Summary
A medRxiv preprint reports that a RoBERTa model classified depression-related Twitter users with 0.822 accuracy on an 80/20 training–testing split. The study also used GPT-4o to identify themes in posts, including societal pressure and low self-esteem.
A medRxiv preprint reports that a RoBERTa language model classified Twitter users identified as having depression with 0.822 accuracy on an 80/20 training–testing split. The researchers also used GPT-4o models to analyse posts for themes related to depression. The work is a computational study of social-media text, not a clinical trial, and the preprint has not yet been peer reviewed.
How the study was built
The researchers collected Twitter/X posts dated April 20, 2023, to July 24, 2024. Their search used the phrase “diagnosed with depression” alongside “I” or “me”. After cleaning the data and using GPT-4o for labelling, they identified 2,275 users in the depressive group. They assembled a comparison group of 1,661 non-depressive users from tweets containing the keyword “today”.
A RoBERTa model was trained on tweets from both groups, using an 80/20 training–testing split. RoBERTa is a language model adapted to process text; here, it was used to classify posts associated with the two user groups. The reported test results were an accuracy of 0.822, an F1 score of 0.855 and an area under the receiver operating characteristic curve (AUC) of 0.809. Accuracy is the share of classifications that were correct; F1 combines precision and recall, while AUC measures how well a model ranks examples from the two groups across classification thresholds.
The authors then applied GPT-4o models to posts from users in the depressive group. The themes they reported included societal pressure, low self-esteem, cultural influences and identity-related challenges. These are themes identified in the text, rather than evidence that those factors caused depression.
What the results can tell us
The study shows one way language models can be used to examine social-media text: a classifier separates posts associated with two constructed groups, while a separate LLM analysis summarises recurring themes. Such approaches could be investigated as tools for population-level research or early identification, as the authors suggest.
The figures describe performance within this study’s dataset and split. The groups were assembled using particular search terms and GPT-4o labelling, so the results should not be read as the accuracy of a depression-screening tool for Twitter users generally. The abstract also does not describe an independent clinical assessment of participants. The reported performance therefore concerns classification of these study groups, rather than a confirmed diagnosis of depression in individuals.