zlacker

>>giulio+(OP)
As with all studies in the social sciences, one of two principles apply.

First, if the conclusions are counterintuitive or unexpected, then when you look closer, you will find that the methodology is garbage and that it does not support the conclusions given.

Second, if the conclusions reflect things that you believe are true, when you look closer, you will find that the methodology is garbage and that it does not support the conclusions given.

>>andrew+HQ
That's a low-effort, shallow dismissal that doesn't even address anything specific to the article.

If you have specific criticism regarding the methodology of this study - which doesn't, prima facie, appear unsound - please let the rest of us participate.

>>Tainno+OZ
Unfortunately I was not able to locate a preprint for the paper itself, so we only have this article summarizing.

First I'll say that without preregistration of the methodology, there's a lot that is immediately suspicious.

> The researchers built an index based on 15 relevant questions in the NCHA, in which students were asked about their mental health in the past year

Why these 15? What was the "relevance" criteria?

To their credit, they don't just look at a summary metric of "mental health" which would be kind of absurd since the relative weighting is also arbitrary (although that appears to be the main conclusion). The article here notes several axes on which significant differences were found. Why these axes? What about other "mental health" metrics? Did they get better or stay neutral or just have no detectable effect?

Without preregistration it's almost impossible to determine exactly how cherry-picked these differences were, as with a large enough set of potential questions to choose from, you're going to find statistically significant trends on some of them by random chance.

The core methodology is to track the spread of Facebook to different colleges and compare mental health between schools that had Facebook and schools that did not yet have Facebook. This is surprisingly not terrible, but without insight into how the study controlled for the time axis and potential confounding variables about the non-random selection of schools for the rollout, it's difficult to say more.

>>andrew+171
I hate that I was baited into taking a closer look at this rather than just sticking with my trite dismissal. I did locate a preprint of the paper [1], but have not yet looked at it to determine if any of my above criticisms hold water.

Nonetheless I remain blithely confident that this study is not going to be the one to break the mold.

[1] https://www.econstor.eu/bitstream/10419/256787/1/1801812535....

>>andrew+k81
So it is a low-effort shallow dismissal then?

>>rajup+me1
Certainly a dismissal but at this point it seems rather disingenuous to call it low-effort and shallow.

(Also, please consider this friendly piece of advice: check yourself!)