First, if the conclusions are counterintuitive or unexpected, then when you look closer, you will find that the methodology is garbage and that it does not support the conclusions given.
Second, if the conclusions reflect things that you believe are true, when you look closer, you will find that the methodology is garbage and that it does not support the conclusions given.
If you have specific criticism regarding the methodology of this study - which doesn't, prima facie, appear unsound - please let the rest of us participate.
"The have used a correlational model, not a causal model. There are several confounding variables the paper doesn't consider, hence it is not proven from the evidence that Facebook has a negative impact "
The article discusses how the study looked at different universities during the same time period, some of which had access to facebook and some of which didn't, and discovered that in the first case there was an increase in mental health issues over that period. There could still be confounders, sure, (or the sample size could be too small etc.), but at a first glance, that's not an unreasonable approach, as it tries to isolate the variable "facebook yes/no".
That said, if you haven't read the article, I'm not sure why you even felt the need to comment? This is exactly the same kind of shallow dismissal I was calling out.
> While many studies have found a correlation between the use of social media and various symptoms related to mental health, so far, it has been challenging to ascertain whether social media was actually the cause of poor mental health. By applying a novel research method, researchers have now succeeded in establishing such a causality
But doesn't elaborate on the new method. We'll have to wait for the study to be published I guess.
First I'll say that without preregistration of the methodology, there's a lot that is immediately suspicious.
> The researchers built an index based on 15 relevant questions in the NCHA, in which students were asked about their mental health in the past year
Why these 15? What was the "relevance" criteria?
To their credit, they don't just look at a summary metric of "mental health" which would be kind of absurd since the relative weighting is also arbitrary (although that appears to be the main conclusion). The article here notes several axes on which significant differences were found. Why these axes? What about other "mental health" metrics? Did they get better or stay neutral or just have no detectable effect?
Without preregistration it's almost impossible to determine exactly how cherry-picked these differences were, as with a large enough set of potential questions to choose from, you're going to find statistically significant trends on some of them by random chance.
The core methodology is to track the spread of Facebook to different colleges and compare mental health between schools that had Facebook and schools that did not yet have Facebook. This is surprisingly not terrible, but without insight into how the study controlled for the time axis and potential confounding variables about the non-random selection of schools for the rollout, it's difficult to say more.
Some were surely acting in their own personal financial interests but I'm also certain that a lot of it was more nuanced and personal. People need to think of themselves as, for the most part, good people who do mostly good things. Knowingly contributing to something that makes life much worse for many people doesn't align with that and they will need to deny it. I know if you polled phillip morris employees about cancer in the late 60s after the link was confirmed you'd hear a lot about correlation and uncertainty.
HN isn't a random slice of the population. A lot of us here work in this domain or on similar products. There are certainly people in this comment section who directly worked on the core facebook product being discussed. They need to think of themselves as good still, too.
They looked at the mental health (as measured by self-reported surveys) among schools over time and cross-referenced that with the rollout of Facebook over time. So they could compare the change in mental health at schools the received Facebook access and compare it to the change in mental health at schools that did not receive Facebook access at the same time.
The methodology appears to be fairly novel and does isolate them from several reverse-causation biases, as it is difficult to imagine that the rollout of Facebook was influenced by factors that led to the decline of mental health in student bodies.
Going back to your specific comments. Clearly the universities were not randomly assigned the treatment and control. And the actual number of independent sample sizes is extremely unlikely to give stat sig results at the single percentage digit impact shown. And no matter what they do, for something as complex as mental health, listing out all the confounding factors is hopeless - unlike lung cancer where you are literally sucking tar into your lungs and the sample sizes and effects are huge. Its a useful observational study, but it is ridiculous to call it a proof.
> We know that smoking is linked to cancer through decades of correlational studies and careful analysis of confounding factors, for example.
Yes, it took decades, when there is no proper control set. There are work arounds like backdoor and front door criteria, but yeah - it will take decades of work and looking inside the "black box".
Nonetheless I remain blithely confident that this study is not going to be the one to break the mold.
[1] https://www.econstor.eu/bitstream/10419/256787/1/1801812535....
These 2 are vastly different situations.
To give an example. Establishing causal effect between nicotine and lung cancer is an open question, even as the causal effect of smoking on cancer is very clear.
I assume to do that you have to establish the complete pathway and mechanism from someone using facebook to an increase in depression, like showing observations of changes in neurotransmitters or brain structure that have been proven to cause changes in mental health, and then proving that facebook caused the changes in those levels. (FWIW I assume this could be done and that we may see those kinds of results if it were done, but I haven't actually seen a study like that. I also assume the hypothesis in general.)
For instance, using the example of smoking from another commenter, from the CDC website [0]:
> - Poisons in cigarette smoke can weaken the body’s immune system, making it harder to kill cancer cells. When this happens, cancer cells keep growing without being stopped.
> - Poisons in tobacco smoke can damage or change a cell’s DNA. DNA is the cell’s “instruction manual” that controls a cell’s normal growth and function. When DNA is damaged, a cell can begin growing out of control and create a cancer tumor.
These seem more like things that can be tested in laboratory settings that are easily reproducible and rely on more objective observations than self-reporting.
I'm neither a neuroscientist or social scientist so I'm just trying to understand, not saying they're wrong or that the research is even flawed.
[0]: https://www.cdc.gov/tobacco/campaign/tips/diseases/cancer.ht...
Of course, it'd be nice to see if the difference in increased rates of depression & anxiety are themselves abnormal in the first place... Not sure if the study goes into that depth.
(Also, please consider this friendly piece of advice: check yourself!)
Like there's that one finding that came up while researching how scientists in the hard sciences achieved recognition. The soft science researcher discovered that every single one of the scientists insisted questions are more important than answers. But there was no margin of error, so they couldn't write a paper about that. It's not a statistic, it's just absolute. They should have by all means written a paper about it, no shame in being absolutely right.
And there's sociologists like Andrés Pascal Allende, on whom the Mandalorian is based, who was considered a counter-terrorist by the rightful president, and also a terrorist by the usurper, like the Mandalorian. I should clarify he mostly carried out sociology with machine guns and grenades, killed many carabineros, hard target, came in and out of Chile as he pleased, highly persecuted, outraced the persecutors every time, was Minister of Tourism in Cuba--that's a really good job, incredibly good, dude that's like that's a huge reward for standing up to death and torture, oh man, that's recognition, on top of the other recognition, medals and all the rest. That is all second only to being the hero of the absolute most oppressed and repressed (both) worthy victims, meaning those who wish to do what he did for them if they could like watching the Mandalorian wishing they could do that and then going back into the grind and struggle day after day of exploitation and dealing with the betrayal contest set up by the dictatorship. Nothing compares to that recognition, the recognition of the worthy victim. That is heroism definitionally. Really his heroism and those he led determined were the only thing holding up the dignity and living conditions of like 80% of Chileans, fear of the hero.
He studied sociology before becoming the Mandalorian. Must have learned something if he was determined to graduate.
It is hard to credibly preregister studies that use observational data. It also seems hard to design an experiment around the roll-out of a social-media service that we know ahead of time to be successful.
Instead, what is usually done on observational data is (1) making clear what the statistical assumptions are that are required to establish causality, (2) testing possible violations of the assumptions, and (3) testing whether the data is consistent with alternative explanations.
So in such papers, results don't come for free. We need to think seriously about what reasonable theories we can have, and whether the data matches each theory.
> without insight into how the study controlled for the time axis and potential confounding variables about the non-random selection of schools for the rollout, it's difficult to say more.
The paper does also use alternative assumptions that lead to alternative statistical specifications. They also look at various intermediate outcomes to see if they are consistent with their proposed narrative. Such defensive writing is what blows the PDF up to almost 80 pages.
Proofs are for mathematics, not for science. (I share your distaste for science journalism that throws big words like "prove" around without much care, but that's probably not something you can fault the study authors for.)
This is evidence in favour of a theory. It is to be understood within a larger body of evidence. Eventually, hopefully, there is enough evidence in one direction or another that we may draw more or less definitive conclusions.
> I made it a point to not read it, because virtually all social science papers are like these. It's really not worth my time
Nobody is forcing you to read this study, but somehow you seem to assume that your shallow dismissals (to which you are of course entitled privately) are worth anyone's time.
Not sure why OP considers themselves to have been "baited" when the conversation IMHO has been greatly improved by them substantiating their criticism (which may have its merit).
I agree it's not unreasonable, but you have to account for the fact that back then, most of the colleges that had it were top tier/high stress/highly selective colleges. Facebook started at Harvard, then went to Yale and Princeton, and then on to basically most of the US News top 50.
The comment I responded to was seeming to attribute those to OP's later comments, which would be unfair. The dismissal of the dismissal still comes across as low-effort and shallow.
As always, it's better to go to another thread if a topic doesn't interest you, rather than disrespect people's time & energy by attacking the validity of the topic itself.
I do not take offense to the response calling out OP's first comment as low-effort and shallow because it was both of those things. I just can't see the comment I responded to as defensible with such a strong combination of irony and infelicity.