🇨🇱Chilean Researchers: School-Based Mental Health Programs Show "No Evidence of Any Clinically Important Difference" in Outcomes
But U.S. education "experts" still insist otherwise - so here are some bits of ammo, some intellectual capital, to counter the science deniers.
(Disclosure - this Stack entry is long. You’ll need to click through to the website to read it all.)
Continuing in a series examining academic research supporting the thesis that school-based behavioral health counseling programs are ineffective, here’s one from Chile. Let me explain why I keep making these deep dives:
Everywhere we are seeing a push in U.S. public education to expand the presence of school counselors, school social workers, and psychologists.
We are constantly inundated with assertions that kids’ mental health is spiraling out of control and its the school’s near-sacred responsibility - to expand behavioral health services in schools for their “social, emotional, and academic” development.
Against this constant drum beat, parents are not seeing results - certainly when it comes to academic outcomes.
If anything, whatever mental health problems are there, they are persisting…and measured academic outcomes continue to slide nationwide.
Parents everywhere assert that U.S. schools have lost focus, but the “experts” are unfazed and double down - demanding wholesale transformation of American schools into something more akin to health clinics with classrooms.
At the edge of this phenomena, parents are seeing schools - usually led by a behavioral health worker - inserting themselves in their family relationships, concealing interventions at school that seem purposely designed to turn their children against them - and almost always involve a zeal to proselytize sexualized identities, as my friend Jane Coleman documents at the Legal Insurrection.
As parents we must meet the irrational zealot with a rational zeal for counterfactuals, higher standards of proof, and honest assessment - plus some mockery, too, when we can.
My goal is to provide ammo - brass alloy bits of intellectual capital - to parents, educators and policy makers to combat this mass hysteria-level of single-mindedness that has now failed a generation of kids. Like those in Selma, Alabama.
But combating the “experts” does require some background knowledge, a little effort - so this is a longer read but I promise you, put in the time. You’ll be equipped to more deeply conceptualize the problem, see the limitations of their position, and gain articulable, intelligent objections to this new, absurdly misguided Therapeutic Model of Modern Education Theory to the experts like this:
So, on to Santiago!
In 2013 Chilean researchers, with the assistance of U.K. colleagues from the medical schools of the University of Bristol, University of Bath, and the University of Nottingham, spent three years designing, planning and rolling-out a randomized controlled trial in some of the poorest communities of Santiago that would assess the effectiveness of a school-based, universal, psychological intervention programs to reduce or prevent depression among adolescents from low-income families.1
From the title of the piece, you already know the result: their comprehensive, school-based mental health counseling program failed to produce any evidence of “clinically important” reductions in depressive symptoms. From the abstract:
“There was no evidence of any clinically important difference in mean depression scores between the [control group and intervention group] or for any other outcomes 3 months after completion of the intervention. No significant differences were found in any of the outcomes at 12 months.”
That’s the result, but there are a number of aspects to this study I want to highlight to support my thesis that these intervention programs - all very similar to what U.S. school programs do now - are simply ineffective.
Let’s dig in.
First, the researchers were quite-consciously motivated to assess the possibility of implementing school-based counseling programs in a “resource poor” environment, selecting 22 participant schools involving over 2500 students in low-income districts of Santiago. They were missionaries for the cause:
“To our knowledge, no large randomized clinical trial of a universal, school-based intervention for depressive symptoms from a low- or middle-income country has been published. There is an urgent need to develop potentially scalable programs for adolescents with emotional symptoms in resource-poor settings.”2
It’s reasonable to conclude they really, really, in a non-scientific way, wanted this to succeed. Their project was sponsored by the Wellcome Trust, a large UK charity established by a pharmaceutical baron that was probably prepared to fund “potentially scalable programs” worldwide before the researchers had to admit, regrettably, it didn’t work.
Unlike the Australian and UK meta-analysis and systematic review studies I started this series with - which are performed in the quiet sanctums of a faculty office - this was a randomized controlled trial out in the field. RCTs are a gold standard in the hierarchy of the medical evidence pyramid, but require buckets of money to carry out. That’s why research by pharmaceutical companies is notoriously suspect - sponsors are presumed to act in their own interest, and researchers want to please their source of funding. In the venture capital-speak of my day job, “He who has the gold makes the rules.”
(N.B., see where “expert opinion” ranks?)
Why is this important? Because, again drawing on my day job as a transactional lawyer, I suspect project funding was secured with a pitch to roll-out these “potentially scalable” school-based mental health programs in minority, low-income neighborhood schools, something that would appeal to the Wellcome Trust’s investment committee members. They just needed a test case, a proof of concept. They obviously received a lot of funding to make that case - over three years of grant money, the study involved 13 PhD and MD researchers from 8 universities, 2500 students, 22 schools and 24 on-site Chilean psychotherapy “facilitators.” They were under pressure to deliver results.
That is perhaps why the authors went to great lengths to explain in the Discussion section of the report why their project involved Herculean difficulties, and how they tried anything and everything they could, before reluctantly, but candidly, concluding:
“In conclusion, given the lack of effect in this and other universal school-based trials for depressive symptoms, it is legitimate to ask whether we should continue to invest in these programs. This is important because there are many mental health school-based programs around the world, including in low-to-middle-income countries, with no evidence to support it.”3
Without a deep-pocket sponsor behind the study, we would not have been able to get such a golden nugget quote. They had a reputation to uphold. They had just spent three years and large sums of the Trust’s money with dangling promises to deliver more than a honest assessment of the evidence justified. They had to bite the bullet, take the loss, but in a way their Trust contacts would still return calls, their integrity intact.
This level of honesty and scientific integrity is what we expect of “experts” - but not what parents get now from smarmy, know-it-all public education “experts” with a PhD in public administration. Parents can see tangible, measured, empirically proven test scores dropping. Parents can see the rising violence in schools, children brutally attacked on school buses, teachers assaulted - and yet time-and-time again, the district coddles the bully with a restorative justice, equity-something non-disciplinary counseling session, without consequences. And we all sense it is getting worse.
Bottom line: the therapeutic educational model is not working - and solid, well-funded, scientific research has demonstrated - for over a decade - that these programs do not work. And rather than face up to the loss like these researchers did to their sponsor, public educators just double-down on stupid.
Now let’s look at the form of intervention and methodologies employed by the researchers and consider the implications in the context of school-based mental health programs in the U.S.
After 18 months of “formative research”4 the study designers chose a cognitive behavioral therapy (CBT) intervention method called “I Think, Feel, and Act” counseling, consisting of 11 weekly and 2 ex-post booster psychotherapy sessions of 1 hour each. This is a control-your-thoughts-change-your-feelings, change your life kind of CBT program. Sounds New Age, but CBT is a real form of psychotherapy:
“Cognitive behavioral therapy (CBT) is a common type of talk therapy (psychotherapy). You work with a mental health counselor (psychotherapist or therapist) in a structured way, attending a limited number of sessions. CBT helps you become aware of inaccurate or negative thinking so you can view challenging situations more clearly and respond to them in a more effective way.”5
The goal with “universal” intervention programs is to prevent mental health conditions like depression from arising in the first place. So the study designers sought to rollout a universal program that could be administered to all students in a school environment without regard to risk status. Students chosen to participate were thus picked randomly and not specifically targeted because of any prior assessment of risk.
Half of the roughly 2500 students were put into the control group receiving psychotherapy, and the other half were simply kept in a classroom where “problems could be discussed” but no form of psychotherapy was given.
Eight groups of 3 “young facilitators” delivered the intervention using a detailed operational manual provided by the research team. All of them are identified as either psychologists, occupational therapists or social workers, trained and supervised by senior clinical professionals, but their professional licensing status is undisclosed. As “young facilitators” requiring experienced supervision, I suspect they were not licensed to practice their specialty independently in a clinical setting.6 Teachers were excluded on the premise that students might hold back in their presence.7
To establish a baseline to measure outcomes, each group of students completed clinically-accepted depression and anxiety measuring surveys8 consisting of questions like these:
On the basis of these self-reported surveys, researchers assigned a baseline number for both the intervention and control arms that could then be measured and analyzed for effect post-counseling:
Given the size of the study, a psychiatric interview was not conducted. These type of mental health surveys are not considered formal diagnostic tests like those used by professionals in a clinical setting; they are simply tools to measure self-reported depressive symptoms and anxiety. This is consistent with the researchers’ goal of creating a “scalable” universal intervention program that can port to any school environment with low income children.
“[W]e aimed to implement a pragmatic trial, and if this intervention were to be extended to a larger study population, only questionaries would be used.”9
The distinction between using formal diagnostic tests in an individualized, one-on-one psychiatric interview versus passing out a questionnaire to everyone in the room is important and has implications in the U.S. school context.
As a practical matter, a non-diagnostic survey suits the kind of scalable universal intervention program the study designers envisioned. As a school-based mental health intervention, a “scalable program” should be administrable by school district employees who, on a district budget, are not likely senior, experienced clinical practitioners. Universal programs, premised on a “first responder/frontline” type of intervention, are implemented across all student groups without regard to risk. So a formal diagnosis of each individual student is simply impractical.
For these types of intervention programs which do not conduct a formal diagnosis interview, proponents of school-based mental health programs believe, or at least want US to believe, they are not engaged in the practice behavioral health requiring a board license. But is that really true in light of what comes next in these programs?
If the program consists solely of administrating surveys - and nothing more - I think the answer is yes. But that’s not what happens in practice. In practice, the programs take student data of anxiety levels or depressive symptoms and, just like the Chilean study intervention program, start some form of psychotherapy treatment either individually or in a group session. And that cross-over into a treatment phase has the potential to be problematic. There’s a reason they call it “intervention.”
Let me explain.
In the counseling world, a “formal diagnosis” of a mental condition requires the application of highly defined criteria contained in the official reference book of the American Psychiatric Association, the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) The DSM-5 is used by all types of mental health professionals around the world who are trained to use it. Symptoms are matched with DSM-5’s recognized categories of disorders, and out pops an officially recognized diagnosis. On the basis of that diagnosis, a treatment plan is crafted and the associated insurance code is identified for billing purposes. This is standard practice for professional therapists in a clinical, non-school setting.
For public health, safety and welfare, behavioral health services like counseling and social work are a regulated health profession, just like medicine, dentistry or nursing. The diagnosis or treatment of a physical or mental condition in most U.S. states may only be performed by a board-licensed behavioral health professional.10 It’s safe to assume that is true most anywhere in the world. Most school-based counselors and social workers, like the “young facilitators” of the Chilean study I suspect11, are not board-licensed; they are merely certified to work in schools on the basis of the educational credentials by a state education department. In the U.S., they likely have completed the same masters-level course of study as their licensed counterparts, but they do not sit for a board licensing exam. They are not mandated to undergo years of supervised practice before earning the right to practice independently. They do not have scope of practice limitations beyond what a school district policy (if any) allows. And they are not mandated to follow legally enforceable practice rules set in regulation by a board of behavioral health with investigatory and adjudicative powers.
So, for unlicensed school-based behavioral health workers, if they don’t apply the DSM-5, they aren’t “diagnosing” anything, and that’s okay right?
Well, let’s look at that. Because diagnosing a condition is not the first step in the practice of behavioral health.
Ask yourself, if those school counselors haven’t diagnosed anything, what exactly are they doing in the followup part of the school-based program? Why administer a survey or solicit a student’s feelings and emotions at all? What “services” are actually included when school districts use the term “student support services” to describe a structured program to deal with an identified student problem?
If the school was making an academic intervention by tutoring a child to raise math scores, fine. But that’s not what’s happening in school-based counseling programs. Behavioral health programs in U.S. schools do not consist solely of classroom instruction on mental health like some kind of lesson plan or lecture, although psychoeducational classes are often part of the job description.
Lectures have a start and end point. They intellectualize a topic for study. The topic is the subject of the lesson, not the child challenged to engage with that topic. Quizzes given at the start of a course to assess a child’s knowledge coming into a class are not used in the same manner as a mental health survey. The former is used to tailor a curriculum of study to a class of students with varying background knowledge on a subject matter. The later is used to tailor a treatment of a child.
These counselors may not be pulling out the DSM-5 textbook and assigning a diagnosis - but, like teachers that use first week quizzes to assess baseline subject knowledge, they are assessing mental conditions. And that is where the problem starts.
“Assessment” is an actual mental health term to describe the process of gathering information about a patient prior to applying the DSM-5, through collecting information and drawing conclusions through the use of observation, interviews, tests.12 Assessments lead to diagnoses, and, like anything in a regulated health profession, performing an assessment requires experience, knowledge and professional judgments to make.
In the Chilean study, prior to giving the students CBT psychotherapy, the researchers used two assessment tests to create a baseline measurement of the student participants’ depressive symptoms and anxiety levels. But completing a long-form, written questionarie is not the only means of making an assessment. Assessments can take the form of other means of gathering information about a person - interviews and observations can form the basis, in a professional’s judgment, of an assessment prior to diagnosis.
Assessing an individual is a component of, and the first step for the practice of behavioral health. Use an intake form, a questionnaire, before your appointment with a therapist starts? You’re starting an assessment process. And the therapist is now engaged in the practice of behavioral health. It’s right there in Arizona’s statutory definition of counseling and social work:
Proponents of school-based behavioral health programs by unlicensed counselors and social workers tend to hide behind the claim they aren’t diagnosing anything.
But they are treating something. They aren’t working with nothing. They have assessments, however incomplete or informal. Teacher referrals. School resource officer comments. The lunch room monitor who witnesses an altercation. A principal who observes a student in her new “Goth” fashion style and signs of cutting on her arm. These observations are transmitted to the school counselor who’s tasked to “look into it” for a possible referral to a “student support services” program.
And that’s how it starts. A few observations and your child is in a counseling program, feeling pressure from adults to examine her feelings, to psychologize her emotions, question herself and her life’s direction.
On the basis of a professional, seasoned judgment? No, on the basis of a 22-year old graduate student with no board license under pressure to perform at her first new job. Or that purple-haired school social worker who looks everywhere for signs of “homophobia” at home and carries a starter kit of pamphlets for “gender questioning” students.
You weren’t consulted in advance. You didn’t give consent. But too late now. Your child’s mind has been pried open and other adults are looking in, making recommendations.
All of it without the protection of a professional board of licensing regulating their activities.
It IS behavioral health. Refusing to make a formal diagnosis does not give some kind of plausible deniability to the charge that school counselors and social workers are practicing a regulated behavioral health activity.
Don’t let them claim otherwise.
And it’s getting worse. Their foot is on the accelerator.
End.
Araya R, Ritsch R, Spears M, et al. School intervention to improve mental health students in Santiago, Chile: a randomized clinical trial. JAMA Pediatr 2013; 167(11):1004-1010; abstract available here: https://pubmed.ncbi.nlm.nih.gov/23999656/
Arya R, at 1005.
Arya R, at 1009.
Arya R, at 1005.
Mayo Foundation for Medical Education and Research. (2019, March 16). Cognitive behavioral therapy. Mayo Clinic. https://www.mayoclinic.org/tests-procedures/cognitive-behavioral-therapy/about/pac-20384610 (last accessed June 7, 2024).
I have attempted to engage the author of the study to verify and will update if needed.
Arya R, at 1005, 1006.
The primary outcome measure was the Beck Depression Inventory-II (BDI-II); secondary outcomes measure was the Revised Child Anxiety and Depression Scale adapted from the Spence Child Anxiety Scale, as well as the Personal Failure Subscale of the Children’s Automatic Thought Scale and 5 subscales of the Short Form of the Social Problem-Solving Inventor-Revised scale. Id. at 1006.
Arya R, at 1008.
Cf, Arizona Revised Statutes 32-3286(C) makes it unlawful for any unlicensed person “to practice behavioral health, including diagnosing or treating any mental ailment, disease or disorder or other mental condition of any person....”
See FN 6 supra.
Bridley A, Daffin L, Fundamentals of Psychological Disorders, 3rd Ed. 5-TR, (May 2024); available at: https://opentext.wsu.edu/abnormal-psych/front-matter/title-page/