ROI Mental Health Claims Are Everywhere. Here’s How to Know What’s Real.

A phenomenon I’ve been noticing is the prevalence of return on investment (ROI) claims by digital health point solutions. These claims are so overstated they can effectively make the term ROI meaningless without careful consideration, especially for consultants attempting to sort through the tangle for their clients.

As Head of Actuarial Analytics at Spring Health, I spend a lot of time thinking about the value of mental health solutions. I enjoy drawing from my background in healthcare program valuation, medical economics, and trend forecasting at health plans and applying what I’ve learned to the unique world of mental healthcare.

In my experience, consultants have a tough job ensuring that the assumptions made in ROI claims are reasonable and the data being used is analyzed, interpreted, and reported correctly. So, let’s talk about ROI, how to measure it accurately, and the metrics driving positive ROI for employers. 

A nuanced approach

Simply stated, return on investment (ROI) is the total dollars a customer saves for every one dollar invested. When an employer invests in a healthcare solution, they typically look for positive ROI—one dollar out and more than one dollar back in. 

Mental health solutions have proven to significantly impact employers’ health plan spending, and in some cases, go beyond cost-offset to deliver on the promise of positive ROI.   

However, the value of a mental health solution is broader than just financial savings. Clinical and productivity improvements are central to assessing the true population impact of a mental health solution, and are a precursor to ROI:

• Clinical outcomes: Is the solution showing measurable improvements for the population? The quality of a solution is interconnected with savings, which we’ll talk about later.
• Improvements in employee retention and productivity: These are measures of whether employees like the solution, are using it, and whether it’s making a difference in their lives.

Although the question of dollar savings sounds simple, there are complexities to measuring these mechanisms. Without understanding the interplay of a mental health solution and the people using it, it’s impossible to know if there’s a real return on investment.

Meticulous validation is key

If every mental health vendor claims ROI for their solution, how might consultants evaluate them? First of all, for credibility, an ROI study must be open to outside analysis. When reviewing ROI claims, we recommend asking whether they are:

1. Independently verified and validated?
2. Reviewed by third parties?
3. Peer-reviewed? 
4. Up to academic research standards?
5. Published in academic journals?

Outside validation can help pinpoint any problems with the study design or conclusions drawn.

Study development and design are essential

A mental health solution must use a rigorous thought process to design and conduct an ROI study. While it’s tempting to compile a high-level analysis and label it as ROI or savings, these conclusions may not be universally applicable.

To gauge the credibility of an ROI study, consider these questions:

• Risk-adjustment: Does the study provide evidence confirming the comparability of the two groups in terms of risk?
• Baseline: Were the spending patterns of the treatment and comparison groups similar before the intervention?
• Credibility: How precise is the savings estimate? Is there a substantial volume that distinguishes it significantly from zero?
• Expectation: How does the result align with external evidence and actuarial estimates?
• Transparency: Does the study reveal the “I” in ROI?
• Validation: Has a trusted third party endorsed the results, lending credibility?
• Cost-offset: Is the study using your data or assumptions?

A well-designed study can be reasonably expected to have a similar effect on a customer’s population—an important point for consultants helping clients make informed decisions about which solution to adopt. 

Cost of the solution is often ignored

Many ROI reports don’t include the cost of the mental health solution in their calculations, which means they’re not genuinely talking about ROI. As consultants look at reports claiming savings, it’s important to check if they show costs. If they’re not showing costs, why not? 

Verify that these claims are not simply a cost-offset, shifting the spend from the health plan to a vendor who now manages the benefit. If it is, it’s possible the employer isn’t saving money overall, but just moving costs around.

Common challenges of measuring ROI

For any medical intervention, we need to think about sample size, access barriers, population differences, and severity level as we’re trying to understand whether an intervention is working for a particular population.

Sample size

Sample size means the number of people in the population analyzed for an ROI report. 

There needs to be enough people using the benefit so the results are statistically significant. If only three employees engage with the benefit, they may have great results, but those results aren’t generalizable to a larger population.

Access challenges

Something else to keep in mind is that due to a lack of provider access and continuing stigma around mental health, there are a lot of undiagnosed mental health needs. Typically, when we think of ROI, it’s about claims data, which doesn’t capture people who haven’t received care. 

This can be a dilemma when we’re trying to draw conclusions based on the number of people with mental health conditions. That data might simply be inaccessible. It’s not as straightforward as identifying people with diabetes or other chronic health conditions.

Population differences

We’ve found that comparing populations with mental health conditions to a general benchmark is less useful than comparing them to similar populations with mental health conditions. When measuring the success of a mental health program’s success, they must compare apples to apples to understand what’s working or not working. 

When creating a comparison group, we must be thoughtful and careful about designing a matched cohort. This is a way to go deeper than just looking at benchmark or population-level statistics and helps measure ROI more meaningfully.

For example, someone with hypertension may be prescribed exercise as part of a treatment plan. However, someone with hypertension and depression may struggle to exercise, as inactivity is a symptom of depression. Treating the mental health condition empowers the person to manage their physical health better. 

Severity level

There are significant variations in the severity of mental health conditions and how those relate to health care costs. Depression and generalized anxiety are prevalent conditions, but even within the same diagnosis, there are wide severity ranges and levels of treatment needed.

One person with depression may need short-term medication and therapy, while another person may require hospitalization. So, total healthcare costs among these groups also vary widely. This is why we need to be very specific about comparing these populations. 

We can’t just group all mental health conditions into one big bucket and then compare that to the general population. You have to account for severity. 

Now that we’ve covered some challenges of measuring mental health solution ROI, let’s discuss the importance of thorough validation.

Measuring the success of a benefit

We’ve covered why thorough validation is important and the challenges of measuring ROI. Let’s discuss measuring whether a benefit works and the link to cost savings.

Clinical improvement

In value-based healthcare, there’s a reason why savings and quality are so closely tied together. ROI and clinical improvement are very similar. If an employer saves money with lower-quality mental healthcare but doesn’t help people, they haven’t accomplished anything. 

You need both quality and savings for this to work. A quality, comprehensive solution creates better clinical outcomes, lowers overall healthcare spend, and functions as an early intervention before conditions become more acute.

There are a lot of mental health solutions that show good outcomes but can’t demonstrate ROI, or show some ROI but no good outcomes. 

Engagement

A population must use a mental health solution to be effective and show real dollar savings. So, when reviewing an ROI claim, it can be helpful to check engagement levels and ensure that the client understands this relationship so they’re actively pushing for benefit engagement.

Reduction in total cost of care

Employers want to avoid looking into a specific portion of spend and seeing impacts without understanding the entire cost of care. They might be noticing shifts in spending where costs are lowered in one area but inflated in another.

A solution should show it’s not only reducing costs in the specific, promised areas, but also bringing the entire cost down, while considering how much they’ve invested. 

This comprehensive picture is the only way to truly know if ROI is real.

The nuances matter

Consultants are influential in impacting the narratives around workplace mental health benefits. Understanding the complexities of ROI in this space allows them to guide their clients more effectively amid the proliferation of claims, saving them money and creating a more supportive work environment for employees at the same time.

To go deeper into ROI, check out the largest study of its kind on ROI in mental healthcare—and see how a comprehensive mental health solution can reduce workplace and healthcare costs.