Maximizing Meaningful Engagement in Digital Therapeutics

Digital therapeutics represent a promising opportunity for healthcare. Its potential to improve accessibility, reduce stigma and integrate seamlessly into people's daily routines is very auspicious. However, one of the most important challenges is to ensure meaningful engagement to optimize its effectiveness. The level and type of user engagement with digital therapeutics has been a topic of conversation at industry events and talks, as a key area to understand and optimize. In this article I will attempt to cover what I believe to be the three key areas of engagement: assessment, strategies and innovations, based on a scientific foundation of recent articles and information that have come to my attention.

A 4-step meaningful engagement assessment for DTx

When talking about engagement for digital therapies, we should first think about how we can assess them. As each of these therapies serves a different pathology, audience, and demographic, meaningful engagement has no single definition. This assessment can, however, be made in a general way using a framework developed by the authors Strauss et al. (2022) and published in Frontiers in Digital Health.

The authors built upon the ideas of Lean Startup and Design Thinking that consist of a loop of build-measure-learn activities. Based on these elements, and taking into account the unique challenges faced by the DTx products, they created a four-step process-oriented framework for driving meaningful engagement:

STEP 1: Defining the Measure of Value

Contrary to conventional technology products, the value in digital therapeutics centers on clinical outcomes. As an example, in an entertainment product like a regular game, companies strive to keep their users entertained, which makes retention (consistently returning to the app) a great indicator of the product's value. In contrast, DTx aims to improve whichever pathology they are treating, so even if user retention is high, but clinical outcomes do not improve, the product doesn't achieve the primary intended value.

Therefore, regular metrics of user engagement may not be valuable for all DTx cases, or may not be the only metric that should be taken into account. The goal should be to identify which outcomes are crucial to users and stakeholders, and then (which will be part of the next step) translate these into the metrics of meaningful engagement that will be measured.

STEP 2: Operationalizing Meaningful Engagement

The next step would be to actually translate the defined 'measures of value' into actionable metrics and behaviors within the DTx, focusing on the engagement behaviors that are strongly associated with the desired clinical outcomes.

Strauss et al. (2022) recommends taking a theory-driven approach, identifying clinical outcomes and then validating this with the product data.  In the case of new digital therapeutic or programs with limited data, a large part of this challenge may be the lack of prior evidence of the leading clinical outcomes within this modality, making theory the only guide. It is possible, however, to refine theory-based hypotheses using exploratory analyses for products that have existing data.

Nevertheless, although engagement metrics used for evaluating digital therapeutics differ, research indicates adherence to recommended usage is a strong indicator of positive outcomes and thus should be taken as an initial reference target for development and design teams (Fuhr et al., 2018).

STEP 3: Implementing Solutions

The third step consists of identifying engagement techniques and tailoring them for maximum impact. Drawing from developmental, behavioral, and clinical science, tailored engagement techniques are designed to meet users' unique needs and preferences.

Recent advances highlighted in a study by Nwosu et al., published last year in Frontiers in Psychiatry shed light on several pivotal strategies enhancing engagement in mental health apps which could be also useful when thinking about treating other pathologies. In this paper gamification, personalization, in-app symptom monitoring, and integration with clinical services emerge as key pillars.

Gamification leverages innate human tendencies by incorporating game-like elements, fostering sustained app usage through challenges, rewards, and progress tracking. We discussed this topic on a previous blogpost on our website, this is definitely worth checking out if you are interested in this topic!

On the other hand, in-app symptom monitoring promotes self-awareness and active management, while integration with clinical services provides comprehensive support within the app. Real-world data analysis emphasizes factors such as ease of use, personalized experiences, and clinical service integration in maintaining sustained engagement (Nwosu et al., 2022).

Moreover, personalization tailors content and features, fostering relevance and ownership for users. To drive engagement, an understanding of behavioral change techniques is necessary, but not sufficient. There can be large differences in effect sizes for the same behavioral change technique. This is due in large part to the fact that behavioral change techniques can be applied in a wide range of contexts and specific characteristics of the user base such as demographics can heavily influence the results. User-centered design thus has a fundamental role to play.

A great real world example of this can be found in a recently published patent by the inventors Caroline Pena and Katie Rodammer, whose assignee is the well-known DTx company Click Therapeutics (Rodammer and Pena, 2022). By monitoring user behavior and deploying targeted in-app alerts and messages, it aims to ensure compliance and active user engagement. After being prescribed a DTx, users must fill out basic demographic information (e.g., their age, weight, location, health history, etc). In the initial weeks of treatment, baseline user habits are recorded, and threshold limits are monitored based on these inputs. The app will deploy in-app alerts and messages to encourage users to engage with the product if a threshold is exceeded (e.g., user click speed exceeding or below a defined personal limit). In order to determine whether alerts were effective and whether the user was more or less engaged, the software measures the user response to the app. In the event that the app deems that the thresholds have been passed, similar types of alerts will be used to promote user treatment engagement on an ongoing basis. In contrast, if the software determines that alerts were ineffective, it will select different alerts from a pre-built library/database to determine whether other alerts are more effective at changing user behavior.

STEP 4: Iteratively Evaluating Impact

The last step is not actually the end of the process. To actually have a meaningful engagement it is crucial to continuously assess the solutions built on metrics and clinical outcomes while permanently integrating user feedback to refine the digital therapeutic.

To iteratively assess the impact of DTx, it may be useful to test product changes. However, for safety reasons, and even more so if the DTx is FDA-regulated, it is not possible to conduct periodic A/B testing as with regular digital products. Instead, conducting small-scale IRB-regulated studies to measure the impact on meaningful engagement and clinical outcomes before a full user base release is a more appropriate approach.

Even when having these types of tests in place, understanding the most efficient mechanisms of action can be complex. Determining the causality and how engagement affects outcomes is severely difficult. It is therefore important to rely on theory and triangulate findings. Strauss et al. (2022) recommend utilizing a cross-functional approach, combining quantitative data with qualitative insights, emphasizing transparent study designs and prioritizing effect size over p-values to avoid spurious conclusions.

Overall, there is no single recipe for success. The framework presented by the authors Strauss et. al. (2022) serves as a useful tool for shaping and structuring DTx engagement assessments. Nevertheless, the multifaceted and innovative strategies analyzed show a high complexity in defining the right path for each of the solutions. What is certain is that in order to take advantage of the full potential of these transformative solutions, it is essential to integrate the experience of various functions, leverage data, and prioritize user participation. On a personal note, I also believe that collaboration between teams and companies, information transfer, research, and other elements are extremely valuable and should be fostered even further, leading to better clinical outcomes, and consequently, the growth of the DTx market.

Sources:

Fuhr, Kristina, Johanna Schröder, Thomas Berger, Steffen Moritz, Björn Meyer, Wolfgang Lutz, Fritz Hohagen, Martin Hautzinger, and Jan Philipp Klein. ‘The Association between Adherence and Outcome in an Internet Intervention for Depression’. Journal of Affective Disorders 229 (March 2018): 443–49. https://doi.org/10.1016/j.jad.2017.12.028.

Nwosu, Adaora, Samantha Boardman, Mustafa M. Husain, and P. Murali Doraiswamy. ‘Digital Therapeutics for Mental Health: Is Attrition the Achilles Heel?’ Frontiers in Psychiatry 13 (5 August 2022): 900615. https://doi.org/10.3389/fpsyt.2022.900615.

Rodammer, Katie Nicole, and Caroline Pena. APPARATUSES, SYSTEMS, AND METHODS FOR INCREASING MOBILE APPLICATION USER ENGAGEMENT. US 20220399108 A1, filed 30 December 2020, and issued 15 December 2022. https://patents.google.com/patent/US20220399108A1/en.

Strauss, Gabriel, Jessica E. Flannery, Elise Vierra, Xin Koepsell, Emily Berglund, Ian Miller, and Jessica I. Lake. ‘Meaningful Engagement: A Cross Functional Framework for Digital Therapeutics’. Frontiers in Digital Health 4 (11 August 2022): 890081. https://doi.org/10.3389/fdgth.2022.890081.

Yeager, Carolyn M., and Charles C. Benight. ‘If We Build It, Will They Come? Issues of Engagement with Digital Health Interventions for Trauma Recovery’. mHealth 4 (September 2018): 37–37. https://doi.org/10.21037/mhealth.2018.08.04.