Multiphase optimisation strategy (MOST)
How to use a multiphase optimisation strategy (MOST) to evaluate your digital health product.
This page is part of a collection of guidance on evaluating digital health products.
Digital health products and services are usually made up of various features (multicomponent interventions). However, they are often assessed as a package and we are unable to tell which components of the interventions are effective and which are non-essential.
Multiphase optimisation strategy (MOST) involves first evaluating various different components to optimise the intervention. The components with the most promising results are then included in the intervention package while others are discarded.
What to use it for
Use the MOST approach to assess different elements of your digital product or service. This method is particularly useful when you evaluate several different elements of a product or service against certain criteria, for example efficiency or cost.
You would generally use it when developing your product (formative or iterative evaluation). In MOST, the last phase includes a summative trial of the final product.
Pros
Benefits of MOST include:
- you can test several different options for your intervention
- you can discard those components of the intervention that show no effect
- it can help to answer the question of which components are best matched to what users
Cons
Drawbacks of MOST include:
- it consists of 3 stages, so it will take more time and resources to conduct an evaluation using MOST
- you may need some expert knowledge to help you analyse the data
How to carry out a study using MOST
The MOST approach has 3 stages:
- preparation
- optimisation
- evaluation
Preparation
The preparation phase involves selecting the intervention components to examine. You should base this on information that will help you decide what components might be effective in the context of your digital product, such as:
- available scientific literature
- clinical experience
- previous data
This will also involve creating a detailed model mapping how your product will create change. You might consider piloting the components at this stage.
At this stage, make clear decisions about how you will assess which components to keep or discard in the next phase (for example, the choice of benchmarks). These are the optimisation criteria and they need to be defined and put into use in this phase. This step is unique to the MOST method.
Examples of optimisation criteria
Efficiency criterion
Only components which are necessary will be included, removing any inessential (inactive) components.
Cost criterion
Only components up to a maximum cost per user will be included.
Time criterion
You could consider the time needed from the people delivering the intervention. For example, only the most effective components that can be delivered in less than or equal to a specific number of minutes could be included.
There are many possible optimisation criteria and they can be used in combination. Your choice will depend on what is important for your digital product.
Optimisation
The optimisation process aims to help you select a set of components (the intervention package) that will give you the most effective intervention you can get based on the optimisation criteria you set and the constraints defined in your preparation phase.
This phase involves:
- testing the effects of each of the components (and how the components impact each other) using randomised experiments
- screening out the least effective components
Using MOST, participants are assigned to several conditions, so the sample size in each group is reduced. This is done using:
- factorial studies
- fractional factorial studies
- sequential, multiple assignment, randomised trials (MRT)
Evaluation
An RCT is the last phase of the MOST approach. The optimised intervention is evaluated in an RCT when the digital intervention shows enough preliminary efficacy.
Example: testing 4 features of an online smoking cessation intervention
McClure and others (2014), Exploring the ‘Active Ingredients’ of an Online Smoking Intervention: A Randomized Factorial Trial
The team considered 4 design features to include in an online intervention for smoking cessation. They used MOST to design their evaluation.
Each participant was randomised to one of 2 levels of the 4 components, so the study had 2 x 2 x 2 x 2 = 16 groups. 1,865 participants were recruited. The main outcome was self-reported smoking abstinence and the use of adjunct cessation treatment services at a 12-months follow-up.
The 4 potential candidate components tested in the optimisation phase were:
- message tone (prescriptive or motivational) *website navigation autonomy (dictated or free)
- email reminders (received or not)
- tailored testimonials (received or not)
The analysis looked at the main effects of the 4 contrasting levels of the components and two-way interactions (interactions between 2 components). At the 12-month follow-up, they found that 13.7% of participants reported abstinence and 26.0% used adjunct treatment services. They found that none of the design features significantly increased the outcomes at the 12-month follow-up and they found no interaction effects.
However, they found a significant negative effect of testimonials on the use of an adjunct treatment, suggesting that people assigned to receive the testimonials were less likely to use adjunct treatment service at the 6-month follow-up. The authors concluded that, although this result should be viewed with caution, the potential for negative effects that reduce the treatment effects needed to be considered.
More information and resources
Collins L.M., Murphy S.A., Strecher V., The Multiphase Optimization Strategy (MOST) and the Sequential Multiple Assignment Randomized Trial (SMART): New Methods for More Potent eHealth Interventions, American Journal of Preventive Medicine 2007, 32(5): S112–S118. This article argues for the use of MOST in evaluating digital health interventions.
Introduction to MOST with Linda Collins provided by the developer of the MOST
Overview of MOST by the Methodology Centre at PennState
Huffman and others (2020), Developing a Psychological-Behavioral Intervention in Cardiac Patients Using the Multiphase Optimization Strategy: Lessons Learned From the Field. This article details lessons learnt from a 6-year intervention development using MOST, describing the challenges and how these can be overcome.
Examples of studies using MOST in digital health
Healthy Campus Trial: a multiphase optimization strategy (MOST) fully factorial trial to optimize the smartphone cognitive behavioral therapy (CBT) app for mental health promotion among university students: study protocol for a randomized controlled trial. An example of a protocol for a factorial RCT using MOST. Researchers plan to assess 5 app components to increase wellbeing and decrease stress.
Crane and others (2018), A smartphone app to reduce excessive alcohol consumption: Identifying the effectiveness of intervention components in a factorial randomised control trial. This study used a factorial design to assess components of a smoking cessation app.
Kugler and others (2016), Application of the multiphase optimization strategy to a pilot study: an empirical example targeting obesity among children of low-income mothers. This describes a remotely-delivered intervention to prevent childhood obesity using MOST.