Given the stated aims, proposed evaluation questions and the various assumptions we’ve made on the MAM prevention package/s and their planned implementation in Sudan, we propose a stepped wedge cluster controlled trial design. The stepped wedge design addresses the assumed incremental roll-out of the MAM prevention package/s into identified areas or clusters. The design allows for intra-cluster controlled comparison (horizontal comparison) in which each area or cluster is compared to itself at baseline status (control status) at each step at a certain time interval. This analysis will enable us to answer the question on the impact of the addition of the MAM prevention package/s onto MAM treatment. The design also allows inter-cluster controlled comparison (vertical comparison) between a number of study areas or clusters with each cluster serving as a control at varying steps or stages of the evaluation study. This analysis allows us to answer the question on how the addition of varying combinations of MAM prevention package/s components impact on MAM treatment. Both horizontal and vertical analysis provide the necessary information to model the effects of time, in terms of both when intervention has been started and how long intervention has been on-going, on the effectiveness of the MAM prevention package/s. A graphical representation of a stepped wedge design is found in Figure 1 showing these practical and analytic features that makes it suited for answering the proposed evaluation questions indicated in the previous section.
Initial Study Design
We initially planned for a study design that would conduct 1 baseline measurement where all areas or clusters are at ‘baseline’ status (as defined previously) and that assumes that rollout of intervention will be staged at 4 month intervals over a one year period hence 3 steps. We planned to conduct 2 measurements at each step for each of the clusters or areas. First measurement will be done 2 months after the start of each step (i.e., turning of some clusters into intervention areas) and the second measurement will be done 2 months after the first measurement. This will allow for a measurement to be made at the start phase of the intervention when the organisation, logistics and protocols of the intervention are getting refined and institutionalised and at 2 months thereafter when the intervention has already been well-established and potentially has had an effect. This initial design is based on a one year data collection period. However, decision on awarding the grant to Valid International took longer than expected which left us with only about 9 months of data collection for the study. This prompted a re-design.
First Design Revision
We then planned to rollout the programme in 4 steps at 2 month intervals with 1 measurement made at each step. This design will maintain the 2 monthly measurement interval of the previous design but with a shorter gap between steps. This design was chosen so as to stay as close as possible to the temporal resolution of the original design and without inflating the sample size to an unreasonable level.
The new design requires a sample size of 1140 which is 296 more than the original design (see Sample Size Update 1). However, this sample size does not increase the number of clusters needed as six clusters of 192 sample size each will give just enough overall sample size required for this new design. Given that we have inflated the number of clusters to 6 in the original design, then there is no net change in the planned number of clusters and subsequently the total number of sample size for this new design.
Second Design Revision
In late March 2016, an operational requirement for WFP has arisen that necessitates the starting of MAM prevention activities in the localities of North Delta, Telkuk and Rural Aroma in Kassala state which contradicts current planned implementation rollout based on the proposed impact evaluation study (where Rural Aroma and Kassala Urban are the starting localities for implementation while Telkuk is planned for the third round and North Delta for the fourth (last) round. Whilst this change has come quite abruptly, it was also recognised that the change had to be accommodated as much as possible given the recognised complexity of programming realities in Sudan.
In response to these changes, we have put in place the following adaptations:
- We took one step out of the rollout plan from the current 4 steps to 3 steps.
- Increased the number of clusters from 6 to 8
- Changed the rollout plan
This will allow us to re-balance the number of clusters in each step appropriately. However, there is a sample size implication for this change (see Sample Size Update 2).
As mentioned in point 1, the increase in sample size can be managed more cost-effectively by increasing the number of clusters by 2 (from 6 to 8 clusters). This will also help in re-balancing the number of study clusters per step. We plan to add the locality of Wad El Helew and the locality of Western Kassala as study clusters in addition to the previous 6 study clusters selected earlier. This adaptation was something we still had to re-consider given that additional clusters would have cost implications.
We propose to change the rollout plan as follows:
First step: Rural Aroma, North Delta and Telkuk
Second step: River Atbara + Rural El Girba, Western Kassala and Wad El Helew
Third step: Rural Kassala and Urban Kassala
This rollout plan re-balances the number of clusters more evenly across the three steps of the rollout. It also addresses the need for more number of distribution sites for the first and third steps as per requirement of the incidence study.
We aimed to start the study based on this re-design by the second week of April, specifically on the 9th or 10th of April 2016.
Current Design Revision
Since the last study design update, further administrative challenges have arisen that require a re-think of the design of the proposed study and how such design will be operationalised.
To date, there is still no contractual agreement in place between the different organisations involved in the study . This means that there is no binding agreement that will enable the allocation and disbursement of funds needed to fully start-up the study.
As per previous update, the baseline data collection has been earmarked to start on the second week of April 2016 (around the 9th or 10th of April 2016). This is now untenable as there are no available resources that can be used to start the baseline.
There are two alternatives that can address this issue.
First is to push the data collection plan by another month. This would mean conducting baseline in May instead of April (one month later than planned) and then subsequent data collection for each step of the study at 2-month intervals. Keeping the number of steps at 4 , this would mean having the last data collection round for the stepped wedge study at the end of January 2016. This scenario will not involved any sample size increase as compared to previous designs that had a baseline data collection along with 4 steps of data collection. However, this scenario will not be feasible, as this will significantly reduce the time allocated for data analysis given that a March 2016 deadline has been specified by 3ie. Requesting an extension of deadline is most likely not an option because the current timeframe we have is already based on a two-month extension that has been requested by Valid International given the previous delays experienced.
Second option is to drop the baseline altogether and keep the start of interventions in the three clusters needing to be implemented in by WFP to May 2016. We would then start with incidence data collection by the first week of May 2016 for 2 weeks and then continue with the second round of incidence data collection for June and then have the first round of stepped wedge data collection by the end of June 2016. Keeping the number of steps to 4, this would mean that the final stepped wedge data collection will be on the last 2 weeks of December 2016. This option will roughly maintain the amount of time previously allocated for data analysis and will ensure deliver out outputs to 3ie by the March 2016 deadline. However, dropping baseline has a sample size implication. It should be remembered that a baseline round has two benefits to the study. It reduces the overall study sample size requirement and per cluster sample size requirement. Also, it increases the power of the study to detect variances and differences. In general, a baseline makes the study so much stronger and better. Losing baseline would require a relevant increase in sample size to make up for the variance lost by giving up baseline (see current sample size estimate).
This sample size is higher by 206 samples as compared to the original design. This sample size will require 7 clusters with a size of 192 each. It will still be possible to keep the 6 study cluster structure but we will have to get a minimum of 224 samples within each of the study clusters.
This sample size increase is not outrageously large and can be accommodated with minor adjustments in design. This option is also the least disruptive compared to the previous option. This option can be implemented without changing the study design considerably and without needing to negotiate for the deadline to be extended yet again.