A recent operational requirement for WFP has arisen that necessitates the starting of MAM prevention activities in the localities of North Delta, Telkuk and Rural Aroma in Kassala state which slightly contradicts current planned implementation rollout based on the proposed impact evaluation study (where Rural Aroma and Kassala Urban are the starting localities for implementation while Telkuk is planned for the third round and North Delta for the fourth (last) round. Whilst this change has come quite abruptly, it was also recognised that the change had to be accommodated as much as possible given the recognised complexity of programming realities in Sudan.
Current rollout plan and proposed changes
The current rollout plan is in 4 steps across 6 study clusters as follows:
First step: Kassala Urban and Rural Aroma (2 clusters)
Second step: Rural Kassala (1 cluster)
Third step: Telkuk (1 cluster)
Fourth step: North Delta and River Atbara + Rural El Girba (2 clusters as River Atbara and Rural
El Girba was grouped as a single cluster)
This rollout plan was based on the stepped wedge study design proposed and clusters were randomly selected into the different steps. WFP allowed for this type of random selection of rollout areas and committed to trying to work out the operational implementation as much as possible with the caveat that should operational necessities require it, changes will be accommodated.
Prior to rollout, a baseline measurement of the indicators of interest will be performed in all study clusters. This is planned in the month before the start of rollout.
An incidence sub-study will also be conducted as a nested study within the main stepped wedge study design. For this, the study clusters in the first step will be the intervention group while the study clusters in the fourth step will serve as the control group.
WFP now needs to implement its FBPM in three localities specifically North Delta, Telkuk and Rural Aroma first.
Implications of the proposed changes
The most critical implication of the proposed change is the loss of randomisation of the study clusters into the specified rollout steps. The effect of this is that the study becomes more quasi-experimental (though in general stepped wedge studies are generally quasi-experimental anyway because randomisation of programme rollout is rarely feasible operationally).
The secondary effect of the proposed changes is that three clusters will now have to be included in the first step of the rollout. This will have a knock-on effect on subsequent clusters that need to be selected and the balance in the number of clusters in each step. A re-balancing of the number of clusters within each step will therefore be required but will most likely imply the need for reducing the number of steps. A reduction of the number of steps in the rollout has a further implication of increased sample size.
Study design adaptation in response to the changes
Given these changes, we propose the following adaptations to the study areas or clusters selection:
- Increase number of clusters from 6 to 8
- Change the rollout plan for the 8 clusters selected
The increase in sample size can be managed more cost-effectively by increasing the number of clusters by 2 (from 6 to 8 clusters). This will also help in re-balancing the number of study clusters per step.
We suggest adding the locality of Wad El Helew and the locality of Western Kassala as study clusters in addition to the previous 6 study clusters selected earlier.
We propose to change the rollout plan as follows:
First step: Rural Aroma, North Delta and Telkuk
Second step: River Atbara + Rural El Girba, Western Kassala and Wad El Helew
Third step: Rural Kassala and Urban Kassala
This rollout plan re-balances the number of clusters more evenly across the three steps of the rollout. It also addresses the need for more number of distribution sites for the first and third steps as per requirement of the incidence study.
Operational implications of the adapted study design
The adapted study design will face the following challenges:
- The increased number of study clusters has effects on the costs of running the study as it can potentially increase the length of time that each round of data collection will take.
- We will potentially have to deal with new additional partners who cover the new selected clusters.
- This will have additional challenges with the management and supervision of the study as we are covering a bigger geographic area than before.
- Issue of contamination and spill over will potentially be a challenge
However, as earlier suggested, since we are not randomising anymore, the way the selection of clusters have been done allows for some potential of minimising this additional cost through more strategic positioning of the enumerators into assigned clusters and more efficient data collection mechanics that will optimise the time available for data collection to be able to get data from more areas for the roughly the same amount of time as before. In general, this is affected by distance of data collection points from where enumerators will be based and the structure of the survey teams. We are currently reviewing these systems and structures and will test the mechanics in the baseline to see what kinds of time savings we can achieve.
From our knowledge, this should not be overly difficult as partners are known to the study team and will be easy to get them on board quickly.
As with point 1 above, we have to be creative with our data collection and management systems such that we will still be able to properly monitor and supervise data collection even with the increased number of study clusters.
In the study design, we have articulated that selected study clusters should be separated from each other by clear boundaries and geographical gaps so as to avoid contamination and spill over effects. With the current selection, and the increase into two more clusters, avoiding this issue will be more challenging. Hence, the recommendations we made earlier regarding how this can be avoided is all the more important. Some ways of trying to avoid this would be by avoiding sampling from villages in the borders of the localities. There are also additional information during data collection that can be used to ascertain possible contamination and spill over.
Given these changes and considering that it is now at the end of March 2016, it has been suggested to move the start of the programme implementation to May 2016 to allow for the position of the required food inputs to start the programme. This is a welcome suggestion as this means the baseline for the study can be done for the month of April 2016 which allows for data collection mechanics and organisation to be tested, revised and refined to be as efficient as possible. These changes along with the previously stated changes are depicted in the revised schematics of the adapted study design (see schematics update 2).