Given the delays we are currently experiencing with initiating the impact evaluation study for 3ie, this document shows various operational scenarios and their implications on the study design so as to provide nuanced information to aid in planning and decision-making with regard to the study implementation.

The scenarios we present here are based on an assumption that 3ie will be unwilling to move the end date of data collection. As it stands now, 3ie has stipulated that all data collection must end by October 2016. As per current design, this gives us a 10 month data collection period starting at end of January (baseline data collection) with 4 steps each with 1 data collection point at 2 monthly intervals. This design also allows for a maximum of 6 months data collection for the incidence study. If the current study design is to be implemented as per submitted timeframe, we have anticipated that the sample size requirements are well within achievable limits. The scenarios we present here show what the implications of any delays with the study implementation are specifically on sample size and operational considerations.
 

Scenario 1

Given a fixed end point of October 2016 for data collection and a delay of a month that will negate the possibility of conducting a baseline assessment and assuming that WFP will still be able to start rollout by February 2016 for the first 2 clusters and then subsequent rollout into other clusters every 2 months, we adjust the sample size calculation parameters as follows:
 

$ k = 4 \ \text {steps} $
$ b = 0 \ \text {baseline measurement} $
$ t = 1 \ \text {measurement after each step} $
$ p = 0.034 \ \text {intra-cluster correlation coefficient} $
$ n = 192 $
 

Given these parameters, we arrive at the following sample size:
 

$$ \begin{align}
n_{\text {stepped wedge}} &= 1804 \times \frac {1 + 0.034(4 \times 1 \times 192 + 0 \times 192 – 1}{1 + 0.034 \left (\frac {1}{2} \times 4 \times 1 \times 192 + 0 \times 192 – 1 \right )} \times \frac {3(1-0.034)}{2 \times 1 \left (4 – \frac {1}{4} \right )} \\
\\
&= 1804 \times \frac {1 + 0.034(767)}{1 + 0.034(383)} \times \frac {3(0.966)}{2 \left (\frac{15}{4} \right )} \\
\\
&= 1804 \times \frac {27.078}{14.022} \times \frac {2.898}{7.5} \\
\\
&= 1804 \times 1.93110826 \times 0.3864 \\
\\
&\approx 1346
\end{align}$$
 

This sample size is higher 206 samples as compared to the original design. This sample size will require 7 clusters with a size of 192 each. It will be an option to still maintain the 6 study cluster structure but we will have to get a minimum of 224 samples within each of the study clusters.

This scenario is potentially feasible and is the least disruptive of all the possible scenarios. This scenario can still be implementable changing the study design considerably and without needing to negotiate that the end point be moved later.
 

Scenario 2

Given a fixed end point of October 2016 for data collection and a delay of a month that will negate the possibility of conducting a baseline assessment but with another minimum delay of another month due to WFP and partners unable to implement in first study clusters by February 2016, we adjust the following sample size parameters and the corresponding sample size calculations as follows based on monthly increments of delay:

Scenario 2a: Additional delay of 1 month for intervention rollout with rollout start planned for March 2016 instead of February 2016.
 

$ k = 4 \ \text {steps} $
$ b = 1 \ \text {baseline measurement} $
$ t = 1 \ \text {measurement after each step} $
$ p = 0.034 \ \text {intra-cluster correlation coefficient} $
$ n = 192 $
 

Given these parameters, we arrive at the following sample size:
 

$$ \begin{align}
n_{\text {stepped wedge}} &= 1804 \times \frac {1 + 0.034(4 \times 1 \times 192 + 1 \times 192 – 1}{1 + 0.034 \left (\frac {1}{2} \times 4 \times 1 \times 192 + 1 \times 192 – 1 \right )} \times \frac {3(1-0.034)}{2 \times 1 \left (4 – \frac {1}{4} \right )} \\
\\
&= 1804 \times \frac {1 + 0.034(959)}{1 + 0.034(575)} \times \frac {3(0.966)}{2 \left (\frac{15}{4} \right )} \\
\\
&= 1804 \times \frac {33.606}{20.55} \times \frac {2.898}{7.5} \\
\\
&= 1804 \times 1.63528 \times 0.3864 \\
\\
&\approx 1140
\end{align}$$
 

This sample size is exactly the same as the original sample size as parameters of the study design are kept intact but just moved back by 2 months. However, issues with data collection may occur as this will conflict with Ramadan period on the 6th of June to the 5th of July. Round 2 of data collection may encounter difficulties. This is less than ideal but something that can be potentially addressed.
 

Scenario 2b: Additional delay of 2-3 months for intervention rollout with rollout start planned for either April or May 2016 instead of February 2016.
 

$ k = 3 \ \text {steps} $
$ b = 1 \ \text {baseline measurement} $
$ t = 1 \ \text {measurement after each step} $
$ p = 0.034 \ \text {intra-cluster correlation coefficient} $
$ n = 192 $
 

Given these parameters, we arrive at the following sample size:
 

$$ \begin{align}
n_{\text {stepped wedge}} &= 1804 \times \frac {1 + 0.034(3 \times 1 \times 192 + 1 \times 192 – 1}{1 + 0.034 \left (\frac {1}{2} \times 3 \times 1 \times 192 + 1 \times 192 – 1 \right )} \times \frac {3(1-0.034)}{2 \times 1 \left (3 – \frac {1}{3} \right )} \\
\\
&= 1804 \times \frac {1 + 0.034(767)}{1 + 0.034(479)} \times \frac {3(0.966)}{2 \left (\frac{8}{3} \right )} \\
\\
&= 1804 \times \frac {26.078}{16.286} \times \frac {2.898}{5.3} \\
\\
&= 1804 \times 1.601253 \times 0.5468 \\
\\
&\approx 1580
\end{align}$$
 

This sample size is 440 more than the original sample size. It will take 8 clusters of 192 samples each to reach this sample size or we can maintain the 6 study cluster structure but we’ll need to collect 263 samples from each cluster. This per cluster target is a much difficult target to reach and will require more time per village and per study cluster to complete which will have implications with the length of the per step surveys.
 

Scenario 2c: Additional delay of 4 months for intervention rollout with rollout start planned for June 2016 instead of February 2016.
 

$ k = 2 \ \text {steps} $
$ b = 1 \ \text {baseline measurement} $
$ t = 1 \ \text {measurement after each step} $
$ p = 0.034 \ \text {intra-cluster correlation coefficient} $
$ n = 192 $
 

Given these parameters, we arrive at the following sample size:
 

$$ \begin{align}
n_{\text {stepped wedge}} &= 1804 \times \frac {1 + 0.034(2 \times 1 \times 192 + 1 \times 192 – 1}{1 + 0.034 \left (\frac {1}{2} \times 2 \times 1 \times 192 + 1 \times 192 – 1 \right )} \times \frac {3(1-0.034)}{2 \times 1 \left (2 – \frac {1}{2} \right )} \\
\\
&= 1804 \times \frac {1 + 0.034(575)}{1 + 0.034(383)} \times \frac {3(0.966)}{2 \left (\frac{3}{2} \right )} \\
\\
&= 1804 \times \frac {19.55}{13.022} \times \frac {2.898}{3} \\
\\
&= 1804 \times 1.501305 \times 0.966 \\
\\
&\approx 2617
\end{align}$$
 

This sample size is 1,477 more than the original sample size. It will take 14 clusters of 192 samples each to reach this sample size or we can maintain the 6 study cluster structure but we’ll need to collect 436 samples from each cluster. This per cluster target is an almost impossible target to reach and will require more time per village and per study cluster to complete which will have implications with the length of the per step surveys.
 

Given all these scenarios, it is crucial that the study starts no later than March 2016 if no extension beyond October 2016 is granted.