The qualitative investigation will focus primarily on two key streams of enquiry. First, it will look more in depth into the coverage of the various components of the prevention programme and second, it will collect further information on the effects of the SBCC interventions specifically with regard to the mechanisms by which they change/do not change current practices relevant to children and women’s nutrition.

The qualitative investigations will employ key informant interviews, informal group discussions, semi-structured interviews and case studies among others to elicit information from specific target person/s or groups relevant to the implementation of the programme . These include influential leaders, staff of health services and the community-at-large. The aim of the investigations is to be able to provide qualitative context to help understand and explain the quantitative information gathered.
 

Why coverage?

Coverage – the proportion of all people needing or eligible to receive a service who actually receive that service – is an important measure of a programme’s ability to meet need and is directly related to a programme’s effectiveness. An effective programme is one that has good coverage and a programme with good coverage is one that is effective. Hence, timely, appropriate and high-quality monitoring of coverage is essential in planning, implementing and tracking the progress of programmes such as those for maternal, newborn and child health and nutrition such as this MAM prevention programme.

The proposed coverage investigation will assess each of the prevention components of the CNIP and determine its coverage using iterative investigations modelled or patterned after methods developed and used to investigate coverage of SAM treatment1. The results of the investigation will in turn provide critical insight into the various aspects of the programme that contributes to or impedes coverage thereby promoting or hindering programme effectiveness2.
 

Objectives

The qualitative investigation aims to identify the factors that promote or impede access to and coverage of the various components of the food-based prevention of moderate acute malnutrition programme in Kassala state in Sudan.

Specifically, the study will:

  1. Synthesise findings from the on-going MAM incidence study and the stepped-wedge cluster randomised trial along with the information from the routine monitoring programme data and use this to develop relevant themes of further investigation;
  2. Conduct as appropriate key informant interviews, focus group discussions, semi-structured interviews and document case studies that would provide additional information along the lines of the relevant themes identified;
  3. Develop various hypotheses on the mechanisms and dynamics that impact on coverage and then perform various easy to implement small area studies and/or surveys that will test these hypotheses; and,
  4. Summarise and consolidate all information from the quantitative studies and the qualitative studies into a concept map that would illustrate the interconnections and relationships between different factors.

 

Methods
Summary of approach

The qualitative study will investigate access to and coverage of the various components of the prevention package (indicated above) so as to illuminate factors that either hinder (“barriers”) or facilitate/promote (“boosters”) access to the various programme components. The investigation will initially be guided by the routine programme monitoring data and coverage results from the quantitative study with identified issues related to access serving as the first lines of enquiry. This investigation will be iterative and will engage as broad a set of respondents as possible in informal group discussions, key informant interviews, semi-structured interviews and other similar methods eliciting more nuanced responses regarding the various mechanisms and conditions that either support or hinder access to the programme. Then using triangulation of information and inductive reasoning, a concept map of the relationships and connections between various barriers and reasons for non-coverage will be drawn. The following flowchart (see Figure 1) shows the general approach that will be taken to implementing the qualitative study.
 


Figure 1: Flowchart for qualitative investigation
kandhamalcoverage
 

Step 1a: Review of existing or routine programme monitoring data

A review and analysis of the existing routine programme monitoring data will be performed. The data is taken from the various databases used by WFP and its implementing partners for each of the components of the prevention programme. The review will entail simple and easy to implement analysis such as admission and defaulter trends, performance outcomes (cure, default, death rates) and other types of analysis based on the type of available routine programme monitoring data. Following are some examples of the type of analysis that can be performed:
 

Analysis of admissions and defaulters data over time

Common and easily accessible data relating directly to a programme’s coverage include admission and defaulter data. These will first be analysed in isolation before being compared to international standards for indicators (SPHERE) in order to evaluate the programme’s capacity to respond to changes in demand for its services. The number of admissions to the programme will be compiled and presented in graphic form as discussed in the FANTA SQUEAC and SLEAC manual p. 12-16 and as shown in the online SQUEAC and SLEAC toolkit (see discussion of admissions and defaulters data analysis).
 

Building seasonal and work calendars
In order to assess whether changes in admissions over the lifespan of the programme are as expected different calendars such as diseases, labour, hunger, population movement, etc. will be developed for the programme’s environment. Seasonal and work calendars may also be developed and used to establish whether or not they contribute to the pattern of admissions and defaulting. The most commonly used calendars include: diseases affecting young children, periods of intensive labour demand, periods of food insecurity (hunger season) and migration periods (for certain population groups). The graphs showing the trends in admissions and defaulters will be compared with each calendar in order to identify if the programme responds to the increased or the decreased expected numbers. The pattern of admissions should vary over the course of the year accordingly which should correspond to the prevalence of GAM.

A high admissions record in a community-based therapeutic feeding programme when the prevalence of malnutrition is normally high (due to disease or scarcity of food) usually indicates that the programme is responding to need. Malnutrition is expected to be more pronounced during the period when the incidence of other under five diseases (such as diarrhoea and malaria) are high which would impact on malnutrition rates. It is also expected that these diseases will be more common during the rainy season. More admissions are expected in periods when food is available. If there are more admissions during the food secure months/periods then the malnutrition rates can be attributed to the disease incidence such as malaria and diarrhoea. Defaulting patterns will be compared to the labour demands. See a discussion of seasonal calendars in the online toolkit on the topic on admissions over time.
 

MUAC at Admission
MUAC measurement on admission will be collected from the programme beneficiary cards. A tally of recorded MUAC on admission will be done and a histogram created with the MUAC measurements on the x-axis and the corresponding frequencies on the y-axis. The median MUAC on admission can be used as proxy indicator of beneficiaries’ treatment-seeking behaviour. More specifically, it reflects how early or late they seek care. The higher the MUAC on admission the earlier they seek care and the lower the MUAC on admission the later they seek care. Refer to the FANTA SQUEAC and SLEAC manual on p.13-22 or at the online toolkit discussion on MUAC at admission for further details.
 

Discharge Outcomes
Discharge outcomes are also another important element of the routine data analysis. Ideally, a selective feeding programme is considered to be good if it meets SPHERE standards (cure – >75%, defaulters – <15%, death – <10%). To have a clear picture over time it will be important to analyse the discharge outcomes by monthly figures in percentages. This can be done by calculating the percentage of each exit category against the total exits. It should be noted that no standard is currently set for prevention of acute malnutrition programme. However, the current standards for therapeutic programmes can still be used as the higher the coverage the better. A discussion on how discharge outcomes or performance outcomes can be calculated from routine monitoring data is shown in the online toolkit. Variations of these calculations will have to be made for the different components of the prevention programme and should be based on the calculations specified in the CNIP document.
 

Spatial coverage mapping
It is important to be able to visualise the locations where admissions and defaulters are coming from in relation to the programme sites and in relation to where community/ outreach works or volunteers are located. In order to do this, a mapping of the home locations of beneficiaries, defaulters and community volunteers should be done and requires a fairly detailed map of each locality included in the study and the locations of the various clinics/health posts/distribution sites and their respective catchment areas. For the purpose of the spatial sampling approach used in the stepped wedge study, detailed maps for each of the localities in the study were collected which include all the villages in the different locality and the various health clinics/health posts in which the programme is being implemented (see Figure 2). An interactive version of this map has been made and will be used for the spatial mapping of coverage described here.
 


Figure 2: Image of the detailed map of the study localities showing all villages and health centres.

 

Distance to the programme sites
Related to the spatial coverage mapping is distance to the distribution sites. This gives insight as to how much time beneficiaries spend to get to the clinic. This data can be collected from the programme registers or cards where the self-reported distance in hours or km to the site is recorded and categorised based on the minimum and maximum distances recorded in the cards or registers. The frequency of each category for all the sites being investigated over a previous six months period can be compiled in a table form and used to generate a bar graph with the time categories on the x-axis and the corresponding frequencies on the y-axis.
 

Step 1b: Review of stepped wedge and incidence study findings

The results of the stepped wedge and incidence studies can provide additional information that can be triangulated with the information and analysis shown above. This will strengthen the findings or learning that is being gathered from the investigation. Some relevant findings from the stepped wedge study are discussed below.
 

Spatial mapping of study results
The stepped wedge study has been collecting GIS data on the locations of the households from which mother and child data is being collected. The analysis of this data will produce study results of surface maps that will show the various indicator levels at various localities and at high-resolution. An example of this type of output is shown in Figure 3 below.
 


Figure 3: Examples of coverage maps that will be produced from analysis of the stepped wedge data


 

Given this type of outputs from the stepped wedge study, we will be able to have high-resolution information on prevalence and incidence of SAM, MAM and at-risk children across the localities under the study both as aggregates at the locality level and at high-resolution as shown in Figure 3. This can then be triangulated with spatial information on where admissions and defaulters come from (as discussed above) which will identify whether areas of high prevalence or incidence report high admission levels and/or high defaulting levels. Similar maps can also be produced for the coverage of the various programmes. These can then be corroborated with the locations of the health facilities to get an idea of whether distance from a health facility has an impact on coverage. Another spatial analysis that will be performed on the stepped wedge data is a cost surface map. This is basically a high-resolution grid map showing cost to travel (in terms of time)3 from current location to a health facility. This can again be compared to admissions and defaulters data and to coverage maps to show whether areas with high cost of travel have low admissions, high defaulters and low coverage.
 

Step 2: Collection of qualitative data

Further qualitative data can be collected based on the observations made with the routine and quantitative study data. These can be collected through key informant interviews, focused group discussions, semi-structured interviews and documented case studies. This is an important part of these investigations because it allows further development of the hypotheses with regard to coverage. The spatial distribution of the beneficiaries in each of the different aspects of the MAM treatment and prevention packages can guide as to where the qualitative investigations should be conducted. For example, qualitative investigations can be conducted in a sample of sites with high number of admissions versus those with low admissions. Other sites with high default rates can be investigated in comparison to those with low default rates etc. In other words, qualitative data provides vital information concerning the underlying causes of low or high coverage including principle barriers to programme access.
 

Step 3: Synthesise boosters and barriers to coverage into concept map

This qualitative investigation collects a broad set of data using a variety of methods from diverse sources in an intelligent and purposive manner. It therefore requires a means of storing, organising, and analysing data that is designed to generate, visualise, structure, and classify data and ideas in order to solve problems, make decisions, and aid in summarising and reporting complex data. Concept mapping is one of these methods.

Concept mapping is a graphical data analysis technique that is useful for representing relationships between findings. Concept maps show findings and the connections (relationships) between findings. Figure 4 is an example of a concept map using only ‘results in’ or ‘leads to’ relationships.

Concept maps are useful for working out and communicating how different findings (e.g., barriers) are related and interact with each other in complex or cyclical processes (e.g., vicious or virtuous cycles), and in forming hypotheses for further investigation.

For the purpose of this qualitative study, the concept maps will be the key output summarising the voluminous information that will be used and produced by the investigation. The concept map will also serve as the actual “mechanism of change”4 of the programme as it will depict how the programme is actually effecting change.
 


Figure 4: Example of concept map of boosters and barriers to coverage for OTP
conceptmapcoverage
 

Step 4: Generate hypothesis

Based on the concept map, the investigator will draw from the various causal links to generate hypotheses with regard to factors that are considered to either bring coverage up or down. The generated hypotheses will have to be regarding the factors themselves rather than on coverage directly. Some examples of hypotheses that can be generated are:

  • Areas with more outreach workers tend to have increased programme awareness leading to more admissions, less defaulters.
  • Areas with high admissions have good health seeking behaviour and adherence to treatment regimens.
  • Low participation of men in education sessions leads to poor utilisation of OTP, TSFP, FBPM and HF among women.
  • Insecurity in the programme area hinders women from joining or participating in Mother Groups or Lady Groups.
  • HF is highly acceptable in rural sites than in the urban sites

 

Step 5: Perform small area studies and small surveys to test hypotheses

Based on the various hypothesis generated in Step 4, design and implement various small area studies and small surveys to test hypotheses. The design of the studies or surveys will be purposive in nature taking into account the factors being tested and then selecting the parameters of the study or survey accordingly.

The table below shows a list of topic areas and possible hypotheses to be investigated under the Food Based Prevention of Malnutrition, Home Fortification and SBCC prevention intervention programmes.
 

Area of investigation Hypothesis example Proposed method of hypothesis testing
SBCC HF FBPM SBCC HF FBPM
Knowledge of existence of intervention Majority of men not aware of the SBCC CNIP services hence denying their spouses permission to attend or participate in the programme Mothers living close to the distribution sites are more likely to be aware of home fortification programme and related products Populations close to the distribution sites and with more outreach workers are more likely to be aware of FBPM services Informal group discussions with men and women in sites reporting few men participating in education sessions Informal group discussions and key informant interviews with mothers with under-five children Informal group discussions and key informant interviews with mother with under-five children
Utilisation of services or product(s) offered/given by the programme Distance affects the uptake of SBCC messages by the majority of mothers Low utilisation of MNP observed in villages with high default and death rates for the TSFP Migration is a limiting factor for FBPM services amongst pastoralist communities Small studies comparing service utilisation (among mothers with under-five children) in near and far villages to investigate SBCC messages uptake Small studies and small area surveys of defaulters comparing MNP use in areas with high and low defaulters Small studies an key informant interviews with key community figures and mothers with under-five children in select communities
Information source of programme or service or product Community radio awareness raising campaigns has enhanced programme coverage in the target area Mother-to-mother communication is the most effective information source for the MNP Word of mouth and self-referrals contribute a large percentage of FBPM beneficiaries Informal group discussions, key informant interviews with mothers with under-five children, PLWs, key community figures Informal group discussions, key informant interviews with mothers with under-five children Small studies in sites with high and low numbers of beneficiaries using a simple questionnaire administered to programme staff. Reviewing programme registers and beneficiary cards.
Perceptions of the service or product There are positive perceptions or the SBCC messages in villages whose religious leaders have been trained or sensitised about the CNIP services. Negative perceptions of the MNP in villages that are non-Moslems. Community misconceptions of FBPM products lead to high defaulting Informal Group with religious leaders in villages whose religious leaders that have been trained/sensitised compared to villages whose religious leaders have had contact with the SBCC intervention. Informal Group Discussions or Small studies in Non-Muslim sites or villages to check MNP perceptions Small study: Informal Group Discussions with randomly selected men and women in sites/villages with high defaulters and with anecdotal reports of the FBPM products misconceptions

 

Step 6: Update concept map based on new findings

Once the hypotheses have been tested, the initial concept map would need updating accordingly. The confirmed boosters and/or barriers from the concept map can now be used to determine the level of impact each factor has on coverage.

A final concept map is produced. Further analysis can be performed with the concept map by comparing it to the programmes theory of change to see whether they show similar causal links and relationships. This will give a good idea of how the theorised pathways to change were actually realised or not by the programme.

 
 
 

Endnotes

1 Specifically the Semi-quantitative Evaluation of Access and Coverage (SQUEAC) as described in Myatt, M. et al., 2012. Semi-Quantitative Evaluation of Access and Coverage (SQUEAC)/ Simplified Lot Quality Assurance Sampling Evaluation of Access and Coverage (SLEAC) Technical Reference, Washington, DC: FHI 360/FANTA.

2 It should be noted that the focus on coverage as a central topic of investigation for the qualitative study implies the strong causal relationship of coverage to programme effectiveness. Whilst WFP has yet to provide the evaluators a formal and explicit statement of their theory of change, we believe that this will invariably include an element of access and coverage to services as key causal factor to programme effectiveness.

3 Data on time to travel to and from health facility/distribution site is asked of all mothers who have children visiting the facility for any of the programmes.

4 As opposed to the theory of change which is the assumed pathway that the programme will take to effect change.