We identified three lessons: 1 develop a robust theory of change TOC ; 2 define optimization outcomes, which are used to assess the effectiveness of the intervention across development phases, and corresponding criteria for success, which determine whether the intervention has been sufficiently optimized to warrant full-scale evaluation; and 3 create and capture variation in the implementation intensity of components.
When applying these lessons to the BetterBirth intervention, we demonstrate how a TOC could have promoted more complete data collection. We propose an optimization outcome and related criteria for success and illustrate how they could have resulted in additional development phases prior to the full-scale trial. These lessons learned can be applied during both early and advanced stages of complex intervention development and evaluation.
By using examples from a real-world study to demonstrate the relevance of these lessons and illustrating how they can be applied in practice, we hope to encourage future researchers to collect and analyze data in a way that promotes more effective complex intervention development and evaluation. Complex interventions often fail to produce the desired results in full-scale trials. However, there is little guidance on how to best develop complex interventions, especially in early stages of the intervention development process.
This study complements existing literature on complex intervention development by highlighting the importance of producing a robust theory of change and demonstrating how quantitative analysis can be used to refine theories of change and promote the development of more effective complex interventions.
Complex interventions consist of a package of several interacting components or implementation strategies [ 1 — 3 ] and are widely used in public health including HIV prevention [ 4 ], smoking cessation [ 5 ], and childhood obesity prevention [ 6 ].
Complex interventions are ideally both effective, or able to produce a heath impact, and optimized, or able to efficiently use available resources in a way that produces the greatest impact possible. Currently, there is little consensus on how to best develop complex interventions. Methodologically rigorous approaches, such as the factorial or fractional-factorial designs often used in the Multiphase Optimization Strategy MOST , can estimate causal effects of individual package components [ 7 , 8 ].
However, these designs require researchers to specify detailed information on candidate components at the beginning of the study, which may not be feasible early in the development process. They can also be prohibitive in cluster randomized studies where few units are available for randomization [ 9 ] or the cost per unique treatment condition is high [ 10 ]. The recently developed Learn-as-You-Go LAGO design allows researchers to estimate the effects of individual components using data collected in phases, with data from previous phases being used to recommend interventions for subsequent phases [ 11 ].
However, this design has yet to be used in a real-world study. In practice, complex interventions are often developed and refined using qualitative research and expert and stakeholder consensus [ 12 — 15 ].
These approaches are rarely accompanied by quantitative analyses estimating the effectiveness of individual implementation components or demonstrating that the intervention has been sufficiently optimized to warrant a full-scale evaluation. Consequently, many interventions fail to produce desired impacts on health outcomes in a full-scale trial [ 16 ]. One recent example of this phenomenon is the BetterBirth intervention, a complex intervention designed to improve the quality of care in childbirth facilities with the goal of improving maternal and neonatal health.
Despite extensive preliminary research during the intervention development process [ 17 ], the intervention did not improve maternal and newborn health in a recent high-profile trial, although it did improve birth attendant adherence to evidence-based practices [ 18 ].
This paper identifies lessons learned from the BetterBirth experience and provides illustrative examples showing how these lessons could be applied to the development and evaluation of future complex interventions. The item checklist is intended to help birth attendants successfully complete evidence-based essential birth practices EBPs that focus on prevention and early identification of complications during facility-based deliveries [ 19 ]. BetterBirth was developed through a multi-phase process.
This initial package was refined over three sequential development phases conducted in primary-level health facilities in Uttar Pradesh, India. The first two phases, pilot 1 and pilot 2, were pre-post studies conducted in two and four facilities, respectively. The third development phase occurred among the first 15 control-intervention pairs enrolled in a matched-pair, cluster randomized trial CRT designed to assess the effectiveness of the BetterBirth intervention on reducing maternal morbidity and maternal and newborn mortality [ 17 , 23 ].
We consider these 15 pairs to constitute a development phase because researchers originally planned to conduct preliminary analyses among these facilities and further adapt the intervention as needed prior to enrolling the remaining CRT facilities. However, time and budgetary constraints ultimately prevented further adaptations to the final implementation package. Across the three development phases, the content, delivery, and intensity of implementation package components assigned to facilities varied, as described by Hirschhorn et al.
During pilot 1, the BetterBirth intervention included three package components: leadership engagement, an educational and motivational program launch, and ongoing coaching visits to promote checklist use and EBP adherence. The fourth package component, a data feedback cycle in which birth attendants were provided with quantitative information on performance, was added to the pilot 2 and CRT phases.
BetterBirth is considered a complex intervention because it uses these multiple, potentially interacting implementation strategies to promote birth attendant checklist use and behavior change. However, practical considerations related to the timing and duration of labor prevented all births from being continuously observed from admission through discharge such that not all EBPs were observed for each birth.
Data on EBP adherence were available on births from pilot 1, births from pilot 2, and births from the 15 pairs of sites from the CRT development phase.
The intervention that was developed as a result of these three development phases was tested in a large-scale matched-pair, cluster randomized trial CRT designed to assess the effectiveness of the BetterBirth intervention on reducing maternal morbidity and maternal and newborn mortality [ 23 ]. The results of this trial showed that the intervention did not improve maternal and newborn health, although it did improve birth attendant adherence to evidence-based practices [ 18 ].
To identify barriers preventing the identification of an optimal BetterBirth implementation package, we reviewed published articles, research protocols, internal reports, data collection tools, implementation team weekly updates, and data from all three development phases of the BetterBirth intervention. The results of this review were used to identify barriers that hindered the identification of an optimal intervention package and corresponding lessons learned.
For each lesson, we described its importance and used material motivated by the BetterBirth Program to illustrate how this lesson could be applied in practice. These illustrative examples are designed to aid in the development and evaluation of future complex interventions. The theory of change TOC proposed in this paper was retrospectively developed following a review of the study materials and refined through discussion with members from the BetterBirth team.
When assessing the effectiveness of coaching, we added coaching intensity to the model, calculated for each birth as the number of coaching visits occurring at their facility in the 30 days prior to their birth.
In pilot 2, only the first and last dates of coaching and the total number of coaching visits per site were recorded, so we calculated coaching intensity metrics by imputing the missing coaching dates assuming a uniform distribution bounded by the first and last coaching dates. To account for facility-level clustering, all standard errors were estimated using the empirical variance with an exchangeable working covariance structure.
We identified three key lessons learned: 1 develop a robust theory of change; 2 define optimization outcomes, which are used to assess the effectiveness of the intervention across development phases, and corresponding criteria for success, which determine whether the intervention has been sufficiently optimized to warrant full-scale evaluation; and 3 create and capture variation in the implementation intensity of intervention components. For each lesson, we describe its importance, discuss how it applies to the BetterBirth Program, and provide an illustrative example.
The assumptions and hypotheses encoded in a TOC can be informed by a wide range of generalized theories commonly used in implementation science [ 28 ] including the Theory of Planned Behavior [ 29 ] or the Theoretical Domains Framework [ 30 ]. However, TOCs differ from generalized theories because they describe causal relationships between variables in a way that is specific to both the intervention of interest and the context in which that intervention is being implemented [ 26 , 27 ].
Additionally, TOCs should contain information on contextual factors expected to modify the relationship between these variables. Although many researchers use the terms logic model and TOC interchangeably, TOCs necessarily include information about the assumed causal connections between variables while logic models often assume simplistic progressions between groups of variables, such as inputs, outputs, outcomes, and impacts e.
The causal assumptions encoded in TOCs provide a structure for identifying and addressing the challenging hallmarks of complex intervention research [ 1 , 2 ].
Testing for the existence of causal links hypothesized in the TOC can help identify ineffective intervention components, highlight incorrect assumptions about the underlying mechanism of change or context in which the intervention is being implemented, and inform future adaptions to the intervention [ 26 , 33 — 35 ]. TOCs can also be used to identify appropriate data sources and units of analysis for each variable and can highlight which data sources will need to be linked together for analysis [ 36 , 37 ].
Finally, TOCs can strengthen collaborations between interdisciplinary team members who may not otherwise share common assumptions or vocabulary for describing the intervention [ 25 , 26 , 38 ]. Implementation strategies and theories of change used during the development of the BetterBirth intervention. This retrospective TOC was developed primarily by program evaluators and implementors, but prospectively engaging community members and frontline healthcare providers can provide additional insight into local context and enhance community buy-in [ 41 , 42 ].
Many variables identified in this TOC as playing important roles in the BetterBirth Program, such as birth attendant ability, were not measured during the development phases. Low baseline birth attendant ability has been hypothesized to be one factor that affected the overall success of the trial [ 18 , 43 , 44 ], and assessing this contextual factor earlier may have helped implementers address this barrier.
Other variables, such as birth attendant attitudes towards the checklist, were measured in only a single phase and therefore could not be compared across phases, preventing deep understanding of how changes to the intervention affected these variables. Finally, the TOC contains hypothesized causal links that exist between variables that were assessed at different units of analysis. For example, attitude towards the checklist, which was assessed at the birth attendant level, was hypothesized to impact checklist use, which was assessed at the individual birth level.
However, because the data collection process did not allow for individual birth attendants to be linked to individual births, this hypothesized link could only be assessed using data aggregated at the facility level. Developing a robust TOC at the start of the intervention development process could have highlighted these limitations, promoted more complete data collection, and provided additional opportunities to learn about the strengths and weakness of the intervention.
After creating a TOC, a subset of process outcomes can be selected as optimization outcomes. Optimization outcomes serve two functions during complex intervention development. Second, comparing optimization outcomes against criteria for success identifies whether the complex intervention is sufficiently optimized to warrant evaluation in a full-scale trial.
Interventions that fail to meet these criteria require additional phases of development before progressing to a full-scale evaluation. To serve these two functions, optimization outcomes should be defined and assessed consistently across all development phases. They should also be valid surrogate outcomes for the primary outcome.
Surrogate outcomes are sometimes used in clinical efficacy trials when collecting data on the primary outcome is expensive, time-consuming, or otherwise infeasible [ 46 , 47 ].
If a surrogate outcome is valid, then the effect of the intervention on the surrogate outcome will correspond to the effect of the intervention on the primary outcome. However, it can be difficult to identify valid surrogate outcomes [ 48 , 49 ].
Empirically validating surrogate outcomes requires data on intervention status, optimization outcomes, and primary outcomes [ 55 ], which are usually unavailable to researchers developing a new intervention. Without empirical verification of surrogate validity, researchers must rely on expert knowledge about the intervention and its expected effects to determine whether a candidate optimization outcome is likely to be a valid surrogate for the primary outcome. In settings where the intervention is hypothesized to improve the primary outcome, researchers typically select an optimization outcome that a is positively correlated with the primary outcome and b is also expected to improve as a result of the intervention.
In this setting, the following conditions will nearly always guarantee that the optimization outcome is a valid surrogate. More general conditions can be found elsewhere [ 48 , 49 ]. First, the positive correlation between the surrogate and the primary outcome should reflect a positive causal effect and not be induced by confounding bias.
For example, the BetterBirth team believed that increased checklist use would be positively correlated with maternal and newborn survival. However, checklist use would not have been a valid surrogate if this correlation was explained by bias, as would have occurred if more educated birth attendants were both more likely to use the checklist and more likely to have good patient outcomes.
Second, if there are mechanisms through which the intervention could have unintended adverse effects on the primary outcome, those mechanisms should also adversely impact the surrogate outcome. In this case, adherence to any single EBP would have been unlikely to serve as a valid surrogate because it would not have reflected the potential unintended consequences of decreased adherence to other tasks. Third, if improvements in the surrogate are only beneficial among a specific subgroup of individuals, then the intervention should improve the surrogate outcome within that subgroup.
For example, in the BetterBirth intervention, antibiotics were only expected to improve survival among the subgroup of women and newborns who were at risk of infection. Therefore, increases in antibiotic prescription rates would have only been a valid surrogate for survival if those increases occurred specifically among women and newborns identified as being at risk of infection.
In addition to selecting a valid optimization outcome, researchers must also specify criteria for success. Determining what criteria the optimization outcome must achieve for the intervention to have a reasonable chance of improving the primary health outcomes in a full-scale trial depends on both the strength of the relationship between the optimization outcome and the primary health outcomes and whether that relationship exhibits non-linear trends.
This relationship can be relatively weak if the intervention does not target all major determinants of the primary health outcome. For example, in the BetterBirth Program, the association between EBP adherence and maternal and neonatal mortality may have been weaker than expected if key determinants of maternal and neonatal health that were not targeted by the intervention, such as inadequate antenatal care, were responsible for a substantial proportion of deaths.
Situational considerations should also dictate whether criteria for success are set in relative or absolute terms. For example, if researchers believe that the optimization outcome must cross a certain threshold to impact the primary outcome, the criteria for success should reflect that absolute threshold, not relative improvements.
If multiple process outcomes are selected as optimization outcomes, the criteria for success should account for each of these outcomes, either by combining them into a single composite outcome e. Selecting optimization outcomes and specifying criteria for success are both critical steps in the optimization process. If either is misspecified, then researchers could develop an intervention that improves the optimization outcome and meets the criteria for success but still fails to impact the primary outcome.
Finally, the sample size for each development phase should be calculated with respect to the optimization outcomes and their corresponding criteria for success. However, each phase should be powered such that estimates for the effect of the intervention on the optimization outcome are precise enough to inform a decision to proceed to the full-scale trial.
For example, if the criteria for success are defined as observing a confidence interval for the optimization outcome that includes or exceeds some pre-specified value [ 56 ], then power calculations should ensure that the confidence intervals for the optimization outcome will be informatively narrow [ 45 , 57 ].
Adherence to each individual EBP was analyzed and reported independently, with different sets of EBPs used in different development phases [ 17 , 18 ]. The criteria for success were not specified for any EBP. This approach had several limitations. First, adherence to a single EBP is unlikely to constitute a valid surrogate. As previously discussed, adherence to individual tasks would not capture potential unintended adverse consequences of the intervention. Furthermore, secondary analysis of BetterBirth Trial data suggests that individual EBPs were not correlated with improved health outcomes, suggesting that adherence to any individual EBP is not a valid surrogate for maternal and newborn health [ 58 ].
Second, without pre-defined criteria for success, it is unclear how EBP adherence data informed the decision to progress to a full-scale trial. If criteria for success had been defined in terms of absolute EBP adherence, these improvements may have been recognized as too modest to result in meaningful health improvements, triggering additional development phases.
Third, these limitations were compounded by inconsistent data collection across the three development phases. Not all EBPs were assessed in all phases, and the timing and duration of EBP data collection relative to the start of coaching differed from phase to phase. We defined overall EBP adherence as the proportion of observed EBPs that were successfully completed at each birth out of a set of eight EBPs that were measured consistently across all three phases Table 2.
This composite outcome was expected to serve as a valid surrogate for maternal and newborn survival for several reasons. First, overall EBP adherence was correlated with improved newborn survival [ 58 ], and it was assumed that this correlation reflected a causal effect. Second, the set of EBPs included in the composite outcome included a wide range EBPs which were measured at all three pause points and included EBPs performed on both women and newborns, so potential unintended adverse effects occurring at any stage in the birthing process would likely have been reflected by a reduction in overall EBP adherence.
Finally, each EBP included in the composite outcome was believed to be beneficial for all births, not just for a certain subgroup. Our TOC supported the use of overall EBP adherence as an optimization outcome because a it was proximal to the primary outcome of newborn and maternal mortality and b there were no hypothesized causal pathways from the intervention components to newborn and maternal mortality that did not go through EBP adherence.
Our analysis suggested that, although the intervention increased EBP adherence in all phases, changes to the implementation package across phases did not meaningfully improve overall EBP adherence.
The intervention did not cross the threshold needed to achieve the criterion for success, suggesting that the implementation package was not sufficiently optimized to produce the desired health impact at the time of the trial. Effectiveness of each phase of the BetterBirth intervention on overall essential birth practice EBP adherence, which was calculated as the percentage of observed EBPs that were successfully completed out of eight EBPs consistently observed across all three phases: 1 use of a partograph, 2 maternal blood pressure at admission, 3 maternal temperature at admission, 4 appropriate hand hygiene prior to a push, 5 provision of oxytocin to the mother within 1 min of delivery, 6 assessment of baby weight, 7 assessment of newborn temperature, and 8 initiation of breastfeeding within 1 h.
If the criteria for success are not satisfied, investigating relationships between individual implementation components and other variables in the TOC can help identify strategies for improving the intervention.
Implementation intensity can be quantified using domains that include content, quality, frequency, and duration [ 59 , 60 ]. For each component, implementation intensity can be coded as a categorical variable e. Variation in implementation intensity can arise from both planned and unplanned factors.
Planned variation occurs when researchers assign study participants to receive different intensities of an intervention component, as is the case in multi-arm studies [ 61 ] and factorial designs [ 8 ]. Unplanned variation in implementation intensity is often described in terms of fidelity, or the extent to which the delivered intervention deviates from what was originally planned [ 63 — 65 ]. Although sources of variation in implementation intensity may be unplanned, they can be anticipated and measured.
Observing an association between the intensity of an individual implementation component and relevant process outcomes identified in the TOC can provide evidence to evaluate the effectiveness of that component [ 2 ].
As with all observational research, the extent to which this association reflects causal effects depends on the extent to which other confounders are accounted for [ 67 — 69 ]. In the case of complex interventions, special care should be taken to adjust for the remaining intervention components using either randomization e.
The more strongly two components are correlated, the more difficult it is to identify their independent effects. Strong correlations often arise from the study design. For example, intractable collinearity occurs when researchers simultaneously introduce, intensify, or diminish the intensity of multiple components in a single arm or phase of a study.
Collinearity can also occur if a common factor, such as highly motivated leadership, simultaneously affects fidelity to multiple intervention components. To better estimate the effectiveness of individual implementation components, researchers may wish to both create planned, uncorrelated variation in implementation intensity while also capturing unplanned variation that arises in the field. Although the intensity of implementation package components varied across the BetterBirth development phases by design Table 1 , multiple components were simultaneously intensified in each phase.
This practice created strong collinearities between individual components and prevented the identification of their individual effects. For example, the effect of having non-standardized leadership engagement could not be isolated from the effect of a 3-day launch duration since each of these conditions appeared only in pilot 1.
Fidelity was not systematically measured for any component. In addition to this planned source of variation, the BetterBirth team gathered data on the dates of the coaching visits, allowing us to assess unplanned variation that occurred when sites deviated from the intended coaching schedule. Unfortunately, due to the multi-collinearity of the remaining components, coaching is the only intervention package component whose individual effect can be validly analyzed.
We tested for the existence of this relationship by assessing the association between coaching intensity, defined for each infant as the number of coaching visits occurring at their facility in the 30 days prior to their birth, and overall EBP adherence.
We observed a linear dose-response relationship between the number of coaching visits per month and overall EBP adherence Fig. This association suggested that coaching was an effective intervention component.
Other implementation components may have needed to be added or intensified for the BetterBirth intervention to be effective. Through our review of the BetterBirth intervention and associated trial, we identified three lessons learned that can help future researchers develop and evaluate complex interventions: 1 develop a robust theory of change, 2 define optimization outcomes and criteria for success, and 3 create and capture variation in the implementation intensity of individual components.
Our illustrative examples demonstrate how these lessons could have been applied to BetterBirth. Identifying an optimization outcome that was a valid surrogate for maternal and newborn health and comparing it against pre-defined criteria for success could have led to additional phases of intervention development prior to the full-scale trial.
Finally, capturing and creating variation in implementation intensity for each implementation component could have helped identify which implementation components were effective and which needed additional adaptation. These three lessons can be applied in both exploratory and methodologically rigorous phases of complex intervention development and evaluation.
These lessons are highly compatible with the MOST framework, which also encourages researchers to begin with a well-developed theoretical framework and to only proceed to a full-scale trial after reaching some minimal effectiveness threshold [ 7 , 10 ]. Mousa, H. Treatment for primary postpartum haemorrhage. Cochrane Database of Systematic Reviews. Government of India. Prinja, S. Availability of medicines in public sector health facilities of two North Indian States.
BMC Pharmacology and Toxicology, 16 , Rural Health Statistics. Sabde, Y. The availability of emergency obstetric care in the context of the JSY cash transfer programme in Madhya Pradesh, India. BMC Pregnancy Childbirth, 16 1 , Selvaraj, S. Semrau, K. Effectiveness of the WHO Safe Childbirth Checklist program in reducing severe maternal, fetal, and newborn harm in Uttar Pradesh, India: Study protocol for a matched-pair, cluster-randomized controlled trial.
Trials, 17 , Outcomes of a coaching-based WHO safe childbirth checklist program in India. New England Journal of Medicine, 14 24 , — Sharan, M. Intrapartum oxytocin use for labor acceleration in rural India. Spector, J. Access to essential technologies for safe childbirth: A survey of health workers in Africa and Asia. BMC Pregnancy and Childbirth, 13 , Tsu, V.
New and underused technologies to reduce maternal mortality—ScienceDirect. The Lancet, , 75— Wagstaff, A. Child health: Reaching the poor. American Journal of Public Health, 94 5 , — Wedderburn, R. Quasi-likelihood functions, generalized linear models, and the Gauss—Newton method.
Biometrika, 61 3 , — Google Scholar. Zupan, J. Perinatal mortality in developing countries. New England Journal of Medicine, , — Download references. The authors would like to acknowledge the BetterBirth Team, especially the data collectors who traveled far and wide to collect this data. Tuller, Bridget A. Neville, Atul A. Population Services International, Washington, D. You can also search for this author in PubMed Google Scholar. Correspondence to Katherine E.
No conflicts except AAG receives royalties for books and essays, including on improving quality and delivery of health care using checklists. Reprints and Permissions. Galvin, G. Matern Child Health J 23, — Download citation. Published : 14 November Issue Date : 15 February Anyone you share the following link with will be able to read this content:. Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative. Skip to main content. Search SpringerLink Search. Download PDF. Significance The absence of a single essential supply may mean the difference between life and death for a mother or newborn.
Background Despite global progress, maternal and neonatal mortality rates remain high. Flow diagram of facilities surveyed for safe birth supplies in Uttar Pradesh, India. Full size image. Table 1 Groupings of essentials supplies for safe delivery of normal and uncomplicated birth assessed in facilities in Uttar Pradesh, India Full size table. Table 2 Facility characteristics for public-section labor and delivery facilities across 38 districts in Uttar Pradesh, India Full size table. Table 3 Overall and composite supply measures: infection prevention, newborn supplies, and maternal supplies across facilities in Uttar Pradesh, India Full size table.
Average availability of supplies for districts with six or more facilities. Table 4 Univariate and multivariable regression analysis: correlation of facility characteristics and supply availability Full size table.
Discussion Lack of availability of essential birth supplies was pervasive across the facilities in Uttar Pradesh and varied by district. Conclusion Despite high availability of supplies in some facilities, significant room for improvement remains; the absence of a single essential supply may lead to poor outcomes for mothers and newborns.
References Carvalho, N. Book Google Scholar Foy, H. Retrieved 05 July Hirschhorn, L. Article Google Scholar Hirschhorn, L. Article Google Scholar Kruk, M. Article Google Scholar Maisonneuve, J.
Article Google Scholar Sharan, M. Google Scholar Zupan, J. Acknowledgements The authors would like to acknowledge the BetterBirth Team, especially the data collectors who traveled far and wide to collect this data. Semrau Authors Grace Galvin View author publications. View author publications. Ethics declarations Conflict of interest No conflicts except AAG receives royalties for books and essays, including on improving quality and delivery of health care using checklists.
This includes providing updates on progress and feedback of implementation data, partnering with key figures at each level to hold joint public meetings with national, state, and district leaders to reflect on progress of the program and trial, and providing training to government officials in areas of local interest.
These partnerships have ensured that our program and trial design have been carefully reviewed and tailored to ensure alignment with ongoing and planned government programs.
This process has also provided key insight regarding the feasibility of study implementation within the infrastructure of UP. Challenges in this area will continue as government officials change positions, highlighting the need to ensure sustained, effective relationships even as personnel change over the relatively long period of data collection.
At all levels, one of the most critical challenges is the need for local champions, as well as strategies for sustainability of the program, particularly given the frequent reassignment of facility staff across UP.
To address this potential barrier, we have incorporated the training of a facility-level Childbirth Quality Coordinator CQC to continue supporting the use of the SCC during the study and once study coaches have completed the 8-month-long intervention.
Further, the engagement process includes a complete discussion of Checklist implementation and data collection. The participation of facility leaders in the engagement and launch processes build local buy-in for Checklist use.
Similar processes are held at the district and state levels to ensure that priorities align and health leaders have ownership in the project and dedication to quality of care. All of these processes are designed to sustain the use of the SCC. Partnerships among the investigators and the use of expert local staff for implementation and management ensure that we remain cognizant of cultural and infrastructure determinants impacting the health care and research environments.
These allow us to deal with challenges in ensuring informed consent, with ensuring consistent meaning of questions despite variation in dialects across the state, and with operational issues related to how women use health care facilities and recover postpartum, as well as the role of traditional birth attendants in facility based births.
As has been shown previously, simply adding a checklist to workflow will not create uptake or sustainability [ 28 ]. The greatest challenge is in ensuring that individual health workers successfully adopt the SCC program into clinical practice. Sustained behavior change in any health care setting is challenging, especially in resource-limited settings [ 29 ].
Early pilot studies resulted in the better understanding of how to educate birth attendants on how to use the SCC and be motivated to improve their own clinical practice [ 30 ]. In our lessons learned from the pilot studies, one key finding is the importance of using education and peer-to-peer coaching to empower health workers to believe in their own capacity for progress, and to realize that through their own practices they can ensure that maternal and neonatal outcomes may be improved.
Further, with the development of the CQCs and facility staff to take on responsibility and utilization of the SCC, we hope to empower the CQCs such that they will be able to collect, collate, coach, interpret, and feedback data to the frontline workers and provide information to the facility leaders.
The main risk with the data collection stems from the lack of a comprehensive, robust existing health management information system in the study setting. Based on learning from the pilot, the original plan to use standardized birth registers to collect routinely available demographics and in-facility outcome data has required compilation of multiple primary data sources. In baseline data collection, we have successfully collated the data and been able to follow up women and their newborns after discharge by telephone.
After the pilot studies, the standard maternal severe morbidity definitions were modified to reflect the available resources. We reviewed a range of methodologies for self-report on these and other time-delineated outcomes [ 24 , 31 , 32 ]. In addition, the use of maternal near-missed deaths as a component of a composite indicator that combines maternal and newborn outcomes is novel. Another potential risk is contamination of the control sites in this matched-pair, randomized controlled trial.
To mitigate the above risks, we adopted methodological strategies to maximize the implementation of successful trial conduct in this real-world setting. We completed several pilot studies in facilities in UP, using a quality improvement methodology, and then measured success in both effectiveness of education as well as rates of adoption of the care practices comprising the SCC. Through these pilot tests in nine facilities, we progressively modified our approach until we were certain that we had removed all identifiable impediments to successful implementation, data collection, and monitoring [ 30 ].
These changes included but were not limited to a complete re-evaluation of the methods for launch and coaching support. Data from this extensive pilot phase will not be used in the final analyses. In summary, simple, scalable solutions are essential, and desperately needed, to improve maternal and neonatal outcomes around the time of childbirth. Utilization of pilot studies and iterative learning has improved the design of the trial and intervention, as well as the data collection systems, for implementation of a high-quality, large-scale study.
Studies such as this with large land coverage and sample size require immense coordination at all levels of the health system. If the SCC and coaching intervention are found to reduce maternal, fetal and neonatal harm, patients and other stakeholders stand to benefit from a proven quality improvement strategy that could potentially help influence outcomes in millions of births each year.
Maternal mortality for countries, — a systematic analysis of progress towards Millennium Development Goal 5. Article PubMed Google Scholar. Neonatal mortality levels for countries in with trends since a systematic analysis of progress, projections, and priorities. PLoS Med. Every newborn: progress, priorities, and potential beyond survival. Stillbirths: where?
How to make the data count? An assessment of the impact of the JSY cash transfer program on maternal mortality reduction in Madhya Pradesh.
India Glob Health Action. The checklist—a tool for error management and performance improvement. J Crit Care. A surgical safety checklist to reduce morbidity and mortality in a global population.
N Engl J Med. An intervention to decrease catheter-related bloodstream infections in the ICU. Int J Gynaecol Obstet. Improving quality of care for maternal and newborn health: prospective pilot study of the WHO safe childbirth checklist program. PLoS One. Patabendige M, Senanayake H. Implementation of the WHO safe childbirth checklist program at a tertiary care setting in Sri Lanka: a developing country experience.
BMC Pregnancy Childbirth. Deaths from heart failure: using coarsened exact matching to correct cause-of-death statistics. Popul Health Metrics. Article Google Scholar. J Policy Anal Manage. Reducing HIV risk among transgender women in Thailand: a quasi-experimental evaluation of the sisters program. Coarsened exact matching in stat. Stata J. Google Scholar. The essential role of pair matching in cluster-randomized experiments, with application to the Mexican universal health insurance evaluation.
Stat Sci. Commission GoIP. Press note on poverty estimates, — Uttar Pradesh Population Census data Accessed 1 Nov Health and family welfare statistics in India New Delhi: Development and Research Services; Second Updation Bulletin — Pattinson R.
Near miss audit in obstetrics. Maternal near miss—towards a standard tool for monitoring quality of maternal health care. Evaluating the quality of care for severe pregnancy complications: the WHO near-miss approach for maternal health.
Geneva: World Health Organization; Paediatr Perinat Epidemiol. Haybittle JL. Repeated assessment of results in clinical trials of cancer treatment. Br J Radiol.
Anthes E. Hospital checklists are meant to save lives—so why do they often fail? Enculturating science: community-centric design of behavior change interactions for accelerating health impact. Semin Perinatol. Learning before leaping: integration of an adaptive study design process prior to initiation of BetterBirth, a large-scale randomized controlled trial in Uttar Pradesh. India Implement Sci. Measuring maternal health: focus on maternal morbidity.
Bull World Health Organ. WHO systematic review of maternal morbidity and mortality: the prevalence of severe acute maternal morbidity near miss. Reprod Health. Download references. We thank the Governments of India and Uttar Pradesh for collaboration and support to conduct this trial in public health facilities.
We also thank the past and current members of the BetterBirth study team in Boston and the BetterBirth field team based in Uttar Pradesh for study implementation, especially William Berry for his guidance. The funders will not have input on data collection, management, analysis, or interpretation of the data. Further, they will not have any authority over the writing of the reports or decision to submit findings for publication.
Data related to this study protocol will be made publicly available within 1 year following completion of the study. SL and GK designed the biostatistic approaches and conducted sample size calculations. All authors contributed to development of the study protocol and have contributed to and agreed to the final version of the manuscript.
AAG receives royalties for books and essays focused on patient safety and medicine. The Indian Council of Medical Research also approved the study. The protocol will be reviewed and reapproved on an annual basis. The trial is registered at ClinicalTrials.
0コメント