Onsite training-mentoring intervention improves data quality: an implementation research

Health management information system (HMIS) is one of the six building blocks of health system that integrate data collection, processing, reporting, and use of the information [1]. Low and middle income countries (LMICs) use health information systems as a component of health system reform, though they are experiencing challenges to produce quality data [2, 3]. The quality of health data is an important factor in making decisions and transforming the health sector in order to improve equity and the quality of health care services [1, 4]. The data quality and use remain weak within the health sectors of most LMICs, including Ethiopia [1, 5,6,7].

It is crucial to improve the routine health information system (RHIS) data quality to provide timely information for service provision and to guide intervention strategies in the health sector [4, 8,9,10]. However, there are many challenges to have accurate, timely, and accessible health care data at health care services of most LMICs countries [1, 4]. In Ethiopia, RHIS data quality is not satisfactory for most indicators [11], despite the efforts made to strengthen the health information systems [12, 13]. Accuracy and completeness of facility-based routine data remains a big problem in the country [11, 14, 15]. Thus the quality of data has become a growing concern in the sector, which requires reliable data registration, storage, and management at the facilities and all the health care system [16, 17].

Implementation research aims at scientifically studying the implementation of health interventions, including policies, programs and services, in different real-world settings and within the existing range of health systems [18,19,20]. It is also consider as an efficient and effective tool to accelerate universal health coverage [21, 22]; and more beneficial to sustain evidence based intervention in the health sector [23, 24]. Thus, this implementation research aimed to test the onsite training-mentoring (OTM) intervention and adaptation of the implementation strategy to improve RHIS data quality in the context of public health sector..

Table of Contents

Study setting and period

This implementation research was conducted in selected public health facilities of Jigjiga woreda, Jigjiga Woreda Health Office and Regional Health Bureau of Somali Regional State, Ethiopia. The region shares international borders with Kenya to the south, Somalia to the south-east, and Djibouti to the north-west. The region has 11 administrative zones subdivided into 96 districts (Woreda), and 6 town councils [25]. The region has an estimated total population of 6,506,235 by 2022 (3, 454, 673 males and 3,051,562 females) [26]. Pastoralism, whether nomadic or agro-pastoralism, is practiced by more than 85% of the population. The Regional Health Bureau (RHB), which administers Woreda/District Health Offices (WoHO) and hospitals, is at the top of the health-system structure. The WoHO, in turn, manages health centers and health posts in each district. According to the 2022/2023 Health and Health Related Indicators published by MoH, Ethiopian Somali region has 18 Hospitals, 229 Health Centers and 1496 Health Posts [27]. An onsite training-mentoring intervention strategy was used to improve data quality in public health facilities of Jigjiga woreda from July 2021 to June 2022. Following the intervention, the endline assessment was conducted from July 13 to30, 2022.

Study design and implementation strategy

This research was used an interrupted time series design. The research was guided by a Consolidated Framework Implementation Research (CFIR) framework to identify the main constructs of RHIS data quality and how they apply in the context of the study setting [28]. The research was also aligned with the WHO implementation research steps [20]. This research was used stepwise or cyclic process: Firstly, data quality problems were identified in discussion with implementing institutions. Then, formative assessment was conducted to identify barriers and facilitators to improve data quality in the setting. In this assessment, 179 health care workers were participated in guided self-administered data collection. Additionally, three public health facilities, the Woreda Health Office and the Regional Health Bureau were participated in desk review and observational assessments; while 17 key informants were participated in in-depth interviews of the qualitative study. This assessment found technical, behavioral and organizational problems to improve data quality in the study setting [29, 30].

Secondly, onsite intervention package was developed based on the identified problems of the formative assessment and in consultation with the regional health bureau and health facility managers. The intervention was set based on the context of the study setting to achieve the implementation strategy outcomes. Thirdly, the new intervention was tested and adapted at pre intervention phase. As a result, the intervention strategy modified from horizontal to bottom up approach in consultation with the local stakeholders (Fig. 1). Fourthly, the intervention was implemented for one year with continuous supervision and quarterly review meetings to assess the extent to which implementation intervention was effective and to optimize intervention benefits, and sustain the intervention in the study setting. Finally, post intervention assessment was conducted to assess the improvement of data quality in the study setting. The intervention was focused to improve RHIS data quality, including data recording, documentation, reporting, data analysis and data use. The focus of each level of the intervention was indicated on Fig. 1. A total of seven onsite intervention sessions were given for 154 health workers of the sector in the one year intervention period: First, three round training-mentoring session were given for 67 health workers, who had no prior HIS and related trainings. These training-mentoring sessions were given on data quality, data recording, compilation and documentation. Secondly, two round training-mentoring sessions were given for 44 health workers, who had prior HIS and related trainings. These sessions were focused on data quality, data analysis, interpretation and use. Lastly, two round training-mentoring sessions were given for 43 performance monitoring teams (PMTs) and Woreda and Regional Office workers, who are working on HMIS and leadership positions. These two sessions were focused on integrated DHIS₂ data quality and use. The intervention was given by six trained and experienced public health professionals of the Regional Health Bureau and Haramaya University. Additional, continuous supervision and subsequent review meetings were conducted during the implementation phase.

Sample size and sampling techniques

A single population proportion with a finite population correction formula was used for the post intervention assessment using the following assumption: 77.75% proportion of data content completeness [29], 95% confidence level, 80% power, and a 0.05 margin of error, and a 10% non-response. The total number of health care workers in the study setting was 420. Thus, a correction formula was used. Finally, 187 HCWs were participated on the post intervention assessment.

$$n=Z_\fraca2\,^2 \fracP\left(1-P\right)d^2$$

Initially, Jigjiga woreda was selected from the region for the Doris Duke Charitable Foundation (DDCF) project. Then, three public health facilities were randomly selected from the Woreda (Kara Mara hospital, Jigjiga primary hospital and Ayardaga health center), and the health workers, working in the facilities and who had a direct involvement at least in data recording, compilation and reporting were random selected from each unit in proportion to the size of HWs in each facility. Accordingly, 103 from Kara Mara general hospital, 59 from Jigjiga primary hospital, and 25 from Ayardaga health center were included in the quantitative assessment. Additionally, Jigjiga Woreda Health Office and the Regional Health Bureau of Somali Regional State were included in the qualitative assessment.

Data collection tools and techniques

Similar data collection tools and techniques were used for formative and post intervention assessment surveys. Both the formative and post assessments were collected through a guided self-administered survey, a desk review, and an open observation. A pretest was conducted on neighboring district, Harorays Woreda Health Office and Harorays health center. This pretest was conducted before the formative assessment, which aimed to check the clarity and the flow of questions of the data collection tools and administration of the data collection procedures.

All unites of the health center, the Woreda HMIS focal person, and eight health workers from the facility were participated on the pretest. Accordingly, few questions were slightly modified and also the flow of some questions in the questionnaire and desk review checklist were rearranged. A semi-structured and pre-tested questionnaire and check list were used for data collection. The questionnaire was adapted from previous studies (PRISM) and a WHO document. The questionnaire included questions regarding socio-demographics, knowledge and perception of HIS, HIS training, and basic data analysis and data quality checking related questions. The facility document review check list included questions related with types of services, data sources, documents availability, data reporting, data completeness, timeliness of the reports, and other related questions.

The data quality status of the health facilities was assessed using accuracy/consistency, report/ content completeness, and timeliness of the reports [31].The quality of RHIS data was measured using eight selected main indicators (antenatal care, institutional birth, immunization, VCT, inpatient, tuberculosis, pneumonia and sever acute malnutrition).

The data was collected by six trained and experienced Public Health professionals and three supervisors. The document review assessed the previous three-month reports of the survey for data accuracy/consistency and content completeness, and the previous six-month reports for timeliness and report completeness for the facilities and offices. The desk review was made in all the units where the quantitative data was collected.

Operational definitions

Data accuracy: measured as a similarity between what was in the report and what was in the registrations and/tally sheets. A 10% tolerance level was used to judge the accuracy of data. Based on the 10% tolerance for accuracy, data was classified as follows: Over reporting (110%) respectively [31].

Completeness of facility reporting: Percentage of expected monthly facility reports received for a specified period time. Total number of facility reports received at the unit/total number of expected facility reports at that unit [32].

Data completeness on data recording tools (Registers, cards/forms): This refers to all necessary data elements on registers/forms/cards which should be filled immediately after provision of the service by the care provider [31].

Perceptions of the HWs was collected using five point Likert scale, which ranges from strongly disagree to strongly agree. This dichotomized into do not agree if answer 1 to 3, otherwise coded as agree.

Timeliness of facility reporting is defined as the proportion of reports received from health facilities by subnational administrative units by the deadline for reporting [32].

The HWs HIS Knowledge level was measured using 27 item knowledge questions; and it is coded as “1” if it is correctly answered, otherwise it is coded as “0”. A health workers said to have good knowledge if he/she responds knowledge questions above respondents mean score.

Data quality assurance

We used the same data collection tools and techniques for formative and post intervention assessment surveys. The questionnaire was adapted from previous studies (PRISM) [15, 33] and a WHO document [4]. Refreshment training was given for data collectors and supervisors before the post intervention assessment. There was continuous supervision and monitoring of the data collection process by the supervisors and the investigators.

Data processing and analysis

Data were entered into EpiData 3.1 and exported to SPSS 22.0 Version for data processing and analysis. Descriptive statistical tests like frequency of the outcome variables and other categorical independent variables, as well as mean and standard deviation of continuous independent variables were computed. Then, bivariate analysis using odds ratio was used to compute the strength of the association and statistical significance of the categorical independent variables and the binary outcome variables. Accordingly, HIS knowledge level of the workers, data analysis and presentation skills, socio-demographic variables, HIS training and data recording/documentation valued by PMT variables were considered.

Variables with a P value of 0.25 at bivariate were used as a cutoff point for including independent variables in the final binary logistic regression model. Finally, multivariable binary logistic regression with the enter method was used to identify predictors of the data quality. The odds ratio was calculated with a 95 percent confidence interval to determine the relationship between the dependent and independent variables, and statistical significance was fixed at 0.05. Multi-collinearity was checked using standard errors, all the variables in the model had less than 2.0; and model fitness was checked using the Hosmer–Lemeshow model fitness test, which resulted P value > 0.43.

link