Local health department (LHD) vector control programs have experienced reductions in funding and capacity. Acknowledging this situation and its potential effect on the ability to respond to vector-borne diseases, the U.S. Centers for Disease Control and Prevention and the Public Health Foundation partnered on a performance management initiative for LHD vector control programs. The initiative involved 14 programs that conducted a performance assessment using the Environmental Public Health Performance Standards. The programs, assisted by quality improvement (QI) experts, used the assessment results to prioritize improvement areas that were addressed with QI projects intended to increase effectiveness and efficiency in the delivery of services such as responding to mosquito complaints and educating the public about vector-borne disease prevention. This article describes the initiative as a process LHD vector control programs may adapt to meet their performance management needs. This study also reviews aggregate performance assessment results and QI projects, which may reveal common aspects of LHD vector control program performance and priority improvement areas. LHD vector control programs interested in performance assessment and improvement may benefit from engaging in an approach similar to this performance management initiative.
Introduction
Almost half of the approximately 2,800 local health departments (LHDs) in the U.S. provide vector control services. LHD, as defined by the National Association of County and City Health Officials, refers to “an administrative or service unit of local or state government, concerned with health, and carrying some responsibility for the health of a jurisdiction smaller than the state.”1 LHD vector control programs are commonly considered a responsibility of environmental public health, a foundational area of public health.2,3 Vector control programs provide services to address vectors, primarily mosquitoes, that may include performing site assessments, providing pesticide treatments, establishing policy and regulations, educating the public, and conducting surveillance to detect and prevent vector-borne disease transmission.
LHD vector control programs have experienced reduced capacity resulting from decreased budgets.4 Decreased funding to support state health department and LHD West Nile virus and other arbovirus surveillance activities could be a factor impacting capacity.5 These reductions in funding and capacity may affect the ability of vector control programs to identify and respond to existing and emerging threats, including the spread of the mosquito-borne Chikungunya and Zika viruses in the Americas.6,7 Considering these impacts, the U.S. Centers for Disease Control and Prevention (CDC) and Public Health Foundation (PHF), a nonprofit organization that seeks to improve the quality and performance of public health practice, supported an LHD vector control program performance management initiative. The primary objectives of the initiative were to support a performance assessment process and use the results to inform and implement quality improvement (QI) projects. This study describes the initiative as a process LHD vector control programs may adapt to meet their performance management needs. It also reviews aggregate performance assessment results and QI projects, which may reveal common aspects of LHD vector control program performance and priority improvement areas.
This initiative began in November 2014, as 14 participating programs conducted self-facilitated assessments using the Environmental Public Health Performance Standards (EnvPHPS) Version 2.0 with the goal of determining the level at which they provided the 10 Essential Environmental Public Health Services (EEPHS). The EnvPHPS provides a set of standards describing activities an environmental public health program should conduct to perform the EEPHS.
The EnvPHPS can be used for assessments at varying organizational levels, including environmental public health systems, which involve a broad range of entities and partners, departments, and specific programs such as vector control.8 Environmental public health departments have previously used EnvPHPS and the EEPHS to assess their performance and inform QI efforts.9–1011 PHF assigned a QI expert to each vector control program to assist with using EnvPHPS assessment results to identify and prioritize improvement areas and implement QI projects to address these areas. The QI experts emphasized aspects of efficiency such as reducing mosquito complaint response time and effectiveness, including enhancing vector-borne disease prevention outreach and educational efforts.
Methods
Participants
A total of 14 LHD vector control programs from 13 states participated in the initiative from November 2014 through May 2015. The majority of the programs were identified by contacting Environmental Public Health Leadership Institute (EPHLI) alumni from LHD and state health department. EPHLI, supported by the CDC from 2005 to 2012, accepted approximately 30 practicing environmental health professionals each year into the yearlong program designed to enhance leadership skills and abilities.12 The alumni indicated whether their department had a vector control program and whether they were interested in participating in the initiative. In some cases, the alumni were able to recommend another point of contact within the LHD or another jurisdiction if they were not personally involved with the vector control program or a program did not exist at their department. Additionally, internet searches identified LHD vector control programs in geographic areas were not represented by the EPHLI alumni.
While mosquito control agencies or districts not operated by an LHD serve many jurisdictions across the nation, this initiative focused specifically on LHD vector control programs that may address a wider range of vectors and provide a broader scope of public health services. Two of the LHD vector control programs that participated in the initiative were collocated with a mosquito control agency in their jurisdiction. The vector control programs that took part in the initiative represented different regions of the country and variation in population size served, organizational characteristics, and services. For example, two vector control programs were located in the Midwest, three in the Northeast, eight in the South, and one in the West U.S. Census Regions. A limited number of LHD vector control programs were identified in the West, which may have resulted from a seemingly high prevalence of mosquito control agencies in this region. One vector control program served a population <100,000, five served populations of 100,000-500,000, four served populations of 500,001-1,000,000, and four served populations >1,000,000. Four programs had budgets <$1,000 per 10,000 population, five between $1,000 and 9,999, three between $10,000 and 20,000, and one had a budget of >$20,000 per 10,000 population (data were missing for one program). Nine programs had less than one vector control full-time equivalent (FTE) per 100,000 population, four had 1-3 FTEs per 100,000, and one program had approximately five FTEs per 100,000. The scope of services provided by the vector control programs varied, yet mosquito-related services and activities were common among all the participating programs.
Performance assessment
Each of the 14 LHD vector control programs conducted self-facilitated EnvPHPS assessments in a group setting, with participation from three to eight staff members. The EnvPHPS and supporting resources, including a tool for analyzing results, are publicly available on the CDC website.13 An instructional webinar provided each program with guidance on using the EnvPHPS and conducting the performance assessment. During the assessment, staff members engaged in discussion while considering and responding to 64 measures organized by EnvPHPS model standards that correspond to the 10 EEPHS (see Table 1 for a list of the 10 EEPHS and corresponding EnvPHPS Model Standards). Responses were based on a scale of five activity levels: no activity (0%), minimal activity (>0%-25%), moderate activity (>25%-50%), significant activity (>50%-75%), and optimal activity (>75%-100%). Each vector control program assigned a recorder that indicated the responses on a data collection instrument and noted major discussion points during the self-assessment for later reference when reviewing results and determining QI project topics. The programs emailed the completed instruments to the PHF project manager for entry into the EnvPHPS tool for analyzing results, which computed an overall activity level as a percentage for each EEPHS. Assessment results were provided to the vector control programs for their review and consideration, as they entered the QI phase of the initiative. EnvPHPS performance assessment results for all 14 vector control programs were aggregated to reveal average activity levels (reported as a percentage) according to each EEPHS.
Table 1
10 Essential environmental public health services and corresponding environmental public health performance standards model standards.
Quality improvement
Between January 1 and May 31, 2015, the vector control programs completed QI projects as QI experts provided guidance and consultation during four webinars, regular phone discussions, and two day-long site visits. PHF's QI experts are individuals who have served in senior leadership roles in public health, health care, and international business, are or have been faculty at renowned universities, and have extensive experience developing, refining, and improving QI tools and techniques for public health practitioners. Each program received two copies of the Public Health Quality Improvement Encyclopedia, written by PHF's QI experts, which provides explanations and examples of 75 QI tools.14 One of these experts assisted each vector control program with a review of their EnvPHPS performance assessment results, identifying and prioritizing improvement areas according to the 10 EEPHS and implementing one QI project to address the program's top priority. Programs were encouraged to choose a project that could be completed within the initiative time frame. The QI experts then facilitated the use of QI tools such as Gantt charts, cause-and-effect diagrams, flow charts, and a population health driver diagram framework to assist programs leverage partnerships. The QI experts assisted with the establishment of goals and objectives that programs sought to accomplish over the five-month project period.
Results
Performance assessment
Aggregate EnvPHPS performance assessment results of all 14 programs indicated the average level of activity for each of the 10 EEPHS (see Table 2 for aggregate EnvPHPS performance assessment results). On average, vector control programs performed higher in EEPHS 2 diagnose and investigate (µ = 76.3%, SD = 11.7) and EEPHS 3 inform, educate, and empower (µ = 71.1%, SD = 18.9). Lower assessment results were identified with EEPHS 1 monitor environmental and health status (µ = 43.3%, SD = 30.1) and EEPHS 9 evaluate effectiveness, accessibility, and quality (µ = 43.4%, SD = 26.9). A moderate (>25%-50%) to significant (>50%-75%) level of activity was noted for EEPHS 7 link to services (µ = 47.9%, SD = 27.7), EEPHS 10 research (µ = 53.6%, SD = 18.5), EEPHS 6 enforce laws and regulations (µ = 53.8%, SD = 28.9), EEPHS 4 mobilize partnerships (µ = 56.9%, SD = 26.9), EEPHS 5 develop policies and plans (µ = 57.1%, SD = 15.7), and EEPHS 8 assure a competent workforce (µ = 62.0%, SD = 21.0).
Table 2
Aggregate performance assessment results for 14 local health department vector control programs using the environmental public health performance standards.
Quality improvement
The vector control programs completed QI projects focused on a range of topics that spanned from increasing community outreach and educational efforts to decreasing the time required for resolving complaints. The majority of projects involved mosquito-related services and activities, while one project broadened a community-based rodent survey to include mosquito control aspects and another reviewed rabies investigation processes to examine staff workloads. All vector control programs made accomplishments, yet not all fully achieved established QI project goals and objectives within the initiative time frame. Because the QI projects sometimes reflected more than one EEPHS, the authors reviewed each of the projects and agreed upon the primary EEPHS addressed. Overall, the projects addressed seven of the 10 EEPHS. EEPHS 5 (develop policies and plans), EEPHS 6 (enforce laws and regulations), and EEPHS 10 (research) were not addressed. Projects most frequently addressed EEPHS 2 (diagnose and investigate), EEPHS 4 (mobilize partnerships), and EEPHS 9 (evaluate effectiveness, accessibility, and quality). All 14 programs strengthened existing or newly built partnerships internal or external to the LHD (see Table 3 for a description of the QI projects).
Table 3
Local health department vector control program quality improvement project descriptions.
Discussion
Aggregate performance assessment results of all 14 vector control programs indicated the highest levels of performance in EEPHS 2 (diagnose and investigate) and EEPHS 3 (inform, educate, and empower). This likely represented the emphasis that vector control programs placed upon surveillance as related to EEPHS 2 and outreach and community awareness efforts pertaining to EEPHS 3. EEPHS 1 (monitor environmental and health status) and EEPHS 9 (evaluate effectiveness, accessibility, and quality) revealed the lowest performance levels. Three QI projects addressed EEPHS 9 by emphasizing the evaluation of community satisfaction and effectiveness of services. Lower assessment results in EEPHS 9 may show a need for increased support for evaluating vector control services and determining their efficacy. Vector control programs interested in addressing EEPHS 1 may benefit from efforts to enhance capabilities for conducting community assessments and utilizing technologies such as geographic information systems to monitor vector concerns. The absence of QI projects addressing EEPHS 5 (develop policies and plans), EEPHS 6 (enforce laws and regulations), and EEPHS 10 (research) may reveal lower priority assigned to improve related vector control services, perhaps influenced by higher levels of activity as aggregate results were significant (>50%-75%) for each of these EEPHS. Nearly all of the QI projects focused on mosquitoes. The LHD vector control program that did not address a topic related to mosquitoes was collocated with a mosquito control agency. The presence of the mosquito control agency may have contributed to the LHD vector control program's decision to emphasize a QI project that did not focus on mosquitoes.
Performance assessment results varied among the programs, which may have resulted from wide variation in population size served and organizational characteristics (eg, staffing levels and funding). Participation was voluntary, and results cannot be generalized to represent all U.S. LHD vector control programs. Effective prioritization of assessment results and detailed project planning were essential because of the short five-month duration of the project period. Time limitations, especially as the spring months and peak vector season approached, likely influenced project selection in favor of approaches that could be accomplished within the initiative time frame and supported by existing financial and staffing resources. This situation may have caused programs to avoid addressing EEPHS 1 and EEPHS 9 with monitoring and evaluation-related QI projects.
In the absence of funding to participate in the initiative, the dedicated programs were able to identify means for improvement within the boundaries of available resources. LHD vector control programs may be able to conduct performance management activities, similar to this initiative, without additional funding to support their efforts. All programs indicated that engaging in this initiative resulted in stronger collaboration with partners within the LHD, other governmental agencies, and private entities. This result could be particularly important for supporting collaborative approaches between LHD vector control programs and mosquito control agencies when these two types of organizations are collocated within a jurisdiction.
This performance management initiative provided insight into vector control program performance according to the 10 EEPHS and demonstrated how the EnvPHPS can serve as a performance assessment framework for environmental public health programs, such as vector control, to produce results for informing QI projects. The QI experts provided valuable guidance for the vector control programs; however, programs undertaking similar performance assessment and QI improvement activities may leverage the multitude of resources publicly available online or in references such as the PHF QI encyclopedia. LHD vector control programs interested in performance assessment and improvement may benefit from engaging in an approach similar to this performance management initiative.
Author Contributions
Conceived and designed the experiments: JG, MK, JWM, RB, JS. Analyzed the data: JG, MK. Wrote the first draft of the manuscript: JG, MK, VL. Contributed to the writing of the manuscript: JG, MK, JWM, VL. Agreed with manuscript results and conclusions: JG, MK, JWM, RB, JS, VL. Jointly developed the structure and arguments for the paper: JG, MK, JWM, VL. Made critical revisions and approved the final version: JG, JS, VL. All the authors reviewed and approved the final manuscript.
Acknowledgments
The authors would like to thank the following health departments for participating in the performance management initiative: Alexandria Health Department, VA; Boulder County Public Health Department, CO; Florida Department of Health, Broward County, FL; Cuyahoga County Board of Health, OH; Louisville Metro Public Health and Wellness Department, KY; Madison County Health Department, AL; Madison County Health Department, NY; Maricopa County Department of Public Health, AZ; New Hanover County Health Department, NC; Onondaga County Health Department, NY; City of Philadelphia Public Health Department, PA; Saint Louis County Public Health Department, MO; Shelby County Health Department, TN; and Tulsa Health Department, OK.
