141st APHA Annual Meeting and Exposition

Home Events & Meetings Annual Meeting Schedule & Program

Recorded Annual Meeting Presentations (RAMP) are now available for purchase.
Only $90 for presenters, $140 for registered attendees and $200 for non-attendees.

Online Program

275864
Development of performance measures for wide-ranging clinical research programs in diverse global settings


Monday, November 4, 2013

Susanna Weiss, Ph.D., J.D., CEP, Office of Clinical Research Policy and Regulatory Operations -- Division of Clinical Research, National Institute of Allergy and Infectious Diseases -- National Institutes of Health, Rockville, MD
Yin Li, B.Sc., Office of Planning and Operations Support, National Institute of Allergy and Infectious Diseases, National Institutes of Health, Rockville, MD
Laura McNay, M.S., Office of Planning and Operations Support -- Division of Clinical Research, National Institute of Allergy and Infectious Diseases -- National Institutes of Health, Bethesda, MD
The Office of Planning and Operations Support (OPOS) in NIAID's Division of Clinical Research (DCR) is in the process of developing performance measures to provide data-driven progress reports for projects and employees engaged in the facilitation and administration of clinical research within the Division. The search for appropriate performance measures is no simple task given the eclectic role that DCR plays in facilitating clinical research both domestically and globally. This presentation will describe a multi-factorial picture of incremental research forming the foundation, development, validation and implementation of Key Performance Indicators (KPIs) that reflect the work-product of DCR's seven branches. Fundamental to the research are identification of: (1) comparison measures, e.g., baseline performance data and leading industry/government benchmarks against which DCR's performance can be compared, (2) proposed data collection sources, methods, instruments and analysis, and (3) computer tools for assembling, tracking and sorting performance data. Objectives include refinement and pilot testing of the proposed performance metrics and goal-setting benchmarks to determine if they adequately reflect each branch's productivity, and the gleaning of information about contextual factors and intervening variables that may impact performance. Deployment of appropriate performance measures will enable quantifiable, objective assessment of DCR's clinical research enterprise. Data derived from the metrics will help the Division achieve higher productivity, efficient outcomes, and optimal use of resources.

Learning Areas:

Conduct evaluation related to programs, research, and other areas of practice
Program planning
Public health administration or related administration
Public health or related organizational policy, standards, or other guidelines

Learning Objectives:
Explain performance measurement, including conceptual and methodological issues surrounding performance measurement and its application in strategic planning, quality improvement, performance management, and program evaluation. Discuss efforts to identify Key Performance Indicators (KPIs) that are relevant to, and reflective of, the work of each individual branch, and explain how the performance measures should be fully objective and provide quantifiable data that will enable leadership to prioritize, optimally align, and utilize resources. Identify examples of performance metrics, not limited to customer focused assessments, which include process effectiveness outcomes, leadership outcomes, and workforce-focused outcomes. Identify comparison measures against which the performance of the program can be compared. Comparison reference points may include prior measures of the DCR’s performance (baseline) and recognized standards of industry performance. Benchmarking can help promote continuous improvement by providing targets, and by encouraging work groups to meet or exceed leading industry standards. Describe the components of scientific research, including quantitative and qualitative data collection and analysis, focus group and survey methodologies, and multifactorial analysis that will be used to identify appropriate performance indicators. Describe the methods and instruments that will be developed to discern performance and any contextual information that may impact performance. Explain how the proposed study will be enriched with data gathered from several sources, using a variety of appropriate collection methods and instruments. The goal will be to produce metrics that provide data-driven assessments of performance and productivity and information about contextual factors to answer the “Why” and “How” questions that should accompany routine performance measurement. Explain the importance of buy-in from the stakeholders, and the need to explain to staff what kinds of data and contextual information will be gathered in order to develop appropriate performance measures for each branch. It is crucial for all staff to feel comfortable with the concepts, rationales and methodologies used in the development of the KPIs and metrics, and how the information will be used in the “scorecards” or “dashboards” and final report. Discuss computer software that can automate and expedite the gathering, tracking, updating and analysis of performance measures, and the need for expertise in adapting existing software to an organization’s specific needs.

Keyword(s): Performance Measurement, Evaluation

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: Among my scientific interests and areas of scholarship and practice are program development, performance measurement, and evaluation practice. I am the principle author of a "set-aside funding" evaluation protocol for the development of performance measures, and I hold a Certificate in Evaluation Practice from The Evaluators' Institute, George Washington University.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.

Back to: 3141.0: Health Administration Roundtable I- Workforce Issues