287516
Evaluation and redesign of public health information technology: A cost-effective, reproducible, and user-informed approach
Nikie Sarris, MPH,
Community Health Promotion Research, RTI International, Research Triangle Park, NC
Kim Hayes, MPH,
Center for the Health of At-Risk Populations, RTI International, Research Triangle Park, NC
Background: Ensuring key audiences' access to high-quality information is a primary goal of health informatics. Public health practitioners often face resource constraints that limit the extent to which information technology may be evaluated. This study outlines the process of developing and implementing a cost-effective and rapidly deployed usability protocol that yields actionable results for application redesign in a short time-frame. Objective: To develop and implement a reproducible remote testing protocol for public health practitioners with limited resources to evaluate and redesign health IT tools. Methods: We developed a remote usability testing protocol and applied it to evaluate the Agency for Healthcare Research and Quality's National Resource Center website (a central repository for health IT information). Protocol development, survey implementation, analyses, and summary of results occurred within a 2 month span. First, we identified and recruited a small group of representative users. Second, to reduce logistical burdens, we developed a remote testing protocol where users could work in their natural environment and be observed by evaluators through a web-conferencing tool. Our protocol consisted of 2 main sections: (1) cognitive walk-through where users received minimal guidance as they conducted tasks using the application (while verbalizing their thoughts), and (2) system usability scale (SUS) where users provided quantifiable responses to specific questions related to system usability. Results: The cognitive walk-through yielded a series of themes related to usability issues. The SUS yielded a numeric assessment of usability, which showed whether the tool's usability was above or below average. Together, the results provided a set of actionable recommendations for redesign. Conclusion: By identifying a limited number of representative users, and conducting remote testing that utilized complimentary and established techniques, we developed actionable recommendations in a short time-frame. This protocol will be of value to practitioners faced with application redesign under significant logistical limitations.
Learning Areas:
Other professions or practice related to public health
Public health or related research
Learning Objectives:
Explain the advantages of a remote testing usability protocol for evaluating the usability of public health information technology.
List the key steps in developing a remote testing protocol that is cost-effective, rapidly deployed and provides actionable recommendations.
Describe how to engage key stakeholders through a protocol that is efficient and yet tailored to various user needs.
Keyword(s): Health Information, Information Technology
Presenting author's disclosure statement:Qualified on the content I am responsible for because: I was the Task Lead of the usability testing protocol described in the abstract; I also have graduate-level training in human-computer interaction and health systems.
Any relevant financial relationships? No
I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines,
and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed
in my presentation.
