142nd APHA Annual Meeting and Exposition

Home Events & Meetings Annual Meeting Schedule & Program

Online Program

Annual Meeting Recordings are now available for purchase

299018
Is Face-to-Face Meeting Superior to Virtual Review Meeting?

142nd APHA Annual Meeting and Exposition (November 15 - November 19, 2014): http://www.apha.org/events-and-meetings/annual
Tuesday, November 18, 2014 : 4:50 PM - 5:10 PM

Nghia Vo, MD , Division of Scientific Review, AHRQ, Rockville, MD
Rebecca Trocki, MSHAI , Division of Scientific Review, AHRQ, Rockville, MD
Kishena Wadhwani, PhD, MPH , Department of Health and Human Services, Agency for Healthcare Research and Quality, AHRQ, Rockville, MD
Francis Chesley, MD , OEREP, AHRQ, Rockville, MD
Audio (mp3) recording Recorded Presentation (mp4)
PURPOSE: Virtual review (VR) has been thought not to be equal to face-to-face (FF) meeting in terms of thoroughness of the review process. The purpose of this report is to evaluate the thoroughness of nine consecutive review sessions conducted in 2012 and 2013.  

DESIGNS: The nine SEP sessions reviewed include five were face-to-face (FF) (GI) and four WebEx virtual reviews (VR (GII). Pre- and post-discussion scoring was recorded as usual. Thoroughness of the review is measured in terms of percentage and increments of score changes as well as length of discussion per application. Cost of each meeting has also been recorded.   

RESULTS: There were 88 discussed applications in five FF sessions (GI: 17/session) compared to 52 applications in four VR sessions (GII: 13/session). FF sessions resulted in 73% (64/88) score changes (increase and decrease) compared to 60% (30/51) for virtual reviews (NS). Pre- to post-increases in scoring per application (group of three reviewers) are 1.85+/-1.00 and 2.00+/-1.34 for GI and II respectively (NS). Pre- to post-decreases in scoring per application are 1.58+/-0.79 and 1.12+/-0.35 for GI and GII respectively (NS). The incremental changes in scores are minimal and non-significant between the two groups. The time spent to review each application is 26 vs. 20 minutes for FF and VR, respectively. And the overall cost per reviewer is $1,000 higher for a FF than VR meeting.

CONCLUSIONS:   FF and VR sessions resulted in:

  1. Non-significant percentage changes in scoring,
  2. Non-significant incremental changes in the degree of scoring,
  3. The cost of FF meeting is higher than VR, although time spent to review each application is not significantly different.

We conclude that the two methods are equally effective in evaluating grant applications. 

Learning Areas:

Conduct evaluation related to programs, research, and other areas of practice
Public health administration or related administration
Public health or related education
Public health or related research

Learning Objectives:
Discuss face-to-face and Virtual review meetings. Compare and evaluate the two types of meetings.

Keyword(s): Public Health Research, Outcomes Research

Presenting author's disclosure statement:

Qualified on the content I am responsible for because: I have been a Scientific Review Officer at AHRQ for many years and written many abstracts related to the scientific review process.
Any relevant financial relationships? No

I agree to comply with the American Public Health Association Conflict of Interest and Commercial Support Guidelines, and to disclose to the participants any off-label or experimental uses of a commercial product or service discussed in my presentation.

Back to: 4425.0: Public health systems and services research (PHSSR) - II