Social media as qualitative research: Reports of a data mining experiment

As people broadcast information and solicit advice about health through social media, the medium offers volumes of rapid, real-time information.

We wanted to explore if the freely available information on social media would offer similar insights to and possible replace costly and time-consuming research using focus groups. We also sought to determine if such channels offered actionable information and entry points for health outreach or promotion opportunities.

We used two social media data mining services, Radian6 and Lithium, and a combination of free trials and a paid subscription. Both services search billions of Twitter feeds, blogs, online discussion boards, forums and news articles for key terms and use proprietary algorithms to evaluate and categorize posts. Using these services, we were able to analyze and rank words included along with our key terms for positive and negative sentiment. Sentiment analysis allowed for both big-picture insights and detailed observation of specific posts, shedding light on conversations and influencers.

Eschewing neutral posts, those that skewed positive or negative mirrored focus group insights with surprising accuracy. A key, and highly actionable, negative sentiment surfaced in social media posts as it had in focus groups. The critical difference was that unlike with focus groups, in the social media space we were immediately able to join and help steer the conversation.

Social media offers significant opportunities for understanding consumer health behavior that are more cost-effective than traditional research methods. Observing health “crowdsourcing” and unfiltered experience also affords opportunities for direct engagement.