Adverse Event Reporting

White Paper Background

Social media has created a communications challenge for the Pharmaceutical Industry. Many would like to use social media data to better engage with their customers, but concern over potential exposure to liability for monitoring Adverse Events and Off Label conversations, and the internal resources necessary to manage this new workflow gives the industry pause from participating in a listening program.. Meanwhile, companies in other industries are benefiting from access to genuine customer feedback and consumer sentiment.

The purpose of this white paper is to shed light on the realities of Adverse Event Reporting in social media. It includes hard numbers and meaningful research intended to assist pharmaceutical companies in determining the scope and potential impact of creating a social listening program. We hope the findings will make it easier for these companies to eliminate barriers that are currently preventing them from listening to their consumers via social channels.

Executive Summary

In this study, which was conducted over a 30-day period, social media data was tracked, collected and analyzed containing brand mentions for 224 major pharmaceutical brands. Brands studied included 71 over-the-counter (OTC) drugs and 153 prescription-only (Rx) medications. In order to identify and evaluate any significant differences between OTC and Rx online discussions, we focused on two product types for each category:

OTC Medications:                 antacids, decongestants

  Rx Medications:                    statins, antihypertensives

The Visible Intelligence® social media monitoring and analytics platform was used to perform the data collection and analysis. We analyzed 257,177 posts mentioning one or more of the 224 brand names, and then analyzed a subset of those posts that also contained a comprehensive list of terms and phrases associated with Adverse Events (AEs) along with first-person narrative terms for increased contextual relevance. The purpose of this segmentation was to isolate and target those posts that had the greatest likelihood of containing an AE for further analysis.

An adverse event (AE) is defined as any unfavorable and unintended sign including an abnormal laboratory finding, symptom or disease associated with the use of a medical treatment or procedure, regardless of whether it is considered related to the medical treatment or procedure, that occurs during the course of the study.

A team of Visible® social scientists performed manual analysis, carefully reading each post in a sample set of 825 posts. Additional websites were researched and social mining was performed in applicable cases to determine for all posts where an AE was reported:

• If there was an identifiable first and last name associated with the AE reporter
• If the AE issue was self-reported or was reported about someone else
• If there was a contact method (e.g., phone, physical address, email address, or site-provided means of direct or private communication)
• If other self-identified information was provided (e.g., age, gender or location)

Key Findings

The research revealed that for all posts containing mentions of the 224 brands studied:

• Only 0.3 percent actually contain a report of an AE experience
• 14 percent of those posts have an identifiable full name and contact method (through site* or email)

We analyzed the data further, scrutinizing a more relevant subset of 12,530 posts that had the highest likelihood of containing AE experiences—the results of proactively looking for AE experiences.  This subset of posts was five percent of the total population and contained one or more term or phrase from a comprehensive list of terms and phrases associated with Adverse Events (AEs) along with first-person narrative terms for increased contextual relevance. Our analysis revealed that:

• Only 3.3 percent of those posts actually contain a report of an AE experience.
• Only 1 in 7 of posts with an AER contained enough information to meet the four Adverse Event Reporting criteria established by the FDA to trigger filing a report. 
• Reporting rates for OTC versus prescription drugs were nearly identical
  • 3.33 percent  AE rate for OTC drugs
  • 3.27 percent AE rate for prescription drugs
• 56 percent of AE posts contained a method of contact that could be pursued, such as private messaging through a forum after becoming a member or messaging through site-specific channels (beyond replying to thread) to obtain full name information*

*The FDA has not formalized requirements for companies to proactively reach out to authors in order to obtain this information to be able to fill out an Adverse Event Report.

To put some perspective on these figures, during the 30-day research period, the most talked about antacid brand online had just over 11,000 posts—nearly six times the volume of any other antacid brand studied.  Of those posts, only five met the criteria required to trigger filling out an Adverse Event Report.  This small number of AERs is also significant because this brand had nearly twice the AE reporting rate of the other antacid brands, yet still yielded just five posts meeting the criteria.

If the FDA decides drug companies need to submit website links to communities, forums and sites (e.g., Twitter) that offer some means of private communication or messaging that would allow for follow up by an FDA representative when no email or phone is available, then half, rather than 1 out of 7, of the AERs would trigger paperwork to be filed.  In the case of the antacid brand example that would mean 18 reports in an average month of more than 11,000 brand mentions.

 

The diagram above illustrates the overall volume of brand mentions and the segmented data. The dark blue area on the left represents all posts with mentions of the 224 brands. The purple Filtered for Relevance area within that represents posts filtered to focus on first-person narratives (with unrelated news and general content removed).  In our experience, this is the typical view most brands with higher volumes of online mentions tend to focus their time reading and analyzing.

The turquoise area contains posts that also mention AE terms along with the brand. The actual rate of AEs in the overall data set is 0.3 percent (less than one-quarter of a percent) as seen in the green area.  This is an extremely low figure that equates to 3 out of every 1,000 branded posts having an AE in them.

In order to give greater perspective to pharma brands, we took a worst case scenario view from a reporting workload standpoint and proactively searched out branded AE content.  These are the posts where we expect to find discussions of AE (the turquoise area).  Our research showed that the rate of AEs is 3.3 percent in this targeted data set.  Furthermore, when that 3.3% of posts that are discussing valid AEs were investigated, only 1 out of every 7 of those posts (the red area) contained full name and contact information to trigger filling out an AER form.  This makes the real reporting rate 0.5 percent (half a percent).

What does that mean in real numbers for our total data set?  For 30 days worth of data collected on 224 brands totaling over 257,000 posts, the total number of reportable AEs averaged 3 per brand.

Research Goal

The goal of this project was to quantify the number of times AEs appear in social media (consumer-generated content) for both OTC and Rx products, and assess the potential impact to social teams using social media listening platforms in the pharmaceutical industry as it relates to mandatory FDA Adverse Event Reporting.

Methodology

Brand Focus

We focused our research on four categories of pharmaceutical product types to collect and analyze 30 days of brand-specific data from September 12 – October 12, 2011. To determine if there were any noticeable differences in the reporting rates for OTC versus Rx drugs, we chose two highly discussed product types for each category. Data collection focused on the brand name only as the identifier and did not include posts where the chemical name of a medication made it unique to one specific brand.

• OTC products:
  • 33 antacid brands
  • 38 decongestant brands
• Rx products:
• 10 statins
• 143 antihypertensives

Data Set and Analytics Platform

Social data was collected from millions of sources, such as forums, Twitter, and Facebook, and then cleaned, enriched and analyzed using Visible Intelligence social media monitoring and analytics platform. Visible Intelligence’s proprietary system collects consumer-generated content from all forms of social media including:

• More than 250 million blogs (e.g., WordPress)
• More than 6 million forums, message boards and groups (e.g., BabyCenter, DailyStrength)
• Social networks (e.g. Facebook, LinkedIn)
• Microblogs (e.g., Twitter)
• News sites (e.g., WSJ.com)
• Health sites (e.g., WebMD, HealthBoards)
• Video and photo sites, and social bookmarking (e.g., YouTube, Flickr, Reddit)

Brand mentions were identified using actual brand names and variations such as “Gas X” and “Gas-X”.  Additional terms were included for subsets of data, such as first-person narrative words; “we,” “me,” “my” and “our” and AE terms, such as “headache,” “side effect,” “death,” “pain,” “weakness” and “vomiting.” This approach focused our research specifically on the posts with the highest likelihood of containing an AE and provides key insights and opportunities for companies that want to proactively track AE mentions.

The sample was determined by calculating the given population needed to provide a 95 percent level of confidence with a margin of error of +/- 3.3 percent.

FDA’s Four Adverse Event Reporting Criteria

Our team of social scientists analyzed each post in the sample set to ultimately determine how many met the FDA’s criteria for Adverse Event Reporting:

1. Identifiable Patient: The post contains information sufficient to believe a specific patient was involved, such as “I experienced” or “my brother experienced,” but did not contain general statements such as “many people.”
2. Identifiable Reporter: The post contains sufficient information to follow up with the person reporting, such as an email address, telephone number or physical address.
3. Specific Medication: The post mentions a specific medication by brand or the chemical name of a medication where that compound is unique to one specific brand. (Note:  For this project, only brand names were used.)
4. Adverse Event: The post describes a reaction that a “reasonable person” would consider an adverse experience such as death, hospitalization, vomiting, swelling or a side effect that is not known or expected with the medication.

When manually evaluating each post to determine if it contained an AE, analysts erred on the conservative side, including mentions of experiencing “side effects” when no specific ailments were mentioned. Here are some examples of what was considered an AE:

“I take ___  and ___, both say they will make you dizzy, and they do at times. Most notably if you get up too fast. Lots of my friends are on BP meds and we all have that “get up too fast” problem.”

“I don’t take ___ because of the side effects I suffered with it.”

“___’s drive me completely nuts and I’ll feel like my scalp is itching and get that ‘bees in the bonnet’ thing again.”

Post Evaluation

Each post reviewed looked for an AE being reported by the author, that either they experienced or someone they knew experienced. Posts identified as containing an AE were further evaluated to determine if they also contained or with research could easily identify:

• Full Name of the Author/Reporter – Either on the site where the post was written or by clicking links to the author’s/reporter’s profile or links provided in their profile, such as to their personal blog or Twitter profile.
• Contact Information – The research considered a broader view of contact information, beyond phone, address and email, that included some site-provided methods like Direct Messaging (DM) through Twitter or an internal messaging system within a forum community. Simply replying in a public thread with the hopes the intended recipient would read it was not considered a valid form of contact unless it was on the blogger’s personal blog.
• Reported for Someone Else – The AE mentioned was experienced by another person the author/reporter knows and was identified by relationship.
• Additional Information – To ascertain what percentage of AE reporting also contained other available and publically shared information, such as gender, age, location, medical conditions and medications.

Data Volume Perspective

Each of the four product types of brands studied had significant volumes of data during the 30-day research period. The red areas in the following charts show the highly relevant, filtered portion of each brand data set focused on AE mentions—meaning the posts that are most likely to contain AERs. The 3.3 percent rate of AEs is derived from this red area of targeted posts.

Research Findings

This chart contains the data volumes by product type and category for the 30-day period of September 12 – October 12, 2011.

Product Type	All Brand Mentions	Filtered for Relevance	With AE Terms
OTC Antacids	58,856	29,542	3,108
OTC Decongestants	55,467	21,112	4,835
Rx Statins	116,949	6,268	3,069
Rx Antihypertensives	25,905	2,839	1,518
	257,177 posts	59,671 posts	12,530 posts

Only 3.3 percent of all branded social media posts using terms and phrases associated with an AE (far right column on chart above) are actually discussing an AE. 

There was no significant difference in the rates of AEs when talking about OTC versus Rx-only medications:

•  3.33 percent  of OTC branded posts
•  3.27 percent of Rx branded posts

 

 

 

 

Of those posts containing an AE:

• 26 percent have an identifiable name
• 56 percent have ability to contact via email or site-provided method (beyond replying to thread)
• 14 percent of have an identifiable full name and contact method (through site or email)
• 83 percent of AE mentions for someone else were mothers talking about  problems their children were experiencing
• · 78 percent provide additional self-identified information such as, age, gender and location

 

Conclusion

The results of our research indicates that pharmaceutical marketers that engage in diligent monitoring of social media for mentions of their brands should expect to see some mention of AEs within this discussion. However, the volume of AEs is not likely to exceed what can be handled through existing Adverse Event Reporting channels established for traditional/offline reporting methods.

Pharmaceutical marketers still need clarity in several areas, though, including:

• What is a pharmaceutical company’s responsibility for monitoring online discussion for AEs? (e.g., frequency of monitoring and sites monitored)
• Does a company’s presence online or in social media (e.g., online advertising, posting messages in a forum, sponsoring other bloggers’ posts) change that responsibility?
• In the case of a broader safety incident, should the company reach out to monitor AEs reported online and/or turn to CGM sources to post information for consumers to report AEs directly to the FDA?

Additionally, if a pharmaceutical company observes a message containing AE information, but there is no private communication channel for contacting the message poster, what follow-up approach is appropriate? For instance, should the company post a public message within the forum asking the message poster for more information and/or to contact the company through private channels such as email or an 800-number.

And lastly, if a message is discovered several weeks or months after it was originally posted, how does delay of discovery affect the pharmaceutical company’s responsibility for follow-up?

Because of difficult questions like these, pharmaceutical companies are unsure how to structure a social listening program that will fulfill the FDA’s Adverse Event Reporting requirement. Additionally, they are understandably concerned about the potential impact of not responding appropriately to consumers’ discussions of AEs that they encounter in social channels.

What is certain is that the potential for hearing about AEs in social media discussions exists. To better understand how to structure a social listening and engagement program that will work well for their industry, pharmaceutical firms need better intelligence about these discussions.

One important step for pharmaceutical companies is to partner with vendors who are familiar and sensitive to issues around Adverse Event Reporting. They can help them find the data and insights they need to build an effective social listening program and manage the small number of AEs that may arise. By being proactive rather than reactive, pharmaceutical companies can learn more about their customers and shape their brand messages and campaigns accordingly.

About Visible Technologies

Currently, Visible’s social media monitoring and analytics platform, Visible Intelligence^®, is used by large pharmaceutical companies to support their social media programs. Our consulting services team provides guidance and best practice leadership to assist pharma companies in managing social media programs for maximum results.

To request a demo of Visible Intelligence^® click here.

Jackie Kmetz
Director Community Educational Outreach

425.957.6056

Download the PDF