FDA AERS Data & Quality Control

FDA just released its Q4 2008 AERS data (yes, it’s May 13, but that’s another story altogether).

This post is about Quality Control of AERS data. It seems that every quarter, there’s some type of SNAFU with the data release (last year they released AERS data partially contaminated with a previous quarter's data).

This quarter, we have the classic newline-characters-where-they-don’t-belong error that's screwing up my AERS parser.

A little Background:
FDA releases its data in 2 forms (ASCII and SGML). ASCII is the one that I use and each ASCII file consists of row after row after row of $-delimited Adverse Event Records.

2 sample rows might look something like this:

$12345$abcdef$somestuff here$blah$more blah$blah $34321$blahblah$doscum$etcetc$vixerunt$gaius$cicero

Each row should represent one particular database record and my parser dutifully goes through each row extracting all the little bits of information between the dollar $ign$.

But with this latest quarterly release FDA released its Drug data file (aka DRUG08Q4.txt) with 4 significant quality control errors (see sample screenshot below).

[For those who want gruesome details, the following lines in the DRUG.txt file contain errors: 537-538, 258909-258910, 281285-281286, 408948]

The gist of the issue is that whoever entered the data for these 4 drug-records forgot to remove the newline characters (“carriage returns”) and so the record is actually split across 2 or more lines.

While this doesn’t seem like a big deal, if your parser isn’t “smart” it could inadvertently stuff the wrong data into the wrong slots in your database.

And so, you have to design your parser to look for these types of errors--and then you have to have a human look at the problem just to assure yourself that there wasn’t a bigger error. This wastes time...especially when the file you’re looking at has 416,000 records.

‘t would be nicer if FDA did more quality control on their data releases.

FAERS - AERS, but Fancier

Someone at FDA sent this to me recently (see below). It's an email sent around from within the FDA announcing a new Adverse Event Reporting system (sort of).

Emphases are mine. Editorial comments are mine, too.

Spoiler alert: They forked over $$$$$ and went with Oracle.

Bioinformatics Board (BiB) News

Post Market Safety BRB Announces Selection of Vendor[We've decided to shop at Barney's. Can't tell you what we're buying just yet, but it's gonna be FAB!] for New Enterprise Adverse Event System

The Bioinformatics Board (BIB) News provides FDA employees with an update on the activities and progress of the BiB. This month’s news focuses on the Post Market Safety BRB and the selection of the Adverse Event System Vendor. For more information about the BiB or the contents of this message, please contact your BiB representative or email the Bioinformatics Board, bioinformatics@fda.hhs.gov.

Post Market Safety Business Review Board Announces Selection of Vendor for New Enterprise Adverse Event System FDA’s Bioinformatics Board and the Post Market Safety Business Review Board are pleased to announce a significant milestone for the Agency. After an extensive analysis of FDA scientific needs for the management and analysis of post market product safety reports, and an evaluation of leading industry tools, the FDA Adverse Event Reporting System (FAERS) Program has selected Oracle AERS[$$$] (Adverse Event Reporting System) as the new tool for FDA staff[guess who's not getting access to the data juuuust yet...?] to efficiently[you wouldn't believe what we were doing before] analyze post market safety reports in order to identify potential product safety problems.

Selection of the new FAERS tool represents a significant landmark for the Agency. With the selection of an advanced tool that all centers and offices can leverage, the Agency can now focus on implementing processes for sharing post market safety data across our product centers as well as advancing the science of post market report analysis. Achievement of this milestone represents the efforts of numerous individuals from across the Agency over several years[wow. just...wow]. While in many ways this is just a beginning, it is important to recognize our progress towards meeting the goal of providing modern tools necessary to address many post-market surveillance needs. Selection of Oracle AERS[What? no love for MySQL? They're about to be subsumed by Oracle, too, ya know.] positions the Agency for success in this complex and dynamic arena.

We expect the initial users in CDER and CBER to receive training and begin using the new system as early as fall 2009[but don't quote us on that] with additional users being trained and brought on over time. CDRH users are expected to begin using the tool in 2010, and additional centers and offices expected shortly thereafter.

Tuesdays are a busy day @ FDA

3 non-colinear points define a plane...and 3 Tuesdays in a row is enough for me to jump to 2 conclusions, but I think the first conclusion is correct and the 2nd conclusion has a good chance of being right, too.

Conclusion 1. The FDA updates its warning letters database by hand every Tuesday morning.

Why do I think so?
I have an automated ‘bot’ that fetches new FDA warning letters from FDA.gov daily, but recently (when I bothered paying attention to it) I noticed that most days it doesn’t retrieve any new warning letters.

Then I started paying attention, and for the past 3 weeks, my bot only fetches new letters on Tuesday mornings (EST).

Not Monday night (I checked).

Not Tuesday at 6 AM (I checked).

Only Tuesday mornings between ~9 and ~11AM.

So, this smells like a human who has a Tuesday morning to-do list. Task # 1? Push out last week’s warning letters.

Why this can’t be automated on a daily basis? Beats me.


Conclusion 2. Updating of clinical trial data @ clinicaltrials.gov is done by the same person/entity/thing at FDA. This conclusion is far more tenuous, but hear me out.

(admittedly weak) Reasoning?
Every day, I have another bot that fetches new clinical trials data.

Every day, the bot finds updated data around 10AM--Except on Tuesdays when the new data show up around 11:30AM.

Tuesday’s task # 2? Push out new clinical trial data.

Updated FDA Search add-on for Firefox (WOHOOO!)

Just updated the FDA Search add-on from version 0.3.0 to 0.6.0 (I just make these numbers up as I have no idea what they're supposed to mean).

It's a pretty significant upgrade, so I doubled the version #.

What can it do now?

you can highlight a word or phrase and search

• the FDA's website
• the CDC's website
• drug adverse events
• vaccine adverse events
• drug safety analysis
• clinical trials
• FDA warning letters

Also has some nifty icons added, which took me 4ever to figure out how to add. Mozilla's xul documentation is horrendous.

The FDA has an Area 51 on its web-site??

Here's something you won't see every day...it's the FDA's robots.txt file.

For the uninitiated, robots.txt is a small file placed on a web-site to indicate which pages on your website can be crawled by indexing robots (Googlebot, Yahoo! Slurp, etc.). It basically says "Hey Googlebot, you can index these pages, but stay away from those over there."

Here's the FDA's robots.txt file it is in its entirety--I've colored the parts that intrigue me.

#robots.txt file for http://www.fda.gov

#Added for Bristol-Myers on Sept 2005
User-agent: vspider  Disallow: / 

#For all other crawlers
User-agent: *
Disallow: /scripts/
Disallow: /data/
Disallow: /binn/
Disallow: /cder/test/
Disallow: /opacom/area51/ 
Disallow: /oashi/aids/listserv/
Disallow: /cdrh/ftparea/cdrh/MDR/coll/mdr/mdrcoll/
Disallow: /foi/warning_letters/d1371b.pdf
Disallow: /foi/warning_letters/archive/
Hit-rate: 30 # wait 30 seconds before starting a new URL request default=30
Visiting-hours: 23:00EDT-05:00EDT #index this site between 11PM - 5AM EDT
Concurrent-hits: 2 # limit concurrent active URLS to 2 for each index server

1. What's the deal with Bristol Myers' request to ban vspider? And why did the FDA comply with the request? From what I can tell, vspider is a personal indexing robot that can be used by anyone to index a site. Curious in CT.

2. What's going on in area51 and why can't it be indexed? I tried to look at the contents and got a "denied" error...so perhaps it holds the medical records for the little green men in Nevada.

3. Why block indexing of one specific warning letter (d1371b.pdf)? If you try to go to fda.gov/foi/warning_letters/d1371b.pdf you get a 404 (not found) error, but I have a copy from my own search engine. It's a pretty vanilla warning letter from 1998 sent to Trinity Chemical Corporation. Again, I'd love to hear the rationale behind this decision.

4. Why block indexing of the archived warning letters?

Thanks to Pharmamkting blog

Tip of the hat to John Mack @ pharmamkting blog for mentioning my site!

FDA Search Firefox Add-on

Just tried to publish my Firefox add-on, titled FDA Search 0.1.0. What I didn't realize is that Firefox keeps your add-on in purgatory until some FF regulars actually review it, and only then can you nominate your add-on for inclusion in the public repository. Thanks to James Cook and his add-on, which I used as a template for mine.