FDA AERS Data & Quality Control

FDA just released its Q4 2008 AERS data (yes, it’s May 13, but that’s another story altogether).

This post is about Quality Control of AERS data. It seems that every quarter, there’s some type of SNAFU with the data release (last year they released AERS data partially contaminated with a previous quarter's data).

This quarter, we have the classic newline-characters-where-they-don’t-belong error that's screwing up my AERS parser.

A little Background:
FDA releases its data in 2 forms (ASCII and SGML). ASCII is the one that I use and each ASCII file consists of row after row after row of $-delimited Adverse Event Records.

2 sample rows might look something like this:

$12345$abcdef$somestuff here$blah$more blah$blah $34321$blahblah$doscum$etcetc$vixerunt$gaius$cicero

Each row should represent one particular database record and my parser dutifully goes through each row extracting all the little bits of information between the dollar $ign$.

But with this latest quarterly release FDA released its Drug data file (aka DRUG08Q4.txt) with 4 significant quality control errors (see sample screenshot below).

[For those who want gruesome details, the following lines in the DRUG.txt file contain errors: 537-538, 258909-258910, 281285-281286, 408948]

The gist of the issue is that whoever entered the data for these 4 drug-records forgot to remove the newline characters (“carriage returns”) and so the record is actually split across 2 or more lines.

While this doesn’t seem like a big deal, if your parser isn’t “smart” it could inadvertently stuff the wrong data into the wrong slots in your database.

And so, you have to design your parser to look for these types of errors--and then you have to have a human look at the problem just to assure yourself that there wasn’t a bigger error. This wastes time...especially when the file you’re looking at has 416,000 records.

‘t would be nicer if FDA did more quality control on their data releases.

Tuesdays are a busy day @ FDA

3 non-colinear points define a plane...and 3 Tuesdays in a row is enough for me to jump to 2 conclusions, but I think the first conclusion is correct and the 2nd conclusion has a good chance of being right, too.

Conclusion 1. The FDA updates its warning letters database by hand every Tuesday morning.

Why do I think so?
I have an automated ‘bot’ that fetches new FDA warning letters from FDA.gov daily, but recently (when I bothered paying attention to it) I noticed that most days it doesn’t retrieve any new warning letters.

Then I started paying attention, and for the past 3 weeks, my bot only fetches new letters on Tuesday mornings (EST).

Not Monday night (I checked).

Not Tuesday at 6 AM (I checked).

Only Tuesday mornings between ~9 and ~11AM.

So, this smells like a human who has a Tuesday morning to-do list. Task # 1? Push out last week’s warning letters.

Why this can’t be automated on a daily basis? Beats me.


Conclusion 2. Updating of clinical trial data @ clinicaltrials.gov is done by the same person/entity/thing at FDA. This conclusion is far more tenuous, but hear me out.

(admittedly weak) Reasoning?
Every day, I have another bot that fetches new clinical trials data.

Every day, the bot finds updated data around 10AM--Except on Tuesdays when the new data show up around 11:30AM.

Tuesday’s task # 2? Push out new clinical trial data.

The FDA has an Area 51 on its web-site??

Here's something you won't see every day...it's the FDA's robots.txt file.

For the uninitiated, robots.txt is a small file placed on a web-site to indicate which pages on your website can be crawled by indexing robots (Googlebot, Yahoo! Slurp, etc.). It basically says "Hey Googlebot, you can index these pages, but stay away from those over there."

Here's the FDA's robots.txt file it is in its entirety--I've colored the parts that intrigue me.

#robots.txt file for http://www.fda.gov

#Added for Bristol-Myers on Sept 2005
User-agent: vspider  Disallow: / 

#For all other crawlers
User-agent: *
Disallow: /scripts/
Disallow: /data/
Disallow: /binn/
Disallow: /cder/test/
Disallow: /opacom/area51/ 
Disallow: /oashi/aids/listserv/
Disallow: /cdrh/ftparea/cdrh/MDR/coll/mdr/mdrcoll/
Disallow: /foi/warning_letters/d1371b.pdf
Disallow: /foi/warning_letters/archive/
Hit-rate: 30 # wait 30 seconds before starting a new URL request default=30
Visiting-hours: 23:00EDT-05:00EDT #index this site between 11PM - 5AM EDT
Concurrent-hits: 2 # limit concurrent active URLS to 2 for each index server

1. What's the deal with Bristol Myers' request to ban vspider? And why did the FDA comply with the request? From what I can tell, vspider is a personal indexing robot that can be used by anyone to index a site. Curious in CT.

2. What's going on in area51 and why can't it be indexed? I tried to look at the contents and got a "denied" error...so perhaps it holds the medical records for the little green men in Nevada.

3. Why block indexing of one specific warning letter (d1371b.pdf)? If you try to go to fda.gov/foi/warning_letters/d1371b.pdf you get a 404 (not found) error, but I have a copy from my own search engine. It's a pretty vanilla warning letter from 1998 sent to Trinity Chemical Corporation. Again, I'd love to hear the rationale behind this decision.

4. Why block indexing of the archived warning letters?