The most recent release of Adverse Event Data from FDA's AERS / MedWatch database[1][2] shows the biggest jump in total number of reports submitted ever (if you count by absolute number of reports submitted). It's also one of the biggest percent increases (44.8%) in Adverse Event submissions since the AERS program began. I wish I knew what caused the jump. It's safe to assume that it was unrelated to the Supreme Court's recent ruling [pdf] about Adverse Event reporting related to Zicam as the jump we see in AERS reports took place from July to September of 2010, even before the case was argued before SCOTUS.
Initial upload into FDAble looks like everything was okay.
Interestingly, there are ~111,000 reports in this release compared to ~121,000 for the previous quarter. Not sure how significant this decrease is (is it seasonal? is it just noise? is FDA weeding out duplicates?). Will take closer look later....
Our correspondence to Nature Biotech regarding AERS data came out yesterday. I can't post the article due to copyright restrictions, but I'm sure you can pick up a copy at your local newsstand.
I’ve written a couple of posts on FDA Warning Letters (here and here), but today’s post seems particularly important.
The FDA’s Warning Letter Search Engine is Seriously Flawed. There are at least 2 things that are wrong (in addition to the flaws I outlined earlier).
Certain warning letters that were in the old database have vanished.
The warning letters that are returned when searching by date are often inaccurate.
Allow me to elaborate.
Certain warning letters that were in the old database have vanished If you use the FDA’s Warning Letter Search engine to search for “beef northwest”, you get 0 (zero) results [update: fda has fixed this error--see here]. But there is a warning letter issued to Beef Northwest Feeders LLC issued on August 21, 2007 (see here for the letter). You can also search the FDA’s Warning Letters by Company Name and the record does not show up.[update: fda has fixed this error--see here]
By my estimate there are almost 2,000 missing warning letters (I wrote a small bot that systematically went through the current FDA search engine and recorded warning letter results issued every day from January 1, 1996 to the present day and it returned ~7,700 warning letters, whereas the FDAble Warning Letter Database, made from the FDA’s old search engine, contains ~ 9500 letters).
Warning letters returned when searching by date are often inaccurate If you use the FDA warning letter search engine to search for all warning letters issued from 11/1/1996 to 10/31/1997 you get 152 results.
If you expand your search by 1 day (11/1/96 - 11/1/97) you get 876 results. Here’s a hint: The FDA did not issue 724 warning letters on 11/1/97- it was a Saturday.
I don’t know why the first search yields only 152 results, but it’s clearly wrong, and to be honest, given the errors throughout I’m not confident in the 876 results returned in the 2nd search-The FDAble database says there were 1,008 warning letters issued from 11/1/96 and 11/1/97.
To summarize all 3 posts we have the following problems with FDA Warning Letters Search Engine.
Certain warning letters that were in the old database have vanished.
The # of warning letters returned when searching by date is often inaccurate.
Warning Letter Responses also appear to be missing from the new database.
Downloadable Results are presented as Excel files, but are actually HTML.
Warning Letter results return a maximum of 1,000 records, but this limit is not explicitly noted on the web-site.
I have an email in to FDA asking them to repair and notify others who may have been led astray.
I posted earlier about creating a plugin for Mozilla's Firefox that allowed users to search the FDA and CDCwebsites and FDAble's search engines by right-clicking on a highlighted word.
Turns out that you can submit a plugin, but it's considered experimental until you've received reviews.
You also have to write a short justification of why your plugin is worthy of release to the public at large.
1. download the plugin. 2. if you're on a web-page that contains a drug name or other health related term, highlight the term 3. right-click on the highlighted term and choose whether you'd like to use the term to search, the FDA, CDC or FDAble search engines.
FDA used to have a collection of web-pages that allowed you to search the warning letters and responses that it had issued to various food & drug scofflaws and ne’er-do-wells all the way back to 1996.
There were certainly some strange choices made with the old system that they used.For one thing, they separated the “old” warning letters (those > 1 year old) from the new ones (<= 1 year old) and you had to use a separate search engine for each collection of reports.
With the new search engine, they’ve combined old and new so that both can be searched from one form. However, this appears to be the only thing that they got right with the upgrade.
Another peculiarity was that if you used the old system to download an Excel table of warning letters filtered by date, you got a CSV file that was mistakenly tagged with an .xls extension.This transgression is no big deal as CSV will be read easily by Excel even if it’s mis-tagged, but whoever built the new version seems to have taken the mislabeling one step further (see below). If you dig deeper into the web-pages, you find all sorts of weirdness.
First, your searches are capped at 1,000 results no matter how big the true size of results.The search-form doesn’t say that it will only return the first 1,000 results, but it does.And this initially led to confusion on my part because I was trying to see if the system would retrieve all 9,000+ warning letters that should be in the system.It only returned 1,000.
This is a bit dangerous b/c if a user searches for all warning letters from 1996 to 2009 s/he may mistakenly conclude that there have only been about 1,000 reports issued.What’s the deal?I seriously doubt they’re low on computing power.
The same holds true if you try to download an Excel table of the warning letters (you only get 1,000 results) no matter what you try.
And here’s the really strange bit.Remember how I said that the old system delivered a CSV file that was mis-labeled as an xls file?
Well, the *new* system again lets you download what is ostensibly an Excel file, but it’s not an xls file.It’s also not a CSV file like the old system.And no, it’s not one of those new-fangled Microsoft Office 2007 xlsx files.It’s a file marked with an xls extension, but if you open it up with notepad, you’ll find that it’s HTML !
Specifically, they’ve packaged the HTML table that is returned when a user searches their web-interface for warning letters and passed it off as Excel.Why?I have no explanation, except sheer laziness.
Finally, this section of the FDA’s website is titled “Warning Letters and Responses”[emphasis mine] and there used to be a way to search the responses to the warning letters…and the downloadable 'csv' file would list the location of letters received by the FDA in response to their warning salvos.
Also, they moved the URLs for all of the html versions of their warning letters, thereby breaking all of the fdable warning letter links.It’s not like the FDA is legally obligated to inform me of these changes, but when they do stuff like this they end up breaking the links for anyone/everyone who has ever bothered to link to their warning letter data. (time for me to get back to work…).
FDA has a database of postmarket requirements and study commitments (translation: "we're going to approve your drugs, but you have to promise to run these additional trials after we approve").
This database is updated quarterly and FDA even provides its own search engine to mine this database.
I've only been indexing their collection of data for a couple of quarters, but with the most recent update (April 30), I noticed something that I hadn't realized before.
Specifically, each new quarter's data *wipes out* the previous quarter's data. For example, the January 31 update contained 3381 postmarket requirements that could be searched via FDA.
In contrast, the April 30 data release contains only 1910 requirements. So, it looks like the FDA removes commitments that have been satisfied (I'm not 100% sure about the precise commitments that are removed each quarter, but this seems like the most reasonable explanation).
In practice, this means that you can't use FDA's search-engine to get a historical look at the types of commitments/requirements that were mandated by the FDA.
I imagine that this historical information would be useful to many people (clinicians, persons performing competitive intelligence & strategy regarding drug development, etc.).
Like everyone else on the planet looking to score 99 cents (70 cents after commission), I developed an iPhone app: FDA Mobile News.
Basically it's a glorified RSS reader devoted to FDA's RSS feeds and I charge a whopping $0.00 for it (I get 70% of that--so who's the fool, now?).
It still has some problems that need to be worked out (as my only reviewer pointed out--someone who goes by the name of "Saddam Hussein's Lost Car Keys" [WTF?] it isn't particularly clear that the globe icon that I use is actually a hyperlink to the full RSS article).
Well, the way the App Store is these days, I wouldn't be surprised if my parents were toiling away in their basement crafting some "newfangled iAARP contraption for that iTelephone store where everybody gets rich"--70 cents at a time.
Posts
