I. Finding and Using Healthcare Data

9. How can I perform validity checks of pharmacy data?

Pharmacy data will inevitably contain some missing or erroneous values. Many of these can be located through simple searches based on variable values. Here are some examples:

· NDC begins with '00000' or '99999'

· Days Supplied and Quantity Supplied are not integers greater than 0

· Unit or total cost = $0

Records with these values should not be discarded without investigation; it may be possible to rectify an obvious error using other data on the record.

We recommend using PROC UNIVARIATE in SAS, or similar commands in other applications, as a baseline for checking data quality. When it is possible to focus on a small number of medications, another useful check is to calculate the implied dose per day:

      Daily Dose = (# Units Dispensed * Strength per Unit) / (Days Supplied)

For example, if a 15-day supply consists of 30 pills at 200mg per pill, then the daily dose would be 400 mg/day: (30*200)/15 = 400. Package inserts may be consulted to determine standard dose ranges. Doses that are too small to be clinically meaningful or so high as to be toxic should be flagged as potentially erroneous. Investigation of individual prescription records would then be indicated, and potentially utilization records as well if available.

A potential hazard in this method is variation across facilities in the assignment of dispensed units. For example, a 50ml injection may be recorded at one facility as 50 units and at another as 1 unit. In theory the two may be reconciled based on other information on the record, such as the NDC and the dosage instructions. Variations in dispensed units are quite common. One approach to locating them is to tabulate the range of dispensed units for selected NDCs. Variation by a factor of more than 3 or 4 between greatest and least is probably a sign that further investigation is warranted.

Checking the validity of cost data is likewise important but potentially complicated. Depending on when local pharmacies update their drug cost files, in theory one could observe two prices for a single NDC on the same day. Likewise, Blanket Purchase Agreements may cause disparities between two facilities on the same day for the same product. Moreover, not every local pharmacy updates its own cost file every day. What may be done to find or avoid errors then? Again, univariate statistics can alert researchers to outlying values. And cost changes for particular NDCs should not be too great. Cost variation across facilities and across time within a single facility should be moderate; extreme changes are likely signs of error.

A more comprehensive way to avoid cost errors is to use the Pharmacy Benefit Management staff's historical drug cost file to assign a standard price for each NDC covered by federal contracts. (See http://www.pbm.va.gov/default.aspx for contact information.) This reveals an opportunity for sensitivity analyses as well: determining the difference in outcome caused by using alternative medication cost systems. Aside from observed VA costs, choices include the optimal VA cost based on the historical cost file; federal contract prices without the "Big Four" discounts; and publicly available prices, such as a fixed percentage of AWP plus a nominal dispensing fee. (Without information on Blanket Purchase Agreements, however, it will not be possible to recreate the optimal cost for every VA facility. Ignoring BPAs is unlikely to cause significant errors, however.)

If erroneous values are located, there are several options. Each NDC corresponds to a particular package size, thereby providing an alternative measure of units dispensed. Outlier values of dispensing unit (mg, ml, etc.) may be corrected by reference to other prescriptions with the same NDC. Cost values may be corrected by reference to the historical file created by the PBM staff or by taking an average of costs for the same NDC. As much as possible, draw data for cost corrections from the same facility and day or week.

VIREC staff performed a careful validation of the DSS National Data Extract that features prescription-level pharmacy data, comparing it to the PBM prescription-level database. The report revealed the range of errors and inconsistencies that occur in these two VA databases and shows some methods for dealing with them. The report is available on the VIREC web site.

Judgment of proper dosage requires clinical expertise. The presence of comorbid conditions or concomitant medications can lead dosages to deviate from the typical range, and many medications are started at subclinical dosages in order to detect intolerance or adverse events. Only highly unusual values should be considered suspect.