Technical Report 43: Effects of VA Registry Information on Nosos Risk Scores
Suggested CitationDesai M, Gehlert E, Wagner TH. Effects of VA Registry Information on Nosos Risk Scores. Technical Report 43. Health Economics Resource Center, U.S. Department of Veterans Affairs. September 2023. https://www.herc.research.va.gov/include/page.asp?id=technical-report-43.
All tables for the report are saved in an Excel file. Download the tables here.
For a list of VA acronyms, please visit the VA acronym checker on the VA intranet at http://vaww.va.gov/Acronyms/fulllist.cfm.
1. Introduction
The U.S. Department of Veterans Affairs (VA) created Nosos, a risk score, to support performance measurement. Nosos was developed in 2016, replacing the DxCG software (supported by Verisk). (1) Nosos is based on the Centers for Medicare and Medicaid Services (CMS) Hierarchical Conditions Categories (HCC) risk adjustment system developed for Medicare Advantage, along with additional VA-specific information on mental health and medication history. (2)
Nosos predicts each patient’s average spending for this year and for next year. (3,4) This year’s prediction is referred to as a concurrent score (Nosos_c) and next year’s prediction is referred to as a prospective score (Nosos_p). Both scores have their strengths and weakness. (5) All Nosos scores are centered around 1. A score of 1 means that the Veteran is expected to have costs consistent with the average VA patient; a score of 2.8, means that patient is expected to have costs 2.8 higher than the average VA patient.
One of the additional VA-specific variables used to compute Nosos is a registry variable obtained from the VA Allocation Resource Center. The registry variable identifies whether a Veteran is enrolled in one of 16 VA registries: spinal cord injury, chronically mentally ill, blind, post-traumatic stress disorder (PTSD), alcohol, stroke, hepatitis C virus, home health, domiciliary, long-term care, brain, AIDS, transplant, end stage renal disease, homeless, care coordination home telehealth (see Table 1). The registry variable has been included since the inception of Nosos, but lack of clarity about its source and relation to other health conditions has raised questions about its future inclusion. The goal of this study was to understand the effect of removing the registry variable from Nosos.
2. Methods
We computed the Nosos prospective and concurrent scores with and without the registry variable for all Veteran enrollees in 2022 (n=6 million). For our analyses we extracted two 20% random samples. This enabled us to examine the robustness of our results on a separate sample. We compared Nosos scores to see the effects of VA registry information on the parameters age, sex, HCCs, psychiatric groups, and VA drug class category.
Nosos scores are positive numbers, calibrated around 1. Healthy patients with a low expected cost have a score lower than 1, whereas patients with an above average expected cost will have scores above 1. In both of our samples we suspected that there would be a distributional issue; we confirmed our hypothesis by kernel density plots. The distribution of concurrent Nosos scores with the registry variable included is shown in Figure 1. A similar distribution was seen with prospective Nosos scores, as well as both Nosos scores without the registry variables.
Figure 1. Kernel density plot of concurrent Nosos scores with registry variable
These skewed distributions create analytical problems. To understand the effects of the skewed distribution, we divided the data into four equal quartiles and looked at the effects on the quartiles by each category of interest. Quartile 1 represents the least expensive (healthiest) patients, while quartile 4 represents the most expensive patients.
We started our analysis by restricting the scores to each category of interest and created two new variables that represented the difference between the concurrent score with the registry variable and the concurrent score without the registry variable. Variation in mean scores is affected by sample size, so we computed standardized mean difference (SMD) to gauge the magnitude of change.
We flagged the SMDs that were greater than 0.5. We did the same analysis on both 20% random samples to check the robustness of our analysis and to see if the large differences in means were robust.
3. Results
Sample characteristics
Sample 1 had 1,288,713 observations (1,096,577 males and 192,136 females) and sample 2 had 1,288,714 observations (1,096,577 males and 192,137 females). The median age was 65 with an interquartile range of 49-75. Approximately 7% of the sample was in at least one registry. The most common HCCs were major depression (15%), diabetes with complications (12%) and diabetes without complications (11%). The per person average VA costs in FY22 were $13,509, with a minimum of $0 and maximum of $5,108,666.
Concurrent Nosos scores with the registry variable ranged from 0.15 to 45.29, with a mean of 1, as expected. Computing the concurrent Nosos score without the registry variable resulted in a concurrent Nosos score that ranged from 0.16 to 47.95.
Fifty-two percent of Veterans were prescribed a cardiovascular drug, including commonly prescribed hypertension medications. Many Veterans in the sample had a mental health condition picked up in the PsyCMS condition categories: 14% had PTSD, 12% had nicotine dependence, 8% had recurrent major depression, 5% had anxiety, and 5% had adjustment disorder.
Effect of registry data on age/sex
Table 2 shows the registry variable has little effect on concurrent scores, while Table 3 shows registry information has a large effect on prospective scores for Veterans aged 18 to 34 and 45 to 51, mostly in quartile 4. The registry information predicts costs for the most expensive patients for the following year for patients aged 18 to 51. However, we see no significant differences among the other quartiles or age groups on prospective scores. The distribution of age in the five groups was almost equal, so there were no discrepancies caused by sample size (Figure 2). We also see no effects from VA registry information on sex groupings (Tables 4 and 5; Figures 2 and 3).
Figure 2. Standardized mean differences of concurrent Nosos score, by age
Figure 3. Standardized mean differences of concurrent Nosos score, by sex
Effect of registry data on hierarchical condition categories (HCCs)
Table 6 shows the concurrent risk scores by HCC for both samples and Table 7 shows the prospective risk scores by HCC for both samples. In both samples the SMD was less than 0.5 for quartiles 1, 2, and 3. Standardized mean differences greater than 0.5 were seen in both samples in quartile 4 and are highlighted in red text within Tables 6 and 7.
Including registry information has an impact on high-cost patients in the following 12 HCCs: HIV/AIDS, Quadriplegia, Paraplegia, Amyotrophic Lateral Sclerosis and Other Motor Neuron Disease, Cerebral Palsy, Intracranial Hemorrhage, Ischemic or Unspecified Stroke, Hemiplegia/Hemiparesis, Monoplegia, Other Paralytic Syndromes, Dialysis Status and Chronic Kidney Disease-Stage 5.
Effects of registry data on PsyCMS and VA drug class
Excluding the registry variable had little impact on the PsyCMS conditions (all SMDs were less than 0.5; Tables 8 and 9). Likewise, we see no substantial effect on any of the 25 VA drug class categories after removing the registry variables (Tables 10 and 11).
Robustness Checks
Analyses with the second sample showed very similar results, suggesting that the results are highly robust.
4. Discussion
For most patients in our samples, excluding registry information little discernable effect on their Nosos score. Although the registry variable had little effect on average Nosos scores, we did find that the registry variable was an independent predictor of high costs among a handful of HCC’s. Larger SMDs were correlated with functioning issues (paraplegia, quadriplegia and some post stroke functioning), possibly catching a relationship with motor neuron programs that is not captured within the HCCs. This suggests that the registry may be picking up disability information that the VA has, perhaps through Veterans Benefit Administration (VBA).
Our work shows VA registry data has minimal effect on Nosos scores overall but additional work on subgroup analyses is warranted to fully understand the impact of VA registry information on the Nosos risk score. Additional analyses should examine the impact of excluding the registry information on the 12 HCCs with large SMDs.
In conclusion, VA can exclude the registry information in the Nosos model as the registry variable has minimal effect on overall Nosos scores.
References
- Wagner TH, Upadhyay A, Cowgill E, Stefos T, Moran E, Asch SM, Almenoff P. Risk Adjustment Tools for Learning Health Systems: A Comparison of DxCG and CMS-HCC V21. Health Serv Res. 2016 Oct;51(5):2002-19. doi: 10.1111/1475-6773.12454. Epub 2016 Feb 3. PMID: 26839976; PMCID: PMC5034200.
- Wagner TH, Gehlert E, Rosen A, Valenstein M. Updating the Psychiatric Case Mix System (PsyCMS) Mental Health and Substance Use Grouper for ICD-10-CM. Technical Report 31. Health Economics Resource Center, U.S. Department of Veterans Affairs. November 2016. https://www.herc.research.va.gov/include/page.asp?id=technical-report-psycms-icd10.
- Wagner T, Stefos T, Moran E, Cashy J, Shen ML, Gehlert E, Asch S, Almenoff P. Risk Adjustment: Guide to the V21 and Nosos Risk Score Programs. Technical Report 30. Health Economics Resource Center, U.S. Department of Veterans Affairs. February 2016. https://www.herc.research.va.gov/include/page.asp?id=technical-report-risk-adjustment.
- Wagner T, Moran E, Shen ML, Gehlert E. HERC's Guide to the Nosos Risk Adjustment Score. Health Economics Resource Center, U.S. Department of Veterans Affairs. January 2022. https://www.herc.research.va.gov/include/page.asp?id=guidebook-nosos.
- Health Economics Resource Center. Nosos Risk Adjustment.” August 30, 2022. Accessed August 18, 2023. www.herc.research.va.gov/include/page.asp?id=risk-adjustment.
