This is quite an interesting article. It caught my eye that the logistic regression models used by Nightingale in the article were amusingly named “AI models,” which perhaps I should start using in my own publications. In a sense, it’s true if one understands an AI model very broadly.
In the publication, health check-ups primarily refer to health check-ups performed by occupational health nurses in occupational health care and the potential intervention chain that might be initiated from them; those at high risk get to see a doctor who can prescribe various interventions. It’s worth noting that almost all savings and cost-effectiveness come from discontinuing the nurses’ work and replacing it with Nightingale’s blood test and risk tests.
In the public sector, health check-ups have largely been discontinued as ineffective, so savings in that regard have already been realized. Health check-ups would not necessarily need to be replaced by anything (e.g., NG’s test); they could simply be stopped, and that in itself would be cost-effective and lead to significant savings. Here, savings have mainly been achieved by automating the work done by nurses, which could, of course, be valuable.
The article’s simulation is quite well done, even though it uses “synthetic data,” which may not always increase credibility. This study has particularly well considered what such risk testing could lead to and modeled various common interventions that could follow from different risk levels (here, for example, intervention to stop smoking, tightening blood pressure or cholesterol targets, weight loss coaching) and given the effects on their realization. Finally, after applying all these to those at elevated risk and replacing current health check-ups with Nightingale’s test, the remaining expected quality-adjusted life years (QALY) for the cohort increase from 31.4 to 31.5 in scenarios 1 and 2, and correspondingly decrease by 0.01 years in scenario 3
.
At the population level, the changes achieved in quality-adjusted life years are in the order of a few per mille, i.e., insignificant, but the article appears to be the first to consider that testing with Nightingale’s test could also worsen people’s health and life expectancy at the population level, i.e., compared to the current operating model, which involves a nurse’s appointment where FINRISK and FINDRISC are performed.
This, in my opinion, is the most interesting aspect of the article; it’s a pity that the authors do not elaborate on it further in the text or supplementary materials. It is only stated that there are differences in classification compared to the control group, and later there is talk of “inaccuracies” in NG’s metabolomics modeling and classification ability, which is further mentioned to be “slightly weaker” than presented in previous studies.
However, there are also several limitations. First, there are some inaccuracies remaining in the modelling of the MRS risk of individuals and modelled parametric disease-free survival times (i.e. the ability of MRS to recognize the ones in true risk seems to be a bit weaker in our model than reported in previous studies [13,14]). This makes the results presented weaker than they ought to be.
This could have been elaborated on more, but the main focus of this study seemed to be to evaluate the effects of NG’s test on costs and workloads, and it is stated in passing that these are significant and worthwhile despite possible (small) negative health effects. This is one way to lobby for a cause, but one must consider what it means for NG’s investment case if one pivots to talking only about cost-effectiveness and forgets about health effectiveness.
Of course, this study does not actually say whether NG’s test worsens or improves people’s health. The results are vague, within the margin of error, and were generated with synthetic data and a thousand assumptions. Transparency is poor. But the classification ability of NG tests is very poorly described and especially validated, and even here, worrying inaccuracies were referred to.