A review of NICE appraisals of pharmaceuticals 2000-2016 found variation in establishing comparative clinical effectiveness.

OBJECTIVE: To identify and assess the methods for estimating comparative clinical effectiveness for novel pharmaceutical products licensed on the basis of non-RCT data and to evaluate the corresponding NICE recommendations.

METHODS: Our identification strategy was two-fold. First, we reviewed all NICE appraisals between 2010 and 2016 and identified technologies where comparative clinical effectiveness estimates were calculated using non-RCT data. Second, we checked if NICE appraisals completed from 2000 to 2010 had included pharmaceuticals that were granted EMA marketing authorisation without RCT data between 1999-2014. Information was extracted on the methods used as well as the corresponding NICE recommendations. We also collected information on the rationale for utilising non-RCT data in NICE appraisals.

RESULTS: Of 489 individual pharmaceutical technologies assessed by NICE, 22 (4%) used non-RCT data to estimate comparative clinical effectiveness. Methods for establishing external controls in such studies varied: 13 (59%) used published trials, 6 (27%) used observational data, 2 (9%) used expert opinion, and 1 (5%) used a responder vs non-responder analysis. Only 5 (23%) used a regression model to adjust for covariates. We did not observe a notable difference in the proportion of pharmaceutical technologies that received a positive recommendation from NICE whether the decision was based on RCT or non-RCT data. (83% vs 86%) CONCLUSIONS: To date, the small number of appraisals by NICE based on non-RCT data did not result in substantially different treatment decisions. The majority of the technologies appraised on the basis of non-RCT data either received a positive recommendation or a positive recommendation with restrictions. The methods used to calculate comparative clinical effectiveness estimates varied, highlighting the need to establish clear guidance.