17 Jul Blog: 17 challenging study designs
With the start of the empirical evaluations of 17 integrated care programmes, we have entered a crucial and exciting phase of the SELFIE project. These 17 programmes, spread over 8 countries, include 6 population health management programmes, 5 frail elderly programmes, 3 programmes for persons with problems in multiple life domains besides their health, and 3 palliative care and oncology programmes.
After thorough discussions, we have reached consensus about a core set of outcomes that we will measure in all programme evaluations, and four sets of programme-type specific outcomes for the programmes as grouped above. To ensure that these outcomes include patient-reported outcome and experience measures (PROMs and PREMs) that really matter to people with multi-morbidity, the choice of outcomes was largely guided by what persons with multi-morbidity mentioned during the focus groups that were held in each country.
17 Challenging study designs
Alongside discussing the outcomes, we also debated the study design for each programme. The most heavily debated study designs were those to evaluate the population health management programmes. The reason being threefold: 1) the implementation of these programmes often started many years ago, 2) the programmes target the entire population in a region, and 3) the identification of an appropriate control group is a great challenge. The first requires the acknowledgement of outcome measures used to evaluate these programmes in the past. The second leads to a target population that includes subjects that have never been exposed to a particular intervention and thus are unaware of being part of a population health management programme. The third we managed to overcome because all population-health management programmes will be compared to a comparator group that consists of either people living in a different geographical region, being insured by a different health insurer, receiving care from different providers, or national-level data. However, unlike the other programmes that apply prospective evaluations in which the PROMs and PREMs are repeatedly measured in the same individuals, some population health management programmes could only do just one cross-sectional measurement of the PROMs and PREMs. The good news is that the latter programmes have access to a richness of routinely registered claims data and health surveillance data in both intervention and control group, which allows interrupted time series analysis and difference-in-difference analysis. These data are mostly indicators of structures, processes and proxies of health outcomes.
Relative importance of outcomes
The empirical evaluations will provide evidence about the performance of the integrated care programmes on a wide range of outcomes. In SELFIE we also measure the relative importance of the various outcomes, using a weight-elicitation study consisting of a Discrete Choice Experiment and Swing Weighting. We do this in each of the eight countries among representatives of the 5 Ps, i.e. Patients, their Partners (informal caregivers), Professionals (providers), Payers, and Policy makers. This weight-elicitation study is separate from the specific programme evaluations, but has started at about the same time as the latter. The greatest challenge we face here is recruiting a sufficient number of respondents among the payers and policy makers.
To be continued…
Once data from both studies are available we will combine the performance of the programmes on the outcomes with the importance-weights of these outcomes to perform the Multi-Criteria Decision Analyses of the 17 programmes. We’ll keep you posted on the progress.
Prof.dr. Maureen Rutten-van Mölken, coordinator of SELFIE