Utilising open research practices to conduct a systematic review and meta-analysis of healthcare data

The issue

Pancreatic cancer is rare but has dismal survival rates meaning it’s the sixth cause of UK cancer mortality.  It often presents with non-specific symptoms. This makes early diagnosis challenging. However, because weight loss occurs in most patients with pancreatic cancer, it could be a useful diagnostic marker. Our aim was to improve its utility as a marker for pancreatic cancer by quantifying pre-diagnosis weight loss. We undertook a systematic review with meta-analysis.  

However, healthcare data must remain private and confidential. Healthcare datasets cannot be shared in their original form. Only non-identifiable data can be shared and only in a safe environment. This means it is challenging to ensure transparency in research using healthcare data. 

Our approach and challenges 

Before conducting this review CP attended an open research course at the University of Surrey (1) to improve her knowledge of open research and best practices. We registered the study protocol on PROSPERO (2), which is an international database where researchers can submit their proposals for literature reviews. This helps the research to be transparent and reproducible as well as avoiding duplication of work.  It also allows a comparison of what was planned and what was completed for the research.  We followed PRISMA guidelines which provide a framework for what information must be included when writing up the research, such as a well-defined research question together with reasons studies can be excluded from the research.  

We conducted a systematic review and meta-analysis of studies which included quantifiable information about weight loss prior to pancreatic cancer diagnosis. The search for studies to include was conducted using five different databases in November 2023. We used a tool called ROBINS-I (3) to assess possible bias in how the studies we found were conducted.  Examining possible bias is important as this may influence the results when synthesising the data from many studies and using a tool for this assessment is important so that bias is appraised in a standard and reproducible manner. 

Extracted data has been deposited in a database for openly sharing datasets (Zenodo (4)) and will be submitted as a Research Elements article which focusses on how the data was extracted from the studies. The review will be shared as a preprint to improve the speed of dissemination. It will also be submitted for publication in a peer-reviewed open access journal, increasing communication and knowledge in the area of improving the early detection of pancreatic cancer. 

Transparency of methodology enables reproducibility as well as preventing duplication of effort and cost. Depositing extracted data maximises the findability and accessibility of the data. This encourages data reuse practices which supports data minimisation. This means that only as little data as is required is collected and that the same data is not collected several times, which is especially important for potentially invasive research, such as that involving healthcare data, 

We also encountered challenges when conducting this review which were related to limited open research practices in the original studies.  Extracting data for meta-analysis was difficult due to heterogenous study types and a lack of transparency. Many original studies lacked meta-data, only presented data graphically or only reported summary statistics (e.g. as the result of a regression model) so the raw data is not provided. Authors of the original studies often could not clarify results. Without meta-data to enable data reuse this meant some studies had to be excluded from meta-analysis. 

The outcome 

Out of 6,664 original hits 30 studies were included in the review and meta-analysis. Random effects meta-analysis was conducted across three different ways of reporting weight loss (the amount of weight lost (kg), BMI change and proportion of people losing weight). We found 57.4% of people lose an average of 5.89 kg or have a BMI change of -2.54 kg/m² prior to pancreatic cancer diagnosis. However, there was a great deal of heterogeneity between the studies so there is limited confidence in the results and their wider applicability.   

Conclusions 

Research using healthcare records needs to be approached carefully, due to privacy and other ethical considerations. However, maximising open research practices enables data to be shared to maximise patient benefit. This avoids repetition of work and increases reproducibility. 

Original datasets that consist of healthcare records cannot be made open access. However, open research practices can still be adopted by sharing data that has been aggregated or deindentified. It is important to present meta-data to provide context for the research. This improves standardisation across study types and facilitates study combinability. 

1. Fellows. Open Research training [Internet]. [cited 2024 Apr 8]. Available from: https://www.surrey.ac.uk/library/open-research/open-research-training 

2. Price CA, Cooke D, Smith N, Wynn M, Lemanska A. Weight Loss Prior to Pancreatic Cancer Diagnosis: A systematic review and meta-analysis [Internet]. PROSPERO. 2022 [cited 2023 Jan 6]. Available from: https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=302985 

3. Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ. 2016 Oct 12;355:i4919. 

4. Price CA. PaCaClaire/Systematic-Review-Weight-Loss-Extracted-Data: Weight Loss Prior to Pancreatic Cancer Diagnosis - Systematic Review Extracted Data [Internet]. 2023. Available from: https://zenodo.org/records/10118729