Development of a Model for Differentiating PDAC from Benign Pancreatic Conditions: A Prospective Case-control Study

Materials and methods Study setting

All data utilised for this research were collected from the Barts Pancreas Tissue Bank (BPTB; https://www.bartspancreastissuebank.org.uk/)) study, with written informed consent from patients recruited at Barts Health NHS Trust (BHNT) (Hampshire B Research Ethics Committee: 18/SC/0630). The BPTB is a repository of biospecimens and associated clinical data collected from patients (≥18 years) referred to the specialist clinic of HPB surgery at BHNT (Barts and the London HPB Centre) for confirmed or suspected malignant or benign diseases of the pancreas and other diseases of hepatobiliary origin. The biobank also recruits healthy volunteers to be used as contemporaneous comparison cohort.

Data on BPTB participants’ ‘health’ features are entered into a bespoke, encrypted, secure database using a predefined 49-item questionnaire, of which 37 items are completed by trained tissue collection officers in a face-to-face interview during recruitment.²¹ These include demographic information, anthropometric measurements, persistent symptoms, comorbidities, regular use of specific medication, lifestyle behaviours, family life, and family history of pancreatic and other cancers. Another 12 items of information (with varying granularity) about clinical diagnoses, treatment including surgical and non-surgical interventions, and test results including blood test, urine test, imaging, and histopathology reports are populated from their electronic health records (EHR) data at BHNT. Data is further cross-checked and verified by trained clinicians to ensure accuracy.

Study design

A ‘nested’ case-control study was carried out to examine the association of PDAC diagnosis with a set of predictor variables, in comparison to patients with non-malignant pancreatic diseases (PnC), and subsequently to develop a risk-prediction algorithm that may separate diagnosis of PDAC from PnC. The choice of PnC as a reference group was driven by the perceived clinical utility of the potential findings. Patients with pancreatic diseases, malignant and non-malignant alike, are believed to demonstrate similar symptomatic presentation (e.g., jaundice) and biochemical profile (e.g., elevated CA19-9 or deranged Liver Function Tests),¹⁹ therefore identifying any distinctive clinical profile may have an impact on the evidence-based decision making of the referral and diagnostic pathway.

We extracted relevant clinical data for all patients registered with the BPTB study between January 1, 2008 and September 21, 2021 and diagnosed with any pancreatic condition (N=780 of 1371) by excluding healthy controls and those with non-pancreatic conditions (n=588). A diagnosis of pancreatic neoplasm is recorded in the BPTB database using the ICD-O-3 (International Classification of Diseases - Oncology, 2013) multi-axial classification for the site and the histology of the neoplasm; the non-neoplastic disorders of pancreas are coded using the ICD-10 codes (2019) (Supplementary Table 1). The final diagnosis recorded in the database is recorded using hierarchical ascertainment mechanisms: histology reports, multidisciplinary team meetings outcomes, consultation notes from the oncology or gastroenterology specialist, findings from test results (e.g., radiological and endoscopic reports) and/or commissioning data sets diagnosis entry. The date on the first confirmatory histology report is used as the date of diagnosis, otherwise the earliest of dates from the other evidences is used. Complex diagnoses or cases with incomplete data are agreed on by an adjudication group of clinicians (HMK, KS, MZ).

Patients were finally assigned to one of the three groups (in the order of priority) - PDAC, Other pancreatic cancers (PC), and PnC, based on their final diagnosis entry in the database. For each individual within a specific group, the date associated with the earliest diagnosis was considered as the index date. This is particularly applicable for individuals in the PnC group who can have final diagnoses of different pancreatic diseases. Care was taken to exclude patients, with secondary tumour to the pancreas (n=2), with concomitant primary cancer in another body site (n=6), and those who were assigned a provisional diagnosis but were not reassigned to a confirmed diagnosis (n=1) to avoid biases (Figure 1). Comorbidity and symptoms history were not collected for nine patients. Since the study focused solely on PDAC, we excluded patients with other pancreatic cancer (n=58) (Supplementary Table 2).

• Download figure
• Open in new tab

Figure 1.

CONSORT diagram for selection of patients in the nested case-control study.

Assessment of Predictor variables

Details on the predictor variables and their categorisation are provided in Supplementary Table 3. Demographic details included in the study were gender (Male, Female); age (<55, 55-64, 65-74, 75+); ethnic origin (Caucasian, Afro-Caribbean, South Asian, or Other). Lifestyle variables included smoking history of alcohol consumption (current, past, or never) and body mass index (BMI). Clinical variables relating to comorbidities (present or absent) comprised of; diabetes, hypertension, high cholesterol, chronic respiratory disease, cardiovascular disease, chronic kidney disease, liver disease and previous cancer diagnosis. Both lifestyle and comorbidity variables were recorded based on the patients’ status at the time of recruitment.

The following eleven symptoms that might herald a diagnosis of gastrointestinal problems were included as binary outcomes (present vs absent): pain, jaundice, weight loss, nausea, vomiting, diarrhoea, constipation, fatigue, loss of appetite, pruritus, and steatorrhea. The presence or absence of these symptoms within the one-year period before recruitment were confirmed by the TCOs in a face-to-face interview. A composite symptom was also derived in the form of change in bowel habits, manifested in any of diarrhoea, constipation, or steatorrhea.

Twenty-two laboratory tests, commonly requested by clinicians for suspected HPB problems, were examined (Supplementary Table 3). For each test, the instance closest to the recruitment date and conducted within six months before the recruitment was considered, otherwise the first instance within thirty days of recruitment was used.

Statistical analyses

Differences in demographic and clinical characteristics between the PDAC and PnC groups were assessed using Pearson’s Chi-square test or Kruskal-Wallis rank sum test, as appropriate. Predictor variables with data missing in more than 25% of the final study population (N=704) were excluded from further analyses. We conducted the preliminary observational association study and the subsequent development of the risk-prediction algorithms on 75% of the data (derivation set). We then assessed the prediction algorithms on the remaining 25% (validation set). Both derivation and validation sets had similar proportions of PDAC and PnC patients. Multiple imputation by chained equations method was used to replace missing values for predictor variables.^22,23 Five imputations were carried out separately in derivation and validation datasets. In the imputed datasets, results from each blood test were stratified by the known normal range of respective pathology test guidelines (Supplementary Table 4) and included in the analyses as categorical variables with three possible categories: low, normal and high. To examine the PDAC risk associated with individual predictor variables, the effect size was evaluated with odds ratios (OR) with 95% confidence intervals (CI), using multivariable regression models with a binomial distribution. OR for individual predictor variables were obtained from independent regression models, controlled for potential confounding effect by demographic variables - age group, gender, and ethnicity. Rubin’s rules were used to combine the results across the imputed datasets.²²

Several risk-prediction models were developed utilising the findings from observational association investigation. All predictor variables having at least one category with high statistical significance (p<0.01) after correction for multiple testing were retained to be used in the development of prediction models. Finally, with an aim to capture the effect of presenting symptoms and common blood tests in specific patient subgroups, we examined their interactions to encompass all demographic, comorbidity and lifestyle variables; all statistically significant two-way interactions (p<0.01) were subsequently included. Multivariate logistic regression models were fitted within the supervised machine learning setting. The outcome was the probability that a patient would develop PDAC, derived as a function of the predictor variables. Five variations of logistic regression (LR) models were fitted with: no penalty function, lasso regression with L1 regularisation penalty (LR-Lasso), ridge regression with L2 regularisation penalty utilising Bayesian Information Criterion (LR-Ridge), elastic net regression with L1/L2 regularisation penalty (LR-ElasticNet), and stepwise model selection by backward elimination utilising Akaike Information Criterion (LR-StepAIC). The average receiver operating characteristic (ROC) statistic from repeated 10-fold cross-validation training run was used as the metric to determine the optimal hyperparameters for the prediction models. The regression coefficients in the final LR, LR-Lasso, LR-Ridge and LR-ElasticNet models were pooled from preliminary models fitted to the imputed datasets. The final LR-StepAIC model was a regular LR model fitted with a reduced set of predictor variables, after applying a voting-based variable selection technique to the five LR-StepAIC models on the imputed datasets – variables were excluded if they did not appear in all the preliminary models.

Models’ performance were assessed based on the pooled outcome probability from five multiply imputed validation datasets. The best model was chosen based on the discrimination and calibration ability, which were measured by calculating the area under the receiver operating characteristic curve (AUROC) statistic²⁴ and Spiegelhalter Z-test²⁵ respectively. Once the best model was selected, it was applied on the full dataset to extract the F-scores and decide on the optimal probability threshold for identifying potential high-risk PDAC cases. F₁ score is the harmonic mean of precision (positive predictive value) and recall (sensitivity) with equal importance. F₂ score attaches twice as much importance to recall as precision, which is an important consideration for this study to distinguish PDAC patients from non-malignant pancreatic condition patients rather than “otherwise healthy” control population.

All statistical analyses and visualisations were performed in R (version 3.5.1).

Descriptive analyses

Overall, the study included 344 patients with a primary diagnosis of PDAC and 360 patients with other pancreatic non-malignant conditions (PnC) (Figure 1). The PnC group consisted of patients diagnosed with chronic pancreatic conditions (CP) including chronic pancreatitis (n=113, 31%), benign tumour (n=86, 24%), and pancreatic cyst and pseudocyst (n=57, 16%); the rest of the patients (n=104, 29%) had acute pancreatitis or other benign diseases without progression to chronic conditions. Supplementary Table 5 presents baseline characteristics of the two groups. Both groups were male dominated, but the representation is notably higher in the PnC group (p=0.019). PDAC patients were significantly older (median 68 years, interquartile range (IQR) 60–75 years) compared to PnC (median 55, IQR 45-65.2, p<0.001), and had more prevalent comorbidities, particularly diabetes (p=0.002 and high blood pressure (p<0.001). Both groups had similar majority representation from White ethnic backgrounds, but South Asian representation was notably lower in the PDAC group (4.9%) than in the PnC group (10.3%) (p<0.001). At the time of recruitment (closer to the time of index diagnosis), PDAC patients had relatively lower BMI (median 24.2, IQR 22.1-27.5) compared to PnC patients (median 25.5, IQR 22.8-29.3, p=0.002). Histories of smoking and alcohol consumption were marginally less in PDAC group compared to PnC group, with notably lower current smokers (18.6% vs 27.2%) and past drinkers (12.8% vs 21.1%) among PDAC patients.

In the year prior to recruitment, jaundice and weight loss were reported more frequently by PDAC cases, while pain, nausea, vomiting and diarrhoea were more common among PnC patients (Supplementary Table 6). Although the incidence of pruritus was overall low among pancreatic patients, it was reported around four times more frequently by PDAC cases than PnC patients (p=0.005). The biochemical profiles of PDAC patients show significant association with higher levels of ALT, MCV, PLT, NEUT, CRP (all p<0.01) and lower levels of ALB, AMY, CREA, SOD, RBC, HB and LYMP (all p<0.01). PDAC patients demonstrated nearly two times higher levels of ALP (median 162.5 vs 86) and TBIL (median 14.5 vs 8.0) than PnC patients (Supplementary Table 6). Unsurprisingly, being a known pancreatic cancer biomarker, CA19-9 levels were very high among PDAC patients (median 294.0, IQR 63.8-1144.0) compared to the PnC group (median 15.9, IQR 9.0-35.0, p<0.001) (Supplementary Table 6).

Observational association study

The derivation dataset was used to inspect the association between study variables and odds of PDAC in comparison to PnC (Figure 2). The dataset consisted of 258 PDAC patients and 270 PnC patients. The following predictor variables were excluded from the analyses due to missing data in more than 25% patients: AMY (77.6%), AST (62.4%), CA19-9 (46.6%), CA (40.1%), and CRP (31.4%). After controlling for demographic variables in the regression models, no comorbidity or lifestyle variables appeared to be independently associated with PDAC compared to PnC. Increased odds of PDAC was observed for increasing age group (OR between 3.6 and 10.1; p<0.001), jaundice and weight loss (nearly 3-fold; p<0.001). Compared to the normal levels of blood test results, increased odds of PDAC diagnosis were significantly associated with elevated levels of ALP (OR 2.9 [1.5-5.5]; p=0.003), ALT (OR 2.6 [1.5-4.4]; p=0.002), and TBIL (OR 4.8 [2.9-8.1]; p<0.001), and reduced levels of LYMP (OR 1.9 [1.2-3.0]; p=0.008), CREA (OR 3.1 [1.4-6.6]; p=0.008) and NA (OR 4.2 [1.6-10.6]; p=0.006). Unadjusted odds ratios are reported in Supplementary Figure 1. There was also evidence to suggest that PDAC risk associated with several predictors may vary with lifestyle or comorbidity status, particularly recent weight loss appearing to be a significant indicator of PDAC in hypertensive patients (OR 3.8 [1.6-8.9]; p=0.005) (Supplementary Table 7).

• Download figure
• Open in new tab

Figure 2.

Forest plot showing association between study variables and odds of PDAC in comparison to non-malignant pancreatic disease group. The odds ratio (OR) and 95% confidence interval (CI) are derived from logistic regression model controlled for gender, ethnicity and age group. The reported p values are corrected for multiple testing via Benjamini-Hochberg method.

Prediction models for differential diagnosis of PDAC

Statistically significant predictor variables and interactions from the observational association study were included to develop the prediction models under different algorithms: age group, jaundice, weight loss, ALP, ALT, CREA, LYMP, NA, TBIL, and hypertension – weight loss interaction.

The performances of the final models under different algorithms were checked in the multiply imputed validation dataset comprising of unique 86 PDAC cases and 90 PnC patients. The AUROC curves of all prediction models were identical, and significantly better than the null hypothesis area of 0.50, ranging between 0.88 and 0.89. (Table 1). However, the simplest LR model with no penalty function had the best calibration statistic (Spiegelhalter’s z=-1.25, p=0.21) (Table 1). Hence, LR model was selected as our final model (Table 2) indicating good differentiation of PDAC from the PnC cases whilst having close correspondence between predicted PDAC risk and observed outcome.

The maximum F₂ score from the LR model applied to full dataset was obtained for probability threshold at 0.17, in which case 79.7% (n=561 of 704) of the study population would undergo urgent referral with a PDAC sensitivity of 95.6%. With a biomarker panel of 87.5% sensitivity²⁶ applied, this could lead to early detection of around 83.7% (n=288 of 344) of PDAC patients (Figure 3). For a higher probability threshold of 0.40 from maximum F₁ score, 54.1% (n=381) of the population would undergo urgent referral with early detection rate of 72%. Table 3 presents the sensitivity, specificity and positive predictive value for different probability cut-offs obtained from maximum F₁ and F₂ scores, and comparison with current National Institute for Health and Care Excellence (NICE) referral guidelines for suspected pancreatic cancer through specialist appointment within two weeks or urgent imaging within two weeks.²⁷

• Download figure
• Open in new tab

Figure 3.

A performance measure graph with precision, recall, F₁ and F₂ of the logistic regression model with no penalty function with the resultant performance of the model at maximum F₁ and F₂ scores on the full dataset. The resultant percentage of early PDAC detection includes the use of a 87.5% sensitive biomarker.