Machine learning based preoperative analytics for the prediction of anastomotic insufficiency in colorectal surgery: a single-centre pilot study
================================================================================================================================================

* Stephanie Taha-Mehlitz
* Larissa Wentzler
* Fiorenzo Angehrn
* Ahmad Hendie
* Vincent Ochs
* Victor E. Staartjes
* Markus von Flüe
* Anas Taha
* Daniel Steinemann

## Abstract

**Introduction** Anastomotic insufficiency (AI) is a relatively common but grave complication after colorectal surgery. This study aims to determine whether AI can be predicted from simple preoperative data using machine learning (ML) algorithms.

**Methods and analysis** In this retrospective analysis, patients undergoing colorectal surgery with creation of a bowel anastomosis from the University Hospital of Basel were included. Data was split into a training set (80%) and a test set (20%). The group of patients with AI was oversampled to a ratio of 50:50 in the training set and missing values were imputed. Known predictors of AI were included as inputs: age, BMI, smoking status, the Charlson Comorbidity Index, the American Society of Anesthesiologists score, type of operation, indication, haemoglobin and albumin levels, and renal function.

**Results** Of the 593 included patients, 88 experienced AI. At internal validation on unseen patients from the test set, area under the curve (AUC) was 0.61 (95% confidence interval [CI]: 0.44-0.79), calibration slope was 0.16 (95% CI: −0.06-0.39) and calibration intercept was 0.06 (95% CI: 0.02-0.11). We observed a specificity of 0.67 (95% CI: 0.58-0.76), sensitivity of 0.36 (95% CI: 0.08-0.67), and accuracy of 0.64 (95% CI: 0.55-0.72).

**Conclusion** By using 10 patient-related risk factors associated with AI, we demonstrate the feasibility of ML-based prediction of AI after colorectal surgery. Nevertheless, it is crucial to include multicenter data and higher sample sizes to develop a robust and generalisable model, which will subsequently allow for deployment of the algorithm in a web-based application.

**Strengths and limitations of this study**

*   To the best of our knowledge, this is the first study to establish a risk prediction model for anastomotic insufficiency in a perioperative setting in colon surgery.

*   Data from all patients that underwent colon surgery within 8 years at University Hospital Basel were included.

*   We evaluated the feasibility of developing a machine learning model that predicts the outcome by using well-known risk factors for anastomotic insufficiency.

*   Although our model showed promising results, it is crucial to validate our findings externally before clinical practice implications are possible.

Keywords
*   anastomotic insufficiency
*   anastomotic leakage
*   machine learning
*   colorectal surgery
*   prediction tool
*   prediction of anastomotic insufficiency

## Introduction

Anastomotic insufficiency (AI) is a severe complication following gastrointestinal surgery1. According to Rahbari et al.2, AI is defined as a defect at the anastomotic site which leads to a connection between intra- and extraluminal compartments. AI is itself considered to be an independent risk factor for adverse clinical and oncological outcomes like decreased survival of cancer patients and increased readmission rates after surgery3–5. The approximated incidence of AI is 3.3% after colon anastomosis and 8.6% after colorectal anastomosis in specialised centres4. However, these rates are likely considerably higher in centres lacking dedicated colorectal surgery teams and after emergency surgery. In fact, depending on diagnostic criteria, AI rates of over 10% have been reported in the literature6–10. Hospital stay is extended by twelve days on average, and healthcare-related expenses are increased by up to 30.000 USD in patients who experience AI1,2.

In previous publications, a multitude of risk factors for AI has been identified11,12. Among these risk factors, age13, gender7,14–19, body mass index (BMI)20,21, Charlson Comorbidity Index (CCI)14,22, American Society of Anesthesiologists (ASA) score13,21–24, leukocytosis23, anaemia25, hypoalbuminemia15,26,27, steroid use28,29, renal function23, previous abdominal surgery16, active smoking6,30,31, and alcohol abuse6,32, liver metastasis16, indication16,33, type of surgery33, emergency surgery34, surgical approach35, anastomotic technique36,37, and defunctioning ileostomy33,35 were identified more or less consistently. Integrating all of these risk factors into one holistic clinical prediction of AI is a very challenging task, even for experienced physicians. Indeed, even experienced surgeons were reported to systematically underestimate the risk of AI by clinical assessment38. Undoubtedly, the ability to preoperatively predict AI precisely would allow for better resource allocation and enhanced patient preparation, and an improved patient-physician relationship due to the improved quality of informed consent.

Machine learning (ML) algorithms can be exceptionally competent at integrating various patient variables into a unified risk model, which is able to generate predictions specifically for each patient. Development and rigorous validation of clinical prediction models require large amounts of multicentre data as well as external validation. Before embarking on said multicentre data collection, piloting a modelling strategy to assess feasibility and to identify the most valuable inputs is crucial. Consequently, this pilot study with data from a single centre aims to assess whether AI can be predicted from simple patient-related risk factors.

## Methods

### Overview and Data collection

Data was extracted retrospectively from the patient registry of the University Hospital of Basel. Patients who underwent colon anastomosis for various reasons including neoplasia, diverticulitis, ischaemia, iatrogenic or traumatic perforation, or inflammatory bowel disease between 1st of January 2012 and 31st of December 2019 were eligible. This study was completed based on the transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) statement checklist for the development of clinical prediction models39. Utilizing the aforementioned data, we developed ML models with the aim of predicting AI, and internally validated the models on held-out test data.

### Ethical Considerations

Ethical approval was provided for this study by the Northwestern and Central Ethics Committee Switzerland (BASEC-Nr 2020-02265).

### Predictors and Outcome Measures

AI was defined according to Gessler et al.40 and Rahbari et al.2 as any clinical sign of leakage, confirmed by radiological examination, endoscopy, clinical examination of the anastomosis, or upon reoperation.

Recorded variables included risk factors that already have been reported in literature such as age, gender, BMI, active smoking, alcohol abuse (>2 alcoholic beverages per day), prior abdominal surgery, preoperative leucocytosis (≥10.000 per mm3), preoperative steroid use, CCI, ASA Score, renal function (CKD Stages G1 to G5), albumin (g/dl), and haemoglobin level (g/dl), liver metastasis, indication, type of surgery, emergency surgery, surgical approach (laparoscopic or open), anastomotic technique (hand-sewn or stapler) and defunctioning ileostomy.

### Model Development

Continuous variables and categorical variables were recorded as mean ± standard deviation (SD). Moreover, categorical variables as decimal numbers instead of percentages. Python 3.6.9 was used to perform all analyses. The Keras library41 was used for the artificial neural networks, the XGBoost library42 was used for XGBs, and the scikit-learn library43 was used for all other architectures.

Data was randomly split into two sets: 80% of the data was used as training data, while the remaining 20% was utilized as a test set to validate the models internally. Initially, the two sets had similar class distribution. However, the training set’s minority class (patients with AI) was oversampled using random oversampling until a 50:50 class distribution was achieved to prevent the models from overpredicting the majority class due to class imbalance44,45. Recursive feature elimination was applied to determine the optimum set of relevant features predicting the outcome in the training data set46.

A *k*-nearest neighbor (KNN) imputer was co-trained on the pre-oversampling training data to impute any missing data47. We trialled multiple model architectures, including generalised linear models (GLMs), support vector machines (SVMs), naïve Bayes classifiers, random forests, extreme gradient boosting machines (XGB), and artificial neural networks. Training occurred in repeated 5-fold cross validation with 10 repetitions to select the optimal model architecture and hyperparameters based on the area under the receiver operating characteristics curve (AUC). Recalibration was applied using Platt’s method48 based on the pre-oversampling training set, and the cut-off for binary classification was selected based on AUC on the training set using the closest-to-(0,1)-criterion49.

The models were subsequently internally validated using the held-out test set, with ninety-five percent confidence intervals (Cis) being generated using 1000 bootstrap repetitions.

## Results

### Cohort

In the training set, a total of 474 patients were included in the training set, of which 77 (16.2%) suffered from AI. The mean age was 64.6 ± 15.4. Furthermore, 225 (47.5%) patients were male and 249 (52.5%) were female. The mean BMI was 25.1± 5.3 and 152 (32.1%) patients were active smokers. **Table 1** provides an overview of the cohorts.

View this table:
[TABLE 1.](http://medrxiv.org/content/early/2022/01/13/2021.12.11.21267569/T1)

TABLE 1. 
Overview of patient characteristics including missingness.

As for the test set, a total of 119 patients were included, of which 11 (9.2%) suffered from AI. Furthermore, 58 (48.8%) patients were male, and 61 (51.2%) were female. The mean BMI in the test set was 25.0±5.1 and 30 (25.2%) patients were active smokers.

### Model Performance

The performance of all six models was assessed by internal validation using the test data set. This performance is reported in detail along with resampled training performance in **Table 2**. The random forest provided the highest AUC with a value of 0.61 (95% CI: 0.44 – 0.79) while also producing acceptable probabilities based on the calibration slope [0.16 (95% CI: −0.06 – 0.39)], and intercept [0.06 (95% CI: 0.02 - 0.11)]. Moreover, the random forest exhibited a sensitivity of 0.36 (95% CI: 0.08 – 0.67), specificity of 0.67 (95% CI: 0.58 – 0.76) and accuracy of 0.64 (95% CI: 0.55 – 0.72). Specific feature importance within the random forest is displayed in **Table 3**. The calibration and AUC curves for the random forest classifier are displayed in **Figures 1** and **2**.

View this table:
[TABLE 2.](http://medrxiv.org/content/early/2022/01/13/2021.12.11.21267569/T2)

TABLE 2. 
Performance evaluation of the final random forest classifier. Resampled training performance and test set performance are reported.

View this table:
[TABLE 3.](http://medrxiv.org/content/early/2022/01/13/2021.12.11.21267569/T3)

TABLE 3. 
Predictor importance according to random forest classifier.

![FIGURE 1.](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2022/01/13/2021.12.11.21267569/F1.medium.gif)

[FIGURE 1.](http://medrxiv.org/content/early/2022/01/13/2021.12.11.21267569/F1)

FIGURE 1. 
Resampled training (**a**) and test (**b**) set calibration curves of the random forest classifier.

![FIGURE 2.](http://medrxiv.org/http://medrxiv.stage.highwire.org/content/medrxiv/early/2022/01/13/2021.12.11.21267569/F2.medium.gif)

[FIGURE 2.](http://medrxiv.org/content/early/2022/01/13/2021.12.11.21267569/F2)

FIGURE 2. 
Resampled training (**a**) and test (**b**) set area under the receiver operating characteristics curves (AUC) of the random forest classifier.

## Discussion

In a pilot study using data from a single reference centre for colorectal surgery, we assess the feasibility of predicting AI accurately from simple data using ML techniques. Our findings demonstrate that modelling AI to a certain extent is feasible and identifies the most important input variables, laying the groundwork for more extensive multicentre modelling and external validation.

Even though a plethora of studies have analysed risk factors for AI, up to this day, no reliable clinical prediction model for AI has been established11–37. To the best of the authors’ knowledge, this study is the first one attempting to solve the classification problem of AI (‘*will my patient suffer from AI after colorectal surgery*?’) by application of ML algorithms.

There is a widespread misunderstanding that variable importance measures gleaned from clinical prediction models can discover correlations and causalities in the same way that explanatory modelling does (prediction versus explanation)50. Indeed, this common misconceptualisation exists because predictive and explanatory modelling are often not as explicitly distinguished as attempted here in this study. Indeed, the interchangeable use of the concepts of *in-sample correlation* and *out-of-sample generalisation* can lead to false clinical decision making51. While those variables identified as having high feature importance in this study may indeed be the most crucial ones for precise and generalisable prediction of AI, it cannot safely be concluded that these variables are necessarily also important independent risk factors for AI in their own right.

Another separate question is the initial choice of input variables for clinical prediction modelling, which can be achieved in various ways52: In any case, a balance between performance through the inclusion of many variables and between the goal of arriving at parsimonious models that truly generalise needs to be struck. The choice of variables for this study was focused on patient-related risk factors to minimize the statistical noise from differing standard procedures in distinct clinical centres. The following section aims to clarify why the risk factors chosen have not only been correlated to AI previously but are also part of plausible pathomechanisms.

The CCI and the ASA classification system are two commonly used scoring systems assessing the patients’ pre-anaesthesia medical comorbidities and expected mortality risk, respectively. Even though the ability to predict peri- and postoperative complications is claimed for both scores, the ASA score and the CCI give valuable insight into the patients’ general medical conditions and thus can be used to generate more specific scoring systems47,48. Indeed, the CCI and ASA score represent the most important features in our final model.

Serum levels of albumin and other hepatic proteins have been established as easily obtainable laboratory parameters for malnourishment53. Since malnourishment itself is a risk factor for impaired wound healing in general54, it is conceivable that hypoalbuminemia could be factor predisposing patients for AI. In fact, hypoalbuminemia has already been linked to increased postoperative morbidity and mortality in patients with colon cancer27,55.

The most important pathophysiological basis for AI is believed to be gastrointestinal ischaemia, decreasing the amount of oxygen available to the recovering intestinal wall56. Consequently, the intestinal tissue becomes more dysfunctional, which may lead to delayed wound healing. These suspected pathophysiological processes align well with the factors determined relevant for the prediction of AI risk in our model. In a similar way, haemoglobin levels determine the blood’s capacity to supply peripheral tissues with oxygen57,58.

Smoking has been consistently evaluated as a risk factor for AI15,31. Furthermore, it is well established that tobacco smoking is among the top risk factors for vascular disease59, making it a logical contributor to vascular insufficiency leading to subsequently impaired wound healing and postoperative AI risk.

Our model seems to be more specific than sensitive. This indicates that our model is proficient at ruling in high-risk AI with a positive prediction. Conversely, our model is less precise in ruling out low-risk AI. However, a rule-in model could prove to be of great value for clinicians by simply identifying the very high-risk group. Furthermore, the random forest model seems fairly well-calibrated. Well-calibrated predicted probabilities are arguable more important in clinical practice (“How likely is it that I am going to experience AI?’ – ‘Your probability is 17%.’) instead of binary predictions (‘Am I going to suffer from AI?’ – ‘The model predicts yes/no’). Physicians are experts at dealing with uncertainty and risks, and probabilities are thus more appreciated by patients and physicians than a mere yes or no answer – apart from the fact that patients are never binary but instead represent a spectrum of risk60.

Another difficulty in clinical prediction modelling is choosing the appropriate sample size. According to a common rule of thumb, there should be at least 10 minority class observations in a dataset per feature61. This study relies on 10 patient-related risk factors, thus making a total number of 1000 patients with AI who would be necessary for training, while we finally were able to include 88 patients with AI. Other architectures such as random forests, artificial neural networks, and SVMs seem to require much more data per feature62. Therefore, it is conceivable that including more patients might further refine the model. Consequently, it is crucial to recruit more patient data from other hospitals in a multicentre study.

Clinical prediction models can facilitate assessing individual risks and make more informed decisions based on predictive analytics that are tailored to each patient. However, especially in colorectal surgery, the indication for surgery is only rarely truly elective. Therefore, a prediction model can only help decide whether an intervention should be postponed to improve the risk profile or, especially for emergency interventions, whether a patient would benefit from a diverting stoma to minimize and modify risk factors before re-joining the colon. On the other hand, a comprehensive predictive model may also increase a patient’s acceptance of the primary placement of a protective stoma. Thus, such a model could potentially also help to improve the physician-patient relationship through enhanced patient education.

The results of a predictive model cannot be seen as a clear recommendation pro or contra an intervention as the risk profile it mirrors is tailored only to a specific endpoint and thus does not entirely reflect the patient’s global situation. Indeed, components of decision-making such as the psychological distress of a patient with chronic diverticulitis are not included in the model and have a decisive influence on the indication. Consequently, prediction models should be seen only as adjunctive information to be used in a complementary way for informed shared decision-making. Nevertheless, the necessity for evidence-based clinical prediction models becomes clear when considering the relative inability of even experienced clinicians in predicting clinical outcomes38, while the ethical implications of an ‘AI doctor’ technology independent from human control have to be taken into account, too63. Consequently, ML-based clinical prediction models could be deemed a contemporary optimal trade-off between the clinical experience of human experts and the exploitation of big data by learning algorithms.

### Limitations

The relatively high AI rate of 14.8% in our cohort can be seen as a limitation. Similarly, the difference in AI incidence among training versus test set represents an additional hurdle that is realistic, as AI rate is described inconsistently in the literature4,6–10. The patient population at the University Hospital Basel with 32.9% emergency cases and a cohort that includes transplanted and immunosuppressed patients is expected to have higher complication rates34. Nevertheless, such a difference to other hospitals should be reflected in the ASA score, the CCI, and blood values and thus also in our results. By including patient data from other institutions in future analyses, this number will be balanced out, and a differentiated breakdown according to emergency interventions, immunosuppressant use, previous radio-/chemotherapy, and cancer diagnosis, which additionally reflect a patient’s health status, is conceivable and could be implemented in our ML-algorithm.

One further caveat of any model is the danger of overfitting. In the context of clinical prediction modelling, overfitting means that an algorithm adheres too strictly to the training data, especially its inherent variance and possible noise factors (e.g. noise generated by a hospital’s standardized procedures). With enough training, the algorithm will perform extremely well on the training data while losing its generalisation capability towards new data from other centres. Indeed, it is not unlikely that this study might suffer from slight overfitting due to standardized hospital procedures because the data was exclusively collected in one clinical centre. However, this weakness could be addressed by recruiting more patient data from other hospitals. Lastly, we have purposefully not yet deployed our model for clinical application in, e.g. a web-app, as any single-centre and not yet externally validated clinical prediction model is not yet recommended for clinical use64.

## Conclusion

In this pilot study, by using 10 patient-related risk factors associated with AI, we demonstrate the feasibility of ML-based prediction of AI after colorectal surgery. Nevertheless, it is crucial to include multicentre data and higher sample sizes to develop a robust and generalisable model, which will subsequently allow for deployment of the algorithm in a web-based application.

## Data Availability

All data produced in the present study are available upon reasonable request to the authors.

## Author Contributions

Conceptualization A.T., V.S.; data collection L.W., F.A., S.T-M.; administration and ethics S.T.-M., A.T., F.A., D.S.; analysis A.H., V.S., A.T., V.O.; visualization A.H., S.T.-M.; writing—original draft preparation A.T., S.T.-M., V.O., A.H.; writing—review and editing V.S., F.A., D.S., M.v.F.; All authors have read and agreed to the published version of the manuscript.

## Footnotes

*   **Sources of funding:** This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

*   **Previous communications:** This manuscript has been submitted to medRxiv for preprint.

*   **Ethical approval:** The study was approved by the Northwestern and Central Ethics Committee Switzerland (BASEC-Nr 2020-02265).

*   **Data Availability Statement:** The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

*   **Conflicts of Interest:** The authors declare no conflicts of interest.

*   This version of the manuscript has been revised for a new submission.

*   Received December 11, 2021.
*   Revision received January 13, 2022.
*   Accepted January 13, 2022.


*   © 2022, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution-NonCommercial 4.0 International), CC BY-NC 4.0, as described at [http://creativecommons.org/licenses/by-nc/4.0/](http://creativecommons.org/licenses/by-nc/4.0/)

## References

1.  1.Lee SW, Gregory D, Cool CL (2020) Clinical and economic burden of colorectal and bariatric anastomotic leaks. Surg Endosc 34:4374–4381
    
    

2.  2.Rahbari NN, Weitz J, Hohenberger W, et al. (2010) Definition and grading of anastomotic leakage following anterior resection of the rectum: a proposal by the International Study Group of Rectal Cancer. Surgery 147:339–351
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.surg.2009.10.012&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20004450&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

3.  3.Mirnezami A, Mirnezami R, Chandrakumaran K, Sasapu K, Sagar P, Finan P (2011) Increased local recurrence and reduced survival from colorectal cancer following anastomotic leak: systematic review and meta-analysis. Ann Surg 253:890–899
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/SLA.0b013e3182128929&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21394013&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000289510600009&link_type=ISI) 

4.  4.Krell RW, Girotti ME, Fritze D, Campbell DA, Hendren S (2013) Hospital readmissions after colectomy: a population-based study. J Am Coll Surg 217:1070–1079
    
    

5.  5.Hammond J, Lim S, Wan Y, Gao X, Patkar A (2014) The burden of gastrointestinal anastomotic leaks: an evaluation of clinical and economic outcomes. J Gastrointest Surg 18:1176–1185
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11605-014-2506-4&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24671472&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

6.  6.Sørensen LT, Jørgensen T, Kirkeby LT, Skovdal J, Vennits B, Wille-Jørgensen P (1999) Smoking and alcohol abuse are major risk factors for anastomotic leakage in colorectal surgery. Br J Surg 86:927–931
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1046/j.1365-2168.1999.01165.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10417567&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000081445700015&link_type=ISI) 

7.  7.Law WI, Chu KW, Ho JW, Chan CW (2000) Risk factors for anastomotic leakage after low anterior resection with total mesorectal excision. Am J Surg 179:92–96
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0002-9610(00)00252-X&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10773140&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

8.  8.Kang CY, Halabi WJ, Chaudhry OO, Nguyen V, Pigazzi A, Carmichael JC, Mills S, Stamos MJ (2013) Risk factors for anastomotic leakage after anterior resection for rectal cancer. JAMA Surg 148:65–71
    
    

9.  9.Eberl T, Jagoditsch M, Klingler A, Tschmelitsch J (2008) Risk factors for anastomotic leakage after resection for rectal cancer. Am J Surg 196:592–598
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.amjsurg.2007.10.023&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18571620&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000259910300029&link_type=ISI) 

10. 10.Kruschewski M, Rieger H, Pohlen U, Hotz HG, Buhr HJ (2007) Risk factors for clinical anastomotic leakage and postoperative mortality in elective surgery for rectal cancer. Int J Colorectal Dis 22:919–927
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00384-006-0260-0&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17260142&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

11. 11.Komen N, Dijk J-W, Lalmahomed Z, Klop K, Hop W, Kleinrensink G-J, Jeekel H, Ruud Schouten W, Lange JF (2009) After-hours colorectal surgery: a risk factor for anastomotic leakage. Int J Colorectal Dis 24:789–795
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00384-009-0692-4&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19301016&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000266583100008&link_type=ISI) 

12. 12.Kryzauskas M, Bausys A, Degutyte AE, Abeciunas V, Poskus E, Bausys R, Dulskas A, Strupas K, Poskus T (2020) Risk factors for anastomotic leakage and its impact on long-term survival in left-sided colorectal cancer surgery. World J Surg Oncol 18:205
    
    

13. 13.Bakker IS, Grossmann I, Henneman D, Havenga K, Wiggers T (2014) Risk factors for anastomotic leakage and leak-related mortality after colonic cancer surgery in a nationwide audit. Br J Surg 101:424–32; discussion 432
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/bjs.9395&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24536013&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

14. 14.Trencheva K, Morrissey KP, Wells M, Mancuso CA, Lee SW, Sonoda T, Michelassi F, Charlson ME, Milsom JW (2013) Identifying important predictors for anastomotic leak after colon and rectal resection: prospective study on 616 patients. Ann Surg 257:108–113
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/SLA.0b013e318262a6cd&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22968068&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

15. 15.Suding P, Jensen E, Abramson MA, Itani K, Wilson SE (2008) Definitive risk factors for anastomotic leaks in elective open colorectal resection. Arch Surg 143:907–11; discussion 911–2
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/archsurg.143.9.907&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18794430&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000259089200024&link_type=ISI) 

16. 16.Lipska MA, Bissett IP, Parry BR, Merrie AEH (2006) Anastomotic leakage after lower gastrointestinal anastomosis: men are at a higher risk. ANZ J Surg 76:579–585
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1445-2197.2006.03780.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16813622&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000238485000009&link_type=ISI) 

17. 17.Park JS, Choi G-S, Kim SH, et al. (2013) Multicenter analysis of risk factors for anastomotic leakage after laparoscopic rectal cancer excision: the Korean laparoscopic colorectal surgery study group. Ann Surg 257:665–671
    
    

18. 18.Hamabe A, Ito M, Nishigori H, Nishizawa Y, Sasaki T (2018) Preventive effect of diverting stoma on anastomotic leakage after laparoscopic low anterior resection with double stapling technique reconstruction applied based on risk stratification. Asian J Endosc Surg 11:220–226
    
    

19. 19.Tanaka K, Okuda J, Yamamoto S, Ito M, Sakamoto K, Kokuba Y, Yoshimura K, Watanabe M (2017) Risk factors for anastomotic leakage after laparoscopic surgery with the double stapling technique for stage 0/I rectal carcinoma: a subgroup analysis of a multicenter, single-arm phase II trial. Surg Today 47:1215–1222
    
    

20. 20.Silva-Velazco J, Stocchi L, Costedio M, Gorgun E, Kessler H, Remzi FH (2016) Is there anything we can modify among factors associated with morbidity following elective laparoscopic sigmoidectomy for diverticulitis? Surg Endosc 30:3541–3551
    
    

21. 21.Buchs NC, Gervaz P, Secic M, Bucher P, Mugnier-Konrad B, Morel P (2008) Incidence, consequences, and risk factors for anastomotic dehiscence after colorectal surgery: a prospective monocentric study. Int J Colorectal Dis 23:265–270
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00384-007-0399-3&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=18034250&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000252482700006&link_type=ISI) 

22. 22.Tan WP, Talbott VA, Leong QQ, Isenberg GA, Goldstein SD (2013) American Society of Anesthesiologists class and Charlson’s Comorbidity index as predictors of postoperative colorectal anastomotic leak: a single-institution experience. J Surg Res 184:115–119
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jss.2013.05.039&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23830360&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

23. 23.Alves A, Panis Y, Trancart D, Regimbeau J-M, Pocard M, Valleur P (2002) Factors associated with clinically significant anastomotic leakage after large bowel resection: multivariate analysis of 707 patients. World J Surg 26:499–502
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00268-001-0256-4&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11910487&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000174873200019&link_type=ISI) 

24. 24.Choi H-K, Law W-L, Ho JWC (2006) Leakage after resection and intraperitoneal anastomosis for colorectal malignancy: analysis of risk factors. Dis Colon Rectum 49:1719–1725
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s10350-006-0703-2&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=17051321&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000242029300007&link_type=ISI) 

25. 25.Hayden DM, Mora Pinzon MC, Francescatti AB, Saclarides TJ (2015) Patient factors may predict anastomotic complications after rectal cancer surgery: Anastomotic complications in rectal cancer. Ann Med Surg (Lond) 4:11–16
    
    

26. 26.Ionescu D, Tibrea C, Puia C (2013) Pre-operative hypoalbuminemia in colorectal cancer patients undergoing elective surgery—a major risk factor for postoperative outcome. Chirurgia 108:822–828
    
    

27. 27.Lai C-C, You J-F, Yeh C-Y, Chen J-S, Tang R, Wang J-Y, Chin C-C (2011) Low preoperative serum albumin in colon cancer: a risk factor for poor outcome. Int J Colorectal Dis 26:473–481
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00384-010-1113-4&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21190025&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

28. 28.Slieker JC, Komen N, Mannaerts GH, Karsten TM, Willemsen P, Murawska M, Jeekel J, Lange JF (2012) Long-term and perioperative corticosteroids in anastomotic leakage: a prospective study of 259 left-sided colorectal anastomoses. Arch Surg 147:447–452
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/archsurg.2011.1690&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22249852&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

29. 29.Konishi T, Watanabe T, Kishimoto J, Nagawa H (2006) Risk factors for anastomotic leakage after surgery for colorectal cancer: results of prospective surveillance. J Am Coll Surg 202:439–444
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jamcollsurg.2005.10.019&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16500248&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000236233800009&link_type=ISI) 

30. 30.Kim MJ, Shin R, Oh H-K, Park JW, Jeong S-Y, Park J-G (2011) The impact of heavy smoking on anastomotic leakage and stricture after low anterior resection in rectal cancer patients. World J Surg 35:2806–2810
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21959931&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

31. 31.Kwak HD, Kim S-H, Kang DW, Baek S-J, Kwak JM, Kim J (2017) Risk Factors and Oncologic Outcomes of Anastomosis Leakage After Laparoscopic Right Colectomy. Surg Laparosc Endosc Percutan Tech 27:440–444
    
    

32. 32.Mäkelä JT, Kiviniemi H, Laitinen S (2003) Risk factors for anastomotic leakage after left-sided colorectal resection with rectal anastomosis. Dis Colon Rectum 46:653–660
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s10350-004-6627-9&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=12792443&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000182871800015&link_type=ISI) 

33. 33.Peeters KCMJ, Tollenaar RAEM, Marijnen CAM, Klein Kranenbarg E, Steup WH, Wiggers T, Rutten HJ, van de Velde CJH, Dutch Colorectal Cancer Group (2005) Risk factors for anastomotic failure after total mesorectal excision of rectal cancer. Br J Surg 92:211–216
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/bjs.4806&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15584062&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000227141700015&link_type=ISI) 

34. 34.McDermott FD, Heeney A, Kelly ME, Steele RJ, Carlson GL, Winter DC (2015) Systematic review of preoperative, intraoperative and postoperative risk factors for colorectal anastomotic leaks. Br J Surg 102:462–479
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/bjs.9697&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25703524&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

35. 35.Zheng H, Wu Z, Wu Y, Mo S, Dai W, Liu F, Xu Y, Cai S (2019) Laparoscopic surgery may decrease the risk of clinical anastomotic leakage and a nomogram to predict anastomotic leakage after anterior resection for rectal cancer. Int J Colorectal Dis 34:319–328
    
    

36. 36.Naumann DN, Bhangu A, Kelly M, Bowley DM (2015) Stapled versus handsewn intestinal anastomosis in emergency laparotomy: a systemic review and meta-analysis. Surgery 157:609–618
    
    

37. 37.Choy PYG, Bissett IP, Docherty JG, Parry BR, Merrie A, Fitzgerald A (2011) Stapled versus handsewn methods for ileocolic anastomoses. Cochrane Database Syst Rev CD004320
    
    

38. 38.Karliczek A, Harlaar NJ, Zeebregts CJ, Wiggers T, Baas PC, van Dam GM (2009) Surgeons lack predictive accuracy for anastomotic leakage in gastrointestinal surgery. Int J Colorectal Dis 24:569–576
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00384-009-0658-6&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19221768&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

39. 39.Collins GS, Reitsma JB, Altman DG, Moons KGM (2015) Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): the TRIPOD Statement. Br J Surg 102:148–158
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/bjs.9736&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25627261&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

40. 40.Gessler B, Eriksson O, Angenete E (2017) Diagnosis, treatment, and consequences of anastomotic leakage in colorectal surgery. Int J Colorectal Dis 32:549–556
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s00384-016-2744-x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=28070659&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

41. 41.Chollet F (2015) others,”Keras”, GitHub.
    
    

42. 42.Chen T, Guestrin C (2016) XGBoost: A Scalable Tree Boosting System. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, New York, NY, USA, pp 785–794
    
    

43. 43.Pedregosa F, Varoquaux G, Gramfort A, et al. (2011) Scikit-learn: Machine learning in Python. the Journal of machine Learning research 12:2825–2830
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000298103200003&link_type=ISI) 

44. 44.Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: Synthetic Minority Over-sampling Technique. J Artif Intell Res 16:321–357
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1613/jair.953&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=WOS:00017602&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

45. 45.Staartjes VE, Schröder ML (2018) Letter to the Editor. Class imbalance in machine learning for neurosurgical outcome prediction: are our models valid? J Neurosurg Spine 29:611–612
    
    

46. 46.Guyon I, Weston J, Barnhill S, Vapnik V (2002) Gene Selection for Cancer Classification using Support Vector Machines. Mach Learn 46:389–422
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1023/A:1012487302797&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000171501800018&link_type=ISI) 

47. 47.Templ M, Alfons A, Kowarik A, Prantner B (2019) VIM: Visualization and Imputation of Missing Values, 2012. R package version 1:
    
    

48. 48.Staartjes VE, Regli L, Serra C (eds) (2021) Machine learning in clinical neuroscience: Foundations and applications, 1st ed. Springer Nature, Cham, Switzerland
    
    

49. 49.Perkins NJ, Schisterman EF (2006) The Inconsistency of “Optimal” Cutpoints Obtained using Two Criteria based on the Receiver Operating Characteristic Curve. American Journal of Epidemiology 163:670–675
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/aje/kwj063&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16410346&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000236250900011&link_type=ISI) 

50. 50.Shmueli G (2010) To explain or to predict? Stat Sci 25:289–310
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1214/10-STS330&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000286550700002&link_type=ISI) 

51. 51.Yarkoni T, Westfall J (2017) Choosing Prediction Over Explanation in Psychology: Lessons From Machine Learning. Perspect Psychol Sci 12:1100–1122
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/1745691617693393&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=28841086&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

52. 52.Chowdhury MZI, Turin TC (2020) Variable selection strategies and its importance in clinical prediction modelling. Family Medicine and Community Health 8:e000262
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiZm1jaCI7czo1OiJyZXNpZCI7czoxMToiOC8xL2UwMDAyNjIiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMi8wMS8xMy8yMDIxLjEyLjExLjIxMjY3NTY5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

53. 53.Fuhrman MP, Charney P, Mueller CM (2004) Hepatic proteins and nutrition assessment. J Am Diet Assoc 104:1258–1264
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jada.2004.05.213&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15281044&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000223004300014&link_type=ISI) 

54. 54.Guo S, Dipietro LA (2010) Factors affecting wound healing. J Dent Res 89:219–229
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0022034509359125&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20139336&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

55. 55.Kwag S-J, Kim J-G, Kang W-K, Lee J-K, Oh S-T (2014) The nutritional risk is a independent factor for postoperative morbidity in surgery for colorectal cancer. Ann Surg Treat Res 86:206–211
    
    

56. 56.Rutegård M, Rutegård J (2015) Anastomotic leakage in rectal cancer surgery: The role of blood perfusion. World J Gastrointest Surg 7:289–292
    
    

57. 57.Yudin J, Verhovsek M (2019) How we diagnose and manage altered oxygen affinity hemoglobin variants. Am J Hematol 94:597–603
    
    

58. 58.Shepherd JRA, Dominelli PB, Roy TK, Secomb TW, Hoyer JD, Oliveira JL, Joyner MJ (2019) Modelling the relationships between haemoglobin oxygen affinity and the oxygen cascade in humans. J Physiol 597:4193–4202
    
    

59. 59.the Surgeon General O of, Others (2014) The Health Consequences of Smoking-50 Years of Progress: A Report of the Surgeon General. Atlanta, GA: US Department of Health and Human Services, Centers for Disease Control and Prevention (US). National Center for Chronic Disease Prevention and Health Promotion, Office on Smoking and Health. [https://www.surgeongeneral.gov/library/reports/50-years-of-progress/full-report.pdf](https://www.surgeongeneral.gov/library/reports/50-years-of-progress/full-report.pdf)
    
    

60. 60.Staartjes VE, Kernbach JM (2020) Letter to the Editor. Importance of calibration assessment in machine learning– based predictive analytics. J Neurosurg Spine 32:985–987
    
    

61. 61.Peduzzi P, Concato J, Kemper E, Holford TR, Feinstein AR (1996) A simulation study of the number of events per variable in logistic regression analysis. J Clin Epidemiol 49:1373–1379
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0895-4356(96)00236-3&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=8970487&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1996VY92400008&link_type=ISI) 

62. 62.van der Ploeg T, Austin PC, Steyerberg EW (2014) Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints. BMC Med Res Methodol 14:137
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/1471-2288-14-137&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25532820&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2022%2F01%2F13%2F2021.12.11.21267569.atom) 

63. 63.Keskinbora KH (2019) Medical ethics considerations on artificial intelligence. J Clin Neurosci 64:277–282
    
    

64. 64.Staartjes VE, Kernbach JM (2020) Significance of external validation in clinical machine learning: let loose too early? Spine J 20:1159–1160