Data Integrity and Its Impact on Machine Learning Models: A Critical Audit of a Lung Cancer Dataset
DOI:
https://doi.org/10.29207/dq775h27Keywords:
dataset audit, data provenance, duplicate records, class imbalance, reproducibilityAbstract
Subscription-gated IEEE DataPort datasets are difficult to verify before reuse because independent inspection requires access most researchers do not have; we obtained direct access to one such case. We apply a six-stage, three-layer audit (provenance, structural/duplication plausibility, target validity) to “An Anonymized Feature-Based Dataset for Lung Cancer Prediction Using Explainable AI Models,” a 15-feature, binary-symptom-checklist dataset presented as newly compiled to support explainable-AI research. The delivered file contains 2,457 rows, of which 2,132 (86.8%) are exact duplicates of another row, leaving 325 unique records. The dataset’s own hosting page links a peer-reviewed “Research Article” co-authored by two of this dataset’s three listed authors; that 2024 publication states its dataframe “contains 309 entries” with “no null values” — figures that match neither the delivered file’s row count nor its 325 unique records. The same publication’s introduction and methods describe a multi-source study drawing on major imaging and genomic repositories, none of which appear in the delivered file or in the publication’s own reported results, which instead use the same 16-column schema and encoding as a long-circulating public dataset. Within the 325 unique records, duplication is class-dependent: unique cancer-negative rows are repeated 14.1 times on average (up to 86), versus 4.5 times for cancer-positive rows, shifting the delivered file’s class balance relative to the unique data without documentation. Separately, the linked publication reports identical accuracy, precision, recall, F1, and AUC-ROC for five structurally different classifiers, a pattern inconsistent with independent model evaluation.References
[1] S. Makubhai and G. Pathak, “An Anonymized Feature-Based Dataset for Lung Cancer Prediction Using Explainable AI Models,” IEEE DataPort, 2026, doi: 10.21227/sx52-k564.
[2] S. Makubhai, G. R. Pathak, and P. R. Chandre, “Comparative Analysis of Explainable Artificial Intelligence Models for Predicting Lung Cancer Using Diverse Datasets,” IAES International Journal of Artificial Intelligence (IJ-AI), vol. 13, no. 2, pp. 1980–1991, 2024, doi: 10.11591/ijai.v13.i2.pp1980-1991.
[3] A. D. Gibson, N. M. White, G. S. Collins, and A. G. Barnett, “Evidence of Unreliable Data and Poor Data Provenance in Clinical Prediction Model Research and Clinical Practice,” BMC Medicine, vol. 24, article 386, 2026, doi: 10.1186/s12916-026-04981-y.
[4] G. S. Collins, J. B. Reitsma, D. G. Altman, and K. G. M. Moons, “Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD): The TRIPOD Statement,” BMJ, vol. 350, g7594, 2015, doi: 10.1136/bmj.g7594.
[5] T. Gebru, J. Morgenstern, B. Vecchione, J. W. Vaughan, H. Wallach, H. Daumé III, and K. Crawford, “Datasheets for Datasets,” Communications of the ACM, vol. 64, no. 12, pp. 86–92, 2021, doi: 10.1145/3458723.
[6] R. Y. Wang and D. M. Strong, “Beyond Accuracy: What Data Quality Means to Data Consumers,” Journal of Management Information Systems, vol. 12, no. 4, pp. 5–33, 1996, doi: 10.1080/07421222.1996.11518099.
[7] S. Kaufman, S. Rosset, C. Perlich, and O. Stitelman, “Leakage in Data Mining: Formulation, Detection, and Avoidance,” ACM Transactions on Knowledge Discovery from Data, vol. 6, no. 4, article 15, 2012, doi: 10.1145/2382577.2382579.
[8] S. Kapoor and A. Narayanan, “Leakage and the Reproducibility Crisis in Machine-Learning-Based Science,” Patterns, vol. 4, no. 9, article 100804, 2023, doi: 10.1016/j.patter.2023.100804.
[9] M. Baker, “1,500 Scientists Lift the Lid on Reproducibility,” Nature, vol. 533, no. 7604, pp. 452–454, 2016, doi: 10.1038/533452a.
[10] M. R. Munafò, B. A. Nosek, D. V. M. Bishop, K. S. Button, C. D. Chambers, N. Percie du Sert, U. Simonsohn, E.-J. Wagenmakers, J. J. Ware, and J. P. A. Ioannidis, “A Manifesto for Reproducible Science,” Nature Human Behaviour, vol. 1, article 0021, 2017, doi: 10.1038/s41562-016-0021.
[11] J. P. A. Ioannidis, “Why Most Published Research Findings Are False,” PLoS Medicine, vol. 2, no. 8, e124, 2005, doi: 10.1371/journal.pmed.0020124.
[12] M. Mitchell, S. Wu, A. Zaldivar, P. Barnes, L. Vasserman, B. Hutchinson, E. Spitzer, I. D. Raji, and T. Gebru, “Model Cards for Model Reporting,” in Proc. Conf. on Fairness, Accountability, and Transparency (FAT* ’19), Atlanta, GA, USA, 2019, pp. 220–229, doi: 10.1145/3287560.3287596.
[13] I. P. Fellegi and A. B. Sunter, “A Theory for Record Linkage,” Journal of the American Statistical Association, vol. 64, no. 328, pp. 1183–1210, 1969, doi: 10.1080/01621459.1969.10501049.
[14] H. He and E. A. Garcia, “Learning from Imbalanced Data,” IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 9, pp. 1263–1284, 2009, doi: 10.1109/TKDE.2008.239.
[15] T. Saito and M. Rehmsmeier, “The Precision-Recall Plot Is More Informative than the ROC Plot When Evaluating Binary Classifiers on Imbalanced Datasets,” PLOS ONE, vol. 10, no. 3, e0118432, 2015, doi: 10.1371/journal.pone.0118432.
[16] R. Kohavi, “A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection,” in Proc. 14th International Joint Conference on Artificial Intelligence (IJCAI), Montreal, Canada, vol. 2, 1995, pp. 1137–1143.
[17] S. Ahmad Bhat, “Lung Cancer,” Kaggle, 2021. [Online]. Available: https://www.kaggle.com/datasets/mysarahmadbhat/lung-cancer.
[18] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “SMOTE: Synthetic Minority Over-Sampling Technique,” Journal of Artificial Intelligence Research, vol. 16, pp. 321–357, 2002, doi: 10.1613/jair.953.
[19] IEEE, “IEEE Editorial Style Manual for Authors,” IEEE Editorial, 2019.
[20] O. Tur, A. Tur, V. Shabunina, and E. Chernaia, “The IEEE Style: Peculiarities of the Format and Application Prospects,” in Proc. 25th IEEE Int’l Conf. on Problems of Automated Electric Drive. Theory and Practice (PAEP), 2020, doi: 10.1109/PAEP49887.2020.9240790.
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Journal of Systems Engineering and Information Technology (JOSEIT)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under Creative Commons Attribution 4.0 International License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (Refer to The Effect of Open Access).
