Deep learning applications in medical research are often constrained by the lack of data availability due to the significant labor and cost required to collect data. Such issues cause the convolutional neural networks (CNNs) to suffer with overfitting and a drastic loss in accuracy. To overcome this problem, generative adversarial networks (GANs) have been adopted in medical imaging as a data augmentation technique because of their capability to generate realistic samples that help add variability in the training set. Therefore, this paper proposes a data augmentation based on GAN to overcome the issue of limited data availability in conjunction with pretrained CNN models on detecting pneumonia from chest x-ray images. We use the auxiliary classifier GAN (ACGAN), which extends traditional GAN by making the generation of images conditional on a side information such as labels. The proposed method has further improved the performance of the CNN models most especially the ResNet variants that improved by more than 10%. ResNet-18, the smallest ResNet variant, showed the highest improvement with 13.36% in accuracy and 16.13% in F1-score and also outperformed the other CNN models used in the experiment. The addition of ACGAN-generated images has proven to be effective in adding variability to the training set.
Keywords: machine learning, GAN, ACGAN, deep learning, CNN, transfer learningAlbert, B. (2020). Deep learning from limited training data: Novel segmentation and ensemble algorithms applied to automatic melanoma diagnosis. IEEE Access, 8, 31254–31269. https://doi.org/10.1109/ACCESS.2020.2973188
Dong, X., Lei, Y., Tian, S., Wang, T., Patel, P., Curran, W. J., Jani, A. B., Liu, T., & Yang, X. (2019, December). Synthetic MRI-aided multi-organ segmentation on male pelvic CT using a cycle consistent deep attention network. Radiotherapy and Oncology, 141, 192–199. https://doi.org/10.1016/j.radonc.2019.09.028
Fetty, L., Lofstedt, T., Heilemann, G., Furtado, H., Nesvacil, N., Nyholm, T., Georg, D., & Kuess, P. (2020, May). Investigating conditional GAN performance with different generator architectures, an ensemble model, and different MR scanners for MR-SCT conversion. Physics in Medicine & Biology, 65(10), 105004. https://doi.org/10.1088/1361-6560/ab857b
Frid-Adar, M., Klang, E., Amitai, M., Goldberger, J., & Greenspam, H. (2018, January). Synthetic data augmentation using GAN for improved liver lesion classification. arXiv.org. https://arxiv.org/abs/1801.02385
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2020, November). Generative adversarial networks. Communications of the ACM, 63(11), 139–144. https://doi.org/10.1145/3422622
Han, C., Kitamura, Y., Kudo, A., Ichinose, A., Rundo, L., Furukawa, Y., Umemoto, K., Li, Y., & Nakayama, H. (2019). Synthesizing diverse lung nodules wherever massively: 3D multi-conditional GAN-based CT image augmentation for object detection. In 2019 International Conference on 3D Vision (3DV) (pp. 729–737). IEEE. https://doi.org/10.1109/3DV.2019.00085
He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep residual learning for image recognition. arXiv.org. https://arvix.org/abs/1512.03385
Huang, G., Liu, Z., Maaten, L. V., & Weinberger, K. Q. (2018, January 28). Densely connected
convolutional networks. arXiv.org. https://arxiv.org/abs/1608.06993
Jain, R., Nagrath, P., Kataria, G., Kaushik, V. S., & Hemanth, D. J. (2020). Pneumonia detection in chest x-ray images using convolutional neural networks and transfer learning. Measurement, 165, 108046. https://doi.org/10.1016/j.measurement.2020.108046
Jmour, N., Zyen, S., & Abdelkrim, A. (2018). Convolutional neural networks for image classification. In 2018 International Conference on Advanced Systems and Electric Technologies (IC_ASET) (pp. 397–402). IEEE. https://doi.org/10.1109/ASET.2018.8379889
Kan, C. N. E., Maheenaboobacker, N., & Ye, D. H. (2020). Age-conditioned synthesis of pediatric computed tomography with auxiliary classifier generative adversarial networks. In 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI) (pp. 109–112). IEEE. https://doi.org/10.1109/ISBI45749.2020.9098623
Kandel, I., & Castelli, M. (2020, December). The effect of batch size on the generalizability of the convolutional neural networks on a histopathology dataset. ICT Express, 6(4), 312–315. https://doi.org/10.1016/j.icte.2020.04.010
Kermany, D., Zhang, K., & Goldbaum, M. (2018). Labeled optical coherence tomography (OCT) and chest x-ray images for classification (Version 2) [Data set]. Mendeley Data. https://doi.org/10.17632/rscbjbr9sj.2
Kim, J., Kim, J., Han G., Rim, C., & Jo, H. (2020). Low-dose CT image restoration using generative adversarial networks. Informatics in Medicine Unlocked, 21, 100468. https://doi.org/10.1016/j.imu.2020.100468
Learn about pneumonia. (n.d.). https://www.lung.org/lung-health-diseases/lung-disease-lookup/pneumonia/learn-about-pneumonia#:~:text=Pneumonia%20is%20an%20infection%20of,to%20get%20into%20your%20bloodstream
Li, Y., Zhang, Z., Dai, C., Dong, Q., & Badrigilan, S. (2020). Accuracy of deep learning for automated detection of pneumonia using chest x-ray images: A systematic review and meta-analysis. Computers in Biology and Medicine, 123, 103898. https://doi.org/10.1016/j.compbiomed.2020.103898
Liang, G., & Zheng, L. (2020). A transfer learning method with deep residual network for pediatric pneumonia diagnosis. Computer Methods and Programs in Biomedicine, 187, 104964. https://doi.org/10.1016/j.cmpb.2019.06.023
Mao, X., Li, Q., Xie, H., Lau, R. Y. K., Wang, Z., & Smolley, S. P. (2017). Least squares generative adversarial networks. In 2017 IEEE International Conference on Computer Vision (ICCV) (pp. 2794–2802). IEEE. https://doi.org/10.1109/iccv.2017.304
Mirza, M., & Osindero, S. (2014, November). Conditional generative adversarial nets. arXiv.org. https://arxiv.org/abs/1411.1784
Odena, A., Olah, C., & Shlens, J. (2017). Conditional image synthesis with auxiliary classifier GANs. Proceedings of the 34th International Conference on Machine Learning. PMLR, 70, 2642–2651.
Pneumonia. (n.d.). https://www.who.int/news-room/fact-sheets/detail/pneumonia
Qian, P., Xu, K., Wang, T., Zheng, Q., Yang, H., Baydoun, A., Zhu, J., Traughber, B., & Muzic, R. F. (2020, June). Estimating CT from MR abdominal images using novel generative adversarial networks. Journal of Grid Computing, 18(2), 211–226. https://doi.org/10.1007/s10723-020-09513-3
Qin, C., Yao, D., Shi, Y., & Song, Z. (2018, August 22). Computer-aided detection in chest radiography based on artificial intelligence: A survey. BioMedical Engineering OnLine, 17, Article 113. https://biomedical-engineering-online.biomedcentral.com/articles/10.1186/s12938-018-0544-y
Radford, A., Metz, L., & Chintala, S. (2016, January). Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv.org. https://arvix.org/abs/1511.06434
Saraiva, A., Ferreira, N., de Sousa, L. L., Costa, N., Sousa, J., Santos, D., Valente, A., & Soares, S. (2019). Classification of images of childhood pneumonia using convolutional neural networks. In Proceedings of the 12th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2019) (112–119). https://www.scitepress.org/Link.aspx?doi=10.5220/0007404301120119
Shaikhina, T., & Khovanova, N. A. (2017, Jan). Handling limited datasets with neural networks in medical applications: A small-data approach. Artificial Intelligence in Medicine, 75, 51–63. https://doi.org/10.1016/j.artmed.2016.12.003
Theckedath, D., & Sedamkar, R. R. (2020). Detecting affected states using VGG16, ResNet50 and SE-ResNet50 Networks. SN Computer Science, 1, 79. https://doi.org/10.1007/s42979-020-0114-9
Waheed, A., Goyal, M., Gupta, D., Khanna, A., Al-Turjman, F., & Pinheiro, P. R. (2020). Covid GAN: Data augmentation using auxiliary classifier GAN for improved Covid-19 detection. IEEE Access, 8, 91916–91923. https://doi.org/10.1109/ACCESS.2020.2994762
Wani, M. A., Bhat, F. A., Afzal, S., & Khan, A. I. (2020). Advances in deep learning. Singapore: Springer Singapore. https://doi.org/10.1007/978-981-13-6794-6
Weiss, K., Khoshgoftaar, T. M., & Wang, D. (2016). A survey of transfer learning. Journal of Big Data, 3, 9. https://doi.org/10.1186/s40537-016-0043-6
Wolterink, J. M., Leiner, T., Viergever, M. A., & Isgum, I. (2017, December). Generative adversarial networks for noise reduction in low-dose CT. IEEE Transactions on Medical Imaging, 36(12), 2536–2545. https://doi.org/10.1109/TMI.2017.2708987
Yamashita, R., Nishio, M., Do, R. K. G., & Togashi, K. (2018). Convolutional neural networks: An overview and application in radiology. Insights Into Imaging, 9(4), 611–629.
Yang, H., Xia, K., Anqi, B., Qian, P., & Khosravi, M. R. (2019). Abdomen MRI synthesis based on conditional GAN. In 2019 International Conference on Computational Science and Computational Intelligence (CSCI) (pp. 1021–1025). IEEE. https://doi.org/10.1109/CSCI49370.2019.00195
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014). How transferable are features in deep neural networks? arXiv.org. https://arxiv.org/abs/1411.1792