Modification of SqueezeNet for Devices  with Limited Computational Resources

Rahmadya Trias Handayanto; Herlawati

doi:10.29207/resti.v7i1.4446

Rahmadya Trias Handayanto Universitas Islam 45 Bekasi
Herlawati Universitas Bhayangkara Jakarta Raya

DOI: https://doi.org/10.29207/resti.v7i1.4446

Keywords: deep learning, squeezenet, resnet, imagenet, convolutional layer

Abstract

In recent years, the computational approach has shifted from a statistical basis to deep neural network architectures which process the input without explicit knowledge that underlies the model. Many models with high accuracy have been proposed by training the datasets using high performance computing devices. However, only a few studies have examined its use on non-high-performance computers. In fact, most users, who are mostly researchers in certain fields (medical, geography, economics, etc.) sometimes need computers with limited computational resources to process datasets, from notebooks, personal computers, to mobile processor-based devices. This study proposes a basic model with good accuracy and can run lightly on the average computer so that it remains lightweight when used as a basis for advanced deep neural networks models, e.g., U-Net, SegNet, PSPNet, DeepLab, etc. Using several well-known basic methods as a baseline (SqueezeNet, ShuffleNet, GoogleNet, MobileNetV2, and ResNet), a model combining SqueezeNet with ResNet, termed Res-SqueezeNet, was formed. Testing results show that the proposed method has accuracy and inference time of 84.59% and 8.46 second, respectively, which has an accuracy of 2% higher than the SqueezeNet (82.53%) and is close to the accuracy of other baseline methods (from 84.93% to 0.88.01%) while still maintaining the inference speed (below nine second). In addition, residual part of the proposed method can be used to avoid vanishing gradient, hence, it can be implemented to solve more advanced problems which need a lot of layers, e.g., semantic segmentation, time-series prediction, etc.

Downloads

Download data is not yet available.

References

M. Tan and Q. V. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” 36th International Conference on Machine Learning, ICML 2019, vol. 2019-June, pp. 10691–10700, 2019.

Mathworks, “Pretrained Deep Neural Networks,” 2022. [Online]. Available: https://www.mathworks.com/help/deeplearning/ug/pretrained-convolutional-neural-networks.html. [Accessed: 21-Aug-2022].

M. Z. Alom et al., “The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches,” 2018.

R. Zhang, G. Li, M. Li, and L. Wang, “Fusion of images and point clouds for the semantic segmentation of large-scale 3D scenes based on deep learning,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 143, no. April, pp. 85–96, 2018.

E. Shelhamer, J. Long, and T. Darrell, “Fully Convolutional Networks for Semantic Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 4, pp. 640–651, 2017.

A. A. Ardakani, A. R. Kanafi, U. R. Acharya, N. Khadem, and A. Mohammadi, “Application of deep learning technique to manage COVID-19 in routine clinical practice using CT images: Results of 10 convolutional neural networks,” Computers in Biology and Medicine, vol. 121, no. April, p. 103795, 2020.

H. J. Lee, I. Ullah, W. Wan, Y. Gao, and Z. Fang, “Real-Time vehicle make and model recognition with the residual squeezenet architecture,” Sensors (Switzerland), vol. 19, no. 5, 2019.

X. Zhang, X. Zhou, M. Lin, and J. Sun, “ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 6848–6856, 2018.

K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2016-Decem, pp. 770–778, 2016.

A. Victor Ikechukwu, S. Murali, R. Deepu, and R. C. Shivamurthy, “ResNet-50 vs VGG-19 vs training from scratch: A comparative analysis of the segmentation and classification of Pneumonia from chest X-ray images,” Global Transitions Proceedings, vol. 2, no. 2, pp. 375–381, 2021.

A. G. Howard et al., “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” Computer Vision and Pattern Recognition, 2017.

N. Ma, X. Zhang, H. T. Zheng, and J. Sun, “Shufflenet V2: Practical guidelines for efficient cnn architecture design,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11218 LNCS, pp. 122–138, 2018.

M. Loey, G. Manogaran, M. H. N. Taha, and N. E. M. Khalifa, “A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic,” Measurement: Journal of the International Measurement Confederation, vol. 167, no. July 2020, p. 108288, 2021.

A. A. Pravitasari et al., “UNet-VGG16 with transfer learning for MRI-based brain tumor segmentation,” Telkomnika (Telecommunication Computing Electronics and Control), vol. 18, no. 3, pp. 1310–1318, 2020.

Z. Benbahria, M. F. Smiej, I. Sebari, and H. Hajji, “Land cover intelligent mapping using transfer learning and semantic segmentation,” 7th Mediterranean Congress of Telecommunications 2019, CMT 2019, pp. 1–5, 2019.

M. Imad, O. Doukhi, and D. J. Lee, “Transfer learning based semantic segmentation for 3d object detection from point cloud,” Sensors, vol. 21, no. 12, pp. 1–15, 2021.

J. Yang, Y. Q. Zhao, and J. C. W. Chan, “Learning and Transferring Deep Joint Spectral-Spatial Features for Hyperspectral Classification,” IEEE Transactions on Geoscience and Remote Sensing, vol. 55, no. 8, pp. 4729–4742, 2017.

I. Z. Matovinovic, S. Loncaric, J. Lo, M. Heisler, and M. Sarunic, “Transfer learning with U-net type model for automatic segmentation of three retinal layers in optical coherence tomography images,” International Symposium on Image and Signal Processing and Analysis, ISPA, vol. 2019-Septe, pp. 49–53, 2019.

M. Á. Molina, G. Asencio-Cortés, J. C. Riquelme, and F. Martínez-Álvarez, “A Preliminary Study on Deep Transfer Learning Applied to Image Classification for Small Datasets,” Advances in Intelligent Systems and Computing, vol. 1268 AISC, pp. 741–750, 2021.

A. Zhang, Z. C. Lipton, M. Li, and A. J. Smola, “Dive into Deep Learning,” arXiv preprint arXiv:2106.11342, 2021.

N. D. Trung, T. T. Ngoc, and H. X. Huynh, “Automated Pneumonia Detection in X-Ray Images Via Depthwise Separable Convolution Based Learning,” pp. 1–8, 2020.

K. A. Laube and A. Zell, “ShuffleNASNets: Efficient CNN models through modified Efficient Neural Architecture Search,” Proceedings of the International Joint Conference on Neural Networks, vol. 2019-July, 2019.

F. N. Iandola, S. Han, M. W. Moskewicz, K. Ashraf, W. J. Dally, and K. Keutzer, “SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size,” pp. 1–13, 2016.

J. Wang, P. Lv, H. Wang, and C. Shi, “SAR-U-Net: Squeeze-and-excitation block and atrous spatial pyramid pooling based residual U-Net for automatic liver segmentation in Computed Tomography,” Computer Methods and Programs in Biomedicine, vol. 208, pp. 1–25, 2021.

P. K. Gadosey et al., “SD-UNET: Stripping down U-net for segmentation of biomedical images on platforms with low computational budgets,” Diagnostics, vol. 10, no. 2, 2020.

I. L. Kharisma, R. T. Handayanto, and D. A. Dewi, “Face Mask Detection In The Covid-19 Pandemic Era by Implementing Convolutional Neural Network and Pre-Trained CNN Models,” in 2021 IEEE 7th International Conference on Computing, Engineering and Design (ICCED), 2021, pp. 1–6.

V. Badrinarayanan, A. Kendall, and R. Cipolla, “SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 12, pp. 2481–2495, 2017.

Mathworks, “Data Sets for Deep Learning,” 2022. [Online]. Available: https://www.mathworks.com/help/deeplearning/ug/data-sets-for-deep-learning.html. [Accessed: 21-Aug-2022].