The Formula Study in Determining the Best Number of Neurons in Neural Network Backpropagation Architecture with Three Hidden Layers

Syaharuddin Syaharuddin; Fatmawati Fatmawati; Herry Suprajitno

doi:10.29207/resti.v6i3.4049

Syaharuddin Syaharuddin Universitas Airlangga
Fatmawati Fatmawati Universitas Airlangga
Herry Suprajitno Universitas Airlangga

DOI: https://doi.org/10.29207/resti.v6i3.4049

Keywords: Neural Network, Backpropagation, 3-Layer Hidden, Number of Neurons

Abstract

The researchers conducted data simulation experiments, but they did so unstructured in determining the number of neurons in the hidden layer in the Artificial Neural Network Back-Propagation architecture. The researchers also used a general architecture consisting of one hidden layer. Researchers are still producing minimal research that discusses how to determine the number of neurons when using hidden layers. This article examines the results of experiments by conducting training and testing data using seven recommended formulas including the Hecht-Nelson, Marchandani-Cao, Lawrence & Fredrickson, Berry-Linoff, Boger-Guterman, JingTao-Chew, and Lawrence & Fredrickson modifications. We use rainfall data and temperature data with a 10-day type for the last 10 years (2012-2021) sourced from Lombok International Airport Station, Indonesia. The training and testing data used showed the results that in determining the number of neurons on the hidden-1 screen, it was more appropriate to use the Hecht-Nelson formula and the Lawrence & Fredricson formula which is more suitable for use in the 2nd & 3rd hidden layer. The resulting research was able to provide an accuracy rate of up to 97.79% (temperature data) and 99.94% (rainfall data) with an architecture of 36-73-37-19-1.

Downloads

Download data is not yet available.

References

Haviluddin and R. Alfred, “A genetic-based backpropagation neural network for forecasting in time-series data,” in Proceedings - 2015 International Conference on Science in Information Technology: Big Data Spectrum for Future Information Economy, ICSITech 2015, 2016, pp. 158–163, doi: 10.1109/ICSITech.2015.7407796.

H. Azami, M. R. Mosavi, and S. Sanei, “Classification of GPS satellites using improved back propagation training algorithms,” Wirel. Pers. Commun., vol. 71, no. 2, pp. 789–803, 2013, doi: 10.1007/s11277-012-0844-7.

H. Park, “Study for Application of Artificial Neural Networks in Geotechnical Problems,” in Artificial Neural Networks - Application, Croatia: InTech, 2011, pp. 303–336.

Y. Bai, Y. Li, X. Wang, J. Xie, and C. Li, “Air pollutants concentrations forecasting using back propagation neural network based on wavelet decomposition with meteorological conditions,” Atmos. Pollut. Res., vol. 7, no. 3, pp. 557–566, 2016, doi: 10.1016/j.apr.2016.01.004.

T. Jayalakshmi and A. Santhakumaran, “Statistical Normalization and Back Propagationfor Classification,” Int. J. Comput. Theory Eng., vol. 3, no. 1, pp. 89–93, 2011, doi: 10.7763/ijcte.2011.v3.288.

R. Bagaria, S. Wadhwani, and A. K. Wadhwani, “Bone fractures detection using support vector machine and error backpropagation neural network,” Optik (Stuttg)., vol. 247, 2021, doi: 10.1016/j.ijleo.2021.168021.

N. M. Nawi, F. Hamzah, N. A. Hamid, M. Z. Rehman, M. Aamir, and A. A. Ramli, “An optimized back propagation learning algorithm with adaptive learning rate,” Int. J. Adv. Sci. Eng. Inf. Technol., vol. 7, no. 5, pp. 1693–1700, 2017, doi: 10.18517/ijaseit.7.5.2972.

S. N. Endah, A. P. Widodo, M. L. Fariq, S. I. Nadianada, and F. Maulana, “Beyond back-propagation learning for diabetic detection: Convergence comparison of gradient descent, momentum and Adaptive Learning Rate,” in Proceedings - 2017 1st International Conference on Informatics and Computational Sciences, ICICoS 2017, 2017, vol. 2018-January, pp. 189–193, doi: 10.1109/ICICOS.2017.8276360.

H. Asgari, X. Chen, M. B. Menhaj, and R. Sainudiin, “Artificial neural network-based system identification for a single-shaft gas turbine,” J. Eng. Gas Turbines Power, vol. 135, no. 9, pp. 1-7, 2013, doi: 10.1115/1.4024735.

S. Ch and S. Mathur, “Particle swarm optimization trained neural network for aquifer parameter estimation,” KSCE J. Civ. Eng., vol. 16, no. 3, pp. 298–307, 2012, doi: 10.1007/s12205-012-1452-5.

D. Pratiwi, D. D. Santika, and B. Pardamean, “An Application Of Backpropagation Artificial Neural Network Method for Measuring The Severity of Osteoarthritis,” Int. J. Eng. Technol., vol. 11, no. 3, pp. 102–105, 2011.

E. Suryani, Wiharto, S. Palgunadi, and T. P. Nurcahya Pradana, “Classification of Acute Myelogenous Leukemia (AML M2 and AML M3) using Momentum Back Propagation from Watershed Distance Transform Segmented Images,” in Journal of Physics: Conference Series, 2017, vol. 801, no. 1, pp. 1-8, doi: 10.1088/1742-6596/801/1/012044.

C. C. Gowda and S. G. Mayya, “Comparison of Back Propagation Neural Network and Genetic Algorithm Neural Network for Stream Flow Prediction,” J. Comput. Environ. Sci., vol. 2014, pp. 1–6, 2014, doi: 10.1155/2014/290127.

S. J. Abdulkadir, S. M. Shamsuddin, and R. Sallehuddin, “Moisture Prediction in Maize Using Three Term Back Propagation Neural Network,” Int. J. Environ. Sci. Dev., vol. 3, no. 2, pp. 199–204, 2012, doi: 10.7763/ijesd.2012.v3.215.

S. Karsoliya, “Approximating Number of Hidden layer neurons in Multiple Hidden Layer BPNN Architecture,” Int. J. Eng. Trends Technol., vol. 3, no. 6, pp. 714–717, 2012.

M. J. Berry and G. Linoff, “Data Mining Techniques: For Marketing, Sales, and Customer Relationship Management,” John Wiley Sons, Inc., pp. 1–888, 2011.

R. Hecht-Nielsen, “Kolmogorov’s Mapping Neural Network Existence Theorem,” Proc. Int. Conf. Neural Networks, pp. 11–14, 1987.

Z. Yudong and W. Lenan, “Stock market prediction of S&P 500 via combination of improved BCO approach and BP neural network,” Expert Syst. Appl., vol. 36, no. 5, pp. 8849–8854, 2009, doi: 10.1016/j.eswa.2008.11.028.

G. Mirchandani and W. Cao, “On Hidden Nodes for Neural Nets,” IEEE Trans. Circuits Syst., vol. 36, no. 5, pp. 661–664, 1989, doi: 10.1109/31.31313.

J. Laurence, Introduction to Neural Networks: Design, Theory, and Applications. Nevada City: California Scientific Software, 1994.

A. Blum, Neural networks in C++: an object-oriented framework for building connectionist systems, Cambridge . New York: John Wiley & Sons, Inc., 1992.

R. Faisal, N. S. Surjandari, and S. Setiono, “Prediksi Stabilitas Lereng Menggunakan Adaptive Neuro-Fuzzy Metode Hybrid,” Matriks Tek. Sipil, vol. 6, no. 3, pp. 439-450, 2018, doi: 10.20961/mateksi.v6i3.36549.

Z. Boger and H. Guterman, “Knowledge extraction from artificial neural networks models,” in Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, 1997, vol. 4, pp. 3030–3035, doi: 10.1109/icsmc.1997.633051.

Y. JingTao and L. T. Chew, “Guidelines for financial forecasting with neural networks,” . Neural Inf. Process. Shanghai, 2001.

B. Setyonugroho, A. E. Permanasari, and S. S. Kusumawardani, “Perbandingan Akurasi Algoritme Pelatihan dalam Jaringan Syaraf Tiruan untuk Peramalan Jumlah Pengguna Kereta Api di Pulau Jawa,” J. Metik, vol. 1, no. 1, pp. 50–62, 2017.

Mislan, Haviluddin, S. Hardwinarto, Sumaryono, and M. Aipassa, “Rainfall Monthly Prediction Based on Artificial Neural Network: A Case Study in Tenggarong Station, East Kalimantan - Indonesia,” in Procedia Computer Science, 2015, vol. 59, pp. 142–151, doi: 10.1016/j.procs.2015.07.528.

Z. Zhang, P. Yang, X. Ren, Q. Su, and X. Sun, “Memorized sparse backpropagation,” Neurocomputing, vol. 415, pp. 397–407, 2020, doi: 10.1016/j.neucom.2020.08.055.

S. Almaliki, R. Alimardani, and M. Omid, “Artificial neural network based modeling of tractor performance at different field conditions,” Agric. Eng. Int. CIGR J., vol. 18, no. 4, pp. 262–274, 2016.

G. Lesinski, S. Corns, and C. Dagli, “Application of an Artificial Neural Network to Predict Graduation Success at the United States Military Academy,” in Procedia Computer Science, 2016, vol. 95, pp. 375–382, doi: 10.1016/j.procs.2016.09.348.