EBAQ: An Entropy-Based Bit Allocation Framework for Lightweight Autoencoder Models

Authors

DOI:

https://doi.org/10.29207/resti.v10i1.6925

Keywords:

autoencoder, anomaly detection, edge computing efficiency, entropy-based quantization, latent space representation

Abstract

Autoencoder-based models have shown strong potential for anomaly detection in complex time-series data; however, they often assume equal importance across latent dimensions, resulting in inefficiencies and reduced precision. This study addresses this limitation by introducing the Entropy-Based Bit Allocation Quantizer (EBAQ), a novel quantization framework that adaptively allocates bits to each latent dimension based on its entropy, preserving more precision where information content is highest. The primary objective is to enhance representational efficiency and anomaly detection performance without increasing model complexity or computational cost. EBAQ is implemented as a plug-and-play module within a standard autoencoder architecture, requiring no retraining or architectural modification. The method was evaluated using a publicly available ECG dataset, where reconstruction-based anomaly detection was employed to assess its performance. Results show that EBAQ outperforms the standard autoencoder baseline, achieving higher accuracy (94.9%), precision (99.4%), and recall (91.4%), while also demonstrating more apparent separation between normal and anomalous data in latent space visualizations. These findings confirm that entropy-aware quantization improves both fidelity and interpretability in unsupervised anomaly detection. Overall, this work presents a theoretically grounded and practically efficient solution that bridges information theory and deep learning, offering a human-centered approach to developing more intelligent and efficient AI systems for real-world applications.

Downloads

Download data is not yet available.

References

M. Nawaz and J. Ahmed, “Cloud-based healthcare framework for real-time anomaly detection and classification of 1-D ECG signals,” PLOS ONE, vol. 17, no. 12, p. e0279305, Dec. 2022, doi: 10.1371/journal.pone.0279305.

J. Qin et al., “A novel temporal generative adversarial network for electrocardiography anomaly detection,” Artificial Intelligence in Medicine, vol. 136, p. 102489, Feb. 2023, doi: 10.1016/j.artmed.2023.102489.

H. Zhou and C. Kan, “Tensor-Based ECG Anomaly Detection toward Cardiac Monitoring in the Internet of Health Things,” Sensors, vol. 21, no. 12, 2021, doi: 10.3390/s21124173.https://doi.org/10.3390/s21124173

A. Raza, K. P. Tran, L. Koehl, and S. Li, “AnoFed: Adaptive anomaly detection for digital health using transformer-based federated learning and support vector data description,” Engineering Applications of Artificial Intelligence, vol. 121, p. 106051, May 2023, doi: 10.1016/j.engappai.2023.106051.

T. Dissanayake, T. Fernando, S. Denman, S. Sridharan, and C. Fookes, “DConv-LSTM-Net: A Novel Architecture for Single- and 12-Lead ECG Anomaly Detection,” IEEE Sensors Journal, vol. 23, no. 19, pp. 22763–22776, Oct. 2023, doi: 10.1109/JSEN.2023.3300752.

M. Roy, S. Majumder, A. Halder, and U. Biswas, “ECG-NET: A deep LSTM autoencoder for detecting anomalous ECG,” Engineering Applications of Artificial Intelligence, vol. 124, p. 106484, Sep. 2023, doi: 10.1016/j.engappai.2023.106484.

M. R. Azghadi et al., “Hardware Implementation of Deep Network Accelerators Towards Healthcare and Biomedical Applications,” IEEE Transactions on Biomedical Circuits and Systems, vol. 14, no. 6, pp. 1138–1159, Dec. 2020, doi: 10.1109/TBCAS.2020.3036081.

Y. Liu et al., “Deep Anomaly Detection for Time-Series Data in Industrial IoT: A Communication-Efficient On-Device Federated Learning Approach,” IEEE Internet of Things Journal, vol. 8, no. 8, pp. 6348–6358, Apr. 2021, doi: 10.1109/JIOT.2020.3011726.

Q. Zhang, R. Han, G. Xin, C. H. Liu, G. Wang, and L. Y. Chen, “Lightweight and Accurate DNN-Based Anomaly Detection at Edge,” IEEE Transactions on Parallel and Distributed Systems, vol. 33, no. 11, pp. 2927–2942, Nov. 2022, doi: 10.1109/TPDS.2021.3137631.

Q. Yu, C. Li, Y. Zhu, and T. Kurita, “Convolutional autoencoder based on latent subspace projection for anomaly detection,” Methods, vol. 214, pp. 48–59, Jun. 2023, doi: 10.1016/j.ymeth.2023.04.007.

A. Ghafourian, H. Shui, D. Upadhyay, R. Gupta, D. Filev, and I. Soltani, “Targeted Collapse Regularized Autoencoder for Anomaly Detection: Black Hole at the Center,” IEEE Transactions on Neural Networks and Learning Systems, vol. 36, no. 6, pp. 10348–10358, Jun. 2025, doi: 10.1109/TNNLS.2024.3472456.

K. A. Alaghbari, H.-S. Lim, M. H. M. Saad, and Y. S. Yong, “Deep Autoencoder-Based Integrated Model for Anomaly Detection and Effi cient Feature Extraction in IoT Networks,” IoT, vol. 4, no. 3, pp. 345–365, Aug. 2023, doi: 10.3390/iot4030016.

M. Ivanovska and V. Štruc, “Y-GAN: Learning dual data representations for anomaly detection in images,” Expert Systems with Applications, vol. 248, p. 123410, Aug. 2024, doi: 10.1016/j.eswa.2024.123410.

A. Niaz, S. Ul Amin, S. Soomro, H. Zia, and K. Nam Choi, “Spatially Aware Fusion in 3D Convolutional Autoencoders for Video Anomaly Detection,” IEEE Access, vol. 12, pp. 104770–104784, 2024, doi: 10.1109/access.2024.3435144.

H. Torabi, S. L. Mirtaheri, and S. Greco, “Practical autoencoder based anomaly detection by using vector reconstruction error,” Cybersecurity, vol. 6, no. 1, Jan. 2023, doi: 10.1186/s42400-022-00134-9.

F. Alshameri and R. Xia, “An Evaluation of Variational Autoencoder in Credit Card Anomaly Detection,” Big Data Min. Anal., vol. 7, no. 3, pp. 718–729, Sep. 2024, doi: 10.26599/bdma.2023.9020035.

M. H. Soleimani-Babakamali, R. Soleimani-Babakamali, R. Sarlo, M. F. Farghally, and I. Lourentzou, “On the effectiveness of dimensionality reduction for unsupervised structural health monitoring anomaly detection,” Mechanical Systems and Signal Processing, vol. 187, p. 109910, Mar. 2023, doi: 10.1016/j.ymssp.2022.109910.

J. Shin, Y. Kang, and Y.-S. Jeon, “Vector Quantization for Deep-Learning-Based CSI Feedback in Massive MI MO Systems,” IEEE Wireless Commun. Lett., vol. 13, no. 9, pp. 2382–2386, Sep. 2024, doi: 10.1109/lwc.2024.3415633.

S. Kotyan, T. Ueda, and D. V. Vargas, “k* Distribution: Evaluating the Latent Space of Deep Neural Networks Using Local Neighborhood Analysis,” IEEE Transactions on Neural Networks and Learning Systems, pp. 1–15, 2024, doi: 10.1109/TNNLS.2024.3446509.

Z. Ge, S. Ma, W. Gao, J. Pan, and C. Jia, “NLIC: Non-Uniform Quantization-Based Learned Image Compression,” IEEE Trans. Circuits Syst. Video Technol., vol. 34, no. 10, pp. 9647–9663, Oct. 2024, doi: 10.1109/tcsvt.2024.3401872.

L. Gu, J. Liu, X. Liu, W. Wan, and J. Sun, “Entropy-Optimized Deep Weighted Product Quantization for Image Retrieval,” IEEE Trans. on Image Process., vol. 33, pp. 1162–1174, 2024, doi: 10.1109/tip.2024.3359066.

Z. Chen et al., “Learning vector quantized representation for cancer subtypes identification,” Computer Methods and Programs in Biomedicine, vol. 236, p. 107543, Jun. 2023, doi: 10.1016/j.cmpb.2023.107543.

S. Lee, J.-B. Jeong, and E.-S. Ryu, “Entropy-Constrained Implicit Neural Representations for Deep Image Compression,” IEEE Signal Process. Lett., vol. 30, pp. 663–667, 2023, doi: 10.1109/lsp.2023.3279780.

G. C. Marinò, F. Furia, D. Malchiodi, and M. Frasca, “Efficient and Compact Representations of Deep Neural Networks via Entropy Coding,” IEEE Access, vol. 11, pp. 106103–106125, 2023, doi: 10.1109/access.2023.3317293.

M. Cao et al., “End-to-End Optimized Image Compression With Deep Gaussian Process Regression,” IEEE Trans. Circuits Syst. Video Technol., vol. 35, no. 4, pp. 3770–3785, Apr. 2025, doi: 10.1109/tcsvt.2022.3209661.

H. Li, J. Zhang, Z. Li, J. Liu, and Y. Wang, “Improvement of Min-Entropy Evaluation Based on Pruning and Quantized Deep Neural Network,” IEEE Trans.Inform.Forensic Secur., vol. 18, pp. 1410–1420, 2023, doi: 10.1109/tifs.2023.3240859.

S. Choi, H. Jin, and E. Kim, “Task-Aware Dynamic Model Optimization for Multi-Task Learning,” IEEE Access, vol. 11, pp. 137709–137717, 2023, doi: 10.1109/access.2023.3339793.

B. S. Sharmila and R. Nagapadma, “Quantized autoencoder (QAE) intrusion detection system for anomaly detection in resource-constrained IoT devices using RT-IoT2022 dataset,” Cybersecurity, vol. 6, no. 1, Sep. 2023, doi: 10.1186/s42400-023-00178-5.

M. S. Alam, W. A. Misba, and J. Atulasimha, “Quantized non-volatile nanomagnetic domain wall synapse based autoenco der for efficient unsupervised network anomaly detection,” Neuromorph. Comput. Eng., vol. 4, no. 2, p. 024012, Jun. 2024, doi: 10.1088/2634-4386/ad49ce.

Downloads

Published

2026-02-26

Issue

Section

Artificial Intelligence

How to Cite

Zaidir, Z., & Mohammad Diqi. (2026). EBAQ: An Entropy-Based Bit Allocation Framework for Lightweight Autoencoder Models. Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), 10(1), 171-180. https://doi.org/10.29207/resti.v10i1.6925