Hyperparameter Tuning with Optuna to optimize the YOLOv11n Model for Weed Detection

Authors

DOI:

https://doi.org/10.29207/resti.v9i5.6682

Keywords:

image augmentation, Optuna, residential weed detection, YOLO nano, YOLOv11n

Abstract

Accurate weed detection is essential for maintaining the cleanliness and aesthetic appeal of residential yards. This study aimed to optimize YOLOv11n, a lightweight object detection model, to achieve high precision in weed identification under real-world conditions. The novelty of this study lies in the application of Optuna, an automatic hyperparameter optimization framework, to enhance model performance while maintaining computational efficiency for resource-limited devices such as drones and IoT systems. The research involved data augmentation techniques including crop (0–20% zoom), hue (±20°), saturation (±30%), brightness (±20%), exposure (±15%), and mosaic augmentation. These augmented images were used to train four YOLO nano variants (v5n, v8n, v11n, v12n), which were evaluated using standard metrics: Precision, Recall, F1-Score, and mean Average Precision (mAP). Among the models tested, YOLOv11n with Custom Optuna configuration delivered the highest performance, achieving a 94.6% F1-score and 97.8% mAP@0.5. These results demonstrate that the optimized YOLOv11n model can support accurate and efficient real-time weed detection in household environments, particularly on edge devices with limited hardware capabilities. This makes it a viable solution for practical implementation in precision agriculture and smart gardening.

Downloads

Download data is not yet available.

References

J. Lisek, L. Sas-Paszt, A. Mika, and A. Lisek, “The Response of Weeds and Apple Trees to Beneficial Soil Microorganisms and Mineral Fertilizers Applied in Orchards,” Agronomy, vol. 12, no. 11, 2022, doi: 10.3390/agronomy12112882.

J. Lisek, “Diversity of Summer Weed Communities in Response to Different Plum Orchard Floor Management in-Row,” Agronomy, vol. 13, no. 5, 2023, doi: 10.3390/agronomy13051421.

A. Arato, G. Mengesha, Z. Fentahun, and B. Borano, “Association of apple powdery mildew (Podosphaera leucotricha) and anthracnose (Neofabraea malicorticis) epidemics with environmental factors and agronomic practices in Chencha highlands, southern Ethiopia,” Phytopathol. Res., vol. 7, 2025, doi: 10.1186/s42483-024-00303-3.

C. M. Badgujar, A. Poulose, and H. Gan, “Agricultural object detection with You Only Look Once (YOLO) Algorithm: A bibliometric and systematic literature review,” Comput. Electron. Agric., vol. 223, pp. 1–30, 2024, doi: 10.1016/j.compag.2024.109090.

L. Yang, T. Zhang, S. Zhou, and J. Guo, “AAB-YOLO: An Improved YOLOv11 Network for Apple Detection in Natural Environments,” Agriculture, vol. 15, no. 8, 2025, doi: 10.3390/agriculture15080836.

F. M. Firnando, D. R. I. M. Setiadi, A. R. Muslikh, and S. W. Iriananda, “Analyzing InceptionV3 and InceptionResNetV2 with Data Augmentation for Rice Leaf Disease Classification,” J. Futur. Artif. Intell. Technol., vol. 1, no. 1, pp. 1–11, 2024, doi: 10.62411/faith.2024-4.

R. K. Rachman, D. R. I. M. Setiadi, A. Susanto, K. Nugroho, and H. M. M. Islam, “Enhanced Vision Transformer and Transfer Learning Approach to Improve Rice Disease Recognition,” J. Comput. Theor. Appl., vol. 1, no. 4, pp. 446–460, 2024, doi: 10.62411/jcta.10459.

J. Mei and W. Zhu, “BGF-YOLOv10: Small Object Detection Algorithm from Unmanned Aerial Vehicle Perspective Based on Improved YOLOv10,” Sensors, vol. 24, no. 21, p. 6911, Oct. 2024, doi: 10.3390/s24216911.

L. Nieradzik et al., “WoodYOLO: A Novel Object Detector for Wood Species Detection in Microscopic Images,” Forests, vol. 15, no. 11, p. 1910, Oct. 2024, doi: 10.3390/f15111910.

X. Xi, Z. Huang, L. Ning, and Y. Zhang, “Real-time Pedestrian Detection Algorithm Based on Improved YOLOv3,” J. Phys. Conf. Ser., pp. 180–183, 2021, doi: 10.1088/1742-6596/2002/1/012075.

S. Singha and B. Aydin, “Automated Drone Detection Using YOLOv4,” Drones, vol. 5, no. 3, p. 95, Sep. 2021, doi: 10.3390/drones5030095.

M. Sportelli et al., “Evaluation of YOLO Object Detectors for Weed Detection in Different Turfgrass Scenarios,” Appl. Sci., vol. 13, no. 14, 2023, doi: 10.3390/app13148502.

J. Luo, Z. Liu, Y. Wang, A. Tang, H. Zuo, and P. Han, “Efficient Small Object Detection You Only Look Once: A Small Object Detection Algorithm for Aerial Images,” Sensors, vol. 24, no. 21, 2024, doi: 10.3390/s24217067.

A. Startups, “Weeds Dataset,” Roboflow Universe. [Online]. Available: https://universe.roboflow.com/augmented-startups/weeds-nxe1w

J. A. O. S. Silva et al., “Deep Learning for Weed Detection and Segmentation in Agricultural Crops Using Images Captured by an Unmanned Aerial Vehicle,” Remote Sens., vol. 16, no. 23, 2024, doi: 10.3390/rs16234394.

M. Butt, N. Glas, J. Monsuur, R. Stoop, and A. de Keijzer, “Application of YOLOv8 and Detectron2 for Bullet Hole Detection and Score Calculation from Shooting Cards,” AI, vol. 5, no. 1, pp. 72–90, 2024, doi: 10.3390/ai5010005.

Q. Chen, Q. Xiong, H. Huang, and S. Tang, “An Efficient and Lightweight Surface Defect Detection Method for Micro-Motor Commutators in Complex Industrial Scenarios Based on the CLS-YOLO Network,” Electron., vol. 14, no. 3, pp. 1–26, 2025, doi: 10.3390/electronics14030505.

J. Li, M. Zheng, D. Dong, and X. Xie, “PC-CS-YOLO: High-Precision Obstacle Detection for Visually Impaired Safety,” Sensors, vol. 25, no. 2, pp. 1–15, 2025, doi: 10.3390/s25020534.

R. Khanam and M. Hussain, “YOLOv11: An Overview of the Key Architectural Enhancements,” vol. 2024, pp. 1–9, 2024, [Online]. Available: http://arxiv.org/abs/2410.17725

S. Kaya, B. Kiraz, and A. Y. Çamurcu, “Hyperparameter Optimization in Deep Learning-Based Object Detection of Branching and Endpoints on 2D Brain Vessel Images,” in 2024 Innovations in Intelligent Systems and Applications Conference (ASYU), Oct. 2024, pp. 1–6. doi: 10.1109/ASYU62119.2024.10757007.

O. V. Johnson, C. Xinying, K. W. Khaw, and M. H. Lee, “ps-CALR: Periodic-Shift Cosine Annealing Learning Rate for Deep Neural Networks,” IEEE Access, vol. 11, pp. 139171–139186, 2023, doi: 10.1109/ACCESS.2023.3340719.

T. Zhang, B. Zhu, Y. Ma, and X. Zhou, “A Novel Image Encryption Algorithm Based on Multiple Random DNA Coding and Annealing,” Electron., vol. 12, no. 3, 2023, doi: 10.3390/electronics12030501.

X. Guo et al., “Citrus Disease Detection Based on Dilated Reparam Feature Enhancement and Shared Parameter Head,” Sensors, vol. 25, no. 7, 2025, doi: 10.3390/s25071971.

L. H. Lai et al., “The Use of Machine Learning Models with Optuna in Disease Prediction,” Electron., vol. 13, no. 23, pp. 1–20, 2024, doi: 10.3390/electronics13234775.

D. Şahin, O. Torkul, M. Şişci, D. D. Diren, R. Yılmaz, and A. Kibar, “Real-Time Classification of Chicken Parts in the Packaging Process Using Object Detection Models Based on Deep Learning,” Processes, vol. 13, no. 4, 2025, doi: 10.3390/pr13041005.

L. Song, H. Deng, J. Han, and X. Gao, “Improved Aerial Surface Floating Object Detection and Classification Recognition Algorithm Based on YOLOv8n,” Sensors, vol. 25, no. 6, 2025, doi: 10.3390/s25061938.

J. Liu, X. Liu, H. Chen, and S. Luo, “MDD-YOLOv8: A Multi-Scale Object Detection Model Based on YOLOv8 for Synthetic Aperture Radar Images,” Appl. Sci., vol. 15, no. 4, 2025, doi: 10.3390/app15042239.

Downloads

Published

2025-10-25

Issue

Section

Artificial Intelligence

How to Cite

Candhy Fadhila Arsyad, Pulung Nurtantio Andono, & Moch Arief Soeleman. (2025). Hyperparameter Tuning with Optuna to optimize the YOLOv11n Model for Weed Detection. Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), 9(5), 1153-1162. https://doi.org/10.29207/resti.v9i5.6682