Wheat spikelet detection on RGB images using deep machine learning

This study addresses the challenge of automated high-throughput phenotyping of wheat spike characteristics using modern computer vision and deep learning methods. Accurate estimation of spikelet number is a key indicator of plant productivity, yet traditional manual counting approaches are labor-intensive, slow, and difficult to scale to large breeding datasets. To overcome these limitations, we propose a spikelet detection strategy based on simplified point annotations, where an expert marks only the centers of spikelets rather than drawing detailed segmentation masks or bounding boxes. This significantly reduces annotation time and lowers the overall cost of preparing training datasets for machine learning models. To determine the most effective way of utilizing such simplified annotations, three computational methods were explored: segmentation of binary masks using a U-Net architecture, density regression based on two-dimensional Gaussian distributions optimized via Kullback–Leibler divergence, and detection of fixed-size bounding regions using the YOLOv8 object detection framework. The models were evaluated on dedicated test datasets using both quantitative metrics (MAE, MAPE) and spatial localization metrics (Precision, Recall, F1 score). The results demonstrate that U-Net-based approaches provide consistently high accuracy in spikelet localization and counting while maintaining robustness to annotation imperfections. In contrast, the YOLOv8-based method showed reduced performance, likely due to the geometric mismatch between fixed-size boxes and the natural elongated shape of spikelets. Overall, the proposed methodology highlights the effectiveness of combining minimalistic point-level annotation with advanced segmentation models for automating phenotyping workflows. This approach has the potential to accelerate breeding programs, enhance the efficiency of largescale phenotypic data collection, and support further development of robust computer-vision tools for plant science applications.

1. Afonnikov D.A., Genaev M.A., Doroshkov A.V., Komyshev E.G., Pshenichnikova T.A. Methods of high-throughput plant phenotyping for large-scale breeding and genetic experiments. Russ J Genet. 2016;52(7):688-701. doi 10.1134/S1022795416070024

2. Akiba T., Sano S., Yanase T., Ohta T., Koyama M. Optuna: a nextgeneration hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA. ACM, 2019;2623-2631. doi 10.1145/3292500.3330701

3. Alkhudaydi T., Zhou J., De La Iglesia B. SpikeletFCN: counting spikelets from infield wheat crop images using fully convolutional networks. In: Artificial Intelligence and Soft Computing. ICAISC 2019. Lecture Notes in Computer Science. Vol. 11508. Springer, Cham., 2019;3-13. doi 10.1007/978-3-030-20912-4_1

4. Artemenko N.V., Epifanov R.U., Genaev M.A., Komyshev E.G., Kruchinina Yu.V., Koval V.S., Goncharov N.P., Afonnikov D.A. Image-based classification of wheat spikes by glume pubescence using convolutional neural networks. Front Plant Sci. 2024;14: 1336192. doi 10.3389/fpls.2023.1336192

5. Buslaev A., Iglovikov V.I., Khvedchenya E., Parinov A., Druzhinin M., Kalinin A. Albumentations: fast and flexible image augmentations. Information. 2020;11(2):125. doi 10.3390/info11020125

6. Chen F., Pound M.P., French A.P. Learning to localise and count with incomplete dot-annotations. In: 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW). Montreal, BC, Canada. IEEE, 2021;1612-1620. doi 10.1109/ICCVW54120.2021.00186

7. Deng J., Dong W., Socher R., Li L.J., Li K., Fei-Fei L. ImageNet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009, Miami. IEEE, 2009;248-255. doi 10.1109/CVPR.2009.5206848

8. Efimov V.М., Rechkin D.V., Goncharov N.P. Multivariate analysis of long-term climate data in connection with yield, earliness and the problem of global warming. Vavilovskii Zhurnal Genetiki i Selektsii = Vavilov J Genet Breed. 2024;28(2):155-165. doi 10.18699/vjgb-24-18

9. Gao S.-H., Cheng M.-M., Zhao K., Zhang X.-Y., Yang M.-H., Torr P. Res2Net: a new multi-scale backbone architecture. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2021;43(2): 652-662. doi 10.1109/TPAMI.2019.2938758

10. Genaev M.A., Komyshev E.G., Khao F., Koval V.S., Goncharov N.P., Afonnikov D.A. SpikeDroidDB: an information system for annotation of morphometric characteristics of wheat spike. Vavilovskii Zhurnal Genetiki i Selektsii = Vavilov J Genet Breed. 2018;22(1):132- 140. doi 10.18699/VJ18.340 (in Russian)

11. Genaev M.A., Komyshev E.G., Smirnov N.V., Kruchinina Y.V., Goncharov N.P., Afonnikov D.A. Morphometry of the wheat spike by analyzing 2D images. Agronomy. 2019;9(7):390. doi 10.3390/agronomy9070390

12. Khoroshevsky F., Khoroshevsky S., Bar-Hillel A. Parts-per-object count in agricultural images: solving phenotyping problems via a single deep neural network. Remote Sens. 2021;13(13):2496. doi 10.3390/rs13132496

13. Konopatskaia I., Vavilova V., Blinov A., Goncharov N.P. Spike morphology genes in wheat species (Triticum L.). Proc Latv Acad Sci B Nat Exact Appl Sci. 2016;70(6):345-355. doi 10.1515/prolas2016-0053

14. Li Q., Cai J., Berger B., Okamoto M., Miklavcic S.J. Detecting spikes of wheat plants using neural networks with Laws texture energy. Plant Methods. 2017;13:83. doi 10.1186/s13007-017-0231-1

15. Liu T., Chen W., Wang Y., Wu W., Sun C., Ding J., Guo W. Rice and wheat grain counting method and software development based on Android system. Comput Electron Agric. 2017;141:302-309. doi 10.1016/j.compag.2017.08.011

16. Maslova G., Abdryayev M.R., Sharapov I.I., Sharapova Ju.A. Correlation analysis of yield and elements of productivity of winter soft wheat varieties in the arid conditions of steppe zone of Middle Volga region. Izvestiya of Samara Scientific Center of the Russian Academy of Sciences. 2018;20(2-4):680-683. doi 10.24411/1990-5378-2018-00153 (in Russian)

17. Misra T., Arora A., Marwaha S., Chinnusamy V., Rao A., Jain R., Narayan R., Ray M., Kumar S., Raju D., Ranjan R., Nigam A., Goel S. SpikeSegNet – a deep learning approach utilizing encoder-decoder network with hourglass for spike segmentation and counting in wheat plant from visual imaging. Plant Methods. 2020;16:40. doi 10.1186/s13007-020-00582-9

18. Qiu R., He Y., Zhang M. Automatic detection and counting of wheat spikelet using semi-automatic labeling and deep learning. Front Plant Sci. 2022;13:872555. doi 10.3389/fpls.2022.872555

19. Radosavovic I., Kosaraju R.P., Girshick R., He K., Dollár P. Designing network design spaces. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, Seattle, WA, USA. IEEE, 2020;10428-10436. doi 10.1109/CVPR42600.2020.01044

20. Redmon J., Divvala S., Girshick R., Farhadi A. You Only Look Once: unified, real-time object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016, Las Vegas, NV, USA. IEEE, 2016;779-788. doi 10.1109/CVPR.2016.91

21. Ronneberger O., Fischer P., Brox T. U-Net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Lecture Notes in Computer Science. Vol. 9351. Springer, 2015;234-241. doi 10.1007/978-3-319-24574-4_28

22. Savin V.Yu. Determination of the force required for stripping the wheat ear. Inzhenernyye Tekhnologii i Sistemy = Engineering Technologies and Systems. 2019;29(3):456-466. doi 10.15507/2658-4123.029.201903.456-466 (in Russian)

23. Schneider C.A., Rasband W.S., Eliceiri K.W. NIH Image to ImageJ: 25 years of image analysis. Nat Methods. 2012;9(7):671-675. doi 10.1038/nmeth.2089

24. Shi L., Sun J., Dang Y., Zhang S., Sun X., Xi L., Wang J. YOLOv5s-T: a lightweight small object detection method for wheat spikelet counting. Agriculture. 2023;13(4):872. doi 10.3390/agriculture13040872

25. Skripka O.V., Samofalov A.P., Podgornyî S.V., Gromova S.N. Productivity and main elements of productivity of winter wheat varieties of intensive type, bred in the All-Russian Research Institute of Grain Crops. Dostizheniya Nauki i Tekhniki APK = Achievements of Science and Technology in Agro-Industrial Complex. 2016;30(9): 30-32 (in Russian)

26. Tan M., Le Q.V. EfficientNet: rethinking model scaling for convolutional neural networks. In: Proceedings of the 36th International Conference on Machine Learning, ICML 2019, Long Beach, California, USA. ICML, 2019;6105-6114

27. Vahamidis P., Karamanos A.J., Economou G. Grain number determination in durum wheat as affected by drought stress: an analysis at spike and spikelet level. Ann Appl Biol. 2019;174(2):190-208. doi 10.1111/aab.12487

28. Vavilova V., Konopatskaia I., Kuznetsova A.E., Blinov A., Goncharov N.P. Genomic characterization of DEP1 gene in wheats with normal and compact spike shape. BMC Genetics. 2017;18 (Suppl.1): 106. doi 10.1186/s12863-017-0583-6

29. Vavilova V.Yu., Konopatskaia I.D., Blinov A.G., Goncharov N.P. Interspecific polymorphism of DEP1 genes and the spike shape in wheats. Russ J Genet. 2019;55(7):908-913. doi 10.1134/S1022795419070147

30. Wightman R. PyTorch Image Models (timm). GitHub Repository. 2019. doi 10.5281/zenodo.4414861

31. Xie E., Wang W., Yu Z., Anandkumar A., Alvarez J., Luo P. SegFormer: simple and efficient design for semantic segmentation with transformers. arXiv. 2021. doi 10.48550/arXiv.2105.15203

32. Zhang B., Liu X., Xu W., Chang J., Li A., Mao X., Zhang X., Jing R. Novel function of a putative MOC1 ortholog associated with spikelet number per spike in common wheat. Sci Rep. 2015;5(1):12211. doi 10.1038/srep12211

33. Zhang H., Wu C., Zhang Z., Zhu Y., Zhang Z., Lin H., Sun Y., He T., Mueller J., Manmatha R., Li M., Smola A. ResNeSt: split-attention networks. arXiv. 2020. doi 10.48550/arXiv.2004.08955

34. Zhou H., Riche A.B., Hawkesford M.J., Whalley W.R., Atkinson B.S., Sturrock C.J., Mooney S.J. Determination of wheat spike and spikelet architecture and grain traits using X-ray Computed Tomography imaging. Plant Methods. 2021;17(1):26. doi 10.1186/s13007-021-00726-5