Self-learning virtual organisms in a physics simulator: on the optimal resolution of their visual system, the architecture of the nervous system and the computational complexity of the problem

   Vision plays a key role in the lives of various organisms, enabling spatial orientation, foraging, predator avoidance and social interaction. In species with relatively simple visual systems, such as insects, effective behavioral strategies are achieved through high neural specialization, adaptation to specific environmental conditions, and the use of additional sensory systems such as olfaction or hearing. Animals with more complex vision and nervous systems, such as mammals, have greater cognitive abilities and flexibility, but this comes with increased demands on the brain’s energy costs and computational resources. Modeling the features of such systems in a virtual environment could allow researchers to explore the fundamental principles of sensorimotor integration and the limits of cognitive complexity, as well as test hypotheses about the interaction between perception, memory and decision-making mechanisms.

   In this work, we implement and investigate a model of virtual organisms with a visual system operating in a three-dimensional physical environment using the Unity ML-Agents software – one of the most high-performance simulation platforms currently available.

   We propose a hierarchical control architecture that separates locomotion and navigation tasks between two modules: (1) visual perception and decision-making, and (2) coordinated control of limb movement for locomotion in the physical environment. A series of numerical experiments was conducted to examine the influence of visual system parameters (e. g, resolution of the “first-person” view), environmental configuration and agent architectural features on the efficiency and outcomes of reinforcement learning (using the PPO algorithm). The results demonstrate the existence of an optimal range of resolutions that provide a trade-off between computational complexity and success in accomplishing the task, while excessive dimensionality of sensory inputs or action space leads to slower learning. We performed system performance profiling and identified key bottlenecks in large-scale simulations. The discussion considers biological parallels, highlighting cases of high behavioral efficiency in insects with relatively low-resolution visual systems, and the potential of neuroevolutionary approaches for adapting agent architectures. The proposed approach and the results obtained are of potential interest to researchers working on biologically inspired artificial agents, evolutionary modeling, and the study of cognitive processes in artificial systems.

1. Aksoy V., Camlitepe Y. Spectral sensitivities of ants – a review. Anim Biol. 2018;68(1):55­73. doi: 10.1163/15707563­17000119

2. Attwell D., Laughlin S.B. An energy budget for signaling in the grey matter of the brain. J Cereb Blood Flow Metab. 2001;21(10):1133­-1145. doi: 10.1097/00004647­200110000­0000

3. Bongard J.C. Evolutionary robotics. Commun ACM. 2013;56(8): 74­83. doi: 10.1145/2493883

4. Botvinick M., Wang J.X., Dabney W., Miller K.J., Kurth-Nelson Z. Deep reinforcement learning and its neuroscientific implications. Neuron. 2020;107(4):603­616. doi: 10.1016/j.neuron.2020.06.014

5. Cammaerts M.C. The visual perception of the ant Myrmica ruginodis (Hymenoptera: Formicidae). Biologia. 2012;67(6):1165­1174. doi: 10.2478/s11756­012­0112­z

6. Cammaerts M.­C.T., Cammaerts R. Are ants (Hymenoptera, Formicidae) capable of self recognition? J Sci. 2015;5(7):521­532

7. Chittka L., Muller H. Learning, specialization, efficiency and task allocation in social insects. Commun Integr Biol. 2009;2(2):151­154. doi: 10.4161/cib.7600

8. Chittka L., Niven J. Are bigger brains better? Curr Biol. 2009;19(21): R995­R1008. doi: 10.1016/j.cub.2009.08.023

9. Devyaterikov А.P., Palyanov А.Y. A software system for modeling evolution in a population of organisms with vision, interacting with each other in 3D simulator. Vavilov J Genet Breed. 2022;26(8): 780­786. doi: 10.18699/VJGB­22­94

10. Hassabis D., Humaran D., Summerfield C., Botvinick M. Neuroscience-inspired artificial intelligence. Neuron. 2017;95(2):245­258. doi: 10.1016/j.neuron.2017.06.011

11. Juliani A., Berges V.­P., Teng E., Cohen A., Harper J., Elion C., Goy C., Gao Y., Henry H., Matter M., Lange D. Unity: A general platform for intelligent agents. arXiv. 2018. doi: 10.48550/arXiv.1809.02627

12. Land M.F., Nilsson D.­E. Animal Eyes. Oxford University Press, 2012. Available: https://www.softouch.on.ca/kb/data/Animal%20Eyes.pdf

13. Menzel R. The honeybee as a model for understanding the basis of cognition. Nat Rev Neurosci. 2012;13(11):758­768. doi: 10.1038/nrn3357

14. Niven J.E., Laughlin S.B. Energy limitation as a selective pressure on the evolution of sensory systems. J Exp Biol. 2008;211:1792­1804. doi: 10.1242/jeb.017574

15. O’shea K., Nash R. An introduction to convolutional neural networks. arXiv. 2015. doi: 10.48550/arXiv.1511.08458

16. Peng P., Wen Y., Yang Y., Yuan Q., Tang Z., Long H., Wang J. Multiagent bidirectionally­coordinated nets: Emergence of human­level coordination in learning to play StarCraft combat games. arXiv. 2017. doi: 10.48550/arXiv.1703.10069

17. Schulman J., Wolski F., Dhariwal P., Radford A., Klimov O. Proximal policy optimization algorithms. arXiv. 2017. doi: 10.48550/arXiv.1707.06347

18. Song K.-Y., Behzadfar M., Zhang W.-J. A dynamic pole motion approach for control of nonlinear hybrid soft legs: A preliminary study. Machines. 2022;10(10):875. doi: 10.3390/machines10100875

19. Stanley K.O., Clune J., Lehman J., Miikkulainen R. Designing neural networks through neuroevolution. Nat Mach Intell. 2019;1(1): 24­35. doi: 10.1038/s42256­018­0006­z

20. Tschantz A., Anil K.S., Christopher L.B. Learning action-oriented models through active inference. PLoS Comput Biol. 2020;16(4): e1007805. doi: 10.1371/journal.pcbi.1007805

21. Vezhnevets A.S., Osindero S., Schaul T., Heess N., Jaderberg M., Silver D., Kavukcuoglu K. FeUdal networks for hierarchical reinforcement learning. In: Proceedings of the 34^th International Conference on Machine Learning. Vol. 70 (ICML’17). JMLR.org, 2017;3540-3549

22. Zador A.M. A critique of pure learning and what artificial neural networks can learn from animal brains. Nat Commun. 2019;10(1):3770. doi: 10.1038/s41467­019­11786­6