LocustVR was jointly funded by The Department of Collective Behavior at MPI-AB and the Centre for the Advanced Study of Collective Behaviour. They approached us to create the worlds first, virtual reality system to study freely walking locusts. Using our experience creating sophisticated and unique VR systems for numerous species, we developed LocustVR.
LocustVR is a combination of a locomotion compensator sphere for freely moving animals, and a perspective correct virtual reality (VR) projection. The system simultaneously tracks the position of the locust as it walks on a large sphere, rotates the sphere to keep the locust always near the center of the arena, and projects a panoramic virtual reality from 3 projectors mounted above.
System and Specifications and Architecture
LocustVR consists of three large subsystems:
- Locomotion compensator
- Detection, tracking, and closed-loop control using Motif
- VR software
The 600mm diameter sphere was surface treated to increase its friction to make locomotion easier. It is controlled using 2 servo motors with rotary encoders (based on  and in conjunction with PhenoSys). The locomotion compensator acts as a spherical treadmill and can compensate for the movement of the subject in all directions - always returning the subject back to the center of the arena. This allows the locust to freely move without escaping from the experimental apparatus.
Detection, Tracking, and Closed-loop Control
Video capture and processing use our Motif software. Two cameras were present, a color camera for observation, and a 100fps NIR camera for tracking. Using Motif realtime processing support a contrast-based tracking method detected the center of mass and the orientation of the locust. After Kalman filtering, a control-law keeps the locust near the center of the sphere by adjusting the motor velocity. Simultaneously, the position of the animal on the sphere and the rotation of the sphere was also fed into the VR system.
3 x 120fps projectors project downwards onto a cylindrical projection surface (800mm diameter, 700mm high) and also onto the flat floor which surrounds the sphere. The overlap between projectors was corrected and blended together to achieve uniform brightness. The VR system is based on the FreemoVR engine . The virtual environment can be designed in Blender software and experimental logic is implemented in Python.
Introducing the Subject
LocustVR includes an air-supported hinged upper frame. This makes it easy to change subjects by simply lifting the lid and placing the locust on top of the sphere. As soon as the locust is detected, motion compensation is enabled and the lid can be closed.
In this video the locust explores a virtual world in an experiment to understand decision making (described in ). Locusts are highly attracted to the edges of objects so a virtual world featuring two identical vertical posts against a white background was designed in 3D modelling software. In this paradigm, as the locust approaches the two identical targets, the animal reaches a point at which it 'decides' which post to continue approaching.
By systematically studying the dynamics of this decision process as the relative angular position and the number of posts changes, the researchers hope to understand fundamental processes of how decisions are made.
Coverage in the Media
The LocustVR system has been featured in international media prominently. Have a look at the videos below to see the system in action, and to learn about how LocustVR is used to study collective behavior. Click on the image to be redirected to watch the video, the times where the LocustVR is featured in the video is indicated in parentheses below.