Scientists have reconstructed short films from the brain activity of rats who watched videos for a project that seeks to lift the curtain on how the animals see the world.
The brief movie clips are grainy and pixelated, but provide a glimpse of how the rats processed the footage, which shows people participating in a variety of sports from gymnastics to horse riding and wrestling.
The work is in its infancy, but as the technology advances, scientists hope to monitor a richer set of animals’ perceptions and eventually gain new insights into their experiences and how brains respond to their surroundings more broadly.
“The nice thing with humans is you can just ask someone what did you dream? What did you see? What are you hallucinating?” said Dr. Joel Bauer at University College London. “But with animals we don’t have that same access.”
The center of the work was an artificial intelligence program that won recent scientific competition To predict how electrical activity in the visual cortex of the mouse brain changes depending on what the animal is looking at. The visual cortex receives raw input from the retina and transforms it into a coherent view of the world.
To reconstruct what the rats were seeing, the scientists first used an infrared laser to record how neurons in the visual cortex were firing as the rodents watched a 10-second-long film clip. They then fed blank video data into an AI program and continually tweaked the imagery until the AI predicted the same patterns of brain activity seen in rats. Details are published in Journal eLife.
Rats have poorer eyesight than humans, so the reconstructed videos may never be as clear as the original videos. But as a rough estimate, Bauer suspects scientists could make the footage about seven times sharper than it currently is.
This is not the only area where there is room for improvement. The reconstructed video is essentially a pinhole view of the screen that the rats see, but future work could reconstruct the animal’s entire visual field, based on brain activity generated from information from both eyes individually.
While Bauer is excited about reading the minds of animals, he is more cautious about parallel work in humans. Several research groups are devising ways to reconstruct images and other perceptions from human brain scans. Ultimately, he said, this could lead to technologies that violate people’s privacy.
“The risk in humans would be that you might reconstruct not what they see, but what they imagine,” he said. “We don’t necessarily want to share everything that’s going on in our minds,” he said. “The privacy of our neural data is critical and will become more critical.”
As far as animals are concerned, he believes this approach could give scientists fundamental insights into how they experience the world, providing answers to questions such as what they see in dreams, whether they are fooled by optical illusions similar to humans, and even whether they hallucinate on magic mushrooms.
In the distant future, he said, it may be possible to reconstruct the rich sense of an animal’s experience as well as any kind of emotion, leading to “a very deep kind of empathy” between humans and other species. So can humans finally understand what it’s like to be a bat? “That would be nice,” Bauer said.
