“Visual positioning is not a very new technology,” says Konrad Wenzel at ESRI, a company that develops digital mapping and geospatial analysis software. “But it’s clear that the more cameras we have, the better it will be.”
Niantic Spatial has trained its model on 30 billion images captured in urban environments. Specifically, the images are clustered around hot spots – locations that served as important locations in Niantic’s games that players were encouraged to visit, such as Pokémon battle arenas. “We have over a million locations around the world where we can pinpoint where you are,” says McLendon. “We know where you are standing to within several centimeters of accuracy and, most importantly, where you are looking.”
The result is that for each of those millions of locations, Niantic Spatial has thousands of images taken in more or less the same location, but from different angles, at different times of the day, and in different weather conditions. Each of those images comes with detailed metadata that indicates where the phone was in space at the time the image was captured, including what direction the phone was facing, whether it was moving or not, how fast and in what direction, and much more.
The firm has used this data set to train a model to accurately predict where it is, taking into account what it is seeing – even for locations other than those millions of hot spots, where good sources of image and location data are scarce.
In addition to GPS, Coco’s robots, which are equipped with four cameras, will now use this model to try to figure out where they are and where they are going. The robots’ cameras are hip-height and directed in all directions simultaneously, says Rash, so their perspective is a little different from that of a Pokémon Go player, but optimizing the data was straightforward.
Rival companies also use visual positioning systems. For example, Starship Technologies, a robot delivery firm founded in Estonia in 2014, says its robots use their sensors to create a 3D map of their surroundings, drawing the sides of buildings and the position of streetlights.
But Rash is betting that Niantic Spatial’s technology will give Coco the edge. They claim this will allow their robots to position themselves at the right pickup locations outside the restaurant, ensuring they don’t get in anyone’s way, and stopping right outside the customer’s door instead of a few steps away, which would have been the case in the past.
Cambrian explosion in robotics
When Niantic Spatial began work on its visual positioning system, the idea was to apply it to augmented reality, says Hanke. “If you’re wearing AR glasses and you want the world to be limited to where you’re looking, you need some method to do that,” he says. “But now we’re seeing a Cambrian explosion in robotics.”
