You'd be forgiven for dismissing jumping spiders as pests, or at best the subject of an amusing macro shot. But it turns out that these tiny arachnids, with brains no bigger than a poppy seed, have been quietly solving one of the trickiest problems in imaging for millions of years: how to judge depth accurately and efficiently. Now, engineers at Northwestern University in Illinois have taken note.
The resulting device, called SpiderCam, is a 3D depth-sensing camera that produces real-time depth maps while consuming less than a watt of power. That's less energy than the nightlight on your landing. For photographers and camera engineers accustomed to thinking about power budgets in terms of battery life per shoot or drone flight time, that number is pretty striking.
What makes the achievement really interesting, though, is the elegance of the underlying idea. Rather than reverse-engineering some exotic biological mechanism and spending years trying to replicate it in silicon, the Northwestern team spotted a principle that was hiding in plain sight, in the optics of a creature most of us have been swatting away since childhood.
How spiders see
Jumping spiders have a problem any photographer will recognise. They need sharp, precise depth information to make accurate leaps, either to catch prey or to avoid becoming it. But they're running this entire visual pipeline on what amounts to almost zero processing power.
Their solution is built into the structure of their eyes. Where a human eye has a single retina, jumping spiders have multiple retinal layers in each eye, each focused at a slightly different distance. One layer captures an object sharply; another captures the same object with slight blur. The brain compares those two images, and the difference in sharpness tells the spider exactly how far away the object is.
It's basically the same principle as the defocus cues photographers know well (a shallow depth of field tells you a lot about the spatial relationship between subject and background). Jumping spiders have simply evolved to exploit this systematically, in both eyes, all the time.
How it works
SpiderCam mimics this approach directly. A custom camera simultaneously captures two images of the same scene with slightly different focus settings. A custom algorithm then analyses how sharpness changes between those two images, particularly around edges and textures, and converts those differences into depth measurements in real time.
Rather than running that algorithm on a conventional processor, the team embedded it directly into a low-power FPGA; a type of customisable computer chip optimised for efficient processing. The resulting prototype generates depth maps at 32.5 frames per second while drawing just 624 milliwatts. The researchers say it's the first passive FPGA-based 3D camera to operate below one watt.
Conventional depth cameras typically work in one of two ways: either by projecting structured light or infrared patterns onto a scene and measuring the return, or by comparing images from two spatially separated viewpoints. Both approaches work well, but they come with power costs, hardware costs and bulk that rule them out for the smallest and most battery-constrained applications.
Practical uses
The obvious applications for this breakthrough are drones, small robots and wearables, devices where every milliwatt counts and where conventional depth sensors are simply too hungry. But Emma Alexander, the assistant professor of computer science who led the research, has another idea. "I think it's particularly exciting for applications like augmented reality," she says, 'where you're interfacing with the physical world and need to know the locations of objects around you."
She's right, of course: precise, real-time depth sensing is fundamental to AR overlays that actually sit convincingly in a scene, rather than floating oddly in front of it. Current AR headsets wrestle constantly with the power demands of their depth sensors. A system that could deliver similar results at a fraction of the power would be a great achievement.
Right now, the team plans to improve the optics of the SpiderCam, broaden the field of view, and eventually replace the FPGA with a custom chip that could cut power consumption further still. Not bad for a spider with a brain the size of a poppy seed.
.jpeg?w=600)
