Brown University researchers paired an event camera with a spiking neural network to reconstruct and track moving, two-dimensional targets in controlled fog and turbid-water tests. The study first appeared in Advanced Science on Aug. 24; Brown announced the work on Oct. 1.
An event camera reports changes in pixel brightness rather than recording full image frames. In the researchers’ setup, target motion generated a stream of sparse events while the scattering background changed more slowly. A spiking neural network (SNN) then accumulated those events over time to estimate a target’s position and reconstruct its outline.
In a water-tank test, the team projected bird silhouettes through a 30-centimeter path of water clouded with Intralipid. On 800 sequences held out from training, the system scored 0.9593 on the structural similarity index (SSIM), a measure of how closely a reconstructed image matches its reference. In a separate fog-chamber test, it continued to track and reconstruct targets as fog density shifted during capture, the authors report.
The targets were projected two-dimensional silhouettes, not live birds or physical three-dimensional objects. The system currently recovers outlines rather than full grayscale images and needs a target to move relative to the camera. Rapid changes in the scattering medium can also generate events that interfere with the signal. Brown lists vehicle vision and rescue drones as possible future applications; the study did not test a vehicle or drone in the field.
Sources
Image credit: FrankBanalis / Wikimedia Commons, CC BY 4.0. Illustrative fog photograph; it does not show the research setup.
