New Lensless Camera Captures 3D Depth Photos in a Single Exposure

Lensless Camera

Two researchers from the University of California, Davis have created a camera that uses a microlens array and new image processing algorithms to capture 3D photos in a single exposure.

Weijian Yang, Ph.D., and doctoral student Feng Tian say that because the camera is able to learn from existing data on how to digitally reconstruct a 3D scene it can produce these images in real-time, Phys reports. Yang states that the camera is lensless as it replaces bulky lenses in traditional cameras with an array of microlens made from flexible polymer.

“Because each microlens can observe objects from different viewing angles, it can accomplish complex imaging tasks such as acquiring 3D information from objects partially obscured by objects closer to the camera.”

The duo say that the camera is especially useful for giving robots 3D vision, which would in turn allow them to better navigate spaces or perform complex tasks such as manipulating small or fine objects.

“It could also be used to acquire rich 3D information that could provide content for 3D displays used in gaming, entertainment or many other applications,” Yang adds.

The microlens array is made up of 37 smaller lenses that are distributed in an area just 12 millimeters in diameter. The array takes a complete scene, and then the “simple yet robust” deep learning model that is physics aware depth-resolves photos in real time.

3d imaging experiment
3D imaging experiment. A cell phone camera captured an image of objects arranged at different distances. (b) Captured image by the lensless 3D camera using microlens array. (c)-(e), untrained, trained and enhanced reconstructions at multiple refocused distances. (f-(h), zoom in views starting from the box with a white dashed line (c) to (e). The refocused distances are labeled in panel (c). Ginsey Home Solutions manufactured the toy figurines. Images of the toy figurines were published with permission from Nick.com.

“Our lensless imager does not require calibrating the point spread function and has the capability to resolve depth and ‘see-through’ opaque obstacles to image features being blocked, enabling broad applications in computational imaging,” the research paper, published on Optica, explains.

The resulting images can be refocused to different depths and distances and the camera can also create a depth map that the researchers confirm is in line with the actual arrangement of test objects.

In a final demonstration, we demonstrated that the camera can image objects behind opaque obstacles,” Yang said. “To the best of our knowledge, this is the first demonstration of imaging objects behind opaque obstacles using a lensless camera.”

The two researchers are currently refining the system to reduce artifacts and errors and will then work on miniaturizing the system to make it small enough to fit in a smartphone, greatly expanding its usability.

The full research paper titled Learned lensless 3D camera can be read on Optica.


Image credits: Left: Weijian Yang, Ph.D., and Feng Tian developed a camera that uses a thin microlens array and new image processing algorithms to capture 3D information about multiple objects in single exposure. On the monitor, the raw sub-images of the microarray can be seen. Photo: Savannah Luy (University of California–Davis). | Right: “See through obstacles” experiment using the lensless camera via Learned lensless 3D camera.

Loading...