New Imaging System Creates Pictures By Measuring Time (phys.org) 63
An anonymous reader writes: Photos and videos are usually produced by capturing photons -- the building blocks of light—with digital sensors. For instance, digital cameras consist of millions of pixels that form images by detecting the intensity and color of the light at every point of space. 3-D images can then be generated either by positioning two or more cameras around the subject to photograph it from multiple angles, or by using streams of photons to scan the scene and reconstruct it in three dimensions. Either way, an image is only built by gathering spatial information of the scene. In a new paper published today in the journal Optica, researchers based in the U.K., Italy and the Netherlands describe an entirely new way to make animated 3-D images: by capturing temporal information about photons instead of their spatial coordinates.
Their process begins with a simple, inexpensive single-point detector tuned to act as a kind of stopwatch for photons. Unlike cameras, measuring the spatial distribution of color and intensity, the detector only records how long it takes the photons produced by a split-second pulse of laser light to bounce off each object in any given scene and reach the sensor. The further away an object is, the longer it will take each reflected photon to reach the sensor. The information about the timings of each photon reflected in the scene -- what the researchers call the temporal data -- is collected in a very simple graph.
Those graphs are then transformed into a 3-D image with the help of a sophisticated neural network algorithm. The researchers trained the algorithm by showing it thousands of conventional photos of the team moving and carrying objects around the lab, alongside temporal data captured by the single-point detector at the same time. Eventually, the network had learned enough about how the temporal data corresponded with the photos that it was capable of creating highly accurate images from the temporal data alone. In the proof-of-principle experiments, the team managed to construct moving images at about 10 frames per second from the temporal data, although the hardware and algorithm used has the potential to produce thousands of images per second. Currently, the neural net's ability to create images is limited to what it has been trained to pick out from the temporal data of scenes created by the researchers. However, with further training and even by using more advanced algorithms, it could learn to visualize a varied range of scenes, widening its potential applications in real-world situations.
Their process begins with a simple, inexpensive single-point detector tuned to act as a kind of stopwatch for photons. Unlike cameras, measuring the spatial distribution of color and intensity, the detector only records how long it takes the photons produced by a split-second pulse of laser light to bounce off each object in any given scene and reach the sensor. The further away an object is, the longer it will take each reflected photon to reach the sensor. The information about the timings of each photon reflected in the scene -- what the researchers call the temporal data -- is collected in a very simple graph.
Those graphs are then transformed into a 3-D image with the help of a sophisticated neural network algorithm. The researchers trained the algorithm by showing it thousands of conventional photos of the team moving and carrying objects around the lab, alongside temporal data captured by the single-point detector at the same time. Eventually, the network had learned enough about how the temporal data corresponded with the photos that it was capable of creating highly accurate images from the temporal data alone. In the proof-of-principle experiments, the team managed to construct moving images at about 10 frames per second from the temporal data, although the hardware and algorithm used has the potential to produce thousands of images per second. Currently, the neural net's ability to create images is limited to what it has been trained to pick out from the temporal data of scenes created by the researchers. However, with further training and even by using more advanced algorithms, it could learn to visualize a varied range of scenes, widening its potential applications in real-world situations.
Looks like somebody has rediscovered LIDAR (Score:5, Informative)
The idea is about 60 years old.
whats unique ? (Score:1)
honestly this looks like LIDAR :
pulse of laser light to bounce off each object in any given scene and reach the sensor. The further away an object is, the longer it will take each reflected photon to reach the sensor.
whats unique in this compared to conventional LIDAR ?
Re: (Score:3)
whats unique in this compared to conventional LIDAR ?
It's significantly less useful:
Currently, the neural net's ability to create images is limited to what it has been trained to pick out from the temporal data of scenes created by the researchers.
Re: (Score:2)
I'm guessing that this method, once refined, will allow for better results from a limited data set.
Or not, I'm just guessing, but that would make this novel compared to LIDAR.
Re: (Score:2)
single point and ai trained.. but.. (Score:3)
it's just a single flash and recording a stream of returning photons.
then using some algorithm to turn that into a depth map? which I guess was trained using sample sets? I mean it could distinguish a face in front of it to be a face from a banana in front of it, that I can imagine rather easily from the data it gathers. but I'm kinda confused how it can even with ai know the orientation of the banana if the summary is correct at all, because if it's a single flash of light and a single point sensor (the s
Re: (Score:2)
Re:single point and ai trained.. but.. (Score:5, Interesting)
Re: (Score:2)
Except, it doesn't seem that you created an image. It seems that you mapped out that there are a set of surfaces, each at various distances from the surfaces. Then you used "AI" (ie, glorified filter) to categorize the "surface distance" readings into categories to matched known physical configurations of people and things.
With a picture, if you replaced the person with a dog, a person viewing the output would be able to determine that it is a dog. This is not a "picture". No one would be able to deter
Re: (Score:2)
whats unique in this compared to conventional LIDAR ?
They designed it like a fish finder, and then instead of programming the formulas, they trained an AI, so it is crappier but easier to make.
Re: (Score:2)
They stuck some crummy DeepFake for home furnishings on top of the LIDAR
It is as if they stuck pictures of Billy Bass over the fish finder output and claimed they captured the image
Re: (Score:2)
DeepFake is harder than what they did, it requires a lot of fiddling in practice.
Re: (Score:3)
In lidar, the scene is illuminated by a scanner (a laser beam that moves up & down, back & forth) which lights up the one tiny area of the scene (one pixel) at a time. With their approach the whole scene is illuminated at the same time with wide flashes of light.
Re: (Score:2)
Re: (Score:2)
It seems that I was a little quick on what it does. A second look seems to show that it uses LIDAR data to 'train an AI' to recognize individual objects.
Offhand of course, I'd claim it was more incremental than revolutionary, but could see how this could be useful for self driving cars. By not relying on video, it could reduce some privacy issues (some) and be useful in a number of applications.
Re: (Score:2)
Re: (Score:2)
Not quite. You don't get an image. You get a graph of photons returned per time period. You now know that there are surfaces out there and particular distances, but you don't know what directions the surfaces are.
Re: (Score:2)
Without RTFA, I suspect the difference here is that they're doing it with a single pulse of light. LIDAR is typically scanned (so are CHIRP radars and sonars). You send a laser out in a certain direction, measure the time for the return signal, rotate a bit and send the laser out in the new direction, etc. to build up a 3D map. If you have a phased array of receivers (and a ton of processing power), you can send out
Re: (Score:3)
First off it should be obvious that's not the case given that it's published in a decent journal AND it's a given that everyone has heard of Lidar. Second, if you bother to read the article (instead of relying on the rather misleading description) you'll see that their method doesn't work the same way as LIDAR.
Re:Looks like somebody has rediscovered LIDAR (Score:5, Insightful)
This does seem to be novel and not simply just another LIDAR.
Current LIDAR systems using a single point detector scan the environment in some way, usually by having the light pulses scan over the area so that one vector can be measured at a time. That's why you see spinning optics on LIDAR systems, they are scanning the laser pulses both horizontally and vertically. An alternative is to use a series of special patterns that when combined allow you to get a value for each vector.
These guys have found a way to eliminate the scanning. They flood illuminate the scene and then make a temporal histogram from the returns. Using AI that is trained to recognize what the histograms of various different objects and poses look like they can turn it into an image.
That could be very useful for things like industrial machine vision where you want to check if something has been manufactured or installed correctly, for example. They are also looking at possible consumer applications such as adding 3D data to smartphone cameras.
Re: (Score:2)
What you describe is a "LIDAR scanner". LIDAR is in principle just the ranging idea.
The usual bastardization of the language at work.
Re: (Score:2)
This technique receives light from the entire scene at the same time- there is no directional information, just a histogram of depth. The AI reconstructs the directional information from the depth histrogram. The potential here is amazing.
This could produce color as well by using 3 detectors with RGB filters.
Re: (Score:2)
Sure. If LIDAR used an omnidirectional light source instead of a laser.
To be honest, LIDAR is a dumb idea because the Phonecians invented eyeballs in 2500 BCE.
Not the same as LiDAR (Score:5, Informative)
The summary is a bit obscure on the point, but the article is clear (and open-access): whereas LiDAR uses a scanned beam and time-of-flight to develop a 3D image of a scene, this research uses pulsed flood illumination, not unlike a camera flash. That is to say, there is no raster scanning of the scene. They collect no spatial information but are able to reconstruct the scene structure just by time-of-flight information alone. In contrast, LiDAR, with a scanned beam controlled by the system at all times (just like RADAR), uses the position of the beam as a critical part of 3D reconstruction. This research is different, and because of that is pretty impressive. They also specifically mention LiDAR in their article as a means to develop ground truth for the training algorithm, so it isn't like they don't know about what LiDAR is and how it works. This research is closer to scene reconstruction through echo-location, but with light.
Re: (Score:1)
Not exactly a new discovery (Score:5, Funny)
Having worked to develop several Ag Tech lidar products over the last decade, I can assure you that this is not "new" technology. The price keeps coming down, and the quality keeps going up, but the technology has been out there for decades. It's the heart of many of the collision detection systems in new cars, so you may actually already own a lidar sensor without being aware of it.
Re: (Score:2)
Re: (Score:2)
This works different than lidar. It doesn't have a scanning light source, instead it use a flashes of light that illuminates the whole scene simulataneously.
Re: (Score:1)
Re: (Score:1)
Time-of-flight cameras are nothing new. Wikipedia lists several companies that were selling these cameras in 2011:
https://en.wikipedia.org/wiki/... [wikipedia.org]
Re: (Score:2)
This works different than lidar, it doesn't have a scanning light source, instead it use a flashes of light that illuminates the whole scene simulataneously.
Re: (Score:2)
I doubt this particular technology has existed for decades as it's only fairly recently that AI systems have got good enough to do this kind of scene/object recognition with the accuracy and in the time frames they are talking about.
Also all car anti-collision systems use radar, not LIDAR. Well, a few of the experimental self-driving systems use LIDAR, but all consumer cars use radar. The first LIDAR equipped car will be released by Volvo next year.
Re: (Score:1)
Well, not decades, but this Dutch webpage from 2012 describes a machine using it:
https://www.mechaman.nl/veehou... [mechaman.nl]
optical sensors (Score:2)
Either way, an image is only built by gathering spatial information of the scene.
Except for holograms. They work by creating an interference pattern on/in the detector(film). Pedantic, I know, but let's be accurate.
Re: (Score:2)
Time of flight sensors usually do this as well. It's super expensive to actually time a light pulse, so most use interference between a transmitted and a reference pulse.
Hmm... (Score:4, Insightful)
What you're saying is they've invented a much shittier Kinect?
Re: (Score:1, Insightful)
So you don't say at the top, but I presume you're using VPN, anonymisation, etc. services.
Anything to do with banks / money: Money laundering laws simply do not allow what you want - an anonymous financial transaction. It's really that simple. It's illegal for the banks to facilitate them without knowing who their customer is. Now, association with Bitcoin is particularly worrisome because "the government knows about that", in effect. They don't want to go near it, in case they get flagged as dealing i
On the AI reconstruction (Score:2)
But I don't know what to think of the image reconstruction through AI. All they have is the distance data. As they explain in their publication any object in the same distance gives the same signal.
This means of the whole scene is rotated it would produce the same signal. Same if parts of the scene in a certain angle range are.
This seems to be just too limited data to determine what is in front of the detector. A lo
Re: On the AI reconstruction (Score:2)
Re:On the AI reconstruction (Score:4, Interesting)
It is not LIDAR, but not entirely new either (Score:1)
I'm not expert with this technology, but I try to share what I know. Please correct me if you see any mistakes.
With LIDAR, you shoot multiple lasers to different locations. This is not LIDAR.
With this technology you first need to collect background 3D data, e.g. with LIDAR. Then you shoot pulses or light, which you collect from single point and with this to my understanding you can create a 3D movie of moving people. With top equipment I think you can get into resolutions of 10 cm.
There is also older techno
Re: (Score:1)
It won't go anywhere (Score:3)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
So LIDAR+AI+An educated guess. (Score:1)
Time? (Score:1)
Re: (Score:2)
Conceptually interesting even without the AI (Score:1)
This sounds like a somewhat interesting problem. If you ignore the AI aspect and think about how you could solve for "what is in the room" from the timing frame data with direct mathematical/computational approaches, I can see how you could probably make some reasonable estimates as long as you are allowed to make a few assumptions.
For example:
Start with a mostly empty room, assume it is a normal "rectangular" room with 90 degree angles, etc, and that you can filter out / ignore "secondary" reflections
Similarity to synthetic aperture radar (Score:1)
It is also somewhat similar to the pysical setup for getting sythentic aperature radar (SAR) data: https://en.wikipedia.org/wiki/... [wikipedia.org] In the sense that it it is working with a single-dimensional signal of data.
But differences include:
This has its pros and cons (Score:2)
LIDAR actually captures a 3D image from its perspective.
This sounds like it captures where objects are and their orientation.
This Time Measurement system has an advantage where you can look at the picture from any angle and you will see any object that was in the original "picture" in full detail from that perspective.
With LIDAR the image quickly starts to fall apart if you are only imaging from 1 camera.
Even imaging from 2 ang