Camera that Sees through Smoke and Fog Underway

tomschuring writes "The Age has a story about IATIA, who have been given $2.7 million by the Defence Department to fund development of a military spy camera capable of seeing through fog, smoke and dust storms. The technology uses a highly sophisticated camera that captures three images simultaneously through a single lens. Images thus resolved from between the particles making up fog, smoke, and dust storms are formed into a single picture of the hidden target."

Dense Camera Arrays for seeing through bushes

[vectra14]

These guys at stanford have done some really amazing stuff that's directly related. Except that they has literally dozens of cameras (as seen in their ppt), and their research seems to concentrate on multifocal image reconstruction (see ppt slides, presentation is quite good)

Link [stanford.edu] (has cool results links)

Keith Nugent

[metlin]

Hmm, Keith Nugent [iatia.com.au] is fairly well known in some niche areas of optics. If I remember right, his initial work on the use of x-rays and the like to compensate for normal visible hindrances were met with some opposition, but he is quite famous otherwise.

That was because, ironically, this was developed as a method to visualize biological stuff, and some felt that his methods would not quite be suitable for such a task. His ideas were to use various parameters such as phase, intensity and angle of vision to extract information which could be correlated and converge to recreate images with minimal amount of information, which later gained acceptance.

I guess he developed on that technique, and later on evolved to have the military to take notice. Interesting neverthless.

Article is short on details

[shoolz]

Much is left to the imagination in the article

I am imagining that since it not possible to "see" "through" an object, that these three images must be of various wavelengths (visible light, ultraviolet light, and infrared) and then are run through an interpolation process to get a probable image of what is behind the obstacle.

Am I out to lunch? Can anybody shed more light on how this works?

Hi-res TV stills

[whereiswaldo]

I was recently thinking about a technique which might be used for creating high definition stills of television programs.

The principle goes like this: you can get a view of an entire room with only a slit to look through. All you need to do is move back and forth to get the extra details.

So with the TV stills, you let the camera pan around a bit on a subject and capture all of the detail for each distinct area of the picture (eyes, whatever) since each of the raster lines on the tv are like the slit through the door. The camera panning around is like moving back and forth.

So under the right conditions like I've described, all the detail you want is there, but only when you take all the frames into consideration.

Re:Dense Camera Arrays for seeing through bushes

[ajna]

The Stanford work is actually entirely different. They utilize parallax -- in other words, their cameras are in physically distinct locations and see the scene with different perspectives. The IATIA work utilizes a single point of view, with images captured with the focal plane at the desired location and then slightly fore and aft. Read more here, at a Columbia site [columbia.edu].

Quantitative phase microscopy is a relatively new technique that can generate phase images and phase-amplitude images. In practice, to obtain a quantitative phase image one collects an in-focus image and very slightly positively and negatively defocused images, and uses these data to estimate the differential with respect to the defocus of the image. These images (a through-focal series) can be easily obtained in our system with our z-motion nano-positioner. The resulting data can be solved to yield the phase distribution by Fourier-transform methods. Results are obtained by essentially solving an optical transport equation. Significantly, the phase that is obtained does not have to be unwrapped, as is required for interferometry.

I'd be lying if I told you I completely understand the quoted paragraph, specifically what "essentially solving an optical transport equation" refers to, but I'm sure some cursory googling will lead the curious to specifics, certainly more than googling on terms in the article summary would yield.

I fought the law...

[Rendition]

Hope they can't make this work for speed cameras...

Re:Hi-res TV stills

[swb]

Doesn't changing angles on the subject (a result of moving the camera), cause you to collect not more but different data on the subject, resulting in a higher resolution image that's higher in angular/dimensional data, but not in 2D data?

It seems like you'd end up with a David Hockney-like image, not a higher resolution image.

Already exists

[leabre]

I was in the US Navy from 1994-1996 and the damage control teams already have a special camera (forget what it is called) that can see through dense smoke (the type you would expect from a jet fuel fire or amunition fire on a ship) and help you to see clearly through the smoke.

Wonder what makes the camera in this article so different from the technology the Navy already uses... I'm sure the current navy breed is much more advanced than it was 10 years ago.

Thanks,
Leabre

Parallax

[Anonymous Coward]

"The technology uses a highly sophisticated camera that captures three images simultaneously through a single lens. Images thus resolved from between the particles making up fog, smoke, and dust storms are formed into a single picture of the hidden target."

If it uses the concept of parallax, how can it possibly do this both using the same lense AND at the same time? Isn't parallax based on the concept of different images of overlapping fields of view? IR: two or more eyes/lenses or two or more images slightly timed apart if the object(s) in the foreground are moving?

If it's based on image analysis using different algorythms for three copies of the same original image, wouldn't it be liable to have errors? (Think of those optical illusions of inverted masks...) Or is the third one used to reduce/remove these errors?

my thoughts

[Large Bogon Collider]

I'm not a 100% sure, but the technique involves phase shift. As light of a single frequency passes though an medium, its phase is altered and light propagation is delayed. If you can computationally filter out all out of phase shift information caused by fog, for example, you can "see" what the hidden object looked like. This process is quite CPU intensive. It seemed that about a grayscale SVGA sized image (0.41 mp) took 1.5 secs on a PIV 2.4GHz to calculate. This should improve with algorith tweaking and using FPGAs.

This may also have medical applications in terms of optical imaging - see through the patient (arms and legs only, probably). Shine a bright light at the patient. Capture the ealiest photos that emerge (the ones that had a direct path to the camera). Ignore slow photons (ones that were absorbed and release or bounced around). Voila, instant imaging without x-rays. IIRC, this was in development years ago.

Better solution

[huge colin]

Just use an uncooled microbolometer-based infrared thermal imager. BAE Systems has been producing these for years. They're low-power, lightweight, and efficient.

When receiving this wavelength of IR, you can see through smoke, fog, some plastics (regardless of opacity to visible light), and independent of visible light levels. And seeing radiated heat is, of course, an obvious benefit. A fraction of a degree F is all that's needed to note a difference -- you can even see where things used to be because of the heat shadow they leave.

--Colin

Re:Dense Camera Arrays for seeing through bushes

[ajna]

You've basically said what I wrote elsewhere in this article's comments [slashdot.org]. We both agree that it would have limited utility for real life applications (as in not in fields where confocal microscopy gets people hot and bothered) in as many words.

Incidentally the IATIA link itself held the answer to my above musings, about what the transport equation actually is. I still don't understand it, but it can be viewed by one and all at the bottom of this page: http://www.iatia.com.au/technology/insideQpi.asp [iatia.com.au]

Re:Let's redesign the wheel!

[KD5YPT]

Actually is different, if you read the two article, you'll notice a major differences.

The one you stated is an infrared camera. Which means its only good at seeing objects that give off an appreciative amount of infrared radiation (in this case, runway lights, other planes, and etc).

The one stated in the main post is completely optical. It merely take three consecutive image and a computer compare the images and extract objects that are obscured by fog, dust, and such. Of course, this system would require that SOME visibility exist (in another word, no seeing through solid walls).

Alternate uses...

[d474]

Here are some good targets to test this camera out on, after all, these have some thick fog around them. Maybe we can get a clearer picture of:

1. CBS - the "true" source of the forged documents
2. 
   Dick Cheney's secret Energy group (who are the members)
   CIA - Tenet's "slam dunk" intelligence source on Iraq's WMD (who fabricated that intelligence - afterall, it wasn't real)
   White House - who outed the CIA agent
   FBI & John Ashcroft - why is Sibel Edmond's testimony being "re-classified" after 2 years of being in public domain
   Halliburton - wait, maybe we shouldn't. We don't want to break the camera...

Re:density

[dew-genen-ny]

Regarding cars -

surely we could just sweep a range of values....

or are you of the mind that a TV is impossible because we can only draw one line of dots?

just a case of enough processing power, surely?

Two uses immediately come to mind:

[io333]

1. Car in fog. It would be nice to have a heads up display on my winshield, kind of like Cadillac did with night vision some years ago... Whatever happened to that anyway?

2. Airplanes! No more grounding because of fog.

Re:also

[itwerx]

I wonder if it would let you see through the particles that many dresses consist of.

It would. The technology has actually been around for a long time in spy satellites.
It's devilishly simple. Take pictures/video along a number of wavelengths (e.g. IR through X-ray) along with the fact that they each reflect/refract at different angles of incidence and add some majorly intensive computation and you can "subtract" virtually any sort of dynamic occlusion, including the shifting fabric of a dress. If a woman were walking it would only take a few steps to get a remarkably clear image of what she looked like underneath.
Of course spy satellites (or, rather, some huge rendering farms down below) use it to remove distortion caused by clouds and shifting layers of air but it's all the same process, really.
People doubt that spy satellites can read the time off your watch but if you think about it there's not much you can't see if you've got good enough optics and distortions are no longer an issue.
Now getting all that computation into a camera would be very cool! Although, unless quantum computing makes a giant leap it'll be an analog computer rather than digital...

Re:Already exists

[Anonymous Coward]

Walked through one of the carriers docked in Newport around that same time and they let us use a device (looked like old style night vision binocs) that sounds like what you are talking about. It worked off heat, and wasn't color though. It was cool to see the heat from a lighter swirling around for 2 min after the flame was extinguished.

Re:Already exists

[Stormshadow]

You're thinking about the Navy Firefighter's Thermal Imager AKA NFTI. It's an infrared camera, nothing really special. Just got my Surface pin today, so I could tell you all sorts of useless knowledge about it... runs off 12 AA batteries, can't see through glass, lasts about 90 minutes on a full charge.
-ET3(SW)