A Single Pixel Camera

BuzzSkyline writes "Scientists at Rice University have developed a one pixel camera. Instead of recording an image point by point, it records the brightness of the light reflected from an array of movable micromirrors. Each configuration of the mirrors encodes some information about the scene, which the pixel collects as a single number. The camera produces a picture by psuedorandomly switching the mirrors and measuring the result several thousand times. Unlike megapixel cameras that record millions of pieces of data and then compress the information to keep file sizes down, the single pixel camera compresses the data first and records only the compact information. The experimental version is slow and the image quality is rough, but the technique may lead to single-pixel cameras that use detectors that can collect images outside the visible range, multi-pixel cameras that get by with much smaller imaging arrays, or possibly even megapixel cameras that provide gigapixel resolution. The researchers described their research on October 11 at the Optical Society of America's Frontiers in Optics meeting in Rochester, NY."

I don't get it...

It'll make current cameras, with simpler technology (less micromirror arrays and whatnot) cheaper? How? This stuff sounds expensiver.

Re:I don't get it...

Sure it's expensiverest at the moment. But with economisationalisation from upscalifying the process you could see it cheapifying quickly.

modify parentificator upwardly

Cat got your tongue? (something important seems to be missing from your comment ... like the body or the subject!)

Re:

That's vive la différence. Difference is a girl in French. :)

No real French speaker would make this kind of mistake...

There's the question...

Is it really cheaper to manufacture micromirror arrays that CCD or CMOS sensors?

Also, what degree of photon loss do you have from the arrays? No mirror is perfect...

Re:There's the question...

within a certain wavelength range (down to where actual atomic structures break up the smoothness), a perfectly flat material with no resistance has perfect reflection (that's why the silver back on a glass mirror is so reflective, is very flat and conductive

Re:There's the question...

Is it really cheaper to manufacture micromirror arrays that CCD or CMOS sensors?

Not likely. And it certainly doesn't sound mechanically robust to have moving parts replace a purely electronic chip. Cameras need to be rugged.

Also, what degree of photon loss do you have from the arrays? No mirror is perfect...

Imperfection in the reflectivity is probably secondary to diffraction, which will be a big problem for these small mirrors - and they would have to shrink even further for reasonable (multi-Mpixel) image resolutions. Diffraction is the biggest limiting factor for contrast in DMD projectors.

There are other problems with this design. First off, it is a time-sequential acquisition. The reconstruction algorithm assumes that all measurements are taken from the exact same scene. God knows what garbage it produces if you have moving objects or camera shake.

I guess their biggest motivation is to do the image sensing directly in compression space. Unfortunately, their compression space is vastly inferior to the compression space of, say JPEG. You see, JPEG is very cleverly designed in that it doesn't actually zero out certain frequencies directly - it just quantizes higher frequencies more agressively than lower ones, and that results in data that compresses better with a lossless compression algorithm (Huffman). By contrast, this compressive camera thing essentially directly zeroes out certain frequencies that have low amplitude. Not a very good idea perceptually.

Re:

I'm not sure I agree with you.
The problem with CCDs is you need to clock the values off the capacitors. Either you use a machanical shutter to stop smearing while you do this, or clock it into masked areas, which means you either need to accept a 50% loss

Re:

No camera system is perfect... but I think you might be selling this one short a little too soon.

The idea behind the average consumer camera is to gather photons from a large area in a reasonably short amount of time. Usually we do this with film or with a

101

This is me with Natalie Portman at a Star Wars convention (I'm the second 1).

Re:101

Sorry but due to the lossey process it is impossible to tell if hot grits were present,
Please take another photo and maybe the randomness of the process will enlighten us.

Re:101

The '0' was a hot grit you blind fool!!

Re:101

Nice try, doofus, but that's clearly photoshopped.

Re:

We understand, Mic, but we have ways to cure that. First, you must spend no less than an hour a day watching local news broadcasts. Second, you must spend a full hour reading comments on digg.com, and not allow yourself to post. Finally, you must burn a

Applications

This could have some awesome applications, especially on space missions. Imagine the next generation of mars probes and the resolution of the pictures taken if a camera near the size of current ones could have thousands of times the resolution. And of course, you also need to think about spy satellites. But perhaps the coolest application would be on space telescopes...

Re:Applications

This could have some awesome applications, especially on space missions. Imagine the next generation of mars probes and the resolution of the pictures taken if a camera near the size of current ones could have thousands of times the resolution.

This is unlikely for several reasons 1) resolution is far more limited by optical aperture than by the CCD array, 2) the system reads its images over a longish span of time - not good when your target is passing rapidly beneath you, and 3) the system requires considerable postprocessing - this either means you have to slow down the rate at which you take pictures, or eat scarce communications bandwidth.
 
 

And of course, you also need to think about spy satellites. But perhaps the coolest application would be on space telescopes...

The same objections apply to both applications.

Already done better in 1999

Check this out [osti.gov] In 1999 scientists at Los alamos national lab did essentially the same thing. Except they went one better---they also added in Phase detection by heterodyning the receiver.

Instead of using micro mirrors, the Los alamos team used an LCD which were more mature at the time. And Instead of using random modulation they used a progression of zenike polynomials and thus achieved much more control over the data compression.

patented too

A patent [osti.gov] for "A single element detector acts as an array"

Re:Already done better in 1999

Even better, use your scanner as a camera:
http://www.rit.edu/~andpph/text-demo-scanner-cam.h tml [rit.edu]

Should give you an idea of how to do it yourself to get gigapixel sized pictures.

Re:Applications

Actually a less fancy version of this technique was already used on mars pathfinder where several images were taken of the same objective and then combined to obtain better resolution.

"Superresolution image processing is a computational method for improving image resolution by a factor of n[1/2] by combining n independent images. This technique was used on Pathfinder to obtain better resolved images of Martian surface features."

Taken from the abstract of this article [inist.fr]:

Re:Applications

It makes more sense for small applications, I would think. A 39MPix CCD is several inches in each dimension. A single pixel would easily fit under a fingernail without anyone noticing. Depending on the mirror arrangement, you could probably have a lens-less camera that is not much bigger than a few grains of sand.

that's one big pixel

Scientists at Rice University have developed a one pixel camera.

The camera's one pixel, but when you print it out full size, you get a mega pixel.

photo album

. here's me at the grand canyon . oh man, here's where i got drunk off of my ass . here's me apologizing for this terrible joke

Re:

Next time, remember to remove the lens cover. All your images are black!

Had to be said...

One pixel should be enough for anybody.

Hold still, dammit!!!

I'm trying to take apicture one pixel at a time!

Voyager worked (still works?) like that

Early space cameras were single pixel and scanned their surroundings by their rotation.

Early fax machines worked the same way, but spun the paper around while the single photocell moved linearly left to right.

Hmmfff - Guess I'm giving my age away...

Re:Voyager worked (still works?) like that

Early fax machines worked the same way, but spun the paper around while the single photocell moved linearly left to right.

Hmmfff - Guess I'm giving my age away...

You should, in fact, call the Guinness Book of Records, as you must be the oldest person in the world. Fax machines of some sort or another have existed since the mid-late 19th century. [wikipedia.org]

Nothing for nothing

If you record only (lossy) compressed data, that will limit your image quality.
If you record things "pseudo-randomly", it'll be harder to get a predictable result
If you record a billion pixels instead of a million, you'll need to store them.
If you reduce the number of pixels, you reduce your redundancy.

It's still an interesting idea and probably has some specialist applications that will be very practical. But don't look for this in your Nikon or Canon camera in the next 10 years. Not sure what they are but if it can be made small enough I imagine a gigapixel camera on a space probe or better yet a space telescope (which can have more time to collect data) might be one. Of course it could also end up useless. That doesn't mean the technology shouldn't be explored. You never know what's going to provide the next breakthrough in understanding or application.

Re:Nothing for nothing

I think you may be missing the point (har har).

What they are recording is not solely a pixel, I would suspect, but the configuration of mirrors that achieved that point. So, there is a significant amount of information that they can extrapolate from just a random number seed and the final color. The plenoptic function that describes the transfer of light from the environment to the plane of the sensor is 4D. By capturing from many different non-parallel input rays onto a sensor, you can extrapolate a lot about the environment that isn't present in a purely parallel data set.

What I suspect they're goal is, is ultimately getting an array of mirrors onto a consumer-grade camera, and having it take three or four shots in rapid succession, then merge the information gained from each so that the result is more like having a High Dynamic Range image (well beyond the capabilities of any consumer-grade sensor) and use a tone-mapping algorithm to bring it back into a typical 8-bit range per component. It's complicated, but not impossible. Similar such things that are only a year or two old in the graphics community (flash + non-flash images being merged to give good color in low-light situations, multiple exposure images merged for HDR, etc) should come out in a couple of years as automatic modes for color correction, probably even on low-end cameras.

Of course, I still have a 6 year old point and shoot, so what do I know? :-)

Other wavelengths

I have often thought that it would be really neat if you could get a visual image of radio waves like around for example 2.4ghz and be able to see exactly how your surroundings block/absorb/reflect those wave - in addition to seeing sources of the waves. They mention that might be possible by throwing a different sort of detector instead of a ccd in there? anyone know - would that be possible? do 2.4ghz waves bounce off anything else like light does mirrors, without getting scattered?

Re:Other wavelengths

Radiowaves are big and they go through just about everything. It would look like a bunch of stuff made out of glass with varying degrees of transparency. Metal things would be darker glass, but anything less than one wavelength in size would be fuzzy and impossible to focus on anyway. In the distance, you would see a bunch of different colored lights flashing where ever there's a radio tower or cellphone. (Each different station would be a different color.) At night, you can see flashes in the sky where distant HAM radio stations bounce off the ionosphere. All your household electronics would glow the faintly in the same 60 Hz color, and you could probably make out all your wiring just sitting in one room and looking around, if it weren't for the fact that it all blurs up due to the size of the wavelength.

Re:

Nice vivid description! I would like to render such a scene, but alas, I couldn't model myself out of a wet paper bag. Maybe someone else is up for it?

any astronomy

or low light applications? i wonder what this idea would be like extended to non-electromagnetic phenomena, like electron microscopes, or neutron detectors or nuclear colliders or gravity waves. well, you need mirrors... "micromirrors"... but their are analogs to mirrors in non-electromagnetic phenomena. sort of

can't wait

can't wait for the first four pixel camera. Imagine the resolution of that one! ;-P

Resolution?

About five pixels? :P

Now THERE'S a reality show we need

Lock ten marketdroids in a room and give them a task to try and create a marketing campaign for something impossible and ridiculous. Like a one-pixel digital camera.

I'm envisioning a sticker on the box that reads "THE ONLY MICRO-MEGAPIXEL CAMERA!"

exotic sensors

this could be useful for imaging in frequencies or frequency ranges where production of a pixel array isn't possible or economically feasable

The point is focus and low light capability

This is a lenseless design and therefore does not have problems of focus. The different parts of the scene should all be in focus simultaneously. There is no sensible way of schieving this with a lensed design since the better the light gathering power, the narrower the plane of focus.

The technique in use for years for infra-red cameras involves the use of a single (Peltier-cooled) pixel and a scanner, but scanners have numerous problems one of which is that there is always vibration caused by the two frequency components of the line end switching of the horizontal and vertical scans. This technique, by using pseudo-random switching, should eliminate vibration.

So the ultimate long term goal would appear to be the ability to produce 3-D images with focus throughout the entire scene, low light capability and an absence of blur due to vibration. IANAOR (I am not an optical researcher) but it seems a good line of investigation.

Oh dear, abuse

I hate to tell you this (no, I don't), but an image forming lens does not normally have light intensification properties. You can see this quite easily if you think that, for instance, an f/2 lens on a 35mm camera has a diameter of approx. 25mm, and the light entering that 25mm circle is expanded to a circle approx. 43mm-50mm diameter. If the lens is removed, the light intensity falling on a given area increases. To a first approximation, to get the same intensity with or without the lens, you would need an f/1 lens. I suggest you see how much Noctiluxes sell for, and what is their depth of field.

Like a lot of people who do not know any optics, I suspect you think that the light at the scene is somehow concentrated by the lens to form the image. It isn't; the lens doesn't suck in any extra light other than what impinges on it.

A single pixel is effectively approx f/1.

Oh yes, and you are arrogant, rude, and stupid. Perhaps you really do have a job with Microsoft.

it's probably been said..

...but it'd suck to have a dead pixel.

Practical uses. Why the stupid comments?

Pretty surprised at all the dumb comments on this story. The scientists involved are not demeaned by consumers being used to cheap megapixel cameras, nor by a secret lab having done something that sounds similar, nor by some patent existing. Slashdot really sucks!

If you are interested you can find out a lot about the really fascinating and cutting edge science of computationally assisted optics, or whatever is the correct term. It is the same field as the people who have been experimenting with giant arrays of cheap cameras, capturing entire light fields that can be sliced in time and space and reprojected later on, etc. It is computers plus physics and a big dose of creativity, which is why it is related to SIGGRAPH too.

Anyway this is interesting and is based on different principles from current megapixel cameras, which is why they think it might improve current cameras too. Just like the way the spaghetti physicists were laughed at by Harvard's igNobel, even though they finally solved something Feynman couldn't crack and have discovered a new method for focusing energy.

Just off-hand, the one pixel camera and compressive imaging theory they have looks very interesting:

• A one-chip computer with transmitter, battery and 1 pixel camera could be worn on your cuffs or collar and capture/assemble from random angles through which it is jangled your entire surroundings.
  
• Could be used if mounted on a wire tip and wire oscillated giving many views of an object for cheap 3d scanning
  
• Camera could include one pixel per range of spectrum, recording a full electromangetic spectrum
  
• They are doing only some simple compression right now. If your current camera could do wavelet compression within the ccd you could certainly get much better pictures and reduce the storage needed.
  
• If current cameras can do all the work needed in 1/500 of a second that means they could be doing a lot more if only compression, transmission and storage are solved, that is what they are working on.
  
• The one pixel camera uses random projections to achieve a certain density of information that seems to be constant throughout the light field they are capturing. This means if they store orientation and time accurately, their data can be sliced at constant quality in any direction, so it is homogenous data which is good. Imagine slicing diagonally through Kraft cheese block or through swiss cheese.
  
• Compressive imaging might help video camera manufacturers wrap their heads around recording at far higher frame rates, including side radio bands for orientation, or combining multiple image sources. Compression in the imaging chip means less data to handle elsewhere.
  
• If you read some of the bibliography (the Architecture one) you will see use of Haar wavelets to reconstruct an image from a 3-dimensional (200,000 voxel) data structure which performs much better than a 2-d one due to the sparseness of data. This paper also talks about the use of bands for which CCD use is impossible.
  
  

Spam

The spammers have had these cameras for a long time. They're always emailing me the pictures they took with them.

Random sampling vs compression

How can an image which is constructed psuedorandomly ever compare to an image that is compressed using algorithms designed to preserve 'important' information?
It seems to me you need to assemble the image before you can decide what to throw away.

here we go again

This kind of thing has been used for a long time: Nipkow Disk [wikipedia.org], Drum Scanner [wikipedia.org]. The combination with micromirror arrays is new.

However, there's a reason we "acquire first, ask questions later", as the article talks about current systems: electronics is much better at "asking questions" than mechanical hardware.

Part Number

There has been a single pixel camera available for a long time, under the part number ORP12.

not patentable, prior art exists

I refer, of course, to the flying-spot scanner of early (and sometimes late) television.

it was very difficult to make a working early camera tube with lame phosphors, flaky passive components, and nightmare wiring. but it was pretty simple to paint a raster on a screen by comparison. so the object to be scanned was put in front of the raster and a single photodiode vacuum tube picked up the changes in brightness, and modulated the "spot" created by the line and position sweep signals.

old hat by the end of the 1920s, but used as late as the 1980s in super-quaity scanners to encode 35mm and 16mm film for network-quality television. the indian-head generators that took two racks of tubes, and provided the best signal reference at the start of a broadcast day and the best calibration signal for TV repairmen in the field, were all flying-spot scanners.

no patent forrrrr YOU.

Re:Non-static images

It would be indeed impractical, and that makes this method quite useless in most applications. The researchers asked themselves "what if that single pixel receptor is good and expensive" while most modern answers are quite opposite to that - it's easier to make plenty of medium quality sensors than one good sensor. Not even counting the micro-mechanics needed. Solid state already gives you several megapixels for a few dollars, and the cost is only going down.

Re:

Even those seemingly magic DLP mirrors couldn't possibly be faster.

      Do not underestimate the power of our shiny disco ball.

Re:

Micromirror arrays have been commercially available for ten years now, and had been in design for at least ten years prior to that. They're used in DLP projectors and projection TVs. You can go buy one at Best Buy if you'd like.

The durability of a micr