3D Mars Scenes Recreated From Photos

Hoon Mihn Fao links to this BBC article about Microsoft Research scientists generating 3D models based on pictures beamed back from the Mars rovers. It begins "Using techniques originally applied to paintings, the researchers have written software that works out what flat images would look like from different viewpoints. The result is a navigable 3D model created from one or more pictures snapped by the Mars buggies."

Am I the only one who's thinking...

[Ayaress]

(insert favorite FPS game here) map?

Re:Am I the only one who's thinking...

[Patrik_AKA_RedX]

A FPS map of a Mars desert? Am I the only one who thinks that would be boring?
A desert is a collection of sandy hills with an occasional cactus and Mars even lacks the latter.

And there aren't any pictures of the secret Nazi Mars base, but don't worry, we've send our best Castle Wolfenstein player to deal with it.

Mars in Stereo

[stereo_Barryo]

To repeat myself, my friend Robert Clemenzi has put up a page on Mars in stereo and has written a program, that can be downloaded, that allows you to study the pictures from the rovers in either cross-eyed stereo or over-under. http://www.cpcug.org/user/clemenzi/science/MarsIma ges_3D.html

More Mars stereo goodness...

[TheAngryArmadillo]

A coworker of mine and I wrote a script to download and generate every possible 3D image from NASA's MER website [nasa.gov]. It goes through and finds all matching left and right images then makes them into an anaglyph.

If you've got your red/blue glasses you can see them here [nnin.net]. We update them with new images every morning. Some of the images are useless but there are also a bunch that NASA never generates for us. They are separated by rover and Sol.

Where's the video?

[Gyler St. James]

Wow. They said something to the effect of being able to see other objects that was not completely visible in the image. "We work out what's behind an object and how we can automatically fill in what's not seen by cameras or painted." I wonder if this would allow you to rotate around objects that are halfway hidden, but what about asymmetrical objects? How can they extrapolate what's not known? Interesting none the less, but where's the video they talked about?

Re:Where's the video?

[Uncle_Al]

Look there:
Pictures and Videos [microsoft.com]

Have fun...

Re:Where's the video?

[theslashdude]

Am I the only one who was expecting a flying-through-the-landscape type video rather then just a little wobble?

Are they interpolating between two cameras?

[enosys]

The range of motion seems extremely small. Are they just interpolating views from between two cameras?

This sort of thing isn't new. Panorama Tools [unimelb.edu.au] has PTInterpolate, which given two images of the same scene taken from different viewpoints creates any intermediate view. I've never tried it but this makes me want to.

Re:Are they interpolating between two cameras?

[kazbek]

I agree with you. So they aren't making up any detail, because they don't go beyond the stereo angle.

Re:Are they interpolating between two cameras?

[enosys]

You may have to make up detail even in this case. Imagine you have two photos of a fairly narrow slit. At the midpoint between these two photos an object can be seen through the slit, but from the viewpoints of the photos themselves this object cannot be seen. This sort of situation can happen with for example ground with lots of rocks on it.

Re:Where's the video?

[Tablizer]

How can they extrapolate what's not known?

I don't think they do. You can calculate the position of most objects actually imaged in images taken from multiple viewpoints of the same scene using a modern version of old-fashioned triangulation like surveyers use, but you still have no idea what is behind a rock, for example. At best one can guess what the back looks like based on the front (if back not imaged).

But I notice in some of the 3D models used by the Mars teams that "unknown" areas are filled in

NASA did this from day one

[CXI]

So, NASA has been taking stereo images since the beginning and making true 3D models, and now Microsoft has figured out how to make fake stereo images and fake 3D models?... Um, nice step backwards?

Re:NASA did this from day one

[Xilman]

Try reading further down the page. They are taking a single image and building a steroscopic view. NASA were using pairs of images in the traditional manner.

Paul

Re:NASA did this from day one

[Anonymous Coward]

Grandparent said:

So, NASA has been taking stereo images since the beginning and making true 3D models, and now Microsoft has figured out how to make fake stereo images and fake 3D models?... Um, nice step backwards?

Parent said:

Try reading further down the page. They are taking a

single image and building a steroscopic view.

Try understanding what you read. :) Making a stereoscopic view from a single image is a step backwards. They're interpolating to "resolve" information that doesn't really exist. It's like when crime dramas zoom in about 10x and then magically clean up the image to see details that were not captured on a security camera! If you want to extract depth, you need two pictures taken from separate locations. Otherwise you're just making educated guesses.

X...Y...Z
....w....

A.......B

Picture from camera A: X...Yw..Z
Picture from camera B: X..wY...Z

Please don't tell me you honestly think you can tell that w is in front of Y just by processing the picture from camera A (unless you know the exact dimensions of the WXYZ elements). You need the additional information provided by camera B. Consider a picture where you see a tall person and a short building. You might assume the person is standing in front of the building, but what if it's a miniature building in the foreground? Unless you have a 2nd picture, you can't know for sure.

Re:NASA did this from day one

[CXI]

Try reading further down the page.

Try reading my post. It is a step backwards to take a single image and generate fake data from it, when we already have real data from multiple stereo images!

Re:NASA did this from day one

[Xilman]

Again: try reading the article. It's (relatively) easy to generate stereograms from two images taken from slightly different positions. That's been done for decades.

What is non-trivial is generating a stereogram from a single image. In some circumstances only a single image is available and another will never become available. Are you seriously claiming that it's a step backwards to be able to generate a stereogram from a single image?

The research was first applied (AFAIK) to Renaissance oil painti

Other uses?

[Pumpernickle]

"We extract 3D information out of the image," said Mr Criminisi. "We work out what's behind an object and how we can automatically fill in what's not seen by cameras or painted."

Would this technique work, for, say, objects hidden or obscured by some sort of flimsy semi-transparent thin fibrous material? Or only hard, rock-like substances? ^^

Re:Other uses?

[dnahelix]

"we can automatically fill in what's not seen by cameras"

That's where the little green men come from!

Re:Other uses?

[mandolin]

I have (not) researched this carefully, and the answer to your 1st question is "yes", as long as the object also satisfies the condition raised in your second question

IOW, hope you like dick.

Better Format?

[MagnusDredd]

Anyone have a copy of the movies in an open format???

Both Video Lan Client, and Mplayer refuse to play this thing.

FFMPEG doesn't support it yet, I don't think.

I'm becoming convinced that the only reason "WMV3" encoding exists is to lock out non-M$ users. There are a million different formats that work well enough and are open.

What else should one expect from M$?

Links are:
http://research.microsoft.com/~antcrim/mars _ web_MS RC/MarsVirtualView.wmv
http://research.microsoft. com/~antcrim/mars_web_MS RC/MSR

Re:Better Format?

[HacTar]

try this

http://research.microsoft.com/~antcrim/mars_web_ MS RC/MarsVirtualView.mpg