Stories
Slash Boxes
Comments
Search

News for nerds, stuff that matters

Slashdot Log In

Log In

Create Account  |  Retrieve Password

Searching with Images instead of Words

Posted by CmdrTaco on Thu Jan 13, 2005 03:29 PM
from the blessing-for-those-who-can't-spell dept.
Graphics Software Science
johnsee writes "A computer vision researcher by the name of Hartmut Neven is developing ingenious new technology that allows the searching of a database by submitting an image, for example, off a mobile phone camera. Imagine taking a photo of a street corner to find out where you are, or the photo of a city building to see its history"
+ -
story
This discussion has been archived. No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.

Searching with Images instead of Words 50 Comments More /

 Full
 Abbreviated
 Hidden
More
Loading... please wait.

  • Is A Picture Worth A Thousand Words? (Score:4, Insightful)

    by fembots (753724) on Thursday January 13 2005, @03:30PM (#11352225) Homepage
    Tell me, which is easier? Upload this image [iclod.com] and try to find out where you are via this Visual Google, or enter the street name (street sign in the photo says "Queen Street") in Text Google?

    The article also mentioned this thing should start small, like a movie guide, so is it easier to upload a 2K "I,Robot" billboard photo, or just enter "I,Robot" in Google on your cell phone?

    As long as human input is still required (i.e. you need to submit something), I don't think this is going to be popular. However, if you have a Oakley that automatically takes photos of what you see and feeds you the location details, that'll be something.
    • Actually, what I look forward to is searching with a picture of Waldo. Maybe I can finally find him.
    • It does seem to have some functionality though. Let's say for example, this holiday I received a Thing(tm) as a present. I could take a picture or two of the Thing, and it may be easier to figure out what the it is.

      Of course, for some reason I think it would be difficult to make Visual Google function that well... the only way I would get results for my Thing would be if someone already knew what it was, and defined it for the search engine.
    • Pretty nifty for the functionally illiterate ... except that if they can't read the signs, they won't be able to read the answer either.

      Of course, it could just return the picture with a big red arrow saying "you are here" (or should it be "U R HERE"?)

      Seems to me that just offering a mapping function via cell locators would be more popular.

    • The real killer app will be taking out hotornot's MeetMe funding (you know, if you want to actually talk to someone you click on, you have to pay money); you just google for their picture, and find them that way :)

    • Imgseek. (Score:5, Informative)

      by Adhemar (679794) on Thursday January 13 2005, @04:14PM (#11352552)

      Imagine you're a photographer. Professional or hobbyist, I don't care. You have made thousands of pictures; they all are on your hard drive.

      Imagine you're lazy. (Maybe you don't have to imagine that.) You don't want to describe your photos, you don't want to label them. The only metadata associated to your photos is date and time.

      Imagine you're looking for a particular photo. You know where you'va taken it, you know what is on it, you can remember the subject, the color shades, etc. You just can't remember exactly when you took that picture. How do you search for it?

      Well, you quickly make a drawing in which you try to (sort of) replicate colors and shapes. And you let your computer search for "similar" graphics.

      Such software exists already (for quite some time). There's a beta Free software project (GNU licenced) called imgseek [sourceforge.net]. Current version: 0.8.4. I haven't tried it, I don't know how good it is. But this screenshot [sourceforge.net] looks impressive.

    • Re:Is A Picture Worth A Thousand Words? (Score:5, Interesting)

      by Rei (128717) on Thursday January 13 2005, @04:25PM (#11352641) Homepage
      Also, there's the whole "vaporware" issue. The scale of this programming task is staggering; it's not only image recognition, but image *searching*. Just look at how poor OCR does with handwriting (and sometimes even pre-printed text). Generalized image recognition is orders of magnitude harder than recognizing a small set of print characters lined up in nice rows and clustered into words, and image searching is beyond that.

      He can claim he's developing whatever he wants, but I'll believe it when I see it. It reminds me too much of how many AI researchers in the 60s were convinced that by the 90s computers would regularly converse with humans and be able to reason like them.
    • Actually, I think I read once (might have even been on /.) about another potential application for a similar technology, which seemed much more useful than this. The idea involved using images to search, say, a parts database. If you were holding some unidentified doohickey in your hand, and you needed to know what it was so you could find a replacement, you could sketch a rough outline of the object, and the sketch would be used to search through the design information in the database (say, CAD drawings an

  • Better yet... (Score:5, Funny)

    by punkass (70637) on Thursday January 13 2005, @03:30PM (#11352229)
    ..imagine being the guy who has to photograph EVERY STREET CORNER IN THE WORLD.

  • Fleck's image recognition (Score:5, Funny)

    by 2advanced.net (849238) on Thursday January 13 2005, @03:31PM (#11352237) Homepage
    So when do you combine this with Fleck's nude recognition [psu.edu] algorithms to provide a service that can identify a person by partial nude picture?

    The possibilities are endless!

  • Why not use GPS Technology? (Score:5, Interesting)

    by xmas2003 (739875) * on Thursday January 13 2005, @03:31PM (#11352243) Homepage
    While this looks pretty cool, I'm confused by the examples provided in the writeup - "Imagine taking a photo of a street corner to find out where you are, or the photo of a city building to see its history" since GPS technology would probably be a better enabler for those specific applications.
    • Or why not just look at the street signs to find out where you are? If the street corner is in a database it is probably in an area that is developed enough to have street signs.
    • Imagine you're going through photos of your latest vacation and you find one of a street corner which you snapped while on a drinking binge. Since, in your drunken stupor, you don't remember where it was, you can just submit it to find out the history of the building and perhaps discover other famous people who have similarly vomited in that vicinity.

    • or the photo of a city building to see its history"

      Substitute "person" for "building". (Is that a cameraphone, officer?)

    • I forgot to add to my post above that GPS encoding as part of the JPEG EXIF header had been a standard for some time and a handful of high-end DLSR's have this capability today - this will become more prevelent - heck, as part of E911, your cell phone camera already has GPS info, but I don't know first hand if those add the tags to the EXIF header, but would be trivial to do.

      alek

    • I thought of that too. Why not use GPS instead. It's a problem of machine learning and classification. Given a picture of a street corner, what features make that street corner in any light and weather conditions different than other hundreds of thousands of corners. Also, what about the angle at which the image is taken?

      The database to work will have to understand what 3D objects are (at least in the specific domain) and have an idea of what features of the object are important (like signs for exam

    • GPS would be useful in some situations (if you want to know about a general area), but for the example of taking a "photo of a city building to see its history", GPS itself would not be sufficient.

      GPS can provide a location, but it can't pinpoint what you are looking at. This is the case even with compass data indicating which direction you are pointing your device--what if there are two things in your line of site from that perspective? (Do you want information about the building, or do you want inform

  • Man on man (Score:3, Insightful)

    by sulli (195030) * on Thursday January 13 2005, @03:31PM (#11352244) Journal
    the pr0n industry is going to love this.

    • Re:Man on man (Score:5, Funny)

      by 2advanced.net (849238) on Thursday January 13 2005, @03:33PM (#11352270) Homepage
      "Man on man"? Man oh man, Freud would love this!
    • ...are gonna love this too. Take a picture of the girl you like and do a search. This has some scary connotations I'm afraid.

      • ...are gonna love this too. Take a picture of the girl you like and do a search. This has some scary connotations I'm afraid.

        Straw-man.

        Stalkers already use Google. It's a lot easier to stalk someone with text than with pictures. What are the chances your image search would actually turn up anything for your average Jane Q Public?
        • Not in the near future, of course. I can't image that taking a picture of most of anything would produce valid results in the near future. However, this type of photographic facial recognition [cnn.com] is already being reseached and developed for things like bank robberies and terorism. I can picture this taking off to the point where it applies to the general public...
  • Imagine taking a photo of a street corner to find out where you are, or the photo of a city building to see its history
    or a photo of your wife to see...
    oh never mind.

  • New Query Language (Score:5, Funny)

    by Tablizer (95088) on Thursday January 13 2005, @03:32PM (#11352252) Homepage Journal
    Enter search criteria: (.)(.)

  • Scary Results (Score:3, Funny)

    by The_Rippa (181699) * on Thursday January 13 2005, @03:32PM (#11352261)
    I'm a beta tester for this product and have gotten some scary results. For instance, I was on vacation in Yellowstone and took a photo of Old Faithful with my camera phone. I submitted it and it gave me back search results for tubgirl!
  • it's going to be difficult for slashdotters to find p0rn with this search engine. Not only must you have sex, but you've got to take a picture of the act too...
  • how much harder is it to just use a regular text search for the restaraunt, movie, building, etc. that you want info on? It's like voice dialing on a cell phone, good idea, but it's about ten times faster and more effective to either dial or scroll to the name you want to call manually.
  • Isn't it more efficient to use the GPS in it to tell me where I am than to submit an image?

    Seriously it's a good thing but the uses are somewhat limited. What if you don't have a digital image or a photograph to be scanned? How would you translate the image in your minds-eye into something searchable by the PC? (yah, mindlink... I know...)

  • It's going to be boobs (Score:4, Funny)

    by Nom du Keyboard (633989) on Thursday January 13 2005, @03:34PM (#11352292)
    Everyone is going to try searching on a pair of boobs to see what they get. I can set it now:

    1: Take picture of current date's frontside archtecture.
    2: Submit to search.
    3: Reply: You can do better than that. Try her older sister.

  • i'm going to rush out today and buy a nokia phone so that i can have this functionality the instant it is available in 2038.
    i mean, really. isn't this one of the main roadblocks to having a robot that can operate independently -- object recognition? i mean, if spam bots (the world's most advanced robots) get tripped up trying to read the word 'cat' behind some wavy lines while they're signing up for a hotmail account, you really think that i'll be able to photograph a car in 5 years and get info about it?
    • Image recognition is immensely more difficult than people seem to think. And yet every few weeks someone is claiming to be just around the corner from a system that can easily identify the contents of an image.

      When your brain 'recognises' what it is looking at, it is doing a lot more than just comparing two images (as in the street-corner example from the article). Your brain simply doesnt operate in terms of bitmaps.

      The fact that he is basing his hyper-vaporous product on facial-recognition software should

  • OPTION 1:
    Take picture of street corner with camera phone. Connect cameraphone to Internet connection. Upload to wherami.cooldatabases.com. Wait 30 seconds for processing. Get location.

    OPTION 2:
    Pull out $400 GPS with map software.

    OPTION 3:
    Read street signs. Check index of road atlas.

    Yeah. Option 1 sounds awesome...

  • And we have a winner... (Score:3, Funny)

    by Momoru (837801) on Thursday January 13 2005, @03:36PM (#11352322) Homepage Journal
    Sounds like we may have a winner for Wired's 2005-2010 Vaporware awards.
  • Visual Google? Try visual semantic web. It identifies something and figures out where to go to find answers to it.

    Also, I think the building and street corner thing would work a lot better with GPS than a camera.

    The most interesting thing I saw in this article is that he plans to roll out a first version in about a year. Besides that, it's interesting research, but stuff we've already heard about.

    I would definitely like to experiment with this sort of system.

  • I've heard of more than one service in development for returning music titles against a song clip you play or hum into a phone or microphone.

    Even though satellite/digital radio will reduce the market for this kind of thing, because each displays the artist name and track title, there are still plenty of opportunities to get a song stuck in your head that you don't recognize. A surprising number of people find out about music when it's used as the background tune in TV commercials, for example.

  • How could taking a picture of a street corner possibly tell you where you are? Don't most look alike? There are other, better technologies for telling you where you are, such as GPS or even just looking at the nearest street sign and typing the name of the corner into the map application on your phone.

  • iDating (Score:5, Insightful)

    by El_Smack (267329) on Thursday January 13 2005, @03:38PM (#11352353)
    Or taking a picture of someone and finding out their history.
    click
    "Whoa Dude!, she's been on 4 amature Pr0n sites!"
  • On a run the other day I found a waterfowl that I'd never seen before in the lake. It took some doing to figure out what it was. If I could have snapped its picture on my cell camera and gotten an identification (a hooded merganser, it turned out, after some digging), that would have been cool.

    I can't imagine how well this would work, since orientation fools things pretty easily. But I imagine that if it were available, I'd find a lot of unidentified objects to look up. (A cooking magazine I read has a

  • Yes, Imagine taking a picture (Score:5, Insightful)

    by Nom du Keyboard (633989) on Thursday January 13 2005, @03:39PM (#11352368)
    Imagine taking a photo of a street corner to find out where you are,

    Yes, imagine that.

    1: Take picture with ultra-modern all-features camera phone of building while lost in city.
    2: Submit to search system.
    3: Search system queries phone's built-in GPS for position information.
    4: Search system sends back retrieved GPS location.
    5: Customer is absolutely blown away and immediately sends back picture of self signing virtual 10-year contract at Early Adopter prices.
    6: Profit!

  • The most searched term is 'sex' or something like that. It'd be interesting to see the "search queries" for a picture-based search tool. I bet there would be a lot of stick figures behaving badly.
  • Maybe this can be used for robots to recognise stuff or something like that.
  • If I need to find a specific location, I'll send a text message to 46645 [google.com] (GOOGL). Then, I'll use the street signs or the navigation system in my car.

    If I'm completely lost, the only way object recognition would work is if I'm in an area with a lot of recognizable features...like a city...in which case I'd just ask somebody. I doubt taking a picture of a bush when I'm lost in the middle of nowhere will be helpful (see: car navigation system).

  • Hartmut Neven and hundres of other researchers... (Score:3, Insightful)

    by dioscaido (541037) on Thursday January 13 2005, @03:42PM (#11352422)
    This seems less like a technology article and more like an advertisement for Hartmut Neven himself. Yes, he's built a 'google for images'... But how does it perform? How exactly is it 'ingenious'? What sets his project apart from the handful of people at almost every University with a Computer Vision research department that is tackling the problem. The problem of matching images is well known, and very difficult to solve. Even in my grad school (BU), which has a small number of computer vision grad students, there are two different research projects on this very topic.
  • Can my Powerbook be prompted to surf porn sites when the iSight catches me pulling down my pants?

  • Way to figure out the name of the chick you saw on flashyourrack.com!!!
  • Back in the 1990s, I used to develop Kiosk Software using IBM's Audio-Visual Connection language. The next generation of that software was called Ultim-Media Builder. At the same time, IBM came out with an extension to DB2 that would allow you to do queries against a database of images...you could say "Find all pictures with a red ball and a tree", and it would find them in the database by "looking" at the pictures, not because of any captions or notes. I never heard if that tech became well know, or if it

  • Duh (Score:3, Funny)

    by dfj225 (587560) on Thursday January 13 2005, @03:44PM (#11352458) Homepage Journal
    "Imagine taking a photo of a street corner to find out where you are..."

    Imagine reading that street sign you just took a photo of to find out where you are.
  • That IMHO is the real prize in imaging right now.

    I show an image of a car, and the computer knows, the make, model.

    I show an screenshot of a TV show where they remove the product name/brand from the product... it can ID the product.

    Facial recognition is not to bad at this point (though it seems lots of the pioneers are going under). Nobody seems to have successfully applied it to objects.

    I think that has much more use... think about it:
    1. Indexing and searching images/video
    2. Explaining TV to the bli

All trademarks and copyrights on this page are owned by their respective owners. Comments are owned by the Poster. The Rest © 1997-2009 Geeknet, Inc.