Content-Aware Image Resizing 174
An anonymous reader writes "At the SIGGRAPH 2007 conference in San Diego, two Israeli professors, Shai Avidan and Ariel Shamir, have demonstrated a new method to shrink images. The method is called 'Seam Carving for Content-Aware Image Resizing' (PDF paper here) and it figures out which parts of an image are less significant. This makes it possible to change the aspect ratio of an image without making the content look skewed or stretched out. There is a video demonstration up on YouTube."
The paper via ACM (Score:5, Informative)
Re:The paper via ACM (Score:5, Informative)
Re:The paper via ACM (Score:5, Informative)
Shrink image:
Step 1: Run an edge detection algorithm.
Step 2: Find minimal energy (least amount of edges crossed) path from top to bottom or left to right (graph-cut algorithm).
Step 3: Remove pixels along that path.
Step 4: Repeat steps 2 and 3 as necessary.
Extend image:
Step 1: Run an edge detection algorithm.
Step 2: Find minimal energy (least amount of edges crossed) path from top to bottom or left to right (graph-cut algorithm).
Step 3: Insert pixels along that path (interpolated from neighbors)
Step 4: Repeat steps 2 and 3 as necessary.
Remove objects:
Step 1: Run an edge detection algorithm.
Step 2: Mask object by giving its pixels low/negative energy values.
Step 3: Find minimal energy (least amount of edges crossed) path from top to bottom or left to right (graph-cut algorithm).
Step 4: Remove pixels along that path.
Step 5: Repeat steps 3 and 4 as necessary.
Re:The paper via ACM (Score:5, Insightful)
Step 6: Extend image to match original size using the previous extend image algorithm
(Of course, I leave the obligatory Profit step as an exercise for the reader).
Re: (Score:2, Informative)
Video is on youtube.... (Score:3, Informative)
Clicky [youtube.com]
Tm
Impressive (Score:2)
I Think You'll Find (Score:3, Insightful)
Re: (Score:2)
nice! (Score:5, Interesting)
Other than that though, that's pretty awesome... I'm sure there's more instances where it doesn't look right than what they showed, but it's definitely cool how well it works as it stands!
I can imagine it would be extremely useful for ex-boyfriends or ex-girlfriends; just load up all their photos of them and their ex, wave the magic eraser, and *boom* you don't have to delete all your old vacation shots
I wonder how well it would work for the porn industry too; nice automatic resizing of breasts without ruining the picture! Fetishists will be SO happy!
Re: (Score:2, Interesting)
It seems like a little bit of work is left to make it as completely automated as you would need to have it just "always work"
Completely right, yes. The images in the video have been selected to show this technique in the best possible light. There's a great variety of images that'll really not work quite right with a completely automated treatment. Speaking from experience having implemented this last week.
That said, as pointed out in the paper there's plenty of room for a higher level of analysis over the top of the basic seam-carving procedure. The function used to calculate the energy of a given pixel is easily swapped out wi
Re: (Score:2)
Re:nice! (Score:5, Funny)
Better never get a partner then at all if you are going to hate the person once it doesn't work longer.
But then I'm a regular slashdot visitor and don't have any exs so what do I know.
Re: (Score:2)
Re: (Score:2)
Pay alimony for awhile and I'm sure you'll understand it much better.
Most of the bitterness arises because you didn't get a good return on your investment of time/love/money/etc. You expected lifelong compatibility, but got a partner that has a significant personality change in the first few years of you relationship. I take more of a "high price of a lesson learned" view of relationships that ended badly, but then I just toss the photos and old cutesy stuff o
Re: (Score:2)
Also regarding time I watched some TV program about it and how hormones worked and I got the impression that the first "fall in love" part only lasted for two years or whatever it was, but then it stops and people who stay rather do so because they have become so good friends and like eachother that way. But still, what do I know =P, some peoples attention only s
Re: (Score:2)
He mentioned alimony and "first few years" - in other words, things were presumably good, he married her, and *then* she changed and they split. So yes, he did expect a lifelong relationship (that's what marriage is supposed to be). It's a very common story.
Re: (Score:2)
I'd never understod this hate-your-ex-thing? The person where part of your life for some time but you have decided to hate it and want to erase it from it? Better never get a partner then at all if you are going to hate the person once it doesn't work longer.
Sometimes the reason it doesn't work any longer is because you've grown to hate the person.
I don't think any reasonably adjusted person would culture hatred for a past parter just because the relationship didn't work out. People who do otherwise a
Re: (Score:2)
She can just DIAF (Score:2)
Re: (Score:2)
I guess it's true with life in general, that it's better to have LIVED even if everything didn't turned out as expected/good, some things probably did. And for those people where it didn't, I feel sad for them.
Re: (Score:2)
There's nothing quite like the crushing anguish of being used, deceived, betrayed, lied to, and cheated on by someone you are in love with and trust implicitly.
You can't easily 'cherish' the once happy memories because they are all tainted by your exes deceipt.
Imagine your father or uncle died, and you have all these memories of going to the beach togethe
Re: (Score:2)
Re: (Score:2)
No we don't. Many relationships last a lifetime.
And there is nothing inherently wrong with short relationships. They don't have to end in a crushing heartbreak. You can be in a series of short relationships without emotionally scarring everyone you come into contact with; if you are honest with your partner.
We're untrustworthy creatures at heart.
Maybe you are. I have enough respect for myself and for others that I don't go around needlessly hurting the people who care for me.
Re: (Score:2)
Re: (Score:2)
So it's a great tool not just for breakups but for stalkers as well!
I wonder how good it will be with moving pictures. How well can it remove Jar-Jar Binks while minimizing exposing artifacts across continuous frames?
Practical uses (Score:2, Funny)
Re: (Score:2)
I see your reduced breasts and raise you a 'Seam Carvied Content-Aware Resized Image' midget porn. Guess who Elizabeth Hurley looks like now.
Re: (Score:2)
Slightly Strange (Score:3, Interesting)
Re: (Score:2, Interesting)
According to the video, the added background information is actually the averaging of the extra "low energy" information around it. So it's not quite duplicated.
Re: (Score:2)
You may aswell just add a nice blue frame round the edge of the picture to make it fit.
The whole point is to avoid artificial techniques that stand out, like frames. A trivial application would be expanding an image to fit as your desktop background. If you're trying to fit a picture into a rectangle with particular dimensions, you may want to both squeeze it one dimension *and* expand it in another to give you the most natural looking effect.
For example, think how a 4:3 TV screen often displays wide-screen movies: They scale (shrink) the image to fit and put bars on the top and bottom. In
Re: (Score:2)
Re: (Score:2)
Re:Slightly Strange (Score:5, Insightful)
It's not perfect of course. I'm guessing that if you had a picture of two people next to each other, one with a solid colored shirt, and the other with a striped colored shirt, that the solid colored shirt guy would get skinner than the striped when shrinking, and the reverse when enlarging. However, it's a neat idea, and I look forward to reading the paper.
A picture speaks a thousand words... (Score:4, Insightful)
There are probably a few situations where the 'unimportant' bits of an image are still as relevant as the rest. Sports photos for instance - especially those played on grass - would not give you a true picture (literally) of what's going on in the scene.
This'd be good for reference photos - like the animals at the start of the YouTube video, but applications where precision and distance are required wouldn't benefit. Nice bit of work though and I reckon with some smart scaling embedded too (rather than its 'folding effect'), it'd cater for most image retargetting requirements.
Re: (Score:2)
Re: (Score:2)
There are circumstances where it makes sense to abridge (or retarget) and others where it makes more sense to simply rescale. Since this appears to allow the content provider to choose the method that will be used, the overall effect should be fairly acceptable. For existing content, and future unmarked content, some guesswork will be necessary, and I would imagine this is going to be the hardest part. Computers still can't "see", so getting them to assign t
Re: (Score:2)
Exactly. I keep seeing this as an extension to HTML, and what I'm coming up with is this (or the same in CSS, actually):
All the parameters should be automatic by default, especially scalebelow -- on auto, once the least-important path in the image exceeds a given complexity, the algorithm would switch to simply scaling down the previously-smallest retargeted image. All in all, pretty exciting stuff for web design. Especia
Re:A picture speaks a thousand words... (Score:5, Informative)
he uic bownfoxjumed verthelaz yelowdog
You get:
Th qik brwn fx jmpd ovr th lzy ylo dog
Which reduces the total size by the same amount, but retains more information than treating every bit of information the same.
Re: (Score:2)
lol.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Re:A picture speaks a thousand words... (Score:5, Interesting)
if you have 3 people in a picture and you crop it down to 2, you've erased a person, but you haven't changed who is seated next to whom. if you use this method and the middle person is erased, you make it appear as though the outer two people were in fact seated next to each other when they weren't.
we are used to the idea that a picture can be cropped (mentally considering what might be just outside the frame). We aren't yet used to the concept that the photo has effectively been cut and pasted together to create new relationships between the objects in the photo (though of course photoshop is getting us there).
to continue your analogy, if we take:
the quick brown fox jumped over the lazy dog
and drop letters, we can create:
the cow jumped over the dog
whereas "cropping" might let us say:
the quick brown fox jumped
I think it's clear that one of these is more misleading than the other, though in both cases you're just removing information. (in one case, some of that information happens to be spaces between letters/words)
Re: (Score:3, Insightful)
Hopefully someone will write a GIMP plugin and we can all experiment with it. Also a firefox plugin. Obviously some metadata will eventually nee
Re: (Score:2)
given the choice of an image which is cropped, or an image which had this done to it, if i'm trying to use the image to reconstruct the "truth" of a scene, i want the cropped image. it's at least slightly less artificial.
but for practical applications, such as browsing the web on a 320x320 screen, y
Re: (Score:3, Insightful)
There are probably a few situations where the 'unimportant' bits of an image are still as relevant as the rest. Sports photos for instance - especially those played on grass - would not give you a true picture (literally) of what's going on in the scene.
Sorry -- "true picture?" That assumes such a thing can exist in the first place. Take a color-blind viewer for instance. Can he (and I say he because statistically, most color-blind people are male) look at ANY image and say that he is seeing the "true i
Re: (Score:2)
This brings to mind, what about resizing of video?
DP Approach (Score:4, Interesting)
or entropy of the background is as great as the foreground. Also the paper doesn't go into
too much details about the dynamic programming approach they used to find the path of least
energy, I guess that aspect of it is patentable. Another thing they could investigate is the
use of diagonal seams instead of just staggered vertical and horizontal seams.
All in all a very interesting read.
Re: (Score:2, Insightful)
Re: (Score:2)
Re: (Score:3, Informative)
Also the paper doesn't go into too much details about the dynamic programming approach they used to find the path of least energy, I guess that aspect of it is patentable.
Not so much patentable, as "Easy enough for the reader to implement that it deserves little mention."
Prior art (Score:2, Informative)
Before [wikipedia.org]
After [wikipedia.org]
Insignificant person removed.
Re: (Score:3, Informative)
Whao (Score:5, Funny)
Gimp! (Score:5, Interesting)
Paranoia! It's not just for Gimps (Score:2, Insightful)
Is that check going to cover the removal of their paper from above and the ACM archives, let alone OUR archives?
Re: (Score:2)
I can see the spam now (Score:3, Funny)
Re: (Score:2)
Open source alternative via the GIMP:
Does Anyone Find It Ironic (Score:5, Funny)
Re: (Score:2)
Finally! (Score:2)
(Yes, I know, this thread is worthless without pictures)
Re: (Score:2)
Very ugly (Score:2)
My Implementation (Score:5, Interesting)
Forget about resizing! (Score:2)
removing the intended layout (Score:2, Insightful)
Re: (Score:2)
If you don't like it, don't use it. If you don't use it, nothing gets destroyed.
It's not like cropping pictures was a new invention of the digital age either. Good old analoge photographers have been cropping and "zooming during development" (as opposed to while taking the picture) for ages as well in order to improve the artistic aspects of their work, especially for "action" pictures. And newspapers most definitely have been cropping for decades.
In short, the artistic argument is nonsense. Much more
This is like HTML vs PDF (Score:2)
When I looked at those ads my thought was "man, the PDF version is unreadable at this resolution, but the HTML might look ugly but I can read all of it".
As PDF came into use my fears were realized. PDF documents are all too often unreadable on anything but the largest screens, and I sometimes have to blow them up even on my 23" cinema display.
When it comes t
Better Quality Video (Score:2)
---
Contronyms: for people who are chuffed by antonyms
---
Youtube is definately the best medium.... (Score:2)
Re: (Score:2)
image removal (Score:2)
There's no reason why they co
Image insertion and panorama creation (Score:2)
Ariel Shamir (Score:3, Informative)
some code (Score:4, Interesting)
http://rafb.net/p/jinioy45.html [rafb.net]
(yeah my coding sucks but it produces awesome results and I reversed engineered the algorithm from youtube so please grovel...)
I'll improve it soon to remove an arbitrary number of line, horizontally or vertically
- no recalculation of gradient, only the gradient near the line needs to be recomputed
- precomputes a file that store the order of the pixel needing to be removed
I need help with something though, I understand how the algorithm can precompute a file which says in which order pixel should be removed, but I don't see how this can work in *both* direction. Suppose you want to reduce vertically and horizontally at the same time, the horizontal change should completely break the precomputed vertical changes. How would you handle that?
Re: (Score:3, Interesting)
original
http://img96.imageshack.us/my.php?image=testxq4.jp g [imageshack.us]
somewhat reduced
http://img361.imageshack.us/my.php?image=outew8.pn g [imageshack.us]
very reduced
http://img484.imageshack.us/my.php?image=outas2.pn g [imageshack.us]
Re: (Score:3, Funny)
Re: (Score:2)
Part of it sure beats shredding photo's (Score:2)
I know this isn't adding much to the discussion... (Score:2, Funny)
Hey now.. (Score:2)
Less impressive when you look at the hires movie (Score:2)
Re:I For One (Score:4, Insightful)
I'm really impressed. Again, maybe not too hard to implement at first, but probably damn hard to get working perfectly, and I might just be ignorant (and I'm entitled too, it's far from my field of work), but I've not seen anyone doing it before.
Re:Let us be wholly thankful... (Score:4, Funny)
hehe... gaping... deep deep... rectum... i mean rectify... hehe
i need to get some sleep
Re: (Score:2)
"This technology could render very visually-convincing (but not computer/analytically convincing) image censorship or alteration. I am strongly reminded of this example of photo-editing from the 1940s:
http://www.newseum.org/berlinwall/commissar_vanish [newseum.org] es/vanishes.htm "
Need I remind you, komerade, kommisar Nikolai Yezhov was originally ADDED to pikture, and that our Ministery of Truth only restored the photo to original kondition? Everyone knows that in Soviet Russia, photo alters YOU! Now, your papers
Re: (Score:2)
Re: (Score:2, Interesting)
It could be worse.
In December 2001 The New York Fire Department unveiled plans for a statue based on the photograph [wikipedia.org] to be placed at the Brooklyn headquarters. In an effort to be politically correct, the statue was to include black, white, and Hispanic firefighters. However, it was cancelled in an outcry about rewriting history -- the depicted firefighters are white.
Re: (Score:2)
Re: (Score:2)
Re: (Score:2, Informative)
Fixed [newseum.org] that link for ya.
Re: (Score:3, Insightful)
Re: (Score:2)
Re: (Score:2)
I really like the way where the people are erased and you can hardly notice it. But the when it comes to the faces, the algorithm seems to need more work.
The authors have already demonstrated how automated face recognition could be applied to protect face-like areas on the images. I'm not sure what else you want.Re: (Score:3, Funny)
Re:Great - We can do this, but should we? (Score:5, Insightful)
By your reasoning
Cars can be used by criminals to travel faster.
A knife can be used to kill
Electricity can be used to kill
Computers can be used by the govt to collect more information abt us effectively
Is that really what we want?
see the flaw in the logic?
Re: (Score:3, Funny)
Re:Not ready for Prime Time (Score:5, Insightful)
It has nothing to do with edge detection. The algorithm simply detects paths of minimal gradient which lead from one side of the image to the opposite side. This can be used to produce a "pretty picture" which shows the edges -- but this is merely fallout.
They showed what I thought were several realistic photos with complex backgrounds, and the algorithm did well overall, except on structures where people are closely attuned to exact detail -- such as human faces. If we weren't innately wired to process faces in incredible detail, we wouldn't even notice the distortion.
So it's not perfect. Can you show me something in this world that is? And I don't think there has been any mention of "prime time" application, whatever that means.