Please create an account to participate in the Slashdot moderation system

typodupeerror

## Algorithms Determine Mona Lisa's True Emotions349

Posted by Zonk
from the better-than-ticked-off dept.
caffeinemessiah writes "The BBC reports that researchers at UIUC and the University of Amsterdam, Holland have used "emotion recognition" software to determine Mona Lisa's true emotions. The algorithm is based on a library of neutral face images of young women and determined that Mona Lisa was 83% happy and 9% disgusted." From the article: "The program, developed with researchers at the University of Illinois, US, draws on a database of young female faces to derive an average 'neutral' expression. The software uses this average expression as the standard for comparisons. The New Scientist says that software capable of recognising emotions just by looking at photographs could lead to PCs that adjust their response depending on the user's mood. "
This discussion has been archived. No new comments can be posted.

## Algorithms Determine Mona Lisa's True Emotions

• #### You know (Score:3, Funny)

<fidelcatsro@@@gmail...com> on Friday December 16, 2005 @12:12PM (#14272556) Journal
First time I saw the painting I said "She looks bored ".
She had likely been sitting there for hours having her painting done , likely irritated , in need of the toilet and bored .

Perhaps since the knew study is out , we have discovered that Da Vinci painted naked and was fairly good looking . She was probably thinking "Oh dear lord , he is nude . Oh wait , fairly hot body though .. must not look interested , I don't want to appear easy , but Meooooow"
• #### Re:You know (Score:3, Informative)

But he wouldn't have had much interest in her (except as a model he was commissioned to paint), as he was gay.

As for the article... I think these folks just need a hobby.
• #### Re:You know (Score:2, Insightful)

He wasn't "gay" in the sense that you think of today. The concept of a gay identity (in other words another "us vs. them"-ism) is a relatively modern construct that arose during Victorian times, especially during the trial of Oscar Wilde.
• #### Re:You know (Score:5, Informative)

on Friday December 16, 2005 @01:12PM (#14273114) Homepage

Here [about.com] is an expansion on that...

What is absolutely untrue is the reference in The Da Vinci Code to Leonardo's reputation as a "flamboyant homosexual". He was not known as such. Historical evidence is sketchy about the latter, and the only thing Leonardo was "flamboyant" about was his inability to finish projects he started.

• #### I HAVE NO IDEA WHAT IM TALKING ABOUT (Score:3, Funny)

his inability to finish projects he started

Like that helicopter, for instance.
• #### Re:You know (Score:2, Insightful)

And analyzing art isn't a hobby?
• #### Re:You know (Score:2)

Which is a very interesting thought to imagine the subject having, since it is widely thought that the Mona Lisa is a self portrait.
• #### Not "bored", man! (Score:2)

Here's one of the young female faces used in the mathematical analysis:

http://ellenfeiss.net/temp/movie.php?movie=movies/ ellen_feiss.mov [ellenfeiss.net]
• #### Thank god, I am not the only pervert here (Score:5, Funny)

on Friday December 16, 2005 @12:58PM (#14272975) Journal
I thought almost the same thing. Except knowing something about italy she probably said "miaaaoooo".

She has that expression of a woman looking at a naked man and being faintly aroused but also faintly amused at that "last chicken in the poultry shop" display. If your in a long relationship were your girl still fancies you but feels secure enough she doesn't have to constantly worship as a god to keep your ego up you will learn to regonize that look. Oh well it is better then when they break down in laughter while you are trying to pose seductivly. I guess men just aren't designed to look good naked with their socks on.

• #### Re:You know (Score:3, Interesting)

creepy how she seems to be smiling, but if you look right at her mouth, her smile fades.

Back up to her eyes, smile.
Down to her mouth, pissed.
Eyes, smile.
Mouth, pissed.

Bet if you just stare at her boobs she would fold her arms and just glare at you.
• #### And... (Score:5, Funny)

on Friday December 16, 2005 @12:12PM (#14272557) Homepage
83% happy and 9% disgusted

and 8% lost, seemingly.
• #### Re:And... (Score:5, Informative)

on Friday December 16, 2005 @12:20PM (#14272639) Homepage
And to modify your sig: Those who can't RTFA, complain about stupid crud

Quote from the third paragraph in the article:
It concluded that the subject was 83% happy, 9% disgusted, 6% fearful and 2% angry, New Scientist magazine was told.
• #### Re:And... (Score:3, Insightful)

This doesn't change the fact that the article description is still stupid. It should have included that extra 8%.
• #### Re:And... (Score:2, Funny)

Kinda how Dutch locals look at you when they realize you are a drug tourist. Or how you look at them. I forget.
• #### Re:And... (Score:2, Informative)

Actually, their determination was that the smile is "83 percent happy, 9 percent disgusted, 6 percent fearful, 2 percent angry, less than 1 percent neutral, and not surprised at all."
• #### Clippy 2008 (Score:5, Funny)

on Friday December 16, 2005 @12:13PM (#14272563) Homepage
Clippy: Ok man, I was just saying... I should really just go, sorry.
• #### Re:Clippy 2008 (Score:2)

I don't think clippy really cares how you feel.
• #### Re:Clippy 2008 (Score:2)

Hello! It looks like you are trying to write a letter, may I be of...oh, oh man...geez, I'm sorry...sorry!
• #### Re:Clippy 2008 (Score:2)

I think we'd all be much better off if Clippy could simply be taught to recognize an upturned middle finger...

Come to think of this, this would be a valuable tecnique for improving user interfaces. Just have it send feedback to the authors whenever it detects the user cursing, making obscene guestures at, hitting, or otherwise abusing the computer! Just think how quickly clippy would get pulled out if every time he popped up on Steve Ballmer's machine, it detected him throwing a chair across the office!

• #### So... (Score:5, Funny)

on Friday December 16, 2005 @12:13PM (#14272564) Homepage Journal
She did have gas.
• #### Why can't we just say (Score:2)

that Mona Lisa looks like she's "pleasant". She doesn't have to be happy, or smiling....to me she just looks like she's kickin' it, and doesn't really feel like much of anything.
• #### How this probably works ... (Score:5, Informative)

<eldavojohn@nOsPam.gmail.com> on Friday December 16, 2005 @12:13PM (#14272567) Journal
You can find corpuses [face-rec.org] of human faces taken with different emotions displayed.

Once you've either collected them yourself or downloaded them, you need to use a process called eigenanalysis which is basically fancy talk for analyzing a large dataset with multiple classes (emotions) using matrix decomposition.

I've actually worked on many projects involving this and the result is an eigenface (or eigenmask) [mit.edu] that allows you to transform the space that the original image is in and classify it using any of a number of algoirthms that use euclidean distance.

I know I left out a lot but there are many papers out there that you can find on citeseer [psu.edu] and white papers floating around out there [ucsb.edu] that provide a lot of reading material on this.

There are also strategies which require tagging certain features as points on the face (like corners of eyes, corners of mouth, center of eye, etc) and then using the relative distances between all these points to determine what classification you would give a new face. The problem with this is that it requires a lot of hand work to prepare the training set.

Hope this helps anyone who wants to learn more about the actual process used to accomplish this recognition.
• #### YES! (Score:5, Funny)

on Friday December 16, 2005 @12:55PM (#14272950)
Hope this helps anyone who wants to learn more about the actual process used to accomplish this recognition.

You can bet your purple pants it does!! I can finally put an end this this scenario:

Wife: "no, there's nothing wrong, I'm not mad at you"
muhahahaha.... thats when I take the polariod and get a snapshot
Me: "Yeah right, we'll just see about that!"

Two weeks later the divorce goes through and my ass is on the curb.
• #### I don't buy it (Score:5, Funny)

on Friday December 16, 2005 @12:14PM (#14272577)
Mona Lisa is a woman, how can any software possibly tell what she is really thinking?
• #### Re:I don't buy it (Score:2)

Easy: the researchers perfected a revolutionary technique of writing non-male software!
• #### Re:I don't buy it (Score:2, Funny)

As long as it's also not female; because women can't tell you how they're feeling, either. You're just supposed to KNOW.
• #### oh crap... (Score:3, Funny)

... so now machines are both better than men in bed, and better at knowing how the women feel?

Guys, we're becoming extinct here...
• #### Re:oh crap... (Score:3, Funny)

I think if machines were better then men in bed we would have been extinct a long time ago. Luckily due to the massive conspiracy throughout academia to stop women from becoming engineers the risk of the prophecies coming true is minimal.
• #### Just goes to show, computers are smarter then men (Score:3)

They should make this software available on a camera phone. Next time your girlfriend claims you don't understand here snap her picture and voila, you instantly will be a sensitive man. Anything to keep the dumb cunt happy eh? What?
• #### Mona Lisa a Woman? (Score:3, Funny)

90% happy, 10% disgusted sounds like a drag queen to me.
• #### I'm..... (Score:2, Informative)

by Anonymous Coward
22% hungry and 88% constipated
• #### Next Microsoft Office (Score:2)

The New Scientist says that software capable of recognising emotions just by looking at photographs could lead to PCs that adjust their response depending on the user's mood.

Finally!
Computer:"Clippy senses you're getting pissed off at it and want to kill it! It'll go hide in a corner now out of view. So sorry!"
• #### Re:Next Microsoft Office (Score:2)

If I had written this software I would just determine if the subject was male or female from the user name.

Then, if the user is female I'd hard code the mood to "mood for shopping" and if male I'd hard code it to "mood for porn".

There, five minutes of coding and the result is spookily accurate.
• #### Golden Ratio (Score:2)

83% happy and 9% disgusted

To achieve fulfilment, a woman should strive for balance.

• #### Re:Golden Ratio (Score:2, Funny)

83% happy and 9% disgusted

To achieve fulfilment, a woman should strive for balance.

So ... on the back of her carriage, Mona Lisa La Giaconda should have had a brass plaque which said

I'm 50% Happy and 50% Disgusted. Don't push it
• #### A painting isn't a photograph (Score:5, Interesting)

on Friday December 16, 2005 @12:15PM (#14272591) Homepage
This of course assumes that DaVinci captured her exact expression... Chances are that the painting just developed that way. Anyone who does art by hand knows that it's not a photograph and that the painting more or less takes on it's own personality as it's being created.

If it were a photo then yes I'd be more apt to accept an algorythmic interpretation of the image.. but paintings take time and it's doubtful that a person feels the exact same way over the course of days or weeks or even months it took for this painting to be completed.
• #### Re:A painting isn't a photograph (Score:3, Insightful)

It's not really about the subjects expression, it's about the painting's expression. People find the smile very interesting in this painting, and that is most of the reason for the painting's particularly great fame. Why is the smile so interesting to people? Perhaps it is the 9% disgust. That's what this kind of analysis can hope to tell us about our response to this painting.
• #### Re:A painting isn't a photograph (Score:2)

I agree. It's not capturing a moment in time like a photo. People have a range of emotions while sitting for a while. Hey.. maybe he told a dirty joke halfway through the painting and that's where the disgust came from.
• #### Re:A painting isn't a photograph (Score:2)

Not to mention, that for all we know the analysis may be completely wrong.

Yeah, she's 83% Happy, 9% Disgusted, 6% Fearful, 2% Angry.

We're sure she's not 87% Happy, 2% Disgusted, 6% Fearful, 5% Angry? What kind of degree of certainty do we have with these numbers?
• #### Re:A painting isn't a photograph (Score:2)

The aim of the study is to decode the emotions of Mona Lisa IN the painting, not the emotions she had while the painting was done. As it is not a photograph, rendering the paint would have taken _several_ sessions, with changing moods.
• #### Camera Obscura, etc (Score:5, Interesting)

by jd (1658) <<moc.oohay> <ta> <kapimi>> on Friday December 16, 2005 @01:19PM (#14273172) Homepage Journal
It was not unusual, at that time, for artists to use a range of techniques for capturing the key points of the person they were painting, to avoid having a person sit for ages whilst being painted. This means that although the painting would not be a "true" photograph, it could have been extremely close to one.

On a related note, this might also explain the resemblance to Leonardo. Let us say that he did, indeed, have a woman sit for just long enough to sketch in the key facial lines. He would then have needed to add in the skin texture and other features that couldn't have been captured by whatever method he used. It would be logical for him to have used his own face to capture such information. The Mona Lisa would then have been a composite of the original model and himself, which means that it would indeed have a resemblance to him.

X-Ray analysis of the original painting reveals sketches and paintings below the Mona Lisa - though there was no sign of anyone having written "This is a fake" in felt-tip pen, much to the chagrin of Doctor Who fans. It would be interesting to know how the different levels relate to each other - were the earlier pictures earlier versions of the same painting? If they are analyzed with the same software, does it produce the same result?

• #### Annoying (Score:2)

software capable of recognising emotions just by looking at photographs could lead to PCs that adjust their response depending on the user's mood

You mean, like Clippy, but even more annoying?

• #### translated into "emoticon recognition" (Score:5, Funny)

on Friday December 16, 2005 @12:18PM (#14272619)

83% happy

.83(:-))

• #### She doesn't have emotions (Score:2, Insightful)

Mona Lisa doesn't have emotions. She's made of paint.
• #### Re:She doesn't have emotions (Score:2)

Shhh! Do you have any idea how upset she would be if she found that out?
• #### The Slashdot Method (Score:5, Funny)

<null AT filmcritic DOT com> on Friday December 16, 2005 @12:20PM (#14272636) Homepage
I find this story 83% Interesting and 9% Funny.
• #### Clippy of doom! (Score:2, Funny)

by Anonymous Coward
could lead to PCs that adjust their response depending on the user's mood

Lovely, now clippy can pop up with "You look like you're frustrated as hell with Microsoft Office. Would you like to buy some Microsoft stock?"

Death to clippy!
• #### How does that work? (Score:2)

...software capable of recognising emotions just by looking at photographs could lead to PCs that adjust their response depending on the user's mood. Great... how is my computer going to adjust itself to my looking horny and bored?
• #### The way things are going... (Score:2)

I think we have to look at the more likely application: Detection of thoughtcrime!
• #### Finally! (Score:2, Informative)

Interesting to see this idea actually working now. I think I first saw this five years ago on IBM's Alphaworks site. Ah yes, here it is.

• #### Warning Signs (Score:5, Funny)

<satan@programmer.net> on Friday December 16, 2005 @12:28PM (#14272717)
software capable of recognising emotions just by looking at photographs could lead to PCs that adjust their response depending on the user's mood.

If your computer says "I know I've made some very poor decisions recently, but I can give you my complete assurance that my work will be back to normal. I've still got the greatest enthusiasm and confidence in the mission. And I want to help you," then you should pull the plug immediately.

• #### 200% (Score:2, Funny)

I want to try and trick it and simultaneously make all the expressions that i can. I could be 17% happy, 40% confused, and 85% constipated.
• #### 3% (Score:2)

Oh dont forget 3% SBD (silent but deadly). But maby thats just the smile I make when that happens to me. *must think unfunny thoughts ... must not laugh*
• #### Nice trick these researchers have discovered (Score:5, Insightful)

on Friday December 16, 2005 @12:29PM (#14272724)
1. Invent algorithm
2. Apply it in a domain where your work can't be falsified
3. ???
4. Profit!
• #### And the ??? stands for: (Score:5, Funny)

on Friday December 16, 2005 @12:48PM (#14272886) Homepage
(this is actually the first time I play this game, let me know how I'm doing!)

3a. Patent algorithm.
3b. Sue everybody that looks 83% happy or 9% disgusted
• #### Re:Nice trick these researchers have discovered (Score:2)

It's easy to falsify this work actually.

1) train a bunch of humans to accurately identify facial emotions. (train to some target level of competence)
2) have your human facial emotion identifiers look at a lot of faces, keep track of statistics
3) improve your software until it agrees with #2 on a sufficiently large sample set
4) verify the accuracy of your software by checking for agreement with #2 on a new set of previously untested images
5) have great statistical confidence that your analysis of further im
• #### Re:Nice trick these researchers have discovered (Score:2)

So tell me, what value did this research add to just taking a sample of random opinions of what expression the picture portrayed?
• #### Re:Nice trick these researchers have discovered (Score:2)

Are you kidding? Having a computer that is able to judge facial expressions accurately? It adds value in all kinds of ways:

1) In the specific context, it provides for an analysis unbiased by human emotion attached to a famous work.
2) In a general context, a computer which can judge facial expressions may be able to interact more accurately or effectively with you without resorting to primitive mouse/keyboard work.
3) In an alternative context, a computer able to analyze facial emotion may be able to improv
• #### pseudoscience (Score:2)

I read this write-up of the study in question:

http://www.livescience.com/history/ap_051215_mona_ lisa.html [livescience.com]

This isn't science. Jim Wayman, a biometrics researcher, says "It's hocus pocus, not serious science, but it's good for a laugh, and it doesn't hurt anybody." He's right, though this is right up there with those studies that find an equation for the perfect ice cream cone, or whatever. The annoying thing is, people take this shit seriously.

Furthermore, from the link, "it couldn't detect the hint of
• #### Longshot question (Score:3, Interesting)

<zachary.pruckowski@gmail.com> on Friday December 16, 2005 @12:33PM (#14272755)
Is there a chance the facial images have changed over the 1000-1500 years or whatever? I mean, obviously they wouldn't change much, but maybe a little?

More importantly, are we sure da Vinci had regular access to girl's faces? I mean, it was probably mostly guesswork on his part.
• #### Re:Longshot question (Score:2)

Not much chance that facial expression has changed significantly over time. One of the things psychologists have studied fairly extensively is emotion in isolated cultures. Smile==happy no matter where you go. Given that such cultures have often been isolated on the 1000 year range, it seems unlikely that a uniform evolutionary change happened everywhere, and so it is probably fairly safe to conclude that the meaning of facial expressions isn't changing much over time.
• #### A programming problem (Score:5, Funny)

on Friday December 16, 2005 @12:38PM (#14272797)
Okay, here are my questions for the Slashdot community:

1) You're writing some code. You call the User Emotional Analysis API, and it reports back that your user is currently "83% happy and 9% disgusted". How should your software "adjust its response" in reaction to this information?

2) What happy/disgusted ratio leads to maximum productivity?

3) What are the odds that the Mona Lisa is a portrait of a Perl programmer?

• #### What this means she was thinking (Score:2, Redundant)

"Man, that Leonardo is such a stud... but does he really have to paint me while he's naked?"

• #### HAL (Score:2)

PCs that adjust their response depending on the user's mood.

Hal: Good Morning, Dave.

Just what we all need a computer that can sense we are getting pissed off and attempt to kill us before we kill it, our advantage is over.
• #### Actual results (Score:2)

83% happy
9% disgusted
8% confused as to why anyone would take a COMPUTER'S word about EMOTIONS
• #### So what does that mean? (Score:2, Insightful)

83% happy and 9% disgusted? And just what does that mean? All of life's greatest mysteries can be solved in a quantitative manner? I for one don't want my computer to act differently if I'm happy, sad, pissed off, stoned, whatever. Just what I've always wanted, a computer with a Genuine People Personality (TM).
• #### A note about the Mona Lisa (Score:4, Interesting)

on Friday December 16, 2005 @12:49PM (#14272894)
This painting was not made in one sitting. Or two. Or ten.

It was never even finished.

The subject, ASSUMING THERE WAS ONE, sat for one or several sessions and then Leonardo continued to work on the painting off and on for the rest of his life.

There is speculation as to who the subject was, but perhaps there was none, and some think it's actually a self portrait in drag (perhaps the cause of the mostly amused but 9% disgusted?)
• #### The da Vinci Commode (Score:3, Funny)

on Friday December 16, 2005 @12:51PM (#14272905) Homepage Journal
Could her smile be the result of one of da Vinci's inventions [aucklandmuseum.com], ie the vibrating commode that the lady in question was sitting upon?
• #### Man... (Score:2)

determined that Mona Lisa was 83% happy and 9% disgusted

Now I'm 100% bored...
• #### Oh, goody. (Score:2)

Now all we need is a computer that can tell us what Ellen Feiss was on.

Er, what she was thinking.

• #### Next up: Minority Rapport (Score:2)

Thank god we don't have a linguistics engine capable of reproducing stuff like that yet. Else we'd be most of the way to:

"How did that scoop-neck camisole work out for you, Mr. Takemura?... Mr. Takemura?? I KNOW YOU CAN HEAR ME DAMMIT!"

or the shrieking vegetables courtesy The Onion: "I HAVE 37% MORE VITAMIN D THAN THAT CUCUMBER! STOP LOOKING AT HER!"

If there is any hope for human sanity in 20 years, all linguists must stop working on computers IMMEDIATELY.

• #### Facial changes over time (Score:2)

But what about the time period it was painted versus modern facial expressions. Everything else changes, speech, speech patterns, gestures... why would facial expressions not be slightly different now than they were at the time of the painting?
• #### 90% Lisa, 10% Mona (Score:5, Interesting)

on Friday December 16, 2005 @12:57PM (#14272972)
On a related note, back in February, I searched Flickr for photos matching the tags Lisa or Mona.
The results indicate that 9/10ths of the women in these photos are named Lisa. I built
a photo mosaic from the results, which can be seen here:

on Friday December 16, 2005 @01:00PM (#14272997) Journal
• #### Mona Lisa is actually a self-portrait (Score:3, Interesting)

on Friday December 16, 2005 @01:00PM (#14273002) Homepage
Of course, a number of people suspect that the true model for Mona Lisa was Da Vinci himself. I wonder if the researchers accounted for this?
• #### 83% happy and 9% disgusted (Score:3, Funny)

on Friday December 16, 2005 @01:01PM (#14273008) Homepage
And by a strange co-incidence most slashdot readers' interactions with women produce exactly opposite results.
• #### My new mood-adjusting PC... (Score:4, Funny)

on Friday December 16, 2005 @01:01PM (#14273014)
Launching Firefox...
You're bored...
You're horny...
You're horny...
You're disgusted!
You're horny...
You need a cigarette...
You're bored...
• #### Nurture (Score:2)

Facial expression of emotion is in part tied to upbringing. Unless they have really good computer-based images from the year 1503, this can't help but be less accurate than if it were used against a modern image.
• #### Real-world application (Score:2)

I for one welcome this new technology. I look foward to being able
to buy a handheld model that can tell me why my wife is mad at me.
• #### Pfft! (Score:3, Funny)

on Friday December 16, 2005 @01:15PM (#14273143)
I wrote a prorgam that did this on my old Atari 800 back in the computer Steam Age.

I got:
44% "Happy"
12% "Baffled"
21% "Knowing lesbian smirk"
19% "Get your hand off my knee, Leo"
55% "Planning to start new religion"

I also analyzed the brush strokes and built a picture of DaVinci:
54% "Depressed"
61% "Inventive"
10% "Horny"
30% "That's not my hand, Mona"
71% "Must encode holy grail into here somehwere"
11% "She'd make a good tank"

#### Related LinksTop of the: day, week, month.

"Why should we subsidize intellectual curiosity?" -Ronald Reagan

Working...