Text Analyzer Reveals Emotional 'Temperature' of Novels and Fairy Tales 65
KentuckyFC writes "Stories are a powerful channel for communicating emotions. But while they have been studied in detail by generations of critics, there is little in the way of objective tools for analyzing and comparing their emotional content. That looks set to change thanks to one data mining researcher who has applied the process of sentiment analysis to novels and fairy tales that have been digitized on Project Gutenburg and the Google Books Corpus. The results show the density of emotions in different parts of a story and how the emotional 'temperature' changes throughout the tale. For example, this guy has used the technique to compare the emotional content of the entire collection of the Brothers Grimm fairy tales to reveal that the darkest story is a tale called Gambling Hansel; clearly a lesson to us all."
I usually start with a Gambling Hansel.... (Score:3, Funny)
then I move to a Dirty Gretel and finish all off with a move I call the Sugar Fairy Plums.
Ironic captcha: bodice
Re: (Score:3)
Gambling Hansel: not dark at all (Score:4, Interesting)
I didn't find it dark at all, not nearly as dark as the tales Disney sanitized. I mean, it's about a gambler who beats both God and the Devil even if he has lousy luck with mortals prior to getting rigged cards and dice.
Re: (Score:3)
well it's pretty dark that seemingly people on earth have no choice but to gamble nor do beings in hell and heaven.
other than that, it's pretty light hearted for a grimm.
there's even 7 years in gambling hansel during which nobody dies, since he beat death too - and his soul lives eternally in gamblers.
so the algorithm is pretty contextually unaware. maybe it just counts nots and buts. waste of time anyhow, even if it did get me to read gambling hansel.
Re: (Score:2)
It's actually one of my favorites. I read through the whole collection as a child, and even having not read any of the stories again for over 20 years, still remember how he cheated Death, the Devil, and God Himself. It was amusing at the time, and still is now. What I took away from the story back then was that Someone fucked up big giving him the stacked deck and loaded dice.
As far as I can recall (note that I haven't read them in some 20 years), there are definitely darker ones that don't involve death,
I'd like to see the emotional temperature of A Son (Score:1)
I'd like to see the emotional temperature of A Song of Ice and Fire.
Re: (Score:1)
Eddard dies.
So?
This Guy (Score:2)
The summary doesn't even mention the researcher's name? I mean, I agree that this is useless, pointless "research". But if you're going to piss about and drop Project Gutenburg and "Google Books Corpus" which are only tangentially related, couldn't you at least give "this guy" a fucking name?
Re: (Score:3)
Ahhhh.... This guy... *points at guy...*
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
Re: (Score:2)
It's Project Gutenb*e*rg
And that's what I initially typed. I had to force myself to copy Gutenburg from the summary.
It's a Pot Boiler! (Score:3)
"How yer know that?"
"It's written in charcoal."
i'll get me coat
The "eight fundamental emotions" (Score:4, Interesting)
From TFA:
Analysing the emotional content of text is also becoming easier. In recent years, researchers have built up significant databases of the emotions that a given word evokes. This is part of the new field of sentiment analysis in which common words are categorised as positive, negative or neutral and associated with one of the eight fundamental emotions—joy, sadness, anger, fear, trust, disgust, surprise and anticipation.
I don't know about anyone else, but I found that bit as fascinating as the text analyzer itself. But where does laughter fit? Shouldn't it count as a fundamental emotion? Or is it considered just a sub-category of "surprise" or "joy"?
In any case, I wonder if someone could combine all that with the 36 dramatic situations [wikipedia.org] and a few other components, and create a program that writes stories....
Re: (Score:2)
they didnt fit satisfaction in there, not buying it.
Re: (Score:2)
I could see classifying "satisfaction" as a type of "joy."
Re: (Score:3)
That is a rather limited way to look at satisfaction. Relief, catharsis, vengeance, schadenfreude, comeuppance to name a few could all be descriptive of something satisfactory, but might not cross over into joy so much.
Re:The "eight fundamental emotions" (Score:5, Funny)
That's because the authors couldn't get no satisfaction, though they tried and they tried, and they tried, and they tried.
Re: (Score:2)
they didnt fit satisfaction in there, not buying it.
Guess they.... "couldn't get no..."
Re: (Score:3)
I don't know about anyone else, but I found that bit as fascinating as the text analyzer itself. But where does laughter fit?
Yeah, while this theory of emotions surely has some good aspects, forcing ALL emotionally charged words into these categories will obviously skew the data in certain ways. When a model like this is used to classify something much more complex, the ultimate data analysis often tells you more about the model than about the data. Are we actually tracking the changes in "sadness" and "fear" over the course of Hamlet, or are we tracking some arbitrary dividing line that this model forces us to use to classify
Re: (Score:3)
But they really need to factor in context, multiple meanings, and especially other factors that might lead to high frequencies of their chosen "emotional" words, like proper names or other plot points that may not actually be representative of the vocabulary and emotions of the story overall.
You are correct. Obviously this sort of text analyzer is still in its infancy. It would be interesting to throw some oddball stories at it and see the results. E.g., here's a story filled with unpaired words [newyorker.com]. I wonder what it would say its "emotional temperature" was? And of course the program would totally miss the humor. (Note that the New Yorker blew the formatting when they put this online, and that the actual story starts with the third sentence: "It had been a rough day....")
Re: (Score:3)
Re: (Score:3)
Hmmm, interesting. But what is laughing at a joke? It isn't necessarily nervous, or joyful. It's sort of a relief, in that the tension of the buildup is released in the punch line, but that doesn't seem to be the same as "laughter of relief." Humor, or laughter, just seems to me to be as core of an emotion as fear or anger, but maybe psychologists and data miners don't see it that way.
Re: (Score:2)
Agreed. Laughter of relief is laughing when something turns out not to be as bad as you thought it was going to be, which is the polar opposite of humor.
A sufficiently lighthearted story can often be described as simply the absence of darkness, but that's not necessarily the case for all humor. Consider the whole genre of dark comedy, in which the story is funny, but involves really dark, depressing situations that are merely portrayed in a lighthearted way.
IMO, there's definitely a category of laughter
Re: (Score:2)
From TFA:
In any case, I wonder if someone could combine all that with the 36 dramatic situations [wikipedia.org] and a few other components, and create a program that writes stories....
Someone has ... every Hollywood studio has it running on their server farms
Re: (Score:3)
Codswallop! This notion has been around for a very long time under the name of connotation. Giving something a new name and peddling it as a new concept doesn't exactly inspire confidence in the writer's competence or integrity.
Think of the Children (Score:2)
[the end]
Tropes (Score:3)
Re: (Score:3)
Let me know when it can call out all the TVTropes in a story...or would that cause an endless loop?
This troper believes that the fun in visiting tvtropes is knowing other geeks have enjoyed each story... or *hated* it, and in many other ways been driven to break down the story for your enjoyment due to resonating with that story emotionally. Call it pride in exclusivity, sort of like coming to slashdot looking for fellow geek observations, tips, jokes, and so on. In short, tvtropes is fun because you're finding someone else that you share some knowledge with in a way the corresponding Wikipedia article c
Re: (Score:2)
This troper...
There's a trope for that.
Here is the PowerPoint for the paper (Score:5, Informative)
Here is the a good summary of the work in a PDF of a PPT.
http://www.saifmohammad.com/WebDocs/LaTeCH-emotions-in-books.pdf [saifmohammad.com]
Ted
Re: (Score:3)
1. used Mechanical Turk to get people to report association of words with emotions
2. determine emotion by counting(!) corresponding words with weighting proportional to association, using sliding window in time.
3. generate pretty but almost meaningless plot [postimg.org]
4. profit?
hint: Dramatic tension is often created through irony. The audience knows that the doom of character X (e.g. Walter White) comes when he finally comes to trust character Y (e.g. the white nationalist thugs) implicitly. Goddammit, at the very lea
Re: (Score:3)
Here's a good basic emotion measure: adjective count. It covers the bulk of written text even though one can find ways around it.
The Bible? (Score:2)
> Text Analyzer Reveals Emotional 'Temperature' of Novels and Fairy Tales
How would the Bible be colored?
Per book should be interesting.
Re: (Score:3)
Re: (Score:3)
my favorite book of the Bible is Ecclesiastes [bartleby.com] which, being basically an existential musing on the meaninglessness of life by, ostensibly, King Solomon, is considered so out-of-place that scholars have been trying for about two thousand years to figure out why the hell it was included in the Hebrew canon.
Re: (Score:3)
The best answer that I have heard is that the existential nihilism that is covered by the book is an important aspect of Jewish / Christian traditions and that all wise people must confront it and think about it. The idea is so central that it even suggests the idea that God himself wrestles with this question and more specifically in the Christian Tradition this is what the Christ wrestled with on the cross when he cried out "Why have you forsaken me?" The saints, holy people, and mad men through out hi
Re: (Score:2)
And again, the Christians have to retcon Him into every piece of scripture, thereby fundamentally changing its meaning. (I'm not Jewish, btw, I just find it to be an odd obsession.)
But, back on topic, I agree with your general interpretation.
Something that's perhaps overly interesting to me is that the Christian bible usually has something like this translation ``I returned and saw under the sun that the race is not to the swift, nor the battle to the strong, nor bread to the wise, nor riches to men of unde
Re: (Score:3)
You misunderstood the comment. TedSchopp is saying these are such fundamental questions that every Jew and Christian grapples with, even the Christ who is regarded by Christians as the incarnation of God. The saying of Jesus on the cross "Why have you forsaken me?" is very much of a piece with the existential despair of Eclesiastes. What could be worse for Jesus at that point than feeling that not only is he dying in this horrific way, but that it is a completely meaningless experience, as is the whole o
Re: (Score:2)
Thanks you got my meaning and made it clear than I could myself.
Re: (Score:2)
and more specifically in the Christian Tradition this is what the Christ wrestled with on the cross when he cried out "Why have you forsaken me?"
He was quoting Psalm 22, not wrestling with his conscience. Psalm 22:1 says "My God, my God, why have you forsaken me? Why are you so far from saving me, from the words of my groaning?" It continues, in Psalm 22:14-21
Related stuff by my wife on tagging narratives (Score:5, Informative)
Mainly by hand though. Free book: http://www.workingwithstories.org/ [workingwithstories.org]
Free software for communtieis: http://www.rakontu.org/ [rakontu.org]
Related business process patent (sadly) when at IBM Research: http://www.google.com/patents?hl=en&lr=&vid=USPAT7136791 [google.com]
Past commercial software: http://www.sensemaker-suite.com/ [sensemaker-suite.com]
National security (does have some automatic aspects): http://app.rahs.gov.sg/public/www/content.aspx?sid=2955 [rahs.gov.sg]
There is a lot you can do with stories once they are tagged for emotional intensity, whether automatically, by the teller, or by other people. Stories are all around us, as we try to make sense of our lives and events in our communities. So this sort of technology to tag emotions in stories is much more far reaching than just being about fiction. It can be used to design better products, to help communities figure out what to do about a pressing issue, to resolve conflicts, and to see emerging trends. That is one reason such work is funded by the intelligence sector (as well as businesses and some non-profits). She's been trying to make these ideas freely available to everyone, but it has been a slow going slog to follow the path of free and open source for all this.
By someone else on the relation between emotion and reason:
http://en.wikipedia.org/wiki/Descartes'_Error [wikipedia.org]
Text Analysis... (Score:3)
So, this analyses text and emotional connotation of words to produce an emotional score for each story. Yet, it has no way of divining context, whether or not a particular section of the story is funny, or if a death causes an emotional reaction of sadness or satisfaction (i.e. the character was evil, deserved it, etc.). In other words, it's an arbitrary system that may work at a basic level but will still get a lot of things completely wrong... at least it's a start, I guess...
Why won't it read. (Score:2)
Accuracy? (Score:2)
My guess is that the program wouldn't find that grim or depressing at all. Context is everything.
Simple (Score:3)
Carpe Diem (Score:2)
I preferred the version with Robin Williams.