Using The Web For Linguistic Research 205
prostoalex writes "The Economist says linguists are gradually adopting the World Wide Web as a useful corpus for linguistic research. Google is used, among other resources, to research how the written language evolves and how some non-standard examples of usage become more or less acceptable (The Economist quotes the phrase 'He far from succeeded,' where 'far from' is used as an adverb). LanguageLog is a resource linked in the article, where linguists discuss current peculiarities of the English language."
They should probably avoid Slashdot (Score:5, Funny)
Re:They should probably avoid Slashdot (Score:2)
Re:They should probably avoid Slashdot (Score:2)
Re:They should probably avoid Slashdot (Score:3, Interesting)
Actually, I take that back.
It could actually be very interesting from a lexical or morphological point of view. The phenomenon of abbreviating words, such as "u" for "you" or "ur" for "you're" or "ru" for "are you." Language teachers in classrooms have been seeing it crop up in actual homework assignments. While reading such language may be like having glass
Re:They should probably avoid Slashdot (Score:3, Interesting)
One thing that's always been at the front of my my mind, why aren't these kids learning how to type? Or at least to type with any reasonable amount of skill. The only computer I had as a child was a Commodore 64, and I was still faster than most of todays youth even with their abbreviations. I was somewhat lucky in that our schools somehow foresaw the advent of the home computer and made sure we kne
Re:They should probably avoid Slashdot (Score:1, Insightful)
One thing that's always been at the front of my my mind, why aren't these kids learning how to type?
Because, unlike the parent's assumption, the phenomenon isn't related to computers. It's related to text messaging. It might be just as fast to type "you" instead of "u" with a keyboard, but it's noticably slower on mobile phones, especially before predictive text became popular.
Furthermore, there is a limit on how many characters you can send in a single message. Most service providers automaticall
Re:They should probably avoid Slashdot (Score:2)
I'd still advance the argument that extensive use of computers by a larger portion of the population has contributed to the phenomenon. I remember seeing those abbreviations before cell-phone use became almost ubiquitous.
It's al
It looks like no one read the article (Score:3, Insightful)
Re:It looks like no one read the article (Score:2)
I specifically stated that I was retracting my initial comments, but I kept them in there as a tongue-in-cheek statement. I never said anything about "bad language." I'm well aware of the difference between how language is considered by linguists and English teachers.
As to the relation of written language on the web to spoken language, I don't think that's been established. I know that in dialogue systems, which deal with spoken language as opposed to wr
Re:They should probably avoid Slashdot (Score:2)
Same concept. Whole thing got hijacked by politics.
--Dan
Re:They should probably avoid Slashdot (Score:2)
Right now, yes. But in a generation or two, perhaps they'll lose the distinction.
Whoops.
Re: Who should probably avoid Slashdot ??? (Score:2)
I don't think this has been established one way or another. Linguistic complexity would be an incredibly difficult term to objectively quantify.
Re:They should probably avoid Slashdot (Score:2)
Re:They should probably avoid Slashdot (Score:2)
Re:They should probably avoid Slashdot (Score:2)
WTFDTRLASF?
What the f**k does that really long acronym stand for?
Re:They should probably avoid Slashdot (Score:2)
Yeah...
Linux is a community made of of mostly literate folks who generally understand the language well enough to be understood. Zealots complaining about the use of apostrophe 's' as a possessive, you can do worse than slashdot in terms of grammar, easily.
That said, only pedants can claim perfect spelling and grammar at all times, ever.
Indeed (Score:4, Funny)
I rue the day... (Score:3, Funny)
Re:I rue the day... (Score:3, Interesting)
I've heard it done. I've also heard 'roffle' (an attempt at pronouncing ROTFL I guess). Bizarre, really, since those terms are attempts to turn physical real-life actions into a verbal-only form.
Re:I rue the day... (Score:2)
Re:I rue the day... (Score:2)
Unfortunately, the idea conveyed is "I'm a fucking idiot."
Re:I rue the day... (Score:1)
Re:I rue the day... (Score:2)
Re:I rue the day... (Score:2)
Re:I rue the day... (Score:1)
http://home.planet.nl/~cruij087/vin3.mp
Re:I rue the day... (Score:3, Informative)
lol (de ~) 1 [inf.] plezier
(taken from, www.vandale.nl, an authoritive dutch dictionary)
Re:I rue the day... (Score:2)
inner city teens (Score:1)
Re:inner city teens (Score:3, Insightful)
I'm glad they're telling the youth what is proper; you're clearly incompetent to do so.
using words... is becoming more than just the normal, it is becoming the standard.
Is that right? Using words is "becoming more than just the normal"? I've been using words for years now; I'm glad to hear that's becoming the standard. Your post is a perfect example of why people should learn to write in something a
Re:inner city teens (Score:2)
Accent is a drift in the pronunciation.
Dialect tends to bend the meanings of words.
Dialect cannot be considered proper english due to the fact that it isn't widespread(usu. local) and confusion is likely to exist due to conflicting meanings. If I'd told you to "Rush the growler" would you have any idea what I was talking about?
Would you
Pet the dog?
Fill the pail with beer?
Get a sandwich?
Yes, the answer
Re:inner city teens (Score:2)
Yes, people who think that double negatives are in some way illogical are very irritating.
I don't think they're not logical in the least! Double negatives are not always an ungood thing. People just need to learn to not use them when they don't mean to.
good reasons to speak standard English sometimes, but this doesn't mean it's OK to snee
Re:inner city teens (Score:3, Interesting)
Incomprehension often has very little to do with that. A friend of mine moved to MA from NC at the same time as I moved from CA. She could not understand most people there, most people there could not understand her. I could, on the other hand, understand both of them. I've be
Re:inner city teens (Score:2)
It's an observation. If a language has stopped changing, that's typically due to the fact that nobody uses it anymore and it is in a phase of decline.
Epiphany (Score:2, Funny)
Re:Epiphany (Score:2)
Oh I hope you're right. If I every hear someone actually vocalize "lol" or "rofl", I'll punch them in the face.
Google does it again (Score:3, Interesting)
This is not the first time when Google (and search engines in general) changed how we do things.
Nowadays copyrighters use Google to search for potential violations of their intelectual property. Plagiarism is easy to detect nowadays thanks to Google as well. Instead of using rather expensive [turnitin.com] systems in order to search for duplicate work, teachers are now one search away in distinguishing original work from the rest.
Re: (Score:2)
Don't trademark that! (Score:1)
*BSD be dyin' (Score:2, Funny)
One mo'e cripplin' bombshell hit da damn already beleaguered *BSD community when IDC confirmed dat *BSD market share gots dropped yet again, now waaay down t'less dan some fracshun uh 1 puh'cent uh all servers. Comin' on de heels uh a recent Netcraft survey which plainly states dat *BSD gots lost mo'e market share, dis news serves t'reinfo'ce whut we've knode all along. What it is, Mama! *BSD is collapsin' in complete disarray, as fittin'
HAMMER REVOLUTION --; (Score:1)
Re:HAMMER REVOLUTION --; (Score:1)
Re:HAMMER REVOLUTION --; (Score:2)
Programmer grammar (Score:3, Insightful)
Be carefull thought... (Score:3, Interesting)
native speakers.
In the European community the native English
speaking persons are by far a minority. That way
French expressions are poring into the language
in an unstoppable way. Those expressions are then
used by native speaking politicians and are
broadcasted by television. That way they enter the
mainstream of the English language.
Regards
Re:Be carefull thought... (Score:1, Insightful)
native speakers.
Of course, non-native speakers have generally less trouble distinguishing "then" from "than" than the so-called "native" speakers do. You might speak it natively, but remember, you don't write it natively.
Re:Be carefull thought... (Score:1, Funny)
Re:Be carefull thought... (Score:1, Funny)
Re:Be carefull thought... (Score:2)
Now I know some people would be quite upset at the horrible "loss" of cultural diversity implied by a single global language. But we can be just as diverse in many other ways that don't cause us to be unable to communicate with each other on a basic level. And IMHO, being able to communicate is much more important
Re:Be carefull thought... (Score:2)
Now I know some people would be quite upset at the horrible "loss" of cultural diversity implied by a single global language. But we can be just as diverse in many other ways that don't cause us to be unable to communicate with each other on a basic level. And IMHO, being able to communicate is much more important than some academic's ideal of "cultural identity".
Okay... how about the complete loss of the ability to read any of the world's litera
Re:Be carefull thought... (Score:3, Insightful)
By the time we have a unified language, we'll have a whole new set of literature to go along with it. Today's literature will be like ancient greek literature, and yes, it will only be readable by people with special training. It will need to be translat
Re:Be carefull thought... (Score:2, Interesting)
I think you're a bit wrong on this. There are around 6,800 languages [ethnologue.com]. Most languages have developed their own culture. Do you really think millions of people around the globe would be willing to lose their identity?
For example, after the collapse of the Soviet Union, Uzbeks started replacing Russian loan-words with the original Uzbek words.
Paul Rodrigues
Re:Be carefull thought... (Score:2)
Re:Be carefull thought... (Score:2)
But I can't imagine those super groups ever merging
Of c
Re:Be carefull thought... (Score:2)
Of course, if we all move to speaking some kind of ubermetalanguage, that implies that the languages spoken today would be lost. That would be a sad thing.
Language is a reflection of culture, and culture, to date, is a deeply regional thing. The standard example is that the Inuit people of Alaska and Canada have dozens of words for snow; while this seems to be not entirely accurate [straightdope.com], the general point stands that different groups have richer or poorer ways of expressing concepts based on their collective
Re:Be carefull thought... (Score:2)
Done: nous sommes desolés que notre president (Score:4, Insightful)
Those expressions are then
used by native speaking politicians and are
broadcasted by television.
Dude, it's worse, the French have already infiltrated as far as the advertising business and are using covert channels to spread some dangerous crack i heard was called La Liberte
http://french.about.com/b/a/081281.htm
Slightly more seriously
Apart from pointing out that your use of the word native is rather presumptive of geographic origin in this big wide internet thing, i wonder if this linguistic adoption is more one way towards English since the internet. OK the French got Le Weekend, and tons of anglicised nouns, tried to ban them all and didn't manage. But i read Friday that a British pilot training firm lost a contract to a French one. The reason cited by the Asian airline was that, whilst the training had to be in English, the French trainers spoke better, clearer, more intelligble English than did the English. I can't argue with that. Sadly.
Re:Be carefull thought... (Score:2)
Re:Be carefull thought... (Score:2, Funny)
Re:Be carefull thought... (Score:2)
Good. Less to worry about whenever they get restless.
That way French expressions are poring into the language in an unstoppable way.
Ah, but when you pore into the language, the language also pores back into you.
I've used the web for corpus linguistics research (Score:2, Informative)
I could have gotten a higher accuracy rate, but this was just a simple undergraduate project.
Same here (Score:1)
Re:Same here (Score:1)
print "$badgram{vocab}" (Score:1, Funny)
Compression Prize (Score:2)
Re:Compression Prize (Score:2)
Compression is a stricter test for AI than Turing (Score:2)
Non-official English (Score:2, Informative)
Re:Non-official English (Score:2, Insightful)
I'd say you're fighting a losing battle on this one. I'm not too bothered by it, e
Re:Non-official English (Score:2)
These things can change over time. After all, in German there is in most cases no distinction between adverbs and adjectives, no "ly" suffix (adjectives get suffixes to agree with the gender and case of the nouns that they modify, but in some forms there is no suffix). It is possible that "ly" could disappear over time.
Language Lives! (Score:2)
silly move. The example reminds me of "To boldly go", which was not proper, but its elegance is hard to argue against.
Re:Language Lives! (Score:1)
Apparently written English Grammar varies so much from how it is often spoken as the rules were written down by a Scholar in latin who firmly believed that English should conform to the same rules - even though it doesn't
A careful poke at this 17th century book ( thereabouts - which sets the standard for modern grammar ) means that even Shakespeare wrote bad grammar, and he isn't the only one.
So in fact correct grammar isn't so correct at all an
Re:Language Lives! (Score:1)
My understanding is tht the banning of split infinitives was never a hard and fast rule, even among good writers; Orwell certainly dissented. Infinitives in Latin (and French, German, Italian and Spanish[1]) can't be split anyway, as they are one word.
There's a middle ground. How many moles of NaCl
'Language' == spoken || written? (Score:3, Insightful)
The article addresses this in a weird way, where it first draws attention to the distinction, but once it reaches its crux, where google is used as a tool, the distinction is ignored entirely; instead it opts to focus on stranger things.
Re:'Language' == spoken || written? (Score:1)
Re:'Language' == spoken || written? (Score:1)
Re:'Language' == spoken || written? (Score:2)
"3", of course, is "e". Since "pwned" came from "owned", the most sensible pronunciation is to rhyme the two: "powned." I supposed "pooned" is acceptable, since the only two English-accepted words that use "w" as a vowel, "cwm [reference.com]" and "crwth [reference.com]", are pronounced with an "oo" vowel.
But "pwned" is obviously not a Welsh word. So "powned" may be preferable.
Re:'Language' == spoken || written? (Score:2)
"Powned".
OMG! (Score:2)
Popular usage != wanted usage (Score:3, Informative)
I was pretty taken aback when a council of linguist in Poland suddenly declared some widely-chastised and not even very popular errors to be valid usage. I've been brought up in the circles of people who not only put a lot of stress to the language you use, but also cruelly point out every incorrect word or phrase you use -- and this made me quite intolerant to bad speech.
Being but a dirty foreigner, I know that my English can sound bad in the ears of native English speakers -- that's why I sometimes ask people to correct me if they spot errors.
In other words: some people find careless speech repulsive. Thus, we should do whatever we can to promote correct usage as opposed to legalising incorrect uses.
using google as a spell checker (Score:1)
Then again, my idea of fun is to use google count for finding the words that get misspelt(google ratio with misspelled 5%) the most often.
I thought compatable was common, but i only get a 1% ratio there. Maybe there should be a category 'non native'.
Is conneXion considered an error? I like it much better than connection.
Just now i find out that there are lists , eg at most commonly misspelled w [world-english.org]
Re:using google as a spell checker (Score:1)
It's correct, but British. Just like colour/color, or theatre/theater. Or foetus/fetus, though that doesn't seem to come up so often.
connexion
Pronunciation: k&-'nek-sh&n
chiefly British variant of CONNECTION
Did it never occur to you to check an actual online dictionary [dictionary.com]? I use google to see if my usage of a word or phrase is acceptable (or at least common), but a dictionary is probably a better bet for spelling.
Three types of language (Score:4, Interesting)
I think that for most of the 20th century, English, and most languages in the industrialized world, was largely static, dominated by the written word which was dominated by proper grammar. Since WWII, popular culture and faster communications have increasingly exposed us to local vernaculars, mostly through radio and television. The written word lagged behind in its cultural evolution.
Thanks to the internet (initially email, BBS's and IRC, but more widely known on the Web), we now have a hybrid of the spoken and written word: the "typed word". This form of language evolves at the same rate as the spoken word, and injects its own vernacular as a side effect of the medium: acromyn and abbreviation "words" (rofl, how r u), along with common misspellings (pwned), and mixing letters with numbers or punctuation (133t, n00b). All of these serve at least one purpose, whether as a form of super shorthand, insult, the appearance of being "cool", or are merely the result of laziness on the part of the author. Most typed-word terms don't transfer well when spoken.
One of my hobbies is studying (European) languages and how they are related. Sometimes I worry about the damage the typed word is causing to the spoken and written word (and any proper linguist should at least be interested in the phenomenon). Luckily, most typed word expressions aren't pronounceable, and the ones that are sound absurd, because they are removed from their original context when spoken, and everyone recognizes gibberish when they hear it. How the typed word affects the written word remains to be seen. Yes both are typed now, but only the written word has a chance of going through an editorial process. I think it will take a very long time for the formal lexicon and rules of grammar to embrace, however reluctantly if ever, the typed vernacular.
Re:Three types of language (Score:2)
You do realise that most of the 20th century happened after the second world war, don't you? A condition that became false
Re:Three types of language (Score:2)
Re:Three types of language (Score:2)
Google as a grammar checker (Score:2, Interesting)
I've had the chance to use Google as a grammar or style checker in my day job as a glorified copy editor. I type two nearly identical expressions X and Y in the search box. If expression X gets 10,100 hits and expression Y only 500 hits, I use expression X.
For example, as a non-native speaker, I found myself waffling between the expression (A) "run for mayor of" and the expression (B) "run as mayor of." Letting Google arbitrate, I found 14,900 hits for (A) and only 200 hundred hits for (B). I chose (A).
I
Re:Google as a grammar checker (Score:2)
Tongue Gymnastics (Score:2)
I love a bit of cunning linguistics.
Reminds me of "Meme Tree"... (Score:4, Informative)
Why a tree? Language and geneology seem to have a common thread. Meaning is like genetics. Language is expressive. Information is a kind of tree whose branches grow as reality elaborates and past events accumulate. New terms need to be invented for the dynamics we perceive in reality, just as new names are given to individuals as they emerge into the world. Patterns, continuity, periodicity. Such things lie at the heart of material existence and provide the hooks for consciousness itself. Information theory is the next great frontier, along with particle physics. Already they have converged and diverged and converged again. And playing with artificial trees turns out to be a lot of fun.
As for the "Meme Tree" program
The theory is that the internal consistency of these various lexical maps should roughly reflect many aspects of associative meaning. You could think of the statistical map as a Godelian bubble whose "truth" - if you will - is imposed by the laws governing the statistical associations. We don't derive the laws of language and meaning from these exercises, but we create an internally-complete map that reflects something about the nature of meaning.
There is a practical aim as well. If you can derive the strength of equivalence and the various levels and colors of associative meaning you could in theory build a "Truth Machine" capable of answering any question with a high degree of accuracy. The result of any question could be computed as any other information retrieval problem would be.
I never got around to having my little Meme Tree programs scrape the internet for random sentences. However, this should be a very simple thing to do. Google has had programming contests in the past - programs that use the Google database in interesting ways. Statistical analysis of language is basically what they do. Research projects on their data could provide stunning insights into the nature of information itself, its relation to language and to reality, and likely into our very nature as linguistic beings.
BBC voices (Score:2, Informative)
Another use of Google in Linguistics (Score:1, Informative)
"Automatic Meaning Discovery Using Google":http://arxiv.org/abs/cs.CL/0412098/ [arxiv.org]
Comments welcome, -Rudi.
Writing in Japanese (Score:4, Insightful)
Re:Writing in Japanese (Score:2)
A high number of google hits could mean the translation is correct, but it could also mean there are a lot of idiots using the internet.
Tooting own horn ... (Score:2)
Linguistics 101 (Score:2, Insightful)
Measuring how the internet changes world languages is only a small part o
Using Google as a tagged linguistical data store (Score:2)
operating system
linux kernel
free software
And citations linked to those pairs such as:
Linus torvalds as the moving force behind the operating system t
Ethnologue (Score:2)
LanguageLog is not limited to English (Score:2)
This is misleading in suggesting that LanguageLog [upenn.edu] is limited to English. Actually, it deals with all sorts of linguistic topics and languages.
Re:Hey (Score:2)
Re:Hey (Score:2)
So, yes, it does seem a fracking uselessly mistyped/misknown/miswrote/misthought way to express oneself. But that is changed now, because some people with too much time on their hands think it is a new form of expression and this is the way the English language is changing. So now we are supposed to treat these insentient ideas as the new ways ? Bahh, get lost.