Sign Language Via Cell Phone

QuatumCrypto writes "A project is underway at the University of Washington to enable real-time sign language communication via cell phone. Because of the low-bandwidth wireless cell phone network, a new compression scheme is necessary to capture only the bare essential components of signing to minimize data transfer. Although text messaging is a viable alternative for everyone, signing — like speech — is a much faster and more convenient form of communication."

Video calls

I've already seen sign language being used over video calls. Then again, as one who volunteers with autistic children, I've seen a lot of super-use of technology and hands...

Re:

Just out of curiosity, how does this work? Did they get someone else to hold the phone for them, or can ASL work with only one hand?

Re:

Yes, we hold the phone for him. I've also got a Nokia 6280 with video calls. The video calls at 0.46 NIS/m are cheaper than regular phone calls at 0.67 NIS per minute. The boy is question is not only autistic but also mostly deaf. What's interesting is wa

ob: pr0n gag

I've seen a lot of super-use of technology and hands...

... and we're all very familar with that around here.

They're focusing on video...

But there's another problem with using sign language via cell phone. Look at the screen mock-up on that page - it shows the signers from the waist up. If your phone is far enough away that it can capture your whole body, how are you going to see the screen?

Also, they claim "The current wireless telephone network has inadvertently excluded over one million deaf or hard of hearing Americans", but it's easy to get a cell phone that supports TDD [phonescoop.com], just like a wired phone.

Re:

I was at a presentation of their paper on this in Portland last year: http://portal.acm.org/citation.cfm?id=1169001&jmp= cit&coll=ACM&dl=ACM&CFID=14265233&CFTOKEN=82641255 #CIT [acm.org] From the abstract: "...techniques that exploit the visual

Re:

it's easy to get a cell phone that supports TDD

That's all well and good, but that requires carrying around a TDD keyboard in addition to the cellphone. Those things aren't small. It also requires that the receiving party also have a TDD, unless the ce

How do you hold the cell phone?

The technology for this is very cool and all, but I don't see it as very applicable to use with cell-phones. As far as I know requires the use of both hands, so you would have to put down your phone in a way that you can be seen and you can see the screen and lastly without holding it. This seems like an impossible proposition. But the technology in its own right could be very interesting, at least for desktop video-conferencing units.

Re:

I know requires the use of both hands, so you would have to put down your phone in a way that you can be seen and you can see the screen and lastly without holding it

That was my first thought as well, but it will be nice to have the software ready for when

Language-agnostic?

I hope this compression scheme won't be tied to the semantics of a single sign language like ASL. There are plenty of other sign languages in the world, so hopefully this tech will be "language-agnostic", so to speak.

Re:

They're fairly simple solutions for face and hand tracking and work well under the assumption that everybody has the same color of skin, but we all know this isn't really the case.

Well, there's a pretty simple solution to this problem. We just need to p

Makes sense

I often use sign language to people using cell phones while they're driving.

Hmmm

I still think deaf people should communicate by getting to kick non-deaf people in the crouch. It works similiar to morse code, but with "crunches" and "squishes" instead of "dots" and "lines".

But I'm one for giving handicapped people excuses to hurt the

no subject

"What is the benefit over txt messaging?"

Sign language is much faster obviously, and sign language is based alot on the user's emotions and how they use a certain sign or signs.

But to answer the parent's question, none of the cell phone carriers offer a price break for deaf/hard of hearing users.

BUT the deaf community is fond of using the t-mobile sidekick, all versions, because of the relatively cheap unlimited txt/data plan that comes with it. Sidekicks are almost dominant among deaf people. Some deaf tech sites and companies offer the sidekicks significantly cheaper to deaf users since it is so popular among them.

Re:

A while back, I had a neighbor who was deaf. I helped him and his hearing wife with their computer a few times for free. (And not out of pity, because I didn't know he was deaf when his wife asked for the help.)

Anyhow, he also used a sidekick. Unfort

morse too slow?

Don't some morse characters have 5 or 6 dots or dashes? what's the average length of a character in morse? I would have thought that the current phone keyboards where most characters are between 1 and 4 characters away would be faster. Plus the cognitive l

Too complicated. Doomed to fail.

No, what you need is a pair of wiimote-like gloves that you wear which are connected to a tiny robot monkey on the recipients cellphone that mimics your movements. The recipient, in turn, wears another pair of gloves which are connected to the robot monkey

that doesnt make sense

Although text messaging is a viable alternative for everyone, signing -- like speech -- is a much faster and more convenient form of communication

Speech is flavored in languages, like text. So speech is not convenient at all if this is what they are sayin

I know a few deaf people and

They are flocking to the Sorenson VP-100 system.

I cannot, for the life of me understand this, when there
are so many video based chat sites on the net.

All the deaf people I know have PC's. I met my first
deaf friend on the old BBS's. In the text messages on
F

Re:

I would not want a deaf user signing while driving :P

It's like I learnt in Italy - you NEVER speak to an Italian when he/she is driving, because they are forced to take both hands off the wheel to reply to you!

Re:

I would not want a deaf user signing while driving :P

That might not be so bad... The ability to hear and use one's hands for driving don't seem to do much good for the vast majority of the public. At least deaf people would be are used to it.

Just hack Wiimote!

Another big chance for Nintindo. Can they hack the Wiimote to translate sign language to text?

Re:

Sign language is more in the fingers, the wii-mote would be useless here.

Stop modding him insightful.

Re:

Well, the Wii can handle four wiimotes simultaneously. You need something like ten motion sensors to do sign language. But sign language to text wiimote will be covered by medical insurance as a "medically neccessary prosthetic device". True, gamers are wi

Re:

Now that's an insightful post. I'm sorry I doubted you, but you really spoke like you didn't know what you were on about before. I still don't think that the wiimote is a good basis for a sign-to-texting device (making one at all is still a great idea), bu

Re:

I think you're joking. But just in case, in BSL at least, sign language relies on nuance to form different words - like lip shapes, facial expressions, etc.. For example you can do a sign for "lemonade" and for "to f***" (in the sexual sense, not in the fs

Videophones

At least in Finland, in cooperation with a Finnish hearing-impaired association, there's been some projects with 3G video-phones. Yes - selling a phone to deaf people opens up a nice new market :). Anyway, as far as I know the experiences have been overall positive - and no fancy sign-language-specific codecs or anything, just a normal 64kbps video phone call and a camera phone.

Sign language over mobile works on 3G already

There is nothing new about this story. Sign language over mobile (cell) networks already works with regular 3G (UMTS) phones in Europe. Take a trip to Örebro in Sweden, which has a high concentration of hearing impaired due to a specialist education cent(e)r(e), and you'll see loads of teenagers using their 3G phones to talk using sign language. In the streets, on the bus, in cafes, everywhere. This article http://svt.se/svt/jsp/Crosslink.jsp?d=37482&a=5369 32 [svt.se] (in Swedish) from February 2006 even talks of the local social security services offering customer service to hearing impaired using 3G phones and sign language.

Dumb, dumb.....

There's somthing very stupid with the concept of a sign language chat via cell phone. People are limited by the capabilities of a phone, and at the same time, given the capability, people will use it as they please. That means that no such 'sign language p

Signing is good, but an audio based cell?

Siging is good for the deaf, but the blind have trouble using cell phone because they all react to input graphically instead of audibly.

Sign language text and language

Not all deaf can just use text. Some know only sign and don't know any English. It is impossible for them to use TTY's or other text. And even if it is possible, it can be slow and painful. Learning English text when you can't hear is a very very diffi

Re:

Sorry what?

If they know sign, can't you sign to teach them to read? I can't imagine someone being very functional in society without any written language knowledge.

Tom

Re:

If they know sign, can't you sign to teach them to read? I can't imagine someone being very functional in society without any written language knowledge.

First important thing to know: ASL isn't signed english - it's a language very much of its own. And fo

Re:

I know that ASL is it's own language. I'm just saying if you have a deaf kid, you'd immerse than in English [or whatever the standard is] written text as much as possible as early as possible.

I wouldn't expect them to SPEAK it easily, but reading shouldn'

Re:

I think you're missing the point about ASL being a completely different language. It, unlike most spoken languages, does not have a written component. For someone who grows up in a completely deaf family and culture, it's more than just a language barrier.

Re:

ASL still has a grammar right? Parts of speech (so to speak, er say, er... you know) and all that? Can you not explain in ASL what the letter a looks like? That it's vowel? that it's the first letter of the English alphabet? etc...?

It's like music, you

Re:

Sigh... I'll continue to feed the trolls...

ASL still has a grammar right? Parts of speech (so to speak, er say, er... you know) and all that?

Of course ASL has grammar; do you understand the concept of language?

Can you not explain in ASL what the letter

Re:

I'm not trolling, I just refuse to accept that deaf kids can't learn to read.

There is more than one way to learn a language than just hearing it. I heard pictures and moving pictures work well too... I never said it was easy. Hell, it's not easy to lear

Re:TTY?

The deaf texts a lot as well - one swedish article had a couple of deaf teenagers commenting on 3G phones and they used to send 500-1000 SMS per month. But a quick email exchange isn't the same thing as a actual live conversation (at least not for most of us)... just consider how the time gaps with texting/email makes it harder to judge the other persons mental state.

Another interesting trial project going on now in Sweden is "Translator in a pocket". It allows a deaf person to call a sign language translator who translates using the phone. Very useful for anything where you need a direct conversation with a hearing person and you couldn't plan ahead to get a translator and don't want passing notes (or what they'd use). Btw, 3G phones are very popular here in Sweden with the deaf and especially with the teenagers. I've heard numbers that something like 80% of all deaf teenagers have videophones.

Fun with deaf relay operators :)

From the Phone Losers of America [phonelosers.org]:

EPISODE #3 - Deaf Relay Operators

This episode features my new co-host Mary, a relay operator. It also features songs, skits, messages, commentary, commercials and raps all performed by deaf relay operators. It's approximate

Re:TTY?

What is the benefit of this technology over TTY or using text messaging?

I don't know what you or the GP means (in a mobile phone context) by TTY, but there is one definite benefit: immediateness. That is, lack of latency in message delivery.

When you send text messages back and forth, there's a delay with every delivery. For the equivalent of speech, this would be like calling the moon. Plus, you have to go into the inbox and open new messages all the time -- not very conversation-like or, for that matter, IM-like.

Re:

When you send text messages back and forth, there's a delay with every delivery. For the equivalent of speech, this would be like calling the moon. Plus, you have to go into the inbox and open new messages all the time -- not very conversation-like or, for

Re:TTY?

Well it seems the same effect can be achieved through a video conference...

And that's essentially what this article is about. Rather than using full-bandwidth video communication, they're trying to develop a compression algorithm that is better suited to signing (ie, capturing only the primary hand motions).

Re:

The benifit is that sign language is the first language for a lot of people.
English is their second language.

Re:

Last I checked BBC sells it's feed to other broadcasters. They'd be stupid to just give it away.

That and most of the rest of the planet only gets BBC World [the news] not BBC 1-4. :-(

Tom

Re:

As someone who speaks English and French, can express ideas in a half dozen programming languages, and can read music ... I say so what.

What do you think a musical score is? Bunch of meaningless symbols, lines, dots and squiggles [to the untrained eye].

I

Re:

Not all deaf-from-birth people end up being raised on sign language.

I'm born deaf. I was raised on written and spoken English, known as "oral education".

I do know sign language, but not until I had entered high school. By then, my understanding of the Engl

Re:

Mod parent up, I was gonna write the same thing but searched to see if anyone wrote it first and well, yeah what he said They did it in babel.

Re:Sign language and speech faster than typing?

Either you talk too slow or you've broken some land-speed records for typing on handheld devices. Typical English conversation is roughly 200 words per minute. Most of the population can't type faster than about 60 words per minute on a standard keyboard, let alone a cell phone-sized thumbpad.

Even if you type at double that (120wpm), you're still typing slower than you speak. As for the input device, how would you go about making a pocket-sized keyboard as efficient as a desktop version (which you can put down and use all fingers to type--no such possibility with a cell phone)? Having to have the physical input device AT ALL *is* the problem to be fixed here.

Re:

Umm. I type as fast as I generally speak. I *can* speak faster, but then, I *can* type faster too, if I don't have to stop and think what I'm going to say. I imagine signing would be similar. So I would think text messaging would be just as fast. Unless t