Let Your Pupils Do the Typing 49
New submitter s.mathot writes: Researchers from France and the Netherlands have developed a way to—literally—write text by thinking of letters. (Academic paper [open access], non-technical blog, YouTube video.) This technique relies on small changes in pupil size that occur when you covertly (from the corner of your eye; without moving your eyes or body) attend to bright or dark objects. By presenting a virtual keyboard on which the 'keys' alternate in brightness, and simultaneously measuring the size of the eye's pupil, the technique automatically determines which letter you want to write; as a result, you can write letters by merely attending to them, without moving any part of your body, including the eyes.
Is this really new? (Score:1, Interesting)
I think that eye movement tracking have been studied for a long time as an input method, mostly for handicapped people that lacks movement in a major part of their body.
What about the system Stephen Hawking uses?
Re: (Score:3, Interesting)
Re: (Score:3)
"and you could even use a bogstandard $15 webcam."
Have any scammers tried this technique at ATMs using a tiny, concealable camera, now that EMV card readers are preventing crooks from skimming the users?
Re: (Score:2)
If you can't move your eyes at all, then I don't see how you can "attend" to anything. Vision isn't possible without continuous saccades, your eyes are always moving.
Re:Is this really new? (Score:4, Interesting)
If you can't move your eyes at all, then I don't see how you can "attend" to anything. Vision isn't possible without continuous saccades, your eyes are always moving.
You can actually focus your attention on objects in your periphery without moving you eyes to it. In vision therapy there is an exercise where you stare fixedly at a dot in the center of the page while finding letters that are scattered throughout the page. You can actually do this with some practice, although it's not particularly natural. Here is the exercise:
https://visionhelp.files.wordp... [wordpress.com]
An interesting aspect of the test is that letters away from the center are printed much larger since the resolution of your vision drops sharply the further away you get from the fixation point.
Re: (Score:2)
Except your eyes are still moving. You can keep it focused on a fairly small area, but if they completely stop moving, you can't see anything. When I had laser surgery for a small retinal tear, they injected a paralytic in that eye. As it took effect, I basically went blind. I could sense light and dark, and I could tell when there was movement, but there was no way to actually see anything. The other eye was fine.
Re: (Score:1)
Re: (Score:1)
Re: (Score:2)
It would seem that the main problem is that it probably works great in a darkroom on a monitor specially designed for it. But not in the real word, not on a monitor with a giant glowing power button, and not if you want to show anything else on the screen at the same time.
Re: (Score:2)
If this is accurate, you don't have to look at the letter. It sounds like, as the summary says, you're literally just thinking about a letter in your peripheral vision without looking directly at it and your pupil responds to its brightness.
Which, even if unreliable, is an interesting discovery.
Re:Is this really new? (Score:5, Insightful)
Hawking still uses a system activated by a muscle in his cheek, one of the few over which he still has some level of control, which is then detected by an IR sensor in his glasses. Earlier versions used a small joystick while he still had some control over a few fingers (or maybe it was just one), but the system has been adapted as he's lost more and more control.
This system might allow him to continue working even if he loses the last vestiges of control over his facial muscles.
Re: (Score:2, Informative)
I understand the confusion, but this technique does not rely on eye tracking in the sense of measuring eye position. It relies on measuring the size of the pupil, while the eyes don't move. So our technique is more comparable to so-called brain-computer interfaces than to conventional eye tracking. (Of course, eye tracking and similar devices (such as the cheek system used by Stephen Hawking) have been around for years, and if (eye) movement is possible, these are more efficient.)
Re: (Score:1)
Re: Is this really new? (Score:1)
As an aside, is anyone else having to relog in a lot on the mobile side of Slashdot?
Re: (Score:2)
It's not a direct read, but the idea is the same. You're thinking of a letter, and your attention goes to the letter on the screen even if your eyes don't move at all (they mention this for use in locked-in syndrome, where there's no voluntary movement at all). The iris responds to a lesser degree than it would if it were to center on the letter, but it still responds to the brightness, an involuntary movement based on a thought.
It's not a direct brain interface, but it makes for an indirect one through.
Re: (Score:2)
Re: (Score:2)
If the keys alternate in brightness, that would only tell you which half of the keys the user is looking at, not which particular key... unless I'm missing some handwaving mojo someplace.
There is no eye tracking. As you have observed, the system can only tell which half of the keys the user is looking at, so the system is only asking a binary question at each stage. The trick is that the letters change places within the groups depending on which groups the user attends to in earlier questions. This is an example of a binary search. Suppose the letter is H, it would work something like this:
Is the letter in the set ABCDEFGHIJKLM or NOPQRSTUVWXYZ? (user fixates on first set)
Is the letter in t
Re: (Score:2)
Re: (Score:2)
Indeed, this would only be helpful to someone who could neither type nor speak. It seems that writing this way would be very time consuming.
Re: (Score:2)
Indeed, this would only be helpful to someone who could neither type nor speak.
Nor move any part of their body including their eyes.
It seems that writing this way would be very time consuming.
If the alternative is "not communicating at all" I don't think it matters.
Re: (Score:2)
Well, of course. It may indeed be of help to Doctor Hawking.
Re: (Score:2)
My late wife had an eyegaze computer made by Tobii [tobiiati.com].
My first thought is, "How does this differ from a traditional eyegaze computer?"
Re: (Score:1)
You made me curious so I went looking. The prices aren't that bad though I don't see any options for open source. I wonder (I looked and didn't find any) if there are any projects doing this with open source? It seems like you could patch something like that together for less money and the software would then be the only real hard part. (Unless I'm missing something.) An above poster, seemingly in a university and working with this sort of thing, indicates that you can do that with a standard web cam.
I've n
Re: (Score:2)
Some systems work on detecting the direction the eyes are looking by measuring the shape of the pupil. Advertising and human-computer interaction people use eye-gaze detection systems that are used to measure how long and where a person is looking
Perfect (Score:1)
Oh, by the way, this type of system has already been deployed for paraplegics for many years. And no, you aren't "literally thinking" about letters, you are "looking" at letters. Learn what "literally" means.
Re: (Score:2)
Actually the new and improved part of this invention is exactly that people do NOT look at the letters.
Re: (Score:1)
Re: (Score:3)
Careful when your boss is near, you might spell "asshole" before you're realizing it. :)
Re: (Score:1)
Except that the point of this tech is that you aren't in fact looking at the letters, you're mentally focusing on them outside of the focus of your vision. This is what they mean by "attending" to them, which frankly is a poor wording that doesn't really convey what they're trying to convey.
Re: (Score:1)
Not quite so! Eye tracking has been used for many years (which is looking at letters); but measuring pupil size when participants are looking at a single location. They attend to the letter they want to type rather than actually looking at it. The pupil responds to the bright/dark background, and this is what's picked up by the algorithm. So the study is rather novel, and is a promising system for those who cannot move their eyes.
You could've know this, if you actually read the announcement and/or the paper
Re: (Score:2)
This is perfect. We now don't even need to move our arms to watch netflix. The peak of civilization has now been achieved.
Netflix. Yeah, right.
Teachers having their charges do the typing? (Score:1)
Did anyone else think this was going to be advice to teachers to let their charges do the typing instead of the teachers ?
Phew! (Score:3)
So this isn't about exploitation of students, then?
Let your pupils do the typing (Score:5, Funny)
Right after they clean the blackboard and erasers.
Old technology. (Score:3)
They started morphing into programmer mills churning out dBase III, COBOL, coders and now they teach everything from Java to Ansys Fluid Mechanics R17.1 (Register for two courses and AutoCAD is free!)
Richard Stallman Does This. (Score:2)
Richard Stallman Does This.
He has his graduate students do his typing for him; too many EMACS related Vulcan Nerve Pinches have left him with severe carpal-tunnel syndrome.
Oh. Not *those* kind of "pupils". Never mind.
Re: (Score:3)
EMACS is indeed the only software that ever caused me physical injury.
After a multi-day editing binge on a CKIE keyboard I went to the campus medical center. Muscle strain on my left pinky from rotate/stretch/curl of my large hands to hit control ...
Now, I would never tamper with University property but a couple of days later days later, there was a little piece of plastic next to my keyboard , and the shift-lock no longer toggled allowing me to remap the control key to where God meant it tone. . .
hawk
Re: (Score:1)
Eye Movment? (Score:2)
It sounds to me like they are implying that when the eye focuses on the brighter lights its pupil contracts, and in such a way the device can know which letter you want. Is that not what is happening? Because that definitely requires eye movement; Are they implying that you will just somehow know the differing brightness of the keys without even looking at them?
Re: Eye Movment? (Score:1)
"Attend to"? (Score:2)
Do you mean "look at"?
Think vs look (Score:2)
They have developed a way to—literally—write text by looking at letters. Great advance!!
Why a binary search? (Score:3)
It appears as if the letters are separated into two groups and that within that group the alternation of brightness is perfectly synchronized to the other letters in the group and opposite the color to the letters in the opposing group. Smaller sets of letters are presented over time until one letter is chosen. So this appears to be a binary search that reads one bit at a time, based strictly on the phase of the brightness signal.
What makes me wonder is why this is so constrained. Could the brightness of each letter be controlled independently to encode the letter directly? Perhaps the user could be presented with a full keyboard, with each "key's" brightness modulated to a different binary code. Presumably then the code of the character that the user was fixating on could be read from the pupil diameter variation directly?