HoloLens Can Act As Eyes For Blind Users and Guide Them With Audio Prompts, New Research Shows (techcrunch.com) 25
New research shows that Microsoft's HoloLens augmented-reality headset works well as a visual prosthesis for the vision impaired, not relaying actual visual data but guiding them in real time with audio cues and instructions. TechCrunch reports: The researchers, from Caltech and University of Southern California, first argue that restoring vision is at present simply not a realistic goal, but that replacing the perception portion of vision isn't necessary to replicate the practical portion. After all, if you can tell where a chair is, you don't need to see it to avoid it, right? Crunching visual data and producing a map of high-level features like walls, obstacles and doors is one of the core capabilities of the HoloLens, so the team decided to let it do its thing and recreate the environment for the user from these extracted features. They designed the system around sound, naturally. Every major object and feature can tell the user where it is, either via voice or sound. Walls, for instance, hiss (presumably a white noise, not a snake hiss) as the user approaches them. And the user can scan the scene, with objects announcing themselves from left to right from the direction in which they are located. A single object can be selected and will repeat its callout to help the user find it. That's all well for stationary tasks like finding your cane or the couch in a friend's house. But the system also works in motion.
The team recruited seven blind people to test it out. They were given a brief intro but no training, and then asked to accomplish a variety of tasks. The users could reliably locate and point to objects from audio cues, and were able to find a chair in a room in a fraction of the time they normally would, and avoid obstacles easily as well. Then they were tasked with navigating from the entrance of a building to a room on the second floor by following the headset's instructions. A "virtual guide" repeatedly says "follow me" from an apparent distance of a few feet ahead, while also warning when stairs were coming, where handrails were and when the user had gone off course. All seven users got to their destinations on the first try, and much more quickly than if they had had to proceed normally with no navigation.
The team recruited seven blind people to test it out. They were given a brief intro but no training, and then asked to accomplish a variety of tasks. The users could reliably locate and point to objects from audio cues, and were able to find a chair in a room in a fraction of the time they normally would, and avoid obstacles easily as well. Then they were tasked with navigating from the entrance of a building to a room on the second floor by following the headset's instructions. A "virtual guide" repeatedly says "follow me" from an apparent distance of a few feet ahead, while also warning when stairs were coming, where handrails were and when the user had gone off course. All seven users got to their destinations on the first try, and much more quickly than if they had had to proceed normally with no navigation.
"New research*" (Score:2)
Re: (Score:3)
Indeed... https://en.wikipedia.org/wiki/... [wikipedia.org]
Really I'm more inclined towards the systems that apply tactile stimulation to a patch on the user's skin, instead of crowding out their already overtaxed auditory spectrum. https://jneuroengrehab.biomedc... [biomedcentral.com]
Re: (Score:1)
Re: (Score:1)
You say that as if there was something wrong with it.
Newsflash, every company sponsors research into the possible uses of its product, and they have every right to puff up the results. So long as they don't tell lies in the marketing, it's all good.
Re: (Score:2)
Yep. Sounds like a wonderful way to couple the sale of hundreds of dollars in expensive display technology that's inherently useless to the blind user to the usage of the head-mounted camera that's the only part actually being used.
Re: (Score:2)
I'm no MS fanboi and haven't used any of their products for the past ~10 years (apart from the outlook calendar forced on me by work).
But this is really good, it's fantastic that they're doing this, and it actually sounds like a good idea. Of course they have good incentives apart from the goodness of their heart (good PR, and possible future profits - there's an estimate of 36M (completely) blind people globally), but as far as I care if an "evil" corporation does "good" stuff because of incentives in the
Updates (Score:1)
Re:Updates (Score:5, Insightful)
Sounds like a great idea. Right up until a forced update gets someone killed because their hololens stopped talking to them at a bad time.
You jest (I assume), but assistive equipment does have occasional failures, like any devices.
You don't stop using hearing aids just because the batteries die and needs replacement.
Nor does this kind of tech get worn all the time. At a minimum, it will get taken off to go to bed, to shower, etc. Much like glasses or anything else.
Re: (Score:2)
I would point out that unlike windows updates, hearing aid batteries can be replaced on a schedule the wearer controls. If you don't want to risk it going dead at a bad time, just replace the battery a few days early.
Likewise the times when it's taken off would make good times for an update, but M$ doesn't care about scheduling. Go into the settings and tell it to install updates at 2AM? It's a coin flip whether it will respect that or not.
Yes, assistive devices ha
Re: (Score:2)
Right up until a forced update gets someone killed because their hololens stopped talking to them at a bad time.
Even worse . . . ransomware . . .
"Please deposit 10 Bitcoins to receive instructions for your next 10 steps . . . "
Excellent (Score:2)
Nice! (Score:2)
So each time when a person wearing this enters a room we will hear stuff like:
"The mantlepiece is full of a clutter of photos, candles, a clock and a vase. The fireplace below is a reproduction Victorian one with a marble hearth, although if we wanted to light the fire we would have to clear away a couple of storage baskets that are full of wii controllers, wii games and DVDs.
Next to the fireplace is the TV which is sitting on a storage unit with the DVD player, Sky Box, Wii and an old VHS Recorder that doe
Gazebo (Score:2)
It is pitch black. You are likely to be eaten by a grue.