Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Science

Controlling Robots with the Mind 95

Posted by CowboyNeal from the clango-and-company dept.
loucura! writes "Scientific American has a fairly technical article on the real-time control of robotic limbs using recorded neuron patterns. The researcher's macaque has simultaneously controlled two robotic arms in addition to its own arm motion. The amazing thing? One of the arms was 600 miles away. So, they transmitted and translated the "commands" into motion in less than 300 milliseconds!" It's still a long ways off from helping the disabled or making a Dr. Octopus suit, but the potential uses are pretty cool.
This discussion has been archived. No new comments can be posted.

Controlling Robots with the Mind More

Controlling Robots with the Mind

Comments Filter:
  • Just like in Forbidden Planet. We can try out our subconcious minds controlling these robots!
  • Can't these people see the potential for robotic genitalia?
    Add some good feedback and you could be boning your g.f while you're 600 miles away! :-)

  • You're kidding me, right?! (Score:5, Insightful)

    by ZigMonty ( 524212 ) <slashdot.zigmonty@postinbox@com> on Saturday September 28, 2002 @10:49AM (#4349982)
    The amazing thing? One of the arms was 600 miles away.

    No, the amazing thing was that they successfully decoded the neural impulses of the monkey's motor cortex and generated commands that drove a robotic arm in sync with the monkey's arm.

    Who gives a shit if they also sent those signals 600 miles away? Let me introduce you to something called the Internet...

    • Re:You're kidding me, right?! (Score:4, Insightful)

      by theRhinoceros ( 201323 ) on Saturday September 28, 2002 @11:25AM (#4350089)
      Later mathematical analyses revealed that the accuracy of the robot movements was roughly proportional to the number of neurons recorded, but this linear relation began to taper off as the number increased. By sampling 100 neurons we could create robot hand trajectories that were about 70 percent similar to those the monkeys produced. Further analysis estimated that to achieve 95 percent accuracy in the prediction of one-dimensional hand movements, as few as 500 to 700 neurons would suffice, depending on which brain regions we sampled. We are now calculating the number of neurons that would be needed for highly accurate three-dimensional movements. We suspect the total will again be in the hundreds, not thousands.

      This part amazes me above all the other facts in the article: 100 neurons was all it took to get ~70% similarity in action. That seems (at least to me) to be an incredibly small number and says a great deal about signal redundancy in the human brain. Cool.

      • Re:You're kidding me, right?! (Score:4, Insightful)

        by ZigMonty ( 524212 ) <slashdot.zigmonty@postinbox@com> on Saturday September 28, 2002 @12:02PM (#4350194)
        Yep.

        Another thing that amazed me was the adaptability that the animals showed. I am really starting to think that we greatly underestimate their intelligence. It reminds me of the treatment of immigrants: they can't speak English, therefore they are stupid. I personally can't detect or understand the scent signals that many animals use. With the huge amount of redundancy apparently in the brain, is comparing brain size all that accurate? Maybe we just have more redundancy than other primates and our technological progress is solely due to the sophisticated languages and writing systems we have developed. I still don't believe that yet, but I think the perceived differences between us and other higher level animals will shrink as we learn more.

        Of course I'm just a layman who happens to have an interest in this topic.

        • Re:You're kidding me, right?! (Score:4, Interesting)

          by Illserve ( 56215 ) on Saturday September 28, 2002 @02:33PM (#4350768)
          Apes are certainly physiologically equipped to build their own society. They have the digits to construct things, they have the basic insights into using tools and they have very rudimentary language capabilities.

          They're just too stupid.

          The ability to adapt in impressive ways is just the way the brain is wired up and says little about their intelligence. Even rats often come up with impressive tricks to shortcut their way to a reward that the experimenters hadn't thought of. It doesn't make the rats sentient or smarter than the scientist, it just means they are very efficient at picking up correlations in the environment in the same way a computer might if it were designed to do so.

          What sets us apart is our language and sentience. It is not a mundane detail, nor is it something that can be taught to apes at a better than 4 year old level.

          Now, you can argue that it's wrong to experiment on them, I'm not defending or proposing animal experimentation, I'm just stating a fact, on the scale of humans, with or without language, apes are just plain stupid.

          Remove humans from earth and fast forward 5 million years and chimps could very likely evolve into a human-like species (again), but they're nowhere close yet.

          And yes, it's legitimate to compare brain volume. Brain volume allows manual coordination, executive decision making, memory, image processing. Discounting brain volume puts you dangerously close to separating mind from brain, and if you want to go down that road, I surrender.
          • Remove humans from earth and fast forward 5 million years and chimps could very likely evolve into a human-like species (again), but they're nowhere close yet.

            Once more, and I hope everyone is paying attention this time - humans did not evolve from apes; apes and humans have common ancestry. "We come from monkeys" is a very inadequate simplification of the theory of evolution.

          • Apes are certainly physiologically equipped to build their own society. They have the digits to construct things, they have the basic insights into using tools and they have very rudimentary language capabilities.

            Many animals will make tools to help them get food. This was once thought to be the domain of humans only. What if apes keep inventing tools, but because an ape can only discover a new tool by inventing it or mimicking another ape (due to the lack of sophisticated language), the knowledge doesn't spread far and eventually dies out. There is no evidence to suggest that other primates in the past didn't invent the spear or the bow and arrow. Are you really so sure that you would have invented the spear independently if you hadn't heard about it?

            Like I said, I don't believe that they are our intellectual equals. I merely said that I think we're underestimating them.

            • Simple tool use is instinctual in many animals, but yeah, you need language to pass down complex tool making memes. It's hard to imagine how constructing a bow and arrow could end up encoded in genes to be hardcoded in the developed brain and expressed as instinct later on.

              Instinct and imitation only go so far.

              --


          • And yes, it's legitimate to compare brain volume. Brain volume allows manual coordination, executive decision making, memory, image processing. Discounting brain volume puts you dangerously close to separating mind from brain, and if you want to go down that road, I surrender.


            Looking back ~40k years in Europe we find the Neanderthals, who'd been living here for at least 160k years. Around this time Cro-Magnon (which is us) enters Europe. 10k years later the Neanderthals are extinct.

            Neanderthals was shorter than Cro-Magnon (which is us) but possessed larger hands and did in fact also have larger brains. Evidence show that they were routed out and killed by the Cro-Magnon.

            So how did we manage that? We were smarter, even though our brains were smaller. If you look at the tools the Neanderthals made, they are very simple. Typically it's just a stone with a sharp edge and very general purpose. If you look at the Cro-Magnon tools, you see saws and hammers and drills etc.

            Point being, while brain size might certainly matter, it's not all that does.
      • It's that 30% that really makes a difference. A 30% jitter in hand movement turns a brain surgeon into a useless wreck.

        And that 30% is extremely expensive. Don't think that they could capture it just by samping an extra 30 neurons. The #of neurons required to increase the fidelity of a signal increases in a highly nonlinear fashion.

  • I don't understand the fascination with the 600 mile separation. The people who need to control things directly with their neurons are going to be much more interested in manipulating their immediate environment. Anyway, these days aren't the next room and 600 miles pretty much equivalent?

    Two things in this study did strike me as amazing though. One is that the connection has lasted a year. I remember when they first started this the neural connection didn't last long. The other is the fact that the monkey took only a few days to figure out that she didn't have to use her hand and just had to think about moving the lever.

    • Re:600 miles, who cares? (Score:1, Interesting)

      by Anonymous Coward
      The people who don't NEED to control things directly with their neurons but who would WANT to be, say, 600 miles away from where they currently are without actually having to take 8 hours to go through airport security, would find it really neat to be able to plug themselves into a wall and be able to have a physical presence somewhere else.

      (Especially if they ever get those other appendages to work, too...)

    • I don't understand the fascination with the 600 mile separation. The people who need to control things directly with their neurons are going to be much more interested in manipulating their immediate environment.

      Vacationing parents might care.

      "Dear, my mind nanny is showing that our little Johnny is thinking about throwing a party now that we're away"

      "We'll see about that!" (holds fingers to temples). 600 miles away, thwap!. "That'll show him!"

  • Finally, I'll be able to type and browse with BOTH hands!

  • 600 miles (Score:3, Funny)

    by avandesande ( 143899 ) on Saturday September 28, 2002 @10:54AM (#4349995) Journal
    Wow, 600 hundred miles, that is 100 times more amazing than 6 miles away!

  • ...WITH YOUR MIND?

    [obscure Mr. Show [bobanddavid.com] reference].

  • If Battlemechs(TM) will be much farther off...

    • Daimos, or Gundam. ;)
      • BattleTech! Yes, with the advent of the neurohelm we will find 90 meters IS a sufficient distance for gattling cannon fire, and discover ways to increase bore size and yet somehow DECREASE range in artillary.

        As long as my Locust looks like a Crusher Joe and my Marauder looks like a Glaug, I'll be happy.

  • The loop isn't closed yet... (Score:5, Insightful)

    by wowbagger ( 69688 ) on Saturday September 28, 2002 @11:04AM (#4350023) Homepage Journal
    At the current time, all they are doing with the robot arms is ape-ing (pun intended) the motion of the monkey's arm - the monkey is NOT using the robot arms to accomplish tasks. Rather, as the monkey uses it own arm to accomplish tasks, the robot arms are making the same motions. The monkey is no more "controlling" two arms in addition to her own than I would be controlling two computers just because I had VNC displaying the same thing on both computers.

    In other experiments the researchers HAVE closed the loop, by using the brain activity to control a cursor on a screen the monkey can see. Thus, the control loop is closed: Screen feeds brain feeds computer feeds screen.

    But until they can close the loop controlling the arm, by providing some form of tactile feedback, the system isn't very useful. That is their next step - closing the loop by stimulating the monkey's skin in proportion to the force the arm is experiencing.

    Now, if they can combine this research with the work being done on rats to stimulate the sensation nerves, then they may have something that can help paraplegics. And given how plastic the brain is - how good the brain is at adapting to its feedback, then there is a good chance we might be able to make useful direct brain controlled limbs.
    • Now, had you read the article (always a stickly one, that ;) ), you would have read about a rat which could do exactly that (but moving a lever instead of an arm) using just a pattern in it's neurons. They first trained it to manually press the lever, and moved on (read the article for a more detailed description) to the point where the rat just sat there and [b]thought[/b] of the actions necessary to moving the lever, and it was rewarded. No closed loop, true, but a 'recording' of the neural patterns of the movements necessary was made.
      • Actually, I read the article more thoroughly than you read my post. I was responding to the item that the Slashdot editors and the story submitter had seized upon - the control of the robotic arms, which were completely open loop.

        Indeed, the rat experiment was a closed loop - the rat thought, and could see the results of the thought, i.e. the dispenser fired.

        That is my whole point - IF you close the loop, THEN the brain can learn. The closing of the loop can be by haptic feedback, by visual feedback, by auditory feedback, but in any case the results of the action MUST be communicated back to the brain initiating the action. The monkey wasn't getting feedback from the arms, so the loop wasn't closed.
    • I haven't read it.

      Did they do it by measuring muscle response near the muscle, or by measurng neural activity near the spine? Because if it's the latter, you could amputate that monkey's arm, replace it with the waldo, hook it up, and the monkey wouldn't know the difference, grabbing-a-banana wise. He'd just move the arm.

      But then again, maybe they're not recording individual neurons, just some gross wavepatterns, which means it'd be no more "controlling a limb" than letting your dog drive is the Indy 500.

      --Blair

      • Did they do it by measuring muscle response near the muscle, or by measurng neural activity near the spine? Because if it's the latter, you could amputate that monkey's arm, replace it with the waldo, hook it up, and the monkey wouldn't know the difference, grabbing-a-banana wise.

        Not likely. Moving your arm and grasping an object requires not only that you control the muscles but also that you get feedback from your arm letting you know the current positions of your joints and the pressure you're currently applying to the surface of the object. Unless the waldo could supply at least a rudimentary form of this feedback, the monkey would have a very difficult time. He could probably learn to do with only visual feedback, but it wouldn't be easy or instinctive.

      • Re:The loop isn't closed yet... (Score:1, Interesting)

        by Anonymous Coward
        Even better than that -- they implanted very fine wires into the brain to lie alongside a small number of individual neurons. Then they measured the electrical activity of those neurons. We're not talking about reading from the axons in the spinal column, this is straight from the motor cortex.

        The most amazing thing is that a usable signal can be read from so few neurons. Recording from 100 random neurons in the right part of the brain gave a 70%-complete signal, with diminishing returns as more are added. Wiring up brain using no more than a few hundred connections is reasonable enough to be useful!
        • I wonder (and I should google for info), how much progress has been made in connecting up neurons in the visual cortex to look for patterns in the monkey's vision?

          (I want my NVidia OpticNerv Ti4600! ...after the blind people get theirs, of course :)

          --

    • Had you actually read the article, you would have found out that they later hooked up a monkey to control a cursor on the screen via neuronal impulses, and the monkey was able to control it with no physical movement. The article also goes into how this kind of feedback allowed their software to increase its precision.
      In conclusion, RTFA.
      Same goes for you mods that modded up this guy.
    • Tactile feedback is in the works. Check out this bit from page 5 of the article.
      In May we began modifying the BMI to give her tactile feedback for new experiments that are now beginning. The BMI will control a nearby robot arm fitted with a gripper that simulates a grasping hand. Force sensors will indicate when the gripper encounters an object and how much force is required to hold it. Tactile feedback--is the object heavy or light, slick or sticky?--will be delivered to a patch on Aurora's skin embedded with small vibrators. Variations in the vibration frequencies should help Aurora figure out how much force the robot arm should apply to, say, pick up a piece of fruit, and to hold it as the robot brings it back to her.
      Not exactly stimulating the sense nerves that are directly involved, but they do at least provide some sort of substitute.
  • Well, obviously this experiment proves that links between computers and brain tissue are quite possible and usable. In addition, it beats the pants off other experiments...like that one the air force had where human volunteers would try to move a simulator left and right. It took weeks of training for the humans to "train" their brains to give the correct signal most of the time.

    In this case no training seems to be required...you just move your arm and the software is able to translate that. VERY IMPRESSIVE.

    But there is a price to be paid : the monkey is wired with actual hardware in the brain. Face it, the V.R. systems of the future and the cyborgs will have to have actual surgically inserted wiring. To get that cool V.R. rig you'll have to have a major operation installing thousands of tiny wires to the nerves of your body.

  • Oooops... (Score:3, Funny)

    by Nailer ( 69468 ) on Saturday September 28, 2002 @11:12AM (#4350042)
    I actually read that intro as:

    It's still a long ways off from helping the disabled by making a Dr. Octopus suit

    You shoulda seen what I was imagining...
  • Towards the end of the article they talk about being able to replace a monkey's arm with a robotic one (by aanesthetizing the monkey's arm) and having it control the robot arm as if it were its own.
    If this does prove to be successful, It could open the door to 'human upgrades' where you could buy mods for yourself, like extra limbs. i could see a huge market for this in construction. Though i know there are alot of other field that would benefit from this, but i wont list them all.
    • It would be great to be able to attach artificial limbs that worked right off of one's brainwaves (so long as there wasn't interference or somebody yanking your wire by accident).
      Another important thing they'll need to figure out is how to get and interpret feedback. That is, to allow for the sense of feeling from the hand/etc being moved to be translated back to the brain. I think to some extent it's been done already, and one thing nicer than having a robotic hand would be having a robotic hand you can feel with.

      disclaimer: I claim no responsibility those who respond to this post with comments of a sexual or otherwise immature nature - phorm/I.

  • Big deal! (Score:2, Funny)

    by rocjoe71 ( 545053 )
    Controlling robots with the mind? Pfft! I can levitate birds...

  • SciAm PopSci (Score:1, Offtopic)

    by blair1q ( 305137 )
    Last night, I was sitting on the sofa watching the game, and I glanced over and saw this headline on the front of the magazine, and something about wondercars, and another fluffy sensationalist barely scientific come-on.

    And I thought it was the latest issue of Popular Science, which it turned out was was right underneath this issue of Scientific American.

    Seriously. If you covered up the name, and don't have the UPC memorized, you couldn't hope to tell them apart. They used the same layout template for the covers. And maybe for their websites, because both covers are in about the same spot on their home page:

    exhibit A [sciam.com].
    exhibit B [popsci.com].

    Scientific American should never have started taking ads.

    --Blair

    More persistent-looking links to the cover thumbnails:
    sa [sciam.com]
    ps [popsci.com]

  • Old News (Score:3, Insightful)

    by radoni ( 267396 ) on Saturday September 28, 2002 @11:29AM (#4350097)
    Reads like a hype article for the researchers who wrote it. At the end it says:

    "In the two years since that day, our labs and several others have advanced neuroscience, computer science, micr..."

    this was done 2 years ago, guys. it's old news.

    wake up, johnny, i feel a hurricane comin' on!
    • This is in the current Scientific American. It is news. A lot of science doesn't get published until a while after the fact (so it can be peer reviewed, etc)
    • Just because it happens two years ago doesn't not make it news. They need to evaluate, verify, and understand the results, and a number of the experiments have been ongoing in that intervening time. Do you think they're going to get a result from a preliminary experiment and publish it the next day? That's the kind of thing that leads to inaccuracies/lack of checking that has resulted in a few prominent recent examples of scientific misconduct. Sheesh, you yell at them when it's inaccurate, and yell at them when they take the time to verify it.
      You people....
  • It'd really be much cooler if the monkey controlled the robots with it's mind, and was in turn controlled by me, through marionette strings. It's beyond science; it's art.

    Seriously, though, doesn't this raise the very real potential problem of armies of robots, mechanically flinging monkey poo?

    And imagine a beow . . . Oh, never mind. I'll shut up now.
  • Maybe someone else saw the recent issue of Wired magazine (maybe a month or two back) where some mad scientist-type was able to wire a camera up to a blind patients brain, and through the use of a program that would 'learn' what effects certain signals it would put out on the guys visual cortex had, could then begin to replicate a pretty decent field of vision (albeit at very low resolution).

    Well, it seems that scientists are getting somewhat proficient at interpreting brain signals and even providing direct-to-brain feedback. The reality of this is actually amazing. It's the stuff of science fiction, but immersive systems (the Matrix, anyone?) might not be so far fetched anymore. The stuff from 80's cyberpunk fiction where everyone is walking around with jacks in their heads might not be so far off. But then again, flying cars shouldn't be so far off either but you don't see many of those either.

  • /. time to posting TWO YEARS? (Score:4, Funny)

    by skinfitz ( 564041 ) on Saturday September 28, 2002 @11:36AM (#4350126) Journal
    From the article: In the two years since that day,...

    The incident in question happened two years ago - I guess I'm not the only one who submits articles here only to see them "pending" for a long time. But I'm not bitter.

  • Screw this. (Score:2, Funny)

    by tigertigr ( 610853 )
    I'd rather it was "Controlling the Mind with Robots".
  • Must... control... CowboyNeal... post... my... articles...


    Damn. This is harder than it seems.

  • wtf is going on!

    All the posts are unnested and I can mod things, I just want nested comments!
  • To me, the most amazing part of the article is on page 5:

    If visual and tactile sensations mimic the information that usually flows between Aurora's own arm and brain, long-term interaction with a BMI could possibly stimulate her brain to incorporate the robot into its representations of her body--schema known to exist in most brain regions. In other words, Aurora's brain might represent this artificial device as another part of her body. Neuronal tissue in her brain might even dedicate itself to operating the robot arm and interpreting its feedback.

    So, not only could you teach your brain to replace a damaged limb with a prosthetic one, but you could potentially teach your brain to operate a totally *new* limb! How cool would that be??

    And the whole idea of remotely controlling limbs makes me think that the concept of Hector from Saturn 3 [imdb.com] [www.imdb.com] probably seemed far-fetched at the time, but starts to be less and less so...

  • The data isn't a clean as the article implies. (Score:1, Interesting)

    by Anonymous Coward
    The data is not as clean as the article implies (typical of Scientific American and science p.r. pulp magazines) and the technology right now will detect only gross patterns, some identical patterns are generated by other intensions and attensions so we have big problems of false positives and of course some give no distinguishing patterns at all. There is the big question whether brain patterns will correlate 100% or even good enough with mental patterns (specificity issues), the literature isn't very good in this regard. I seriously doubt 100% correlation for logical, empirical and ontological reasons, but good enough maybe all that's required.

  • If you put a disabled person into an high resolution NMR you can let him control robotics devices so that he can walk around.
  • ... to get me to stop wearing my brainwave deflecting hat [zapatopi.net]! Sinister indeed...
  • Why do I picture a roomfull of monkeys remotely wired up to some ICBM's deep underground in norad?

    "Sir monkey 211 has located osama bin laden and is going in for the kill!"

    "Give him a bannana!"

    Imagine a beowulf cluster of these !
  • The amazing thing? One of the arms was 600 miles away. So, they transmitted and translated the "commands" into motion in less than 300 milliseconds!"

    That sounds very similar to moving a character around in an online game. 300 milliseconds is nothing as far as transmission speed goes. A 300 ping in an online game is awful (even with a 56k modem!) Somehow, I doubt that most of that 300 milliseconds was taken up by transmissing the data 600 miles. More likely, most of that time was actually taken up by computations.

    • Delays like that were programmed into the system to compensate for the way neurons worked and communicated, different impulses had to be selectively delayed to perform the desired motion. Didn't really have anything to do with the connection....

  • Typing (Score:3, Interesting)

    by wizardhat ( 172178 ) on Saturday September 28, 2002 @07:17PM (#4351594)
    I wonder if this technology could be adapted, so that as a person thinks of a letter, the sensors could translate the neuron pattern into an ASCII code. Imagine typing without the risk of carpal tunnel syndrome.
  • i can be less worried that my wife would cheat on me because i can still pleasure with my robotic arm miles away on my business trips
  • I just hope that the software which takes in the decoded signal doesn't have a bug. Otherwise U never know, you may intend to shake the hand of the person in front of you, and you may end up punching him on the nose... and to make it worse, when the person tries to hit you back, ur robotic leg refuses to run saying a critical error has ocurred.
    Actually I can think of many other scenarios, but I think you get the picture.

Slashdot Top Deals

Solutions are obvious if one only has the optical power to observe them over the horizon. -- K.A. Arsdall

Close