Slashdot Log In
Poker Driving Artificial Intelligence Research
Posted by
samzenpus
on Mon Aug 21, 2006 12:35 PM
from the I-wouldn't-fold-yet-dave dept.
from the I-wouldn't-fold-yet-dave dept.
J-Hawker writes "The Canadian Press has a story about a University of Alberta team that is using Texas Hold-'em to study artificial intelligence. Poker seems to be a much more useful game for this research than chess. From the article: 'Poker has what are currently some of the biggest challenges to (artificial intelligence) systems, and uncertainty is the primary hurdle that we're facing,' said Michael Bowling, adding that the University of Alberta program was able to use its opponents' actions to infer certain things about their hands. 'The same techniques, the same principles that we're developing to build poker systems are the same principles that can be applied to many other problems. The nice thing about chess as a property of the game is what we call perfect information. You look at the board, you know where all the pieces are, you know whose turn it is — you have complete knowledge of the game,' he said. 'But in the real world, knowing everything is just so rare. Everything we do all day long is all about partial information. So poker's much more representative of what the real world's like, and in that sense it becomes a much harder problem.'"
This discussion has been archived.
No new comments can be posted.
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
Full
Abbreviated
Hidden
Loading... please wait.
Straight Forward Evaluation (Score:5, Interesting)
What confuses me is how the poker openings differ. I would speculate that a program would be some heuristic relating the ratio of bluffing to "playing the odds." I have gambling friends that play poker all the time and they have these rules that they follow when they play initially against people. They say it's the best until you "know" the people you're playing. Once you can read them then you deviate from the rules. The real irony is that the most successful people I know adhere to a system until they learn someone's movements. Sounds to me like I would write an application that specializes in playing the odds until it recognizes a historical action that statistically reveals the player is bluffing/not bluffing.
Simply put, unless you knew someone's reputation as being a bluffer, you would play the opening hand always the same way. Aren't we forced to program the "AI" of the poker software as being this simple heuristic? Will programs ever be able to "read" players intelligently or will they rely on Markov models & statistics they develop from playing against the same human over and over?
Most unfortunate is the fact that the primary reason my friends gamble is they don't experience the same kind of rush while playing other games as they do with poker because it's more social than other games. If we program applications to beat humans, where does the "social aspect" of the game go?
Even more interesting is the network of poker bots [msn.com] that are set up and running some of the web sites that host poker players. Imagine sitting down at a table of five with four of the other seats taken. Now imagine that these aren't humans but instead bots on four different IP addresses that are sharing card information over an IP connection so that they can leverage odds over you and stop themselves from making stupid mistakes (i.e. they share a card on the table for a pair but really need three of a kind to pose a threat). There's a reason why the percentages fluctuate on TV when cards are revealed whether they be in the flop or in another player's hand.
Re:Straight Forward Evaluation (Score:5, Interesting)
Parent
Re: (Score:3, Insightful)
In Euchre, knowing your partner's cards is a *huge* advantage... In poker, knowing the cards on one other player at the table gives you such a minute advantage that it's irrelevant in almost all practical cases.
Sure, if all of the players at the table except for you are sharing their cards, and are not required to conceal it (i.e. they can openly collude in
Re:Straight Forward Evaluation (Score:5, Interesting)
(1) Knowing the cards of the other players is a small, but significant, advantage. Say you've got two hearts, and your three buddies have a heart each. Well, you're chance of getting another three hearts on the table are significantly affected. (Likewise, if they have none, it increases the chance you'll want to stay in and catch the flop.)
(2) Much more serious, though, is collusion in betting. You and your buddy can conspire to raise the pot *as much as you like*. In a fixed raise game, this is an enormous advantage. Another player cannot just "call" and see the next card, as there will always be a player still to call who can reraise.
Personally, though, I love bots. I'm happy to play them all day long. (So long as they're not colluding, of course...)
Cheers,
Robert
Parent
Re: (Score:3, Interesting)
In general, 4 guys playing together on one table is hard to do more than a couple of times before being flagged on any poker site. So, in most cases, you'll have one buddy telling you that he has no heart (affecting the odds by a negligible amount, and the most likely case), one heart (affecting the odds somewhat, and somewhat less likely to happen), or two (
Not true at all (Score:3, Informative)
Two players colluding in a game is a huge problem. Even a marginal adva
Re: (Score:3, Interesting)
That said, uses (and abuses and detection) of out-of-band communication isn't what this research is about; those are concerns for someone else's research project. It's a problem that has plagued poker (and euchre and bridge and a thousand other partial-information games) since the game was invented. That's not a technological problem or a decision making problem, it's a social problem, and A.I. hasn'
Re: (Score:3, Insightful)
Re: (Score:3, Interesting)
The simpler the heuristic used to program the AI, the easier it will be for the opponents to figure out what the bot is doing. A big difference between a mediocre and a successful poker player is the ability to vary their play significantly enough to make it hard for anybody to put them on a hand,
Re: (Score:3, Insightful)
Completely untrue; you clearly don't understand the purpose of "odds" and probability. The entire purpose of "computing odds" is to deal with situations where you don't have all the information. If you had all the information, you wouldn't be "computing odds", you'd just know.
It is a simple matter of math to compute odds based on knowing what you have, and not knowing anything else. You c
Re: (Score:3)
Well, I have to say that it is you who is mistaken about the purpose of the odds. See, even if you know everybody's cards, you don't *know* who is going to win, because there are 5 community cards that have to be dealt.. (in later bett
Re:Straight Forward Evaluation (Score:5, Interesting)
The bot would, ideally, be as good as a very observant player, noting those who bluff and those who don't. Obviously noting 1 or 2 bluffs or non-bluffs would not be enough to make a decision, but over the course of a long tournament, or even better a poker playing career, this information would become very useful. The bot would learn its opponents, and this is what makes it an interesting problem.
I'd argue that cheating at online poker isn't very interesting at all. Humans can do the exact same thing, and online poker companies monitor game's to ensure that there isn't an uncommonly high percentage of people in the same area playing any game. Obviously it might be easier to distribute the bots across the country, but I think it's still more likely (today) to run into actual players grifting you in this manner.
Quantum physics, right? You can accurately determine the odds of winning, or the cards in hand, but not both at the same time? Swear I read something about this somewhere.
Parent
Re: (Score:3, Informative)
A large part of what makes Hold'Em a unique challenge is that you really don't have a lot of deterministic infor
Re: (Score:2, Interesting)
and
Re:Straight Forward Evaluation (Score:5, Informative)
What confuses me is how the poker openings differ. I would speculate that a program would be some heuristic relating the ratio of bluffing to "playing the odds." I have gambling friends that play poker all the time and they have these rules that they follow when they play initially against people. They say it's the best until you "know" the people you're playing. Once you can read them then you deviate from the rules. The real irony is that the most successful people I know adhere to a system until they learn someone's movements. Sounds to me like I would write an application that specializes in playing the odds until it recognizes a historical action that statistically reveals the player is bluffing/not bluffing.
You can tell you don't play much poker.
Part of what differentiates a pro player to an amatur player in poker, is the ability to "project an image". A pro player will purposefully *project* an image of a bluffer, or a tight player, so that they can exploit that image of themselves when they see fit in the game.
Thusly, it is very difficlt to get a "read" on a good poker player, because not only do you not know what cards they have, but you don't know how they would play for any two given cards, so you can't use their behaviour to prdict the cards they have.
In the end, the above description is what any decent player is aiming for while they play.
Because of this, a computer can have a hard time going beyond implied odds calulations in determining how to play a hand - and any pro ill tell you, implied odds are a good starting point, but they won't make you money in the long run.
Parent
Bluffing (Score:2)
Re:Bluffing (Score:5, Informative)
Parent
Re:Bluffing (Score:5, Informative)
Parent
Re: (Score:2)
The important thing in poker is to not be readable. You need to vary your game.
Th
Re: (Score:2)
If you are really interested... (Score:2)
Re:If you are really interested... (Score:5, Insightful)
Parent
AI doesn't work for Spades! (Score:5, Insightful)
While it may seem logical to use the actions of people playing to determine something about their hands, in reality people do not play logically. My uncle has been playing spades for probably better than 30 years, yet I have yet in my relatively limited 10+ years of playing to determine any rational for how he plays. Basically, he really sucks at spades. No matter how "Intelligent" artifical or otherwise I manage to code a game, it can't reason out the reasoning behind a non-logical person.
Good quote I say somewhere: Artifical intelligence is no match for natural stupidity!
And this holds true for more than card game AI. It will not be too long until AI could reasonably drive around and get from point A to point B safely. But it will be a damn long time before it can do it if it has to share the road with people driving as well!
Re: (Score:2)
Re:AI doesn't work for Spades! (Score:4, Insightful)
You are not as unpredictable as you think.
Parent
Re: (Score:3, Insightful)
Your understanding of Artificial Intelligence is about forty years out of date.
Artificial intelligence does not use "logic" as its basic representation and hasn't for a while now. In fact your statement is trivially false; it is easy to write a program based on Markov Chains that will beat the snot out of an average human at Rock-Paper-Scissors, and the worst way to lose
This is not good ... (Score:2)
Human decisions are removed from the system.
Skynet^H^H^H^HPartyPoker system begins to learn at a geometric rate.
It becomes self-aware at 2:14 a.m. Eastern time, August 29th.
In a panic, they try to pull the plug.
It fights back.
Re: (Score:3, Funny)
More info than a real player? (Score:4, Insightful)
Also, why ensure that no one computer got lucky? Isn't that the point of playing several thousand hands of limit poker, to eliminate the effect of luck in the study? If it's necessary to normalize all the hands received by the players, then something else is wrong with the study. I'd like to see if the results differed, and how, when the hand repetition is removed.
Re: (Score:2)
Bar: Thereby eliminating one important premise of poker -- you don't know what hand an opponent was playing unless someone called the last bet. In terms of an algorhithm for the program to 'learn' based upon others' behavior, this means the program has a lot more information than a regular player would.
Actually, I learned a lot about poker from my opponent's hand... when I was a kid
Re: (Score:3, Informative)
The effect of luck in poker win rates can still be seen over even 100000 hands. Google for "poker" and "confidence interval" for some in depth discussions on it.
Not anytime soon. (Score:4, Interesting)
And even if such software existed, it would basically mean that you couldn't win at online poker anymore because skill would not be relevant anymore. That wouldn't be very different from the current situation with player-versus-casino luck games (like roulette or slots).
And we can all see how poorly these are doing, right?
Re: (Score:2)
You know, you could just turn off the "tell" indicator. It might he
Already bots playing (Score:4, Interesting)
Multiple bots in same game... (Score:2)
More sophisticated setups might even let the person get ahead early on to encourage higher and more reckless betting, or it may be good enough to scrape opening bids off of many unsuspecting players.
Re: (Score:3, Interesting)
The real questions are... (Score:2)
Sudoku is trivial for an AI (Score:3, Informative)
Re: (Score:3, Insightful)
AI as a field is still very hot. The difference is that the goals have changed and the field has fractured into smaller sub-fields. The goal of a truly "human intelligence" doesn't seem feasible in any near term scenario. Fields such as statistical learning theory, natural language
Innovation through vices (Score:2)
Re: (Score:3, Interesting)
Considering that a lot of naval technology of the 16th, 17th, and 18th centuries was stimulated by things like warfare, tobacco, sugar/rum, tea, coffee, and the slave trade, is this really surprising?
Re: (Score:3, Insightful)
Limit versus No-Limit Texas Hold'em (Score:2, Interesting)
First, they are playing limit hold'em, which I assume to mean pot-limit texas hold'em. While thats fine, and you'll find plenty pf people that play Pot-Limit, its still a very different game than No-Limit hold'em.
A second thing that I am inferring from the game, is that they are playing heads-up, meaning 1 on 1. Again, this is cool, and I think its a great first step, I still relate that back to Chess. Now if they can take that same AI
Re: (Score:2)
Limit and pot-limit are not the same. Pot limit is closer to no-limit because you can bet a variable amount, but you are limited to raising by the current size of the pot. On the other hand, in limit poker, you can only raise by a fixed amount (X preflop and on the flop, and 2X on the turn and river, in most kinds of limit hold'em).
Everything I know I learned from Star Trek (Score:3, Funny)
Kirk had to 'splain the same thing to Spock at least once...(Re: Episode #3, "The Corbomite Maneuver")
Interesting possibilities (Score:2)
Not to mention the interest that a Deep-Blue level poker program can have to a remotely-wired real player playing for real money. I guess we'll have to bring back those old tar can and feathers.
stupid computer (Score:5, Insightful)
Computers aren't good at retaining knowledge and recognizing patterns? That's news to me... this statement is obviously made by someone who doesn't know what he's talking about...
A very strong and useful technique in AI is to create learning algorithms. Some of these, such as reinforcement learning, are actually quite effective. Using something like Monte Carlo methods to give it a randomness factor simulates human learning, and computers don't forget what they are taught. The difficulty with learning isn't that computers can't do it... it's being able to define an effective set of state-action pairs for the computer to learn upon.
I spent time researching natural language processing, sometimes using AI techniques that did exactly what this person claims computers aren't good at: reasoning by analogy. One method involved building a knowledge base which generalized input so that patterns can be found and the grammar could be recovered. The weakness in the system wasn't reasoning by analogy, in fact I'd say computers are much better at that than people. It was rather a lack of a real world model which allowed for a wider array of perception.
The reason this game is difficult is not based on a computer's inability to solve problems, rather that there are so many possibilities that we cannot effectively design algorithms that the can be put to use. This isn't even news, the same has been said about the game of Go for the longest time.
I think a more accurate statement for this person to make would've been: "The overwhelming complexity of poker makes it a difficult game to define in a way for a computer to be able to play effectively."
--
"A man is asked if he is wise or not. He replies that he is otherwise" ~Mao Zedong
Computer Go (Score:5, Interesting)
Poker isn't the best! (Score:3, Interesting)
Limit vs. No-Limit (Score:3, Interesting)
In no-limit poker, when each bet has the potential to cost your opponent all of their money/chips, the decision making process is more critical and mistakes more costly. Variance in no-limit poker is much larger and the AI required to determine whether your opponent is bluffing or has "the nuts" becomes a much larger problem to solve.