Puter beats top human poker players

Submitted by catchy on Sun, 07/13/2008 - 9:38pm. Tags:

Science and Tech

I was surprised to read that an AI system outperformed top poker players.

Whenever I've played the game it seems to involve more than mere probability calculations, but also reading the psychological patterns in other players.

Nonetheless,
Polaris beat several top human players at the Man v Machine poker competition in Portland. Most impressive to me are reports of an element of learning, where Polaris identifies which common poker stratagy a human is using and switches its own strategy to counter.

So the competition was a heads-up, hold-em version of online poker. I don't know if that's one of the easier types of poker to play and would be interested to hear any thoughts especially from folk who have played online poker.

One question is whether this could signal the end of online poker. Once the software gets leaked, how would anyone know whether they're playing an AI program or a human?

http://www.eetimes.com/showArticle.jhtml;jsessionid=BDYCRFWYJLEDOQSNDLRS...
--

catchy's blog
Login or register to post comments

I wonder... (#103159)

by heet on Tue, 07/15/2008 - 12:43pm

if there are not already ppl out there who have written highly sophisticated poker playing programs but don't advertise them. In other words, they are more interested in setting them up to make money off the rubes online.

Over here on E Street, we're proud to support Obama for President. - Bruce Springsteen

(

)

Half right (#103204)

by Zorrito on Tue, 07/15/2008 - 7:40pm

There are plenty of programs that win online poker by cheating. They do it by controlling multiple players in a single game and colluding. It started at the same time on-line gambling started. Uni students sat next to each other at their computers showing each other their hands. Automating the process didn't take very long.

AFAIK the poker part of the programs isn't very sophisticated, the hard part is avoiding detection by the game host.

(

| parent )

ahhhhhh... (#103232)

by heet on Tue, 07/15/2008 - 10:16pm

I didn't think of that. Guess I'd qualify as one of the online rubes if I played for real money!

Over here on E Street, we're proud to support Obama for President. - Bruce Springsteen

(

| parent )

So I thought about this, and I have an opinion: (#103041)

by Punditus Maximus on Mon, 07/14/2008 - 6:07pm

this was inevitable.

If you watch professional poker players, they spend a great deal of time and energy hiding their own facial expressions and bodies and much, much time engaging in activities to draw out tells from other players. Clearly, the largest returns are from failing to tell, rather than from reading others' tells. In that context, a machine which is incapable of telling but not particularly good at reading the tells of others -- and cannot make math errors -- is still superior.

It's impossible to debate if people simply hold beliefs that have no grounding in reality.

(

)

Not really (#103111)

by stillnotking on Tue, 07/15/2008 - 7:57am

they spend a great deal of time and energy hiding their own facial expressions and bodies and much, much time engaging in activities to draw out tells from other players.

It's pretty easy to conceal your facial expressions and body language, and it's pretty hard to draw out other people's, if they've been playing long at all. "Angle shooting" is popular in $3-$6 games, but you won't see it much at the high-stakes tables. (The angle shooter will give away more information than he gleans from a capable opponent. If I see someone pushing chips around to watch my reaction, I know he has a marginal hand.)

At least 95% of the intellectual energy expended in poker involves attempting to guess the opponents' strategy on the basis of their betting patterns, not their "tells". This is particularly true now that online poker is more popular than brick-and-mortar.

The other day I heard that ignorance and apathy are sweeping the country. I didn't know that, but I don't really care.

(

| parent )

My thoughts (#102888)

by stillnotking on Mon, 07/14/2008 - 6:32am

as requested... :) (I'm still lurking a bit, just not posting as much)

The link is broken, so I can't read the story. How long was the Man v Machine competition? Was it a tournament or a ring game? Did the human players know they were playing against a machine? All these factors make a difference. If someone has written software that can beat Doyle Brunson over 10k hands in a cash game, I'm impressed. If a computer program won a tournament, I'm not. Poker in the short run is essentially all luck, assuming each player has basic skill at the game.

As far as whether it will kill online poker -- well, the majority of the people who play online are already deluding themselves that they can beat skilled human players. Why shouldn't they also delude themselves that they can beat well-programmed software?

The other day I heard that ignorance and apathy are sweeping the country. I didn't know that, but I don't really care.

(

)

The basics of this model have been around a while. (#102898)

by BlaiseP on Mon, 07/14/2008 - 8:29am

Poker is a problem of incomplete knowledge. It's modeled in a Poisson distribution. The mathematics of poker is well understood: anyone with high-school algebra and a little calculus, enough to find the zero of a first derivative can grasp the math involved.

Polaris currently works against a single player. In that respect, it's really playing a more elaborate version of other games like Rock Paper Scissors. There's no winning against a perfectly random opponent in Rock Paper Scissors. I believe Polaris only plays Limit Hold 'Em, so that's a far simpler game: the betting states are smaller, so's the tree. Makes the problem computable.

The key to Polaris' winning anything is spotting an opponent's style of play and adapting to that style, what a games theorist would call Exploitative Play. The computer is completely immune to bluffing: it will punish the bluffer. It would be vulnerable to collusion among multiple players, but it might be able to spot collusion given time.

Games theory has a lot to teach us: one aspect of the problem resolves to the fact that winning and losing are separate strategy paths: it's possible to master the art of not-losing without learning to win.

Richard Nixon was a legendary poker player in the service: he made quite a pile from his fellow sailors by tossing mediocre hands and gutting it out with winners. Bill Gates was a poker player as well.

The Polaris system will be a huge step forward in the art of not-losing, but it won't teach the rube to win, either. The real benefits of a Polaris type system will emerge in bid-and-ask, contracts and war gaming.

(

| parent )

The article mentioned that players were allowed to compare notes (#102902)

by catchy on Mon, 07/14/2008 - 9:11am

so Polaris seems immune to certain forms of collusion.

You're probably right that: the betting states are smaller, so's the tree. Makes the problem computable.

But note that chess is N-P complete yet programs still use heuristics + approximating algorithms quite effectively.

I have no idea whether the same strategies utilized by Polaris would be useful when larger decision trees are present, but combinatorial explosion isn't an in principle barrier for beating humans.

(

| parent )

Chess is theoretically N-P complete (#102908)

by BlaiseP on Mon, 07/14/2008 - 9:34am

but that's not the way to build a winning strategy. The first eight moves of chess are completely understood, anyone with a copy of Modern Chess Openings has them all down. Bizarre openings are still possible, but they convey no practical advantage.

Polaris and programs of its sort are useful in the way a chess program is useful: it quickly prunes off losing strategies. That alone will improve poker in a Darwinian sense, but it won't make big winners out of the mediocre player, any more than a run-of-the-mill chess program will make a winner.

(

| parent )

Immune to bluffing? (#102901)

by stillnotking on Mon, 07/14/2008 - 9:08am

That makes no sense, Blaise. It's impossible to be immune to bluffing. Game-theoretically, there is an optimal bluff percentage in a particular situation, but that percentage depends on the opponent's strategy. This is why poker cannot be "solved": if I switch my strategy to X, you switch yours to Y, so I have to switch mine to Z, etc.

The reason tight play is generally regarded as "correct" is that the average poker player is much too hard to bluff, i.e. he overvalues mediocre hands and overvalues calling down as a strategy. Against a conservative player, very loose play is correct, because you expect to win more hands with bluffs.

As far as whether limit poker is a simpler game, opinions vary. There are fewer options available to the player, but many of the top players believe limit poker requires more skill at reading the opponent. (Not in the sense of reading tells, but in the sense of correctly divining his strategy at a given time.) The reason is the lack of access to the all-in move, which is actually a tremendous skill leveler.

The other day I heard that ignorance and apathy are sweeping the country. I didn't know that, but I don't really care.

(

| parent )

The computer doesn't care about your bluff. (#102907)

by BlaiseP on Mon, 07/14/2008 - 9:27am

Polaris sees random bids derived from its own bidding tree, so it doesn't perceive strength from a human bidder. It sees rashness.

Look, mostly Polaris plays a don't-lose strategy, something most humans simply don't accept as a different proposition than winning.

(

| parent )

Polaris sees incomplete information (#103109)

by stillnotking on Tue, 07/15/2008 - 7:49am

just like every other player. Obviously it doesn't always see bets as "rashness", or it would be a calling station and ludicrously easy to beat -- the archetypal crusty old retiree. "OK, sonny, let's see what you got." If that were a winning strategy, the WSOP would play very differently!

Polaris assigns some probability that a player is bluffing, based, presumably, on its assessment of that player's recent behavior. Is it better at doing this than the top human players? That's the only question. It can't have a fundamentally different orientation to the game, because there is no different orientation to the game that works. Bear in mind that most champion players have studied game theory pretty seriously; even the ones that haven't, grasp it intuitively.

The other day I heard that ignorance and apathy are sweeping the country. I didn't know that, but I don't really care.

(

| parent )

Well sure, but your point was about immunity to bluffing. (#103168)

by BlaiseP on Tue, 07/15/2008 - 1:43pm

Let me ask you this: how do you spot a bluff? Let's just say I hitched you and Chris Moneymaker up to an expert system and watched you play online Limit Hold 'Em for a few hundred hands. Now you're both immune to all the table shenanigans, and Polaris can get a good look at you both. In a sense, a bluff in poker is like a gambit in chess: it can either be taken up or rejected. But the most important part of a chess gambit, a game I play far better than poker, is that it's been made. Hard to explain to someone who hasn't faced it, but I think I'm making sense when I say a bluff can be spotted as rash behavior.

Now, one part of it is brute force: let's say I also went to pokeronline and got the raw feeds from hundreds of players. I'd isolate specific patterns of play, which as you point out is really the ultimate tell anyway. Very much like chess openings: for every strategy there exist one or more standard opposition strategies. Forget optimal play, that's just not going to win games, and is as unproductive as saying chess is N-P complete. Now it is true that chess is N-P complete, but effective strategy counts for wayway more.

See, Polaris isn't Chris Moneymaker. Lots of people who do well online don't do so well in front of real people and vice versa. It would be more interesting to let Polaris evolve its expert system around the online poker community.

(

| parent )

I added the link at the end (#102892)

by catchy on Mon, 07/14/2008 - 7:34am

Hope that works. Polaris played 4 rounds of 500 hands each and yes, the humans knew they were playing a puter.

That seems like a little more than luck, but not the largest sample-size either.

If the programs get better, tho, you ahve to wonder if people will really stick with online poker. They might delude themselves re: human experts, but surely that's b/c they believe humans are highly fallible and can be overcome by a combo. of luck + determination.

I'm not sure computers are viewed as that kind of advesary.

(

| parent )

Thanks (#103113)

by stillnotking on Tue, 07/15/2008 - 8:05am

Link still doesn't have a ton of info, and 2K hands is not really a reasonable sample size. "Stoxtrader" is a really good player, but he is a grinder rather than a plunger. It'd be interesting to see how Polaris did against a Gus Hansen, heck, even a Phil Hellmuth.

This is the paragraph that really grabbed me:

"There are two really big changes in Polaris over last year," said professor Michael Bowling, who supervised graduate students who programmed Polaris. "First of all, our poker model is much expanded over last year--its much harder for humans to exploit weaknesses. And secondly, we have added an element of learning, where Polaris identifies which common poker stratagy a human is using and switches its own strategy to counter.

(Emphasis mine). You just added an element of learning? This suggests to me that Polaris has a ways to go yet. We'll see.

The other day I heard that ignorance and apathy are sweeping the country. I didn't know that, but I don't really care.

(

| parent )

If They Could Rig It Right. . . (#102875)

by M Scott Eiland on Sun, 07/13/2008 - 10:13pm

. . .it might be a interesting publicity stunt for next year's WSOP main event--have Polaris as a player, with an operator in the chair to enter the actions of the other players and the cards in the computer's hand and on the board (and the chip count). Of course--as with most other "elite" players--the computer would probably be knocked out on the first day by some yokel who went all in with 10-6 offsuit against the computer's big pocket pair--at least Polaris doesn't have an ego to be bruised by the guy whipping out his cellphone to brag about having taken out the big timer.

I haven't seen SNK around for a while--this topic would be right up his alley.