I missed this back in March (I think there was a lot going on back then?) but the feature-length documentary AlphaGo is now available to stream for free on YouTube. The movie documents the development by DeepMind/Google of the AlphaGo computer program designed to play Go and the competition between AlphaGo and Lee Sedol, a Go master.
With more board configurations than there are atoms in the universe, the ancient Chinese game of Go has long been considered a grand challenge for artificial intelligence. On March 9, 2016, the worlds of Go and artificial intelligence collided in South Korea for an extraordinary best-of-five-game competition, coined The DeepMind Challenge Match. Hundreds of millions of people around the world watched as a legendary Go master took on an unproven AI challenger for the first time in history.
During the competition back in 2016, I wrote a post that rounded up some of the commentary about the matches.
Move after move was exchanged and it became apparent that Lee wasn’t gaining enough profit from his attack.
By move 32, it was unclear who was attacking whom, and by 48 Lee was desperately fending off White’s powerful counter-attack.
I can only speak for myself here, but as I watched the game unfold and the realization of what was happening dawned on me, I felt physically unwell.
With just four hours of practice playing against itself and no study of outside material, AlphaZero (an upgraded version of Alpha Go, the AI program that Google built for playing Go) beat the silicon pants off of the world’s strongest chess program yesterday. This is massively and scarily impressive.
AlphaZero won the closed-door, 100-game match with 28 wins, 72 draws, and zero losses.
Oh, and it took AlphaZero only four hours to “learn” chess. Sorry humans, you had a good run.
That’s right — the programmers of AlphaZero, housed within the DeepMind division of Google, had it use a type of “machine learning,” specifically reinforcement learning. Put more plainly, AlphaZero was not “taught” the game in the traditional sense. That means no opening book, no endgame tables, and apparently no complicated algorithms dissecting minute differences between center pawns and side pawns.
This would be akin to a robot being given access to thousands of metal bits and parts, but no knowledge of a combustion engine, then it experiments numerous times with every combination possible until it builds a Ferrari. That’s all in less time that it takes to watch the “Lord of the Rings” trilogy. The program had four hours to play itself many, many times, thereby becoming its own teacher.
Grandmaster Peter Heine Nelson likened the experience of watching AlphaZero play to aliens:
After reading the paper but especially seeing the games I thought, well, I always wondered how it would be if a superior species landed on earth and showed us how they play chess. I feel now I know.
Unpredictable machines. Machines that act more like the weather than Newtonian gravity. That’s going to take some getting used to.
Albert Silver has a good overview of AlphaZero’s history and what Google has accomplished. To many chess experts, it seemed as though AlphaZero was playing more like a human than a machine:
If Karpov had been a chess engine, he might have been called AlphaZero. There is a relentless positional boa constrictor approach that is simply unheard of. Modern chess engines are focused on activity, and have special safeguards to avoid blocked positions as they have no understanding of them and often find themselves in a dead end before they realize it. AlphaZero has no such prejudices or issues, and seems to thrive on snuffing out the opponent’s play. It is singularly impressive, and what is astonishing is how it is able to also find tactics that the engines seem blind to.
So, where does Google take AlphaZero from here? In a post which includes the phrase “Skynet Goes Live”, Tyler Cowen ventures a guess:
I’ve long said that Google’s final fate will be to evolve into a hedge fund.
Why goof around with search & display advertising when directly gaming the world’s financial market could be so much more lucrative?
I have been following with fascination the match between Google’s Go-playing AI AlphaGo and top-tier player Lee Sedol and with even more fascination the human reaction to AlphaGo’s success. Many humans seem unnerved not only by AlphaGo’s early lead in the best-of-five match but especially by how the machine is playing in those games.
Then, with its 19th move, AlphaGo made an even more surprising and forceful play, dropping a black piece into some empty space on the right-hand side of the board. Lee Sedol seemed just as surprised as anyone else. He promptly left the match table, taking an (allowed) break as his game clock continued to run. “It’s a creative move,” Redmond said of AlphaGo’s sudden change in tack. “It’s something that I don’t think I’ve seen in a top player’s game.”
When Lee Sedol returned to the match table, he took an usually long time to respond, his game clock running down to an hour and 19 minutes, a full twenty minutes less than the time left on AlphaGo’s clock. “He’s having trouble dealing with a move he has never seen before,” Redmond said. But he also suspected that the Korean grandmaster was feeling a certain “pleasure” after the machine’s big move. “It’s something new and unique he has to think about,” Redmond explained. “This is a reason people become pros.”
“A creative move.” Let’s think about that…a machine that is thinking creatively. Whaaaaaa… In fact, AlphaGo’s first strong human opponent, Fan Hui, has credited the machine for making him a better player, a more beautiful player:
As he played match after match with AlphaGo over the past five months, he watched the machine improve. But he also watched himself improve. The experience has, quite literally, changed the way he views the game. When he first played the Google machine, he was ranked 633rd in the world. Now, he is up into the 300s. In the months since October, AlphaGo has taught him, a human, to be a better player. He sees things he didn’t see before. And that makes him happy. “So beautiful,” he says. “So beautiful.”
Creative. Beautiful. Machine? What is going on here? Not even the creators of the machine know:
“Although we have programmed this machine to play, we have no idea what moves it will come up with,” Graepel said. “Its moves are an emergent phenomenon from the training. We just create the data sets and the training algorithms. But the moves it then comes up with are out of our hands — and much better than we, as Go players, could come up with.”
Generally speaking,1 until recently machines were predictable and more or less easily understood. That’s central to the definition of a machine, you might say. You build them to do X, Y, & Z and that’s what they do. A car built to do 0-60 in 4.2 seconds isn’t suddenly going to do it in 3.6 seconds under the same conditions.
Now machines are starting to be built to think for themselves, creatively and unpredictably. Some emergent, non-linear shit is going on. And humans are having a hard time figuring out not only what the machine is up to but how it’s even thinking about it, which strikes me as a relatively new development in our relationship. It is not all that hard to imagine, in time, an even smarter AlphaGo that can do more things — paint a picture, write a poem, prove a difficult mathematical conjecture, negotiate peace — and do those things creatively and better than people.
Unpredictable machines. Machines that act more like the weather than Newtonian gravity. That’s going to take some getting used to. For one thing, we might have to stop shoving them around with hockey sticks. (thx, twitter folks)
Update: AlphaGo beat Lee in the third game of the match, in perhaps the most dominant fashion yet. The human disquiet persists…this time, it’s David Ormerod:
Move after move was exchanged and it became apparent that Lee wasn’t gaining enough profit from his attack.
By move 32, it was unclear who was attacking whom, and by 48 Lee was desperately fending off White’s powerful counter-attack.
I can only speak for myself here, but as I watched the game unfold and the realization of what was happening dawned on me, I felt physically unwell.
Generally I avoid this sort of personal commentary, but this game was just so disquieting. I say this as someone who is quite interested in AI and who has been looking forward to the match since it was announced.
One of the game’s greatest virtuosos of the middle game had just been upstaged in black and white clarity.
AlphaGo’s strength was simply remarkable and it was hard not to feel Lee’s pain.
Let’s get the caveats out of the way here. Machines and their outputs aren’t completely deterministic. Also, with AlphaGo, we are talking about a machine with a very limited capacity. It just plays one game. It can’t make a better omelette than Jacques Pepin or flow like Nicki. But not only beating a top human player while showing creativity in a game like Go, which was considered to be uncrackable not that long ago, seems rather remarkable.↩
Stay Connected