Today, OpenAI announced that they have created a bot which beats the world’s top professionals at 1v1 matches of Dota 2 under standard tournament rules.
The bot learned the game from scratch by self-play, and does not use imitation learning or tree search. This is a step towards building AI systems which accomplish well-defined goals in messy, complicated situations involving real humans.
Today they played Dendi on mainstage at The International, winning a best-of-three match. Over the past week, their bot was undefeated against many top professionals including SumaiL (top 1v1 player in the world) and Arteezy (top overall player in the world).
Dota 1v1 is a complex game with hidden information. Agents must learn to plan, attack, trick, and deceive their opponents. The correlation between player skill and actions-per-minute is not strong, and in fact, their AI’s actions-per-minute are comparable to that of an average human player.
Success in Dota requires players to develop intuitions about their opponents and plan accordingly. In the above video you can see that the bot has learned – entirely via self-play – to predict where other players will move, to improvise in response to unfamiliar situations, and how to influence the other player’s allied units to help it succeed.
The full game of Dota is played by two teams of five. Each player chooses from a hundred heroes and hundreds of items. Our next step is to create a team of Dota 2 bots which can compete and collaborate with the top human teams.