Dota2 is one of the most popular, and complex, online games in the world, but an AI has once again shown itself to supersede human skill. In matches over the weekend, OpenAI’s “Five” system defeated two pro teams soundly, and soon you’ll be able to test your own mettle against — or alongside — the ruthless agent.
In a blog post, OpenAI detailed how its game-playing agent has progressed from its younger self — it seems wrong to say previous version, since it really is the same extensive neural network as many months ago, but with much more training.
The version that played at Dota2’s premiere tournament, The International, gets schooled by the new version 99 percent of the time. And it’s all down to more practice:
In total, the current version of OpenAI Five has consumed 800 petaflop/s-days and experienced about 45,000 years of Dota self-play over 10 realtime months (up from about 10,000 years over 1.5 realtime months as of The International), for an average of 250 years of simulated experience per day.
To the best of our knowledge, this is the first time an RL [reinforcement learning] agent has been trained using such a long-lived training run.
One is tempted to cry foul at a data center-spanning intelligence being allowed to train for 600 human lifespans. But really it’s more of a compliment to human cognition that we can accomplish the same thing with a handful of months or years, while still finding time to eat, sleep, socialize (well, some of us) and so on.
Dota2 is an intense and complex game with some rigid rules but a huge amount of fluidity, and representing it in a way that makes sense to a computer isn’t easy (which likely accounts partly for the volume of training required). Controlling five “heroes” at once on a large map with so much going on at any given time is enough to tax a team of five human brains. But teams work best when they’re acting as a single unit, which is more or less what Five was doing from the start. Rather than five heroes, it was more like five fingers of a hand to the AI.
Interestingly, OpenAI also discovered lately that Five is capable of playing cooperatively with humans as well as in competition. This was far from a sure thing — the whole system might have frozen up or misbehaved if it had a person in there gumming up the gears. But in fact it works pretty well.
You can watch the replays or get the pro commentary on the games if you want to hear exactly how the AI won (I’ve played but I’m far from good. I’m not even bad yet). I understand they had some interesting buy-back tactics and were very aggressive. Or, if you’re feeling masochistic, you can take on the AI yourself in a limited-time event later this week.
We’re launching OpenAI Five Arena, a public experiment where we’ll let anyone play OpenAI Five in both competitive and cooperative modes. We’d known that our 1v1 bot would be exploitable through clever strategies; we don’t know to what extent the same is true of OpenAI Five, but we’re excited to invite the community to help us find out!
Although a match against pros would mean all-out war using traditional tactics, low-stakes matches against curious players might reveal interesting patterns or exploits that the AI’s creators aren’t aware of. Results will be posted publicly, so be ready for that.
You’ll need to sign up ahead of time, though: The system will only be available to play from Thursday night at 6 PM to the very end of Sunday, Pacific time. They need to reserve the requisite amount of computing resources to run the thing, so sign up now if you want to be sure to get a spot.
OpenAI’s team writes that this is the last we’ll hear of this particular iteration of the system; it’s done competing (at least in tournaments) and will be described more thoroughly in a paper soon. They’ll continue to work in the Dota2 environment because it’s interesting, but what exactly the goals, means or limitations will be are yet to be announced.
Source : OpenAI Five crushes Dota2 world champs, and soon you can lose to it too
