Last time, I introduced a half part about the session named “Unity for Deep Learning: Tool kit: ML-Agents” which is held in Unite 2018 Tokyo which was started from 7 May 2018 for 3 days at Tokyo International Forum.
This was held by Mike Geig who is working for Unity as Head of Global Evangelism Content.
[Link: Unite 2018 Tokyo: Introduce about “Unity for Deep Learning: Tool kit: ML-Agents”. Part 1]
As the first half of this session, last time, I introduced his explanation about the basic knowledge about Machin Learning and detail settings for car race game controlled by AI which was trained by Imitation Learning.
This time, I’ll introduce 2nd half of this session.
After his explanation about settings of Imitation Learning, some examples which were used ML-Agents were introduced by him. Those examples were also used at some sessions and some blogs.
[Twelve Agents, One Brain, Independent Rewards]
In this case, there are 12 agents, so it can be learned 12 times faster than learning by 1 agent. For just about 20 seconds, it’s balanced.
[Two Agents, One Brain, Cooperative Rewards]
It’s one example to learn cooperating together, because the their rewards are linked together.
[Four Agents, Multi-Brain, Competitive Rewards]
This is an example for Multi-Brain. The way to learn was that he made only strikers learn firstly, and made keepers learn separately later, and finally merging those learning results, execute the training including both strikers and keepers together.
He said that depending on the progress of the training, even no any written logics, strikers started to stay at the location which is good point to get a goal, and keepers started to stay at the location which is good point to block the goals.
[Curriculum Learning]
Next, he introduced other learning way. It’s called “Curriculum Learning”. This is one type of Reinforcement Learning.
In the case which is too complicated, if some good things happen coincidentally, maybe it can be learned very well, but if it’s too complicated and too big, it’s difficult to happen that kind of lucky chance. If so, there is a possibility not to proceed the learning.
To resolve that kind of case, Curriculum Learning starts from small simple tasks to learn, and proceed to step up to more complicated and bigger tasks.
Following image is function list of ML-Agents. For just a half year, these many functions have been released. Their developing speed is very fast! I feel that Unity really focuses on AI too.
By the way, in this session, he’s talking about ver 0.3 of ML-Agents, but currently already released ver 0.4. It’s very fast.
Currently, in Unity, Danny Lange is top for Machine Learning. He worked for Uber as top of Machine Learning division, before that, in Amazon and Microsoft, he worked for many companies leading the each Machine Learning department.
(He explained Machine Learning in the session of GDC before.)
By hiring him, we can understand how Unity focuses on this area.
[Original Japanese Site: http://blog.lab7.biz/archives/10503014.html]