I participated in a study session called ‘Machine Learning Business Workshop‘ hosted by Unity Singapore on June 4.
The main topics were a basic technical explanation of Unity’s machine learning library ‘ML-Agents‘ and some examples of how Unity’s machine learnings are used.
The instructors were Marek Marchlewicz of Unity Technical Evangelist for technical explanation, and Stijn De Keyser of Unity Sales for case studies.
【Contents of this article】
1. Opening talk from sales
2. Basic technical explanation of ML-Agents
3. Examples of ML-Agents applied to Games
4. Effectiveness of Unity in Non-Gaming
5. Examples of ML-Agents used in Non-Gaming
【1. Opening talk from sales】
First of all, from Stijn in sales, some figures related Unity were shared like Unite Keynote.
Because the use of machine learning in Non-Game areas is also actively pushed by Unity, so the use of Unity for Non-Game is expanding. This chart has also been seen in some Unite Keynote before.
And he introduced some Unity use cases. Oh, the IKEA’s AR app was also made with Unity!
【2. Basic technical explanation of ML-Agents】
From here, Technical Evangelist Marek explained machine learning itself and Unity ML-Agents.
First of all, what is machine learning ? (I will skip about the detail explanations of the basic ML knowledge here.)
And he explained basic topic about Unity’s machine learning library ‘ML-Agents’.
Well, this is also the basic basis for those who have used ML-Agents, like ML-Agents uses TensorFlow, or as a learning method it has ‘Supervised Learning‘, ‘Unsupervised Learning‘, ‘Reinforcement Learning‘, ‘Imitation Learning‘, ‘Curriculum Learning‘ etc.
【3. Examples of ML-Agents applied to Games】
Some demos using ML-Agents on Game were shown. First, two very simple games applied ML-Agents were demonstrated.
One is the game applied ML-Agents to hit the ball well to the cube that appears randomly. The other is the game that you control a helicopter which falls by gravity and click it to hover at a fixed height.
And some sample games included in ML-Agents were introduced.
【4. Effectiveness of Unity in Non-Gaming】
Stijn in Sales explained about the use of Unity and ML-Agents in the Non-Gaming industry here.
In fact, it was this part that was most interesting in this session. The other topics were well known. I’ve also heard about Unity for Non-Gaming industry, but there wasn’t much information on the Internet.
The method of using real-world data for Computer Vision in Autonomous Vehicle is to train and evaluate sample cases gathered as a Training Set.
Gathering training data using real-world data for that Autonomous Vehicle is very expensive, and only the data that actually occurred can be collected.
And it is impossible to test every possible situation in the real world. For example, the following abnormal situations.
Certainly, it is abnormal. The man who got a cart comes to play on the high way (upper left), and the tank suddenly crosses the normal road (upper right), it definitely can’t be tested in a realistic world.
Why is Unity ideal for this kind of simulations?
- Unity is a Real-time 3D Engine.
: It can be handled objects in 3D space like in the real world and can reproduce sensors of the real world in virtual world using the Physics of Unity. - Unity is an Open Platform, and there is also a lot of useful assets in Asset Store.
: You can easily create an environment using existing assets, or you can use a large developer community when you cope with problems. - Unity has an organization that supports each industry.
【5. Examples of ML-Agents used in Non-Gaming】
He introduced two use cases of simulation with Unity and ML-Agents in Non-Gaming area.
The first one is the simulation of car control.
SynCity is a data set for simulation provided by CVEDIA. A simulation town built with 3D models can be used for various simulations while changing the brightness and weather. Carry out car control simulation on Unity using SynCity.
Baidu in China is using them to simulate cars driving automatically. Conducting simulations in different environments such as daytime and night.
Simulating various sensors using Unity functions.
Next, use it for simulation of Smart City.
A project in collaboration with Bellevue city, where Unity’s Seattle office is located.
Computer Vision demonstration project to analyze the image of a camera that captures a city and recognize the situation.
In order to do Machine Learning for the situation, it is not possible to collect images of all situations / conditions in the real world, so need to try simulation using 3D space on Unity. The Machine Learning of Computer Vision is done by the picture which taken with the camera in Unity.
I see.
Unity, a 3D game engine, is excellent for creating realistic virtual 3D spaces.
Using it, it seems that it is often used for real-time 3D cinema movie creation, but it is also used for 3D simulation. And the machine learning of this Unity ML-Agents is utilized there.
It is often said in Unite’s Keynote that Unity is also focusing on the industries other than games. As a Unity user it’s very exciting and pleasing.
[Original Japanese Site: http://blog.lab7.biz/archives/19055197.html ]