بازی تیراندازی شخص اول با روش های یادگیری ماشین
Abstract
1- Introduction and research motivation
2- Research on Doom using the VizDoom Game-AI research platform
3- Direct future prediction (DFP)
4- Direct future prediction (DFP) and asynchronous advantage Actor-Critic (A3C)
5- Advantage Actor-critic (A2C) and advantage Actor-Critic- long Short-Term memory (A2C-LSTM)
6- Asynchronous advantage Actor-Critic (A3C) and deep recurrent Q-Network (DRQN)
7- Direct future prediction (DFP) and deep recurrent Q-Network (DRQN)
8- Deep Q-Network (DQN) and deep recurrent Q-Network (DRQN)
9- Dueling deep Q-Network (DDQN) and double deep Q-Network (DDQN)
10- C51_DDQN and Reinforce
11- Techniques with better performance
12- Discussion and future work
Acknowledgment
References
Abstract
Artificial Intelligence in the form of machine learning is employed in games to control non-human computer-players, agents or bots. However, most of these games such as Atari took place in 2D environments that were fully observable to the agents. Currently, it is of extreme significance to employ such machine learning techniques and methods in 3D environments such as Doom. Therefore, In this paper, we train agents on the health gathering scenario of the classical first-person shooter game Doom by first presenting the Direct Future Prediction to train an agent that uses a simple architecture with no additional supervisory signals, then differentiate and compare the performance of the agents trained by using several different machine learning techniques, and the AI reinforcement learning platform ‘VizDoom’, a 3D partially observable environment, with interesting enhanced properties that makes agents to stand out from inbuilt AI agents and human players. We have continued to use computer games as a benchmark for the performance of AI as having been so successful in the past. We also compared the results of our findings to conclude the performance of the agents trained with different machine learning techniques. The agents performed well against both human players and inbuilt game agents.
Introduction and research motivation
In the last few decades, due to the progress in artificial intelligence, a revolution and sudden change have been observed in the technology both in hardware and software [1]. This change is seeping and taking over in our lives up to a certain extent, affecting how we live, work and entertain ourselves such as employing domestic robots servants, healthcare uses, electronic trading, remote sensing, expert systems, traffic control systems, autonomously-powered self-driving vehicles, and from behavioral algorithms to suggestive searches [2], etc. In the same way, gaming is a widely recognized part of our cultural landscape and as old as our human ancestors. The earliest computers were very slow and the interaction with the user was limited to basic principles. In the early ’40s, computers evolved, and programmers started to develop new virtual worlds and surprising ways of interaction between the user and the machine [3]. But now due to advancements in technology such as GPU’s[4], TPU’s[5] and the revolution in deep neural networks [6] it has become possible for artificial intelligence to stepin in video games as well where massive graphical data in the form of frames, or to be more specific a huge amount of multidimensional data is required to be processed and execute [7]. In the recent past, machine learning techniques and methods were employed in Atari games for training agents, where later, the agents performed on 49 different Atari games with better and improved results. However, most of these Atari games took place in 2D environments that were fully observable to the agents [8]. Currently, it is of extreme significance to employ such machine learning techniques and methods in 3D environments such as Doom [9] a first-person-shooter game shown in fig. 1, Starcraft [10] a third person shooter game based on real-time strategies, and sandbox open-world games such as Grand Theft Auto V and Minecraft [11] because the research community in AI think and consider that computer video games are the best test-beds for testing different artificial intelligence techniques, methods, and algorithms before evaluating them in real-world life. Thus, in this paper, state-of-the-art machine learning techniques that were before partially tested in 2D environments are now employed in a 3D environment known as Doom, to train, differentiate and compare agents performances, such as advantage actor-critic (A2C) [12], advantage actor-critic long short-term memory (A2C-LSTM) [13], asynchronous advantage actor-critic (A3C) [۱۴], Deep Q-network (DQN) [15], Deep recurrent Q-network (DRQN) [16], Double deep Q-network (DDQN) [17], C51-DDQN [18], Dueling deep Q-network (DDQN) [19], and Reinforce [20] whereafter applying most of the agents are found useful and effective. In addition, this paper presents one of the 4 best techniques that performed well on the VizDoom AI platform [21]. It was suggested that making such research available is beneficial for the community researching on first-person-shooter games which may set up a base for further research and improvement.