Textbook in PDF format

May 11, 1997, was a watershed moment in the history of artificial intelligence (AI): the IBM supercomputer chess engine, Deep Blue, beat the world Chess champion, Garry Kasparov. It was the first time a machine had triumphed over a human player in a Chess tournament. Fast forward 19 years to May 9, 2016, DeepMind’s AlphaGo beat the world Go champion Lee Sedol. AI again stole the spotlight and generated a media frenzy. This time, a new type of AI algorithm, namely machine learning (ML) was the driving force behind the game strategies.
What exactly is ML? How is it related to AI? Why is deep learning (DL) so popular these days? This book explains how traditional rule-based AI and ML work and how they can be imple mented in everyday games such as Last Coin Standing, Tic Tac Toe, or Connect Four. Game rules in these three games are easy to implement. As a result, readers will learn rule-based AI, deep reinforcement learning, and more importantly, how to combine the two to create powerful game strategies (the whole is indeed greater than the sum of its parts) without getting bogged down in complicated game rules.
Preface
Section I Rule-Based AI
Rule-Based AI in the Coin Game
Look-Ahead Search in Tic Tac Toe
Planning Three Steps Ahead in Connect Four
Recursion and MiniMax Tree Search
Depth Pruning in MiniMax
Alpha-Beta Pruning
Position Evaluation in MiniMax
Monte Carlo Tree Search
Section II Deep Learning
Deep Learning in the Coin Game
Policy Networks in Tic Tac Toe
A Policy Network in Connect Four
Section III Reinforcement Learning
Tabular Q-Learning in the Coin Game
Self-Play Deep Reinforcement Learning
Vectorization to Speed Up Deep Reinforcement Learning
A Value Network in Connect Four
Section IV AlphaGo Algorithms
Implementing AlphaGo in the Coin Game
AlphaGo in Tic Tac Toe and Connect Four
Hyperparameter Tuning in AlphaGo
The Actor-Critic Method and AlphaZero
Iterative Self-Play and AlphaZero in Tic Tac Toe
AlphaZero in Unsolved Games
Bibliography