Deep reinforcement learning for FlipIt security game
Files
Accepted manuscript
Date
2021-12-02
Authors
Greige, Laura
Chin, Sang
Version
Accepted manuscript
OA Version
Citation
Greige, L., Chin, P. (2022). Deep Reinforcement Learning for FlipIt Security Game. In: Benito, R.M., Cherifi, C., Cherifi, H., Moro, E., Rocha, L.M., Sales-Pardo, M. (eds) Complex Networks & Their Applications X. COMPLEX NETWORKS 2021. Studies in Computational Intelligence, vol 1015. Springer, Cham. https://doi.org/10.1007/978-3-030-93409-5_68
Abstract
Reinforcement learning has shown much success in games
such as chess, backgammon and Go [21,24,22]. However, in most of these
games, agents have full knowledge of the environment at all times. In this
paper, we describe a deep learning model in which agents successfully
adapt to different classes of opponents and learn the optimal counter-strategy
using reinforcement learning in a game under partial observability.
We apply our model to FlipIt [25], a two-player security game in
which both players, the attacker and the defender, compete for ownership
of a shared resource and only receive information on the current state of
the game upon making a move. Our model is a deep neural network
combined with Q-learning and is trained to maximize the defender’s
time of ownership of the resource. Despite the noisy information, our
model successfully learns a cost-effective counter-strategy outperforming
its opponent’s strategies and shows the advantages of the use of deep
reinforcement learning in game theoretic scenarios. We also extend FlipIt
to a larger action-spaced game with the introduction of a new lower-cost
move and generalize the model to n-player FlipIt.