Normal-form Games (NFGs) are the backbone of game theory. They are well-studied and have established solution concepts such as Nash, correlated, and coarse correlated equilibria. Traditionally, research has focused on solving NFGs precisely, one at a time, with iterative solvers. Recent work attempts to solve these games using paradigms suitable for machine learning: quickly and approximately, with function approximation. This talk will be split into two sections, first I will establish some useful properties of NFGs that can be exploited in a machine learning context: an equilibrium-invariant embedding for NFGs which simplifies representation and sampling. Secondly I will describe a neural network architecture for very quickly computing (C)CEs in n-player general-sum games. Finally, I will explain why solving NFGs at scale could be an important component in solving temporal game formulations like Markov Games at scale, in a principled manner.