In this talk, I propose that language can be modeled from raw speech data in a fully unsupervised manner with Generative Adversarial Networks (GANs) and that such modeling has implications both for the understanding of language acquisition and for the understanding of how deep neural networks learn internal representations. I propose an extension of the GAN architecture (fiwGAN) in which meaningful linguistic properties emerge from two networks learning to encode and decode information. FiwGAN captures the perception-production loop of human speech and, unlike most other deep learning architectures, has traces of communicative intent. I further propose a technique to identify latent variables in deep convolutional networks that represent linguistically meaningful units in a causal, disentangled, and interpretable way. We can thus uncover symbolic-like representations at the phonetic, phonological, syntactic and lexical semantic levels, analyze how learning biases in GANs match human learning biases in behavioral experiments, how speech processing in the brain compares to intermediate representations in deep neural networks, and what GANs’ innovative outputs can teach us about productivity in human language.