We are in the midst of an extraordinary arms race for building larger, more powerful chatbots that are adapted (“aligned”) to follow human demands by producing appropriate responses to the articulations of human commands. It is exciting to see how far these chatbots can be pushed via scaling large language models (LLMs) and various design parameters. However, since this rapid progress is motivated by market competition pressure, the underlying design choices that enable these models are often opaque, rendering them prone to many issues and inefficiencies. In this talk, I am planning to go over issues regarding aligning LLMs to follow human commands and provide partial results. I will explore issues related to the size and diversity of human feedback, algorithm selection, and other concerns regarding the future of this trajectory. My objective is to raise a broad set of questions that indirectly emphasize the importance of academic research in this new era of AI.