Most machine learning theory is focused on studying the performance of systems under worst-case or even adversarial  circumstances.  However, the world does not always confront a learning agent with worst-case problems. The problem instances that arise in real-world tasks are often easier than the worst case, requiring less computational resources, less data, and lower data procurement costs. In this talk, I will discuss the notion of easiness in two contexts: statistical learning with fixed features and also problems where we learn the features (known as learning a  representation). After giving an overview of the statistical learning setting and the idea of easiness in learning, I'll focus on my recent work in learning at faster rates under a wide-reaching condition known as stochastic mixability. I'll then shift to my work on refining measures of how much data is needed to learn, and provide theoretical justification for learning using less data. This part of the talk will focus primarily on my work in learning a dictionary for sparse coding in a supervised way. Lastly, I'll close with future directions.