This proposal outlines a research agenda focused on improving the performance, efficiency, and robustness of modern deep learning models (with an emphasis on transformer-based neural networks) under real-world constraints. As large language models (LLMs) become increasingly central to AI systems, it is vital to understand how these models behave and perform when deployed outside idealized settings. My work explores this space through three complementary threads: (1) structure-aware modeling in constrained domains like tabular data, where models like STEP demonstrate that simplicity and domain alignment can outperform more complex pipelines; (2) inference-time efficiency, including ongoing work on tokenizer compression and long-context extensions to reduce decoding costs; and (3) robustness and privacy under adversarial pressure, including studies of coercion attacks and information leakage. Together, these projects reflect a cohesive research direction grounded in practicality and broad applicability. Future work will investigate underexplored deployment scenarios, such as tokenization, tabular modeling, and computational trade-offs, with the goal of designing adaptable, efficient, and robust transformer systems.