Artificial intelligence (AI) and large language models (LLMs) specifically, have recently been used to model source code, which has proven to be effective for a variety of software development tasks such as code completion, summarization, translation, and debugging, among others. While the programming languages and software engineering (PL/SE) communities are abuzz with AI-based tools for code development (AIforDev), the application of AIforDev to parallel code has been largely unexplored. Writing, debugging and optimizing parallel code is hard, and the question before the HPC community is -- do AI and LLMs hold the potential for revolutionizing parallel software development. In this talk, I will address the shortcomings of current LLMs when used for parallel code development and how we can close the gap toward building HPC-capable LLMs. I will further highlight emerging areas of research such as improving code modeling capabilities to facilitate various aspects of parallel code development, such as generating correct and efficient parallel code, reasoning about parallel performance, and much more.
Abhinav Bhatele is an associate professor in the department of computer science, and director of the Parallel Software and Systems Group at the University of Maryland, College Park. His research interests are broadly in systems and AI, with a focus on parallel computing and distributed AI. He has published research in parallel programming models and runtimes, network design and simulation, applications of machine learning to parallel systems, parallel deep learning, and on analyzing/visualizing, modeling and optimizing the performance of parallel software and systems. Abhinav has received best paper awards at Euro-Par 2009, IPDPS 2013, IPDPS 2016, and PDP 2024, and a best poster award at SC 2023. He was selected as a recipient of the IEEE TCSC Award for Excellence in Scalable Computing (Early Career) in 2014, the LLNL Early and Mid-Career Recognition award in 2018, the NSF CAREER award in 2021, the IEEE TCSC Award for Excellence in Scalable Computing (Middle Career) in 2023, and the UIUC CS Early Career Academic Achievement Alumni Award in 2024.
Abhinav received a B.Tech. degree in Computer Science and Engineering from I.I.T. Kanpur, India in May 2005, and M.S. and Ph.D. degrees in Computer Science from the University of Illinois at Urbana-Champaign in 2007 and 2010 respectively. He was a post-doc and later computer scientist in the Center for Applied Scientific Computing at Lawrence Livermore National Laboratory from 2011-2019. Abhinav is an associate editor of the IEEE Transactions on Parallel and Distributed Systems (TPDS). He was one of the General Chairs of IEEE Cluster 2022, and Research Papers Chair of ISC 2023.