Abstract
Why this tutorial matters
The rapid growth of foundation models and large-scale AI systems has transformed modern computing
infrastructure. As model sizes and cluster scales continue to grow, system performance is increasingly
constrained not by compute alone, but by communication, memory capacity, memory bandwidth, collective
operations, and interconnect fabrics. Emerging technologies such as Ultra Ethernet, InfiniBand, NVLink,
UALink, CXL-based memory expansion, and memory-semantic fabrics are redefining the architecture of AI
clusters and introducing new research challenges spanning systems, networking, and computer architecture.
This tutorial introduces simulation-driven methodologies for studying and designing next-generation AI
infrastructure. Participants will learn the fundamentals of distributed AI training and inference,
understand the role of communication and memory bottlenecks, and gain hands-on experience using
Astra-Sim and related simulation frameworks. The tutorial further covers emerging research directions in
AI fabrics, memory disaggregation, large-scale model serving, and memory-semantic networking.
The tutorial combines conceptual foundations, practical demonstrations, and guided hands-on exercises,
enabling researchers and practitioners to evaluate AI infrastructure designs without requiring access to
expensive production-scale clusters.
Objectives
- Introduce the architecture of modern AI infrastructure.
- Explain communication and memory bottlenecks in large-scale AI systems.
- Provide hands-on experience using Astra-Sim and LLM-Serving-Sim for modeling AI workloads.
- Discuss emerging research opportunities in memory-semantic fabrics, AI networking, and disaggregated AI infrastructure, and demonstrate how simulation can be used for such research problems.
Intended Audience
- Graduate students and researchers in HPC, systems, networking, and computer architecture.
- Industry practitioners working on AI infrastructure and large-scale training and inference.
Prerequisites & Level
- Basic computer architecture
- Basic networking concepts
- Familiarity with Linux
- No prior experience with AI infrastructure simulators is required.