Google’s TPU 8t and 8i Power the Agentic AI Era

Google unveiled eighth-generation TPUs—TPU 8t for large-scale training and TPU 8i for low-latency agent inference—arguing its vertically integrated stack avoids the high margins of Nvidia-based rentals. TPU 8t boosts FP4 EFlops per pod 2.8x over the prior gen, doubles scale-up bandwidth, quadruples scale-out networking, and can scale beyond one million chips in a single job via new Virgo networking and TPU Direct Storage to cut CPU-mediated data hops. TPU 8i focuses on agent workloads with a redesigned Boardfly network topology to reduce latency, delivering ~9.8x FP8 EFlops per pod and much higher HBM capacity and pod size; it targets real-time sampling and memory-heavy inference. Google says the two-chip roadmap reflects a strategic split for training vs. agent inference and promises better cost-per-token economics for cloud customers.

Google unveiled technical details of its eighth-generation TPU family — TPU 8t and TPU 8i — designed for modern AI workloads like agentic systems, world models, and massive MoEs. TPU 8t targets large-scale pretraining and embedding-heavy tasks with a 9,600-chip 3D torus superpod, a new SparseCore accelerator for irregular embedding access, improved VPU/MXU overlap for higher utilization, and native FP4 to double MXU throughput while cutting memory bandwidth demands. The platform integrates Arm-based Axion CPU headers to reduce host-side preprocessing bottlenecks and is part of Google Cloud’s AI Hypercomputer to support training, fine-tuning, and serving. These changes matter because they address real throughput, latency, and scaling limits for next-gen model training and multi-agent reasoning at cloud scale.