AI & ML
Evolving Google’s Global and Data Center Networks for a New Era of Connectivity
Throughout 25 years of developing Google’s global network, we've transformed our infrastructure through several technological shifts, and we're now advancing into a fourth evolution that prioritizes efficiency and scalability.
The Transition to the AI Era
The landscape of computing has undergone significant transformations over the past quarter of a century, with Google at the forefront of these shifts. The emergence of advanced artificial intelligence (AI) marks a notable inflection point in this evolution. Unlike earlier eras characterized by foundational Internet, streaming, and cloud services, the demands of AI applications are distinct and inherently more complex. These applications impose new requirements not just on computational power, but also on networking capabilities.
An interesting challenge in this domain arises from the fundamental differences in moving data as opposed to electrical signals; it’s a far simpler task to transmit light (data over fiber) than it is to transmit electricity (power). AI workloads can often exceed the capacity limits of individual data centers, leading Google to strategically position facilities near sustainable energy sources or in areas where new renewable energy infrastructure can be integrated into existing grids. By allowing AI workloads to be distributed across a network of campuses, Google effectively pools computing resources, overcoming the energy limitations that single sites would otherwise encounter.
Building a Comprehensive AI Infrastructure
To address the growing demands of AI, Google has developed a vertically integrated AI architecture that encompasses everything from specialized chips to comprehensive platforms. This technology stack is designed to support a variety of AI applications and includes notable components such as the [Gemini Enterprise Agent Platform](https://cloud.google.com/blog/products/ai-machine-learning/introducing-gemini-enterprise-agent-platform?e=48754805) for scaling and governing applications, as well as the [AI Hypercomputer](https://cloud.google.com/solutions/ai-hypercomputer?e=48754805), which combines custom hardware with flexible software options to form the core of its AI capabilities. Essentially, the network serves as the backbone that connects these innovations, integrating the vast improvements from decades of research and development.
It’s crucial for the infrastructure to not only deliver high bandwidth but also maintain performance standards required by AI technologies. The network must extend its capacities both within localized data centers and across wider networks, ensuring that AI training data moves efficiently from its source to processing units.
Innovating Network Architectures
To meet these demands, Google is rethinking the foundational elements of its network infrastructure. This paradigm shift revolves around three focal points: the internal infrastructure of the AI Hypercomputer, intercommunication between units, and the expansive global network necessary for a seamless AI experience.
The sheer scale of current AI models means that training them requires vast computational and networking resources. This results in a need for enhanced network bandwidth and minimization of latency — particularly given AI workloads’ characteristic traffic patterns, which are often defined by bursts of intensive activity. Stability is paramount because the large training jobs can quickly become susceptible to slowdowns caused by hardware failures.
In response to these challenges, Google has embraced a “campus as a computer” philosophy. This approach involves segmenting the network into three distinct areas: a scale-up domain for internal connectivity, an east-west acceleration fabric for optimal resource distribution, and the [Jupiter network](https://cloud.google.com/blog/products/networking/speed-scale-reliability-25-years-of-data-center-networking?e=48754805) for external data and compute access. This separation allows each domain to advance independently, facilitating innovation without compromising overall performance.
A highlight of this strategy is the introduction of the [Virgo Network](https://cloud.google.com/blog/products/networking/introducing-virgo-megascale-data-center-fabric), tailored specifically for AI needs. This innovative fabric employs dense switching technology to enhance bandwidth while cutting down on latency, allowing for flexibility in scaling across multiple data centers—effectively mitigating the limits associated with individual physical sites.
Maximizing Efficiency in AI Operations
Building an efficient infrastructure is critical, especially when dealing with clusters composed of hundreds of thousands of processing units. Failures are an inevitable reality; hence, quick fault identification and mitigation systems must be in place to prevent wasted computational effort.
The Virgo Network integrates autonomous reliability features that maximize output efficiency, a term known as "goodput." These capabilities extend existing straggler detection systems with automated hang detection, ensuring that when a job stalls, the system quickly isolates the issue, minimizes downtime, and allows for smooth job recovery.
To further enhance performance, high-resolution telemetry is deployed for detailed monitoring of micro-bursts that ordinary systems may overlook. This level of insight into network operations promotes superior management and rapid recovery processes.
Beyond Connectivity: A Global Network for AI
Google's global network infrastructure is vital for the demands of AI-driven inference applications. The network spans over 10 million kilometers of fiber optics and connects 43 cloud regions, supplemented by more than 200 edge nodes. This extensive framework is designed for performance, providing the low latency required for an optimal AI experience.
Key to its functionality is the AI-native Cloud Interconnect, which supports the high bandwidth and low-latency demands of AI workloads. The interconnect not only facilitates vast data transfers but also offers flexible connection options for diverse environments—making it easier to integrate AI-heavy workloads whether they reside on-premises or across clouds.
A Vision for the Future
As organizations leverage these advanced networking innovations, they can expect not just improved computational efficiency but a future where AI capabilities are more accessible and can be scaled reliably. Google’s commitment to expanding its networking technologies furthers its aim to support enterprises in their digital transformation journeys, effectively making AI services beneficial to a broader audience. The robust infrastructure is more than just a support system; it's foundational to the future of AI applications.
