From Scale-Out to Scale-Up: Why AI Needs Open Networking and a Unified Software Foundation

AI infrastructure is entering a new phase. The industry is moving from isolated accelerator deployments to large, tightly coupled AI systems built to train, tune and serve increasingly complex models. As the number of GPUs and XPUs inside these systems grows, the network is no longer just a way to move data between servers. It becomes part of the compute architecture itself.

That shift is creating a distinct and urgent requirement for scale-up networking: a fabric designed to connect accelerators with the bandwidth, latency, determinism and reliability needed to make large AI systems behave like a unified pool of compute.

Why the AI Industry Needs the Scale-Up Domain

To serve the first wave of generative AI applications, AI infrastructure discussions have often centered on scale-out networking: connecting many servers, racks and clusters across the data center. Scale-out remains critical, but it is only part of the story. As AI systems become more tightly integrated, the industry also needs a dedicated scale-up domain that connects accelerators inside rack-scale and system-scale environments with extremely high bandwidth and predictable low latency.

The purpose of the scale-up domain is simple: keep expensive AI accelerators fed, synchronized and highly utilized. Mixture of Expert Models (MoE), modern training, distributed inference and emerging agentic workloads require rapid movement of parameters, activations, embeddings and intermediate results across accelerators. When that movement is delayed or inconsistent, overall system performance suffers, even when the accelerators themselves are powerful.

This is why scale-up networking is becoming a defining layer of next-generation AI infrastructure. It is where performance, utilization and total cost of ownership increasingly converge. A scale-up fabric must deliver memory-semantic behavior, deterministic performance and the ability to grow with increasingly dense accelerator systems.

Dell’Oro forecasts nearly $1 trillion in AI back-end switch spending from 2026 to 2030, with scale-up networking representing more than half of the market by 2030. That market trajectory reflects a broader reality: AI compute growth is creating a massive demand for AI networking.

Why Open Scale-Up Networking Matters More than Ever

The AI compute landscape is becoming more heterogeneous. Hyperscalers, neoclouds and AI infrastructure builders are no longer designing around a single type of accelerator. They are integrating GPUs, custom XPUs, merchant silicon, specialized inference engines and evolving accelerator roadmaps. This diversity is needed to mitigate supply risk and cater to various types of workloads and requirements. In turn, growing heterogeneity on the compute side is driving greater diversity and openness on the networking side as well.

A closed scale-up fabric can limit flexibility. It can constrain which accelerators customers deploy, how quickly they adopt new compute technologies and how much control they have over the system architecture. As more GPU and XPU vendors adopt Ethernet-based scale-up approaches, the industry needs an open networking foundation that can connect diverse compute engines without forcing customers into a narrow ecosystem. This is where ESUN and open Ethernet-based scale-up become important. At Dell’Oro Group, we expect Ethernet to win in the scale-up domain, the same way it dominated in the scale-out domain.

Why Customers Need Flexible Consumption Models

AI infrastructure customers are not all starting from the same place. Some have deep system-design teams and want maximum component-level control. Others want integrated systems that reduce qualification time and operational complexity. Still others want a full software-enabled stack that can be deployed and managed as a complete solution.

This flexible consumption model is important for a market that is still moving quickly. Hyperscalers may want silicon-level control to optimize for their own system architecture. GPU and XPU builders may want an open scale-up foundation that helps them bring new accelerator platforms to market. Neocloud operators may value integrated systems and software that improve deployment speed, utilization and operational simplicity.

By offering multiple paths to adoption, customers can be served across different levels of maturity, scale and control. The common foundation is the same: an open, deterministic, AI-optimized scale-up fabric. The difference is how each customer chooses to consume it.

Why Scale-Out Remains Essential

As AI clusters expand across servers and racks, scale-out networks must deliver sufficient bandwidth, effective congestion management and reliable communication to support training and distributed inference. These capabilities help keep accelerators productive across the cluster, making scale-out performance an important complement to advances within the scale-up domain.

Why AI Networking Needs Unified Software

As scale-up and scale-out converge into a broader AI networking architecture, openness at the silicon and system level is only part of the requirement. Customers also need a unified software foundation that can operate these domains as one, providing a common operating system, control plane and orchestration layer across heterogeneous accelerators and fabrics. This software unification can simplify deployment and operations while preserving customer choice at the hardware layer. The Token Fabric™ architecture recently announced by Upscale is one example of this approach.

The Market Is at a Major Inflection Point

The AI networking market is at an inflection point. Accelerator density is increasing. Distributed inference is growing. Agentic workloads are emerging. Custom and merchant XPUs are proliferating. As token factory networks become larger, denser and more diverse, the scale-up fabric will increasingly determine how efficiently AI compute can be deployed and monetized. At the same time, customers are demanding more openness, better utilization, stronger TCO and more flexible deployment models.