The AI Boom Is Fueling a Need for Speed in Chip Networking
The new era of Silicon Valley is fundamentally driven by networking – and not the kind of professional connections one cultivates on LinkedIn. As the global technology industry funnels unprecedented billions into the construction and expansion of AI data centers, a quiet revolution is unfolding in the foundational technology that connects chips to other chips, and entire server racks to other server racks. This surge in investment and innovation underscores a critical realization: the sheer computational might of AI models is only as effective as the speed and efficiency with which data can move through the underlying hardware infrastructure.
Networking technology, in its most basic form, has been an indispensable component of computing since its inception, critically linking mainframes to facilitate data sharing and distributed processing. In the complex world of semiconductors, networking permeates almost every layer of the technological stack. This ranges from the microscopic interconnects that enable transistors to communicate within a single chip, to the expansive external connections that bind together boxes or racks of sophisticated processing units within a data center. Historically, while essential, networking was often viewed as a somewhat utilitarian and less glamorous aspect of tech innovation compared to the dazzling advancements in processor power or software algorithms. However, the insatiable demands of artificial intelligence have thrust it into the spotlight, making it a pivotal battleground for innovation and investment.
The burgeoning AI boom has fundamentally altered this perception. The massive amounts of digital information required to train and run sophisticated AI models, coupled with the need for near-instantaneous processing, are pushing the limits of traditional interconnect technologies. This is where the quest for new networking approaches, specifically designed to accelerate high-speed data flow through vast data centers, becomes paramount. Established chip giants like Nvidia, Broadcom, and Marvell, already possessing well-established networking bona fides, are intensifying their efforts. But the real excitement, and significant venture capital, is flowing into deep-tech startups that are pioneering radical new solutions, particularly those leveraging optical technology. Companies such as Lightmatter, Celestial AI, and PsiQuantum are at the forefront, promising to unlock unprecedented speeds and efficiencies using light rather than electrons.
Optical technology, or photonics, is experiencing a remarkable coming-of-age moment. For decades, specifically about 25 years, this technology was largely dismissed within the industry as "lame, expensive, and marginally useful," according to Pete Shadbolt, cofounder and chief scientific officer of PsiQuantum. The prevailing wisdom favored incremental improvements in electrical signaling. However, the sheer scale and intensity of the AI boom have dramatically reignited interest in photonics, transforming it from a niche academic pursuit into a viable, and increasingly essential, commercial solution. The fundamental advantage lies in light’s ability to transmit data at much higher speeds and with significantly less energy loss compared to electrons, which face resistance and generate heat.
This renewed interest isn’t just theoretical; it’s translating into substantial financial backing. A new wave of venture capitalists and institutional investors, eager to identify the next major wave of chip innovation or secure lucrative acquisition targets, are channeling billions into these pioneering startups. Their conviction stems from a growing consensus that traditional interconnect technology, which relies on the movement of electrons through copper wires, simply cannot keep pace with the escalating need for high-bandwidth, low-latency AI workloads. The sheer volume and velocity of data now required by AI systems are creating bottlenecks that traditional electrical pathways struggle to overcome without significant energy consumption and heat generation.
Ben Bajarin, a seasoned tech analyst and CEO of the research firm Creative Strategies, aptly summarizes this paradigm shift: "If you look back historically, networking was really boring to cover, because it was switching packets of bits. Now, because of AI, it’s having to move fairly robust workloads, and that’s why you’re seeing innovation around speed." This transformation from a backend utility to a critical performance differentiator is what makes the current era of networking so dynamic and vital.
Big Chip Energy: The Incumbents’ Strategic Plays
Credit for foresight in recognizing the strategic importance of networking largely goes to Nvidia. Years ago, the GPU giant made two pivotal acquisitions that solidified its vertically-integrated approach to AI infrastructure. In 2020, Nvidia invested nearly $7 billion to acquire Mellanox Technologies, an Israeli firm renowned for its high-speed networking solutions tailored for servers and data centers. This move immediately bolstered Nvidia’s capabilities in InfiniBand and Ethernet, crucial technologies for connecting their powerful GPUs. Shortly thereafter, Nvidia acquired Cumulus Networks, a company focused on Linux-based software systems for computer networking. These strategic purchases marked a significant turning point for Nvidia, which correctly wagered that its GPUs, with their unparalleled parallel-computing capabilities, would achieve far greater power and efficiency when clustered together in data centers, interconnected by a robust, high-speed, and software-defined network. This integrated approach allowed Nvidia to offer a comprehensive, optimized stack, from the chip to the network, tailored for AI.
While Nvidia dominates in these vertically-integrated GPU stacks, Broadcom has carved out a formidable position as a key player in custom chip accelerators and high-speed networking technology. The colossal $1.7 trillion company collaborates closely with hyperscalers like Google and Meta, and more recently, with OpenAI, on developing specialized chips for their sprawling data centers. Broadcom is also at the forefront of silicon photonics research and deployment. Last month, Reuters reported that Broadcom is preparing to launch a new networking chip, codenamed Thor Ultra. This chip is designed to provide a "critical link between an AI system and the rest of the data center," highlighting Broadcom’s ongoing commitment to pushing the boundaries of data center interconnectivity in the AI era.
The strategic importance of networking is further underscored by recent corporate maneuvers. On its earnings call last week, semiconductor design giant ARM announced its plans to acquire DreamBig for $265 million. DreamBig specializes in AI chiplets—small, modular circuits designed to be packaged together in larger chip systems—in partnership with Samsung. ARM CEO Rene Haas emphasized that DreamBig possesses "interesting intellectual property… which [is] very key for scale-up and scale-out networking." This means DreamBig’s technology is crucial for efficiently connecting components and sending data both within a single chip cluster (scale-up) and between multiple racks of chips (scale-out), which are fundamental requirements for advanced AI architectures.
Light On: The Deep-Tech Disruptors
The scale of AI’s computational appetite is staggering. Nick Harris, CEO of Lightmatter, a company at the cutting edge of optical computing, has highlighted that the amount of computing power required by AI now doubles approximately every three months – a pace far exceeding the traditional exponential growth predicted by Moore’s Law. As computer chips continue to grow in size and complexity, pushing the limits of silicon manufacturing, Harris notes, "Whenever you’re at the state of the art of the biggest chips you can build, all performance after that comes from linking the chips together." This statement encapsulates the core problem that Lightmatter and similar startups are trying to solve.
Lightmatter’s approach is genuinely cutting-edge and departs significantly from traditional electrical networking technology. The company designs and builds silicon photonics solutions that link chips together using light. It claims to have developed the world’s fastest photonic engine for AI chips, which is essentially a 3D stack of silicon connected by light-based interconnect technology. This innovative architecture promises to drastically reduce latency and increase bandwidth, enabling AI models to perform at unprecedented speeds. The market has taken notice: Lightmatter has successfully raised over $500 million in the past two years from prominent investors like GV and T. Rowe Price, with its valuation soaring to $4.4 billion last year.
"The future of computing is really about light," Harris asserts, articulating the company’s vision. While acknowledging the continued importance of electronics and software, he stresses that "at this level of computing you need new ideas, and a big chunk of the new frontier of computers involves light." This sentiment is echoed across the deep-tech landscape.
Another startup that has garnered significant attention and investment for its optical interconnect technology is Celestial AI. Earlier this year, Celestial AI secured $250 million in funding from a syndicate of high-profile investors including Fidelity Management, BlackRock, Tiger Global Management, Temasek, and chip giant AMD. The company’s credibility was further boosted by the recent addition of Intel CEO Lip-Bu Tan to its board of directors, signaling a strong belief in the potential of its technology. Similarly, PsiQuantum, a company focused on using optical technology to build chips for quantum computers, raised an astounding $1 billion in September from investors such as BlackRock, Ribbit Capital, and Nvidia’s venture arm, NVentures. This massive infusion of capital has propelled PsiQuantum’s valuation to an impressive $7 billion, underscoring the immense confidence in the long-term prospects of optical computing, even for the nascent field of quantum computing.
Challenges and the Long Road Ahead
Despite the undeniable promise and the surge in investment, optical networking technology is not without its hurdles. It remains inherently expensive to build and deploy, requiring highly specialized manufacturing processes and equipment. Furthermore, a significant technical challenge lies in its ability to seamlessly "plug in" and integrate with existing electrical systems, which still form the backbone of most computing infrastructure. Bridging this gap between optical and electrical domains efficiently is crucial for widespread adoption.
Ben Bajarin points out that established companies like Broadcom and Marvell possess a distinct advantage. They have the deep expertise, extensive resources, and established relationships to work closely with hyperscalers, catering to their highly specific and customized needs in both AI data center chips and networking solutions. Regardless of whether these giants are employing traditional electrical networking tech or venturing into cutting-edge photonics, they have the proven ability to scale their solutions to meet massive demands. "Networking is the thing that makes computers function," Bajarin observes, "but it just feels like the industry is moving towards much more customization, which might be harder for the small guys."
This doesn’t diminish the value of the intellectual property being developed by these innovative upstarts. The demand for faster data speeds, and consequently, superior networking technology, is only going to intensify as AI continues its exponential growth. However, Bajarin cautions that the ultimate payoff for these experimental startups might still be years down the line. "We all believe there will be a world with a photonics future," he concludes, "but it’s still a ways away." The race for speed in chip networking, fueled by the relentless demands of AI, is just beginning, promising a future where light may well become the ultimate currency of data transfer.
