Overcoming AI Hardware Bottlenecks Through Vertical Systems Integration

The AI hardware bottleneck is not just a supply issue; it is a design philosophy problem. While incumbents optimize for general-purpose versatility, Etched founders Gavin Uberti and Rob Wachen argue that the industry is trapped by outdated constraints, such as designing for temperature ranges that never exist in data centers. By rejecting the buffer mindset of the semiconductor industry and integrating vertically from silicon to rack, they show that the current shortage is partially a result of inefficient, retrofit architectures. For the reader, this conversation provides a blueprint for identifying where conventional wisdom creates artificial ceilings. The advantage lies in recognizing that when the underlying physics of a market change, incumbents are often the last to realize their own constraints are no longer binding.

The Hidden Cost of General Purpose Design

The semiconductor industry has historically prioritized general-purpose utility, building chips that function in everything from IoT devices to data centers. Etched argues this creates a massive performance tax. By designing specifically for inference, they identified that GPU architectures are burdened by buffer requirements, such as operating at freezing temperatures, which are irrelevant in modern data centers.

"The entire semiconductors and data center industry is built on buffer. And what I mean by that is every part of the stack from the EDA tools to the power modules to the circuit boards to the chip design and standard cells, everything is built to be general purpose for everything."

-- Gavin Uberti

This insight reveals a downstream consequence: because incumbents optimize for broad compatibility, they fail to reach the voltage efficiency required for massive inference scale. By ignoring these default constraints, Etched designed chips that run at less than half the voltage of traditional AI hardware, creating a performance gap that compounds as data centers grow.

The 40-Day Payoff: Why Vertical Integration Wins

Most teams treat the rack as an afterthought, focusing solely on the chip. Etched systems-level thinking treats the entire rack, including power delivery, cooling, and interconnects, as the product. This creates a competitive moat built on speed of iteration. By building thermal-mock chips and running software stacks on FPGA clusters before their silicon arrived, they reduced the time-to-production from ten months to 40 days.

"The best ability is availability. If I have a thousand chips a day, someone's going to use them and we need to build a chip that's not just way better than what's been built before but it needs to be available at many Gigawatt scale."

-- Rob Wachen

This provides a systems-thinking lesson: immediate pain, such as the cost and complexity of building a rack, creates a lasting advantage by allowing deployment at scale while others are still debugging.

When the System Responds to Your Architecture

A non-obvious dynamic revealed in the conversation is the cluster scale memory problem. Most AI chips are fast in isolation but slow when communicating across a cluster due to high latency between chips. By building a custom interconnect stack, Etched effectively treats the memory of an entire cluster as a single pool. This changes the incentive structure for model builders: they no longer need to compromise model size to fit into a single GPU memory. Over time, this shifts the industry away from handcrafted token generation toward a model where economies of scale apply to intelligence just as they do to hardware manufacturing.

Key Action Items

• Audit Your Default Constraints: Identify where your industry or team accepts limitations, like needing to support all legacy formats, that no longer serve your specific use case. (Immediate)
• Prioritize Production as Product: If you are building complex hardware or systems, stop treating the final assembly as a separate phase. Parallelize your software and hardware development using mock-ups and FPGAs. (Over the next quarter)
• Adopt Project-Based Recruiting: When seeking elite talent, stop selling the company vision and start mapping the hardest technical problems in the world. Target the individuals who actually solved those problems. (Ongoing)
• Embrace Hard Vertical Integration: Assess which parts of your stack are currently outsourced to vendors that slow your iteration speed. Bring the most critical bottlenecks in-house to reclaim control over your schedule. (12-18 months)
• Optimize for Wall-Clock Time: Stop measuring efficiency by theoretical metrics like peak FLOPS and start measuring it by the time it takes to iterate on a full model run. (Immediate)
• Assume Solvability: When faced with impossible technical hurdles, force the team to operate under the assumption that a solution exists. This prevents the paralysis that causes teams to abandon projects prematurely. (Ongoing)