Google and Amazon are now both selling AI chips directly to data center operators. Not just cloud rental. The hardware itself. Who buys, and is “neocloud” still the right term for what comes next?
Also today: HBM test bottlenecks, floating datacenters, Hyundai absorbing Boston Dynamics, and really interesting data about the optics market.
Let’s get into it. — Austin & Vik
Quick hits, high signal. Takes from semi industry experts. Sign up for free daily updates!
Be sure to check out the Semi Doped podcast on YouTube or your favorite podcast player.
Amazon to sell Trainium chips externally
Amazon plans to sell its in-house Trainium AI chips directly to data center operators, joining Google, which announced a similar move in April, in offering custom silicon outside their own cloud platforms. The shift positions both hyperscalers as direct alternatives to Nvidia, whose chip business is valued at roughly $5 trillion and which counts Amazon and Google among its largest customers. EE Times reported the development, noting it marks a departure from the prior model in which hyperscaler-designed accelerators were available only through AWS and Google Cloud rentals. (EE Times)
Vik: First Google started selling externally, and now Amazon. Their CUDA equivalent is called Neuron, and the software ecosystem is just as important. It is inevitable that companies making AI chips will feed themselves first, and then start selling to others. Competition is good.
Austin: Who will be buying these TPUs and Trainiums? Surely neoclouds. And the obvious end customers renting those chips will be AI labs. But I’ll be super interested to understand which enterprise end customers sign up. According to the WSJ,
“Among them is Citadel Securities, a longtime Google Cloud customer that recently began using TPUs for some of its research software workloads. Josh Woods, the firm’s chief technology officer, said the company can run key workloads at a 30% lower cost and up to four times as fast with TPUs.”
Relatedly, neocloud originally referred to “new GPU-only cloud” players like CoreWeave, and was meant to differentiate from traditional cloud service providers that also rent CPU, storage, etc like Amazon Web Services, Microsoft Azure, and Google Cloud. Will the term neocloud now also encompass XPUs or more broadly, any AI accelerator? I’d expect said neoclouds to eventually also stand up racks of AI accelerators from startups.
Of course, if customers are demanding XPUs, one has to stop and ask why AWS and GCP don’t just expand their own fleet of XPU rentals and are instead selling them? Some reasons come to mind. The hardware rental business has a longer time-to-payback for the CapEx investment, plus it requires more datacenter buildouts and access to power. Selling the XPUs is a lot simpler and banks a quick ROI…
MangoBoost moves from DPU chips to server racks
MangoBoost, an AI data center DPU startup, is expanding from chip sales into complete server rack systems, CEO of MangoBoost Korea Kim Jang-woo said on June 19. The company is betting that customers increasingly prefer turnkey AI infrastructure over assembling servers themselves. (The Elec)
Bold claim by MangoBoost: “Outside of Nvidia, MangoBoost is the only company capable of delivering everything from chips to software as a unified offering.”
Vik: Delivering optimized hardware is a much bigger deal than people think. MangoBoost’s software platform is called LLM Boost, and they integrate networking, storage, CPUs, and GPUs (from AMD) into racks providing turnkey solutions to deploy AI hardware.
Austin: SemiAnalysis had a nice post discussing the idea of “goodput”, namely:
“In the context of training, goodput is defined as the amount of useful work users can perform on their cluster. Goodput plays on the term throughput to mean that not all throughput is “good”. Lots of training throughput can be “bad” if a GPU fell of the bus, NCCL is stalling, or there is an OOM hiding around the corner during the next checkpoint save.”
While that definition discusses training, obviously there’s tons of money on the table if inference isn’t fully optimized either. DPUs play an underappreciated role in offloading networking tasks from host CPUs and enabling high-throughput data access for KV caches, for example in Nvidia’s STX platform:
If MangoBoost is good at leveraging DPUs, they should be able squeeze more tokens out of the same hardware.
I wonder what they can do with AMD’s MI455X Helios platform, which also has a Pensando DPU in it:
If MangoBoost can squeeze more tokens out of Helios than AMD… well then, they ought to be a great acquisition target.
TSE builds HBM test handler doubling throughput
South Korean semiconductor test component maker TSE is developing a next-generation test handler aimed at doubling inspection throughput for high-bandwidth memory devices, the company said. The handler targets HBM production lines, where rising stack heights and tighter quality requirements have stretched test times and created a bottleneck for memory makers including SK Hynix, Samsung, and Micron. TSE, which supplies probe cards and test sockets to the major DRAM producers, said the new equipment is intended to improve manufacturing efficiency as HBM volumes scale to meet AI accelerator demand. (The Elec)
Austin: The article says the engineering development will be done in 9 months from now or so:
“The companies plan to complete development of both the handler and die socket by March 2027. Once commercialized, the system is expected to alleviate testing bottlenecks that currently constrain throughput in HBM manufacturing lines.”
But when will it actually be deployed at the big three and actually move the needle on amount of tested wafers per hour? Not until 2028 or later?
Imec details ferroelectric memory advances at VLSI Symposium
Belgian research institute imec presented two ferroelectric memory developments at the 2026 IEEE/JSAP Symposium on VLSI Technology & Circuits, targeting capacity, bandwidth, and energy-efficiency constraints in AI memory systems. The work covers low-voltage ferroelectric capacitors and vertically stacked ferroelectric field-effect transistors (FeFETs), positioned as alternatives to conventional DRAM and SRAM for high-density applications. Imec framed the research as a response to AI workloads pushing existing memory architectures beyond their limits. (EE News Europe)
Vik: New memory technologies are one of the long term escape hatches from our current memory shortage predicament. We need a breakthrough now, more than ever.
Austin: imec is so cool. Vik we need to get there and talk to folks. It’s one of the most interesting R&D labs in the world.
Phison Demos 28GB/s Pascari Gen6 SSD
Phison demonstrated a PCIe Gen6 Pascari SSD reaching 28GB/s sequential throughput at Computex 2026, alongside new additions to its Pascari enterprise lineup. The demo doubles the bandwidth of current Gen5 enterprise drives and positions Phison among the first controller vendors to publicly show working Gen6 silicon. The expanded Pascari portfolio targets AI training, inference, and high-capacity storage tiers in data center deployments. (The SSD Review)
Samsung Heavy, Supermicro plan floating AI data centers
Samsung Heavy Industries has partnered with Supermicro and a Greek shipowner to develop 50MW floating AI data centers, according to Hydrogen Central. The vessels are designed to be powered by solid oxide fuel cells running on liquefied natural gas. Samsung Heavy will lead the shipbuilding, while Supermicro supplies the compute infrastructure housed onboard. (Hydrogen Central)
Vik: Ha! First we want to chuck data centers in space, now we want to bob them on the oceans. I bet they could double as AI powered weather stations making sea-faring even safer. Also, BE 0.00%↑ for floating data centers?
Hyundai acquires Boston Dynamics, plans 25,000 Atlas robots for factories
Hyundai Motor Group is paying $325 million to acquire SoftBank’s remaining stake in Boston Dynamics, bringing its ownership to 100%. Hyundai plans to deploy over 25,000 Atlas robots across its global plants, starting with its Metaplant near Savannah, Georgia, by 2028. A dedicated humanoid factory is targeting 10,000 to 30,000 units annually by around 2030. (finance.yahoo.com)
Austin:Well, we lost Boston Dynamics from America’s grip, but at least we’ll see some deployment and operation know-how reside in America. Savannah Bananas home game in 2028 + a Metaplant visit? Hyundai call us!
SK Hynix surpasses Samsung as South Korea’s most valuable company
SK Hynix’s market capitalization reached $1.362 trillion, overtaking Samsung Electronics. This occurred as SK Hynix shares closed 5.6% higher on Monday, lifting its market capitalization to 2.080 quadrillion won. Samsung shares ended 0.1% lower, taking its market value to 2.067 quadrillion won. (wsj.com)
Key Data #1
A single AI training cluster now requires more optical and photonic components than the entire worldwide volume of such components in 2008. (via jwt0625)
Key Data #2
According to Goldman Sachs, here is the breakdown of the BoM of a CPO switch. (via omercheema)
How AI interconnects are built — a nice video by Lightmatter.
Worth a Read
FADU’s blog post on CMX using SSDs is highly informative. The table below explains how the needs for Agentic AI differs from traditional storage.
Sector Watch
Foundry & Logic
Intel and Apple Intel shares rose ~9% after Trump announced Apple agreed to design and manufacture chips in the U.S. with Intel, shifting significant volume from TSMC and validating Intel’s foundry strategy with a lighthouse customer. (TechMeme)
Samsung and AMD Samsung leveraging excess fab capacity to secure AMD as advanced node customer, carving supply share from TSMC and testing pricing power in the foundry oligopoly. (com.tw)
Advanced Packaging
TSMC Advancing CoPoS panel-level packaging with glass substrates to replace CoWoS silicon interposers, targeting mass production Q4 2028 or Q1 2029 to align with Nvidia’s next-gen accelerators. (com.tw)
MPI Corporation Weighing prepayment mechanism to secure probe card capacity amid AI-driven supply constraints, highlighting test infrastructure as binding constraint on accelerator throughput. (DigiTimes)
Networking
Nvidia and optical interconnects Acceleration to 1.6T and 3.2T standards exposing copper limits, driving structural shift toward silicon photonics and MicroLED in Taiwan supply chain. (DigiTimes)
Arista Networks Deploying 1.6T Ethernet switches for AI cluster interconnects, challenging Nvidia’s NVLink dominance with open, disaggregated alternative for hyperscale datacenters. (eenewseurope.com)
Optics & CPO
JX Nippon Expanding Indium Phosphide substrate capacity to meet rising demand for optical interconnects in datacenters, supporting transition toward optical I/O as critical bottleneck for hyperscale AI. (Semiconductor Today)
Power
BWX Technologies Licensed small modular reactor design following activist pressure; NRC reforming licensing framework to accelerate nuclear plant approvals for AI datacenter energy demand. (Bloomberg Tech)
Components
MLCC shortage AI accelerator density driving structural shortage in multi-layer ceramic capacitors, forcing hyperscalers to secure long-term supply contracts before chip deliveries. (South China Morning Post)
Thanks for reading! This post is public so feel free to share it.
