FuriosaAI Explores 16-Card Server Tray Design with Supermicro

FuriosaAI and Supermicro are testing a new 16-card AI server tray design that changes how data moves between computing units. This architecture matters because it reduces the need for data to cross between separate server trays, which directly impacts operational efficiency for large-scale AI clusters. Organizations running large-scale AI workloads may find this design appealing as it simplifies the physical interconnect topology needed to synchronize thousands of chips.

New architecture reduces cross-tray data flow for AI clusters

The hardware concept relies on a dual-layer motherboard structure that stacks two boards in a sandwich configuration. Each motherboard provides eight PCIe slots, allowing the chassis to hold a total of sixteen accelerator cards. FuriosaAI is collaborating with Supermicro on this layout, aiming to address scaling bottlenecks that often challenge dense AI server designs.

Specifications

• XPU Capacity: 16 cards
• Architecture: Dual-layer motherboard sandwich
• PCIe Slots: 8 per motherboard
• NIC to Switch Ratio: 16:1
• Switch Chip Vendor: Astera Labs

A key technical goal of this design is to expand the interconnect domain so that more chips communicate within a single physical unit. The system uses high-channel switch chips from Astera Labs to manage this traffic. This approach allows the server to achieve a 16:1 ratio of network interface cards to switches, which is a significant jump from the traditional 4:1 ratio found in many current setups.

According to reports from THE ELEC, the expanded interconnect domain allows for larger data flow domains within the tray. This reduction in cross-tray data movement helps improve overall operational efficiency by minimizing latency and bandwidth contention between separate chassis. The design is currently in an exploration phase with the partners, meaning the final specifications and availability remain subject to change as the collaboration progresses.