This Reddit user has managed to build his own supercomputer cluster under $10,000. It shows you do not need enterprise-grade money for an AI-grade supercomputing system. The build is based around a used NVIDIA GH200 engineering sample chip.
Hobbyist reverse engineers server chip to build 1.5TB memory system
It gives us an idea of how a hobbyist might use second-hand server-grade components to build their own AI model training system. This build is a nice showcase of the ease of access to these huge memory systems today. For the DIY and adventurous researcher or enthusiast, just to configure the booting right can be a challenge.
Specifications
- Cluster Memory: 1500GB
- GH200 Memory: 500GB
- DGX Spark Units: 8
- DGX Spark Memory Contribution: 1TB
- Interconnect Speed: 400Gbps and 200Gbps
This supercomputer cluster build is based on the NVIDIA GH200 which gives you around 500GB of ultra fast unified memory. However, this user has managed to add eight NVIDIA DGX Spark units to give you a further 1TB memory, this way they're able to pool the memory to give you this amazing 1500GB or so of unified memory. This will be a great resource for you to train your own LMs at home. This build has some challenges that he's had to solve, especially getting the server chip to boot by using a Ranger Compute Tray which was acquired through special channels.
He's managed to pool all that memory by combining the GH200 memory with those eight DGX Spark modules. The way they connect everything together is through MikroTik switches for 400Gbps and 200Gbps high speed connections. With the custom interconnects, i.e. MikroTik switches for 400/200Gbps connections, they can connect to the larger memory resource with a low latency.
He's managed to bring together all 1500GB of memory resource. This makes everything a cohesive whole for AI use-cases.
The entire system build with all this hardware comes in under $10,000 USD. That's a bargain compared to commercial options. We've been keeping an eye on GH200.
We covered it earlier here: NVIDIA JUPITER Supercomputer Simulates 104 Qubits. The model of the engineering sample chip for GH200 isn't specified but what matters is this working example. It's possible to get all that memory resource for your personal AI training without having to remortgage your house.



Comments (0)