Skymizer has announced the release of its new AI inference decoding accelerator chip, the HTX301. The company stated that this product is designed to speed up the output phase when paired with GPUs handling the pre-fill stage.
New chip integrates six units on single card for local AI models
The hardware integrates six HTX301 chips onto a single PCIe AIC card and includes 384GB of memory per unit. Each card consumes 240W of power while supporting SoC and PCIe AIC deployment form factors.
Skymizer claims the chip is built for local running of models with 700B parameters using its HyperThought software design platform. The architecture relies on the LISA instruction set to focus on bandwidth-intensive workloads that require high compute system utilization.
The company has confirmed availability in the Taiwan region as part of its AI IP enterprise portfolio. Skymizer noted that the launch is intended to accelerate the inference decoding stage for customers in that market.
Discussion
0 comments
Log in to join the thread with a thoughtful take, question, or correction.