MangoBoost 與 PFN 合作開發異構 AI 推理系統
MangoBoost, PFN Partner on Heterogeneous AI Inference Systems
MangoBoost 與 Preferred Networks(PFN)簽署諒解備忘錄,合作研發 AI 推理基礎設施,並將為新一代「AI Token Factory」架構進行概念驗證。

From left: Takahiro Ogura, division head at PFN, and Jangwoo Kim, CEO of MangoBoost. (Photo: MangoBoost)
MangoBoost said on November 18 that it had signed a memorandum of understanding (MOU) with Preferred Networks (PFN) to collaborate on artificial intelligence (AI) inference infrastructure technologies.
The companies will conduct a proof of concept (PoC) for a next-generation "AI Token Factory" architecture. They will explore ways to improve system utilization by increasing AI inference throughput and reducing latency.
The proposed architecture combines MangoBoost's data processing units (DPUs) and other solutions with PFN's MN-Core AI inference processors and AMD graphics processing units (GPUs). The heterogeneous computing system is designed to optimize computing resources, networking and storage at the system level.
GPUs will handle the prefill stage, in which AI models process input instructions, while PFN's AI inference chips will perform the decode stage, which generates tokens. MangoBoost's DPUs, key-value (KV) cache pooling storage and prefill-decode disaggregation (PDD) solutions will be used to optimize data transfers and KV cache management.
MangoBoost's DPU is a chip dedicated to handling communications, storage input/output and security processing in AI infrastructure. Together with its KV cache solutions, it reduces system overhead associated with data movement and memory management in heterogeneous computing environments.
The companies will initially validate the architecture in a simulation environment. Once samples of the MN-Core L1000 chip become available, they will expand testing to performance validation on actual hardware. They will also consider publishing a joint research paper based on the evaluation results. An MOU signing ceremony will be held at SC26, a supercomputing conference taking place in Chicago from November 15 to 20, local time.
來源:The Elec · thelec.net