SK海力士与英特尔提出共享内存池架构：内部测试首个词元等待由80.76秒降至15.45秒、降幅约81%，11月12日在首尔公布
韩国首尔江南区，SK海力士11月12日在英特尔人工智能峰会首尔站提出共享内存池架构，把大容量内存放进独立的内存机架，让多台图形处理器服务器与中央处理器服务器共同取用，而不是只靠每台机器各自附带的内存。SK海力士副社长朱永杓（音译自英文报道 Joo Young-pyo）在会上介绍了这一方案。
这一架构针对的是超级智能（SI）模型在多轮对话与任务中不断增长的键值缓存。键值缓存是模型在生成过程中保存的工作记忆，上下文越长，占用的内存越大。传统做法把高带宽内存直接附在单个图形处理器上，SK海力士称这种方式存在利用率不高的问题。
连接方式上，机架之间通过远程直接内存访问相连，机架内部使用 Compute Express Link，即 CXL。内存池的容量设想为数十至数百 TB，与图形处理器机架和中央处理器机架并排部署，原有服务器自带的本地内存保留，共享池是在其上叠加的新一层。
SK海力士公布了与伙伴进行的内部测试数字。在多轮超级智能（SI）推理评估中，采用淘汰加重新计算方式时，生成首个词元的平均时间为80.76秒；把缓存放在外置固态硬盘上时为33.56秒；使用共享池中的远程 CXL 动态随机存取存储器时降至15.45秒，较重新计算方式减少约81%，约为其5.2倍的速度。
SK海力士表示，将从系统设计阶段起与英特尔、客户及其他战略伙伴验证这一架构。公布材料同时说明，连接内存池与服务器的规格以及配套软件生态仍需共同制定，基于 CXL 的共享内存在数据中心规模下的延迟与可靠性也仍待验证。
SK hynix and Intel Propose Shared Memory Pool as Internal Test Cuts Time to First Token From 80.76 Seconds to 15.45 Seconds, About 81 Percent, Presented in Seoul on November 12
In Gangnam District, Seoul, South Korea, SK hynix presented a shared memory pool architecture on November 12 at the Intel AI Summit in Seoul. The design places high-capacity memory in dedicated racks that multiple GPU and CPU servers can access together, instead of relying only on memory attached to each individual machine. SK hynix Vice President Joo Young-pyo introduced the proposal at the summit.
The architecture targets the key-value cache that SI models build up during multi-turn conversations and tasks. The KV cache is the working memory a model keeps as generation proceeds, and the longer the context, the more memory it occupies. SK hynix said the conventional approach, in which high-bandwidth memory is attached directly to individual GPUs, leaves inefficiencies in utilization.
For connections, racks are linked by remote direct memory access, or RDMA, while connections within a rack use Compute Express Link, or CXL. The memory pool is envisioned at tens to hundreds of terabytes, installed alongside GPU racks and CPU racks. Local memory attached to existing servers stays in place, with the shared pool added on top.
SK hynix released internal test figures from an evaluation conducted with partners. In the multi-turn SI inference test, average time to first token was 80.76 seconds when eviction and recompute were used, 33.56 seconds when the cache was stored on an external solid-state drive, and 15.45 seconds when remote CXL DRAM from the shared pool was used. That is a reduction of about 81 percent from the recompute method, or roughly 5.2 times faster.
SK hynix said it will validate the architecture with Intel, customers and other strategic partners starting from the system-design phase. The materials also noted that the connection specifications between the memory pool and servers, along with the supporting software ecosystem, still need to be developed together, and that the latency and reliability of CXL-based shared memory at data center scale remain to be validated.
来源：SK海力士
发布：GSM·美国（发布链接）
图片来源：图：SK海力士在英特尔人工智能峰会首尔站介绍共享内存池架构。来源：SK海力士官方发布。 / Photo: SK hynix presenting its shared memory pool architecture at the Intel AI Summit in Seoul. Credit: SK hynix.
GSM Global · 本篇英文版 English edition of this story
