亚马逊SGLang-Neuron公开可用：同一套SGLang接口可在Trainium芯片上服务模型
亚马逊于2026年11月12日在其机器学习博客宣布，SGLang-Neuron现已公开可用，可经一个AWS深度学习容器获取，该容器由RadixArk与Annapurna Labs合作发布。开发者可用同一套SGLang界面在AWS Trainium芯片上服务模型并生成强化学习rollout，即策略展开样本。
博客称，SGLang-Neuron是一个树外插件，把SGLang与AWS Trainium连接起来，运行于原生PyTorch之上。TorchNeuron把Trainium注册为原生PyTorch设备，模型执行路径使用标准PyTorch算子，并结合以Neuron Kernel Interface编写的内核，该插件还支持预填充与解码分离部署。
博客的演练在一台Trainium3的亚马逊EC2实例上，使用pytorch-inference-sglang-neuronx容器加载openai/gpt-oss-120b模型。演练给出的示例设置包括张量并行与专家并行各为8、键值缓存数据类型为fp8_e4m3、最多同时运行64个请求、总词元上限655360，随后向本地聊天补全接口发送请求。博客说明这些是示例部署设置，并指引开发者查阅项目文档确认受支持的模型、功能与已验证配置，文中未给出吞吐、延迟或价格对比数字。
Amazon Makes SGLang-Neuron Publicly Available: The Same SGLang Interface Now Serves Models on Trainium Chips
Amazon announced on November 12, 2026, on its machine learning blog that SGLang-Neuron is publicly available and can be accessed through an AWS Deep Learning Container released by RadixArk in collaboration with Annapurna Labs. With SGLang-Neuron, developers can serve models and generate reinforcement learning rollouts with SGLang on AWS Trainium using the same SGLang interface they use today.
The post said SGLang-Neuron is an out-of-tree plugin that connects SGLang with AWS Trainium and runs on native PyTorch. TorchNeuron registers Trainium as a native PyTorch device, the model execution path uses standard PyTorch operations together with kernels written with the Neuron Kernel Interface, and the plugin also supports prefill and decode disaggregation.
The walkthrough uses a Trainium3 Amazon EC2 instance and a pytorch-inference-sglang-neuronx container to load the openai/gpt-oss-120b model. The example settings include tensor parallel size 8 and expert parallel size 8, an fp8_e4m3 key-value cache data type, up to 64 running requests and a maximum of 655,360 total tokens, followed by a request to a local chat completions endpoint. The post presents these as example deployment settings, directs developers to the project documentation for supported models, features and validated configurations, and reports no throughput, latency or price comparisons.
来源：亚马逊
发布：GSM·美国（发布链接）
图片来源：亚马逊 AWS 位于俄勒冈州的数据中心园区航拍。／Aerial view of an Amazon AWS data center campus in Oregon.（Tedder，CC BY-SA 4.0，Wikimedia Commons）
GSM Global · 本篇英文版 English edition of this story
