文档(金鹏): 2026-08-06 章节 48 篇文章摘要归档
- 46 篇原文+摘要双文件归档（按 来源/作者 分层，复用本地归档 20 篇+新抓取 26 篇） - 即梦生成 9 组主题配图（大图+列表缩略图）存入 知识/金鹏/20260806/ - 章节重组为 9 个主题分组并挂接摘要引用
This commit is contained in:
@@ -0,0 +1,219 @@
|
||||
# How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
|
||||
|
||||
> **来源**：arXiv
|
||||
> **作者**：Hanjiang Wu, Abhimanyu Rajeshkumar Bambhaniya, Sarbartha Banerjee, Tuhin Khare, Sudarshan Srinivasan, Suvinay Subramanian（等，共 12 位）
|
||||
> **发布日期**：2026-05-27
|
||||
> **原文链接**：https://arxiv.org/abs/2605.28302
|
||||
|
||||
---
|
||||
|
||||
## 论文元数据
|
||||
|
||||
- **arXiv ID**：2605.28302
|
||||
- **学科分类**：Distributed, Parallel, and Cluster Computing (cs.DC)
|
||||
- **作者机构**：佐治亚理工学院（Georgia Institute of Technology）、Intel、Google、Google DeepMind、Infravana
|
||||
- **提交历史**：v1: 2026-05-27
|
||||
- **DOI**：https://doi.org/10.48550/arXiv.2605.28302
|
||||
|
||||
---
|
||||
|
||||
Hanjiang Wu1 Abhimanyu Rajeshkumar Bambhaniya1,5 Sarbartha Banerjee1
|
||||
|
||||
Tuhin Khare1 Sudarshan Srinivasan2 Suvinay Subramanian3
|
||||
|
||||
Souvik Kundu2 Madhu Kumar2 Midhilesh Elavazhagan2
|
||||
|
||||
William Won1 Amir Yazdanbakhsh4 Tushar Krishna1,5
|
||||
|
||||
1Georgia Institute of Technology 2Intel 3Google 4Google DeepMind 5Infravana
|
||||
|
||||
## 摘要（Abstract）
|
||||
|
||||
Modern large language model (LLM) inference has progressively disaggregated to keep pace with growing model sizes and tight TTFT and TPOT service-level objectives: from chunked-prefill aggregation, to prefill–decode (P/D) disaggregation, and most recently to operator-level Attention–FFN Disaggregation (AFD). This trend is especially important for mixture-of-experts (MoE) models, where memory-bound attention, compute-intensive expert FFNs, and MoE dispatch/combine communication create distinct resource demands across the serving pipeline. AFD further exposes this heterogeneity by placing attention and MoE-FFN execution on separate GPU groups. Each level of disaggregation deepens the scheduling design space across workload characteristics, resource allocation, and interconnect topology, leaving open the central question: When does each level of disaggregation actually pay off? We systematically characterize this trade-off for MoE inference across realistic workload use cases defined by input/output sequence lengths, prefix-KV reuse, and per-user latency constraints. Using chunked-prefill and P/D disaggregation as strong baselines, we study the benefits and limits of AFD at scale through a framework that fuses rich on-device kernel measurements with high-fidelity network simulation. Our findings deliver a practical map of when and where deeper disaggregation pays off for MoE serving at scale. Under strict TTFT/TPOT SLOs, AFD sustains around 4k tokens/s of system throughput on DeepSeek-V3.2 across chat, coding, and agentic-coding workloads, regimes in which non-AFD deployments are infeasible. Our design and analysis further distill concrete takeaways for jointly optimizing system throughput and user interactivity, including how to partition attention and FFN across GPUs as a function of workload and model architecture, providing design principles for current rack- and cluster-scale deployments as well as future disaggregated AI infrastructure.
|
||||
|
||||
## 1 引言（Introduction）
|
||||
|
||||
The rapid scaling of agentic large language models (LLMs) has enabled AI systems to perform increasingly complex tasks, including multi‑turn reasoning, code generation, and autonomous decision‑making. As these models grow to hundreds of billions of parameters and operate on long input contexts, their deployment places unprecedented pressure on inference infrastructure, particularly due to the expanding KV‑cache footprint and the need to scale across multiple compute nodes. At the same time, emerging agentic workloads and model architectures exhibit increasing compute characteristic heterogeneity, exposing limitations in existing LLM serving paradigms that struggle to simultaneously achieve high performance, efficiency, and scalability.
|
||||
|
||||
A central challenge stems from the heterogeneous execution characteristics of different components within modern LLM architectures, which impose conflicting demands on compute, memory bandwidth, and communication resources. Prior systems mitigate these effects through batching and scheduling techniques such as chunked prefill Agrawal et al. (2023) and continuous batching Yu et al. (2022), pipelining, or coarse‑grained prefill–decode (P/D) disaggregation Zhong et al. (2024); Patel et al. (2023). While effective at reducing phase‑level interference, these approaches implicitly treat the model as a monolithic execution unit, obscuring fine‑grained trade‑offs that become dominant at scale.
|
||||
|
||||
|
||||
|
||||
Figure 1: AIC++ Framework Overview. AIC++ takes model architecture, hardware configuration, and workload constraints as inputs and performs design-space exploration to identify optimal scheduling across token-level parallelism—data (DP), sequence (SP), tensor (TP), pipeline (PP), and expert parallelism (EP)—as well as phase-level prefill/decode (P/D) and operator-level attention–FFN disaggregation (AFD) [C1]. The framework leverages AIConfigurator to model heterogeneous GPU clusters (GPUA,GPUBGPU_{A},GPU_{B}) interconnected via scale-up (NVLink) and scale-out (InfiniBand) fabrics simulated with AstraSim [C2]. Based on this analysis, attention (PA,DA1,DA2P_{A},D_{A,D_{A) and FFN (PF,DF1,DF2P_{F},D_{F,D_{F) operators are placed on specific GPUs to maximize overall system throughput [C3].
|
||||
|
||||
Recent studies such as MegaScale-Infer Zhu et al. (2025) show that coarse-grained inference abstractions break down for large heterogeneous models, especially MoE architectures, where substantial compute heterogeneity exists within each Transformer block. Attention variants such as MHA Ashish (2017), GQA Hudson and Manning (2019), and MLA DeepSeek-AI et al. (2024) are largely memory-bound due to KV-cache access and data movement, whereas FFNs are compute-bound and dominated by dense GEMMs. Although MegaScale-Infer highlights inefficiencies from attention–FFN heterogeneity, it remains unclear how Attention–FFN Disaggregation (AFD) composes with existing parallelism strategies, prefill–decode (P/D) disaggregation, and different attention architectures.
|
||||
|
||||
Beyond architectural heterogeneity, AFD effectiveness also depends on workload characteristics such as input/output sequence length (ISL/OSL), prefix length, and system load (tokens/s/user). These factors directly affect scheduling decisions and determine when disaggregation is beneficial. Understanding these trade-offs is critical for current cluster-scale LLM serving and for future disaggregated inference platforms, including NVIDIA Groq 3 LPX NVIDIA (2026a) and Intel/SambaNova-style systems Intel (2026).
|
||||
|
||||
In this paper, we present a systematic study of AFD for LLM inference across diverse application domains. We analyze efficient deployment across three dimensions: distributed parallelism, including tensor, data, pipeline, sequence, and expert parallelism; phase-level P/D disaggregation; and operator-level attention–FFN disaggregation, as shown in Figure˜1(C1). We formulate scheduling selection as a design-space exploration (DSE) problem that captures operator-level compute heterogeneity and inter-node communication costs.
|
||||
|
||||
To support this study, we develop AIConfigurator++ (AIC++), a co-design framework that combines operator-level compute modeling from NVIDIA AIConfigurator Xu et al. (2026) with distributed communication modeling using AstraSim Rashidi et al. (2020). Grounded in a customized vLLM-based AFD prototype Kwon et al. (2023), AIC++ integrates kernel measurements with system-level simulation to evaluate compute–communication trade-offs across scheduling strategies, as illustrated in Figure˜1(C2).
|
||||
|
||||
We evaluate chatbot, coding, and agentic workloads using DeepSeek-V3.2, GPT-OSS-120B, Nemotron3-120B, and Qwen3-235B, covering MLA, GQA, sparse attention, and Mamba-based architectures. By varying ISL, OSL, prefix length, and system load, our framework identifies when AFD is beneficial, how micro-batching and operator placement improve compute–communication overlap, and which scheduling strategy maximizes throughput and interactivity under SLO constraints. Importantly, our analysis translates workload and model characteristics into concrete attention-to-FFN GPU ratios, providing actionable guidance for cluster-scale deployment.
|
||||
|
||||
More broadly, our findings suggest that as LLM workloads become increasingly heterogeneous, system optimization must move beyond coarse-grained placement toward operator-level disaggregation. Modeling-driven frameworks such as AIC++ can guide the co-design of future heterogeneous inference platforms. From Figure˜2, we observe that under stringent SLOs for TTFT and TPOT on DeepSeek-V3.2, AFD is able to achieve 4k tokens/s system throughput while the non-AFD deployment is infeasible to run. Through exhaustive design-space exploration to analyze the best deployment strategy on a cluster of 128 B200 GPUs (Section˜4), we see that AFD is not the best option when system throughput is the main target compared to the pure P/D disaggregation and chunked prefill. But our analysis that dynamically searches the attention-to-FFN ratios finds that it will always achieve the best latency and user interactivity with the AFD-specific microbatch overlapping technique that best utilizes the compute and communication resources. Finally, we present a case study highlighting AFD’s memory-segmentation benefit: by placing most model weights on FFN GPUs, AFD leaves more memory on attention GPUs for KV cache, enabling higher throughput under the same memory constraint.
|
||||
|
||||
Our key contributions are:
|
||||
|
||||
- C1:
|
||||
|
||||
Multi-dimensional DSE: We jointly optimize LLM serving across token-level parallelism, P/D disaggregation, and AFD, and translate workload characteristics into concrete attention-to-FFN GPU ratios.
|
||||
- C2:
|
||||
|
||||
System modeling for disaggregated architectures: We build AIC++, which models compute and data-transfer costs for scaled-out disaggregated inference systems.
|
||||
- C3:
|
||||
|
||||
Optimal AFD operator placement: We demonstrate that careful placement of attention and FFN operators is critical for effective AFD deployment.
|
||||
|
||||
|
||||
|
||||
Figure 2: System throughput at the best feasible deployment on DeepSeek-V3.2 (128× B200 with trtllm backend) under strict SLOs (TTFT<50/100/150 ms for Chat/Coding/Agentic Coding; TPOT caps 15 ms). Red cross marks infeasibility — the auto-con
|
||||
|
||||
advisory chatbots - typically exhibit large context length and small ISLs.
|
||||
These workloads favor aggregated deployments, as frequent cross‑node data transfers in disaggregated systems incur high communication overheads.
|
||||
In contrast, workloads with long ISLs, common in code‑completion and code‑generation tasks, benefit more from asynchronous function decomposition (AFD), which introduces an additional dimension to the scheduling design space.
|
||||
In this work, we characterize different workloads with different ISL, OSL and context length to find the Pareto-optimal scheduling that balances system throughput and user interactivity under SLO constraints (detailed in section 4).
|
||||
|
||||
### 2.2 Finding the optimal scheduling with disaggregated architecture
|
||||
|
||||
The emergence of disaggregated architectures with heterogeneous compute units within a node—such as NVIDIA Groq‑3 LPX, Rubin CPX Nvidia (2024), and Intel SambaNova—has made fine‑grained AFD an effective scheduling strategy despite the increased communication volume. High‑bandwidth scale‑up interconnects in these heterogeneous clusters enable memory‑bound attention operators to execute on memory‑rich devices, while compute‑intensive FFN blocks benefit from accelerators optimized for high arithmetic throughput.
|
||||
|
||||
System modeling requirements of disaggregated architecture:
|
||||
Exploring a large design space by evaluating hundreds of candidate configurations is prohibitively expensive for large‑scale disaggregated systems as observed by Miao et al. (2022); Zheng et al. (2022).
|
||||
Hence, accurate system modeling of compute and communication infrastructure is necessary for evaluation.
|
||||
AIConfigurator Xu et al. (2026) provides a compute modeling framework to estimate the compute and memory bandwidth of a wide range of modern GPUs.
|
||||
However, we additionally need accurate estimation of the data-transfer cost for fine-grained operator-level AFD.
|
||||
To address this, we augment AIConfigurator with the AstraSim Rashidi et al. (2020) network simulator to build AIC++,
|
||||
a disaggregated modeling framework that captures the compute-communication effects, maximizing their overlap for fine-grained AFD.
|
||||
Moreover, we evaluate the impact of AFD in heterogeneous scale-up AIC++ deployments.
|
||||
|
||||
|
||||
|
||||
Figure 3: (a) Runtime breakdown and (b) memory component breakdown for different model architectures running prompts with different context lengths.
|
||||
|
||||
Precise placement of attention and FFN operators:
|
||||
Fine‑grained data transfers introduced by AFD can lead to significant interconnect congestion if attention and FFN operators are placed arbitrarily across a disaggregated infrastructure. Consequently, effective scheduling of fine‑grained operator disaggregation requires jointly reasoning about compute affinity and data‑movement costs—a challenge explicitly addressed in our work. In particular, optimal operator placement must account for compute–communication overlap by co‑locating operators with high data‑exchange intensity on nearby or tightly coupled compute nodes. Going beyond prior approaches that primarily model heterogeneous architectures, our work jointly optimizes operator placement by explicitly considering interconnect congestion, identifying optimal mappings of attention and FFN operators that maximize overall system efficiency.
|
||||
|
||||
## 3 AIC++ 框架总览
|
||||
|
||||
In this section, we present AIC++, a framework for exploring the design space of AFD‑enabled MoE serving at scale. AIC++ combines AIConfigurator Xu et al. (2026) for accelerator‑level performance modeling with ASTRA‑Sim Rashidi et al. (2020) for high‑fidelity network simulation, enabling evaluation across attention architectures and application domains. While AIConfigurator accurately models modern LLM kernels, it does not capture AFD‑specific communication paths; we extend it by partitioning execution into attention and MoE‑FFN phases and binding each phase to an independent GPU backend, faithfully modeling runtime and memory behavior on disaggregated devices.
|
||||
|
||||
### 3.1 AFD design for MoE architectures
|
||||
|
||||
As shown in Figure˜4, the transition between the Attention and MoE–FFN phases is mediated by the MoE-Dispatch and MoE-Combine communication operators. In non-AFD deployments, these operators involve matched source and destination counts, as communication is confined to GPU ranks participating in expert parallelism (EP). Under AFD, attention and MoE–FFN phases execute on physically disaggregated GPUs, resulting in asymmetric fan-out or fan-in communication patterns depending on the scheduling strategy. For example, while an MoE model deployed on 8 GPUs with EP=8 exchanges tokens among all 8 GPUs, an AFD configuration with 2 attention GPUs and 6 FFN GPUs induces a fan-out pattern, requiring tokens generated by the attention GPUs to be distributed to a larger set of FFN GPUs.
|
||||
|
||||
AIC++ models these interactions using AstraSim to simulate scale-up and
|
||||
scale-out communication at packet granularity. By coupling AstraSim with
|
||||
AIConfigurator, AIC++ enforces communication-dependent execution, capturing
|
||||
contention, GPU utilization, and data-transfer efficiency under dynamic workloads.
|
||||
|
||||
Concretely, each transformer layer incurs two cross-AFD transfers. In the all-pairs mode used for the main results, the AFD worker establishes pairwise communicators between every attention rank and every FFN rank, so tokens generated by a single attention rank may be routed to any FFN rank hosting the selected experts.
|
||||
A2F/A2E (MoE-Dispatch) transfers post-attention hidden states, token ids, and per-expert routing metadata from attention ranks
|
||||
to FFN ranks hosting the selected experts, where each FFN rank filters tokens according to its hosted experts. Due to the top-kk routing and token dispatch, A2F exhibits a fan-out, making FFN-side ingress congestion dominant. F2A/E2A (MoE-Combine) aggregates expert outputs and
|
||||
returns a single reduced hidden state per token to the originating attention rank,
|
||||
forming a fan-in transfer in which attention-side ingress becomes the bottleneck. AIC++ expands both transfers into a full bipartite
|
||||
traffic matrix and feeds them into AstraSim’s tiered, congestion-aware network model,
|
||||
capturing contention when A2F egress and F2A ingress overlap on full-duplex
|
||||
interconnects. This part of the implementation with its communication pattern is based on our prototype implementation (refer to subsection 6.1).
|
||||
|
||||
### 3.2 Batch Overlap (BO) in AFD
|
||||
|
||||
As shown in Figure˜4 (left), AFD decomposes execution into four stages mapped to separate compute or communication resources: attention computation on the attention GPUs, dispatch of post-attention hidden states, token ids, and per-expert routing metadata to experts, MoE-FFN computation on the FFN GPUs, and aggregation/transfer of expert outputs back to the attention side for the next layer.
|
||||
|
||||
The return communication may share the same channel on half-duplex links, or proceed concurrently over a dedicated channel on full-duplex networks, enabling the pipelined execution shown in Figure˜4(B). Since modern datacenter GPU deployments commonly provide full-duplex interconnects such as NVLink and InfiniBand, AFD can exploit compute-communication overlap across GPUs and the network fabric. Accordingly, in the following discussion, we assume an AFD implementation with four-stage micro-batch overlap enabled. In contrast, under aggregated execution, all devices jointly execute all stages as a single group, as shown in Figure˜4(A).
|
||||
|
||||
To model batch overlap behavior in AIC++, we partition the per-step token budget
|
||||
Tbudget=batch_size×ISLT_{{budget}}=it{batch\_size}it{ISL} into MM microbatches.
|
||||
The number of microbatches MM is chosen to match the effective pipeline depth—three
|
||||
for half-duplex links and four for full-duplex links. For each microbatch,
|
||||
AIC++ queries AIConfigurator’s empirically measured GPU-cluster cost database
|
||||
to obtain the attention and FFN execution costs for a microbatch size of
|
||||
Tbudget/MT_{{budget}}/M, thereby preserving the nonlinear scaling behavior of small
|
||||
GEMMs and collective operations.
|
||||
|
||||
Let sis_{i} denote the measured per-microbatch execution cost of pipeline stage ii,
|
||||
aggregated across all LL transformer layers, and let
|
||||
smax=maxisis_{}=_{i}s_{i} be the bottleneck stage. Under steady-state cross-layer
|
||||
pipelining, the bottleneck stage processes all MM microbatches back-to-back,
|
||||
whereas each non-bottleneck stage incurs only a one-time pipeline fill and drain
|
||||
overhead of si/Ls_{i}/L. This cost is amortized over the full execution rather than
|
||||
charged per layer, consistent with the cross-layer scheduling strategies used by
|
||||
MegaScale-Infer and Step-Fun. The resulting end-to-end pipelined latency is shown in Equation˜1.
|
||||
|
||||
| | | | |
|
||||
| --- | --- | --- | --- |
|
||||
| | tpipe=M⋅smax+∑i:si≠smaxsiL.t_{{pipe}}=M s_{}+_{i:s_{i} s_{}}{s_{i}}{L}. | | (1) |
|
||||
|
||||
The first term captures the steady-state throughput dictated by the bottleneck
|
||||
resource, while the second term accounts for pipeline fill and drain bubbles
|
||||
introduced by non-bottleneck stages. We apply this formulation to both prefill and
|
||||
decode phases, enabling AIC++ to accurately reason about computation and
|
||||
communication costs under batch-overlapped execution.
|
||||
|
||||
|
||||
|
||||
Figure 4: Mapping AFD to 4 different stages in the model execution
|
||||
|
||||
### 3.3 Location-aware GPU placement
|
||||
|
||||
Because AstraSim labels each GPU with its physical position—node, scale-up domain, and link tier—and resolves congestion at packet granularity, AIC++ additionally enables a study of optimal GPU placement under combined AFD and P/D disaggregation. The policy is frequency-driven: the most frequent intra-layer A2F/F2A traffic (𝒪(layer){O}({layer}) per request) is bound to the highest-bandwidth scale-up domain (intra-node NVLink), while the less frequent inter-node KV-cache transfer (𝒪(1){O}(1) per request) is deferred to the scale-out domain (InfiniBand). This grouping co-locates GPUs with high communication affinity onto faster interconnects, avoiding contention on slower links. The detailed placement study—including segregated vs. paired P/D layouts and the resulting KV-transfer speed-up—is given in Appendix 6.2.
|
||||
|
||||
We also consider asymmetric prefill and decode configurations reflecting their distinct compute characteristics. AIC++ enables such asymmetry through independent prefill/decode scheduling and asymmetric attention and FFN worker allocation, while the network simulator co‑locates operators to reduce interconnect congestion and improve efficiency.
|
||||
|
||||
## 4 Evaluation
|
||||
|
||||
### 4.1 Design-Space Exploration (DSE) of different serving strategies
|
||||
|
||||
#### 4.1.1 Input workloads
|
||||
|
||||
To demonstrate the applicability of AFD, we evaluate a diverse set of model architectures spanning multiple application domains, as summarized in Table˜1. Specifically, we consider recent production‑deployed models including Qwen3‑235B, commonly used for chatbot workloads; GPT‑OSS‑120B, targeting medium‑scale reasoning tasks; DeepSeek‑V3.2, designed for reasoning‑intensive and agentic coding workflows; and Nemotron3‑120B, optimized for large‑context applications such as RAG serving Nvidia (2026). The corresponding prefix sizes, ISL, OSL, and architectural characteristics for each workload are detailed in Table˜1.
|
||||
|
||||
Table 1: Representative application workloads and model architectures.
|
||||
|
||||
Application Workload
|
||||
|
||||
| | | | |
|
||||
| --- | --- | --- | --- |
|
||||
| Use Case | Prefix | ISL | OSL |
|
||||
| Chat | 4096 | 512 | 256 |
|
||||
| Coding | 2048 | 4096 | 1024 |
|
||||
| Agentic Coding | 524k 111Models listed in this table may not natively support a 524k context window. We use 524k to model long-context agentic workloads and quantify the system impact of large KV-cache residency. | 256 | 8192 |
|
||||
|
||||
Model Architecture
|
||||
|
||||
| | | | |
|
||||
| --- | --- | --- | --- |
|
||||
| Model | Attention | # Experts | Precision |
|
||||
| GPT-OSS-120B | Full GQA + Window GQA | 128 | FP8 |
|
||||
| Qwen3-235B | GQA | 128 | FP8 |
|
||||
| Nemotron3-120B | Mamba 2 + GQA | 512 | FP8 |
|
||||
| DeepSeek V3.2 | MLA + Sparse Attention | 256 | FP8 |
|
||||
|
||||
#### 4.1.2 Cluster-scale DSE analysis
|
||||
|
||||
Figure˜5 reports the Pareto curve (tokens/s/user vs system tokens/s) produced by AIC++ on a 128 B200SXM cluster with TensorRT-LLM NVIDIA (2026b) as the performance backend, with each panel constrained by the workload’s SLO from Table˜1. To exhaustively probe the throughput frontier, our replica search enumerates every replica size from 2 to 128 GPUs and dynamically composes the per-replica parallelism plan across TP, DP, EP, and (for AFD) attention/FFN GPU groups. This lets the optimizer surface asymmetric layouts that only become feasible when prefill and decode have very different compute and memory profiles, including off-grid replica sizes that pack a long-context KV cache more efficiently than the obvious power-of-two choices. The cluster-scale results in this section are model-based DSE estimates that combine backend cost measurements with AstraSim communication modeling. Appendix 6.1 describes our vLLM-based AFD prototype, which we use to verify functional correctness of the all-pairs attention–FFN execution path and to ground the communication pattern modeled by AIC++.
|
||||
|
||||
|
||||
|
||||
Figure 5: Evaluation of a Cluster with 128 B200 GPUs for the representative model architectures and application workloads shown in Table 1, Y-axis denotes the system throughput (total tokens/s), X-axis denotes the interactivity (tokens/s/user)
|
||||
|
||||
No single strategy dominates the throughput frontier. Aggregated serving with chunked prefill, deployed as 16 single-node 8-GPU replicas that hold the expert FFNs through Expert Parallelism, wins most panels by amortizing the chunked-prefill bubble across the replica fleet and ingesting many disjoint token batches in parallel. Disaggregation takes the rest, but only after the wider search exposes asymmetric replica shapes: a single off-grid replica with many small 2-GPU prefill workers feeding a few large 8-GPU decode workers (Qwen3-235B-A22B chat, DeepSeek-V3.2 coding), or many tiny xPyD shards under _disagg+AFD_ that double aggregated’s throughput on Nemotron-3-Super coding. AFD on its own wins the throughput frontier on a single panel, GPT-OSS-120B chat, where the cheap sliding-window GQA layers let four 32-GPU replicas with a near-symmetric 16A+16F split outpace 16-replica agg. The wider TP-16 enumeration also unlocks a feasible _disagg-standard_ layout for DeepSeek-V3.2 agentic coding, where a narrower TP,≤,8 search returned no SLO-feasible configuration for the 524k-prefix workload.
|
||||
|
||||
On the latency axis, AFD wins every panel, with the optimal attention/FFN split tracking each model’s intrinsic attention/mixer cost relative to its FFN cost. DeepSeek-V3.2, whose MLA combined with sparse (DSA) attention shrinks both per-token attention compute and the KV-cache footprint, collapses the attention shard to its minimum on long-context workloads, dedicating almost the entire cluster to FFN (2A+126F on agentic, 16A+112F on chat). The 2A+126F layout is initially counter-intuitive given the 524k-token KV demand, but consistent with rate-matching: MLA compresses the latent KV cache so aggressively that the entire prefix fits in two GPUs’ HBM, and DSA keeps per-token attention cheap enou
|
||||
|
||||
manov.
|
||||
Vidur: A large-scale simulation framework for llm inference, 2024.
|
||||
URL https://arxiv.org/abs/2405.05465.
|
||||
- Ashish [2017]
|
||||
|
||||
Vaswani Ashish.
|
||||
Attention is all you need.
|
||||
_Advances in neural information processing systems_, 30:I, 2017.
|
||||
- Bambhaniya et al. [2026]
|
||||
|
||||
Abhimanyu Rajeshkumar Bambhaniya, Hanjiang Wu, Suvinay Subramanian, Sudarshan Srinivasan, Souvik Kundu, Amir Yazdanbakhsh, Midhilesh Elavazhagan, Madhu Kumar, Minlan Yu, Arijit Raychowdhury, and Tushar Krishna.
|
||||
Mist: A co-design framework for heterogeneous, multi-stage llm inference, 2026.
|
||||
URL https://arxiv.org/abs/2504.09775.
|
||||
- DeepSeek-AI et al. [2024]
|
||||
|
||||
DeepSeek-AI, Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Hanwei Xu, Hao Yang, Haowei Zhang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Li, Hui Qu, J. L. Cai, Jian Liang, Jianzhong Guo, Jiaqi Ni, Jiashi Li, Jin Chen, Jingyang Yuan, Junjie Qiu, Junxiao Song, Kai Dong, Kaige Gao, Kang Guan, Lean Wang, Lecong Zhang, Lei Xu, Leyi Xia, Liang Zhao, Liyue Zhang, Meng Li, Miaojun Wang, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Mingming Li, Ning Tian, Panpan Huang, Peiyi Wang, Peng Zhang, Qihao Zhu, Qinyu Chen, Qiushi Du, R. J. Chen, R. L. Jin, Ruiqi Ge, Ruizhe Pan, Runxin Xu, Ruyi Chen, S. S. Li, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shaoqing Wu, Shengfeng
|
||||
@@ -0,0 +1,94 @@
|
||||
# 📊 文章摘要：How Far Can Disaggregation Go? A Design-Space Exploration of Attention-FFN Disaggregation for Efficient MoE LLM Serving
|
||||
|
||||
> **原文**：[2026-05-27_AFD_How_Far_Can_Disaggregation_Go.md](./2026-05-27_AFD_How_Far_Can_Disaggregation_Go.md)
|
||||
> **原文链接**：https://arxiv.org/abs/2605.28302
|
||||
> **来源**：arXiv
|
||||
> **作者**：Hanjiang Wu, Abhimanyu Rajeshkumar Bambhaniya, Sarbartha Banerjee, Tuhin Khare, Sudarshan Srinivasan, Suvinay Subramanian 等 12 位（佐治亚理工学院、Intel、Google、Google DeepMind、Infravana）
|
||||
> **发布日期**：2026-05-27
|
||||
> **摘要日期**：2026-08-06
|
||||
> **价值评级**：⭐⭐⭐ 高
|
||||
|
||||
---
|
||||
|
||||
## 核心命题
|
||||
|
||||
> **算子级解耦** — 解耦不止于阶段（prefill/decode），可下探到算子级：把内存受限的 attention 与计算密集的 MoE-FFN 拆分到不同 GPU 组（AFD），并以设计空间探索给出"何时、何处更深层解耦才划算"的实用地图。
|
||||
|
||||
---
|
||||
|
||||
## 文章概要
|
||||
|
||||
随着 MoE 模型（如 DeepSeek-V3.2）规模扩大与 TTFT/TPOT SLO 收紧，LLM serving 的解耦粒度不断加深：从 chunked-prefill 聚合，到 prefill/decode（P/D）解耦，再到算子级 attention-FFN 解耦（AFD）。本文系统回答"每一层解耦何时真正划算"：基于 vLLM 的 AFD 原型 + AIConfigurator（算子级计算建模）+ AstraSim（包粒度网络模拟）构建 AIC++ 框架，对 128 卡 B200 集群上 4 种模型（DeepSeek-V3.2、GPT-OSS-120B、Nemotron3-120B、Qwen3-235B）× 3 类负载（chat/coding/agentic coding）做全维设计空间探索。核心发现：严格 SLO 下 AFD 在 DeepSeek-V3.2 上可维持约 4k tokens/s 系统吞吐，而非 AFD 部署不可行；吞吐维度没有单一策略通吃（chunked prefill 聚合常胜），但延迟/交互性维度 AFD 全胜，且最优 attention/FFN GPU 比例可依模型与负载导出（如 DeepSeek-V3.2 长上下文下 2A+126F 的极端配比）。局限：结果基于建模 + 模拟的 DSE 估计，原型仅验证功能正确性。
|
||||
|
||||
---
|
||||
|
||||
## 关键要点
|
||||
|
||||
1. **解耦粒度演化谱系** — chunked-prefill 聚合 → P/D 阶段解耦 → AFD 算子级解耦，每一层解耦都加深调度设计空间（工作负载特性 × 资源分配 × 互连拓扑），本文首次系统回答"解耦能走多远"这一问题。`[分类: 范式突破]`
|
||||
2. **MoE 架构放大算子异构** — attention（MHA/GQA/MLA）内存受限、FFN 计算密集，加上 MoE dispatch/combine 通信，一个 transformer 块内就存在显著计算异构；粗粒度抽象（MegaScale-Infer 已指出的问题）在 MoE 上尤其失效。`[分类: 共识]`
|
||||
3. **严格 SLO 下 AFD 使不可能变为可能** — DeepSeek-V3.2 上以 TTFT<50/100/150ms、TPOT≤15ms 的严格约束（chat/coding/agentic coding），AFD 部署维持约 4k tokens/s 系统吞吐，而非 AFD 部署不可行（infeasible）。`[分类: 范式突破]`
|
||||
4. **吞吐前沿无通吃方案** — 128×B200 集群 DSE：chunked prefill 聚合部署（16 个单节点 8-GPU 副本）赢得多数面板；AFD 单独赢得一个面板（GPT-OSS-120B chat，16A+16F 近似对称拆分）；非对称副本形态（如 2-GPU prefill 工人喂 8-GPU decode 工人）是解耦方案胜出的关键。`[分类: 争议]`
|
||||
5. **延迟维度 AFD 全胜** — 每个面板的最优 attention/FFN 拆分都追踪模型内在的 attention/mixer 成本与 FFN 成本之比；AFD 特定微批重叠（four-stage pipeline，全双工链路）最大化计算-通信重叠，始终给出最佳延迟与用户交互性。`[分类: 未探索]`
|
||||
6. **反直觉的极端配比** — DeepSeek-V3.2 的 MLA + 稀疏注意力把每 token attention 计算与 KV cache 足迹压得极小，长上下文 agentic 负载（524k prefix）下最优配置为 2A+126F——整个 524k 前缀的 KV cache 可装入 2 个 GPU 的 HBM，几乎整个集群给 FFN。`[分类: 范式突破]`
|
||||
7. **内存分割是隐藏收益** — 把多数模型权重放在 FFN GPU 上，attention GPU 腾出显存装 KV cache，同一内存约束下可支撑更高吞吐——AFD 的收益不止调度层面。`[分类: 未探索]`
|
||||
8. **位置感知放置原则** — 最频繁的层内 A2F/F2A 流量（每请求 O(layer)）绑定最高带宽 scale-up 域（NVLink），低频的跨节点 KV cache 传输（每请求 O(1)）走 scale-out（InfiniBand）；A2F 扇形广播（fan-out）使 FFN 侧入口拥塞、F2A 扇形汇聚（fan-in）使 attention 侧入口成为瓶颈。`[分类: 未探索]`
|
||||
9. **AIC++ 方法论** — 内核级实测成本库（AIConfigurator）+ 包粒度拥塞感知网络模拟（AstraSim）联合建模，以 vLLM AFD 原型锚定通信模式与功能正确性；论文明示集群级结果均为"模型驱动 DSE 估计"。`[分类: 争议]`
|
||||
|
||||
---
|
||||
|
||||
## 批判性分析
|
||||
|
||||
### 假设前提
|
||||
|
||||
- 异构/解耦硬件时代将到来：节点内出现计算/内存不对称的加速器组（NVIDIA Groq-3 LPX、Rubin CPX、Intel/SambaNova 等），且存在 NVLink 级高带宽 scale-up 互连。
|
||||
- AFD 的通信模式（all-pairs 的 A2F/F2A 双向传输）在真实网络上可被微批重叠技术有效隐藏。
|
||||
- 内核级测量成本库 + AstraSim 网络模拟的组合足以逼近真实集群行为，从而替代全系统原型评估。
|
||||
- 以系统吞吐与用户交互性（tokens/s/user）为目标的 SLO 框架能代表生产诉求。
|
||||
|
||||
### 论据与逻辑
|
||||
|
||||
- 论据结构严谨：先以运行时分解与内存分解数据（图 3）确立算子异构事实，再用 AIC++ 在 128 卡规模做穷举式 DSE（副本规模 2-128 卡全枚举 + TP/DP/EP/SP/PP 与 P/D、AFD 联合搜索），结论（无通吃方案、AFD 延迟全胜、配比可推导）有模拟数据支撑。
|
||||
- 关键量化结论（约 4k tokens/s、2A+126F、2.35×/1.4× 类收益的可比性）均限定在各自严格语境中，未过度外推。
|
||||
- 弱点：集群级结论全部来自建模估计，AIC++ 的准确性未与真实 AFD 集群端到端对照；vLLM 原型仅验证功能正确性，无法排除建模误差对"吞吐前沿无通吃方案"等核心结论的影响；对非 AFD 部署"不可行"的判定依赖 SLO 设定与搜索范围的完整性（论文也承认窄 TP≤8 搜索曾漏掉可行配置）。
|
||||
|
||||
### 边界与局限
|
||||
|
||||
- AFD 的优势集中于延迟/交互性维度与严格 SLO 场景；纯吞吐目标下 chunked prefill 聚合常更优——结论不可泛化为"AFD 总是更好"。
|
||||
- 依赖高带宽 scale-up 互连；半双工/低带宽网络下微批重叠收益收缩（pipeline 深度由 4 降为 3）。
|
||||
- 配比指导依赖模型架构（MLA/GQA/Mamba 差异显著），换架构需重跑 DSE。
|
||||
- 结果基于模拟，需真实异构集群验证；524k prefix 属极端长上下文建模，超出部分模型原生窗口。
|
||||
|
||||
---
|
||||
|
||||
## 可引用金句
|
||||
|
||||
> "Our findings deliver a practical map of when and where deeper disaggregation pays off for MoE serving at scale."
|
||||
> （我们的发现提供了一张实用地图：在 MoE 规模化服务中，何时、何处更深层的解耦才真正划算。）
|
||||
|
||||
> "Under strict TTFT/TPOT SLOs, AFD sustains around 4k tokens/s of system throughput on DeepSeek-V3.2 across chat, coding, and agentic-coding workloads, regimes in which non-AFD deployments are infeasible."
|
||||
> （在严格的 TTFT/TPOT SLO 下，AFD 在 DeepSeek-V3.2 上于 chat、coding 与 agentic-coding 负载中维持约 4k tokens/s 的系统吞吐，而这是非 AFD 部署不可行的场景。）
|
||||
|
||||
---
|
||||
|
||||
## 总体评价
|
||||
|
||||
**亮点**：
|
||||
- 首次把解耦粒度推进到算子级（AFD）并系统回答"解耦能走多远"，把 2023-2024 年的阶段解耦共识延伸为新范式
|
||||
- "吞吐无通吃方案、延迟 AFD 全胜"的差异化结论打破了"解耦必然更好"的朴素预期，结论颗粒度精细
|
||||
- 给出可操作的 attention/FFN GPU 配比指导（含 2A+126F 这类反直觉结论背后的推理），并揭示内存分割这一隐藏收益
|
||||
- AIC++ 把内核实测与网络模拟结合的建模方法论，为异构推理基础设施设计提供了可复用工具
|
||||
|
||||
**不足**：
|
||||
- 集群级结论全部基于建模与模拟，缺少真实 AFD 集群的端到端验证
|
||||
- 部分结论（如"非 AFD 不可行"）对 SLO 设定与搜索范围敏感，泛化需谨慎
|
||||
- 评估硬件模型（B200 + TensorRT-LLM）与实际异构平台（Groq-3 LPX 等）仍有差距
|
||||
|
||||
**适用场景**：MoE 模型 serving 的系统架构师；面向异构加速器（Groq-3 LPX、Rubin CPX 等）与解耦集群的设计者；研究 LLM serving 设计空间探索的研究人员。
|
||||
|
||||
**关联建议**：向上游追溯 DistServe（P/D 解耦起点）、Splitwise（异构硬件）、Mooncake（生产实践），可看出"解耦粒度深化"的完整脉络；后续可关注 MegaScale-Infer（同方向的算子级优化）、Mist（同团队的异构多阶段 co-design 框架）以及 NVIDIA 解耦平台（LPX/CPX）的真实部署验证。
|
||||
|
||||
---
|
||||
|
||||
## 配图
|
||||
|
||||
|
||||
Reference in New Issue
Block a user