[TRTLLM-16217][test] Add VR200 4-GPU serve perf cases for DeepSeek V4 Pro and Kimi K3 by ruodil · Pull Request #19260 · NVIDIA/TensorRT-LLM

ruodil · 2026-09-16T07:21:33Z

Summary

Extend the shared B200/GB200/B300/GB300/VR200 4-GPU condition block (compute_capability 10.0-10.7) in llm_perf_core.yml with DEP4/TEP4 serve-backend perf cases for deepseek_v4_pro_base_fp8, deepseek_v4_pro_fp4, and kimi_k3.
Both models are currently pinned to ep:8-tp:8-gpus:8 on B300 only (compute_capability capped at 10.3); neither has a 4-GPU config validated there (the Kimi K3 deployment guide states TEP8 is the minimum topology that fits GB300-class per-GPU memory). VR200's larger per-GPU HBM fits the same DEP4/TEP4 shapes, so these cases are scoped to the existing Rubin-inclusive 4-GPU condition block rather than the B300-only one.
Case shapes mirror the existing 8-GPU B300 recipes 1:1 (same maxbs/maxnt/kv_frac/ISL-OSL/reqs/con), just at ep:4-tp:4-gpus:4, and all use the serve backend.
Also registers matching serve-mode config patterns in pytorch_model_config.py for deepseek_v4_pro_fp4 (its existing patterns were bench-only), so these cases pick up the correct MoE backend / KV-cache dtype / MTP config instead of silently falling back to defaults.

Test plan

Validate on VR200 hardware once available; will report actual TPS/TTFT/TPOT and adjust kv_frac/maxbs if any case OOMs or fails to fit.
Verified tests/integration/test_lists/qa/llm_perf_core.yml parses as valid YAML.
Verified tests/integration/defs/perf/pytorch_model_config.py parses as valid Python.

🤖 Generated with Claude Code

Dev Engineer Review

Adds VR200 4-GPU serve-performance cases for three models: deepseek_v4_pro_base_fp8, deepseek_v4_pro_fp4, and kimi_k3.
Adds matching deepseek_v4_pro_fp4 serve configuration patterns.
VR200 hardware validation remains pending.

QA Engineer Review

Updates the manual-QA performance list with streaming-throughput, latency, maximum-throughput, interactive, and short-context workloads.
Uses ep=4, tp=4, four GPUs, and 120-second timeouts.
YAML and Python syntax validation is complete. Hardware validation remains pending.
Coverage verdict: needs follow-up.

Per-File QA Perspective

tests/integration/test_lists/qa/llm_perf_core.yml: Adds manual-QA coverage for three models on VR200 4-GPU configurations. Verify model loading, workload selection, throughput, latency, and timeout behavior.
tests/integration/defs/perf/pytorch_model_config.py: Adds serve configuration patterns for deepseek_v4_pro_fp4. Verify the selected MoE backend, KV-cache dtype, attention-DP behavior, and MTP settings.

coderabbitai · 2026-09-16T07:26:29Z

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: NVIDIA/TensorRT-LLM/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 72d6cd39-0e7f-4a71-a965-b1548ee01f43

📥 Commits

Reviewing files that changed from the base of the PR and between eb58957 and 64ccf94.

📒 Files selected for processing (1)

tests/integration/test_lists/qa/llm_perf_core.yml

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

The QA configuration adds eleven VR200 4-GPU serving performance tests for DeepSeek V4 Pro Base FP8, DeepSeek V4 Pro FP4, and Kimi K3. The tests cover throughput, latency, interactive, and context-length workloads with model-specific settings and 120-second timeouts.

Changes

VR200 performance tests

Layer / File(s)	Summary
VR200 QA test registrations `tests/integration/test_lists/qa/llm_perf_core.yml`	Added three DeepSeek V4 Pro Base FP8 tests, three DeepSeek V4 Pro FP4 tests, and five Kimi K3 tests. The entries define 4-GPU serving settings for throughput, latency, interactive, and context-length scenarios, with 120-second timeouts.

Priority: ⬇️ Low

Estimated code review effort: 2 (Simple) | ~10 minutes

Change: Other

Merge Risk: ⚪ Minimal · up to 64ccf

This change adds VR200 serving performance coverage for three models without altering production runtime behavior. It is mergeable, with hardware validation still appropriate as follow-up.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the ticket, test type, VR200 4-GPU serving performance coverage, and affected models. It accurately summarizes the primary change.
Description check	✅ Passed	The description explains the change, rationale, configuration details, affected files, and test plan. It also clearly states that VR200 hardware validation is pending. The headings differ from the tem…
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 1 functions across 1 files. (1 skipped: 1 …
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-09-17T04:07:36Z

Automatically added "ci: full pre-merge approved" because this PR has satisfied the required GitHub review approvals. Unresolved review conversations and other required checks remain independent merge requirements.

… Pro and Kimi K3 Extend the shared B200/GB200/B300/GB300/VR200 4-GPU condition block (compute_capability 10.0-10.7) with DEP4/TEP4 serve-backend shapes for deepseek_v4_pro_base_fp8, deepseek_v4_pro_fp4, and kimi_k3, mirroring the ep:8-tp:8-gpus:8 B300 cases already in the 8-GPU-only condition block. Neither model has a 4-GPU config validated on B200/B300 (both are pinned to ep=tp=8 there; the Kimi K3 deployment guide states TEP8 is the minimum topology that fits GB300-class per-GPU memory), but VR200's larger per-GPU HBM makes the same DEP4/TEP4 shapes fit. Signed-off-by: Ruodi Lu <ruodil@users.noreply.github.com>

ruodil · 2026-09-22T02:03:13Z

/bot run

tensorrt-cicd · 2026-09-22T02:09:24Z

PR_Github #74914 [ run ] triggered by Bot. Commit: 64ccf94 Link to invocation

tensorrt-cicd · 2026-09-22T04:37:52Z

PR_Github #74914 [ run ] completed with state SUCCESS. Commit: 64ccf94
/LLM/main/L0_MergeRequest_PR pipeline #61682 completed with status: 'SUCCESS'

CI Report

Link to invocation

github-actions · 2026-10-01T20:52:40Z

Semantic conflict review

The verdict of record is the Semantic conflict with target branch / PR #19260 commit status on the requested head commit. This summary updates on reply events and may lag between a new request and its reply.

Latest recorded state: Possible semantic conflict for head 64ccf94d837058f7d849ef9f093d23423c6559c8, target a81da8a5a8380c4ec8c2d3eafabb8cfc55d7ad56, merge base f7aeaef16348080a2d87fa80ee69f18936a78d93 (request 57f032ff-3be5-414a-aaf1-7d592123472a). CodeRabbit analysis.

Best-effort AI judgment for the recorded revisions. PASS, FAIL and INCONCLUSIVE may be incomplete or incorrect. PR authors and reviewers should independently verify the evidence and relevant behavior. This semantic review and its status/workflow are advisory, not required merge checks under current repository rules; other merge requirements still apply. Advisory status does not make a confirmed defect safe to ignore.

Requested (UTC)	Head	Target	Verdict	Comment
2026-10-02T10:39:28Z	`64ccf94d8370`	`a81da8a5a838`	FAIL	reply
2026-10-02T04:42:17Z	`64ccf94d8370`	`8a3c90311ea9`	FAIL	reply
2026-10-01T20:49:26Z	`64ccf94d8370`	`80509acfc073`	FAIL	reply
2026-10-01T14:42:02Z	`64ccf94d8370`	`ee510fc85d39`	FAIL	reply
2026-10-01T06:56:51Z	`64ccf94d8370`	`0d3bbd257d35`	FAIL	reply
2026-10-01T01:24:19Z	`64ccf94d8370`	`534e1f8ad9d5`	FAIL	reply
2026-09-30T16:41:35Z	`64ccf94d8370`	`324a51deff2a`	FAIL	reply
2026-09-30T10:39:49Z	`64ccf94d8370`	`abddfc991456`	FAIL	reply
2026-09-30T02:49:52Z	`64ccf94d8370`	`111687f92f4d`	FAIL	reply
2026-09-29T18:45:24Z	`64ccf94d8370`	`bcb288a21105`	FAIL	reply
2026-09-29T12:52:14Z	`64ccf94d8370`	`bcdc2d5aaf0c`	FAIL	reply
2026-09-29T04:42:54Z	`64ccf94d8370`	`ae4aa5d6c932`	FAIL	reply
2026-09-28T22:34:52Z	`64ccf94d8370`	`27ba69c78189`	FAIL	reply
2026-09-28T14:42:33Z	`64ccf94d8370`	`0c58480ca680`	FAIL	reply
2026-09-28T08:52:55Z	`64ccf94d8370`	`c2220eef33f4`	FAIL	reply

Processed request and reply comments are minimized to reduce timeline noise; they remain expandable for audit.

ruodil requested review from a team as code owners September 16, 2026 07:21

ruodil requested review from BowenFu, LarryXFly, ZhanruiSunCh, brnguyen2 and tburt-nv September 16, 2026 07:21

github-actions Bot assigned ruodil Sep 16, 2026

ruodil mentioned this pull request Sep 16, 2026

Rubin/VR200 4-GPU perf coverage missing for Qwen 3.8 MAX and GLM 5.2 #19262

Closed

BowenFu approved these changes Sep 16, 2026

View reviewed changes

LarryXFly approved these changes Sep 17, 2026

View reviewed changes

trtllm-agent added the ci: full pre-merge approved label Sep 17, 2026

ruodil force-pushed the user/ruodil/rubin-4gpu-v4pro-kimik3 branch from 75540ae to eb58957 Compare September 21, 2026 07:38

Merge branch 'main' into user/ruodil/rubin-4gpu-v4pro-kimik3

64ccf94

This comment has been minimized.

Sign in to view

ruodil merged commit a40d97a into NVIDIA:main Oct 7, 2026
10 of 12 checks passed

Conversation

ruodil commented Sep 16, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Dev Engineer Review

QA Engineer Review

Per-File QA Perspective

Uh oh!

coderabbitai Bot commented Sep 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

github-actions Bot commented Sep 17, 2026

Uh oh!

ruodil commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

tensorrt-cicd commented Sep 22, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions Bot commented Oct 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Semantic conflict review

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ruodil commented Sep 16, 2026 •

edited by coderabbitai Bot

Loading

coderabbitai Bot commented Sep 16, 2026 •

edited

Loading

github-actions Bot commented Oct 1, 2026 •

edited

Loading