Skip to content

[https://nvbugs/6008183][fix] Use extra_visual_gen_options to help de…#12487

Merged
JunyiXu-nv merged 1 commit into
NVIDIA:mainfrom
JunyiXu-nv:user/junyix/fix-bug-6008183
Mar 29, 2026
Merged

[https://nvbugs/6008183][fix] Use extra_visual_gen_options to help de…#12487
JunyiXu-nv merged 1 commit into
NVIDIA:mainfrom
JunyiXu-nv:user/junyix/fix-bug-6008183

Conversation

@JunyiXu-nv
Copy link
Copy Markdown
Collaborator

@JunyiXu-nv JunyiXu-nv commented Mar 24, 2026 •
edited by coderabbitai Bot
Loading

…tect visual gen model

Summary by CodeRabbit

  • Enhancements
    • Improved server routing for visual generation models by expanding detection logic to recognize additional configuration scenarios, enabling better support for diverse visual generation workloads.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

@JunyiXu-nv JunyiXu-nv requested a review from a team as a code owner March 24, 2026 07:14
@JunyiXu-nv JunyiXu-nv requested a review from schetlur-nv March 24, 2026 07:14
@JunyiXu-nv
Copy link
Copy Markdown
Collaborator Author

JunyiXu-nv commented Mar 24, 2026

/bot run

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 24, 2026 •
edited
Loading

📝 Walkthrough

Walkthrough

The server routing logic in serve.py was modified to check for visual generation by evaluating whether extra_visual_gen_options is provided (non-None) in addition to checking if the model is a diffusion-based model. The routing decision between _serve_visual_gen() and _serve_llm() now uses this combined condition.

Changes

Cohort / File(s) Summary
Visual Generation Routing Logic
tensorrt_llm/commands/serve.py		 Added extra_visual_gen_options as an additional condition for determining is_visual_gen, allowing visual generation to be triggered by either explicit options or diffusion model detection.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The PR description is incomplete. While it includes the required template structure, all substantive sections (Description, Test Coverage) are empty placeholders with only comments. Fill in the Description section explaining what the fix does and why it's needed. Add Test Coverage section listing relevant tests that verify the changes to visual model detection logic.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically describes the main change: using extra_visual_gen_options to help detect visual generation models in the serve.py file.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

tensorrt-cicd commented Mar 24, 2026

PR_Github #40082 [ run ] triggered by Bot. Commit: fffcfed Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

tensorrt-cicd commented Mar 24, 2026

PR_Github #40082 [ run ] completed with state SUCCESS. Commit: fffcfed
/LLM/main/L0_MergeRequest_PR pipeline #31235 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@JunyiXu-nv
Copy link
Copy Markdown
Collaborator Author

JunyiXu-nv commented Mar 25, 2026

/bot run --disable-fail-fast

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

tensorrt-cicd commented Mar 25, 2026

PR_Github #40219 [ run ] triggered by Bot. Commit: fffcfed Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

tensorrt-cicd commented Mar 25, 2026

PR_Github #40219 [ run ] completed with state SUCCESS. Commit: fffcfed
/LLM/main/L0_MergeRequest_PR pipeline #31356 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

@JunyiXu-nv
Copy link
Copy Markdown
Collaborator Author

JunyiXu-nv commented Mar 26, 2026

/bot run

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

tensorrt-cicd commented Mar 26, 2026

PR_Github #40402 [ run ] triggered by Bot. Commit: fffcfed Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

tensorrt-cicd commented Mar 26, 2026

PR_Github #40402 [ run ] completed with state SUCCESS. Commit: fffcfed
/LLM/main/L0_MergeRequest_PR pipeline #31497 completed with status: 'SUCCESS'

CI Report

Link to invocation

@JunyiXu-nv JunyiXu-nv merged commit 8ce0518 into NVIDIA:main Mar 29, 2026
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zhenhuaw-me zhenhuaw-me zhenhuaw-me approved these changes

@schetlur-nv schetlur-nv Awaiting requested review from schetlur-nv schetlur-nv is a code owner automatically assigned from NVIDIA/trt-llm-llmapi-devs

@QiJune QiJune Awaiting requested review from QiJune

Assignees

@JunyiXu-nv JunyiXu-nv

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@JunyiXu-nv @tensorrt-cicd @zhenhuaw-me