[None][fix] always sync sampler_event in update_requests by Funatiq · Pull Request #12585 · NVIDIA/TensorRT-LLM

Funatiq · 2026-03-30T08:04:02Z

Summary by CodeRabbit

Bug Fixes
- Fixed sampler synchronization to ensure proper event handling even when processing empty request lists, preventing potential race conditions.

Description

Always synchronize the sampler_event at the beginning of update_requests, regardless of whether there are sampling requests or not.
This fixes performance metrics, as the GPU events need to be completed for the performance measurements to work.
This may introduce a small overhead when there are no sampling requests, but it ensures that the performance metrics are accurate and consistent.

Test Coverage

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

- Always synchronize the sampler_event at the beginning of update_requests, regardless of whether there are sampling requests or not. - This fixes performance metrics, as the GPU events need to be completed for the performance measurements to work. - This may introduce a small overhead when there are no sampling requests, but it ensures that the performance metrics are accurate and consistent. Signed-off-by: Robin Kobus <[email protected]>

Funatiq · 2026-03-30T08:04:11Z

/bot run

tensorrt-cicd · 2026-03-30T08:11:13Z

PR_Github #40695 [ ] completed with state FAILURE. Commit: ``

Link to invocation

Funatiq · 2026-03-30T10:08:04Z

/bot run

tensorrt-cicd · 2026-03-30T10:14:28Z

PR_Github #40719 [ run ] triggered by Bot. Commit: fe572dd Link to invocation

ixlmar

Might be good to also add a test case to prevent regression.

tensorrt-cicd · 2026-03-30T17:08:31Z

PR_Github #40719 [ run ] completed with state SUCCESS. Commit: fe572dd
/LLM/main/L0_MergeRequest_PR pipeline #31745 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

Funatiq · 2026-03-30T21:19:10Z

/bot run

tensorrt-cicd · 2026-03-30T21:24:58Z

PR_Github #40773 [ run ] triggered by Bot. Commit: fe572dd Link to invocation

tensorrt-cicd · 2026-03-31T01:00:55Z

PR_Github #40773 [ run ] completed with state SUCCESS. Commit: fe572dd
/LLM/main/L0_MergeRequest_PR pipeline #31789 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

Funatiq · 2026-03-31T08:32:53Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-03-31T09:09:49Z

PR_Github #40912 [ run ] triggered by Bot. Commit: fe572dd Link to invocation

tensorrt-cicd · 2026-03-31T15:25:29Z

PR_Github #40912 [ run ] completed with state SUCCESS. Commit: fe572dd
/LLM/main/L0_MergeRequest_PR pipeline #31913 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

Funatiq · 2026-03-31T16:12:49Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-03-31T16:18:37Z

PR_Github #40969 [ run ] triggered by Bot. Commit: fe572dd Link to invocation

tensorrt-cicd · 2026-03-31T19:33:34Z

PR_Github #40969 [ run ] completed with state SUCCESS. Commit: fe572dd
/LLM/main/L0_MergeRequest_PR pipeline #31955 completed with status: 'SUCCESS'

CI Report

Link to invocation

coderabbitai · 2026-04-01T07:11:11Z

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 93e82757-dd32-4a1c-810f-0396386f49eb

📥 Commits

Reviewing files that changed from the base of the PR and between 58d6975 and fe572dd.

📒 Files selected for processing (1)

tensorrt_llm/_torch/pyexecutor/sampler.py

📝 Walkthrough

Walkthrough

The early-exit condition for empty request lists in update_requests methods is repositioned to occur after synchronizing the sampler event, ensuring proper event synchronization even when no requests are present.

Changes

Cohort / File(s)	Summary
Event Synchronization Reordering `tensorrt_llm/_torch/pyexecutor/sampler.py`	Moved early-exit condition `if not state.requests: return` to occur after `state.sampler_event` synchronization in both `TorchSampler.update_requests` and `TRTLLMSampler.update_requests` to ensure CUDA event synchronization happens regardless of request list emptiness.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~3 minutes

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

Signed-off-by: Robin Kobus <[email protected]>

github-actions Bot assigned Funatiq Mar 30, 2026

Funatiq changed the title ~~[fix] always sync sampler_event in update_requests~~ [None][fix] always sync sampler_event in update_requests Mar 30, 2026

Funatiq requested a review from ixlmar March 30, 2026 08:05

ixlmar approved these changes Mar 30, 2026

View reviewed changes

Funatiq marked this pull request as ready for review April 1, 2026 07:09

Funatiq requested a review from a team as a code owner April 1, 2026 07:09

Funatiq requested a review from lancelly April 1, 2026 07:09

Funatiq merged commit d036f74 into NVIDIA:main Apr 1, 2026
7 of 10 checks passed

Funatiq mentioned this pull request Apr 1, 2026

[None][refactor] Improve request management in sampler #11861

Merged

1 task

Funatiq deleted the dev/fix/perf_metrics branch April 1, 2026 07:11

karen-sy pushed a commit to karen-sy/TensorRT-LLM that referenced this pull request Apr 7, 2026

[None][fix] always sync sampler_event in update_requests (NVIDIA#12585)

fa9c526

Signed-off-by: Robin Kobus <[email protected]>

Conversation

Funatiq commented Mar 30, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

Funatiq commented Mar 30, 2026

Uh oh!

tensorrt-cicd commented Mar 30, 2026

Uh oh!

Funatiq commented Mar 30, 2026

Uh oh!

tensorrt-cicd commented Mar 30, 2026

Uh oh!

ixlmar left a comment

Choose a reason for hiding this comment

Uh oh!

tensorrt-cicd commented Mar 30, 2026

Uh oh!

Funatiq commented Mar 30, 2026

Uh oh!

tensorrt-cicd commented Mar 30, 2026

Uh oh!

tensorrt-cicd commented Mar 31, 2026

Uh oh!

Funatiq commented Mar 31, 2026

Uh oh!

tensorrt-cicd commented Mar 31, 2026

Uh oh!

tensorrt-cicd commented Mar 31, 2026

Uh oh!

Funatiq commented Mar 31, 2026

Uh oh!

tensorrt-cicd commented Mar 31, 2026

Uh oh!

tensorrt-cicd commented Mar 31, 2026

Uh oh!

Uh oh!

coderabbitai Bot commented Apr 1, 2026

Review failed

Walkthrough

Changes

Estimated code review effort

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Funatiq commented Mar 30, 2026 •

edited by coderabbitai Bot

Loading