Advisory

PYSEC-2026-3542

Details, part 2 of 2. The advisory

Details

Observed result

Local repro typically ends with:

  • a recovered-token trace showing:
  • sample_recovered_tokens_return -> recovered_token_ids[0] = 151936
  • prepare_next_token_ids_padded -> next_token_ids[0] = -1
  • set_inputs_first_pass -> input_ids_after[0] = -1
  • embed_input_ids_out_of_range -> input_ids[0] = -1
  • CUDA error: device-side assert triggered
  • a fatal engine-side failure

gRPC repro typically ends with:

  • the triggering gRPC requests failing with INTERNAL: EngineCore encountered an issue. See stack trace (above) for the root cause.
  • server logs showing the worker dies with CUDA error: device-side assert triggered
  • a later public probe request also failing after the worker is dead

This demonstrates that the issue is reachable through the public gRPC request surface, not only through a local reproducer.

Log snippets

Local recovered-chain trace

sample_recovered_tokens_return:
  recovered_token_ids = [151936, ...]
  vocab_size = 151936

prepare_next_token_ids_padded:
  sampled_token_ids_head = [[151936, -1, -1, ...], ...]
  next_token_ids = [-1, ...]

set_inputs_first_pass:
  input_ids_after = [-1, 0, 0, 0, ...]

embed_input_ids_out_of_range:
  input_ids = [-1, 0, 0, 0, ...]

gRPC server log

torch.AcceleratorError: CUDA error: device-side assert triggered
...
vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.
...
Error in Generate for request post_crash_probe
vllm.v1.engine.exceptions.EngineDeadError: EngineCore encountered an issue. See stack trace (above) for the root cause.

Root cause

This is a speculative-decoding state-handling bug, not an invalid frontend token-id input bug.

The root cause is that a recovered speculative token can become equal to vocab_size, then be selected as the live next token for a request, then be converted to -1, and that converted -1 is still written back into live drafter input ids and later consumed by the drafting / embedding / attention path.

For the Qwen3 checkpoint used here:

  • 151936 == vocab_size

This value should be described as the model vocab_size boundary value, not as a legal token id.

Attachments

The attached bundle for this report should contain:

  • repro_g4_recovered_minus1_local.py
  • repro_g4_recovered_minus1_grpc.py

These two standalone scripts are sufficient to reproduce the issue and its public gRPC reachability.

Fix

A fix for this vulnerability has been merged in: https://github.com/vllm-project/vllm/pull/44744

Sources & licences