2 Commits
Author SHA1 Message Date
Jesus Gulfo b0dcb8192b server: fix speculation after an image (#28715)
* server: fix speculation after an image

Pass the actual position to the drafter after an image, instead of the
token count. Affects every drafter, not just DFlash.

* rename draft n_past to pos0

n_past is used to denote number of tokens and this parameter is meant to be a position
2026-09-11 11:33:26 +03:00
Jesus Gulfo fa67698187 spec: fix failed to decode mtmd chunk with DFlash (#28587)
* speculative: fix failed to decode mtmd chunk with DFlash

When using DFlash w/ vision models, the drafter memory fails to
allocate new tokens because images report a fixed offset. Stop copying
them to allow the drafter to continue.

* address PR feedback

limit M-RoPE skip to images only, allow audio to pass through. Clean up
comments to align to the updated implementation
2026-09-10 17:10:55 +02:00