 Ruixiang Wang
|
88a39274ec
|
spec: add EAGLE3 speculative decoding support (#18039)
* llama : enable layer input extraction
* spec: support eagle3
* eagle3: fix params bug
* eagle3: support Gemma4 eagle3 from RedHatAI
* eagle3: set sync when get features from target
Co-authored-by: tnhnyzc <115956684+tnhnyzc@users.noreply.github.com>
* eagle3 : fix ubatch handling in embd_layer_inp extraction and encoder
Co-authored-by: Doğaç Eldenk <dogacel@gmail.com>
* eagle3: adapt to upstream changes
* eagle3: fix rebase issues and adapt to upstream changes
* eagle3:exclude the eagle3 arch from test-llama-archs
* eagle3: fix editorconfig check failures
* eagle3: fix multi-seq issue in d2t vocab mapping
* cont : minor style / clean-up
* spec : remove `common_speculative_setup_draft_model()`
* llama : clean-up unused API
* eagle3: set d2t vocab mapping in decode graph
* cont : assert layer inputs are configured
* hparams : use n_embd_inp instead of n_embd_target_features
* eagle3: make output.weight optional and inherit from target model when needed
* haparams : generic norm-before-residual param
* llama-ext : consistent names
* cont : fix
* hparams : remove target_hidden_size
* cparams : rename output_layer_inp -> embeddings_layer_inp
* arch : reuse ATTN_NORM_2 instead of adding new hidden norm
* llama : clean-up names
* cont : add assert + comment
* Update conversion/llama.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
Co-authored-by: tnhnyzc <115956684+tnhnyzc@users.noreply.github.com>
Co-authored-by: Doğaç Eldenk <dogacel@gmail.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
|
2026-06-12 10:21:06 +03:00 |
|