Georgi Gerganov
c8f8e2364c
cont : simplify
2026-05-11 10:54:07 +03:00
Aman Gupta
c417ddfc74
fix batch size
2026-05-11 12:22:37 +08:00
Aman Gupta
a428b010ab
spec: support MTP
2026-05-11 11:28:30 +08:00
Georgi Gerganov
db8e326913
spec : introduce common_speculative_process()
2026-05-09 17:12:24 +03:00
Georgi Gerganov
0d5dd61d66
spec : reset drafting flag at the end
2026-05-09 17:12:06 +03:00
Georgi Gerganov
ec8bc44854
cont : minor
2026-05-09 16:38:17 +03:00
Georgi Gerganov
b3bd3bd4cc
cont : clean-up
2026-05-09 15:03:20 +03:00
Georgi Gerganov
ce0acf03ea
server, spec : clean-up
2026-05-09 10:21:57 +03:00
Georgi Gerganov
55b62bce15
llama : reuse device buffers when possible
2026-05-08 20:42:56 +03:00
Georgi Gerganov
f1652197dd
server : support parallel drafting
2026-05-08 19:30:31 +03:00
Georgi Gerganov
f88c942861
spec : support parallel drafts
2026-05-08 18:53:33 +03:00
Georgi Gerganov
927d6635d3
cont : prepare params
2026-05-08 17:50:20 +03:00
Georgi Gerganov
8822c122be
cont : prepare params
2026-05-08 17:06:24 +03:00
Georgi Gerganov
6582523eaa
spec : refactor for multi-sequence speculative context
2026-05-08 15:43:36 +03:00
Georgi Gerganov
efa2f8e5a7
naming : improve consistency
2026-05-08 12:24:57 +03:00
Georgi Gerganov
778f9e247e
tools : update readme
2026-05-08 11:55:16 +03:00
Georgi Gerganov
1dbc054da5
server : fix slot ctx_drft ptr
2026-05-08 11:55:05 +03:00
Georgi Gerganov
161eae0adf
spec : fix n_past type
2026-05-08 11:54:32 +03:00
Georgi Gerganov
e5b1401318
speculative-simple : update
2026-05-08 11:09:34 +03:00
Georgi Gerganov
3b1a8df8fd
server : clean-up + dry
2026-05-08 10:20:01 +03:00
Georgi Gerganov
233d1aee69
server : add comment
...
[no ci]
2026-05-08 08:50:23 +03:00
Georgi Gerganov
12c7cfbe83
server : fix URL for draft model
2026-05-08 08:03:49 +03:00
Georgi Gerganov
6a4b05a030
server : fix mtmd draft processing
2026-05-08 08:02:11 +03:00
Georgi Gerganov
8be14e40de
spec : handle draft running out of context
2026-05-08 07:11:51 +03:00
Georgi Gerganov
7e118cdce0
cont : process images throught the draft context
2026-05-07 21:44:09 +03:00
Georgi Gerganov
ae6703fa89
cont : pass correct n_past for drafting
2026-05-07 21:44:08 +03:00
Georgi Gerganov
0239f4c611
cont : handle non-ckpt models
2026-05-07 21:44:08 +03:00
Georgi Gerganov
c7facb0fe1
cont : async drft eval when possible
2026-05-07 21:44:08 +03:00
Georgi Gerganov
08c8012bde
cont : sync main and drft contexts
2026-05-07 21:44:08 +03:00
Georgi Gerganov
de35b1255c
server, spec : transition to unified spec context
2026-05-07 21:44:08 +03:00
Georgi Gerganov
1afee5b262
server : improve ctx names
...
[no ci]
2026-05-07 21:44:08 +03:00
Georgi Gerganov
11fd5e7272
server : draft prompt cache and checkpoints
...
[no ci]
2026-05-07 21:44:08 +03:00
Georgi Gerganov
c97dc3605e
server : sketch the ctx_dft decode loop
...
[no ci]
2026-05-07 21:44:08 +03:00
Georgi Gerganov
8a50f6f0b9
cont : dedup ctx_seq_rm_type
...
[no ci]
2026-05-07 21:44:07 +03:00
Georgi Gerganov
77269ad8a7
cont : pass seq_id
...
[no ci]
2026-05-07 21:44:07 +03:00
Georgi Gerganov
4550f0f08b
spec : update common_speculative_init()
...
[no ci]
2026-05-07 21:44:07 +03:00
Georgi Gerganov
befc7ef635
spec : drop support for incompatible vocabs
...
[no ci]
2026-05-07 21:44:07 +03:00
Georgi Gerganov
2c9a40849f
spec : refactor
...
[no ci]
2026-05-07 21:44:07 +03:00
Georgi Gerganov and GitHub
e43431b381
llama : fix device state save/load ( #22805 )
2026-05-07 21:43:40 +03:00
shaofeiqi and GitHub
ceb7e14b96
opencl: add opfilter regex for debugging ( #22782 )
2026-05-07 11:00:20 -07:00
Aldehir Rojas and GitHub
093be624cc
common/chat : preserve media markers for typed-content templates ( #22634 )
2026-05-07 12:50:56 -05:00
HaoJun ZHANG and GitHub
deab41ec68
tests: add long-sequence cases and fix inputs for gated_delta_net ( #22794 )
...
* tests : add long-seq + tail cases for gated_delta_net
* tests : realistic input ranges for gated_delta_net
2026-05-08 00:23:36 +08:00
ad09224658
sycl: add FILL, CUMSUM, DIAG, SOLVE_TRI, SSM_SCAN, GATED_DELTA_NET ( #22149 )
...
* sycl: add FILL, CUMSUM, DIAG, SOLVE_TRI, SSM_SCAN, GATED_DELTA_NET
Signed-off-by: Chun Tao <chun.tao@intel.com >
* Fix abort during test-backend-ops
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com >
* Regenerate ops.md
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com >
* Add scope_dbg_print to newly added SYCL ops.
Also add scope_dbg_print to existing ssm_conv op.
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com >
---------
Signed-off-by: Chun Tao <chun.tao@intel.com >
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com >
Co-authored-by: Chun Tao <chun.tao@intel.com >
Co-authored-by: Todd Malsbary <todd.malsbary@intel.com >
2026-05-07 18:51:33 +03:00
b9afc19cb4
Write a readme on Multi-GPU usage in llama.cpp ( #22729 )
...
* Write a readme on Multi-GPU usage in llama.cpp
* Apply suggestions from code review
Co-authored-by: Johannes Gäßler <johannesg@5d6.de >
* Address review comments
* Apply suggestions from code review
Co-authored-by: Johannes Gäßler <johannesg@5d6.de >
---------
Co-authored-by: Johannes Gäßler <johannesg@5d6.de >
2026-05-07 17:48:40 +02:00
Georgi Gerganov and GitHub
803627f121
llama : remove unnecessary seq_id check during state restore ( #22797 )
2026-05-07 16:37:26 +03:00
pl752 and GitHub
68380ae11b
ggml-cpu: Optimized risc-v cpu q1_0 dot
2026-05-07 21:09:25 +08:00
Pascal and GitHub
cc97e45a14
mtmd: fix whisper audio tail truncation by exposing padded buffer to FFT ( #22770 )
2026-05-07 14:01:01 +02:00
8e52631d55
model: Add Mimo v2.5 model support ( #22493 )
...
* add mimo-v2.5 support
* mimo-v2.5: fix modify_tensors row split
* mimi-v2.5: forgot `add_attn_value_scale` plumbing
* mimi-v2.5: fix tp dequant to detect tp rows
* mimo-v2.5: fix TP iteration to be descending
* mimo-v2.5: fix comment
* mimo-v2.5: retain fused qkv
* mimo-v2.5: missed the attn_value scale during merge
* mimo-v2.5: fused QKV needs contiguous for scaling attention value
* mimo-v2.5: move `speech_embeddings.` to TextModel filter_tensors
* Update src/llama-hparams.h
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update src/models/mimo2.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update src/models/mimo2.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update convert_hf_to_gguf.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update convert_hf_to_gguf.py
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* Update src/models/mimo2.cpp
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
* mimo-v2.5: include MTP weights in gguf
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com >
2026-05-07 13:21:58 +02:00
Pascal and GitHub
f4b5a2ee91
webui: fix ?model= URL param race in router mode ( #22771 )
...
* webui: fix ?model= URL param race in router mode
* chore: update webui build output
2026-05-07 13:09:32 +02:00
Vishal Singh and GitHub
97f06e9eed
codeowners : add ZenDNN backend codeowner ( #22772 )
...
* codeowners : add ZenDNN backend codeowner
* codeowners : fix zendnn owners to use individual github handles
2026-05-07 14:46:51 +08:00
Georgi Gerganov