llama.cpp-turboquant

claudeopus46/llama.cpp-turboquant

Fork 0

457e76fc0e fix: match Ollama's proven gfx1103 approach — gfx1102 target + rocBLAS master b8588 claudeopus46andClaude Opus 4.6 2026-04-03 03:05:06 +02:00
94127d7b33 fix: force MMQ kernels to bypass rocBLAS TensileLibrary on gfx1103 claudeopus46andClaude Opus 4.6 2026-04-03 01:44:34 +02:00
cec946de76 fix: add gfx1103 TensileLibrary alias from gfx1102 claudeopus46andClaude Opus 4.6 2026-04-03 00:49:45 +02:00
7eb4c866ff fix: disable WMMA flash attention for gfx1103 iGPU compatibility claudeopus46andClaude Opus 4.6 2026-04-03 00:32:57 +02:00
1ea0b4798d fix: FWHT butterfly loop warp divergence on RDNA3 iGPU (gfx1103) claudeopus46andClaude Opus 4.6 2026-04-03 00:19:35 +02:00
d1b8f40933 docs: replace README with TurboQuant fork documentation b8583 Pascal WachowskiandClaude Opus 4.6 2026-03-29 20:57:01 +02:00
04b2771048 docs: add TurboQuant benchmark results and documentation Pascal WachowskiandClaude Opus 4.6 2026-03-29 20:32:53 +02:00
bd571adc99 feat: TurboQuant KV-cache quantization for AMD ROCm (turbo3/turbo4) Pascal WachowskiandClaude Opus 4.6 2026-03-29 19:24:47 +02:00
7c203670f8 add missing ROPE_FACTORS_LONG/SHORT for MiniCPM (#21150) Sigbjørn SkjæretandGitHub 2026-03-29 19:45:40 +02:00
ec16a072f0 Optimize MOE GEMV kernel for BS > 1. (#20905)
Gaurav Garg GitHub Aman Gupta
2026-03-29 22:05:18 +05:30
f5d1c4179f hexagon: dma optimizations (mostly fixing regressions) (#21137) Max KrasnyanskyandGitHub 2026-03-29 06:40:13 -07:00
2405d59cb6 devops: including compute-runtime for intel.Dockerfile (#21076) Davi Henrique LinharesandGitHub 2026-03-29 02:34:03 -03:00
afe65aa282 [SYCL] Enhance build script to use half cores to build, avoid OS hang (#21093) Neo ZhangandGitHub 2026-03-29 09:02:45 +08:00
65097181e4 fix **/x glob matching (#21129) Sigbjørn SkjæretandGitHub 2026-03-28 22:27:38 +01:00
98ae0a0d36 common/parser: fix handling of tool definition with missing properties key (#21128) Piotr Wilkin (ilintar)andGitHub 2026-03-28 20:41:32 +01:00
3a14a542f5 common : add character class support to glob_match (#21111) Sigbjørn SkjæretandGitHub 2026-03-28 19:57:37 +01:00
968189729f WebUI: Replace illegal nested button elements (#21026) BlueMöhreandGitHub 2026-03-28 17:57:59 +01:00
e397d3885c common/json-schema: fix: handle non-capturing groups (?:...) in JSON schema pattern converter (#21124) AdrienandGitHub 2026-03-28 17:55:38 +01:00
e6f2ec01ff common : add reasoning_format = none support to gpt-oss (#21094) Aldehir RojasandGitHub 2026-03-28 09:33:39 -05:00
edfb440a2f server : fix processing of multiple back-to-back mtmd chunks (#21107) Georgi GerganovandGitHub 2026-03-28 16:27:36 +02:00
3d66da1809 ci : gracefully shut down the server (#21110) Adrien GallouëtandGitHub 2026-03-28 14:49:57 +01:00
82b703f8bc Document custom default webui preferences in server README (#19771) Woof DogandGitHub 2026-03-28 13:19:16 +00:00
51a84efc53 webui: Conversation forking + branching improvements (#21021) Aleksander GrygierandGitHub 2026-03-28 13:38:15 +01:00
b0f0dd3e51 vendor : update cpp-httplib to 0.40.0 (#21100) Adrien GallouëtandGitHub 2026-03-28 08:59:44 +01:00
0eb4764182 vulkan: add noncontiguous GLU support (#21081) Ruben OrtlamandGitHub 2026-03-28 08:44:56 +01:00
1f5d15e665 common/parser: fix reasoning whitespace bugs + extra parser tests (#21085) Piotr Wilkin (ilintar)andGitHub 2026-03-28 07:29:26 +01:00
c46758d28f cli : add /glob command (#21084) Sigbjørn SkjæretandGitHub 2026-03-28 02:33:04 +01:00
bf934f28db docker : fix and enable ARM64 image build (#20929)
Ts-sound GitHub Aaron Teo
2026-03-28 08:45:09 +08:00
5c1a7b8355 server : add custom socket options to disable SO_REUSEPORT (#21056) Adrien GallouëtandGitHub 2026-03-28 01:12:43 +01:00
59d840209a common : inhibit lazy grammar sampler while reasoning is active (#20970)
Aldehir Rojas GitHub Piotr Wilkin
2026-03-27 12:30:40 -05:00
ff934e29bc server: Introduce LLAMA_BUILD_WEBUI build flag to allow disabling the embedded web ui (#20158) Kusha GharahiandGitHub 2026-03-27 11:25:55 -05:00
ee051c1e4e hexagon: support for IQ4_NL and MXFP4 (#21018) Yiwei ShaoandGitHub 2026-03-27 09:22:41 -07:00
e6f6770515 webui: Improve Chat Messages initial scroll + auto-scroll logic + add lazy loading with transitions to content blocks (#20999) Aleksander GrygierandGitHub 2026-03-27 17:01:36 +01:00
48cda24c11 server: remove the verbose_prompt parameter (#21059) AN LongandGitHub 2026-03-27 19:36:13 +08:00
871f1a2d2f mtmd: add more sanity checks (#21047) Xuan-Son NguyenandGitHub 2026-03-27 11:00:52 +01:00
20197b6fe3 server: add built-in tools backend support (#20898) Xuan-Son NguyenandGitHub 2026-03-27 10:07:11 +01:00
ba38f3becc rpc : proper handling of data pointers to CPU buffers (#21030) Radoslav GerganovandGitHub 2026-03-27 10:59:35 +02:00
37f230dd7c completion : session_tokens insert range in completion tool (no-op → correct) (#20917) mtmcpandGitHub 2026-03-27 05:25:58 -03:00
a308e584ca completion : Fix segfault on model load failure (#21049) mtmcpandGitHub 2026-03-27 05:01:13 -03:00
d0fa2c9fbb Send reasoning content back to the model across turns via the reasoning_content API field (#21036) PascalandGitHub 2026-03-27 08:17:35 +01:00
9bcb4eff4d metal : Fix dimension constraint violation in matmul2d descriptor (#21048) renandGitHub 2026-03-27 00:05:21 -07:00
6861f6509a CANN: update docker images to 8.5.0 and improve CANN.md (#20801) KokerZhouandGitHub 2026-03-27 08:53:00 +08:00
1743d98057 mtmd: fix "v.patch_embd" quant and unsupported im2col ops on Metal for deepseek-ocr (#21027)
Saba Fallah GitHub Sigbjørn Skjæret
2026-03-27 00:07:55 +01:00
7ca0c9cca7 hip: use fnuz fp8 for conversion on CDNA3 (#21040) uvosandGitHub 2026-03-26 23:06:33 +01:00
8c60b8a2be ci: pin external actions to exact commit SHA (#21033) Xuan-Son NguyenandGitHub 2026-03-26 20:44:00 +01:00
287b5b1eab common : add getpwuid fallback for HF cache when HOME is not set (#21035) Adrien GallouëtandGitHub 2026-03-26 20:34:23 +01:00
a73bbd5d92 mtmd: refactor image preprocessing (#21031) Xuan-Son NguyenandGitHub 2026-03-26 19:49:20 +01:00
ded446b34c opencl: allow large buffer for adreno (#20997) lhezandGitHub 2026-03-26 08:52:21 -07:00
f8d4abae86 convert : support Qwen3.5/Qwen3.5 Moe NVFP4 and add input scales (#20505)
Michael Wand GitHub Sigbjørn Skjæret
2026-03-26 08:52:06 -07:00
3d5acab3e7 convert : add RuGPT3XL (RuGPT3XLForCausalLM) support (#21011)
Pavel Zloi GitHub Sigbjørn Skjæret
2026-03-26 18:49:09 +03:00
9900b29c3a common : filter out imatrix when finding models (#21023) Adrien GallouëtandGitHub 2026-03-26 15:37:18 +01:00
dc8d14c582 fix(ggml): correct RISC-V ISA string canonical ordering for RVV in CMake (#20888) ihb2032andGitHub 2026-03-26 19:08:41 +08:00
93dfbc1291 common : make LLAMA_CACHE the one cache for everything (#21009) Adrien GallouëtandGitHub 2026-03-26 12:04:57 +01:00
3cba8bba18 common : fix split model migration (#21019) Adrien GallouëtandGitHub 2026-03-26 12:04:37 +01:00
112c78159f ggml-cuda: Add NVFP4 dp4a kernel (#20644)
Michael Wand GitHub Johannes Gäßler
2026-03-26 01:54:03 -07:00
0fac87b157 imatrix : fix crash when using --show-statistics with zero counts (#19532) SamareshSinghandGitHub 2026-03-26 02:14:36 -05:00
0a524f2404 CUDA & CPU: support F32 kernel type for CONV_TRANSPOSE_2D (#17094)
Yihao Wang GitHub Aman Gupta
2026-03-25 19:19:14 -07:00
c0159f9c1f common : do not delete old files from the old cache when updating (#21000) Adrien GallouëtandGitHub 2026-03-25 22:28:04 +01:00
a970515bdb mtmd: Add DeepSeekOCR Support (#17400)
Saba Fallah GitHub Sigbjørn Skjæret bluebread Xuan Son Nguyen Xuan-Son Nguyen
2026-03-25 19:57:40 +01:00
056b50c319 common : fix verbosity setup (#20989) Adrien GallouëtandGitHub 2026-03-25 19:41:01 +01:00
f2c72b8f1f common : fix gguf selection in common_list_cached_models (#20996) Adrien GallouëtandGitHub 2026-03-25 19:18:06 +01:00
ec54ac13a8 ci : fix parsing of vgpr counts in hip-quality-check (#20987) uvosandGitHub 2026-03-25 19:00:37 +01:00
80322ebdaf model: codefuse-ai/F2LLM-v2 support Saba FallahandGitHub 2026-03-25 18:33:42 +01:00
44c51e526b model : allow causal_attn and pooling_type on all architectures (#20973) DowonandGitHub 2026-03-26 02:12:38 +09:00
1922f87c2f snapdragon: add missing features to WoS scripts to achieve parity with ADB scripts (#20884)
Aparna M P GitHub Max Krasnyansky
2026-03-25 22:13:12 +05:30
345de3cd87 Use docker in build-android.yml (#20928)
Shreya Jain GitHub Sigbjørn Skjæret Max Krasnyansky
2026-03-25 09:36:27 -07:00
9c600bcd4b llama-bench: print -n-cpu-moe when offloaded layers > 1 (#20984) Aman GuptaandGitHub 2026-03-25 21:17:27 +08:00
b2704f9028 ci: Allow ninja to be used during unit test (#20742)
Masato Nakasaka GitHub Aaron Teo Aaron Teo
2026-03-25 06:00:49 -07:00
3fab96cd04 ci : disable self-hosted mac jobs (#20985) Georgi GerganovandGitHub 2026-03-25 14:46:40 +02:00
914eb5ff0c jinja: fix macro with kwargs (#20960)
Xuan-Son Nguyen GitHub Sigbjørn Skjæret
2026-03-25 12:22:48 +01:00
8fc17493c3 gguf-split : clarify operation of gguf-split (#19749) Francisco HerreraandGitHub 2026-03-25 06:12:50 -05:00
36dafba5c4 llama: fix llama-model-saver (#20503)
Johannes Gäßler GitHub Siddhesh2377
2026-03-25 11:53:16 +01:00
69e0ecef06 webui: Fix editing assistant message without branching (#20944) Aleksander GrygierandGitHub 2026-03-25 11:47:33 +01:00
062cca58fc Add SLEEPING status to the WebUI model selector (#20949)
Pascal GitHub Aleksander Grygier
2026-03-25 11:02:32 +01:00
406f4e3f61 android : fix-pointer-dangling (#20974) yikechayedanandGitHub 2026-03-25 17:51:26 +08:00
53dc8b59bf sycl : fix wrong variable check by assert (#20903) Neo ZhangandGitHub 2026-03-25 17:48:37 +08:00
403c9c9cef ci : bump gguf publish python version (#20982) Sigbjørn SkjæretandGitHub 2026-03-25 10:04:59 +01:00
8fc85db9d2 ci : limit requirements versions (#20980) Sigbjørn SkjæretandGitHub 2026-03-25 09:55:37 +01:00
3a60d06ad9 convert : register Qwen3Model architecture (#20967) DowonandGitHub 2026-03-25 17:37:59 +09:00
abd86ef175 docs : Update OpenVINO backend docs (#20968)
Ravi Panchumarthy GitHub Aaron Teo
2026-03-25 01:33:51 -07:00
9f102a1407 models : move the token embedding norms to the first layer (#20943) Georgi GerganovandGitHub 2026-03-24 17:00:30 +02:00
3fc6f1aed1 ggml-backend: re-enable graph reuse with pipeline parallelism (#20927) Aman GuptaandGitHub 2026-03-24 20:47:00 +08:00
29771a0a4c vendor : update cpp-httplib to 0.39.0 (#20933) Alessandro de Oliveira Faria (A.K.A.CABELO)andGitHub 2026-03-24 09:33:33 -03:00
42ebce3beb common : fix get_gguf_split_info (#20946) Adrien GallouëtandGitHub 2026-03-24 13:33:14 +01:00
a94fdb090a WebUI: fix edit msg form textarea height (#20830) BlueMöhreandGitHub 2026-03-24 13:17:45 +01:00
c9dc43333f readme : clarify MODEL_ENDPOINT usage (#20941) Adrien GallouëtandGitHub 2026-03-24 10:35:07 +01:00
2d2d9c2062 common : add a WARNING for HF cache migration (#20935) Adrien GallouëtandGitHub 2026-03-24 09:24:39 +01:00
92080b4396 metal : add FLOOR, CEIL, ROUND, TRUNC unary ops (#20930)
nuri GitHub nryoo
2026-03-24 17:13:07 +09:00
342d6125bc metal : add FA instantiations for HSK=512, HSV=512 (#20902) Georgi GerganovandGitHub 2026-03-24 10:03:09 +02:00
c2e224d829 issues: add openvino backends (#20932) Aaron TeoandGitHub 2026-03-24 14:41:10 +08:00
8c7957ca33 common : add standard Hugging Face cache support (#20775) Adrien GallouëtandGitHub 2026-03-24 07:30:33 +01:00
e852eb4901 llama-fit: fix regex pattern for gate_up tensors (#20910)
Aman Gupta GitHub Johannes Gäßler
2026-03-24 12:57:57 +08:00
312d870a89 common : replace wrap_for_generation with a prefix convenience function and fix gpt-oss (#20912) Aldehir RojasandGitHub 2026-03-23 22:21:47 -05:00
7cadbfce10 hexagon: general DMA and Binary Op fixes for large strides (#20918) Max KrasnyanskyandGitHub 2026-03-23 15:33:49 -07:00
1fb2290a51 Add codeowners for scripts/snapdragon and docs/snapdragon (#20915) Max KrasnyanskyandGitHub 2026-03-23 14:57:18 -07:00
1772701f99 opencl: add q6_K gemm and gemv kernels for Adreno (#20089) lhezandGitHub 2026-03-23 12:44:18 -07:00
39bf0d3c6a rpc : RCE patch (#20908) las7andGitHub 2026-03-23 10:54:57 -07:00
bd6992180b contrib: add "Requirements" section to PR template (#20841)
Xuan-Son Nguyen GitHub Piotr Wilkin
2026-03-23 16:59:02 +01:00
fd18364755 devops: upgraded default oneAPI version (#20731) Davi Henrique LinharesandGitHub 2026-03-23 10:47:34 -03:00
11fb11b901 webui: Improve chat form positioning (#20901) Aleksander GrygierandGitHub 2026-03-23 14:30:55 +01:00

1 2 3 4 5 ...

Commit Graph Select branches Hide Pull Requests master b8583 b8588 Mono Color

Commit Graph

Select branches

Hide Pull Requests

master

b8583

b8588