mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-09-15 18:13:29 +02:00
Adjusts the thread/block count to be proportional to the size of the quant, reducing under/over subscription. Largest perf improvement is the q4_0 -> f32 path, with, on a Arc 70, throughput goes from 20.21 GB/s to 158.19 GB/s The rest of the quants are flat in performance uplift.