mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-09-15 18:13:29 +02:00
Foad Abo DahoodandGitHub
aac810230f
* metal : fix idle threads in the remaining iq mul_mv kernels for ne00 < 1024 Generalize the row split from #28086 to the six other kernels that use the same lane-to-block mapping: iq1_s, iq1_m, iq2_xxs, iq2_xs, iq2_s and iq3_s. Each of them assigns one 32-element chunk per thread, so when a row has fewer than 32 chunks the rest of the simdgroup is idle. When nb32 < 32 and nb32 divides 32, 32/nb32 threads now share each chunk and each takes a slice of the rows, reusing the FC_mul_mv_split function constant and the dispatch wrapper introduced for iq3_xxs. The plain path is untouched: wide matrices keep one thread per chunk and N_R0_<TYPE> = 4. Only the split path uses N_R0_<TYPE>_SPLIT = 8. The K-quants have the same idle-thread issue but a different lane mapping, so they are left for a separate change. * metal : offset the src0 row pointer once in the iq mul_mv kernels q2, dh, sc, qh and signs are all derived from xr, so the row slice offset only has to be applied to xr. * metal : fold iq mul_mv row split into offset0 Compute row0 and row1 before initializing the source pointers and apply the row slice directly to offset0. This keeps x and its derived pointers on the existing path while applying the split row offset once.