mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-09-07 16:37:57 +02:00
AnjielonandGitHub
8fe90e1fbf
* vulkan: add TQ1_0 support (mm, mat-vec, dequant, get_rows) * vulkan: pack TQ1_0 powers of 3 into a 32-bit constant Replaces the constant array with a packed 32-bit value (7 bits per entry, max 81 < 128) extracted with shift/mask, as suggested in review — avoids a constant array that may not be kept in registers. test-backend-ops on gfx1151: tq1_0 MUL_MAT 11/11, MUL_MAT_ID 6/6, GET_ROWS 4/4, unchanged. * vulkan: address review - shared TQ1_0 decode helpers, fix standalone dequant shader Review feedback from jeffbolznv, all points: - Move the packed-pow3 decode into shared helpers in types.glsl (tq1_0_byte_of / tq1_0_digit_of / tq1_0_trit) and use them from dequant_funcs.glsl, mul_mm_funcs.glsl, dequant_funcs_cm2.glsl and dequant_tq1_0.comp instead of repeating the logic. The cm2 path also drops its constant array for the packed-constant extraction. - Translate all remaining comments to English. - dequant_tq1_0.comp: use dequant_head.glsl. The shader previously declared its own single-field push constant while the pipeline is created with the 5-field layout, so p.ne read the wrong field - confirmed broken, as suspected in review. - Fix wg_denoms for the standalone dequant pipeline: one invocation decodes 4 elements with local_size 256, so a workgroup covers 256*4 elements, not 256*16. With the old value the dispatcher launched a quarter of the required workgroups. Verified by temporarily forcing the dequant + f16 matmul path for TQ1_0 (hack not committed): test-backend-ops MUL_MAT passes through the rewritten standalone shader, and the standard MUL_MAT / MUL_MAT_ID / GET_ROWS tq1_0 cases still pass on Vulkan (AMD gfx1151). * vulkan: address review — English comments, shared tq1_0_trit, trim TQ1_0 test cases - mul_mat_vec_tq1_0.comp: drop leftover non-English comment and the local POW3_PACKED constant; all decode sites now call tq1_0_trit() from types.glsl - types.glsl / dequant_funcs_cm2.glsl: ASCII-only, drop stale reviewer note - test-backend-ops: remove the oversized MUL_MAT_ID case (432 MiB A tensor, ~172 GFLOP reference); move the two remaining ones next to the other backend-specific mul_mat_id one-offs and document why they are needed * metal: decline TQ1_0 for GET_ROWS and mat-mul in supports_op The new TQ1_0 cases in test-backend-ops exposed that the Metal backend claimed support for GET_ROWS/MUL_MAT/MUL_MAT_ID with TQ1_0 sources while having no such kernels (ggml_metal_library_compile_pipeline aborted on the missing kernel_get_rows_tq1_0). Decline the type so the ops fall back to the CPU, matching the existing NVFP4 handling on the same lines. Assisted-by: Claude Fable 5 * vulkan: trim the TQ1_0 comments Addresses @0cc4m's review: keep only what the code does not already say. Removed the block-format recaps (the layout is right there in the struct) and the step-by-step decode walkthrough. Kept the two facts a reader cannot infer: the 8-bit truncation is part of the format, not an optimisation, and the powers of 3 are packed into one uint so they do not end up in a constant array that may miss the registers. No functional change. * vulkan: address review — trim comments, fold Metal check, drop unused _v Per @0cc4m's review: - dequant_funcs.glsl, dequant_funcs_cm2.glsl: drop the "see types.glsl" pointers — they apply to every quant and say nothing specific. - dequant_tq1_0.comp: drop the wg_denoms note. It is a precondition, not information. - mul_mm_funcs.glsl: same pointer removed. - types.glsl: the comment on tq1_0_trit is down to the one fact the code cannot show — the 8-bit truncation is part of the format, matching the C reference, not an optimisation. - dequant_funcs_cm2.glsl: removed dequantFuncTQ1_0_v and its define. You were right that it is optional: it wrapped four scalar decodes and vectorised nothing, and mul_mm_cm2.comp already guards the path with `#if defined(dequantFuncA_v)` (DATA_A_F32 omits it the same way). - ggml-metal-device.m: folded TQ1_0 into the existing NVFP4 check instead of a separate block, and dropped both comments. - test-backend-ops.cpp: the two mul_mat_id cases stay — they cover the block-stride loop and the per-expert base offset that k == 256 alone never reaches — but the comment is now one line instead of five. Kept: the one-line labels on the three block regions in mul_mat_vec_tq1_0.comp and on tq1_0_byte_of(). Those state the 5-trits-per-byte packing, which the loop bounds do not show. Happy to remove them too if you prefer. Re-verified on AMD gfx1151 (Vulkan), test-backend-ops, 2/2 backends passed: MUL_MAT 9 TQ1_0 cases, MUL_MAT_ID 5, GET_ROWS 4 — all OK, no failures. The coopmat2 path is unchanged apart from the removed _v define.