* vulkan: add hoisting support for row IDs and expert count in shaders
* use hoisted row ids in coopmat2
* vulkan: address review feedback on count_experts
- use vk_op_count_experts_push_constants instead of a raw uint vector
- apply the fastdiv trick to the ne00 div/mod in count_experts
- compute the per-expert offsets with subgroupExclusiveAdd when the
device supports it, keeping the serial path as fallback
- document the data_d layout and the hoisted_row_id_words bound
- drop a leftover debug print in ggml_vk_matmul_id
* vulkan: use init_pushconst_fastdiv for count_experts push constants
* vulkan: refine comments for row ID hoisting and data layout in count_experts shader
* Whitespace
---------
Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
This adds fa_vec_tuned_table records for Apple M4 to ggml-metal-tuning.cpp.
Includes F16, Q4_0, Q4_1, Q5_0, Q5_1, and Q8_0. (M4, 10 GPU Cores)
Co-authored-by: Strongtut <8432058+Strongtut@users.noreply.github.com>
* OpenVINO Backend: Fuse IM2COL + MatMul convolution into OpenVINO convolution
* ci:ggml-ov: Skip recurrent state rollback tests
* ci:ggml-ov: Skip recurrent state rollback tests
* Update OPENVINO.md
* ggml-openvino : add env-var gated op support debugging
* Fix ggml_rope_set_offset case
* OpenVINO backend: Support Whisper.cpp
* Fix code style
* openvino : enable qwen35 on NPU
Static shapes:
- get_graph_input_shape() left the s_copy / s_copy-leaf inputs dynamic
([1,1,1,-1]) even in static mode, which propagated a dynamic slot dim through
GET_ROWS into the conv/GDN state, the state reshapes and the GDN output.
- With -np 1 the s_copy defrag remainder gathers zero rows; short-circuit that
CPY to the untouched cache instead of emitting a degenerate Slice/Concat, and
skip binding its zero-byte ggml tensor as an output (the dynamic path already
did the latter, the static path wrote the full cache over a 0-byte buffer).
Token-count independence:
- In static mode the compiled model's token count is the prefill chunk size or
1, not the captured cgraph's. Offsets derived from the captured count were
therefore wrong. Anchor the GDN state slice at the end of the packed
[attn | state] output and drop the rs_src_begin runtime inputs, and make
VIEWs over the GDN output / conv_input pass through so the consumer does the
slicing.
- CONT could not identify its token axis when the graph was captured with a
single token (every trailing dim has the same stride and size 1) and baked
the captured shape into the prefill model.
Chunked prefill:
- The last chunk is padded with fabricated tokens. Attention masks them, but
the recurrent path folded them into cache_r/cache_s permanently. Add a
chunk_valid_len runtime input, use it to zero g and beta for padded steps
(making the recurrence an exact identity) and to end the conv snapshot window
at the last valid token, and disable the recurrent-cache reset after the
first chunk so earlier chunks are not wiped.
- get_is_prefill() and the chunk loop bound read inp_pos->ne[0] directly, but
IMROPE stacks 4 position planes, so every decode step was run through the
padded prefill model and the loop ran extra out-of-bounds chunks.
cache_rs_reset_idx/len now stay runtime Parameters in static mode, since
can_reuse_statically() does not invalidate the cached model on ComputeParams
changes. Add GGML_OPENVINO_FORCE_STATIC to exercise the static path on CPU.
* Update to OpenVINO 2026.3.1
* ggml-openvino: forward NPU compilation mode parameters
Add GGML_OPENVINO_NPU_COMPILE_CONFIG to the backend's cached environment so callers can configure the NPU compiler without using the generic property escape hatch.
When the value is non-empty, pass it to OpenVINO as NPU_COMPILATION_MODE_PARAMS. This enables settings such as optimization-level=3 for NPU compilation while preserving the existing behavior when the variable is unset and leaving CPU and GPU configuration unchanged.
Document the variable, its NPU-only scope, and the optimization-level=3 example in the OpenVINO backend runtime configuration table.
* ggml-openvino : support RELU, POOL_2D, QUICK_GEGLU, and ROLL ops
* reorder op table
* exclude GPU/NPU failing POOL_2D case
* move op type detection to compute_op_case
* Relax rope supported cases
* Fix pool case
* Update openvino doc, gpu driver in ov docker
* openvino: remove unused static remote context branch
* openvino: parallelize static model build
* Apply editorconfig
---------
Co-authored-by: Mostafa Faheem <mostafaaafaheem@gmail.com>
Co-authored-by: Ravi Panchumarthy <ravi.panchumarthy@intel.com>
Co-authored-by: zhaixuejun1993 <xuejun.zhai@intel.com>
Measured at a live KV length of 34816 (32768 depth plus one 2048 ubatch),
on Qwen3.8 27B Q4_K_S:
per tensor 4 * 34816 * 256 * 2 B = 71.3 MB
staged per call K and V, so 2x = 142.6 MB
traffic per call read once, write once = 285.2 MB
traffic per ubatch 285.2 MB * 16 calls = 4.56 GB
One ubatch is one ggml_cgraph submission (llama_context::process_ubatch ->
graph_compute), so that 4.56 GB is the cost of a single 2048-token prefill
chunk, and it scales with the live KV length: the first ubatch of the same run,
at seq = 2048, moves 0.27 GB.
Reproduce the two measured inputs with:
GGML_SCHED_DEBUG=2 llama-bench -m MODEL -p 8 -n 0 -r 1 -ngl 0 \
-fa on -ctk f16 -ctv f16 -v > nd.txt 2>&1
grep -E 'n_layer|n_head_kv|n_embd_head_k' nd.txt
awk '/node # 0 /{g++} g==1 && /\(FLASH_ATTN\)/{n++} END{print n+0}' nd.txt
* metal : add fa-vec tunings for M5
This is a followup contribution to efeda76b94 as requested in https://github.com/ggml-org/llama.cpp/discussions/27668 to add support for additional Apple GPUs. I generated this output using the provided instructions:
```sh
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_METAL=ON
cmake --build build --target ggml-metal-tuning -j
./build/bin/ggml-metal-tuning fa-vec --dtype f16,q8_0 > fa_vec_rows.txt 2> fa_vec_sweep.log
```
This ran on a machine with Apple M5.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* metal : add fa-vec tunings for M5 Pro
This adds fa_vec_tuned_table records for Apple M5 Pro to ggml-metal-tuning.cpp.
Contributed by SerayaEryn in https://github.com/ggml-org/llama.cpp/discussions/27668#discussioncomment-18157544 (F16, Q4_0, Q8_0; M5 Pro, 20 GPU cores).
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* metal : add fa-vec tunings for M3 Max
This adds fa_vec_tuned_table records for Apple M3 Max to ggml-metal-tuning.cpp.
Contributed by TeeAaTeeUu in https://github.com/ggml-org/llama.cpp/discussions/27668#discussioncomment-18175220 (F16, Q8_0; M3 Max, MacBook Pro 64GB, low power mode).
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* cont : whitespaces
This is a followup contribution to efeda76b94 as requested in https://github.com/ggml-org/llama.cpp/discussions/27668 to add support for additional Apple GPUs. I generated this output using the provided instructions:
```sh
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_METAL=ON
cmake --build build --target ggml-metal-tuning -j
./build/bin/ggml-metal-tuning fa-vec --dtype f16,q8_0 > fa_vec_rows.txt 2> fa_vec_sweep.log
```
This ran on a MacBook Pro (14-inch, Nov 2024) with Apple M4 Pro. The `ggml-metal-tuning` command completed successfully in 1h 13m 1s with no other notable load on the system.
Add HVX-accelerated implementations for GGML_OP_LOG and
GGML_UNARY_OP_ABS on the HTP backend.
- Register HTP_OP_UNARY_ABS and HTP_OP_UNARY_LOG in op_remap_to_htp()
- Add ABS and LOG to ggml_backend_hexagon_device_supports_op()
- Implement hvx_abs_f32_aa() in hvx-arith.h using hvx_vec_abs_f32()
- Implement hvx_log_f32_aa() in hvx-log.h using hvx_vec_log_f32()
- Add abs_f32() and log_f32() row-wise dispatch in unary-ops.c
- Define tiled and non-tiled task functions via DEFINE_UNARY_TASK and
DEFINE_UNARY_TILED_TASK macros
- Route HTP_OP_UNARY_ABS and HTP_OP_UNARY_LOG through execute_op()
in main.c
* hexagon: use non-host bufs by default and make the backend fully async
* hex-hb: remove optional hostbuf support and fix async copy
* hex-unary: relax supported unary check
* hex-bufs: use same get_alignment for host bufs
* snapdragon: bump android_platform to 34
* hex-rows: super hacky get/set rows for q8_0
* hex-get-rows: fix q8_0
* hex-get-rows: supprot for f16 and cleanup for q8_0
* hex-get-rows: generic macros and specialized thread funcs
* hex-get-rows: add DMA pipeline, vtcm_layout and kernel params
* hex-set-rows: fix q8_0 support, add dma and tracing
* hex-tests: override nmse threshold for HTP of Q8_0 quants
* hex-fa: add support for Q8_0 with inplace dequantizers
* hex-get-rows: simplify type dispatch
* hex-rows: simplify GET/SET_ROWS DMA pipeline
* hex-async: add events, set/get-tensor-async and rest of the async api support
* hex-repack: use slice instead of expert in repack functions
* hex-cpy: update event/async-cpy logging
* hex-set-rows: optimize smaller tensors
* hex-geglu: fix perf regression with larger tensors
* hex-get-rows: add missing header
* hex-set-rows: add missing header
* hex-bufs: ressurect GGML_HEXAGON_HOSTBUF but disable it by default
* hexagon: do not reject ops with non-heaxon buffers
* hex-get-rows: apply >=32 restriction only for q8_0
* hex-res: bump vtcm acquire timeout to 10 seconds
* hex-bufs: add support for cloning buffers between sessions to speed up tensor copies
* hex-async: rework event recording and batch flushing and integrate with meta backend
* hex-bufs: improved handling of repacked tensors
* hex-repack: handle get_tensor_2d offsets
* hex-dev: add support for devices with multiple NPUs
* hex-sync: add support for sync tokens to synchronize npu devices for async splits
* hex-mmap: cleanup mmap calls and add a retry for robustness
* hex-sync: add failsafe if sync wait gets stuck
* hex-sync: use sync_seq to check for completed events
* hex-sync: rotate tokens for extra robustness
* hex-devs: add supprot for legacy device names for now
* hex-bufs: add support for auto-cloning buffers from diff sessions
* hex-fusion: simplify and optimize htp-opnode fusion handling
* hex-sync: override opnode name so that it shows up in the profiles
* hex-trace: update scripts to handle multiple devices
* hex-sync: bump the size of the opbatch queue and number of sync tokens
* hex-cpy-sync: do not explicitly flush opbatches in cpy_tensor_async and add support for cpy-dma
* hex-sync: add graph-flush threshold to avoid single op batches
* hex-sync: add sync_peer so that we can flush peers we depend on during cross-device ops
* hex-bufs: introduce tensor->extra and shadow_bufs for repacking
* hex-l2: flush tiny tensors inline
* hex-sync: use explicit l2flush for sync tokens
* hex-extra: track weight flags via tensor extra
* hex-fence: rename sync to fence
* hex-repack: proper handling of set-tensor-2d in the shadow_buf
* hex-trace: remove obsolete opstage mask that we used for profiling
* hex-env: remove obsolete use_hmx variable
* hexagon: new unified run.py and build.py and updated docs
* snapdragon: update run script to auto-escapt test-backend-op -p argument
* hex-scripts: fix trailing spaces
* hex-scripts: fix flake8 warnings
* snapdragon: cleanup dst lib/bin dirs before copying new build
* hex-ops: add support for allreduce
* hex-ar: improved allreduce with dma pipeline
* hex-ar: align macros
* hex-ar: consistent use of fence_seq
* hex-ar: add AR_SELECT env var to select ALLREDUCE kernel or fallback
* hex-ar: add proper synchronize handling for ALLREDUCE
* hex-opbatch: looks like we now just rely on backend.synchronise to flush the batches, no need to flush them by threshold
* hex-ar: bump block size to improve dma efficiency
* hex-ar: fused ALLREDUCE+ADD
* hex-ar: cleaner fence buffer management
* hex-ar: futher allreduce tweaking to remove race conditions
* hex-ar: add simple solver and remove non-dma kernels
* hex-ar: add row-broadcast to fuse with bias ADD
* hex-fence: pass seq numbers via op_params
* hex-ar: allow for both entry/exit seq for completing entry wait
* hex-ar: align macros
* hex-ar: do not refetch broadcast row
* hex-fusion: move all fusion into opbatch::add_op for consistency with ALLREDUCE and things
* hex-fusion: fix incorrect MUL_MAT reordering
* hex-mm: make fused 2x and 3x matmuls more generic
* hex-fusion: move tensor fusion tagging to graph_compute
* hexagon: make sure to copy tensor->extra by value
* hex-get-rows: fix offset calc with row-chunking
* hex-repack: get_tensor_2d fixes for non-zero offsets
* snapdragon: make profile/trace scripts more robust and donot mix stdout/stderr by default
* hex-devices: use legacy device nameing by default to ease the transition
* hex-devices: hardcode CDSP domain IDs for current devices for now
* hex-optrace: improve multi-NPU timestamp alignment and overall handling of cycle values
* hex-optrace: more robust handling of the fence events
* cuda: unblock mmq for MoE on sm_60
* cuda: duplicate mmq-config-pascal for dp4a and older
* cuda: reduce occupancy on non-dp4a pascal for Q2_K, Q4_K, Q5_K, Q6_K
* metal: WIP chunked SSD SSM_SCAN kernels for multi-token prefill
* metal: drop scalar SSD path; MMA + sequential tail
* drop WIP ssm scan test noise
* remove state_from_dst and rename CS and NSG constants
* remove unrelated added whitespace padding
* added clarity to mma_tokens calculation
* added clarity to use_mma bool checks
* added comments to metal ssd op constants for clarity
* reserve K tokens for sequential kernel rollback snapshots
* reset concurrency between mma and seq tail
* remove print args no longer used
* fixed comment to no longer point to specific line
* add FC_SSM_SCAN so seq path skips token offlset unless it's mma tail
* added changes to new ssm.metal for rebase after ggml-metal.metal refactor
* specialize ssm_scan tail with a template instead of a function constant
---------
Co-authored-by: dpantaleoni <dominikpantaleoni@gmail.com>
Co-authored-by: forforever73 <690105611@qq.com>
* rpc: support apple RDMA as an RPC transport
* remove set_tensor micro optimization, rpc socket pinning per CR
* remove transparent reconnect
* trigger apple builds on RPC changes
---------
Co-authored-by: Ryan Churaman <rschu@meta.com>
* metal : null-check ggml_metal_buffer_init result to avoid OOM crash
ggml_backend_metal_buffer_type_alloc_buffer used the result of
ggml_metal_buffer_init without checking for NULL. ggml_metal_buffer_init
returns NULL when the underlying Metal allocation fails (e.g. an
out-of-memory condition), and the following ggml_metal_buffer_is_shared(res)
call dereferences it, turning a recoverable allocation failure into a hard
crash (EXC_BAD_ACCESS). This is easy to hit on memory-constrained devices
such as iOS when a model/context exceeds the available Metal budget.
Log the failure using the existing GGML_LOG_ERROR convention and return
NULL so the allocator surfaces a diagnosable error up the stack instead of
crashing.
* cont : fix log
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* metal : per-device tuned (Q, NE) for flash-attn vec (#25750)
* rebase Q-generic FA vec body from 01dc93607 (#23114)
* add 53 f16 (Q,NE) flash-attn vec instantiations (vec 80 -> 133)
* add FA vec (Q,NE) tuning table + dispatch wiring + SMEM cap fallback
* add FA vec (Q,NE) perf sweep
* fill tuning result
* fold family table into a per-family representative SKU
* refactor tuning result format
* extend FA vec tuning to quantized KV caches
* sync fa vec tuner bucketing with runtime, use pointwise tuning regret
* update tuned table
* format and cleanup
* prefix fa_vec tuning procs with ggml_backend_metal_tuning_, drop unused fa_vec_override_active
* add device id -> token lookup for the offline tuning tool
* add ggml-metal-tuning skeleton
* add op-agnostic perf cell + median timing for the tuner
* add FA-vec graph build + tensor init to the tuner
* tools : add FA-vec (Q,NE) sweep, compression and table emit
* cool down and re-measure the dirty window on thermal drift
* test-backend-ops : replace the FA vec tune mode with a bounded (Q,NE) slice
* tools : document the Metal tuner, point the table comment at it
* abort on unknown KV type, single-source fa_vec_legal_ne
* cleanup
* honor -o in the FA vec (Q,NE) slice
* retune FA-vec (Q, NE) under a pointwise no-harm gate
* cont : add fa-vec tunings for M1 Pro, M2 Ultra, M5 Max
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* metal : per-op source split + parallel compile (#24021)
* preliminary extract common header
* op source split
* split metallib into 8 libs && load in parallel
* derive kernel->library routing from functionNames
* x-macro lib list + underscore filenames, dedup QK_NL, MRC fixes
* op source split 8 to 20
* improve robustness of source fallback
* clean up
* change bool -> atomic_bool
* only prepend headers that source actually includes
* no semaphore, use GCD global queue
* dedup library compile path, fix NSError lifetime, rename gla
* relocate upstream concat/rope_back/repeat kernel changes into split files
* move ggml-common.h from common.h into dequantize.h to shrink binary size
---------
Co-authored-by: lvyichen <lvyichen@stepfun.com>
* metal: add col2im_1d op (f32/f16/bf16) (#25176)
* metal : add set_rows with src0 f16 (#25434)
* metal : add CONV_2D_DW (depthwise convolution) support (#21565)
* metal : add Q2_0 support (#25419)
* metal: fuse snake activation (mul, sin, sqr, mul, add) (#25459)
* ggml-metal: FWHT kernel for metal backend (#25924)
* metal : port new kernels into the split sources
Move the kernels added on master after the split (lightning indexer,
DSv4 hyper-connections, silu_back, f16 bin ops, TQ2_0, the flash-attn KV
dequantization pass, rope offset/inplace, ssm_scan rollback, packed q8_0
dequantization and the tensor-API mat-mat K clamp) into the corresponding
kernels/*.metal sources. Copied verbatim, no functional change.
---------
Co-authored-by: lvyichen <lvyichen@stepfun.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* DSV4: sm tensor
* set coarser granularity for head splits
* fix dspark
* add model saving for dsv4 + allow dflash to return on specific device
* add comment about dsv4 seq_rm
* simplify
* add shared expert delayed allreduce
* remove special test for dsv4
* opencl: fold the gpt-oss MoE bias adds into swiglu_oai
Default on, opt out with GGML_OPENCL_FUSE_MOE_BIAS_GLU=0.
* opencl: fold the MoE down-projection bias into the combine
Default on, opt out with GGML_OPENCL_FUSE_MOE_BIAS_COMBINE=0.
* Add DMMV Q4_K and Q6_K ESIMD kernels
Configure cmake build with -DGGML_SYCL_ESIMD=ON to enable.
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Refactor ESIMD kernels to share common code
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Move control of ESIMD from compile to runtime
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Use ESIMD by default when available
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Fix possible error when using ESIMD by default
While not an issue in the current version, this will become an
issue when additional QK ESIMD kernels are added (such as Q2_K).
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Add explicit unroll to ESIMD kernels
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Tidy up ESIMD kernels a bit
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Add a reordered Q2_K MMVQ kernel
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Add DMMV Q2_K ESIMD kernel
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
---------
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Add DMMV Q4_K and Q6_K ESIMD kernels
Configure cmake build with -DGGML_SYCL_ESIMD=ON to enable.
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Refactor ESIMD kernels to share common code
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Move control of ESIMD from compile to runtime
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Use ESIMD by default when available
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Fix possible error when using ESIMD by default
While not an issue in the current version, this will become an
issue when additional QK ESIMD kernels are added (such as Q2_K).
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Add explicit unroll to ESIMD kernels
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Tidy up ESIMD kernels a bit
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Add DMMV Q5_K ESIMD kernel
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* Remove redundant copyright notice
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
---------
Signed-off-by: Todd Malsbary <todd.malsbary@intel.com>
* opencl: keep the vocab-scale K-quant lm_head on the CPU on the Adreno A7X
* opencl: revise comments
---------
Co-authored-by: Li He <lih@qti.qualcomm.com>
The Tensor API mat-mat path of kernel_mul_mm (GGML_METAL_HAS_TENSOR) fed a
static K=32 tile to the matmul2d op on every iteration. On the last, partial
K tile (ne00 % 32 != 0) the src1 slice extends past the K extent of the
tensor, and the op reads those out-of-bounds elements (undefined behavior per
the MSL specification, section 2.22.2). Depending on stale memory contents,
this corrupted the result or produced NaN.
Make the matmul2d op use dynamic_extent for K, and clamp the K extent of both
operand tensor views to the remaining valid K range (min(32, K - loop_k)) per
iteration, so the op reads exactly the valid K range on every iteration
(mirroring the tail handling of the MPP matmul2d examples). On K-aligned
inputs the clamp degenerates to the full 32-wide tile: the only difference
from the static-K op is that the dynamic-K op derives K from the operand
extents and edge-checks the tile against the tensor extents (a handful of
integer ops per iteration).
Add test-backend-ops MUL_MAT cases with K not a multiple of 32 to exercise
the unaligned K path.
Assisted-by: pi:llama.cpp/Qwen3.8-27B
* opencl: decline KV-convert flash_attn variants on Adreno A7X (compiler SIGSEGV)
The Adreno 740 (A7X) compiler E031.41 crashes inside clBuildProgram when
building the flash_attn programs whose KV path is mixed-type or dequantized:
flash_attn_f32_f16, flash_attn_f32_q8_0, flash_attn_f32_q4_0. It is a driver
crash rather than a compile-error return, so build_program_from_source_ex()
cannot catch it. The uniform f32 and f16 programs build correctly.
Decline the three KV-convert variants on the A7X in supports_op so they never
lazy-compile; those attention layers run on the CPU backend instead. Same
idiom as the existing Intel DK=512 and X1E carve-outs.
test-backend-ops FLASH_ATTN_EXT on the 740: 226 OK / 0 FAIL, previously exit
139. Other parts are unaffected - the gate is dead code there.
* opencl: fix q6_K flat mul_mat on older Adreno E031 compilers, gated
kernel_mul_mv_q6_K_f32_flat produces ~10x-wrong output on the older Adreno
E031 compilers while q4_K and q5_K are correct. Four codegen defects, each
confirmed on-device against the CPU reference:
1. 64-bit ulong arithmetic is miscompiled, so every weight and scale read
hit the wrong address - the primary cause, and why q5_K (int offsets)
was unaffected. The block index is computed in int and widened only
inside the pointer expression.
2. The vectorized dequant (int4/float4 bit-ops, convert_*4, dot()) is
miscompiled; the 6-bit weights are reconstructed and the dot done
scalar.
3. vload4 of the f32 activations is miscompiled; replaced by a
scalar-indexed load.
4. The accumulation is miscompiled unless a side effect forces the partial
sums to materialize. A printf under a guard the compiler cannot prove
false acts as a zero-cost optimizer barrier; its placement is
load-bearing.
The defect tracks the compiler, not the GPU generation: it reproduces on
E031.38 (Adreno 642L) and E031.41 (Adreno 740) and is fixed by E031.45
(Adreno 619), so the workarounds are gated on the compiler version. Where
they are not needed they cost real throughput - 42.4 -> 35.1 GFLOPS on an
Adreno 840 q6_K GEMV. The explicit compiler-type check is required, not
redundant: newer_than_or_same() is false for every non-E031 compiler, so
negating it alone would enable the workarounds on E17 and DX.
test-backend-ops MUL_MAT is 919/919 on the Adreno 740, 642L, 619, 840 and
850; the 740 and 642L were 909/919 before. The 642L additionally needs the
A6X per-kernel-program support to reach these tests at all.
