* metal : per-device tuned (Q, NE) for flash-attn vec (#25750)
* rebase Q-generic FA vec body from 01dc93607 (#23114)
* add 53 f16 (Q,NE) flash-attn vec instantiations (vec 80 -> 133)
* add FA vec (Q,NE) tuning table + dispatch wiring + SMEM cap fallback
* add FA vec (Q,NE) perf sweep
* fill tuning result
* fold family table into a per-family representative SKU
* refactor tuning result format
* extend FA vec tuning to quantized KV caches
* sync fa vec tuner bucketing with runtime, use pointwise tuning regret
* update tuned table
* format and cleanup
* prefix fa_vec tuning procs with ggml_backend_metal_tuning_, drop unused fa_vec_override_active
* add device id -> token lookup for the offline tuning tool
* add ggml-metal-tuning skeleton
* add op-agnostic perf cell + median timing for the tuner
* add FA-vec graph build + tensor init to the tuner
* tools : add FA-vec (Q,NE) sweep, compression and table emit
* cool down and re-measure the dirty window on thermal drift
* test-backend-ops : replace the FA vec tune mode with a bounded (Q,NE) slice
* tools : document the Metal tuner, point the table comment at it
* abort on unknown KV type, single-source fa_vec_legal_ne
* cleanup
* honor -o in the FA vec (Q,NE) slice
* retune FA-vec (Q, NE) under a pointwise no-harm gate
* cont : add fa-vec tunings for M1 Pro, M2 Ultra, M5 Max
---------
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* metal : per-op source split + parallel compile (#24021)
* preliminary extract common header
* op source split
* split metallib into 8 libs && load in parallel
* derive kernel->library routing from functionNames
* x-macro lib list + underscore filenames, dedup QK_NL, MRC fixes
* op source split 8 to 20
* improve robustness of source fallback
* clean up
* change bool -> atomic_bool
* only prepend headers that source actually includes
* no semaphore, use GCD global queue
* dedup library compile path, fix NSError lifetime, rename gla
* relocate upstream concat/rope_back/repeat kernel changes into split files
* move ggml-common.h from common.h into dequantize.h to shrink binary size
---------
Co-authored-by: lvyichen <lvyichen@stepfun.com>
* metal: add col2im_1d op (f32/f16/bf16) (#25176)
* metal : add set_rows with src0 f16 (#25434)
* metal : add CONV_2D_DW (depthwise convolution) support (#21565)
* metal : add Q2_0 support (#25419)
* metal: fuse snake activation (mul, sin, sqr, mul, add) (#25459)
* ggml-metal: FWHT kernel for metal backend (#25924)
* metal : port new kernels into the split sources
Move the kernels added on master after the split (lightning indexer,
DSv4 hyper-connections, silu_back, f16 bin ops, TQ2_0, the flash-attn KV
dequantization pass, rope offset/inplace, ssm_scan rollback, packed q8_0
dequantization and the tensor-API mat-mat K clamp) into the corresponding
kernels/*.metal sources. Copied verbatim, no functional change.
---------
Co-authored-by: lvyichen <lvyichen@stepfun.com>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* feat: support step3-vl-10b
* use fused QKV && mapping tensor in tensor_mapping.py
* guard hardcoded params and drop crop metadata
* get understand_projector_stride from global config
* img_u8_resize_bilinear_to_f32 move in step3vl class
* Apply suggestions from code review
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* fix the \r\n mess
* add width and heads to MmprojModel.set_gguf_parameters
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
