700 Commits
Author SHA1 Message Date
Apoorv ParleandGitHub b4fa47d226 release : added gfx1103 to ubuntu rocm build (#28423) 2026-09-14 17:09:46 +03:00
Georgi GerganovandGitHub bbdd9f246e tests : add fusion baseline README and broaden fusion CI triggers (#28893)
* tests : add README for updating the per-backend fusion baselines

Assisted-by: pi:llama.cpp/Qwen3.8-27B

* ci : trigger fusion on changes to test-llama-archs.cpp and src/models

the dummy models and their architectures drive the fusion baselines, so a
change to either can alter the per-fusion counters and should re-run the
fusion job.

Assisted-by: pi:llama.cpp/Qwen3.8-27B

* tests : merge the fusion build commands in the README

assisted-by: pi:llama.cpp/Qwen3.8-27B

* pi : require explicit permission before posting PR/issue comments

assisted-by: pi:llama.cpp/Qwen3.8-27B
2026-09-14 15:45:05 +03:00
Aaron TeoandGitHub be2c6d7d1f tests(s390x): add non-vxe build to tests (#28776)
* tests: add non-vxe build to tests

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

ggml-cpu: add unused macro to fix ci

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

Revert "ggml-cpu: temporarily add #28775 patch until its merged"

This reverts commit d4645257b6.

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

* ggml-cpu: revert back to upstream/master

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>

---------

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
2026-09-14 14:04:58 +03:00
Georgi GerganovandGitHub 89fe242405 ci : trigger self-hosted CI on changes to ci/run.sh (#28859)
The workflow's push/pull_request path filters did not include the
ci/run.sh script that all of its jobs execute, so changes to it never
re-triggered the self-hosted CI.

Assisted-by: pi:llama.cpp/Qwen3.8-27B
2026-09-14 11:51:06 +03:00
Georgi GerganovandGitHub c95f8e47b8 ci : run editorconfig and code-style checks on ubuntu-slim (#28854)
Move the EditorConfig Checker and Code Style Checker workflows from the
`[self-hosted, fast]` runners to `ubuntu-slim`, which is an established
runner label in the repo.

Assisted-by: pi:llama.cpp/Qwen3.8-27B
2026-09-13 18:16:45 +03:00
Sigbjørn SkjæretandGitHub 37b3a9e0cc ci : remove leftover command (#28839) 2026-09-13 10:41:27 +03:00
Aldehir RojasandGitHub acecd56032 common : implement common_schema internal representation for JSON schemas (#28736)
* common : implement common_schema types

* common : implement a json schema optimizer

* common : reduce optimizations

* common : refactor json-schema-to-grammar to use common_schema

* common : use common_trie

* common/schema : implement type/kind resolution

* cont : cleanup

* cont : remove common_chat_tool_parameters

* cont : simplify schema resolution

* cont : pass common_schema through the json-schema-to-grammar builder

* cont : cleanup

* cont : move enums under common_schema and add type enum

* cont : reduce test cases

* cont : clean up

* cont : clean up

* refactor : rename common_schema_parse to common_schema_from_json

* tests : fix gcc dangling-reference warning in test-json-schema

* tests : take the schema label as const char * to satisfy gcc dangling-reference

* refactor : rename common_schema_builder parse_* methods to build_*

* cont : fix may_be_string

* cont : properly handle empty tool parameters

* cont : add tests for empty $ref

* cont : remove dead code

* cont : update docs

* cont : make "{}" mean any object for json_object as well

* cont : restore (min|max)Length to imply string type

* cont : rename common_schema to common_chat_schema
2026-09-12 16:14:50 -05:00
d3146f2b56 ggml-webgpu: Update to a recent version of Dawn (#28683)
* ggml-webgpu: Update to a recent version of Dawn

* No module scanning

* Accept review suggestion to update comment

Co-authored-by: Masashi Yoshimura <yoshimura.masashi.frbs@gmail.com>

---------

Co-authored-by: Masashi Yoshimura <yoshimura.masashi.frbs@gmail.com>
2026-09-12 10:47:29 +09:00
Georgi GerganovandGitHub b78a39a2f9 ci : run test-backend-ops as a dedicated ci/run.sh test (#28740)
* ci : run test-backend-ops as a dedicated gg test

Run test-backend-ops as a separate gg test in ci/run.sh so it is executed outside ctest. With GG_BUILD_HIGH_PERF it keeps the existing CPU-only invocation (-b CPU); otherwise it runs all available backends without a backend filter.

Remove the dedicated backend-ops workflow and keep test-backend-ops as a built target that is not registered with ctest to avoid duplicate runs.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp

* ci : run test-backend-ops earlier and enable high-perf on kleidiai

Move the test-backend-ops gg test before test-llama-archs.

Enable GG_BUILD_HIGH_PERF and LLAMA_ARG_THREADS on the Graviton4 KleidiAI job and use the standard self-hosted results/mnt paths.

Add TODO markers for decoupling tests from libllama.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp

* ci : run test-backend-ops in parallel

Pass -j $(nproc) to test-backend-ops in both high-perf and all-backend modes.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp

* ci : disable parallel tests for ROCm

* cont : disable parallel tests with MoltenVK
2026-09-11 22:00:57 +03:00
Georgi GerganovandGitHub a2878d30df metal : single-source fusion table + fusion debug rework (#28164)
* metal : rework fusion patterns into a single table

All fusable op patterns for the Metal backend are now declared once in a
fusion table (ggml-metal-fuse.cpp) and consumed by both the graph optimizer
(ggml_metal_fuse_max, packing) and the op encoders (ggml_metal_fuse_next,
compute). The two phases share the same pattern table plus ggml_can_fuse_subgraph_ext
for the structural checks, and differ only in the mode used for the pattern
check (STRUCTURAL at optimize time, since tensors are not allocated yet, and
FULL at compute time, including Metal buffer placement). This also protects the
snake activation (MUL + SIN + SQR + MUL + ADD) from being reordered during graph
optimization, which was previously unprotected.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* metal : fix absolute output indices in fusion patterns

ggml_can_fuse_subgraph_ext expects the outputs array to contain absolute graph
node indices (it indexes cgraph->nodes[outputs[i]]), but the fusion table query
was passing a relative index (n_ops - 1). As a result the last node of every
pattern was not recognized as an output and was subjected to the elidable
use-count check, which failed for essentially all fusions. This silently
disabled the norm/MUL fusion and caused a ~5% token-generation regression.

Pass the absolute graph index of the last node instead.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* metal : fuse gated_delta_net with cache cpy

Add GGML_METAL_FUSE_GDN_CACHE to the fusion table: when the gated_delta_net
kernel is followed by a cpy that scatters its recurrent state snapshots into
the KV cache, the kernel writes the snapshots straight into the cache buffer
and the trailing cpy is elided.

The gdn output has other consumers (the attn scores view), so unlike the
elision-chain patterns this is not a simple chain: a 'raw' flag on the fusion
pattern skips the generic chain/shape and ggml_can_fuse_subgraph_ext checks,
making the pattern-specific check callback the sole validator. Packing
(ggml_metal_fuse_max) now matches on the same view-transparent node sequence
that the compute phase uses, so the gdn + cache cpy group is packed along with
any intermediate views and stays adjacent through the reorder.

The fused cpy is a view consumer of the gdn (it writes the cache directly),
so its mem-range is skipped in the encoder; the skip is restricted to CPY
nodes consuming the previous fused node through a view so other fusions are
unaffected.

Add test_gated_delta_net_cache_fusion and register 5 cases.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* metal : drop is_view_consumer mem-range skip

The is_view_consumer skip was carried over from the upstream gated_delta_net
cache-fusion draft, but it is not needed: keeping the elided cpy's mem-range in
the concurrency tracker only ever adds a (conservative) memory barrier at the
fusion point. It can never remove a barrier, so it cannot introduce a race. The
worst case is one spurious barrier per gdn+cache-cpy fusion, which is within
run-to-run noise on Qwen3.5-0.8B Q8_0.

Dropping the check keeps the mem-range loop uniform for all fused groups.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* metal : rename gated_delta_net fused state output args

Rename the fused cache-write kernel argument to match the rest of the kargs:
state_out_stride -> nb_out (and widen it to uint64_t), and the local buffer id
bid_state_out -> bid_out.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* metal : rename raw fusion flag to unsafe

raw did not convey that the flag opts a fusion pattern out of the generic
elision-chain safety net (ggml_can_fuse_subgraph_ext + chain/shape checks).
rename it to 'unsafe' to make explicit that the pattern's check callback is the
sole validator and must re-establish the safety guarantees itself.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* metal : tidy fusion pattern checks and table

- const-correct ggml_metal_fuse_outputs buffer
- annotate unused check-callback parameters
- drop a redundant size_t cast
- align the ops/table initializers and add blank-line separation

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* metal : add generic fusion stats via ad-hoc proc-address API

Add a device-owned fusion context that lets a test tool count how many
times each fusion pattern fires and toggle fusion. It is exposed through
the ad-hoc ggml_backend_reg_get_proc_address mechanism with generic names
so the testing tool is backend-agnostic:

- ggml_backend_fusion_stats_init: start collecting fusion stats; when a
  context is created afterwards it registers the labels/counters and
  encodes single-threaded (n_cb == 0) so the counters are race-free
- ggml_backend_fusion_stats_reset / _get_stats / _set_enabled

The context lives on the metal device (not on the last backend context),
so counters accumulate across contexts and reads are always consistent.
The enable/disable toggle is initialized from GGML_METAL_FUSION_DISABLE
and can be overridden by the test through set_enabled. Labels are
synthesized from the fuse table via ggml_metal_fuse_label (e.g.
"GATED_DELTA_NET+CPY").

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* tests : add fusion count regression test with per-backend baseline

test-fusion runs every dummy model generated by test-llama-archs on a
single backend (single-threaded encoding, n_cb == 0) with fusion enabled
and disabled, and for each mode (prefill / decode) reports the per-fusion
counters and the NMSE between the fused and unfused logits, plus the NMSE
against a CPU reference.

A fusion pattern that silently stops matching (or fires when it should
not) is caught as a regression by comparing the counters against a
committed per-backend TSV baseline:

- --record writes the golden baseline, --check (default) validates it
- the unfused run doubles as a control: its counters must be all-zero
- NMSE is skipped when it is NaN or the arch is already broken on the
  device (e.g. plamo2 on Metal), so the count check is the hard gate
- baseline counts depend only on graph structure, not weights (verified
  stable across weight seeds)
- the fusion stats API is resolved through the ad-hoc get_proc_address
  mechanism with generic names; a backend that does not export it makes
  the test fail with an error

The committed MTL0.tsv baseline covers 110 dummy archs (298 rows).

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* tests : rename fusion api helpers to match stats_init signature

Align the test with the ad-hoc fusion stats API: fusion_stats_init no
longer takes an enable bool (stats are turned on by calling it), so the
proc-address wrappers and typedefs are renamed to the api_* convention.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* tests : rename backend to device in fusion test CLI

The fusion test operates on a compute device (e.g. MTL0), not a backend,
so rename the --backend argument to --device and the backend_name
variable to device_name. Keep "backend" where it refers to the ggml
backend interface (the ad-hoc proc-address mechanism).

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* tests : add --model and --help to fusion test

--model FILE runs the fusion regression test over a single model file
instead of enumerating a --models DIR. --models and --model are mutually
exclusive. Also add a --help/-h option that prints the usage.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* tests : use backend base name for fusion baseline output

The fusion test is invoked with a specific device name (e.g. MTL0), but
its output - the recorded baseline and the header it writes - should be
named after the backend base name (e.g. MTL, via ggml_backend_reg_name),
since the counters depend on the backend, not on the specific device
index. Rename the committed baseline to MTL.tsv.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* tests : run fusion test from ci instead of ctest

The fusion test needs Metal and generates a lot of dummy models, so it
does not belong in the generic ctest suite. Move it to ci/run.sh as
gg_run_test_fusion, gated on GG_BUILD_METAL like
gg_run_test_llama_archs_tensor_split: it generates the dummy models with
test-llama-archs -o and then validates the fusion counts against the
committed baseline. test-fusion.cpp is still built (llama_build) but no
longer registered as a ctest.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* tests : align fusion baseline TSV columns

Pad the TSV fields to fixed widths so the columns line up regardless of
the variable arch and fusion-label lengths, and trim each field on parse
so the padded file is still accepted. Regenerate the committed MTL.tsv
baseline in the padded format (data unchanged, verified identical modulo
padding).

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* tests : widen label column and align fusion TSV header

Give the label column more room (28 chars) and fix the column header
widths so they match the data rows (moe/mode/label), keeping the header
aligned with the values. Regenerate the MTL.tsv baseline in the new
format (data unchanged).

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* tests : switch fusion baseline from TSV to CSV

Use comma-separated values like the rest of the project, keeping the
padded, aligned columns. Split on ',' and trim on parse. Rename the
committed baseline to MTL.csv (data unchanged, verified identical modulo
padding/separator). Update the ci/run.sh check path accordingly.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* cont : rebase + update MTL stats

* tests : avoid graph reallocations for some archs

* metal : tidy fusion debugging context and op init

- simplify the shared fusion debugging context comments
- shorten the ggml_metal_fusion struct comment
- align the ggml_metal_fuse struct fields and comments
- move the fusion parameter of ggml_metal_op_init right after dev

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* tests : dedup fusion baseline into any mode

prefill and decode always produce the same per-graph fusion count, so
store a single row per label with mode = "any" and the per-graph count
instead of two rows. this halves the baseline size and keeps the check
stable.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* ci : move fusion model generation to a separate step

the dummy models generated by test-llama-archs are reused by other tests,
so generate them once in their own step instead of inside test_fusion.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* tests : bump nmse thold

* models : fix plamo2 graph

* tests : remove "skip" logic from test-fusion

* tests : set qwen3tts dummy vocab to codec head size

the dummy qwen3tts model used a vocab of 4096 while the codec head is
3072, so the graph padded the output with -inf which made the NMSE in
test-fusion produce NaN. use the exact codec head size instead so the
padding is not generated at all.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* tests : regen fusion baseline

reflect the plamo2 graph fix, which changed its fusion pattern split
(RMS_NORM+MUL 11->10, RMS_NORM+MUL+ADD 3->4; same total).

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* ci : skip dummy model generation on OpenVINO

test-llama-archs does not build on the OpenVINO platform, so do not try
to generate the dummy models there.

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-0731

* cont : minor

* tests : enable test-llama-archs on windows

* cont : disable on windows + workaround

* metal : naming nits

* test-fusion : add instructions to update baseline

* context : fix Kimi-K3 graph reserve

* fusion : update MTL

* cont : fix naming

* metal : rework fusion info storage

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp

* metal : align fusion info API

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp

* metal : use opaque fusion handle in ad-hoc API

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp

* ci : move fusion test to dedicated workflow

Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp

* cont : run only on ggml changes

* cont : simplify

* fusion : remove multi-output stuff for now

* ci : fix typo
2026-09-11 12:41:54 +03:00
Sigbjørn SkjæretandGitHub 451b89bae0 ci : key cache to sanitizer matrix (#28708) 2026-09-11 07:56:18 +02:00
Sigbjørn SkjæretandGitHub 52d4268656 ci : add self-hosted-gpu-cuda and server-sanitize to hf-jobs (#28693) 2026-09-10 18:11:32 +02:00
shivamkumard-ctrlandGitHub 18c17b4d66 ci : Update WoA CUDA 13.4 release to use 13.4.1 GA redistributables (#28687)
- Move Windows ARM64 CUDA 13.4 builds from the Developer Preview archives to the 13.4.1 GA redistributables
2026-09-10 18:10:55 +02:00
EveandGitHub 434ddbbc0e ci: fix sanitizer tests (#28583) 2026-09-09 19:46:28 +00:00
Sigbjørn SkjæretandGitHub 03fa73cb27 ci : disable npm gha cache (#28600)
* disable npm gha cache

* lies
2026-09-08 14:06:15 +02:00
Daniel BeveniusandGitHub 1744c6bde8 ci : add PYTEST_WORKERS=1 to fix server-self-hosted job (#28603)
* ci : add PYTEST_WORKERS=1 to fix server-self-hosted job

This commit adds the `PYTEST_WORKERS=1` environment variable to the
hf-jobs-t4-small:cuda13 runner steps.

This is an attempt to address CI failure of this job that I might have
introduced in Commit 42f0225fea
("server : use pytest-xdist for server tests (#28298)").

Refs: https://github.com/ggml-org/llama.cpp/actions/runs/34126971262/job/101757819134

* apply same changes to server-metal steps
2026-09-08 13:36:03 +02:00
Sigbjørn SkjæretandGitHub 67672dc5b7 ci : bump ty to 0.0.78 (#28548)
* bump ty to 0.0.78

* type fixes

* more type fixes

* add --exit-zero-on-warning

* remove Callable again
2026-09-07 21:10:06 +02:00
Daniel BeveniusandGitHub 2092353c8b ci : add container image checking and tagging (wip) (#28394)
This commit contains a suggestion for handling container images which
are currently not semver tagged, they only have build numbers in there
tags.

The proposed solution here is to first add a check to make sure that
there are container images built for the build number of the release and
if not fail the build. The container images are build nightly but they
can be triggered manually as well.
If the the container images check passes then the make-release workflow
will re-tag the images with the semver.
2026-09-07 07:23:39 +02:00
Johannes GäßlerandGitHub 74a7c897f0 Github: limit blank issues to maintainers (#28435) 2026-09-05 22:42:35 +02:00
Sigbjørn SkjæretandGitHub 8b4b3558f1 ci : move more jobs to ccache-buckets (#28375)
* move more jobs to ccache-buckets

* add venv deps

* also jq
2026-09-04 15:50:33 +02:00
Ravi PanchumarthyandGitHub 0ef4d560e1 ci : disable failing openvino tests (#28347) 2026-09-04 09:18:11 +03:00
Daniel BeveniusandGitHub 42f0225fea server : use pytest-xdist for server tests (#28298)
* server : use pytest-xdist for server tests

This commit adds pytest-xdist to the server tests. This is pytest
plugin that distributes test execution across multiple CPU cores.

Assisted-by: pi:llama.cpp/qwen3.8-27B

Refs: https://github.com/ggml-org/llama.cpp/pull/26734#issuecomment-5220707042

* remove server_base_port and BASE_PORT

* use worksteal and pytest builting tmp_path
2026-09-03 15:04:30 +02:00
Niklas WenzelandGitHub 03dbcc53e1 ci : check for missing autoreleasepools (#27884)
* ci: check for missing autoreleasepools

* ci : generalize graphics device name pattern
2026-09-02 20:54:48 +03:00
Mario LimoncielloandGitHub cff184438e Update ROCm to 10.0.0 release (#27803) 2026-09-02 19:49:11 +02:00
Sigbjørn SkjæretandGitHub d5fec32a87 ci : enable hf-jobs on server-cuda (#28258) 2026-09-02 20:13:20 +03:00
Ludovic HenryandGitHub 234a6ebaa0 ci: Bump ggml-org/ccache-action to v1.2.24 (#28083) 2026-09-01 12:00:17 +03:00
511f9c1379 OpenVINO: Update OV to 2026.3.1, whisper.cpp support, Qwen3.5 on NPU, and new ops (#27843)
* OpenVINO Backend: Fuse IM2COL + MatMul convolution into OpenVINO convolution

* ci:ggml-ov: Skip recurrent state rollback tests

* ci:ggml-ov: Skip recurrent state rollback tests

* Update OPENVINO.md

* ggml-openvino : add env-var gated op support debugging

* Fix ggml_rope_set_offset case

* OpenVINO backend: Support Whisper.cpp

* Fix code style

* openvino : enable qwen35 on NPU

Static shapes:
- get_graph_input_shape() left the s_copy / s_copy-leaf inputs dynamic
  ([1,1,1,-1]) even in static mode, which propagated a dynamic slot dim through
  GET_ROWS into the conv/GDN state, the state reshapes and the GDN output.
- With -np 1 the s_copy defrag remainder gathers zero rows; short-circuit that
  CPY to the untouched cache instead of emitting a degenerate Slice/Concat, and
  skip binding its zero-byte ggml tensor as an output (the dynamic path already
  did the latter, the static path wrote the full cache over a 0-byte buffer).

Token-count independence:
- In static mode the compiled model's token count is the prefill chunk size or
  1, not the captured cgraph's. Offsets derived from the captured count were
  therefore wrong. Anchor the GDN state slice at the end of the packed
  [attn | state] output and drop the rs_src_begin runtime inputs, and make
  VIEWs over the GDN output / conv_input pass through so the consumer does the
  slicing.
- CONT could not identify its token axis when the graph was captured with a
  single token (every trailing dim has the same stride and size 1) and baked
  the captured shape into the prefill model.

Chunked prefill:
- The last chunk is padded with fabricated tokens. Attention masks them, but
  the recurrent path folded them into cache_r/cache_s permanently. Add a
  chunk_valid_len runtime input, use it to zero g and beta for padded steps
  (making the recurrence an exact identity) and to end the conv snapshot window
  at the last valid token, and disable the recurrent-cache reset after the
  first chunk so earlier chunks are not wiped.
- get_is_prefill() and the chunk loop bound read inp_pos->ne[0] directly, but
  IMROPE stacks 4 position planes, so every decode step was run through the
  padded prefill model and the loop ran extra out-of-bounds chunks.

cache_rs_reset_idx/len now stay runtime Parameters in static mode, since
can_reuse_statically() does not invalidate the cached model on ComputeParams
changes. Add GGML_OPENVINO_FORCE_STATIC to exercise the static path on CPU.

* Update to OpenVINO 2026.3.1

* ggml-openvino: forward NPU compilation mode parameters

Add GGML_OPENVINO_NPU_COMPILE_CONFIG to the backend's cached environment so callers can configure the NPU compiler without using the generic property escape hatch.

When the value is non-empty, pass it to OpenVINO as NPU_COMPILATION_MODE_PARAMS. This enables settings such as optimization-level=3 for NPU compilation while preserving the existing behavior when the variable is unset and leaving CPU and GPU configuration unchanged.

Document the variable, its NPU-only scope, and the optimization-level=3 example in the OpenVINO backend runtime configuration table.

* ggml-openvino : support RELU, POOL_2D, QUICK_GEGLU, and ROLL ops

* reorder op table

* exclude GPU/NPU failing POOL_2D case

* move op type detection to compute_op_case

* Relax rope supported cases

* Fix pool case

* Update openvino doc, gpu driver in ov docker

* openvino: remove unused static remote context branch

* openvino: parallelize static model build

* Apply editorconfig

---------

Co-authored-by: Mostafa Faheem <mostafaaafaheem@gmail.com>
Co-authored-by: Ravi Panchumarthy <ravi.panchumarthy@intel.com>
Co-authored-by: zhaixuejun1993 <xuejun.zhai@intel.com>
2026-08-28 14:42:07 +03:00
Harkirat GillandGitHub 32176338a6 ci : build only the ggml-hip backend for windows-rocm release (#27753) 2026-08-27 22:18:34 +02:00
Slobodan JosicandGitHub 6fdd0ac890 ci : bundle HIP runtime DLLs with Windows ROCm release (#26973)
Copy amdhip64_7, amd_comgr and rocm_kpack next to the binaries so the correct
HIP runtime loads over the driver's copy in System32. Fixes #26929.
2026-08-27 19:27:57 +02:00
Aleksander GrygierandGitHub 4d19b28769 ci: Clean up UI builds from releases (#27706)
* ci : inline UI version resolution into ui-build.yml

* ci : build UI once and reuse the artifact in release jobs

Server jobs now extract the ui-build artifact into tools/ui/dist instead of npm-building the UI. Also removes the get-version job and the no-op -DHF_UI_VERSION flags.

Assisted-by: pi:Kimi-K3

* ui : disable the npm UI build by default (LLAMA_BUILD_UI=OFF)

The flag now only controls building the UI from source via npm. The UI
is still embedded by default from local tools/ui/dist or the prebuilt
download (LLAMA_USE_PREBUILT_UI=ON). CI jobs no longer npm-build the
UI; server-sanitize does not need node anymore.

Assisted-by: pi:Kimi-K3

* ci : rename the ui-build artifact to llama-ui.zip

Consistent with the other artifact names in the Actions summary.

Assisted-by: pi:Kimi-K3

* ci : clarify the windows artifact merge in release.yml

The windows-cuda/vulkan/sycl jobs build only the backend library;
llama-server (with the embedded UI) is injected into their zips from
the windows-cpu package during the release. State this in the job
comments and use accurate wording in the merge step.

Assisted-by: pi:Kimi-K3
2026-08-26 14:12:09 +02:00
Sigbjørn SkjæretandGitHub da9b5d68c3 ci : make cache bucket public (#27728)
* check for hf token

* make bucket public
2026-08-26 12:08:23 +02:00
Mario LimoncielloandGitHub eab8ee41f8 ci : update OS used for ROCM to Ubuntu 24.04 (#27681)
This matches what other build targets use and also what AMD advertises
wheels as supporting.
2026-08-25 20:14:16 +03:00
b114b47397 rpc: support apple RDMA as an RPC transport (#26421)
* rpc: support apple RDMA as an RPC transport

* remove set_tensor micro optimization, rpc socket pinning per CR

* remove transparent reconnect

* trigger apple builds on RPC changes

---------

Co-authored-by: Ryan Churaman <rschu@meta.com>
2026-08-25 20:12:15 +03:00
Sigbjørn SkjæretandGitHub 790b5713ca ci : store ccache on HF buckets (test with cuda-ubuntu for now) (#27699)
* add ccache-buckets action

* use ccache-buckets

* only save on master

* install python3-venv for hip

* add jq and python3 for cuda

* only delete caches older than 5 minutes
2026-08-25 17:28:25 +03:00
Georgi GerganovandGitHub c1d0e7a004 llama.cpp : bump version to 0.3.0 (#27696)
* llama.cpp : bump version to 0.3.0

* ci : update release default desc

* scripts : add prompt for generating release summary
2026-08-25 12:42:21 +03:00
Georgi GerganovandGitHub 985b14912b ci : apply ccache-clear with older/min/dry-run to all ccache jobs (#27602)
* ci : apply ccache-clear with older/min/dry-run to all ccache jobs

Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731

* ci : install gh in ccache-clear if missing (container jobs)

The ccache-clear action relies on the gh CLI, which is not present in
container-based jobs. Install it on demand so those jobs can clear caches.

Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731

* ci : install gh via apt repo in ccache-clear

The install.sh script used previously is no longer served (404). Switch to
the official GitHub CLI apt repository, which is still available.

Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731

* ci : pass --repo to gh cache commands in ccache-clear

In container jobs gh cannot auto-detect the repository from git, so
gh cache list/delete fail with 'failed to run git: not a git repository'.
Pass the repository explicitly via --repo using GITHUB_REPOSITORY.

Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731

* ci : drop -new suffix from vulkan ccache key

The -new suffix was only needed to force a fresh cache. With
ccache-clear now evicting stale caches, the original key can be used
again. The old ccache-vulkan-ubuntu-24.04-arm-new entries still match
the ccache-clear key prefix and are cleaned up automatically.

Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731

* ci : fix ccache-clear date parsing on macOS (BSD date)

macOS ships BSD date, which has no -d option. The older cutoff check
was silently disabled there: 'date: illegal option -- d' errors in the
log and the loop was only stopped by the min limit, risking deletion
of caches not older than the cutoff (e.g. saved by a concurrent job).

Parse the ISO-8601 timestamps with GNU date when available and fall
back to BSD date otherwise (TZ=UTC, fractional seconds dropped).

Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731

* ci : extract ccache-clear logic into scripts/ccache-clear.sh

The composite action now consists of a dedicated step that installs the
GitHub CLI when missing (e.g. in container jobs) and a thin step that
calls the new script. The script follows the make-release-checks.sh
conventions (usage/env header, set -euo pipefail, CLI flags) and only
checks that gh is available. The action inputs are unchanged, so the
workflow steps are untouched.

Assisted-by: llama.cpp:DeepSeek-v4-Flash-0731

* ci : remove unused apple ccaches
2026-08-24 10:49:20 +03:00
e85caa81ea ci : Restore ROCm job for Ubuntu (#27399)
* Revert "ci : disable ubuntu-rocm (#26969)"

This reverts commit 9558fa44c9.

* ci: set ccache compiler_check=content for ROCm build

The ROCm toolchain is pip-installed fresh on every run, so the clang binary's
mtime changes each time. With ccache's default compiler_check=mtime that
invalidates the whole cache and warm builds only reached ~70% hits. Hash the
compiler contents instead so the cache survives toolchain reinstalls.

* Update ccache size to 1GB

We're waivering with so many architectures built, we need a bigger
ccache limit.

* merge fix

---------

Co-authored-by: Jim Wu <ywu@xilinx.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-08-22 13:28:30 +03:00
Georgi GerganovandGitHub 3a653fea93 ci : add older, min and dry-run options to ccache-clear (#27504)
* ci : add older, min and dry-run options to ccache-clear

Assisted-by: pi:llama.cpp/Qwen3.8-27B

* pi : add note about not wrapping lines in PR descriptions

[no ci]

Assisted-by: pi:llama.cpp/Qwen3.8-27B
2026-08-22 11:31:30 +03:00
Georgi GerganovandGitHub a3b9c23ead ci : fix empty release_id in make-release upload step (#27516)
The 'Create release' step had no id, so steps.create_release.outputs.id
resolved to an empty string in the 'Upload nightly-tag.txt' step. The
uploadReleaseAsset call then hit /releases//assets and failed with HTTP
404 (Unhandled error: HttpError), e.g. run 32513839499.

Add id: create_release to the step; the action already exposes the id
output.

Assisted-by: pi:llama.cpp/Qwen3.8-27B
2026-08-21 21:41:25 +03:00
d7fa69b7de ci : run ccache-clear as the last step of release jobs (#27503)
* ci : run ccache-clear as the last step of release jobs

Assisted-by: pi:llama.cpp/Qwen3.8-27B

* update disabled job too to force rebase

---------

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
2026-08-21 18:56:04 +03:00
Georgi GerganovandGitHub 353b32d8b9 ci : remove duplicate flag (#27488) 2026-08-21 14:10:06 +03:00
Georgi GerganovandGitHub e467c2ff61 ci : add nightly-tag.txt to make-release (#27485)
As agreed in ggml discussion #1579, the official semver releases now
include a nightly-tag.txt asset containing the tag of the corresponding
nightly release (e.g. b10485). The Web UI assets are published to the
HF bucket under the nightly tag, so this makes them discoverable for
each official release.

- make-release-desc.sh: expose the resolved nightly tag as a
  nightly_tag output
- make-release.yml: create nightly-tag.txt from that tag, upload it
  as a release asset (skipped on dry-run), mention it in the release
  body and in the dry-run summary

Assisted-by: pi:llama.cpp/Qwen3.8-27B
2026-08-21 13:20:44 +03:00
Georgi GerganovandGitHub 1719747451 ci : release clean-up (#27477) 2026-08-21 11:33:40 +03:00
EveandGitHub 0e1d9185c5 ci: use shell script to check cmake pkg (#27414)
* use regular script to build cmake pkg

* use old grep without perl
2026-08-20 20:01:32 +00:00
Oliver SimonsandGitHub bf0040e15f CI: Use LLVM's OpenMP over MSVC_DEBUG_non_redist on Windows (#26678)
* CI: Use LLVM's OpenMP over MSFT_DEBUG_non_redist on Windows

Currently, we ship the non-redist debug version of microsoft's libomp.
This PR changes this to official LLVM's release, also packaging
the license as needed.

* Remove LLVM SHA from job name to increase legibility

* Add temp validations to CI

* Revert "Add temp validations to CI"

This reverts commit eef97c88b5.

* Build OpenMP in CI

* Make OpenMP fetch self-contained in cmake and cache in CI

* Robustify Licens-packaging

1. Ship OpenMP license, not LLVM's.
2. Invalidate cache also on checksum of the license

* Remove stale reference in docs/build.md

* No longer package base license in release

This was scope-creep

* Add explanatory comment to OpenMP license

* Remove arm64 smoke

Forgot this during conflict resolution during rebase of
c54c0e9cf6

* Remove GGML_OPENMP_FETCH_CACHE_DIR as requested by @CISC

* whitespace changes
2026-08-20 15:42:26 +02:00
Sigbjørn SkjæretandGitHub 01ac3ad761 ci : add release attestation url (#27389) 2026-08-19 16:40:47 +02:00
Sigbjørn SkjæretandGitHub 645ca2834b ci : re-enable release dependency for sycl (#27385) 2026-08-19 16:44:16 +03:00
Jeremie MillerandGitHub 0adcc3bb57 ci : add attestation for signed release artifacts (#25933) 2026-08-19 10:23:52 +03:00
9d77fa1725 ci : Update OpenVINO to 2026.3, skip nemotron-h rollback test (#27292)
* update to ov-2026.3, update device drivers

* ci: skip nemotron-h rollback test on OpenVINO

The OpenVINO backend does not support SSM_SCAN, so the Nemotron-H recurrent state rollback graph is split and cannot preserve the recurrent cache output shape. Keep the test enabled for other backends and retain the qwen35 OpenVINO rollback coverage.

---------

Co-authored-by: ravi9 <ravi.panchumarthy@intel.com>
2026-08-18 12:02:22 +02:00
shivamkumard-ctrlandGitHub c0296022f3 ci: add Windows ARM64 CUDA support to the manual workflow (#27300)
- Add a CUDA 13.4 ARM64 matrix entry.
- Build only ggml-cuda for x64 and ARM64.
2026-08-18 11:55:45 +02:00