wylab/llama.cpp
0
0
Fork 1
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-07 16:37:57 +02:00
Code Issues Packages Projects Releases Wiki Activity
10,841 Commits 778 Branches 7,351 Tags
0cc4m/vulkan-mul-mm-refactor
Commit Graph
74 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Neo ZhangandGitHub 0df974d777 sycl : enhance the api to support peer-to-peer copy (#27550) 2026-09-03 10:41:07 +03:00
Neo ZhangandGitHub 5eec3ad017 sycl : support limit max alloc memory within 2GB for host-pinned memory (#27559) 2026-09-01 13:35:47 +03:00
Neo ZhangandGitHub a32af33de2 sycl : Enhance to get the free memory of Intel GPU (#27968) 
* enhance get mem info by l0 an SYCL API

* remove debug code, format the code

* update SYCL.md for GGML_SYCL_GET_MEM_API
 2026-08-31 13:33:02 +03:00
Neo ZhangandGitHub 814d84bc9d sycl : mark tq2_0 as not supported (#27660) 2026-08-25 09:04:58 +03:00
Neo ZhangandGitHub 9e96cf77ff sycl : fix load model with mlock issue (#27250) 2026-08-21 08:14:54 +03:00
Neo ZhangGitHubNeo Zhang Jianyu
 		37a215c9e9 [SYCL] support OP OPT_STEP_ADAMW, OPT_STEP_SGD (#25268) 
* fix conflict

* fix conflict of ops.md

* fix conflict of ops.md

* update the ops.md

---------

Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2026-08-17 07:31:29 +03:00
Neo ZhangandGitHub a97123e497 [SYCL] Support host pinned mem to improve SYCL Host-to-Device Memory Access (#26789) 
* support host pinned mem, ggml_backend_sycl_host_buffer_type_get_max_size,

* fix the thread-safe issue
 2026-08-13 20:05:33 +03:00
Neo ZhangandGitHub d415e65a57 sycl : enhance concat to support Q4_0, Q4_1, Q5_0, Q5_1, Q8_0 (#26800) 2026-08-13 11:24:28 +03:00
Neo ZhangandGitHub 31558dbb76 sycl : Support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PRE (#26568) 
* support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PREwq

* update ops.md

* fix format issue
 2026-08-07 08:22:23 +03:00
Neo ZhangandGitHub c1f4109898 sycl : update guide Q&A and script for device setting (#26442) 2026-08-07 08:18:47 +03:00
Neo ZhangandGitHub eef5f3e343 sycl : fix error Error OP FLASH_ATTN_EXT on arc770 (#26441) 2026-08-07 08:17:56 +03:00
Neo ZhangandGitHub c074cb3f76 sycl : enhance OP set_rows to support all missed data types (#26515) 
* support fp16 to fp16/fp32

* support all missed data types in set_rows

* refactor the code to support all data types
 2026-08-07 07:52:52 +03:00
Neo ZhangandGitHub d5d3e05bf8 [SYCL] support the missed types in cpy (#26005) 
* support the missed types in cpy

* use correct funct

* rm unused code
 2026-07-31 10:25:16 +03:00
Neo ZhangGitHubNeo Zhang Jianyu
 		1c5b89ff63 sycl : support dev2dev memcpy by DEV2DEV_MEMCPY_FORWARD (#26234) 
Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2026-07-31 09:20:28 +03:00
Neo ZhangandGitHub a2be61dc87 [SYCL] Support q2 mul_mat (#26231) 
* support q2_0 in mul_mat

* support more q2_0 case
 2026-07-31 09:19:41 +03:00
Neo ZhangGitHubGeorgi Gerganov
 		a6a77bc48d [UT] enhance UT to show all real unsupported backends (#25234) 
* enhance UT to show real unsupported backends

* cont : simplify

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-07-30 14:04:58 +08:00
Neo ZhangandGitHub d3fba0c79d sycl : fix get_rows Q2_K, Q4_K, Q5_K (#25656) 2026-07-15 10:32:28 +03:00
Neo ZhangandGitHub ae9291e16b sycl : support kernel type fp16 for conv2d_dw (#25653) 2026-07-15 10:31:10 +03:00
Neo ZhangandGitHub 108f186d17 [SYCL] fix unsupported UT cases of CONT & CPY (#25231) 
* fix unsupported UT cases of CONT & CPY

* update ops.md

* rm unused head file
 2026-07-07 12:20:52 +03:00
Neo ZhangandGitHub 47e1de77aa [SYCL] support op col2im_1d (#25264) 
* support op col2im_1d

* update ops.md

* rm unused words

* update for bf16

* optimize 1%-11% as the review comments

* fix the format issue

* update as the review comments
 2026-07-07 11:07:46 +03:00
Neo ZhangandGitHub 55edb2de44 [SYCL] support OP cross_entropy_loss, cross_entropy_loss_back (#25236) 
* support OP cross_entropy_loss, cross_entropy_loss_back

* correct format issue
 2026-07-07 10:48:50 +03:00
Neo ZhangandGitHub 95e5254c0a [SYCL] fix unsupport ACC UT cases for noncontiguous (#25124) 
* fix unsupport ACC UT cases for noncontiguous

* update ops.md
 2026-07-07 10:40:38 +03:00
Neo ZhangandGitHub 9e5ef0dbb1 sycl : enhance argsort to support all UT cases (#25125) 2026-07-07 10:39:29 +03:00
Neo ZhangandGitHub 3d4cbdf18a sycl : use sycl func to fix AOT double type issue (#25081) 2026-07-07 10:38:33 +03:00
Neo ZhangandGitHub 26145b3db7 sycl : rename the env vars from "disable" to "enable" (#25042) 2026-07-07 10:33:51 +03:00
Neo ZhangandGitHub 9bebfcb4bc sycl : fix failed ut cases of norm (#25044) 2026-06-27 12:13:43 +03:00
Neo ZhangandGitHub 9c10954865 sycl : fix the failed UT cases of conv_3d (#24900) 2026-06-25 08:27:58 +03:00
Neo ZhangandGitHub f8cc15f163 [SYCL] support bf16 on bin_bcast OP and unary OPs (#24838) 
* support bf16 on bin_bcast OP and unary OPs

* support the older Intel compiler than 2026.0
 2026-06-22 14:09:02 +03:00
Neo ZhangandGitHub 9724f664e8 [SYCL] rename GGML_SYCL_SUPPORT_LEVEL_ZERO (#24719) 
* rename GGML_SYCL_SUPPORT_LEVEL_ZERO to GGML_SYCL_SUPPORT_LEVEL_ZERO_API, and GGML_SYCL_ENABLE_LEVEL_ZERO to  GGML_SYCL_USE_LEVEL_ZERO_API

* fix code format

* fix error when rebase
 2026-06-18 11:18:26 +03:00
Neo ZhangandGitHub dd69db2924 sycl : support MUL_MAT and OUT_PROD with Q1_0 (#24721) 2026-06-18 11:17:37 +03:00
Neo ZhangandGitHub 6f1034b32a [SYCL] support OPs: conv_2d, conv_2d_dw, conv2d_transpose (#24600) 
* fix conflict

* fix format issue, rename

* rm debug code

* correct the file name
 2026-06-18 09:40:03 +03:00
Neo ZhangGitHubNeo Zhang <NA>
 		74a80dd9c0 [SYCL] add dev2dev memcpy by SYCL API (#24476) 
* add dev2dev memcpy by SYCL API

* mv GGML_SYCL_DEV2DEV_MEMCPY to runntime table

* update the detect method for p2p comm

* fix the erro created during fix confilct

---------

Co-authored-by: Neo Zhang <NA>
 2026-06-17 17:21:34 +03:00
Neo ZhangandGitHub d1759e4156 [SYCL] Add conv_3d (#24691) 
* add conv_3d

* optimize

* update ops.md

* restore test script

* rm unused code

* rm copyright notes
 2026-06-17 17:20:01 +03:00
Neo ZhangandGitHub 58728bdbf0 sycl : Enable to support fp16 by OPs: SQR, SQRT, LOG, SIN, COS, CLAMP (#24692) 2026-06-17 08:58:03 +03:00
Neo ZhangandGitHub fdd109883d [SYCL] Support OP EXPM1, support all UT cases of FLOOR, TRUNC, ROUND (#24363) 
* support OP EXPM1, support all UT cases of FLOOR, TRUNC, ROUND

* fix conflict

* rebase, support new UT case of repeat, concat
 2026-06-16 08:34:29 +03:00
Neo ZhangandGitHub 72be44f1d2 sycl : fix reorder function; add fp32/fp16 in build script (#24578) 2026-06-15 10:08:34 +03:00
Neo ZhangandGitHub 8872ab5467 sycl : enhance set_rows to support q1_0, mxfp4, nvfp4 (#24564) 2026-06-15 10:01:40 +03:00
Neo ZhangandGitHub 987fbd821d [SYCL] add to support pool_1d, move pool_1d/2d code to pool.cpp/hpp (#24584) 
* add to support pool_1d, move pool_1d/2d code to pool.cpp/hpp

* update ops.md
 2026-06-15 10:01:07 +03:00
Neo ZhangandGitHub 099ea76fb4 [SYCL] Fix CI build & release for SYCL backend (#24387) 
* restore SYCL build and release, remove github cache

* modify for test only

* verify the ccache is used

* remove debug code change

* rm duplicate action, update key in ccache

* add action ccache-clear after building in both ubuntu and windows

* set %NUMBER_OF_PROCESSORS% in widnows build
 2026-06-12 09:30:24 +03:00
Neo ZhangandGitHub d403f00ec3 [SYCL] Update compute runtime version to 26.x in docker (#24070) 
* update compute runtime from 25 to 26 in docker

* add comment with old driver for multiple GPUs
 2026-06-08 10:35:18 +08:00
Neo ZhangandGitHub a51142497a [SYCL] Support Q4_1, Q5_0, Q5_1 in Flash-attention (#23812) 
* support Q4_1, Q5_0, Q5_1

* update ut case
 2026-06-01 09:53:53 +03:00
Neo ZhangandGitHub 4162522688 [SYCL] Add more types in GET_ROWS OP (#23710) 
* add to support Q1_0, NVFP4, IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ1_S, IQ1_M, IQ3_S, IQ4_NL, IQ4_XS, I32, MXFP4, Q2_K, Q3_K, Q5_K, and Q6_K in GET_ROWS OP

* correct the link
 2026-06-01 09:53:04 +03:00
Neo ZhangandGitHub 44e211cecf sycl : Optimize Q3_K mul_mat by reorder (#23725) 2026-06-01 09:50:55 +03:00
Neo ZhangandGitHub aabee047d8 [SCYL] add chapter for performance reference in SYCL.md (#23315) 
* add chapter for performance reference

* rm unsupported GPU
 2026-05-19 09:44:51 +03:00
Neo ZhangandGitHub e98bcfec28 sycl : fix error when use -mg 1 error (#23140) 2026-05-18 08:11:19 +03:00
Neo ZhangandGitHub 0f45f1a35c docker : revert stable version of intel compute-runtime (#22968) 2026-05-14 11:30:40 +02:00
Neo ZhangandGitHub 7d442abf5c [SYCL] Add OP im2col_3d (#22903) 
* add im2col_3d

* format code

* update the ops.md
 2026-05-11 08:01:47 +03:00
Neo ZhangandGitHub 6a2a2513dc fix script error (#22795sycl : ) 2026-05-08 06:54:57 +03:00
Neo ZhangandGitHub eddd7a13a5 [SYCL] Optimize Q4_0 mul_mat for Arc770, add scripts (#22291) 
* opt arc770 for Q4_0

* add for Q4_0

* update the script

* add help script for windows

* update guide

* fix format issue

* convert from dos to unix for format issue

* fix missed -sm parameter
 2026-04-25 09:20:14 +03:00
Neo ZhangandGitHub f51fd36d79 sycl : handle other FA case (#21377) 2026-04-06 13:28:00 +03:00
Neo ZhangandGitHub 4888137b17 sycl : fix llama_kv_cache hang when kv_cache is huge: 5GB (#21283) 2026-04-02 10:08:32 +03:00
Neo ZhangandGitHub 6b949d1078 sycl : support nvfp4 type in mul_mat (#21227) 2026-04-01 13:54:15 +03:00
Neo ZhangandGitHub 62278cedde sycl : enhance fattn perf (#21185) 2026-03-31 13:31:50 +03:00
Neo ZhangandGitHub afe65aa282 [SYCL] Enhance build script to use half cores to build, avoid OS hang (#21093) 
* use half cores to build, avoid OS hang

* reduce the output text num to short test time

* avoid to return 0
 2026-03-29 09:02:45 +08:00
Neo ZhangandGitHub 53dc8b59bf sycl : fix wrong variable check by assert (#20903) 
* fix wrong variable check by assert

* use GGML api
 2026-03-25 11:48:37 +02:00
Neo ZhangandGitHub f40a80b4f3 support bf16 and quantized type (#20803) 2026-03-22 22:06:27 +08:00
Neo ZhangandGitHub b6c83aad55 [SYCL] ehance UPSCALE to support all UT cases (#20637) 
* [SYCL] ehance UPSCALE to support more cases

* rm test case result of SYCL1
 2026-03-17 10:01:52 +08:00
Neo ZhangandGitHub a93c0ef0fa add op gated_delta_net (#20455) 2026-03-14 22:01:57 +08:00
Neo ZhangandGitHub ecac98ee53 [SYCL] Update SYCL.md for binary package for Windows (#20401) 
* add download binary package

* update prefix
 2026-03-11 22:21:22 +08:00
Neo ZhangandGitHub 0cec84f999 fix op rope, add rope_back (#20293) 2026-03-11 09:53:34 +08:00
Neo ZhangandGitHub b2e1427c9b fix for failed UT case: ACC, L2_NORM, UPSCALE, fused_glu, unary (#20283) 2026-03-11 09:53:05 +08:00
Neo ZhangandGitHub 213c4a0b81 [SYCL] supprt Flash Attention for fp32/fp16/Q4/Q5/Q8 (#20190) 
* support flash-attention for fp32/fp16/Q4/Q5/Q8

* rm warining

* update for JIT
 2026-03-08 12:00:07 +08:00
Neo ZhangGitHubNeo Zhang Jianyu
 		c17dce4f5c replace the magic nunber 768 by max work group size to support iGPU (#19920) 
Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2026-02-27 09:26:07 +08:00
Neo ZhangGitHubNeo Zhang Jianyu
 		2943210c1e support permuted, remove check s0/s10 (#19889) 
Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2026-02-26 10:27:20 +08:00
Neo ZhangandGitHub bf38346d13 Remove support for Nvidia & AMD GPU, because the oneAPI plugin for Nvidia & AMD GPU is unavailable: download/installation channels are out of work. (#19246) 
User can't build up the software for Nvidia & AMD GPU.
rm the oneMath since it is only used in NV and AMD code path.
 2026-02-02 21:06:21 +08:00
Neo ZhangandGitHub 2634ed207a create test.sh to enhance the parameters for testing, update the guide, rm useless script (#19243) 2026-02-01 18:24:00 +08:00
Neo ZhangGitHubNeo Zhang Jianyu
 		d4964a7c66 sycl: fix norm kernels: l2_norm, group_norm, rms_norm by remove assert to support more cases (#19154) 
Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2026-01-29 09:20:22 +08:00
Neo ZhangGitHubNeo Zhang Jianyu
 		cb6caca191 [SYCL] use malloc to support both iGPU and dGPU in same time (#18992) 
* use malloc to support both iGPU and dGPU in same time

* support windows

---------

Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2026-01-23 20:54:10 +08:00
Neo ZhangGitHubSigbjørn SkjæretNeo Zhang Jianyu
 		a6a552e4ec [SYCL] replace llama-cli by llama-completion to rm the impact to test script (#18290) 
* replace llama-cli by llama-completion to rm the impact to test script

* Update examples/sycl/run-llama2.sh

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update examples/sycl/run-llama2.sh

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update examples/sycl/run-llama3.sh

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update examples/sycl/run-llama3.sh

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update examples/sycl/win-run-llama2.bat

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update examples/sycl/win-run-llama3.bat

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2025-12-23 12:59:12 +08:00
Neo ZhangandGitHub 68522c678d ci : support bfloat16 SYCL release package (#17855) 
* support bfloat16 release package

* add fallback file
 2025-12-08 15:09:39 +01:00
Neo ZhangGitHubNeo Zhang Jianyu
 		7d2add51d8 sycl : support to malloc memory on device more than 4GB, update the doc and script (#17566) 
Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2025-11-29 14:59:44 +02:00
Neo ZhangGitHubNeo Zhang <>
 		a21c6fd450 update guide (#8909) 
Co-authored-by: Neo Zhang <>
 2024-08-11 14:07:43 +05:30
Neo ZhangandGitHub d4ff847153 [SYCL] correct cmd name (#8877) 2024-08-06 09:09:12 +08:00
Neo ZhangandGitHub df68d4fa5d [SYCL] Update README-sycl.md for Chapter "Recommended release" and "News" (#7946) 
* Update README-sycl.md

* Update README-sycl.md

* Update README-sycl.md

* Update README-sycl.md
 2024-06-17 11:17:07 +08:00