wylab/llama.cpp
0
0
Fork 1
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-07 16:37:57 +02:00
Code Issues Packages Projects Releases Wiki Activity
10,809 Commits 778 Branches 7,351 Tags
v0.4.0
Commit Graph
74 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Neo ZhangandGitHub 0df974d777 sycl : enhance the api to support peer-to-peer copy (#27550) 2026-09-03 10:41:07 +03:00
Neo ZhangandGitHub 5eec3ad017 sycl : support limit max alloc memory within 2GB for host-pinned memory (#27559) 2026-09-01 13:35:47 +03:00
Neo ZhangandGitHub a32af33de2 sycl : Enhance to get the free memory of Intel GPU (#27968) 
* enhance get mem info by l0 an SYCL API

* remove debug code, format the code

* update SYCL.md for GGML_SYCL_GET_MEM_API
 2026-08-31 13:33:02 +03:00
Neo ZhangandGitHub 814d84bc9d sycl : mark tq2_0 as not supported (#27660) 2026-08-25 09:04:58 +03:00
Neo ZhangandGitHub 9e96cf77ff sycl : fix load model with mlock issue (#27250) 2026-08-21 08:14:54 +03:00
Neo ZhangGitHubNeo Zhang Jianyu
 		37a215c9e9 [SYCL] support OP OPT_STEP_ADAMW, OPT_STEP_SGD (#25268) 
* fix conflict

* fix conflict of ops.md

* fix conflict of ops.md

* update the ops.md

---------

Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2026-08-17 07:31:29 +03:00
Neo ZhangandGitHub a97123e497 [SYCL] Support host pinned mem to improve SYCL Host-to-Device Memory Access (#26789) 
* support host pinned mem, ggml_backend_sycl_host_buffer_type_get_max_size,

* fix the thread-safe issue
 2026-08-13 20:05:33 +03:00
Neo ZhangandGitHub d415e65a57 sycl : enhance concat to support Q4_0, Q4_1, Q5_0, Q5_1, Q8_0 (#26800) 2026-08-13 11:24:28 +03:00
Neo ZhangandGitHub 31558dbb76 sycl : Support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PRE (#26568) 
* support DSv4 OPs: LIGHTNING_INDEXER,DSV4_HC_COMB,DSV4_HC_POST,DSV4_HC_PREwq

* update ops.md

* fix format issue
 2026-08-07 08:22:23 +03:00
Neo ZhangandGitHub c1f4109898 sycl : update guide Q&A and script for device setting (#26442) 2026-08-07 08:18:47 +03:00
Neo ZhangandGitHub eef5f3e343 sycl : fix error Error OP FLASH_ATTN_EXT on arc770 (#26441) 2026-08-07 08:17:56 +03:00
Neo ZhangandGitHub c074cb3f76 sycl : enhance OP set_rows to support all missed data types (#26515) 
* support fp16 to fp16/fp32

* support all missed data types in set_rows

* refactor the code to support all data types
 2026-08-07 07:52:52 +03:00
Neo ZhangandGitHub d5d3e05bf8 [SYCL] support the missed types in cpy (#26005) 
* support the missed types in cpy

* use correct funct

* rm unused code
 2026-07-31 10:25:16 +03:00
Neo ZhangGitHubNeo Zhang Jianyu
 		1c5b89ff63 sycl : support dev2dev memcpy by DEV2DEV_MEMCPY_FORWARD (#26234) 
Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2026-07-31 09:20:28 +03:00
Neo ZhangandGitHub a2be61dc87 [SYCL] Support q2 mul_mat (#26231) 
* support q2_0 in mul_mat

* support more q2_0 case
 2026-07-31 09:19:41 +03:00
Neo ZhangGitHubGeorgi Gerganov
 		a6a77bc48d [UT] enhance UT to show all real unsupported backends (#25234) 
* enhance UT to show real unsupported backends

* cont : simplify

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2026-07-30 14:04:58 +08:00
Neo ZhangandGitHub d3fba0c79d sycl : fix get_rows Q2_K, Q4_K, Q5_K (#25656) 2026-07-15 10:32:28 +03:00
Neo ZhangandGitHub ae9291e16b sycl : support kernel type fp16 for conv2d_dw (#25653) 2026-07-15 10:31:10 +03:00
Neo ZhangandGitHub 108f186d17 [SYCL] fix unsupported UT cases of CONT & CPY (#25231) 
* fix unsupported UT cases of CONT & CPY

* update ops.md

* rm unused head file
 2026-07-07 12:20:52 +03:00
Neo ZhangandGitHub 47e1de77aa [SYCL] support op col2im_1d (#25264) 
* support op col2im_1d

* update ops.md

* rm unused words

* update for bf16

* optimize 1%-11% as the review comments

* fix the format issue

* update as the review comments
 2026-07-07 11:07:46 +03:00
Neo ZhangandGitHub 55edb2de44 [SYCL] support OP cross_entropy_loss, cross_entropy_loss_back (#25236) 
* support OP cross_entropy_loss, cross_entropy_loss_back

* correct format issue
 2026-07-07 10:48:50 +03:00
Neo ZhangandGitHub 95e5254c0a [SYCL] fix unsupport ACC UT cases for noncontiguous (#25124) 
* fix unsupport ACC UT cases for noncontiguous

* update ops.md
 2026-07-07 10:40:38 +03:00
Neo ZhangandGitHub 9e5ef0dbb1 sycl : enhance argsort to support all UT cases (#25125) 2026-07-07 10:39:29 +03:00
Neo ZhangandGitHub 3d4cbdf18a sycl : use sycl func to fix AOT double type issue (#25081) 2026-07-07 10:38:33 +03:00
Neo ZhangandGitHub 26145b3db7 sycl : rename the env vars from "disable" to "enable" (#25042) 2026-07-07 10:33:51 +03:00
Neo ZhangandGitHub 9bebfcb4bc sycl : fix failed ut cases of norm (#25044) 2026-06-27 12:13:43 +03:00
Neo ZhangandGitHub 9c10954865 sycl : fix the failed UT cases of conv_3d (#24900) 2026-06-25 08:27:58 +03:00
Neo ZhangandGitHub f8cc15f163 [SYCL] support bf16 on bin_bcast OP and unary OPs (#24838) 
* support bf16 on bin_bcast OP and unary OPs

* support the older Intel compiler than 2026.0
 2026-06-22 14:09:02 +03:00
Neo ZhangandGitHub 9724f664e8 [SYCL] rename GGML_SYCL_SUPPORT_LEVEL_ZERO (#24719) 
* rename GGML_SYCL_SUPPORT_LEVEL_ZERO to GGML_SYCL_SUPPORT_LEVEL_ZERO_API, and GGML_SYCL_ENABLE_LEVEL_ZERO to  GGML_SYCL_USE_LEVEL_ZERO_API

* fix code format

* fix error when rebase
 2026-06-18 11:18:26 +03:00
Neo ZhangandGitHub dd69db2924 sycl : support MUL_MAT and OUT_PROD with Q1_0 (#24721) 2026-06-18 11:17:37 +03:00
Neo ZhangandGitHub 6f1034b32a [SYCL] support OPs: conv_2d, conv_2d_dw, conv2d_transpose (#24600) 
* fix conflict

* fix format issue, rename

* rm debug code

* correct the file name
 2026-06-18 09:40:03 +03:00
Neo ZhangGitHubNeo Zhang <NA>
 		74a80dd9c0 [SYCL] add dev2dev memcpy by SYCL API (#24476) 
* add dev2dev memcpy by SYCL API

* mv GGML_SYCL_DEV2DEV_MEMCPY to runntime table

* update the detect method for p2p comm

* fix the erro created during fix confilct

---------

Co-authored-by: Neo Zhang <NA>
 2026-06-17 17:21:34 +03:00
Neo ZhangandGitHub d1759e4156 [SYCL] Add conv_3d (#24691) 
* add conv_3d

* optimize

* update ops.md

* restore test script

* rm unused code

* rm copyright notes
 2026-06-17 17:20:01 +03:00
Neo ZhangandGitHub 58728bdbf0 sycl : Enable to support fp16 by OPs: SQR, SQRT, LOG, SIN, COS, CLAMP (#24692) 2026-06-17 08:58:03 +03:00
Neo ZhangandGitHub fdd109883d [SYCL] Support OP EXPM1, support all UT cases of FLOOR, TRUNC, ROUND (#24363) 
* support OP EXPM1, support all UT cases of FLOOR, TRUNC, ROUND

* fix conflict

* rebase, support new UT case of repeat, concat
 2026-06-16 08:34:29 +03:00
Neo ZhangandGitHub 72be44f1d2 sycl : fix reorder function; add fp32/fp16 in build script (#24578) 2026-06-15 10:08:34 +03:00
Neo ZhangandGitHub 8872ab5467 sycl : enhance set_rows to support q1_0, mxfp4, nvfp4 (#24564) 2026-06-15 10:01:40 +03:00
Neo ZhangandGitHub 987fbd821d [SYCL] add to support pool_1d, move pool_1d/2d code to pool.cpp/hpp (#24584) 
* add to support pool_1d, move pool_1d/2d code to pool.cpp/hpp

* update ops.md
 2026-06-15 10:01:07 +03:00
Neo ZhangandGitHub 099ea76fb4 [SYCL] Fix CI build & release for SYCL backend (#24387) 
* restore SYCL build and release, remove github cache

* modify for test only

* verify the ccache is used

* remove debug code change

* rm duplicate action, update key in ccache

* add action ccache-clear after building in both ubuntu and windows

* set %NUMBER_OF_PROCESSORS% in widnows build
 2026-06-12 09:30:24 +03:00
Neo ZhangandGitHub d403f00ec3 [SYCL] Update compute runtime version to 26.x in docker (#24070) 
* update compute runtime from 25 to 26 in docker

* add comment with old driver for multiple GPUs
 2026-06-08 10:35:18 +08:00
Neo ZhangandGitHub a51142497a [SYCL] Support Q4_1, Q5_0, Q5_1 in Flash-attention (#23812) 
* support Q4_1, Q5_0, Q5_1

* update ut case
 2026-06-01 09:53:53 +03:00
Neo ZhangandGitHub 4162522688 [SYCL] Add more types in GET_ROWS OP (#23710) 
* add to support Q1_0, NVFP4, IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ1_S, IQ1_M, IQ3_S, IQ4_NL, IQ4_XS, I32, MXFP4, Q2_K, Q3_K, Q5_K, and Q6_K in GET_ROWS OP

* correct the link
 2026-06-01 09:53:04 +03:00
Neo ZhangandGitHub 44e211cecf sycl : Optimize Q3_K mul_mat by reorder (#23725) 2026-06-01 09:50:55 +03:00
Neo ZhangandGitHub aabee047d8 [SCYL] add chapter for performance reference in SYCL.md (#23315) 
* add chapter for performance reference

* rm unsupported GPU
 2026-05-19 09:44:51 +03:00
Neo ZhangandGitHub e98bcfec28 sycl : fix error when use -mg 1 error (#23140) 2026-05-18 08:11:19 +03:00
Neo ZhangandGitHub 0f45f1a35c docker : revert stable version of intel compute-runtime (#22968) 2026-05-14 11:30:40 +02:00
Neo ZhangandGitHub 7d442abf5c [SYCL] Add OP im2col_3d (#22903) 
* add im2col_3d

* format code

* update the ops.md
 2026-05-11 08:01:47 +03:00
Neo ZhangandGitHub 6a2a2513dc fix script error (#22795sycl : ) 2026-05-08 06:54:57 +03:00
Neo ZhangandGitHub eddd7a13a5 [SYCL] Optimize Q4_0 mul_mat for Arc770, add scripts (#22291) 
* opt arc770 for Q4_0

* add for Q4_0

* update the script

* add help script for windows

* update guide

* fix format issue

* convert from dos to unix for format issue

* fix missed -sm parameter
 2026-04-25 09:20:14 +03:00
Neo ZhangandGitHub f51fd36d79 sycl : handle other FA case (#21377) 2026-04-06 13:28:00 +03:00
Neo ZhangandGitHub 4888137b17 sycl : fix llama_kv_cache hang when kv_cache is huge: 5GB (#21283) 2026-04-02 10:08:32 +03:00
Neo ZhangandGitHub 6b949d1078 sycl : support nvfp4 type in mul_mat (#21227) 2026-04-01 13:54:15 +03:00
Neo ZhangandGitHub 62278cedde sycl : enhance fattn perf (#21185) 2026-03-31 13:31:50 +03:00
Neo ZhangandGitHub afe65aa282 [SYCL] Enhance build script to use half cores to build, avoid OS hang (#21093) 
* use half cores to build, avoid OS hang

* reduce the output text num to short test time

* avoid to return 0
 2026-03-29 09:02:45 +08:00
Neo ZhangandGitHub 53dc8b59bf sycl : fix wrong variable check by assert (#20903) 
* fix wrong variable check by assert

* use GGML api
 2026-03-25 11:48:37 +02:00
Neo ZhangandGitHub f40a80b4f3 support bf16 and quantized type (#20803) 2026-03-22 22:06:27 +08:00
Neo ZhangandGitHub b6c83aad55 [SYCL] ehance UPSCALE to support all UT cases (#20637) 
* [SYCL] ehance UPSCALE to support more cases

* rm test case result of SYCL1
 2026-03-17 10:01:52 +08:00
Neo ZhangandGitHub a93c0ef0fa add op gated_delta_net (#20455) 2026-03-14 22:01:57 +08:00
Neo ZhangandGitHub ecac98ee53 [SYCL] Update SYCL.md for binary package for Windows (#20401) 
* add download binary package

* update prefix
 2026-03-11 22:21:22 +08:00
Neo ZhangandGitHub 0cec84f999 fix op rope, add rope_back (#20293) 2026-03-11 09:53:34 +08:00
Neo ZhangandGitHub b2e1427c9b fix for failed UT case: ACC, L2_NORM, UPSCALE, fused_glu, unary (#20283) 2026-03-11 09:53:05 +08:00
Neo ZhangandGitHub 213c4a0b81 [SYCL] supprt Flash Attention for fp32/fp16/Q4/Q5/Q8 (#20190) 
* support flash-attention for fp32/fp16/Q4/Q5/Q8

* rm warining

* update for JIT
 2026-03-08 12:00:07 +08:00
Neo ZhangGitHubNeo Zhang Jianyu
 		c17dce4f5c replace the magic nunber 768 by max work group size to support iGPU (#19920) 
Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2026-02-27 09:26:07 +08:00
Neo ZhangGitHubNeo Zhang Jianyu
 		2943210c1e support permuted, remove check s0/s10 (#19889) 
Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2026-02-26 10:27:20 +08:00
Neo ZhangandGitHub bf38346d13 Remove support for Nvidia & AMD GPU, because the oneAPI plugin for Nvidia & AMD GPU is unavailable: download/installation channels are out of work. (#19246) 
User can't build up the software for Nvidia & AMD GPU.
rm the oneMath since it is only used in NV and AMD code path.
 2026-02-02 21:06:21 +08:00
Neo ZhangandGitHub 2634ed207a create test.sh to enhance the parameters for testing, update the guide, rm useless script (#19243) 2026-02-01 18:24:00 +08:00
Neo ZhangGitHubNeo Zhang Jianyu
 		d4964a7c66 sycl: fix norm kernels: l2_norm, group_norm, rms_norm by remove assert to support more cases (#19154) 
Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2026-01-29 09:20:22 +08:00
Neo ZhangGitHubNeo Zhang Jianyu
 		cb6caca191 [SYCL] use malloc to support both iGPU and dGPU in same time (#18992) 
* use malloc to support both iGPU and dGPU in same time

* support windows

---------

Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2026-01-23 20:54:10 +08:00
Neo ZhangGitHubSigbjørn SkjæretNeo Zhang Jianyu
 		a6a552e4ec [SYCL] replace llama-cli by llama-completion to rm the impact to test script (#18290) 
* replace llama-cli by llama-completion to rm the impact to test script

* Update examples/sycl/run-llama2.sh

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update examples/sycl/run-llama2.sh

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update examples/sycl/run-llama3.sh

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update examples/sycl/run-llama3.sh

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update examples/sycl/win-run-llama2.bat

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

* Update examples/sycl/win-run-llama3.bat

Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>

---------

Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
 2025-12-23 12:59:12 +08:00
Neo ZhangandGitHub 68522c678d ci : support bfloat16 SYCL release package (#17855) 
* support bfloat16 release package

* add fallback file
 2025-12-08 15:09:39 +01:00
Neo ZhangGitHubNeo Zhang Jianyu
 		7d2add51d8 sycl : support to malloc memory on device more than 4GB, update the doc and script (#17566) 
Co-authored-by: Neo Zhang Jianyu <jianyu.zhang@intel.com>
 2025-11-29 14:59:44 +02:00
Neo ZhangGitHubNeo Zhang <>
 		a21c6fd450 update guide (#8909) 
Co-authored-by: Neo Zhang <>
 2024-08-11 14:07:43 +05:30
Neo ZhangandGitHub d4ff847153 [SYCL] correct cmd name (#8877) 2024-08-06 09:09:12 +08:00
Neo ZhangandGitHub df68d4fa5d [SYCL] Update README-sycl.md for Chapter "Recommended release" and "News" (#7946) 
* Update README-sycl.md

* Update README-sycl.md

* Update README-sycl.md

* Update README-sycl.md
 2024-06-17 11:17:07 +08:00