wylab/llama.cpp
0
0
Fork 1
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-06 17:57:44 +02:00
Code Issues Packages Projects Releases Wiki Activity

vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize (#27812)

Browse Source 
* vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize

is_src_of doesn't treat two views of one tensor as dependent, so the optimizer reorders nodes across aliased reads and writes. 

Result: silently wrong tokens under greedy decoding, different output on every server start, and invalid speculative-decoding acceptance, with nothing logged.

Hits Qwen3.8's recurrent state (and any model with view-aliased state) on AMD and NVIDIA Vulkan.  CUDA is clean. 

Compare view_src bases on both sides.

Fixes #27805

* vulkan: don't treat view/no-op nodes as aliasing dependencies

Nodes whose op is NONE, RESHAPE, TRANSPOSE, VIEW or PERMUTE execute nothing, so aliasing through them is not a real dependency. The previous base comparison matched them anyway, which only costs the optimizer reordering freedom.

Co-authored-by: Jeff Bolz <jbolz@nvidia.com>

* vulkan: make the lambda parameter const and capture is_empty in is_src_of

Code will not compile without these changes.  
is_src_of has an empty capture list, so is_empty was not visible inside it, and is_empty took a non-const pointer, while is_src_of receives const ones. Other call sites pass non-const pointers, which still convert as usual.

---------

Co-authored-by: Jeff Bolz <jbolz@nvidia.com>
This commit is contained in:
Eric A Stalee
2026-08-28 19:12:33 +02:00
committed by GitHub
co-authored by Jeff Bolz
parent a43c3986b4
commit b387ddfd84
1 changed files with 17 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ggml/src/ggml-vulkan/ggml-vulkan.cpp
+17 -5
View File
@@ -17800,20 +17800,32 @@ static void ggml_vk_graph_optimize(ggml_backend_t backend, struct ggml_cgraph *
return;
}
auto const &is_empty = [](ggml_tensor * node) -> bool {
auto const &is_empty = [](const ggml_tensor * node) -> bool {
return node->op == GGML_OP_NONE || node->op == GGML_OP_RESHAPE || node->op == GGML_OP_TRANSPOSE || node->op == GGML_OP_VIEW || node->op == GGML_OP_PERMUTE;
};
auto const &is_src_of = [](const ggml_tensor *dst, const ggml_tensor *src) -> bool {
auto const &is_src_of = [&is_empty](const ggml_tensor *dst, const ggml_tensor *src) -> bool {
auto const &base = [](const ggml_tensor * tensor) {
return tensor->view_src ? tensor->view_src : tensor;
};
for (uint32_t s = 0; s < GGML_MAX_SRC; ++s) {
if (dst->src[s] == src) {
return true;
}
if (is_empty(dst) || is_empty(src)) {
continue;
}
// A source view of dst may read storage written through a different view by src.
if (dst->src[s] && base(dst->src[s]) == base(src)) {
return true;
}
// Moving dst forward may overwrite storage still read through a view by src.
if (src->src[s] && base(dst) == base(src->src[s])) {
return true;
}
}
// implicit dependency if they view the same tensor
const ggml_tensor *dst2 = dst->view_src ? dst->view_src : dst;
const ggml_tensor *src2 = src->view_src ? src->view_src : src;
if (dst2 == src2) {
if (base(dst) == base(src)) {
return true;
}
return false;