mtmd: support multi-row batching for deepseek-ocr (#26154 )

* mtmd: support multi-row batching for deepseek-ocr * mtmd: weave deepseek-ocr rows in one shot instead of per row (#26615) --------- Co-authored-by: Saba Fallah <sabafallah@gmail.com>
fit: Fix memory allocation for MTP layers (#26605 )
2026-08-05 19:28:03 +02:00 · 2026-08-05 13:34:52 +02:00 · 2026-08-05 13:29:45 +02:00 · 2026-08-05 13:27:06 +02:00 · 2026-08-05 12:36:01 +02:00 · 2026-08-05 18:06:09 +08:00
50 changed files with 1456 additions and 792 deletions
@@ -60,7 +60,6 @@ jobs:
            -DCMAKE_BUILD_RPATH="@loader_path" \
            -DLLAMA_FATAL_WARNINGS=ON \
            -DLLAMA_BUILD_BORINGSSL=ON \
-            -DGGML_METAL_USE_BF16=ON \
            -DGGML_METAL_EMBED_LIBRARY=OFF \
            -DGGML_METAL_SHADER_DEBUG=ON \
            -DGGML_RPC=ON \
@@ -127,7 +126,6 @@ jobs:
        run: |
          sysctl -a
          cmake -B build -G Xcode \
-            -DGGML_METAL_USE_BF16=ON \
            -DGGML_METAL_EMBED_LIBRARY=ON \
            -DLLAMA_OPENSSL=OFF \
            -DLLAMA_BUILD_APP=OFF \
@@ -178,7 +176,6 @@ jobs:
        run: |
          sysctl -a
          cmake -B build -G Xcode \
-            -DGGML_METAL_USE_BF16=ON \
            -DGGML_METAL_EMBED_LIBRARY=ON \
            -DLLAMA_BUILD_COMMON=OFF \
            -DLLAMA_BUILD_APP=OFF \
@@ -212,7 +209,6 @@ jobs:
        run: |
          sysctl -a
          cmake -B build -G Xcode \
-            -DGGML_METAL_USE_BF16=ON \
            -DGGML_METAL_EMBED_LIBRARY=ON \
            -DLLAMA_BUILD_COMMON=OFF \
            -DLLAMA_BUILD_APP=OFF \
@@ -257,7 +253,6 @@ jobs:
        run: |
          sysctl -a
          cmake -B build -G Xcode \
-            -DGGML_METAL_USE_BF16=ON \
            -DGGML_METAL_EMBED_LIBRARY=ON \
            -DLLAMA_OPENSSL=OFF \
            -DLLAMA_BUILD_APP=OFF \
@@ -93,13 +93,13 @@ jobs:
          - build: 'arm64'
            arch: 'arm64'
            os: macos-26
-            defines: "-DGGML_METAL_USE_BF16=ON -DGGML_METAL_EMBED_LIBRARY=ON -DCMAKE_OSX_DEPLOYMENT_TARGET=13.3"
+            defines: "-DGGML_METAL_EMBED_LIBRARY=ON -DCMAKE_OSX_DEPLOYMENT_TARGET=13.3"
          # TODO: this build is disabled to save Github Actions resources (https://github.com/ggml-org/llama.cpp/pull/23780)
          #       in order to enable it again, we have to provision dedicated runners  to run it
          #- build: 'arm64-kleidiai'
          #  arch: 'arm64'
          #  os: macos-14
-          #  defines: "-DGGML_METAL_USE_BF16=ON -DGGML_METAL_EMBED_LIBRARY=ON -DCMAKE_OSX_DEPLOYMENT_TARGET=13.3 -DGGML_CPU_KLEIDIAI=ON"
+          #  defines: "-DGGML_METAL_EMBED_LIBRARY=ON -DCMAKE_OSX_DEPLOYMENT_TARGET=13.3 -DGGML_CPU_KLEIDIAI=ON"
          - build: 'x64'
            arch: 'x64'
            os: macos-15-intel
@@ -1402,7 +1402,6 @@ jobs:
        run: |
          sysctl -a
          cmake -B build -G Xcode \
-            -DGGML_METAL_USE_BF16=ON \
            -DGGML_METAL_EMBED_LIBRARY=ON \
            -DLLAMA_OPENSSL=OFF \
            -DLLAMA_BUILD_APP=OFF \
@@ -21,11 +21,18 @@ Please disclose it as a private [security advisory](https://github.com/ggml-org/

 A team of volunteers on a reasonable-effort basis maintains this project. As such, please give us at least 90 days to work on a fix before public exposure.

+### AI-powered code scan
+
+llama.cpp has an AI security scanner that scans the code periodically. The full prompts and tool set can be found in [ggml-org/security-scan-prompt](https://github.com/ggml-org/security-scan-prompt).
+
+We greatly appreciate reports that reflect genuine research effort, and we are happy to spend our time reviewing them. Findings that an autonomous AI agent can surface on its own add little on top of the scans we already run.
+
 ### Requirements

 Before submitting your report, ensure you meet the following requirements:

 - You have read this policy and fully understand it.
+- You have searched for existing discussions of the issue. If it has already been reported, your report will likely be rejected as a duplicate.
 - AI is only permitted in an assistive capacity as stated in [AGENTS.md](AGENTS.md). We do not accept reports that are written exclusively by AI.
 - Your report must include a working Proof-of-Concept in the form of a script and/or attached files.

@@ -46,6 +53,8 @@ Only vulnerabilities that fall within these parts of the project are considered

 Note that none of the topics under [Using llama.cpp securely](#using-llamacpp-securely) are considered vulnerabilities in LLaMA C++.

+Denial-of-Service (DoS) bugs are generally not treated as vulnerabilities. We don't reject them outright, but we look at them case-by-case and only accept those that are genuinely worth fixing.
+
 For vulnerabilities that fall within the `vendor` directory, please report them directly to the third-party project.

 ## Using llama.cpp securely
@@ -17,7 +17,6 @@ LLAMA_BUILD_MTMD=ON
 GGML_METAL=ON
 GGML_METAL_EMBED_LIBRARY=ON
 GGML_BLAS_DEFAULT=ON
-GGML_METAL_USE_BF16=ON
 GGML_OPENMP=OFF

 COMMON_C_FLAGS="-Wno-macro-redefined -Wno-shorten-64-to-32 -Wno-unused-command-line-argument -g"
@@ -44,7 +43,6 @@ COMMON_CMAKE_ARGS=(
    -DGGML_METAL_EMBED_LIBRARY=${GGML_METAL_EMBED_LIBRARY}
    -DGGML_BLAS_DEFAULT=${GGML_BLAS_DEFAULT}
    -DGGML_METAL=${GGML_METAL}
-    -DGGML_METAL_USE_BF16=${GGML_METAL_USE_BF16}
    -DGGML_NATIVE=OFF
    -DGGML_OPENMP=${GGML_OPENMP}
 )
@@ -2008,9 +2008,9 @@ common_params_context common_params_parser_init(common_params & params, llama_ex
    ).set_sampling());
    add_opt(common_arg(
        {"--repeat-last-n"}, "N",
-        string_format("last n tokens to consider for penalize (default: %d, 0 = disabled, -1 = ctx_size)", params.sampling.penalty_last_n),
+        string_format("last n tokens to consider for penalize (default: %d, 0 = disabled)", params.sampling.penalty_last_n),
        [](common_params & params, int value) {
-            if (value < -1) {
+            if (value < 0) {
                throw std::runtime_error(string_format("error: invalid repeat-last-n = %d\n", value));
            }
            params.sampling.penalty_last_n = value;
@@ -2081,9 +2081,9 @@ common_params_context common_params_parser_init(common_params & params, llama_ex
    ).set_sampling());
    add_opt(common_arg(
        {"--dry-penalty-last-n"}, "N",
-        string_format("set DRY penalty for the last n tokens (default: %d, 0 = disable, -1 = context size)", params.sampling.dry_penalty_last_n),
+        string_format("set DRY penalty for the last n tokens (default: %d, 0 = disable)", params.sampling.dry_penalty_last_n),
        [](common_params & params, int value) {
-            if (value < -1) {
+            if (value < 0) {
                throw std::runtime_error(string_format("error: invalid dry-penalty-last-n = %d\n", value));
            }
            params.sampling.dry_penalty_last_n = value;
@@ -1302,23 +1302,12 @@ common_init_result::common_init_result(common_params & params, bool model_only)
                params.sampling.logit_bias_eog.begin(), params.sampling.logit_bias_eog.end());
    }

-    //if (params.sampling.penalty_last_n == -1) {
-    //    LOG_TRC("%s: setting penalty_last_n to ctx_size = %d\n", __func__, llama_n_ctx(lctx));
-    //    params.sampling.penalty_last_n = llama_n_ctx(lctx);
-    //}
-
-    //if (params.sampling.dry_penalty_last_n == -1) {
-    //    LOG_TRC("%s: setting dry_penalty_last_n to ctx_size = %d\n", __func__, llama_n_ctx(lctx));
-    //    params.sampling.dry_penalty_last_n = llama_n_ctx(lctx);
-    //}
-
    // init the backend samplers as part of the context creation
    pimpl->samplers.resize(cparams.n_seq_max);
    pimpl->samplers_seq_config.resize(cparams.n_seq_max);

-    const int32_t n_ctx = cparams.n_ctx > 0 ? (int32_t) cparams.n_ctx : llama_model_n_ctx_train(model);
    for (int i = 0; i < (int) cparams.n_seq_max; ++i) {
-        pimpl->samplers[i].reset(common_sampler_init(model, params.sampling, n_ctx));
+        pimpl->samplers[i].reset(common_sampler_init(model, params.sampling));
        pimpl->samplers_seq_config[i] = { i, common_sampler_get(pimpl->samplers[i].get()) };
    }

@@ -235,14 +235,14 @@ struct common_params_sampling {
    float   temp               = 0.80f;  // <= 0.0 to sample greedily, 0.0 to not output probabilities
    float   dynatemp_range     = 0.00f;  // 0.0 = disabled
    float   dynatemp_exponent  = 1.00f;  // controls how entropy maps to temperature in dynamic temperature sampler
-    int32_t penalty_last_n     = 64;     // last n tokens to penalize (0 = disable penalty, -1 = context size)
+    int32_t penalty_last_n     = 64;     // last n tokens to penalize (0 = disable penalty)
    float   penalty_repeat     = 1.00f;  // 1.0 = disabled
    float   penalty_freq       = 0.00f;  // 0.0 = disabled
    float   penalty_present    = 0.00f;  // 0.0 = disabled
    float   dry_multiplier     = 0.0f;   // 0.0 = disabled;      DRY repetition penalty for tokens extending repetition:
    float   dry_base           = 1.75f;  // 0.0 = disabled;      multiplier * base ^ (length of sequence before token - allowed length)
    int32_t dry_allowed_length = 2;      // tokens extending repetitions beyond this receive penalty
-    int32_t dry_penalty_last_n = -1;     // how many tokens to scan for repetitions (0 = disable penalty, -1 = context size)
+    int32_t dry_penalty_last_n = 64;     // how many tokens to scan for repetitions (0 = disable penalty)
    float   adaptive_target    = -1.0f;  // select tokens near this probability (valid range 0.0 to 1.0; negative = disabled)
    float   adaptive_decay     = 0.90f;  // EMA decay for adaptation; history ≈ 1/(1-decay) tokens (0.0 - 0.99)
    int32_t mirostat           = 0;      // 0 = disabled, 1 = mirostat, 2 = mirostat 2.0
@@ -136,7 +136,10 @@ static std::vector<llama_device_memory_data> common_get_device_memory_data_impl(
        devs.push_back(llama_model_get_device(model, i));
    }

-    hp_ngl         = llama_model_n_layer(model) + llama_model_n_layer_nextn(model);
+    hp_ngl         = llama_model_n_layer(model);
+    if (mparams->load_mtp) {
+        hp_ngl    += llama_model_n_layer_nextn(model);
+    }
    hp_n_ctx_train = llama_model_n_ctx_train(model);
    hp_n_expert    = llama_model_n_expert(model);

@@ -186,8 +186,7 @@ std::string common_params_sampling::print() const {

 struct common_sampler * common_sampler_init(
        const struct llama_model * model,
-        struct common_params_sampling & params,
-        int32_t n_ctx) {
+        struct common_params_sampling & params) {
    if (!std::isfinite(params.penalty_repeat) ||
        params.penalty_repeat <= 0.0f ||
        !std::isfinite(1.0f/params.penalty_repeat)) {
@@ -199,10 +198,6 @@ struct common_sampler * common_sampler_init(
    if (!std::isfinite(params.penalty_present)) {
        throw std::invalid_argument("penalty_present must be finite");
    }
-    if (params.penalty_last_n == -1) {
-        params.penalty_last_n = n_ctx > 0 ? n_ctx : llama_model_n_ctx_train(model);
-    }
-
    const llama_vocab * vocab = llama_model_get_vocab(model);
    llama_sampler_chain_params lparams = llama_sampler_chain_default_params();

@@ -355,7 +350,7 @@ struct common_sampler * common_sampler_init(
                        for (const auto & str : params.dry_sequence_breakers) {
                            c_breakers.push_back(str.c_str());
                        }
-                        samplers.push_back(llama_sampler_init_dry(vocab, llama_model_n_ctx_train(model), params.dry_multiplier, params.dry_base, params.dry_allowed_length, params.dry_penalty_last_n, c_breakers.data(), c_breakers.size()));
+                        samplers.push_back(llama_sampler_init_dry(vocab, params.dry_multiplier, params.dry_base, params.dry_allowed_length, params.dry_penalty_last_n, c_breakers.data(), c_breakers.size()));
                    }
                    break;
                case COMMON_SAMPLER_TYPE_TOP_K:
@@ -39,8 +39,7 @@ struct common_sampler;
 // note: can mutate params in some cases
 struct common_sampler * common_sampler_init(
        const struct llama_model * model,
-        struct common_params_sampling & params,
-        int32_t n_ctx = 0);
+        struct common_params_sampling & params);

 void common_sampler_free(struct common_sampler * gsmpl);

@@ -59,11 +59,29 @@ def byteswap_q6_k(tensor, block_offs):
    delta.byteswap(inplace=True)


+def byteswap_q1_0(tensor, block_offs):
+    # Each block_q1_0 consists of an f16 delta followed by 16 int8 quantizations.
+
+    # Byte-Swap f16 sized delta field
+    delta = tensor.data[block_offs:block_offs + 2].view(dtype=np.uint16)
+    delta.byteswap(inplace=True)
+
+
+def byteswap_tq2_0(tensor, block_offs):
+    # Each block_tq2_0 consists of 64 int8 values followed by 1 f16 value.
+
+    # Byte-Swap f16 sized field
+    delta = tensor.data[block_offs + 64:block_offs + 66].view(dtype=np.uint16)
+    delta.byteswap(inplace=True)
+
+
 byteswap_tensors = {
+    gguf.GGMLQuantizationType.Q1_0:  byteswap_q1_0,
    gguf.GGMLQuantizationType.Q4_0:  byteswap_q4_0,
    gguf.GGMLQuantizationType.Q8_0:  byteswap_q8_0,
    gguf.GGMLQuantizationType.Q4_K:  byteswap_q4_k,
    gguf.GGMLQuantizationType.Q6_K:  byteswap_q6_k,
+    gguf.GGMLQuantizationType.TQ2_0: byteswap_tq2_0,
    gguf.GGMLQuantizationType.MXFP4: byteswap_noop,
    gguf.GGMLQuantizationType.NVFP4: byteswap_noop,
 }
@@ -1425,7 +1425,7 @@ extern "C" {
    /// NOTE: Avoid using on the full vocabulary as searching for repeated tokens can become slow. For example, apply top-k or top-p sampling first.
    LLAMA_API struct llama_sampler * llama_sampler_init_penalties(
                             int32_t   n_vocab,
-                             int32_t   penalty_last_n,   // last n tokens to penalize (0 = disable penalty, -1 = context size)
+                             int32_t   penalty_last_n,   // last n tokens to penalize (0 = disable penalty)
                               float   penalty_repeat,   // must be > 0.0, 1.0 = disabled
                               float   penalty_freq,     // must be finite, 0.0 = disabled
                               float   penalty_present); // must be finite, 0.0 = disabled
@@ -1433,11 +1433,10 @@ extern "C" {
    ///  @details DRY sampler, designed by p-e-w, as described in: https://github.com/oobabooga/text-generation-webui/pull/5677, porting Koboldcpp implementation authored by pi6am: https://github.com/LostRuins/koboldcpp/pull/982
    LLAMA_API struct llama_sampler * llama_sampler_init_dry(
            const struct llama_vocab *  vocab,
-                             int32_t    n_ctx_train,
                               float    dry_multiplier,
                               float    dry_base,
                             int32_t    dry_allowed_length,
-                             int32_t    dry_penalty_last_n,
+                             int32_t    dry_penalty_last_n, // last n tokens to penalize (0 = disable penalty)
                          const char ** seq_breakers,
                              size_t    num_breakers);

@@ -24,10 +24,31 @@ vendor = {
    "https://raw.githubusercontent.com/sheredom/subprocess.h/8671cee1fc09f11a70ce3782a0ee13177c3aa387/subprocess.h": "vendor/sheredom/subprocess.h",
 }

+# TODO @ngxson : this is temporary, to be removed in the future
+patches = [
+    # https://github.com/sheredom/subprocess.h/pull/102
+    "vendor/sheredom/patch-bsd.patch",
+    # https://github.com/sheredom/subprocess.h/pull/101
+    "vendor/sheredom/patch-windows-quote-backslash.patch",
+    # https://github.com/sheredom/subprocess.h/pull/104
+    # note: must be applied after patch-bsd.patch, they touch adjacent lines
+    "vendor/sheredom/patch-glibc-older-than-2.29.patch",
+]
+
 for url, filename in vendor.items():
    print(f"downloading {url} to {filename}") # noqa: NP100
    urllib.request.urlretrieve(url, filename)

+for patch in patches:
+    print(f"applying {patch}") # noqa: NP100
+    try:
+        subprocess.check_call([
+            "git", "apply", "--directory", os.path.dirname(patch), patch
+        ])
+    except Exception as e:
+        print(f"Error: {e}") # noqa: NP100
+        sys.exit(1)
+
 print("Splitting httplib.h...") # noqa: NP100
 try:
    subprocess.check_call([
@@ -123,15 +123,15 @@ function(npm_build out_var)
    endif()

    if(need_install)
-        message(STATUS "UI: running npm install")
+        message(STATUS "UI: running npm ci")
        execute_process(
-            COMMAND ${NPM_EXECUTABLE} install
+            COMMAND ${NPM_EXECUTABLE} ci
            WORKING_DIRECTORY "${WORK_DIR}"
            RESULT_VARIABLE rc
            ERROR_VARIABLE  err
        )
        if(NOT rc EQUAL 0)
-            message(STATUS "UI: npm install failed (${rc})")
+            message(STATUS "UI: npm ci failed (${rc})")
            message(STATUS "  stderr: ${err}")
            return()
        endif()
@@ -2890,6 +2890,21 @@ void llama_model_base::create_tensor_qkv(llama_layer & layer, int bid,
        int64_t n_embd_, int64_t n_embd_q_, int64_t n_embd_k_, int64_t n_embd_v_,
        int flags) {
    const int64_t n_embd_qkv = n_embd_q_ + n_embd_k_ + n_embd_v_;
+
+    if (flags & TENSOR_SKIP) {
+        const int skip = TENSOR_NOT_REQUIRED | TENSOR_SKIP;
+
+        create_tensor(tn(LLM_TENSOR_ATTN_QKV, "weight", bid), {n_embd_, n_embd_qkv}, skip | TENSOR_SKIP_IF_VIRTUAL);
+        create_tensor(tn(LLM_TENSOR_ATTN_QKV, "bias",   bid), {n_embd_qkv},          skip | TENSOR_SKIP_IF_VIRTUAL);
+        create_tensor(tn(LLM_TENSOR_ATTN_Q,   "weight", bid), {n_embd_, n_embd_q_},  skip);
+        create_tensor(tn(LLM_TENSOR_ATTN_K,   "weight", bid), {n_embd_, n_embd_k_},  skip);
+        create_tensor(tn(LLM_TENSOR_ATTN_V,   "weight", bid), {n_embd_, n_embd_v_},  skip);
+        create_tensor(tn(LLM_TENSOR_ATTN_Q,   "bias",   bid), {n_embd_q_},           skip);
+        create_tensor(tn(LLM_TENSOR_ATTN_K,   "bias",   bid), {n_embd_k_},           skip);
+        create_tensor(tn(LLM_TENSOR_ATTN_V,   "bias",   bid), {n_embd_v_},           skip);
+        return;
+    }
+
    layer.wqkv = create_tensor(tn(LLM_TENSOR_ATTN_QKV, "weight", bid), {n_embd_, n_embd_qkv}, TENSOR_NOT_REQUIRED | TENSOR_SKIP_IF_VIRTUAL);
    if (layer.wqkv) {
        layer.wqkv_b = create_tensor(tn(LLM_TENSOR_ATTN_QKV, "bias", bid), {n_embd_qkv}, TENSOR_NOT_REQUIRED | TENSOR_SKIP_IF_VIRTUAL);
@@ -3078,8 +3078,6 @@ struct llama_sampler * llama_sampler_init_top_n_sigma(float n) {
 // DRY

 struct llama_sampler_dry {
-    int32_t total_context_size;
-
    const float   dry_multiplier;
    const float   dry_base;
    const int32_t dry_allowed_length;
@@ -3155,8 +3153,7 @@ static void llama_sampler_dry_apply(struct llama_sampler * smpl, llama_token_dat
        return;
    }

-    int32_t effective_dry_penalty_last_n = (ctx->dry_penalty_last_n == -1) ? ctx->total_context_size : std::max(ctx->dry_penalty_last_n, 0);
-    int last_n_repeat = std::min(std::min((int)ctx->last_tokens.size(), effective_dry_penalty_last_n), ctx->total_context_size);
+    int last_n_repeat = std::min((int) ctx->last_tokens.size(), ctx->dry_penalty_last_n);

    if (last_n_repeat <= ctx->dry_allowed_length) {
        return;
@@ -3369,7 +3366,7 @@ static struct llama_sampler * llama_sampler_dry_clone(const struct llama_sampler
    llama_vocab dummy_vocab;

    // dummy vocab is passed because it is only needed for raw sequence breaker processing, which we have already done and will simply be copying
-    auto * result = llama_sampler_init_dry(&dummy_vocab, ctx->total_context_size, ctx->dry_multiplier, ctx->dry_base, ctx->dry_allowed_length, ctx->dry_penalty_last_n, NULL, 0);
+    auto * result = llama_sampler_init_dry(&dummy_vocab, ctx->dry_multiplier, ctx->dry_base, ctx->dry_allowed_length, ctx->dry_penalty_last_n, NULL, 0);

    // Copy the state, including the processed breakers
    {
@@ -3400,8 +3397,8 @@ static struct llama_sampler_i llama_sampler_dry_i = {
    /* .backend_set_input = */ nullptr,
 };

-struct llama_sampler * llama_sampler_init_dry(const struct llama_vocab * vocab, int32_t n_ctx_train, float dry_multiplier, float dry_base, int32_t dry_allowed_length, int32_t dry_penalty_last_n, const char** seq_breakers, size_t num_breakers) {
-    int32_t effective_dry_penalty_last_n = (dry_penalty_last_n == -1) ? n_ctx_train : std::max(dry_penalty_last_n, 0);
+struct llama_sampler * llama_sampler_init_dry(const struct llama_vocab * vocab, float dry_multiplier, float dry_base, int32_t dry_allowed_length, int32_t dry_penalty_last_n, const char** seq_breakers, size_t num_breakers) {
+    dry_penalty_last_n = std::max(dry_penalty_last_n, 0);
    std::unordered_multimap<llama_token, std::vector<llama_token>> processed_breakers;
    const int MAX_CHAR_LEN = 40;
    const int MAX_SEQ_LEN = 20;
@@ -3438,23 +3435,22 @@ struct llama_sampler * llama_sampler_init_dry(const struct llama_vocab * vocab,
    return llama_sampler_init(
        /* .iface = */ &llama_sampler_dry_i,
        /* .ctx   = */ new llama_sampler_dry {
-            /* .total_context_size     = */ n_ctx_train,
            /* .dry_multiplier         = */ dry_multiplier,
            /* .dry_base               = */ dry_base,
            /* .dry_allowed_length     = */ dry_allowed_length,
            /* .dry_penalty_last_n     = */ dry_penalty_last_n,
            /* .dry_processed_breakers = */ std::move(processed_breakers),
-            /* .dry_repeat_count       = */ dry_enabled ? std::vector<int>(effective_dry_penalty_last_n, 0) : std::vector<int>{},
+            /* .dry_repeat_count       = */ dry_enabled ? std::vector<int>(dry_penalty_last_n, 0) : std::vector<int>{},
            /* .dry_max_token_repeat   = */ {},
-            /* .last_tokens            = */ dry_enabled ? ring_buffer<llama_token>(effective_dry_penalty_last_n) : ring_buffer<llama_token>(0),
+            /* .last_tokens            = */ dry_enabled ? ring_buffer<llama_token>(dry_penalty_last_n) : ring_buffer<llama_token>(0),
        }
    );
 }

 // wrapper for test-sampling.cpp
-struct llama_sampler * llama_sampler_init_dry_testing(int32_t context_size, float dry_multiplier, float dry_base, int32_t dry_allowed_length, int32_t dry_penalty_last_n, const std::vector<std::vector<llama_token>>& seq_breakers) {
+struct llama_sampler * llama_sampler_init_dry_testing(float dry_multiplier, float dry_base, int32_t dry_allowed_length, int32_t dry_penalty_last_n, const std::vector<std::vector<llama_token>>& seq_breakers) {
    llama_vocab dummy_vocab;
-    auto * result = llama_sampler_init_dry(&dummy_vocab, context_size, dry_multiplier, dry_base, dry_allowed_length, dry_penalty_last_n, NULL, 0);
+    auto * result = llama_sampler_init_dry(&dummy_vocab, dry_multiplier, dry_base, dry_allowed_length, dry_penalty_last_n, NULL, 0);
    auto * ctx = (llama_sampler_dry *) result->ctx;

    // Process the token-based sequence breakers
@@ -34,7 +34,6 @@ struct llama_sampler_chain {
 };

 struct llama_sampler * llama_sampler_init_dry_testing(
-        int32_t context_size,
        float   dry_multiplier,
        float   dry_base,
        int32_t dry_allowed_length,
@@ -101,6 +101,14 @@ static void test(void) {

    {
        common_params penalty_params;
+        assert(penalty_params.sampling.penalty_last_n == 64);
+        assert(penalty_params.sampling.dry_penalty_last_n == 64);
+
+        argv = {"binary_name", "--repeat-last-n", "-1"};
+        assert(false == common_params_parse(argv.size(), list_str_to_char(argv).data(), penalty_params, LLAMA_EXAMPLE_COMMON));
+
+        argv = {"binary_name", "--dry-penalty-last-n", "-1"};
+        assert(false == common_params_parse(argv.size(), list_str_to_char(argv).data(), penalty_params, LLAMA_EXAMPLE_COMMON));

        argv = {"binary_name", "--repeat-penalty", "0"};
        assert(false == common_params_parse(argv.size(), list_str_to_char(argv).data(), penalty_params, LLAMA_EXAMPLE_COMMON));
@@ -10,7 +10,7 @@
 #include <string>
 #include <vector>

-extern struct llama_sampler * llama_sampler_init_dry_testing(int32_t context_size, float dry_multiplier, float dry_base, int32_t dry_allowed_length, int32_t dry_penalty_last_n, const std::vector<std::vector<llama_token>>& seq_breakers);
+extern struct llama_sampler * llama_sampler_init_dry_testing(float dry_multiplier, float dry_base, int32_t dry_allowed_length, int32_t dry_penalty_last_n, const std::vector<std::vector<llama_token>>& seq_breakers);

 static void dump(const llama_token_data_array * cur_p) {
    for (size_t i = 0; i < cur_p->size; i++) {
@@ -168,7 +168,7 @@ static void test_dry(

    sampler_tester tester(probs, expected_probs);

-    auto * sampler = llama_sampler_init_dry_testing(1024, dry_multiplier, dry_base, dry_allowed_length, dry_penalty_last_n, seq_breakers);
+    auto * sampler = llama_sampler_init_dry_testing(dry_multiplier, dry_base, dry_allowed_length, dry_penalty_last_n, seq_breakers);

    for (size_t i = 0; i < last_tokens.size(); i++) {
        llama_sampler_accept(sampler, last_tokens[i]);
@@ -116,14 +116,14 @@
 | `--xtc-probability N` | xtc probability (default: 0.00, 0.0 = disabled) |
 | `--xtc-threshold N` | xtc threshold (default: 0.10, 1.0 = disabled) |
 | `--typical, --typical-p N` | locally typical sampling, parameter p (default: 1.00, 1.0 = disabled) |
-| `--repeat-last-n N` | last n tokens to consider for penalize (default: 64, 0 = disabled, -1 = ctx_size) |
+| `--repeat-last-n N` | last n tokens to consider for penalize (default: 64, 0 = disabled) |
 | `--repeat-penalty N` | penalize repeat sequence of tokens (default: 1.00, 1.0 = disabled) |
 | `--presence-penalty N` | repeat alpha presence penalty (default: 0.00, 0.0 = disabled) |
 | `--frequency-penalty N` | repeat alpha frequency penalty (default: 0.00, 0.0 = disabled) |
 | `--dry-multiplier N` | set DRY sampling multiplier (default: 0.00, 0.0 = disabled) |
 | `--dry-base N` | set DRY sampling base value (default: 1.75) |
 | `--dry-allowed-length N` | set allowed length for DRY sampling (default: 2) |
-| `--dry-penalty-last-n N` | set DRY penalty for the last n tokens (default: -1, 0 = disable, -1 = context size) |
+| `--dry-penalty-last-n N` | set DRY penalty for the last n tokens (default: 64, 0 = disable) |
 | `--dry-sequence-breaker STRING` | add sequence breaker for DRY sampling, clearing out default breakers ('\n', ':', '"', '*') in the process; use "none" to not use any sequence breakers |
 | `--adaptive-target N` | adaptive-p: select tokens near this probability (valid range 0.0 to 1.0; negative = disabled) (default: -1.00)<br/>[(more info)](https://github.com/ggml-org/llama.cpp/pull/17927) |
 | `--adaptive-decay N` | adaptive-p: decay rate for target adaptation over time. lower values are more reactive, higher values are more stable.<br/>(valid range 0.0 to 0.99) (default: 0.90) |
@@ -199,14 +199,14 @@ llama-completion.exe -m models\gemma-1.1-7b-it.Q4_K_M.gguf --ignore-eos -n -1
 | `--xtc-probability N` | xtc probability (default: 0.00, 0.0 = disabled) |
 | `--xtc-threshold N` | xtc threshold (default: 0.10, 1.0 = disabled) |
 | `--typical, --typical-p N` | locally typical sampling, parameter p (default: 1.00, 1.0 = disabled) |
-| `--repeat-last-n N` | last n tokens to consider for penalize (default: 64, 0 = disabled, -1 = ctx_size) |
+| `--repeat-last-n N` | last n tokens to consider for penalize (default: 64, 0 = disabled) |
 | `--repeat-penalty N` | penalize repeat sequence of tokens (default: 1.00, 1.0 = disabled) |
 | `--presence-penalty N` | repeat alpha presence penalty (default: 0.00, 0.0 = disabled) |
 | `--frequency-penalty N` | repeat alpha frequency penalty (default: 0.00, 0.0 = disabled) |
 | `--dry-multiplier N` | set DRY sampling multiplier (default: 0.00, 0.0 = disabled) |
 | `--dry-base N` | set DRY sampling base value (default: 1.75) |
 | `--dry-allowed-length N` | set allowed length for DRY sampling (default: 2) |
-| `--dry-penalty-last-n N` | set DRY penalty for the last n tokens (default: -1, 0 = disable, -1 = context size) |
+| `--dry-penalty-last-n N` | set DRY penalty for the last n tokens (default: 64, 0 = disable) |
 | `--dry-sequence-breaker STRING` | add sequence breaker for DRY sampling, clearing out default breakers ('\n', ':', '"', '*') in the process; use "none" to not use any sequence breakers |
 | `--adaptive-target N` | adaptive-p: select tokens near this probability (valid range 0.0 to 1.0; negative = disabled) (default: -1.00)<br/>[(more info)](https://github.com/ggml-org/llama.cpp/pull/17927) |
 | `--adaptive-decay N` | adaptive-p: decay rate for target adaptation over time. lower values are more reactive, higher values are more stable.<br/>(valid range 0.0 to 0.99) (default: 0.90) |
@@ -388,11 +388,11 @@ Example usage: `--temp 0`
 ### Repeat Penalty

 -   `--repeat-penalty N`: Control the repetition of token sequences in the generated text default: 1.0, 1.0 = disabled).
-   `--repeat-last-n N`: Last n tokens to consider for penalizing repetition (default: 64, 0 = disabled, -1 = ctx-size).
+-   `--repeat-last-n N`: Last n tokens to consider for penalizing repetition (default: 64, 0 = disabled).

 The `repeat-penalty` option helps prevent the model from generating repetitive or monotonous text. A higher value (e.g., 1.5) will penalize repetitions more strongly, while a lower value (e.g., 0.9) will be more lenient. The default value is 1.

-The `repeat-last-n` option controls the number of tokens in the history to consider for penalizing repetition. A larger value will look further back in the generated text to prevent repetitions, while a smaller value will only consider recent tokens. A value of 0 disables the penalty, and a value of -1 sets the number of tokens considered equal to the context size (`ctx-size`).
+The `repeat-last-n` option controls the number of tokens in the history to consider for penalizing repetition. A larger value will look further back in the generated text to prevent repetitions, while a smaller value will only consider recent tokens. A value of 0 disables the penalty.

 ### DRY Repetition Penalty

@@ -401,7 +401,7 @@ DRY (Don't Repeat Yourself) sampling is an effective technique for reducing repe
 - `--dry-multiplier N`: Set the DRY sampling multiplier (default: 0.0, 0.0 = disabled).
 - `--dry-base N`: Set the DRY sampling base value (default: 1.75).
 - `--dry-allowed-length N`: Set the allowed length for DRY sampling (default: 2).
- `--dry-penalty-last-n N`: Set DRY penalty for the last n tokens (default: -1, 0 = disable, -1 = context size).
+- `--dry-penalty-last-n N`: Set DRY penalty for the last n tokens (default: 64, 0 = disable).
 - `--dry-sequence-breaker STRING`: Add a sequence breaker for DRY sampling. Can be used more than once to add multiple sequence breakers. Using this clears out the default breakers, which consist of: `['\n', ':', '"', '*']`. If the string `"none"` is supplied, no sequence breakers are used.

 The `dry-multiplier` option controls the strength of the DRY sampling effect. A value of 0.0 disables DRY sampling, while higher values increase its influence. A typical recommended value is 0.8.
@@ -410,13 +410,13 @@ The `dry-base` option sets the base value for the exponential penalty calculatio

 The `dry-allowed-length` option sets the maximum length of repeated sequences that will not be penalized. Repetitions shorter than or equal to this length are not penalized, allowing for natural repetitions of short phrases or common words.

-The `dry-penalty-last-n` option controls how many recent tokens to consider when applying the DRY penalty. A value of -1 considers the entire context. Use a positive value to limit the consideration to a specific number of recent tokens.
+The `dry-penalty-last-n` option controls how many recent tokens to consider when applying the DRY penalty. A value of 0 disables the penalty. Use a positive value to limit the consideration to a specific number of recent tokens.

 The `dry-sequence-breaker` option adds a single sequence breaker and can be used more than once to specify multiple sequence breakers. Sequence breakers interrupt sequence matching and break the input into parts where matching can be applied.

 DRY sampling provides more nuanced control over text generation, particularly for reducing long-range repetitions and maintaining global coherence.

-Example usage: `--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2 --dry-penalty-last-n -1 --dry-sequence-breaker "—" --dry-sequence-breaker "##"`
+Example usage: `--dry-multiplier 0.8 --dry-base 1.75 --dry-allowed-length 2 --dry-penalty-last-n 64 --dry-sequence-breaker "—" --dry-sequence-breaker "##"`

 ### Top-K Sampling

@@ -47,6 +47,7 @@ struct split_params {
    std::string output;
    bool no_tensor_first_split = false;
    bool dry_run = false;
+    bool delete_splits = false;
 };

 static void split_print_usage(const char * executable) {
@@ -65,6 +66,7 @@ static void split_print_usage(const char * executable) {
    printf("  --split-max-size N(M|G) max size per split\n");
    printf("  --no-tensor-first-split do not add tensors to the first split (disabled by default)\n");
    printf("  --dry-run               only print out a split plan and exit, without writing any new files\n");
+    printf("  --delete-splits         delete the split files during merge to free up disk space WARNING: this option is unsafe and will leave you in an unrecoverable state if something fails during the merge\n");
    printf("\n");
 }

@@ -147,6 +149,9 @@ static void split_params_parse_ex(int argc, const char ** argv, split_params & p
            }
            params.mode = MODE_SIZE;
            params.n_bytes_split = split_str_to_n_bytes(argv[arg_idx]);
+        } else if (arg == "--delete-splits") {
+            arg_found = true;
+            params.delete_splits = true;
        }

        if (!arg_found) {
@@ -509,6 +514,7 @@ static void gguf_merge(const split_params & split_params) {
    }

    // Write tensors data
+    bool merge_error = false;
    for (int i_split = 0; i_split < n_split; i_split++) {
        llama_split_path(split_path, sizeof(split_path), split_prefix, i_split, n_split);
        std::ifstream f_input(split_path, std::ios::binary);
@@ -554,6 +560,16 @@ static void gguf_merge(const split_params & split_params) {
        ggml_free(ctx_meta);
        f_input.close();
        fprintf(stderr, "\033[3Ddone\n");
+
+        if (!split_params.dry_run && split_params.delete_splits) {
+            int delete_result = std::remove(split_path);
+            if (delete_result != 0) {
+                merge_error = true;
+                fprintf(stderr, "error: failed to delete %s\n", split_path);
+            } else {
+                fprintf(stderr, "%s: deleted file %s\n", __func__, split_path);
+            }
+        }
    }

    if (!split_params.dry_run) {
@@ -568,6 +584,10 @@ static void gguf_merge(const split_params & split_params) {

    fprintf(stderr, "%s: %s merged from %d split with %d tensors.\n",
            __func__, split_params.output.c_str(), n_split, total_tensors);
+
+    if (merge_error) {
+        exit(EXIT_FAILURE);
+    }
 }

 int main(int argc, const char ** argv) {
@@ -66,12 +66,12 @@ echo PASS
 echo

 # 5. Merge
-#$SPLIT --merge $WORK_PATH/ggml-model-split-32-tensors-00001-of-00012.gguf $WORK_PATH/ggml-model-merge-2.gguf
+#$SPLIT --merge $WORK_PATH/ggml-model-split-32-tensors-00001-of-00011.gguf $WORK_PATH/ggml-model-merge-2.gguf
 #echo PASS
 #echo

 # 5b. Test the merged model is loading properly
-#$MAIN -no-cnv --model $WORK_PATH/ggml-model-merge-2.gguf --n-predict 32
+#$MAIN -no-cnv --model $WORK_PATH/ggml-model-merge-2.gguf -p "I believe the meaning of life is" --n-predict 32
 #echo PASS
 #echo

@@ -85,5 +85,25 @@ $MAIN -no-cnv --model $WORK_PATH/ggml-model-split-500M-00001-of-00002.gguf -p "I
 echo PASS
 echo

+# 7. Merge with delete splits
+#for i in $(seq -w 1 11); do
+#    cp "$WORK_PATH/ggml-model-split-32-tensors-000${i}-of-00011.gguf" "$WORK_PATH/ggml-model-split-32-tensors-copy-000${i}-of-00011.gguf"
+#done
+#$SPLIT --merge --delete-splits $WORK_PATH/ggml-model-split-32-tensors-copy-00001-of-00011.gguf $WORK_PATH/ggml-model-merge-3.gguf
+#echo PASS
+#echo
+
+# 7b. Test the merged model is loading properly
+#$MAIN -no-cnv --model $WORK_PATH/ggml-model-merge-3.gguf -p "I believe the meaning of life is" --n-predict 32
+#echo PASS
+#echo
+
+# 7c. Test the files were deleted
+#for i in $(seq -w 1 11); do
+#    test ! -f "$WORK_PATH/ggml-model-split-32-tensors-copy-000${i}-of-00011.gguf"
+#done
+#echo PASS
+#echo
+
 # Clean up
 rm -f $WORK_PATH/ggml-model-split*.gguf $WORK_PATH/ggml-model-merge*.gguf
@@ -253,6 +253,9 @@ ggml_cgraph * clip_graph_deepseekocr::build() {

    bool is_overview = img.add_viewsep;
    int n_tiles_per_row = 0;
+    // number of separate "row" images batched together in this graph call
+    // (captured now, before n_batch below gets repurposed as the SAM/ViT batch size)
+    const int n_rows_batch = n_batch;

    // note: we expect either a batch of rows or a batch of overviews, but not a mix of both

@@ -272,16 +275,18 @@ ggml_cgraph * clip_graph_deepseekocr::build() {
        GGML_ASSERT(img.ny() % img.nx() == 0);
        n_tiles_per_row = img.ny() / img.nx();

-        // input shape: [tile_size, tile_size * n_tiles_per_row, 3]
-        // we want to reshape it to [tile_size, tile_size, 3, n_tiles_per_row]
-        inp_raw = ggml_reshape_4d(ctx0, inp_raw, img.nx(), img.nx(), n_tiles_per_row, 3);
-        inp_raw = ggml_cont(ctx0, ggml_permute(ctx0, inp_raw, 0, 1, 3, 2));
+        // each entry is one "row" image of shape [tile_size, tile_size * n_tiles_per_row, 3];
+        // merge the tile axis into the batch axis, giving a combined SAM input of shape
+        // [tile_size, tile_size, 3, n_tiles_per_row * n_rows_batch] (tile fast, row slow)
+        inp_raw = ggml_reshape_4d(ctx0, inp_raw, img.nx() * img.nx(), n_tiles_per_row, 3, n_rows_batch);
+        inp_raw = ggml_cont(ctx0, ggml_permute(ctx0, inp_raw, 0, 2, 1, 3));
+        inp_raw = ggml_reshape_4d(ctx0, inp_raw, img.nx(), img.nx(), 3, n_tiles_per_row * n_rows_batch);
    }

    ggml_tensor * sam_out = build_sam(inp_raw);

    if (!is_overview) {
-        n_batch = n_tiles_per_row;
+        n_batch = n_tiles_per_row * n_rows_batch;
    }

    const int clip_n_patches = sam_out->ne[0] * sam_out->ne[1];
@@ -354,34 +359,36 @@ ggml_cgraph * clip_graph_deepseekocr::build() {
        const auto w     = h;
        const auto n_dim = cur->ne[0];

-        ggml_tensor * imgnl = ggml_repeat_4d(ctx0, model.image_newline, n_dim, 1, h, 1);
-        cur = ggml_reshape_3d(ctx0, cur, n_dim, w, h);
-        cur = ggml_reshape_2d(ctx0, ggml_concat(ctx0, cur, imgnl, 1), n_dim, (w + 1) * h);
-        cur = ggml_concat(ctx0, cur, model.view_seperator, 1);  // (n_dim, h*(w+1) + 1)
+        ggml_tensor * imgnl = ggml_repeat_4d(ctx0, model.image_newline, n_dim, 1, h, n_batch);
+        cur = ggml_reshape_4d(ctx0, cur, n_dim, w, h, n_batch);
+        cur = ggml_reshape_3d(ctx0, ggml_concat(ctx0, cur, imgnl, 1), n_dim, (w + 1) * h, n_batch);
+        ggml_tensor * vs = ggml_repeat_4d(ctx0, model.view_seperator, n_dim, 1, n_batch, 1);
+        cur = ggml_concat(ctx0, cur, vs, 1);  // (n_dim, h*(w+1) + 1, n_batch)
    } else {
        // tile row: interleave tiles within each row, add newline per row
-        const int grid_x      = static_cast<int>(std::sqrt(static_cast<float>(clip_n_patches)));
-        const int grid_y      = grid_x;
-        const auto n_dim      = cur->ne[0];
+        const int  grid_x = static_cast<int>(std::sqrt(static_cast<float>(clip_n_patches)));
+        const int  grid_y = grid_x;
+        const auto n_dim  = cur->ne[0];

-        // (n_dim, clip_n_patches, n_batch) -> (n_dim, grid_x, grid_y, n_batch)
-        cur = ggml_reshape_4d(ctx0, cur, n_dim, grid_x, grid_y, n_batch);
+        // merge n_dim into the grid_x axis, freeing the 4th axis for n_rows_batch
+        // (n_dim, clip_n_patches, n_tiles_per_row * n_rows_batch) -> (n_dim*grid_x, grid_y, n_tiles_per_row, n_rows_batch)
+        cur = ggml_reshape_4d(ctx0, cur, n_dim * grid_x, grid_y, n_tiles_per_row, n_rows_batch);

        // tiles: re-order from A.row0 A.row1 B.row0 B.row1 ...
        //        to A.row0 B.row0 A.row1 B.row1 ...
        //        then add nl: A.row0 B.row0 [nl] A.row1 B.row1 [nl] ...
-        // interleave tiles: (n_dim, grid_x, grid_y, n_batch) -> (n_dim, grid_x, n_batch, grid_y)
-        cur = ggml_cont(ctx0, ggml_permute(ctx0, cur, 0, 1, 3, 2));
+        // interleave tiles: -> (n_dim*grid_x, n_tiles_per_row, grid_y, n_rows_batch)
+        cur = ggml_cont(ctx0, ggml_permute(ctx0, cur, 0, 2, 1, 3));

-        // merge: (n_dim, grid_x, n_batch, grid_y) -> (n_dim, grid_x*n_batch, grid_y, 1)
-        cur = ggml_reshape_4d(ctx0, cur, n_dim, grid_x * n_batch, grid_y, 1);
+        // merge: -> (n_dim, grid_x*n_tiles_per_row, grid_y, n_rows_batch)
+        cur = ggml_reshape_4d(ctx0, cur, n_dim, grid_x * n_tiles_per_row, grid_y, n_rows_batch);

-        // append newline per row: (n_dim, grid_x*n_batch+1, grid_y, 1)
-        ggml_tensor * imgnl = ggml_repeat_4d(ctx0, model.image_newline, n_dim, 1, grid_y, 1);
+        // append newline per row: (n_dim, grid_x*n_tiles_per_row+1, grid_y, n_rows_batch)
+        ggml_tensor * imgnl = ggml_repeat_4d(ctx0, model.image_newline, n_dim, 1, grid_y, n_rows_batch);
        cur = ggml_concat(ctx0, cur, imgnl, 1);

-        // flatten: (n_dim, (grid_x*n_batch+1)*grid_y)
-        cur = ggml_reshape_2d(ctx0, cur, n_dim, (grid_x * n_batch + 1) * grid_y);
+        // flatten: (n_dim, (grid_x*n_tiles_per_row+1)*grid_y, n_rows_batch)
+        cur = ggml_reshape_3d(ctx0, cur, n_dim, (grid_x * n_tiles_per_row + 1) * grid_y, n_rows_batch);
    }

    cb(cur, "dsocr_output", -1);
@@ -14,8 +14,9 @@ ggml_cgraph * clip_graph_deepseekocr2::build() {
    {
        ggml_tensor * inp;

-        inp = ggml_reshape_2d(ctx0, sam_out, sam_out->ne[0] * sam_out->ne[1], sam_out->ne[2]); // H*W, C
-        inp = ggml_cont(ctx0, ggml_permute(ctx0, inp, 1, 0, 2, 3));
+        // H*W, C, B
+        inp = ggml_reshape_3d(ctx0, sam_out, sam_out->ne[0] * sam_out->ne[1], sam_out->ne[2], sam_out->ne[3]);
+        inp = ggml_cont(ctx0, ggml_permute(ctx0, inp, 1, 0, 2, 3)); // C, H*W, B

        auto num_image_tokens = inp->ne[1]; // H*W
        GGML_ASSERT(num_image_tokens == 144 || num_image_tokens == 256);
@@ -32,8 +33,10 @@ ggml_cgraph * clip_graph_deepseekocr2::build() {
            num_queries = 144;
        }

-        // (B, num_image_tokens + num_queries, C)
-        inp = ggml_concat(ctx0, inp, ggml_cast(ctx0, query_embed, inp->type), 1);
+        // repeat the query embedding per batch item, then append: (C, num_image_tokens + num_queries, B)
+        query_embed = ggml_cast(ctx0, query_embed, inp->type);
+        query_embed = ggml_repeat_4d(ctx0, query_embed, query_embed->ne[0], num_queries, inp->ne[2], 1);
+        inp = ggml_concat(ctx0, inp, query_embed, 1);

        auto seq_len = inp->ne[1];

@@ -57,7 +60,7 @@ ggml_cgraph * clip_graph_deepseekocr2::build() {
                                      /* learned_pos_embd */ nullptr, add_rope, vit_opts);

        cur = ggml_cont(ctx0,
-                        ggml_view_2d(ctx0, cur, cur->ne[0], num_queries, cur->nb[1],
+                        ggml_view_3d(ctx0, cur, cur->ne[0], num_queries, cur->ne[2], cur->nb[1], cur->nb[2],
                                     cur->nb[1] * (cur->ne[1] - num_queries))); // only take query tokens for output

        ggml_build_forward_expand(gf, cur);
@@ -71,7 +74,8 @@ ggml_cgraph * clip_graph_deepseekocr2::build() {

    // view_seperator only after the global view
    if (img.add_viewsep) {
-        cur = ggml_concat(ctx0, cur, model.view_seperator, 1); // (n_dim, 257)
+        ggml_tensor * vs = ggml_repeat_4d(ctx0, model.view_seperator, model.view_seperator->ne[0], 1, cur->ne[2], 1);
+        cur = ggml_concat(ctx0, cur, vs, 1); // (n_dim, 257, n_batch)
    }

    cb(cur, "dsocr2_output", -1);
@@ -138,12 +138,13 @@ struct clip_graph_deepseekocr : clip_graph {
    clip_graph_deepseekocr(clip_ctx * ctx, const clip_image_f32 & img) : clip_graph(ctx, img) {}
    ggml_cgraph * build() override;
    ggml_tensor * build_sam(ggml_tensor * inp); // build the SAM model
-    // bool support_batch() const override { return true; } // TODO: support batch for DeepSeek-OCR v1
+    bool support_batch() const override { return true; }
 };

 struct clip_graph_deepseekocr2 : clip_graph_deepseekocr {
    clip_graph_deepseekocr2(clip_ctx * ctx, const clip_image_f32 & img) : clip_graph_deepseekocr(ctx, img) {}
    ggml_cgraph * build() override; // reuses build_sam() from base
+    bool support_batch() const override { return true; }
 };

 struct clip_graph_conformer : clip_graph {
@@ -556,10 +556,8 @@ bool mtmd_audio_preprocessor_whisper::preprocess(const float *                 s
    }

    std::vector<float> smpl;
-    // if input is too short, pad with zeros
-    // this is to avoid potential issues with stage1/2 padding in log_mel_spectrogram
-    // TODO: maybe handle this better
-    size_t min_samples = (size_t) hparams.audio_sample_rate * (hparams.audio_chunk_len + 1);  // +1 second margin
+    // reflection padding needs one sample plus half an FFT window
+    size_t min_samples = (size_t) hparams.audio_n_fft / 2 + 1;
    if (n_samples < min_samples) {
        smpl.resize(min_samples, 0.0f);
        std::memcpy(smpl.data(), samples, n_samples * sizeof(float));
@@ -215,7 +215,7 @@ def run_mtmd_cli(spec: "ModelSpec", model_path, mmproj_path, image_path, bin_pat
            "--dry-multiplier", "0.8",
            "--dry-base", "1.75",
            "--dry-allowed-length", "2",
-            "--dry-penalty-last-n", "-1",
+            "--dry-penalty-last-n", "64",
            "--dry-sequence-breaker", "none",
        ]
    if spec.n_ctx is not None:
@@ -133,14 +133,14 @@ For the full list of features, please refer to [server's changelog](https://gith
 | `--xtc-probability N` | xtc probability (default: 0.00, 0.0 = disabled) |
 | `--xtc-threshold N` | xtc threshold (default: 0.10, 1.0 = disabled) |
 | `--typical, --typical-p N` | locally typical sampling, parameter p (default: 1.00, 1.0 = disabled) |
-| `--repeat-last-n N` | last n tokens to consider for penalize (default: 64, 0 = disabled, -1 = ctx_size) |
+| `--repeat-last-n N` | last n tokens to consider for penalize (default: 64, 0 = disabled) |
 | `--repeat-penalty N` | penalize repeat sequence of tokens (default: 1.00, 1.0 = disabled) |
 | `--presence-penalty N` | repeat alpha presence penalty (default: 0.00, 0.0 = disabled) |
 | `--frequency-penalty N` | repeat alpha frequency penalty (default: 0.00, 0.0 = disabled) |
 | `--dry-multiplier N` | set DRY sampling multiplier (default: 0.00, 0.0 = disabled) |
 | `--dry-base N` | set DRY sampling base value (default: 1.75) |
 | `--dry-allowed-length N` | set allowed length for DRY sampling (default: 2) |
-| `--dry-penalty-last-n N` | set DRY penalty for the last n tokens (default: -1, 0 = disable, -1 = context size) |
+| `--dry-penalty-last-n N` | set DRY penalty for the last n tokens (default: 64, 0 = disable) |
 | `--dry-sequence-breaker STRING` | add sequence breaker for DRY sampling, clearing out default breakers ('\n', ':', '"', '*') in the process; use "none" to not use any sequence breakers |
 | `--adaptive-target N` | adaptive-p: select tokens near this probability (valid range 0.0 to 1.0; negative = disabled) (default: -1.00)<br/>[(more info)](https://github.com/ggml-org/llama.cpp/pull/17927) |
 | `--adaptive-decay N` | adaptive-p: decay rate for target adaptation over time. lower values are more reactive, higher values are more stable.<br/>(valid range 0.0 to 0.99) (default: 0.90) |
@@ -476,7 +476,7 @@ These words will not be included in the completion, so make sure to add them to

 `repeat_penalty`: Control the repetition of token sequences in the generated text. Default: `1.1`

-`repeat_last_n`: Last n tokens to consider for penalizing repetition. Default: `64`, where `0` is disabled and `-1` is ctx-size.
+`repeat_last_n`: Last n tokens to consider for penalizing repetition. Default: `64`, where `0` is disabled.

 `presence_penalty`: Repeat alpha presence penalty. Default: `0.0`, which is disabled.

@@ -488,7 +488,7 @@ These words will not be included in the completion, so make sure to add them to

 `dry_allowed_length`: Tokens that extend repetition beyond this receive exponentially increasing penalty: multiplier * base ^ (length of repeating sequence before token - allowed length). Default: `2`

-`dry_penalty_last_n`: How many tokens to scan for repetitions. Default: `-1`, where `0` is disabled and `-1` is context size.
+`dry_penalty_last_n`: How many tokens to scan for repetitions. Default: `64`, where `0` is disabled.

 `dry_sequence_breakers`: Specify an array of sequence breakers for DRY sampling. Only a JSON array of strings is accepted. Default: `['\n', ':', '"', '*']`

@@ -796,7 +796,7 @@ By default, it is read-only. To make POST request to change global properties, y
      "dry_multiplier": 0.0,
      "dry_base": 1.75,
      "dry_allowed_length": 2,
-      "dry_penalty_last_n": -1,
+      "dry_penalty_last_n": 64,
      "dry_sequence_breakers": [
        "\n",
        ":",
@@ -1076,6 +1076,10 @@ In *router mode* the query param `?model={model_id}` has to be set. This endpoin
 | `llamacpp:n_tokens_max` | Counter | High watermark of the context size observed. |
 | `llamacpp:n_decode_total` | Counter | Total Number of llama_decode() calls. |
 | `llamacpp:n_busy_slots_per_decode` | Gauge | Average number of busy slots per llama_decode() call. |
+| `llamacpp:spec_decode_num_draft_tokens_total` | Counter | Total draft tokens generated (0 when spec-decode is off). |
+| `llamacpp:spec_decode_num_accepted_tokens_total` | Counter | Total draft tokens accepted by the target model (0 when spec-decode is off). |
+| `llamacpp:spec_decode_num_drafts_total` | Counter | Total speculative decoding verification steps (0 when spec-decode is off). |
+| `llamacpp:spec_decode_num_accepted_tokens_per_pos_total` | Counter | Accepted tokens per draft position (labeled `position="N"`; absent when spec-decode is off or before the first completed speculative request). |

 ### POST `/slots/{id_slot}?action=save`: Save the prompt cache of the specified slot to a file.

@@ -845,6 +845,11 @@ struct server_metrics {
    uint64_t n_decode_total     = 0;
    uint64_t n_busy_slots_total = 0;

+    uint64_t n_draft_tokens_total      = 0;
+    uint64_t n_draft_accepted_total    = 0;
+    uint64_t n_draft_verif_steps_total = 0;
+    std::vector<uint64_t> n_accepted_per_pos_total;
+
    void init() {
        t_start = ggml_time_us();
    }
@@ -863,6 +868,17 @@ struct server_metrics {
        n_tokens_predicted         += slot.n_decoded;
        t_tokens_generation        += slot.t_token_generation;
        t_tokens_generation_total  += slot.t_token_generation;
+
+        n_draft_tokens_total      += slot.n_draft_total;
+        n_draft_accepted_total    += slot.n_draft_accepted;
+        n_draft_verif_steps_total += slot.n_draft_verif_steps;
+
+        if (n_accepted_per_pos_total.size() < slot.n_accepted_per_pos.size()) {
+            n_accepted_per_pos_total.resize(slot.n_accepted_per_pos.size(), 0);
+        }
+        for (size_t i = 0; i < slot.n_accepted_per_pos.size(); i++) {
+            n_accepted_per_pos_total[i] += slot.n_accepted_per_pos[i];
+        }
    }

    void on_decoded(const std::vector<server_slot> & slots) {
@@ -1807,8 +1823,7 @@ private:
        // initialize samplers
        if (task.need_sampling()) {
            try {
-                slot.smpl.reset(common_sampler_init(
-                        model_tgt, task.params.sampling, (int32_t) llama_n_ctx(ctx_tgt)));
+                slot.smpl.reset(common_sampler_init(model_tgt, task.params.sampling));
            } catch (std::exception & e) {
                std::string err_msg = std::string("Failed to initialize samplers: ") + e.what();
                send_error(task, err_msg, ERROR_TYPE_INVALID_REQUEST);
@@ -2553,6 +2568,11 @@ private:
                    res->n_decode_total          = metrics.n_decode_total;
                    res->n_busy_slots_total      = metrics.n_busy_slots_total;

+                    res->n_draft_tokens_total      = metrics.n_draft_tokens_total;
+                    res->n_draft_accepted_total    = metrics.n_draft_accepted_total;
+                    res->n_draft_verif_steps_total = metrics.n_draft_verif_steps_total;
+                    res->n_accepted_per_pos_total  = metrics.n_accepted_per_pos_total;
+
                    if (task.metrics_reset_bucket) {
                        metrics.reset_bucket();
                    }
@@ -4148,7 +4168,6 @@ std::unique_ptr<server_res_generator> server_routes::handle_completions_impl(
            task.params = server_schema::eval_llama_cmpl_schema(
                    ctx_server.vocab,
                    params,
-                    meta->slot_n_ctx,
                    meta->logit_bias_eog,
                    data);

@@ -4442,6 +4461,18 @@ void server_routes::init_routes() {
                    {"name",  "n_tokens_max"},
                    {"help",  "Largest observed n_tokens."},
                    {"value",  res_task->n_tokens_max}
+            }, {
+                    {"name",  "spec_decode_num_draft_tokens_total"},
+                    {"help",  "Total draft tokens generated"},
+                    {"value",  res_task->n_draft_tokens_total}
+            }, {
+                    {"name",  "spec_decode_num_accepted_tokens_total"},
+                    {"help",  "Total draft tokens accepted by the target model"},
+                    {"value",  res_task->n_draft_accepted_total}
+            }, {
+                    {"name",  "spec_decode_num_drafts_total"},
+                    {"help",  "Total speculative decoding verification steps"},
+                    {"value",  res_task->n_draft_verif_steps_total}
            }}},
            {"gauge", {{
                    {"name",  "prompt_tokens_seconds"},
@@ -4483,6 +4514,17 @@ void server_routes::init_routes() {
            }
        }

+        // labeled counter: one time series per draft position
+        if (!res_task->n_accepted_per_pos_total.empty()) {
+            prometheus << "# HELP llamacpp:spec_decode_num_accepted_tokens_per_pos_total"
+                          " Accepted tokens per draft position\n"
+                       << "# TYPE llamacpp:spec_decode_num_accepted_tokens_per_pos_total counter\n";
+            for (size_t i = 0; i < res_task->n_accepted_per_pos_total.size(); i++) {
+                prometheus << "llamacpp:spec_decode_num_accepted_tokens_per_pos_total{position=\""
+                           << i << "\"} " << res_task->n_accepted_per_pos_total[i] << "\n";
+            }
+        }
+
        res->headers["Process-Start-Time-Unix"] = std::to_string(res_task->t_start);
        res->content_type = "text/plain; version=0.0.4";
        res->status = 200;
@@ -124,8 +124,8 @@ std::vector<std::unique_ptr<field>> make_llama_cmpl_schema(const common_params &
        ->set_desc("Dynamic temperature exponent, controls how entropy maps to temperature"));

    add((new field_num("repeat_last_n", params.sampling.penalty_last_n))
-        ->set_hard_limits(-1, INT32_MAX)
-        ->set_desc("Last n tokens to consider for penalizing repetition (0 = disabled, -1 = ctx-size)"));
+        ->set_hard_limits(0, INT32_MAX)
+        ->set_desc("Last n tokens to consider for penalizing repetition (0 = disabled)"));

    add((new field_num("repeat_penalty", params.sampling.penalty_repeat))
        ->set_desc("Control the repetition of token sequences in the generated text (1.0 = disabled)"));
@@ -151,8 +151,8 @@ std::vector<std::unique_ptr<field>> make_llama_cmpl_schema(const common_params &
        ->set_desc("Tokens that extend repetition beyond this length receive exponentially increasing penalty: multiplier * base ^ (sequence_length - allowed_length)"));

    add((new field_num("dry_penalty_last_n", params.sampling.dry_penalty_last_n))
-        ->set_hard_limits(-1, INT32_MAX)
-        ->set_desc("How many tokens to scan for repetitions (0 = disabled, -1 = context size)"));
+        ->set_hard_limits(0, INT32_MAX)
+        ->set_desc("How many tokens to scan for repetitions (0 = disabled)"));

    add((new field_num("mirostat", params.sampling.mirostat))
        ->set_limits(0, 2)
@@ -515,7 +515,6 @@ std::vector<std::unique_ptr<field>> make_llama_cmpl_schema(const common_params &
 task_params eval_llama_cmpl_schema(
                const llama_vocab * vocab,
                const common_params & params_base,
-                const int n_ctx_slot,
                const std::vector<llama_logit_bias> & logit_bias_eog,
                const json & data) {
    task_params params;
@@ -549,15 +548,6 @@ task_params eval_llama_cmpl_schema(

    // post-processing
    {
-        if (params.sampling.penalty_last_n == -1) {
-            // note: should be the slot's context and not the full context, but it's ok
-            params.sampling.penalty_last_n = n_ctx_slot;
-        }
-
-        if (params.sampling.dry_penalty_last_n == -1) {
-            params.sampling.dry_penalty_last_n = n_ctx_slot;
-        }
-
        // if "reasoning_format" is not provided, its handler will not be called, we will need to handle it here
        auto reasoning_format = params.chat_parser_params.reasoning_format;
        params.chat_parser_params.reasoning_in_content = params.stream && (reasoning_format == COMMON_REASONING_FORMAT_DEEPSEEK_LEGACY);
@@ -98,7 +98,6 @@ std::vector<std::unique_ptr<field>> make_llama_cmpl_schema(
 task_params eval_llama_cmpl_schema(
                    const llama_vocab * vocab,
                    const common_params & params_base,
-                    const int n_ctx_slot,
                    const std::vector<llama_logit_bias> & logit_bias_eog,
                    const json & data);

@@ -1560,6 +1560,11 @@ json server_task_result_metrics::to_json() {
        { "n_decode_total",                  n_decode_total },
        { "n_busy_slots_total",              n_busy_slots_total },

+        { "n_draft_tokens_total",            n_draft_tokens_total },
+        { "n_draft_accepted_total",          n_draft_accepted_total },
+        { "n_draft_verif_steps_total",       n_draft_verif_steps_total },
+        { "n_accepted_per_pos_total",        n_accepted_per_pos_total },
+
        { "slots",                           slots_data },
    };
 }
@@ -532,6 +532,11 @@ struct server_task_result_metrics : server_task_result {
    uint64_t n_decode_total     = 0;
    uint64_t n_busy_slots_total = 0;

+    uint64_t n_draft_tokens_total      = 0;
+    uint64_t n_draft_accepted_total    = 0;
+    uint64_t n_draft_verif_steps_total = 0;
+    std::vector<uint64_t> n_accepted_per_pos_total;
+
    // while we can also use std::vector<server_slot> this requires copying the slot object which can be quite messy
    // therefore, we use json to temporarily store the slot.to_json() result
    json slots_data = json::array();
@@ -17,12 +17,60 @@
 #include <functional>
 #include <memory>

+#if defined(_WIN32)
+#   ifndef NOMINMAX
+#       define NOMINMAX
+#   endif
+#   include <windows.h>
+#endif
+
 namespace fs = std::filesystem;

 //
 // internal helpers
 //

+#if defined(_WIN32)
+// A chunk can end in the middle of a multi-byte sequence, so the incomplete
+// tail is dropped before validating what precedes it.
+static bool is_utf8_text(const std::string & text) {
+    return is_valid_utf8(text.substr(0, validate_utf8(text)));
+}
+
+// A child process writes its output in the OEM code page, which is not UTF-8
+// on a western Windows install, so accented text reaches the JSON layer as
+// invalid bytes and is replaced there. Text that already decodes as UTF-8 is
+// returned untouched, so a child that emits UTF-8 is never decoded twice.
+// run() spawns without a console, so the console code page does not apply.
+static std::string console_output_to_utf8(const std::string & text) {
+    if (text.empty() || is_utf8_text(text)) {
+        return text;
+    }
+
+    const UINT cp = GetOEMCP();
+
+    // fail rather than emit replacement characters when the code page is wrong
+    const int wide_len = MultiByteToWideChar(cp, MB_ERR_INVALID_CHARS, text.data(), (int) text.size(), nullptr, 0);
+    if (wide_len <= 0) {
+        return text;
+    }
+    std::wstring wide(wide_len, L'\0');
+    MultiByteToWideChar(cp, MB_ERR_INVALID_CHARS, text.data(), (int) text.size(), wide.data(), wide_len);
+
+    const int utf8_len = WideCharToMultiByte(CP_UTF8, 0, wide.data(), wide_len, nullptr, 0, nullptr, nullptr);
+    if (utf8_len <= 0) {
+        return text;
+    }
+    std::string utf8(utf8_len, '\0');
+    WideCharToMultiByte(CP_UTF8, 0, wide.data(), wide_len, utf8.data(), utf8_len, nullptr, nullptr);
+    return utf8;
+}
+#else
+static std::string console_output_to_utf8(const std::string & text) {
+    return text;
+}
+#endif
+
 json server_tool::to_json() const {
    return {
        {"display_name", display_name},
@@ -246,14 +294,14 @@ public:
                    size_t len = strlen(buf);
                    if (output.size() + len <= max_output) {
                        output.append(buf, len);
-                        if (on_chunk && !on_chunk(std::string(buf, len))) {
+                        if (on_chunk && !on_chunk(console_output_to_utf8(std::string(buf, len)))) {
                            proc.terminate();
                            break;
                        }
                    } else {
                        size_t remaining = max_output - output.size();
                        output.append(buf, remaining);
-                        if (on_chunk && remaining > 0) on_chunk(std::string(buf, remaining));
+                        if (on_chunk && remaining > 0) on_chunk(console_output_to_utf8(std::string(buf, remaining)));
                        truncated = true;
                    }
                }
@@ -267,7 +315,7 @@ public:

        res.exit_code = proc.join();

-        res.output    = output;
+        res.output    = console_output_to_utf8(output);
        res.timed_out = timed_out.load();
        if (truncated) {
            res.output += "\n[output truncated]";
@@ -11,7 +11,8 @@ const config: StorybookConfig = {
 		'@chromatic-com/storybook',
 		'@storybook/addon-vitest',
 		'@storybook/addon-a11y',
-		'@storybook/addon-docs'
+		'@storybook/addon-docs',
+		'@storybook/addon-mcp'
 	],
 	framework: '@storybook/sveltekit',
 	viteFinal: async (config) => {
@@ -1,12 +0,0 @@
-import * as a11yAddonAnnotations from '@storybook/addon-a11y/preview';
-import { setProjectAnnotations } from '@storybook/sveltekit';
-import * as previewAnnotations from './preview';
-import { beforeAll } from 'vitest';
-
-const project = setProjectAnnotations([a11yAddonAnnotations, previewAnnotations]);
-
-beforeAll(async () => {
-	if (project.beforeAll) {
-		await project.beforeAll();
-	}
-});
@@ -89,7 +89,7 @@ Llama UI supports two server operation modes:

 ```bash
 cd tools/ui
-npm install
+npm ci
 ```

 ### 2. Start llama-server
@@ -27,20 +27,20 @@
 		"cleanup": "rm -rf .svelte-kit build node_modules test-results dist dev-dist debug-storybook.log static/pwa-*.png static/maskable-icon-*.png static/apple-touch-icon-*.png static/apple-splash-*.png static/favicon*.ico"
 	},
 	"devDependencies": {
-		"@chromatic-com/storybook": "5.0.0",
+		"@chromatic-com/storybook": "5.2.1",
 		"@eslint/compat": "1.4.1",
 		"@eslint/js": "9.39.2",
 		"@internationalized/date": "3.12.2",
 		"@lucide/svelte": "1.25.0",
-		"@modelcontextprotocol/sdk": "1.26.0",
+		"@modelcontextprotocol/sdk": "1.30.0",
 		"@playwright/test": "1.56.1",
-		"@storybook/addon-a11y": "10.2.4",
-		"@storybook/addon-docs": "10.2.4",
-		"@storybook/addon-svelte-csf": "5.0.10",
-		"@storybook/addon-vitest": "10.2.4",
-		"@storybook/sveltekit": "10.2.4",
+		"@storybook/addon-a11y": "10.5.6",
+		"@storybook/addon-docs": "10.5.6",
+		"@storybook/addon-svelte-csf": "5.1.2",
+		"@storybook/addon-vitest": "10.5.6",
+		"@storybook/sveltekit": "10.5.6",
 		"@sveltejs/adapter-static": "3.0.10",
-		"@sveltejs/kit": "2.60.1",
+		"@sveltejs/kit": "2.70.2",
 		"@sveltejs/vite-plugin-svelte": "6.2.1",
 		"@tailwindcss/forms": "0.5.10",
 		"@tailwindcss/typography": "0.5.16",
@@ -48,16 +48,16 @@
 		"@types/node": "24.13.0",
 		"@vite-pwa/assets-generator": "1.0.2",
 		"@vite-pwa/sveltekit": "1.1.0",
-		"@vitest/browser": "4.1.8",
-		"@vitest/browser-playwright": "4.1.8",
-		"@vitest/coverage-v8": "4.1.8",
+		"@vitest/browser": "4.1.10",
+		"@vitest/browser-playwright": "4.1.10",
+		"@vitest/coverage-v8": "4.1.10",
 		"bits-ui": "2.18.1",
 		"clsx": "2.1.1",
 		"dexie": "4.4.3",
-		"dompurify": "3.4.11",
+		"dompurify": "3.4.13",
 		"eslint": "9.39.4",
 		"eslint-config-prettier": "10.1.8",
-		"eslint-plugin-storybook": "10.4.2",
+		"eslint-plugin-storybook": "10.5.6",
 		"eslint-plugin-svelte": "3.19.0",
 		"fflate": "0.8.3",
 		"globals": "16.5.0",
@@ -82,7 +82,7 @@
 		"remark-math": "6.0.0",
 		"remark-rehype": "11.1.2",
 		"sass": "1.100.0",
-		"storybook": "10.4.2",
+		"storybook": "10.5.6",
 		"svelte": "5.56.1",
 		"svelte-check": "4.6.0",
 		"svelte-sonner": "1.1.1",
@@ -95,13 +95,16 @@
 		"unified": "11.0.5",
 		"unist-util-visit": "5.1.0",
 		"uuid": "13.0.2",
-		"vite": "7.3.5",
+		"vite": "7.3.6",
 		"vite-plugin-devtools-json": "0.2.1",
-		"vitest": "4.1.8",
+		"vitest": "4.1.10",
 		"vitest-browser-svelte": "2.1.1",
-		"workbox-window": "7.4.1"
+		"workbox-window": "7.4.1",
+		"@storybook/addon-mcp": "0.7.0"
 	},
 	"overrides": {
-		"cookie": "1.1.1"
+		"cookie": "1.1.1",
+		"sharp": "0.35.3",
+		"valibot": "1.4.2"
 	}
 }
@@ -14,7 +14,7 @@ cd ../../
 # Ensure node_modules are installed
 if [ ! -d "tools/ui/node_modules" ]; then
    echo "📦 Installing npm dependencies..."
-    cd tools/ui && npm install && cd ../../
+    cd tools/ui && npm ci && cd ../../
 fi

 # Check and install git hooks if missing
@@ -14,7 +14,7 @@ cd "$REPO_ROOT/tools/ui"

 # Check that node_modules exists
 if [ ! -d "node_modules" ]; then
-    echo "❌ node_modules not found. Run 'npm install' first."
+    echo "❌ node_modules not found. Run 'npm ci' first."
    exit 1
 fi

@@ -30,7 +30,7 @@ cd "$REPO_ROOT/tools/ui"

 # Check that node_modules exists
 if [ ! -d "node_modules" ]; then
-    echo "❌ node_modules not found. Run 'npm install' first."
+    echo "❌ node_modules not found. Run 'npm ci' first."
    exit 1
 fi

@@ -198,7 +198,7 @@ const SETTINGS_REGISTRY: Record<string, SettingsSectionEntry> = {
 				key: SETTINGS_KEYS.SHOW_MESSAGE_STATS,
 				label: 'Show message generation statistics',
 				help: 'Display generation statistics (tokens/second, token count, duration) below each assistant message.',
-				defaultValue: false,
+				defaultValue: true,
 				type: SettingsFieldType.CHECKBOX,
 				section: SETTINGS_SECTION_SLUGS.DISPLAY
 			},
@@ -30,7 +30,7 @@ describe('ParameterSyncService', () => {
 				dry_multiplier: 0.0,
 				dry_base: 1.75,
 				dry_allowed_length: 2,
-				dry_penalty_last_n: -1,
+				dry_penalty_last_n: 64,
 				mirostat: 0,
 				mirostat_tau: 5.0,
 				mirostat_eta: 0.1,
@@ -96,7 +96,7 @@ describe('ParameterSyncService', () => {
 				dry_multiplier: 0.0,
 				dry_base: 1.75,
 				dry_allowed_length: 2,
-				dry_penalty_last_n: -1,
+				dry_penalty_last_n: 64,
 				mirostat: 0,
 				mirostat_tau: 5.0,
 				mirostat_eta: 0.1,
@@ -76,8 +76,7 @@ export default defineConfig({
 				extends: './vite.config.ts',
 				test: {
 					name: 'ui',
-					browser: { ...browserBaseConfig, instances: [{ browser: 'chromium', headless: true }] },
-					setupFiles: ['./.storybook/vitest.setup.ts']
+					browser: { ...browserBaseConfig, instances: [{ browser: 'chromium', headless: true }] }
 				},
 				plugins: [
 					storybookTest({
@@ -0,0 +1,19 @@
+Fix build on NetBSD, which provides posix_spawn_file_actions_addchdir()
+but not the _np() variant.
+
+Upstream PR: https://github.com/sheredom/subprocess.h/pull/102
+Applied locally by scripts/sync_vendor.py until it is merged upstream.
+
+diff --git a/subprocess.h b/subprocess.h
+index 5e809023a4..74a4e006c7 100644
+--- a/subprocess.h
+++ b/subprocess.h
+@@ -1205,7 +1205,7 @@ cleanup:
+ 
+   // Set working directory
+   if (process_cwd) {
+-#if defined(__APPLE__) && MAC_OS_X_VERSION_MIN_REQUIRED >= 260000
+#if defined(__NetBSD__) || (defined(__APPLE__) && MAC_OS_X_VERSION_MIN_REQUIRED >= 260000)
+     posix_error = posix_spawn_file_actions_addchdir(&actions, process_cwd);
+ #else
+ #if defined(__APPLE__) && defined(__clang__)
@@ -0,0 +1,47 @@
+Fix building against glibc older than 2.29, which has no
+posix_spawn_file_actions_addchdir_np (the symbol is genuinely absent from
+libc.so, so no feature-test macro helps). Affects manylinux2014 (glibc 2.17)
+and manylinux_2_28, and was reported on RHEL 8.1. A requested process_cwd now
+fails with ENOSYS there instead of failing the build.
+
+Upstream PR: https://github.com/sheredom/subprocess.h/pull/104
+Applied locally by scripts/sync_vendor.py until it is merged upstream.
+
+(the README.md and test/ changes from the PR are omitted, we only vendor
+subprocess.h; rebased on top of patch-bsd.patch, so apply it after that one)
+
+diff --git a/subprocess.h b/subprocess.h
+index 1ef424a..c363393 100644
+--- a/subprocess.h
+++ b/subprocess.h
+@@ -274,6 +274,21 @@ subprocess_weak int subprocess_alive(struct subprocess_s *const process);
+ #include <unistd.h>
+ #endif
+ 
+/* Whether subprocess_create_ex can honour process_cwd. glibc only gained
+   posix_spawn_file_actions_addchdir_np in 2.29. Define this yourself to
+   override the detection, for instance on musl older than 1.1.24. */
+#if !defined(SUBPROCESS_HAVE_CWD)
+#if defined(__GLIBC__)
+#if __GLIBC_PREREQ(2, 29)
+#define SUBPROCESS_HAVE_CWD 1
+#else
+#define SUBPROCESS_HAVE_CWD 0
+#endif
+#else
+#define SUBPROCESS_HAVE_CWD 1
+#endif
+#endif
+
+ #if defined(_WIN32)
+ 
+ #include <wchar.h>
+@@ -1219,6 +1234,8 @@ cleanup:
+   if (process_cwd) {
+ #if defined(__NetBSD__) || (defined(__APPLE__) && MAC_OS_X_VERSION_MIN_REQUIRED >= 260000)
+     posix_error = posix_spawn_file_actions_addchdir(&actions, process_cwd);
+#elif !SUBPROCESS_HAVE_CWD
+    posix_error = ENOSYS;
+ #else
+ #if defined(__APPLE__) && defined(__clang__)
+ #pragma clang diagnostic push
@@ -0,0 +1,107 @@
+Fix Windows command line quoting of backslash runs: a trailing backslash, or
+backslashes preceding a double quote, were not doubled, so CommandLineToArgvW
+in the child parsed them as escapes and mangled the argument list.
+
+Upstream PR: https://github.com/sheredom/subprocess.h/pull/101
+Applied locally by scripts/sync_vendor.py until it is merged upstream.
+
+(the test/ changes from the PR are omitted, we only vendor subprocess.h)
+
+diff --git a/subprocess.h b/subprocess.h
+index 5e80902..b06ad4d 100644
+--- a/subprocess.h
+++ b/subprocess.h
+@@ -653,6 +653,7 @@ int subprocess_create_ex(const char *const commandLine[], int options,
+   int wide_len;
+   int i, j;
+   int need_quoting;
+  subprocess_size_t bs_run;
+   unsigned long flags = 0;
+   unsigned long last_error = 0;
+   int result = subprocess_error_unknown;
+@@ -906,25 +907,29 @@ int subprocess_create_ex(const char *const commandLine[], int options,
+     len++;
+ 
+     // Quote the argument if it has a space in it
+-    if (strpbrk(commandLine[i], "\t\v ") != SUBPROCESS_NULL ||
+-        commandLine[i][0] == SUBPROCESS_NULL)
+    need_quoting = strpbrk(commandLine[i], "\t\v ") != SUBPROCESS_NULL ||
+                   commandLine[i][0] == SUBPROCESS_NULL;
+    if (need_quoting)
+       len += 2;
+ 
+    bs_run = 0;
+     for (j = 0; '\0' != commandLine[i][j]; j++) {
+-      switch (commandLine[i][j]) {
+-      default:
+-        break;
+-      case '\\':
+-        if (commandLine[i][j + 1] == '"') {
+-          len++;
+-        }
+      len++;
+ 
+-        break;
+-      case '"':
+-        len++;
+-        break;
+      if ('\\' == commandLine[i][j]) {
+        bs_run++;
+      } else {
+        if ('"' == commandLine[i][j]) {
+          // Duplicate the preceding run and escape the quote.
+          len += bs_run + 1;
+        }
+        bs_run = 0;
+       }
+-      len++;
+    }
+
+    if (need_quoting) {
+      // Duplicate trailing slashes before the generated closing quote.
+      len += bs_run;
+     }
+   }
+ 
+@@ -949,22 +954,29 @@ int subprocess_create_ex(const char *const commandLine[], int options,
+       commandLineCombined[len++] = '"';
+     }
+ 
+-    for (j = 0; '\0' != commandLine[i][j]; j++) {
+-      switch (commandLine[i][j]) {
+-      default:
+-        break;
+-      case '\\':
+-        if (commandLine[i][j + 1] == '"') {
+-          commandLineCombined[len++] = '\\';
+-        }
+    for (j = 0; '\0' != commandLine[i][j];) {
+      bs_run = 0;
+      while ('\\' == commandLine[i][j]) {
+        bs_run++;
+        j++;
+      }
+
+      if ('"' == commandLine[i][j]) {
+        // 2n + 1 slashes preserve n slashes and escape the quote.
+        bs_run = (bs_run * 2) + 1;
+      } else if ('\0' == commandLine[i][j] && need_quoting) {
+        // 2n slashes preserve n slashes before the closing quote.
+        bs_run *= 2;
+      }
+ 
+-        break;
+-      case '"':
+      while (bs_run > 0) {
+         commandLineCombined[len++] = '\\';
+-        break;
+        bs_run--;
+       }
+ 
+-      commandLineCombined[len++] = commandLine[i][j];
+      if ('\0' != commandLine[i][j]) {
+        commandLineCombined[len++] = commandLine[i][j++];
+      }
+     }
+     if (need_quoting) {
+       commandLineCombined[len++] = '"';
@@ -274,6 +274,21 @@ subprocess_weak int subprocess_alive(struct subprocess_s *const process);
 #include <unistd.h>
 #endif

+/* Whether subprocess_create_ex can honour process_cwd. glibc only gained
+   posix_spawn_file_actions_addchdir_np in 2.29. Define this yourself to
+   override the detection, for instance on musl older than 1.1.24. */
+#if !defined(SUBPROCESS_HAVE_CWD)
+#if defined(__GLIBC__)
+#if __GLIBC_PREREQ(2, 29)
+#define SUBPROCESS_HAVE_CWD 1
+#else
+#define SUBPROCESS_HAVE_CWD 0
+#endif
+#else
+#define SUBPROCESS_HAVE_CWD 1
+#endif
+#endif
+
 #if defined(_WIN32)

 #include <wchar.h>
@@ -653,6 +668,7 @@ int subprocess_create_ex(const char *const commandLine[], int options,
  int wide_len;
  int i, j;
  int need_quoting;
+  subprocess_size_t bs_run;
  unsigned long flags = 0;
  unsigned long last_error = 0;
  int result = subprocess_error_unknown;
@@ -906,25 +922,29 @@ int subprocess_create_ex(const char *const commandLine[], int options,
    len++;

    // Quote the argument if it has a space in it
-    if (strpbrk(commandLine[i], "\t\v ") != SUBPROCESS_NULL ||
-        commandLine[i][0] == SUBPROCESS_NULL)
+    need_quoting = strpbrk(commandLine[i], "\t\v ") != SUBPROCESS_NULL ||
+                   commandLine[i][0] == SUBPROCESS_NULL;
+    if (need_quoting)
      len += 2;

+    bs_run = 0;
    for (j = 0; '\0' != commandLine[i][j]; j++) {
-      switch (commandLine[i][j]) {
-      default:
-        break;
-      case '\\':
-        if (commandLine[i][j + 1] == '"') {
-          len++;
-        }
-
-        break;
-      case '"':
-        len++;
-        break;
-      }
      len++;
+
+      if ('\\' == commandLine[i][j]) {
+        bs_run++;
+      } else {
+        if ('"' == commandLine[i][j]) {
+          // Duplicate the preceding run and escape the quote.
+          len += bs_run + 1;
+        }
+        bs_run = 0;
+      }
+    }
+
+    if (need_quoting) {
+      // Duplicate trailing slashes before the generated closing quote.
+      len += bs_run;
    }
  }

@@ -949,22 +969,29 @@ int subprocess_create_ex(const char *const commandLine[], int options,
      commandLineCombined[len++] = '"';
    }

-    for (j = 0; '\0' != commandLine[i][j]; j++) {
-      switch (commandLine[i][j]) {
-      default:
-        break;
-      case '\\':
-        if (commandLine[i][j + 1] == '"') {
-          commandLineCombined[len++] = '\\';
-        }
-
-        break;
-      case '"':
-        commandLineCombined[len++] = '\\';
-        break;
+    for (j = 0; '\0' != commandLine[i][j];) {
+      bs_run = 0;
+      while ('\\' == commandLine[i][j]) {
+        bs_run++;
+        j++;
      }

-      commandLineCombined[len++] = commandLine[i][j];
+      if ('"' == commandLine[i][j]) {
+        // 2n + 1 slashes preserve n slashes and escape the quote.
+        bs_run = (bs_run * 2) + 1;
+      } else if ('\0' == commandLine[i][j] && need_quoting) {
+        // 2n slashes preserve n slashes before the closing quote.
+        bs_run *= 2;
+      }
+
+      while (bs_run > 0) {
+        commandLineCombined[len++] = '\\';
+        bs_run--;
+      }
+
+      if ('\0' != commandLine[i][j]) {
+        commandLineCombined[len++] = commandLine[i][j++];
+      }
    }
    if (need_quoting) {
      commandLineCombined[len++] = '"';
@@ -1205,8 +1232,10 @@ cleanup:

  // Set working directory
  if (process_cwd) {
-#if defined(__APPLE__) && MAC_OS_X_VERSION_MIN_REQUIRED >= 260000
+#if defined(__NetBSD__) || (defined(__APPLE__) && MAC_OS_X_VERSION_MIN_REQUIRED >= 260000)
    posix_error = posix_spawn_file_actions_addchdir(&actions, process_cwd);
+#elif !SUBPROCESS_HAVE_CWD
+    posix_error = ENOSYS;
 #else
 #if defined(__APPLE__) && defined(__clang__)
 #pragma clang diagnostic push
Author	SHA1	Message	Date
Xuan-Son Nguyen GitHub Saba Fallah	717dad5c8e	mtmd: support multi-row batching for deepseek-ocr (#26154 ) * mtmd: support multi-row batching for deepseek-ocr * mtmd: weave deepseek-ocr rows in one shot instead of per row (#26615) --------- Co-authored-by: Saba Fallah <sabafallah@gmail.com>	2026-08-05 13:34:52 +02:00
Sergey MalininandGitHub	9a688e51e6	fit: Fix memory allocation for MTP layers (#26605 )	2026-08-05 13:29:45 +02:00
Xuan-Son NguyenandGitHub	9303cdd8d3	security : clarify about AI-generated reports (#26579 ) * security : clarify about AI-generated reports * nits * nits 2	2026-08-05 13:27:06 +02:00
Bhavik ShardaandGitHub	a035a88878	server: Adding spec-decode counters to /metrics endpoint (#26389 ) * * server: add spec-decode counters to /metrics endpoint * server: fixed review comments and now aligned param names exactly with vLLM.	2026-08-05 12:36:01 +02:00
Andreas Krebbel GitHub Sigbjørn Skjæret	020760adfc	convert: Add endianness conversion for Q1 and TQ2 quantizations (#26618 ) * Add endianness conversion for Q1 and TQ2 quantizations * lint --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>	2026-08-05 18:06:09 +08:00
Xuan-Son NguyenandGitHub	61881b1f7f	vendor : apply patches for subprocess.h (#26606 )	2026-08-05 11:26:20 +02:00
Aleksander GrygierandGitHub	3e3a7a416d	ui: show generation statistics by default in chat settings (#26624 )	2026-08-05 11:03:23 +02:00
Niklas WenzelandGitHub	d52ec04a66	build : remove GGML_METAL_USE_BF16 from all build scripts (#26604 )	2026-08-05 10:44:34 +02:00
Aleksander GrygierandGitHub	e031d95679	ui: Update vulnerable packages + cleanup Storybook config (#26607 ) * chore: Upgrade Storybook * chore: Bump package-lock * chore: bump vitest to 4.1.10 * ui: bump fast-uri to 3.1.5 * ui: bump ip-address to 10.4.0 * ui: bump js-yaml to 4.3.1 * ui: bump immutable to 5.1.9 * ui: bump postcss to 8.5.25 * ui: bump brace-expansion to safe versions * ui: bump sharp to 0.35.3 via override * ui: bump body-parser to 2.3.0 * ui: bump vite to 7.3.6 and esbuild to 0.28.1 Assisted-by: Claude Sonnet * ui: bump hono to 4.13.0 * ui: bump dompurify to 3.4.13 * ui: bump @sveltejs/kit to 2.70.2 * ui: bump @modelcontextprotocol/sdk to 1.30.0 * ui: bump valibot to 1.4.2 via override * chore: Remove legacy setup file * refactor: Nits cleanup	2026-08-05 08:06:37 +02:00
Evan HuusandGitHub	6ea215d171	Prefer npm ci over install for security (#26601 )	2026-08-05 00:14:22 +02:00
PascalandGitHub	4308a4f035	server: decode Windows OEM output to UTF-8 in built-in tools (#26597 ) a child process writes in the OEM code page, which is not UTF-8 on a western Windows install, so accented output reaches the JSON layer as invalid bytes and gets replaced there, silently losing the characters run() spawns without a console, so the child never inherits the console code page and GetOEMCP is the one that applies decode with MB_ERR_INVALID_CHARS so a wrong code page returns the text untouched instead of emitting replacement characters, and pass text that already decodes as UTF-8 through so a child emitting UTF-8 is never decoded twice the check drops an incomplete trailing sequence before validating, since a streamed chunk can end in the middle of a multi-byte character	2026-08-04 22:24:55 +02:00
Abhinay KrishnaandGitHub	474c92e722	mtmd: correcting duplicate empty audio chunks for short inputs (#26536 ) * correcting duplicate empty audio chunks for short inputs * tests.sh code restored	2026-08-04 22:05:56 +02:00
Oliver SimonsandGitHub	a6aa6f5450	sampler : remove "full-context windows" from history-based samplers (#26524 ) * Resolve -1 to 1024 instead of ctx-len for samplers Because of backend-sampling we initialize samplers before the complete llama_context is there. Therefore, we cannot infer the resolved context length yet at the time we construct the samplers. * Shared default of 64 for history-based samplers, remove context_size	2026-08-04 21:28:55 +03:00
Guilherme Quintino GitHub Sigbjørn Skjæret	76c956c137	gguf-split: Add option to delete split parts during merge (#26538 ) * Add delete-files option to split parameters Added a new option to delete split files during execution to free up disk space. * Add test for delete files on merge option * Fix tests * Update tools/gguf-split/gguf-split.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * Update tools/gguf-split/gguf-split.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * Uncomment tests * Improvements to address PR comments * Fix formatting * Fix formatting * Rename --delete-files to --delete-splits * Comment tests * Move delete inside loop * style cleanup --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>	2026-08-04 21:27:47 +03:00