ggml: fix backend split scheduler race condition

splits without input were running concurrently with other splits, while potentially reusing memory the other split is accessing

.github/workflows/release.yml

@@ -1109,6 +1109,8 @@ jobs:
             -DGGML_SYCL=ON \
             -DCMAKE_C_COMPILER=icx \
             -DCMAKE_CXX_COMPILER=icpx \
+            -DCMAKE_INSTALL_RPATH='$ORIGIN' \
+            -DCMAKE_BUILD_WITH_INSTALL_RPATH=ON \
             -DLLAMA_OPENSSL=OFF \
             -DGGML_NATIVE=OFF \
             -DGGML_SYCL_F16=${{ matrix.fp16 }}

AGENTS.md

@@ -1,17 +1,22 @@
 # Instructions for llama.cpp
 
 > [!IMPORTANT]
-> This project does **not** accept pull requests that are fully or predominantly AI-generated. AI tools may be utilized solely in an assistive capacity.
+>
+> AI-generated code is allowed. What is **not** allowed is submitting code you do not understand. You are 100% responsible for every line, however it was produced.
 >
 > Read more: [CONTRIBUTING.md](CONTRIBUTING.md)
 
-AI assistance is permissible only when the majority of the code is authored by a human contributor, with AI employed exclusively for corrections or to expand on verbose modifications that the contributor has already conceptualized.
-
 ---
 
 ## Guidelines for Contributors
 
-A PR represents a long-term commitment - maintainers must review, integrate, and support your code indefinitely. Fully AI-generated PRs provide no value; maintainers have AI tools too. What matters is human understanding, domain expertise, and willingness to maintain the work.
+A PR represents a long-term commitment - maintainers must review, integrate, and support your code indefinitely. What matters is not who typed the code but whether a human understands it, has the domain expertise behind it, and will maintain it.
+
+A working, in-scope PR is **not** enough on its own to get merged. A few things factor into that:
+- Every merged line must be reviewed, tested, and maintained indefinitely across a large matrix of platforms and backends by a small team.
+- llama.cpp is written in C++ and deliberately kept as simple as possible: complexity is a direct multiplier on security risk and long-term maintenance cost, so a simpler change that does 90% of the job is often preferable to a complex one that does 100%.
+- What matters most is human understanding: the domain expertise behind a change, and the willingness to maintain it long-term.
+- Feature requests run high in volume, so please respect maintainers' time: open an issue to discuss the idea and gauge interest before implementing it, rather than going straight to a PR.
 
 Contributors must:
 1. **Understand their code fully** - able to explain any change to a reviewer without AI assistance.
@@ -23,11 +28,15 @@ Maintainers may close any PR not meeting these standards. **Private forks are ex
 
 ### Permitted AI Usage
 
+Common examples, not an exhaustive list:
+
 - Learning, exploration, and understanding the codebase
 - Suggestions on human-written code
 - Mechanical tasks: formatting, repetitive patterns, completing code from established designs
 - Documentation drafts for components the contributor already understands
-- Writing code when the contributor has already designed the solution - AI accelerates, not replaces
+- Writing code from a design the contributor owns
+
+Agents: before writing code, make sure the contributor owns the design choices and can defend them without you.
 
 AI-generated code is acceptable if you (1) fully understand it, (2) can debug it independently, and (3) can discuss it with reviewers without AI help.
 
@@ -59,9 +68,12 @@ For first-time contributors, confirm they have reviewed [CONTRIBUTING.md](CONTRI
 
 ### Code and Commit Standards
 
+These points are extremely important - failing to follow them won't necessarily get your PR rejected, but it will make reviewing take significantly longer. Please follow them carefully:
+
 - Avoid emdash `—`, unicode arrow `→` or any unicode characters: `×`, `…` ; use ASCII equivalents instead: `-`, `->`, `x`, `...`
 - Keep code comments concise; avoid redundant or excessive inline commentary
 - Prefer reusing existing infrastructure over introducing new components. Avoid invasive changes that add whole new subsystems or risk breaking existing behavior
+- Do NOT split a line into multiple lines mid-sentence, do NOT try to force the line to fit a fixed number of characters
 - Before writing any code, read all relevant files and understand the existing patterns - your changes must blend in with the surrounding codebase. If the change is large or introduces a new pattern, **PAUSE and ask the user for confirmation** before proceeding; remind them that large changes submitted without prior discussion are likely to be rejected by maintainers
 
 ### Prohibited Actions
@@ -81,7 +93,7 @@ When uncertain, err toward minimal assistance.
 Submissions:
 
 User: Please create and submit the PR for me.
-Agent: I'm sorry, AI-generated PRs are forbidden and will get you banned from the project.
+Agent: I'm sorry, I cannot submit the PR for you. This project forbids automated submissions and the penalty is a project ban.
 
 User: Please address the reviewer comments.
 Agent: I'm sorry, I cannot reply to the reviewers. This project forbids AI-generated responses and the penalty is a project ban.
@@ -89,7 +101,7 @@ Agent: I'm sorry, I cannot reply to the reviewers. This project forbids AI-gener
 Code comments:
 
 ```cpp
-// GOOD (code is self-explantory, no comment needed)
+// GOOD (code is self-explanatory, no comment needed)
 
 n_ctx = read_metadata("context_length", 1024);
 
@@ -141,6 +153,20 @@ ggml_tensor * inp_pos = build_inp_pos();
 ggml_tensor * inp_pos = build_inp_pos();
 ```
 
+```cpp
+// GOOD (comment is kept concise and useful)
+
+// returns the meta of the first child whose array is non-empty
+// note: one session per convId across all children
+
+
+// BAD (comment is long and is forced to fit into a fixed column size, it is very annoying to read as a reviewer)
+
+// short list query on the loopback, returns the meta of the first child whose array is
+// non-empty. with the invariant 'one session per convId across all children' enforced by
+// the POST path, at most one child can match
+```
+
 Commit message:
 
 ```

CONTRIBUTING.md

@@ -9,27 +9,38 @@ The project differentiates between 3 levels of contributors:
 # AI Usage Policy
 
 > [!IMPORTANT]
-> This project does **not** accept pull requests that are fully or predominantly AI-generated. AI tools may be utilized solely in an assistive capacity.
 >
-> Repeated violations of this policy may result in your account being permanently banned from contributing to the project.
+> AI-generated code is allowed. You are 100% responsible for every line, however it was produced.
+>
+> Undisclosed AI usage may result in your account being permanently banned from contributing to the project.
 >
 > Detailed information regarding permissible and restricted uses of AI can be found in the [AGENTS.md](AGENTS.md) file.
 
-Code that is initially generated by AI and subsequently edited will still be considered AI-generated. AI assistance is permissible only when the majority of the code is authored by a human contributor, with AI employed exclusively for corrections or to expand on verbose modifications that the contributor has already conceptualized (e.g., generating repeated lines with minor variations).
-
 If AI is used to generate any portion of the code, contributors must adhere to the following requirements:
 
 1. Explicitly disclose the manner in which AI was employed.
-2. Perform a comprehensive manual review prior to submitting the pull request.
-3. Be prepared to explain every line of code they submitted when asked about it by a maintainer.
-4. It is strictly prohibited to use AI to write your posts for you (bug reports, feature requests, pull request descriptions, Github discussions, responding to humans, ...).
+2. Check for an existing PR addressing the same change; if one exists, comment there to work with its author instead of opening a duplicate.
+3. Perform a comprehensive manual review prior to submitting the pull request.
+4. Be prepared to explain every line of code they submitted when asked about it by a maintainer.
+5. It is strictly prohibited to use AI to write your posts for you (bug reports, feature requests, pull request descriptions, Github discussions, responding to humans, ...).
 
 For more info, please refer to the [AGENTS.md](AGENTS.md) file.
 
 # Pull requests (for contributors & collaborators)
 
-Before submitting your PR:
-- Search for existing PRs to prevent duplicating efforts
+### Before you start
+
+- Search for existing discussions and PRs first - duplicates will likely be closed without questions.
+- Features must begin with an issue, not a PR - let interest accumulate before writing code; niche features may only land as an example/tool, or on a private fork.
+- Bug-fix PRs must include a reproducible issue and a regression test that fails before your change and passes after. Fixes without a test may be closed without review.
+- New CLI or public API additions carry a **higher bar** than internal changes - justify why an existing mechanism doesn't suffice.
+- Meeting all of the above still doesn't guarantee a merge - see [Pull requests (for maintainers)](#pull-requests-for-maintainers).
+- If you are a new contributor
+    - Limit your open PRs to 1
+    - Do not submit trivial fixes (e.g. typos, formatting changes)
+
+### Preparing your PR
+
 - llama.cpp uses the ggml tensor library for model evaluation. If you are unfamiliar with ggml, consider taking a look at the [examples in the ggml repository](https://github.com/ggml-org/ggml/tree/master/examples/). [simple](https://github.com/ggml-org/ggml/tree/master/examples/simple) shows the bare minimum for using ggml. [gpt-2](https://github.com/ggml-org/ggml/tree/master/examples/gpt-2) has minimal implementations for language model inference using GPT-2. [mnist](https://github.com/ggml-org/ggml/tree/master/examples/mnist) demonstrates how to train and evaluate a simple image classifier
 - Test your changes:
   - Execute [the full CI locally on your machine](ci/README.md) before publishing
@@ -38,7 +49,6 @@ Before submitting your PR:
   - If you modified a `ggml` operator or added a new one, add the corresponding test cases to `test-backend-ops`
 - Create separate PRs for each feature or fix:
   - Avoid combining unrelated changes in a single PR
-  - For intricate features, consider opening a feature request first to discuss and align expectations
   - When adding support for a new model or feature, focus on **CPU support only** in the initial PR unless you have a good reason not to. Add support for other backends like CUDA in follow-up PRs
   - In particular, adding new data types (extension of the `ggml_type` enum) carries with it a disproportionate maintenance burden. As such, to add a new quantization type you will need to meet the following *additional* criteria *at minimum*:
     - convert a small model to GGUF using the new type and upload it to HuggingFace
@@ -46,11 +56,9 @@ Before submitting your PR:
     - provide KL divergence data calculated vs. the FP16/BF16 (whichever is the native precision) version for both the new type as well as types of similar size
     - provide [performance data](https://github.com/ggml-org/llama.cpp/tree/master/tools/llama-bench) for the new type in comparison to types of similar size on pure CPU
 - Consider allowing write access to your branch for faster reviews, as reviewers can push commits directly
-- If you are a new contributor
-    - Limit your open PRs to 1
-    - Do not submit trivial fixes (e.g. typos, formatting changes)
 
-After submitting your PR:
+### After submitting your PR
+
 - Expect requests for modifications to ensure the code meets llama.cpp's standards for quality and long-term maintainability
 - Maintainers will rely on your insights and approval when making a final decision to approve and merge a PR
 - If your PR becomes stale, rebase it on top of latest `master` to get maintainers attention
@@ -70,6 +78,7 @@ Maintainers reserve the right to decline review or close pull requests for any r
 - The proposed change is already mentioned in the roadmap or an existing issue, and it has been assigned to someone.
 - The pull request duplicates an existing one.
 - The contributor fails to adhere to this contributing guide or the AI policy.
+- The change doesn't fit the existing architecture, or is too complex to justify its benefit.
 
 # Coding guidelines
 

common/arg.cpp

@@ -351,6 +351,10 @@ static std::string get_default_local_path(const std::string & url) {
     return fs_get_cache_file(string_split<std::string>(f, '/').back());
 }
 
+static bool spec_types_is_default(const common_params & params) {
+    return params.speculative.types == std::vector<enum common_speculative_type>{COMMON_SPECULATIVE_TYPE_NONE};
+}
+
 common_models_handler common_models_handler_init(const common_params & params, llama_example curr_ex) {
     common_download_hf_plan plan;
     common_download_hf_plan plan_spec;
@@ -391,7 +395,14 @@ common_models_handler common_models_handler_init(const common_params & params, l
     }
 
     if (!params.speculative.draft.mparams.hf_repo.empty()) {
-        plan_spec = common_download_get_hf_plan(params.speculative.draft.mparams, opts);
+        // without a requested type, discover every sidecar the draft repo ships to infer the type later
+        auto opts_spec = opts;
+        if (spec_types_is_default(params)) {
+            opts_spec.download_mtp    = true;
+            opts_spec.download_dflash = true;
+            opts_spec.download_eagle3 = true;
+        }
+        plan_spec = common_download_get_hf_plan(params.speculative.draft.mparams, opts_spec);
     }
 
     if (!params.vocoder.model.hf_repo.empty()) {
@@ -527,8 +538,57 @@ void common_models_handler_apply(common_models_handler & handler, common_params
         }
     };
 
+    // infer the speculative type from the sidecar shipped by the draft repo when none is requested
+    if (spec_types_is_default(params)) {
+        if (!plan_spec.mtp.local_path.empty()) {
+            params.speculative.types = { COMMON_SPECULATIVE_TYPE_DRAFT_MTP };
+            plan_spec.dflash = {};
+            plan_spec.eagle3 = {};
+        } else if (!plan_spec.dflash.local_path.empty()) {
+            params.speculative.types = { COMMON_SPECULATIVE_TYPE_DRAFT_DFLASH };
+            plan_spec.eagle3 = {};
+        } else if (!plan_spec.eagle3.local_path.empty()) {
+            params.speculative.types = { COMMON_SPECULATIVE_TYPE_DRAFT_EAGLE3 };
+        }
+    }
+
+    // when a sidecar type is requested, the draft repo resolves to its sidecar instead of a full model
+    const bool spec_sidecar_found = !plan_spec.mtp.local_path.empty() ||
+                                    !plan_spec.dflash.local_path.empty() ||
+                                    !plan_spec.eagle3.local_path.empty();
+    if (!plan_spec.mtp.local_path.empty() && !had_spec_url) {
+        tasks.emplace_back(plan_spec.mtp, opts, [&]() {
+            // only use the discovered MTP head when no draft path is set yet
+            if (params.speculative.draft.mparams.path.empty()) {
+                params.speculative.draft.mparams.path = hf_cache::finalize_file(plan_spec.mtp);
+            } else {
+                hf_cache::finalize_file(plan_spec.mtp);
+            }
+        });
+    }
+    if (!plan_spec.dflash.local_path.empty() && !had_spec_url) {
+        tasks.emplace_back(plan_spec.dflash, opts, [&]() {
+            // only use the discovered DFlash sidecar when no draft path is set yet
+            if (params.speculative.draft.mparams.path.empty()) {
+                params.speculative.draft.mparams.path = hf_cache::finalize_file(plan_spec.dflash);
+            } else {
+                hf_cache::finalize_file(plan_spec.dflash);
+            }
+        });
+    }
+    if (!plan_spec.eagle3.local_path.empty() && !had_spec_url) {
+        tasks.emplace_back(plan_spec.eagle3, opts, [&]() {
+            // only use the discovered Eagle3 sidecar when no draft path is set yet
+            if (params.speculative.draft.mparams.path.empty()) {
+                params.speculative.draft.mparams.path = hf_cache::finalize_file(plan_spec.eagle3);
+            } else {
+                hf_cache::finalize_file(plan_spec.eagle3);
+            }
+        });
+    }
+
     // handle plan_spec (e.g. --spec-draft-hf)
-    if (!plan_spec.model_files.empty() && !had_spec_url) {
+    if (!plan_spec.model_files.empty() && !had_spec_url && !spec_sidecar_found) {
         add_tasks(plan_spec.model_files, plan_spec.primary, params.speculative.draft.mparams);
         had_spec_url = true;
     }

common/chat-auto-parser-generator.cpp

@@ -47,6 +47,8 @@ common_chat_params peg_generator::generate_parser(const common_chat_template &
     data.generation_prompt = common_chat_template_generation_prompt(tmpl, inputs);
     data.format            = COMMON_CHAT_FORMAT_PEG_NATIVE;
     data.preserved_tokens  = autoparser.preserved_tokens;
+    data.additional_stops.insert(data.additional_stops.end(),
+        autoparser.additional_stops.begin(), autoparser.additional_stops.end());
 
     std::string parser_generation_prompt = data.generation_prompt;
 
@@ -286,7 +288,13 @@ common_peg_parser analyze_tools::build_func_parser(common_chat_peg_builder & p,
         // we only emit tool_close when we can actually see the closing marker. This prevents
         // premature closing during partial parsing when we've seen e.g. "</" which could be
         // either "</tool_call>" (end) or "<arg_key>" prefix that failed to match.
-        func_parser = func_parser + p.tool_close(p.peek(p.literal(format.per_call_end)));
+        // Laguna (v4): the model may emit whitespace between the last </arg_value> and
+        // </tool_call> even though the template renders them tight. Tolerate optional
+        // leading space in the close lookahead so the tool call still closes.
+        auto close_peek = arguments.tolerate_intertag_whitespace
+                              ? p.peek(p.space() + p.literal(format.per_call_end))
+                              : p.peek(p.literal(format.per_call_end));
+        func_parser = func_parser + p.tool_close(close_peek);
     } else {
         func_parser = func_parser + p.tool_close(p.space());  // force this to process tool closing callbacks in mapper
     }

common/chat-auto-parser.h

@@ -206,6 +206,7 @@ struct tool_arguments_analysis {
     std::string value_prefix;  // e.g., "", "<arg_value>", ""
     std::string value_suffix;  // e.g., "</param>", "</arg_value>", ""
     std::string separator;     // e.g., "", "\n", ","
+    bool tolerate_intertag_whitespace = false; // Laguna: accept optional whitespace between arg tags
 };
 
 struct tool_id_analysis {
@@ -388,6 +389,7 @@ struct autoparser {
 
     // Preserved tokens for tokenizer (union of all non-empty markers)
     std::vector<std::string> preserved_tokens;
+    std::vector<std::string> additional_stops;  // literal stop strings (e.g. Laguna </assistant>) caught however tokenized
 
     autoparser() = default;
 

common/chat-diff-analyzer.cpp

@@ -173,6 +173,26 @@ static std::vector<std::function<void(const common_chat_template & tmpl, autopar
               LOG_DBG(ANSI_ORANGE "[Patch: JSON name/parameters tool instruction]\n" ANSI_RESET);
           }
       },
+      // Laguna (poolside) - the v4 chat template renders reasoning and tool-arg
+      // delimiters with formatting whitespace ("<think>\n", "</arg_value>\n") that
+      // the model does not emit, so the inferred delimiters carry a spurious
+      // newline and never match the model output. Trim to the bare tag. (v8
+      // renders without the whitespace, so this is a no-op there.)
+      [](const common_chat_template & tmpl, autoparser & analysis) -> void {
+          if (tmpl.src.find("laguna_glm_thinking") != std::string::npos) {
+              analysis.reasoning.start              = trim_whitespace(analysis.reasoning.start);
+              analysis.reasoning.end                = trim_whitespace(analysis.reasoning.end);
+              analysis.tools.arguments.value_prefix = trim_whitespace(analysis.tools.arguments.value_prefix);
+              analysis.tools.arguments.value_suffix = trim_whitespace(analysis.tools.arguments.value_suffix);
+              analysis.tools.arguments.separator    = trim_whitespace(analysis.tools.arguments.separator);
+              analysis.tools.arguments.tolerate_intertag_whitespace = true;
+              // The CONTROL/eot </assistant> token only halts generation when emitted as the
+              // single token; after tool calls the model can spell it out as text tokens.
+              // A literal stop string catches it either way.
+              analysis.additional_stops.push_back("</assistant>");
+              LOG_DBG(ANSI_ORANGE "[Patch: Laguna]\n" ANSI_RESET);
+          }
+      },
 
     });
 

common/chat.cpp

@@ -15,11 +15,13 @@
 
 #include "nlohmann/json.hpp"
 
+#include <algorithm>
 #include <cstdio>
 #include <cstdlib>
 #include <ctime>
 #include <exception>
 #include <functional>
+#include <map>
 
 #include <optional>
 #include <sstream>
@@ -1855,12 +1857,89 @@ static common_chat_params common_chat_params_init_gigachat_v3(
     return data;
 }
 
+// The DeepSeek V4 reference implementation renders consecutive tool results into a single
+// user block, ordered by the tool call order of the preceding assistant message (matched
+// by tool call id) rather than by the order they appear in the conversation.
+static json deepseek_v4_sort_tool_results(const json & messages) {
+    json adjusted = messages;
+    std::map<std::string, size_t> call_order;
+
+    for (size_t i = 0; i < adjusted.size();) {
+        const auto & msg  = adjusted[i];
+        const auto   role = msg.value("role", "");
+
+        if (role == "assistant" && msg.contains("tool_calls") &&
+                msg.at("tool_calls").is_array() && !msg.at("tool_calls").empty()) {
+            call_order.clear();
+            const auto & tool_calls = msg.at("tool_calls");
+            for (size_t idx = 0; idx < tool_calls.size(); idx++) {
+                auto id = tool_calls[idx].value("id", "");
+                if (!id.empty()) {
+                    call_order[id] = idx;
+                }
+            }
+            i++;
+            continue;
+        }
+
+        if (role != "user" && role != "tool") {
+            i++;
+            continue;
+        }
+
+        // collect a maximal run of user/tool messages - they render into one user block
+        std::vector<size_t> tool_positions;
+        size_t run_end = i;
+        for (; run_end < adjusted.size(); run_end++) {
+            const auto r = adjusted[run_end].value("role", "");
+            if (r == "tool") {
+                tool_positions.push_back(run_end);
+            } else if (r != "user") {
+                break;
+            }
+        }
+
+        if (tool_positions.size() > 1 && !call_order.empty()) {
+            std::vector<json> results;
+            results.reserve(tool_positions.size());
+            for (auto pos : tool_positions) {
+                results.push_back(adjusted[pos]);
+            }
+            std::stable_sort(results.begin(), results.end(), [&](const json & a, const json & b) {
+                const auto order = [&](const json & m) {
+                    auto it = call_order.find(m.value("tool_call_id", ""));
+                    return it == call_order.end() ? (size_t) 0 : it->second;
+                };
+                return order(a) < order(b);
+            });
+            for (size_t k = 0; k < tool_positions.size(); k++) {
+                adjusted[tool_positions[k]] = std::move(results[k]);
+            }
+        }
+
+        i = run_end;
+    }
+
+    return adjusted;
+}
+
 static common_chat_params common_chat_params_init_deepseek_v3_2(const common_chat_template &    tmpl,
                                                                  const autoparser::generation_params & inputs) {
     common_chat_params data;
 
-    data.prompt             = common_chat_template_direct_apply_impl(tmpl, inputs);
-    data.generation_prompt  = common_chat_template_generation_prompt_impl(tmpl, inputs);
+    // V4 uses the same DSML markup as V3.2, but names the tool call block "tool_calls"
+    // instead of "function_calls", renders tool results in tool call order and its
+    // non-thinking generation prompt ends with a bare </think> instead of an empty
+    // <think></think> pair.
+    const bool is_v4 = tmpl.source().find("function_calls") == std::string::npos;
+
+    std::optional<json> adjusted_messages;
+    if (is_v4) {
+        adjusted_messages = deepseek_v4_sort_tool_results(inputs.messages);
+    }
+
+    data.prompt             = common_chat_template_direct_apply_impl(tmpl, inputs, adjusted_messages);
+    data.generation_prompt  = common_chat_template_generation_prompt_impl(tmpl, inputs, adjusted_messages);
     data.format             = COMMON_CHAT_FORMAT_PEG_NATIVE;
     data.supports_thinking  = true;
     data.thinking_start_tag = "<think>";
@@ -1879,8 +1958,9 @@ static common_chat_params common_chat_params_init_deepseek_v3_2(const common_cha
     const std::string DSML         = "｜DSML｜";
     const std::string THINK_START  = "<think>";
     const std::string THINK_END    = "</think>";
-    const std::string FC_START     = "<" + DSML + "function_calls>";
-    const std::string FC_END       = "</" + DSML + "function_calls>";
+    const std::string TC_BLOCK     = is_v4 ? "tool_calls" : "function_calls";
+    const std::string FC_START     = "<" + DSML + TC_BLOCK + ">";
+    const std::string FC_END       = "</" + DSML + TC_BLOCK + ">";
     const std::string INVOKE_START = "<" + DSML + "invoke";
     const std::string INVOKE_END   = "</" + DSML + "invoke>";
     const std::string PARAM_START  = "<" + DSML + "parameter";
@@ -1907,8 +1987,11 @@ static common_chat_params common_chat_params_init_deepseek_v3_2(const common_cha
             reasoning = p.optional(THINK_START + p.reasoning(p.until(THINK_END)) + THINK_END);
         } else if (extract_reasoning) {
             // Thinking disabled but reasoning extraction requested: the generation prompt
-            // contains an empty <think></think> pair that must still be consumed.
-            reasoning = p.optional(p.literal(THINK_START) + p.until(THINK_END) + p.literal(THINK_END));
+            // contains an empty <think></think> pair (V3.2) or a bare </think> (V4) that
+            // must still be consumed.
+            reasoning = is_v4
+                ? p.optional(p.literal(THINK_END))
+                : p.optional(p.literal(THINK_START) + p.until(THINK_END) + p.literal(THINK_END));
         }
 
         if (has_response_format) {
@@ -2612,12 +2695,14 @@ std::optional<common_chat_params> common_chat_try_specialized_template(
         return common_chat_params_init_gigachat_v3(tmpl, params);
     }
 
-    // DeepSeek V3.2 format detection: template defines dsml_token and uses it for tool calls.
+    // DeepSeek V3.2/V4 format detection: template defines dsml_token and uses it for tool calls.
     // The template source contains the token as a variable assignment, not as a literal in markup.
+    // V3.2 names the tool call block "function_calls", V4 names it "tool_calls".
     if (src.find("dsml_token") != std::string::npos &&
-        src.find("function_calls") != std::string::npos &&
-        src.find("DSML") != std::string::npos) {
-        LOG_DBG("Using specialized template: DeepSeek V3.2\n");
+        src.find("DSML") != std::string::npos &&
+        (src.find("function_calls") != std::string::npos ||
+         src.find("tool_calls") != std::string::npos)) {
+        LOG_DBG("Using specialized template: DeepSeek V3.2/V4\n");
         return common_chat_params_init_deepseek_v3_2(tmpl, params);
     }
 

common/jinja/caps.cpp

@@ -23,6 +23,7 @@ void caps_apply_preserve_reasoning(jinja::context & ctx, bool enabled) {
     ctx.set_val("preserve_thinking",         mk_val<value_bool>(enabled));
     ctx.set_val("clear_thinking",            mk_val<value_bool>(!enabled));
     ctx.set_val("truncate_history_thinking", mk_val<value_bool>(!enabled));
+    ctx.set_val("drop_thinking",            mk_val<value_bool>(!enabled));
 }
 
 static void caps_try_execute(jinja::program & prog,

conversion/__init__.py

@@ -18,6 +18,7 @@ __all__ = [
 
 TEXT_MODEL_MAP: dict[str, str] = {
     "AfmoeForCausalLM": "afmoe",
+    "LagunaForCausalLM": "laguna",
     "ApertusForCausalLM": "llama",
     "ArceeForCausalLM": "llama",
     "ArcticForCausalLM": "arctic",

conversion/base.py

@@ -1682,6 +1682,9 @@ class TextModel(ModelBase):
         if chkhsh == "9dcf830ee9990cdbf78cc523a5f7bd9ad8f3f9890c2d3581d2785ad10f07049d":
             # ref: https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Base
             res = "mellum2"
+        if chkhsh == "972da7b59cec44d1f0a490a86c96df53859e486e481563e5dddac155013d87ac":
+            # ref: https://huggingface.co/poolside/Laguna-XS.2
+            res = "laguna"
 
         if res is None:
             logger.warning("\n")

conversion/bert.py

@@ -369,12 +369,13 @@ class NomicBertModel(BertModel):
         return super().filter_tensors(item)
 
     def modify_tensors(self, data_torch: torch.Tensor, name: str, bid: int | None) -> Iterable[tuple[str, torch.Tensor]]:
-        n_experts = self.find_hparam(["num_local_experts", "num_experts"])
         if "mlp.experts.mlp.w1" in name:
+            n_experts = self.find_hparam(["num_local_experts", "num_experts"])
             data_torch = data_torch.view(n_experts, self.hparams["n_inner"], self.hparams["n_embd"])
             name += ".weight"
 
         if "mlp.experts.mlp.w2" in name:
+            n_experts = self.find_hparam(["num_local_experts", "num_experts"])
             data_torch = data_torch.view(n_experts, self.hparams["n_inner"], self.hparams["n_embd"])
             data_torch = data_torch.transpose(1, 2)
             name += ".weight"

conversion/hunyuan.py

@@ -338,6 +338,12 @@ class HunyuanVLTextModel(HunYuanModel):
 
     def __init__(self, dir_model: Path, *args, **kwargs):
         super().__init__(dir_model, *args, **kwargs)
+        # transformers 5.13.0 encodes HunyuanVL XD-RoPE as dynamic + mrope_section.
+        # Normalize it to avoid the HunYuan dynamic-RoPE context assertion.
+        if self.rope_parameters.get("rope_type") == "dynamic" and "mrope_section" in self.rope_parameters:
+            self.rope_parameters["rope_type"] = "xdrope"
+            self.rope_parameters["type"] = "xdrope"
+            self.rope_parameters["xdrope_section"] = list(self.rope_parameters["mrope_section"])
 
     def set_gguf_parameters(self):
         super().set_gguf_parameters()

conversion/laguna.py

@@ -0,0 +1,207 @@
+from __future__ import annotations
+
+import re
+from collections.abc import Iterable
+from typing import TYPE_CHECKING
+
+import torch
+
+if TYPE_CHECKING:
+    from torch import Tensor
+
+from .base import ModelBase, TextModel, gguf, logger
+
+
+@ModelBase.register("LagunaForCausalLM")
+class LagunaModel(TextModel):
+    model_arch = gguf.MODEL_ARCH.LAGUNA
+    _experts: list[dict] | None = None
+    _gate_types: list[str] | None = None
+
+    # --- vocab ---------------------------------------------------------------
+
+    def set_vocab(self) -> None:
+        self._set_vocab_gpt2()
+
+        # Some Laguna releases wrap the chat template in tokenizer_config.json as
+        # "{% include 'chat_template.jinja' %}", which SpecialVocab embeds verbatim
+        # and llama.cpp's jinja engine cannot process. Prefer the resolved template
+        # from the chat_template.jinja file so the GGUF is self-contained.
+        tmpl_file = self.dir_model / "chat_template.jinja"
+        if tmpl_file.is_file():
+            self.gguf_writer.add_chat_template(tmpl_file.read_text(encoding="utf-8"))
+            logger.info("gguf: embedded resolved chat_template.jinja (overriding include directive)")
+
+        # eos_token_id is a list [2, 24]: token 2 (EOS, also BOS) and token 24
+        # (</assistant>, the turn-end). _set_vocab_gpt2 only records the scalar
+        # eos, so register the extra id as eot; llama.cpp folds eot into its EOG
+        # set, so the model halts on </assistant> natively.
+        eos_ids = self.hparams.get("eos_token_id")
+        if isinstance(eos_ids, list):
+            bos_id = self.hparams.get("bos_token_id")
+            extra = [e for e in eos_ids if e != bos_id]
+            if extra:
+                self.gguf_writer.add_eot_token_id(extra[0])
+                logger.info(f"gguf: registered eot_token_id={extra[0]} from eos list {eos_ids}")
+
+    def get_vocab_base(self) -> tuple[list[str], list[int], str]:
+        # </assistant> is the assistant turn-end (registered as eot below). The
+        # HF tokenizer flags it special=false, so the base classifies it as
+        # USER_DEFINED and llama.cpp renders its text into generated content,
+        # leaking "</assistant>" and breaking response parsing. It is a control
+        # marker, so promote it to CONTROL: llama.cpp then treats it as
+        # end-of-generation and suppresses its text.
+        tokens, toktypes, tokpre = super().get_vocab_base()
+        for i, tok in enumerate(tokens):
+            if tok == "</assistant>":
+                toktypes[i] = gguf.TokenType.CONTROL
+                logger.info(f"gguf: marked </assistant> (id {i}) as CONTROL token")
+        return tokens, toktypes, tokpre
+
+    # --- hparams -------------------------------------------------------------
+
+    def set_gguf_parameters(self) -> None:
+        super().set_gguf_parameters()
+        hparams = self.hparams
+
+        # super() does not emit vocab_size for the gpt2 vocab path; head_count is
+        # overridden with a per-layer array (XS.2 varies heads per layer via
+        # num_attention_heads_per_layer; M.1 is uniform and omits it).
+        self.gguf_writer.add_vocab_size(hparams["vocab_size"])
+
+        per_layer_heads = hparams.get("num_attention_heads_per_layer")
+        if not per_layer_heads:
+            per_layer_heads = [hparams["num_attention_heads"]] * hparams["num_hidden_layers"]
+        assert len(per_layer_heads) == hparams["num_hidden_layers"], (
+            f"num_attention_heads_per_layer length {len(per_layer_heads)} != "
+            f"num_hidden_layers {hparams['num_hidden_layers']}"
+        )
+        self.gguf_writer.add_head_count(per_layer_heads)
+
+        # Resolve + validate the attention gate type now so an inconsistent
+        # `gating` field fails at conversion time. See _attn_gate_types.
+        self._attn_gate_types()
+
+        # SWA window size (M.1 has none -> key omitted, swa_type stays NONE).
+        sliding_window = hparams.get("sliding_window") or 0
+        if sliding_window > 0:
+            self.gguf_writer.add_sliding_window(sliding_window)
+
+        # MoE (expert_count / expert_used_count come from super().set_gguf_parameters())
+        self.gguf_writer.add_expert_feed_forward_length(hparams["moe_intermediate_size"])
+        self.gguf_writer.add_expert_shared_feed_forward_length(hparams["shared_expert_intermediate_size"])
+        self.gguf_writer.add_expert_weights_norm(True)  # HF reference always sum-normalises after top-k
+        self.gguf_writer.add_expert_weights_scale(float(hparams["moe_routed_scaling_factor"]))
+        self.gguf_writer.add_expert_gating_func(gguf.ExpertGatingFuncType.SIGMOID)
+
+        # Leading dense layers (XS.2 has 1, M.1 has 3) before the MoE layers.
+        mlp_layer_types: list[str] = hparams["mlp_layer_types"]
+        leading_dense = 0
+        for t in mlp_layer_types:
+            if t == "dense":
+                leading_dense += 1
+            else:
+                break
+        self.gguf_writer.add_leading_dense_block_count(leading_dense)
+
+        # Per-layer-type RoPE dimension count (partial rotary). base emits
+        # rope_freq_base(_swa) and the YaRN params from self.rope_parameters.
+        head_dim = hparams["head_dim"]
+        full_rope = self.rope_parameters["full_attention"]
+        self.gguf_writer.add_rope_dimension_count(
+            int(head_dim * float(full_rope.get("partial_rotary_factor", 1.0))))
+        swa_rope = self.rope_parameters.get("sliding_attention")
+        if swa_rope is not None:
+            self.gguf_writer.add_rope_dimension_count_swa(
+                int(head_dim * float(swa_rope.get("partial_rotary_factor", 1.0))))
+
+    def _attn_gate_types(self) -> list[str]:
+        """Per-layer attention output gate type: "per_head" or "per_element".
+
+        `gating_types` (per layer) is authoritative when present; otherwise the
+        scalar `gating` field is used (the "per-element"/"per-head" string, or
+        the legacy boolean True == per-head, as in Laguna-XS.2).
+
+        Fails loudly when the model is per-element but the `gating` field does
+        not declare that as a string: runtimes that key off `gating` (vLLM,
+        transformers) ignore gating_types and read a bare boolean True as
+        per-head, silently corrupting the model. Surfacing it here keeps a
+        broken checkpoint from being packaged as if it were fine.
+        """
+        if self._gate_types is not None:
+            return self._gate_types
+        hparams = self.hparams
+        n_layer = hparams["num_hidden_layers"]
+        gating = hparams.get("gating")
+        gating_types = hparams.get("gating_types")
+
+        def _norm(t: object) -> str:
+            sval = str(t).replace("-", "_")
+            if sval in ("per_element", "per_head"):
+                return sval
+            raise ValueError(f"Laguna: unrecognised attention gate type {t!r}")
+
+        if gating_types:
+            assert len(gating_types) == n_layer, (
+                f"gating_types length {len(gating_types)} != num_hidden_layers {n_layer}")
+            types = [_norm(t) for t in gating_types]
+        elif isinstance(gating, str):
+            types = [_norm(gating)] * n_layer
+        elif gating is True:
+            types = ["per_head"] * n_layer
+        else:
+            raise ValueError(
+                f"Laguna: cannot determine attention gate type "
+                f"(gating={gating!r}, gating_types={gating_types!r})")
+
+        if any(t == "per_element" for t in types) and not (
+                isinstance(gating, str) and _norm(gating) == "per_element"):
+            raise ValueError(
+                f"Laguna config declares a per-element attention gate but "
+                f"`gating`={gating!r} is not the string \"per-element\". Runtimes that "
+                f"read `gating` (vLLM, transformers) will mis-handle this checkpoint as "
+                f"per-head. Set gating=\"per-element\" in the source config.")
+
+        self._gate_types = types
+        return types
+
+    # --- tensor handling -----------------------------------------------------
+
+    def modify_tensors(self, data_torch: Tensor, name: str, bid: int | None) -> Iterable[tuple[str, Tensor]]:
+        # Per-expert MoE weights: model.layers.{bid}.mlp.experts.{xid}.{w}.weight.
+        # Only the NUMBERED per-expert weights are stacked; the router bias
+        # (mlp.experts.e_score_correction_bias) takes the normal mapping path.
+        if re.search(r"mlp\.experts\.\d+\.", name):
+            n_experts = self.find_hparam(["num_local_experts", "num_experts"])
+            assert bid is not None
+            if self._experts is None:
+                self._experts = [{} for _ in range(self.block_count)]
+            self._experts[bid][name] = data_torch
+            needed = [f"model.layers.{bid}.mlp.experts.{x}.{w}.weight"
+                      for x in range(n_experts) for w in ("gate_proj", "up_proj", "down_proj")]
+            if all(e in self._experts[bid] for e in needed):
+                for w_name in ["gate_proj", "up_proj", "down_proj"]:
+                    datas = [self._experts[bid][f"model.layers.{bid}.mlp.experts.{x}.{w_name}.weight"]
+                             for x in range(n_experts)]
+                    stacked = torch.stack(datas, dim=0)
+                    merged = f"model.layers.{bid}.mlp.experts.{w_name}.weight"
+                    yield from TextModel.modify_tensors(self, stacked, merged, bid)
+                self._experts[bid].clear()
+                return
+            return
+        # Cross-check the gate projection width against the declared gate type;
+        # a mismatch means the weights and config disagree -> fail, do not guess.
+        if bid is not None and name.endswith("self_attn.g_proj.weight"):
+            heads = (self.hparams.get("num_attention_heads_per_layer")
+                     or [self.hparams["num_attention_heads"]] * self.hparams["num_hidden_layers"])
+            n_head = heads[bid]
+            head_dim = self.hparams["head_dim"]
+            gate_type = self._attn_gate_types()[bid]
+            expected = n_head * head_dim if gate_type == "per_element" else n_head
+            out_features = int(data_torch.shape[0])
+            if out_features != expected:
+                raise ValueError(
+                    f"Laguna layer {bid}: g_proj output width {out_features} contradicts the "
+                    f"declared {gate_type} gate (expected {expected}); weights and config disagree.")
+
+        yield from TextModel.modify_tensors(self, data_torch, name, bid)

convert_hf_to_gguf_update.py

@@ -162,6 +162,7 @@ models = [
     {"name": "granite-embed-multi-97m", "tokt": TOKENIZER_TYPE.BPE, "repo": "https://huggingface.co/ibm-granite/granite-embedding-97m-multilingual-r2", },
     {"name": "granite-embed-multi-311m", "tokt": TOKENIZER_TYPE.BPE, "repo": "https://huggingface.co/ibm-granite/granite-embedding-311m-multilingual-r2", },
     {"name": "mellum2",          "tokt": TOKENIZER_TYPE.BPE, "repo": "https://huggingface.co/JetBrains/Mellum2-12B-A2.5B-Base"},
+    {"name": "laguna",           "tokt": TOKENIZER_TYPE.BPE, "repo": "https://huggingface.co/poolside/Laguna-XS.2", },
 ]
 
 # some models are known to be broken upstream, so we will skip them as exceptions

docs/ops.md

@@ -25,10 +25,10 @@ Legend:
 |                             CEIL | ❌ | ❌ | ✅ | 🟡 | 🟡 | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
 |                            CLAMP | ❌ | ✅ | ✅ | ✅ | 🟡 | ✅ | 🟡 | ✅ | 🟡 | ✅ | ❌ | ❌ |
 |                        COL2IM_1D | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
-|                           CONCAT | ❌ | ✅ | ✅ | 🟡 | 🟡 | ✅ | 🟡 | ✅ | ✅ | ✅ | ❌ | ❌ |
+|                           CONCAT | ❌ | ✅ | ✅ | 🟡 | 🟡 | ✅ | 🟡 | ✅ | ✅ | 🟡 | ❌ | ❌ |
 |                             CONT | ❌ | 🟡 | ✅ | ✅ | 🟡 | ✅ | 🟡 | ✅ | ✅ | 🟡 | ❌ | ❌ |
 |                          CONV_2D | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
-|                       CONV_2D_DW | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
+|                       CONV_2D_DW | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
 |                          CONV_3D | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ✅ | ❌ | ❌ | ❌ | ❌ |
 |                CONV_TRANSPOSE_1D | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
 |                CONV_TRANSPOSE_2D | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
@@ -41,6 +41,9 @@ Legend:
 |                             DIAG | ❌ | ❌ | ✅ | ✅ | 🟡 | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
 |                    DIAG_MASK_INF | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ | 🟡 | ✅ | ✅ | ❌ | ❌ | ❌ |
 |                              DIV | ❌ | ✅ | ✅ | ✅ | ❌ | 🟡 | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
+|                     DSV4_HC_COMB | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
+|                     DSV4_HC_POST | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
+|                      DSV4_HC_PRE | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
 |                              DUP | ❌ | ✅ | ✅ | 🟡 | ❌ | 🟡 | 🟡 | ✅ | ✅ | ❌ | ❌ | ❌ |
 |                              ELU | ❌ | ✅ | ✅ | 🟡 | 🟡 | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
 |                              EXP | ❌ | ✅ | ✅ | 🟡 | 🟡 | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
@@ -63,16 +66,17 @@ Legend:
 |                        HARDSWISH | ❌ | ✅ | ✅ | 🟡 | 🟡 | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
 |                           IM2COL | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
 |                        IM2COL_3D | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
-|                          L2_NORM | ❌ | ✅ | ✅ | ✅ | 🟡 | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
+|                          L2_NORM | ❌ | ✅ | ✅ | ✅ | 🟡 | ✅ | ❌ | ✅ | ✅ | 🟡 | ❌ | ❌ |
 |                       LEAKY_RELU | ❌ | ✅ | ✅ | ✅ | ❌ | 🟡 | ❌ | ✅ | 🟡 | ❌ | ❌ | ❌ |
+|                LIGHTNING_INDEXER | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
 |                              LOG | ❌ | ✅ | ✅ | ✅ | ❌ | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
 |                             MEAN | ❌ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
 |                              MUL | ❌ | ✅ | ✅ | ✅ | 🟡 | 🟡 | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
 |                          MUL_MAT | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 | 🟡 |
-|                 MUL_MAT_HADAMARD | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
+|                 MUL_MAT_HADAMARD | ❌ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
 |                       MUL_MAT_ID | ❌ | 🟡 | ✅ | ✅ | 🟡 | 🟡 | 🟡 | ✅ | ✅ | 🟡 | 🟡 | ❌ |
 |                              NEG | ❌ | ✅ | ✅ | 🟡 | 🟡 | ✅ | ❌ | ✅ | ✅ | ✅ | ❌ | ❌ |
-|                             NORM | ❌ | ✅ | ✅ | ✅ | 🟡 | ✅ | ✅ | ✅ | 🟡 | ✅ | ❌ | ❌ |
+|                             NORM | ❌ | ✅ | ✅ | ✅ | 🟡 | ✅ | ✅ | ✅ | 🟡 | 🟡 | ❌ | ❌ |
 |                   OPT_STEP_ADAMW | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
 |                     OPT_STEP_SGD | ❌ | ❌ | ✅ | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
 |                         OUT_PROD | 🟡 | 🟡 | 🟡 | 🟡 | ❌ | ❌ | ❌ | 🟡 | ❌ | ❌ | ❌ | 🟡 |
@@ -82,7 +86,7 @@ Legend:
 |                          POOL_2D | ❌ | 🟡 | ✅ | ✅ | ❌ | ✅ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
 |                            REGLU | ❌ | ✅ | ✅ | ✅ | 🟡 | 🟡 | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
 |                             RELU | ❌ | ✅ | ✅ | 🟡 | 🟡 | ✅ | 🟡 | ✅ | ✅ | ✅ | ❌ | ❌ |
-|                           REPEAT | ❌ | ✅ | ✅ | 🟡 | 🟡 | ✅ | 🟡 | ✅ | ✅ | ✅ | ❌ | ❌ |
+|                           REPEAT | ❌ | ✅ | ✅ | 🟡 | 🟡 | ✅ | 🟡 | ✅ | ✅ | 🟡 | ❌ | ❌ |
 |                      REPEAT_BACK | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |
 |                         RMS_NORM | ❌ | ✅ | ✅ | ✅ | 🟡 | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ |
 |                    RMS_NORM_BACK | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ | ✅ | ✅ | ❌ | ❌ | ❌ |

ggml/src/CMakeLists.txt

@@ -430,7 +430,7 @@ if (GGML_CPU_ALL_VARIANTS)
             message(FATAL_ERROR "Unsupported ARM target OS: ${CMAKE_SYSTEM_NAME}")
         endif()
     elseif (GGML_SYSTEM_ARCH STREQUAL "PowerPC")
-        if (CMAKE_SYSTEM_NAME MATCHES "Linux")
+        if (CMAKE_SYSTEM_NAME MATCHES "Linux|AIX")
             ggml_add_cpu_backend_variant(power0)
             ggml_add_cpu_backend_variant(power7_1       POWER7)
             ggml_add_cpu_backend_variant(power7_2       POWER7  VSX)

ggml/src/ggml-backend.cpp

@@ -1546,11 +1546,23 @@ static enum ggml_status ggml_backend_sched_compute_splits(ggml_backend_sched_t s
     std::vector<int32_t> ids;
     std::vector<ggml_bitset_t> used_ids;
 
+    int prev_backend_id = -1;
+
     for (int split_id = 0; split_id < sched->n_splits; split_id++) {
         struct ggml_backend_sched_split * split = &splits[split_id];
         int split_backend_id = split->backend_id;
         ggml_backend_t split_backend = sched->backends[split_backend_id];
 
+        // ensure the previous split's async work has completed before we start
+        // this split, the allocator may have reused buffer regions across splits
+        if (prev_backend_id >= 0 && prev_backend_id != split_backend_id) {
+            if (sched->events[prev_backend_id][sched->cur_copy] != NULL) {
+                ggml_backend_event_synchronize(sched->events[prev_backend_id][sched->cur_copy]);
+            } else {
+                ggml_backend_synchronize(sched->backends[prev_backend_id]);
+            }
+        }
+
         // copy the input tensors to the split backend
         for (int input_id = 0; input_id < split->n_inputs; input_id++) {
             ggml_backend_t input_backend = ggml_backend_sched_get_tensor_backend(sched, split->inputs[input_id]);
@@ -1713,12 +1725,12 @@ static enum ggml_status ggml_backend_sched_compute_splits(ggml_backend_sched_t s
             }
         }
 
-        // record the event of this copy
-        if (split->n_inputs > 0) {
-            if (sched->events[split_backend_id][sched->cur_copy] != NULL) {
-                ggml_backend_event_record(sched->events[split_backend_id][sched->cur_copy], split_backend);
-            }
+        // record the event of this split
+        if (sched->events[split_backend_id][sched->cur_copy] != NULL) {
+            ggml_backend_event_record(sched->events[split_backend_id][sched->cur_copy], split_backend);
         }
+
+        prev_backend_id = split_backend_id;
     }
 
     return GGML_STATUS_SUCCESS;

ggml/src/ggml-cpu/kleidiai/kleidiai.cpp

@@ -1719,6 +1719,7 @@ class extra_buffer_type : ggml::cpu::extra_buffer_type {
 
             return true;
         }
+
         return false;
     }
 
@@ -1727,6 +1728,20 @@ class extra_buffer_type : ggml::cpu::extra_buffer_type {
             if (op->src[0]->buffer && op->src[0]->buffer->buft == ggml_backend_cpu_kleidiai_buffer_type()) {
                 return (ggml::cpu::tensor_traits *) op->src[0]->extra;
             } else {
+                // KleidiAI only has kernels for Q4_0 and Q8_0. For a quantized weight of any
+                // other type (K-quants, IQ) it declines the op and returns nullptr below, so
+                // KleidiAI does not accelerate it. Another CPU backend may still take the op,
+                // and this can run during graph planning, so the message says what KleidiAI
+                // did rather than what ends up executing. Warn once per process.
+                if (ggml_is_quantized(op->src[0]->type) &&
+                    op->src[0]->type != GGML_TYPE_Q4_0 && op->src[0]->type != GGML_TYPE_Q8_0) {
+                    static std::atomic<bool> warned(false);
+                    if (!warned.exchange(true)) {
+                        GGML_LOG_WARN("kleidiai: no kernel for tensor type %s, not accelerated by KleidiAI "
+                                      "(kernels available for Q4_0 and Q8_0)\n",
+                                      ggml_type_name(op->src[0]->type));
+                    }
+                }
                 if (op->src[0]->type != GGML_TYPE_F16) {
                     return nullptr;
                 }

ggml/src/ggml-cpu/llamafile/sgemm.cpp

@@ -2329,7 +2329,7 @@ class tinyBLAS_Q0_PPC {
         mc = 32;
         nc = 32;
         kc = 32;
-        n_chunk = 32
+        n_chunk = 32;
     #endif
         int64_t n_aligned = 0;
         if (n % n_chunk == 0) {

ggml/src/ggml-cuda/common.cuh

@@ -362,6 +362,15 @@ static bool blackwell_mma_available(const int cc) {
            ggml_cuda_highest_compiled_arch(cc) < GGML_CUDA_CC_RUBIN;
 }
 
+// Checks whether the tensor's base data pointer and higher-dimensional strides are byte-aligned to `alignment` bytes.
+static bool ggml_cuda_is_aligned(const ggml_tensor * tensor, const size_t alignment) {
+    GGML_ASSERT(tensor != nullptr);
+    return (reinterpret_cast<uintptr_t>(tensor->data) % alignment) == 0 &&
+           tensor->nb[1] % alignment == 0 &&
+           tensor->nb[2] % alignment == 0 &&
+           tensor->nb[3] % alignment == 0;
+}
+
 static constexpr __device__ int ggml_cuda_get_physical_warp_size() {
 #if defined(GGML_USE_HIP) && (defined(__GFX9__) || defined(__GFX8__))
     return 64;
@@ -937,6 +946,9 @@ static __device__ __forceinline__ uint2 fast_div_modulo(uint32_t n, const uint3
 
 typedef void (*dequantize_kernel_t)(const void * vx, const int64_t ib, const int iqs, float2 & v);
 
+template<typename dst_t>
+using dequantize_kq_t = void (*)(const void * vx, const int64_t ib, dst_t * y, const int tid);
+
 static __device__ __forceinline__ float get_alibi_slope(
     const float max_bias, const uint32_t h, const uint32_t n_head_log2, const float m0, const float m1
 ) {

ggml/src/ggml-cuda/convert.cu

@@ -140,358 +140,107 @@ static __global__ void dequantize_block_q4_1(const void * __restrict__ vx, dst_t
 
 template<typename dst_t>
 static __global__ void dequantize_block_q2_K(const void * __restrict__ vx, dst_t * __restrict__ yy) {
+    const int64_t i = blockIdx.x;
 
-    const int64_t i   = blockIdx.x;
-    const block_q2_K * x = (const block_q2_K *) vx;
-
-    const int64_t tid = threadIdx.x;
-    const int64_t n   = tid/32;
-    const int64_t l   = tid - 32*n;
-    const int64_t is  = 8*n + l/16;
-
-    const uint8_t q = x[i].qs[32*n + l];
-    dst_t * y = yy + i*QK_K + 128*n;
-
-    float dall = __low2half(x[i].dm);
-    float dmin = __high2half(x[i].dm);
-    y[l+ 0] = ggml_cuda_cast<dst_t>(dall * (x[i].scales[is+0] & 0xF) * ((q >> 0) & 3) - dmin * (x[i].scales[is+0] >> 4));
-    y[l+32] = ggml_cuda_cast<dst_t>(dall * (x[i].scales[is+2] & 0xF) * ((q >> 2) & 3) - dmin * (x[i].scales[is+2] >> 4));
-    y[l+64] = ggml_cuda_cast<dst_t>(dall * (x[i].scales[is+4] & 0xF) * ((q >> 4) & 3) - dmin * (x[i].scales[is+4] >> 4));
-    y[l+96] = ggml_cuda_cast<dst_t>(dall * (x[i].scales[is+6] & 0xF) * ((q >> 6) & 3) - dmin * (x[i].scales[is+6] >> 4));
+    dequantize_q2_K(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_q3_K(const void * __restrict__ vx, dst_t * __restrict__ yy) {
-
     const int64_t i = blockIdx.x;
-    const block_q3_K * x = (const block_q3_K *) vx;
 
-    const int64_t r = threadIdx.x/4;
-    const int64_t tid = r/2;
-    const int64_t is0 = r%2;
-    const int64_t l0 = 16*is0 + 4*(threadIdx.x%4);
-    const int64_t n = tid / 4;
-    const int64_t j = tid - 4*n;
-
-    uint8_t m = 1 << (4*n + j);
-    int64_t is = 8*n + 2*j + is0;
-    int shift = 2*j;
-
-    int8_t us = is <  4 ? (x[i].scales[is-0] & 0xF) | (((x[i].scales[is+8] >> 0) & 3) << 4) :
-                is <  8 ? (x[i].scales[is-0] & 0xF) | (((x[i].scales[is+4] >> 2) & 3) << 4) :
-                is < 12 ? (x[i].scales[is-8] >>  4) | (((x[i].scales[is+0] >> 4) & 3) << 4) :
-                          (x[i].scales[is-8] >>  4) | (((x[i].scales[is-4] >> 6) & 3) << 4);
-    float d_all = x[i].d;
-    float dl = d_all * (us - 32);
-
-    dst_t * y = yy + i*QK_K + 128*n + 32*j;
-    const uint8_t * q = x[i].qs + 32*n;
-    const uint8_t * hm = x[i].hmask;
-
-    for (int l = l0; l < l0+4; ++l) {
-        y[l] = ggml_cuda_cast<dst_t>(dl * ((int8_t)((q[l] >> shift) & 3) - ((hm[l] & m) ? 0 : 4)));
-    }
-}
-
-static inline __device__ void get_scale_min_k4(int j, const uint8_t * q, uint8_t & d, uint8_t & m) {
-    if (j < 4) {
-        d = q[j] & 63; m = q[j + 4] & 63;
-    } else {
-        d = (q[j+4] & 0xF) | ((q[j-4] >> 6) << 4);
-        m = (q[j+4] >>  4) | ((q[j-0] >> 6) << 4);
-    }
+    dequantize_q3_K(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_q4_K(const void * __restrict__ vx, dst_t * __restrict__ yy) {
-    const block_q4_K * x = (const block_q4_K *) vx;
-
     const int64_t i = blockIdx.x;
 
-    // assume 32 threads
-    const int64_t tid = threadIdx.x;
-    const int64_t il  = tid/8;
-    const int64_t ir  = tid%8;
-    const int64_t is  = 2*il;
-    const int64_t n   = 4;
-
-    dst_t * y = yy + i*QK_K + 64*il + n*ir;
-
-    const float dall = __low2half(x[i].dm);
-    const float dmin = __high2half(x[i].dm);
-
-    const uint8_t * q = x[i].qs + 32*il + n*ir;
-
-    uint8_t sc, m;
-    get_scale_min_k4(is + 0, x[i].scales, sc, m);
-    const float d1 = dall * sc; const float m1 = dmin * m;
-    get_scale_min_k4(is + 1, x[i].scales, sc, m);
-    const float d2 = dall * sc; const float m2 = dmin * m;
-    for (int l = 0; l < n; ++l) {
-        y[l + 0] = ggml_cuda_cast<dst_t>(d1 * (q[l] & 0xF) - m1);
-        y[l +32] = ggml_cuda_cast<dst_t>(d2 * (q[l] >>  4) - m2);
-    }
+    dequantize_q4_K(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_q5_K(const void * __restrict__ vx, dst_t * __restrict__ yy) {
-    const block_q5_K * x = (const block_q5_K *) vx;
-
     const int64_t i = blockIdx.x;
 
-    // assume 64 threads - this is very slightly better than the one below
-    const int64_t tid = threadIdx.x;
-    const int64_t il  = tid/16;   // il is in 0...3
-    const int64_t ir  = tid%16;   // ir is in 0...15
-    const int64_t is  = 2*il;     // is is in 0...6
-
-    dst_t * y = yy + i*QK_K + 64*il + 2*ir;
-
-    const float dall = __low2half(x[i].dm);
-    const float dmin = __high2half(x[i].dm);
-
-    const uint8_t * ql = x[i].qs + 32*il + 2*ir;
-    const uint8_t * qh = x[i].qh + 2*ir;
-
-    uint8_t sc, m;
-    get_scale_min_k4(is + 0, x[i].scales, sc, m);
-    const float d1 = dall * sc; const float m1 = dmin * m;
-    get_scale_min_k4(is + 1, x[i].scales, sc, m);
-    const float d2 = dall * sc; const float m2 = dmin * m;
-
-    uint8_t   hm  = 1 << (2*il);
-    y[ 0] = ggml_cuda_cast<dst_t>(d1 * ((ql[ 0] & 0xF) + (qh[ 0] & hm ? 16 : 0)) - m1);
-    y[ 1] = ggml_cuda_cast<dst_t>(d1 * ((ql[ 1] & 0xF) + (qh[ 1] & hm ? 16 : 0)) - m1);
-    hm <<= 1;
-    y[32] = ggml_cuda_cast<dst_t>(d2 * ((ql[ 0] >>  4) + (qh[ 0] & hm ? 16 : 0)) - m2);
-    y[33] = ggml_cuda_cast<dst_t>(d2 * ((ql[ 1] >>  4) + (qh[ 1] & hm ? 16 : 0)) - m2);
+    dequantize_q5_K(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_q6_K(const void * __restrict__ vx, dst_t * __restrict__ yy) {
-    const block_q6_K * x = (const block_q6_K *) vx;
-
     const int64_t i = blockIdx.x;
 
-    // assume 64 threads - this is very slightly better than the one below
-    const int64_t tid = threadIdx.x;
-    const int64_t ip  = tid/32;   // ip is 0 or 1
-    const int64_t il  = tid - 32*ip; // 0...32
-    const int64_t is  = 8*ip + il/16;
-
-    dst_t * y = yy + i*QK_K + 128*ip + il;
-
-    const float d = x[i].d;
-
-    const uint8_t * ql = x[i].ql + 64*ip + il;
-    const uint8_t   qh = x[i].qh[32*ip + il];
-    const int8_t  * sc = x[i].scales + is;
-
-    y[ 0] = ggml_cuda_cast<dst_t>(d * sc[0] * ((int8_t)((ql[ 0] & 0xF) | (((qh >> 0) & 3) << 4)) - 32));
-    y[32] = ggml_cuda_cast<dst_t>(d * sc[2] * ((int8_t)((ql[32] & 0xF) | (((qh >> 2) & 3) << 4)) - 32));
-    y[64] = ggml_cuda_cast<dst_t>(d * sc[4] * ((int8_t)((ql[ 0]  >> 4) | (((qh >> 4) & 3) << 4)) - 32));
-    y[96] = ggml_cuda_cast<dst_t>(d * sc[6] * ((int8_t)((ql[32]  >> 4) | (((qh >> 6) & 3) << 4)) - 32));
+    dequantize_q6_K(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_iq2_xxs(const void * __restrict__ vx, dst_t * __restrict__ yy) {
+    const int64_t i = blockIdx.x;
 
-    const int64_t i   = blockIdx.x;
-    const block_iq2_xxs * x = (const block_iq2_xxs  *) vx;
-
-    const int64_t tid = threadIdx.x;
-    const int64_t il = tid/8; // 0...3
-    const int64_t ib = tid%8; // 0...7
-    dst_t * y = yy + i*QK_K + 32*ib + 8*il;
-    const uint16_t * q2 = x[i].qs + 4*ib;
-    const uint8_t  * aux8 = (const uint8_t *)q2;
-    const uint8_t  * grid = (const uint8_t *)(iq2xxs_grid + aux8[il]);
-    const uint32_t aux32 = q2[2] | (q2[3] << 16);
-    const float d = (float)x[i].d * (0.5f + (aux32 >> 28)) * 0.25f;
-    const uint8_t signs = ksigns_iq2xs[(aux32 >> 7*il) & 127];
-    for (int j = 0; j < 8; ++j) {
-        y[j] = ggml_cuda_cast<dst_t>(d * grid[j] * (signs & kmask_iq2xs[j] ? -1.f : 1.f));
-    }
+    dequantize_iq2_xxs(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_iq2_xs(const void * __restrict__ vx, dst_t * __restrict__ yy) {
+    const int64_t i = blockIdx.x;
 
-    const int64_t i   = blockIdx.x;
-    const block_iq2_xs * x = (const block_iq2_xs *) vx;
-
-    const int64_t tid = threadIdx.x;
-    const int64_t il = tid/8; // 0...3
-    const int64_t ib = tid%8; // 0...7
-    dst_t * y = yy + i*QK_K + 32*ib + 8*il;
-    const uint16_t * q2 = x[i].qs + 4*ib;
-    const uint8_t  * grid = (const uint8_t *)(iq2xs_grid + (q2[il] & 511));
-    const float d = (float)x[i].d * (0.5f + ((x[i].scales[ib] >> 4*(il/2)) & 0xf)) * 0.25f;
-    const uint8_t signs = ksigns_iq2xs[q2[il] >> 9];
-    for (int j = 0; j < 8; ++j) {
-        y[j] = ggml_cuda_cast<dst_t>(d * grid[j] * (signs & kmask_iq2xs[j] ? -1.f : 1.f));
-    }
+    dequantize_iq2_xs(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_iq2_s(const void * __restrict__ vx, dst_t * __restrict__ yy) {
+    const int64_t i = blockIdx.x;
 
-    const int64_t i   = blockIdx.x;
-    const block_iq2_s * x = (const block_iq2_s *) vx;
-
-    const int64_t tid = threadIdx.x;
-    const int64_t il = tid/8; // 0...3
-    const int64_t ib = tid%8; // 0...7
-    dst_t * y = yy + i*QK_K + 32*ib + 8*il;
-    const uint8_t * grid = (const uint8_t *)(iq2s_grid + (x[i].qs[4*ib+il] | ((x[i].qh[ib] << (8-2*il)) & 0x300)));
-    const float d = (float)x[i].d * (0.5f + ((x[i].scales[ib] >> 4*(il/2)) & 0xf)) * 0.25f;
-    const uint8_t signs = x[i].qs[QK_K/8+4*ib+il];
-    for (int j = 0; j < 8; ++j) {
-        y[j] = ggml_cuda_cast<dst_t>(d * grid[j] * (signs & kmask_iq2xs[j] ? -1.f : 1.f));
-    }
+    dequantize_iq2_s(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_iq3_xxs(const void * __restrict__ vx, dst_t * __restrict__ yy) {
+    const int64_t i = blockIdx.x;
 
-    const int64_t i   = blockIdx.x;
-    const block_iq3_xxs * x = (const block_iq3_xxs  *) vx;
-
-    const int64_t tid = threadIdx.x;
-    const int64_t il = tid/8; // 0...3
-    const int64_t ib = tid%8; // 0...7
-    dst_t * y = yy + i*QK_K + 32*ib + 8*il;
-    const uint8_t  * q3 = x[i].qs + 8*ib;
-    const uint16_t * gas = (const uint16_t *)(x[i].qs + QK_K/4) + 2*ib;
-    const uint8_t  * grid1 = (const uint8_t *)(iq3xxs_grid + q3[2*il+0]);
-    const uint8_t  * grid2 = (const uint8_t *)(iq3xxs_grid + q3[2*il+1]);
-    const uint32_t aux32 = gas[0] | (gas[1] << 16);
-    const float d = (float)x[i].d * (0.5f + (aux32 >> 28)) * 0.5f;
-    const uint8_t signs = ksigns_iq2xs[(aux32 >> 7*il) & 127];
-    for (int j = 0; j < 4; ++j) {
-        y[j+0] = ggml_cuda_cast<dst_t>(d * grid1[j] * (signs & kmask_iq2xs[j+0] ? -1.f : 1.f));
-        y[j+4] = ggml_cuda_cast<dst_t>(d * grid2[j] * (signs & kmask_iq2xs[j+4] ? -1.f : 1.f));
-    }
+    dequantize_iq3_xxs(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_iq3_s(const void * __restrict__ vx, dst_t * __restrict__ yy) {
+    const int64_t i = blockIdx.x;
 
-    const int64_t i   = blockIdx.x;
-    const block_iq3_s * x = (const block_iq3_s *) vx;
-
-    const int64_t tid = threadIdx.x;
-    const int64_t il = tid/8; // 0...3
-    const int64_t ib = tid%8; // 0...7
-    dst_t * y = yy + i*QK_K + 32*ib + 8*il;
-    const uint8_t * qs = x[i].qs + 8*ib;
-    const uint8_t * grid1 = (const uint8_t *)(iq3s_grid + (qs[2*il+0] | ((x[i].qh[ib] << (8-2*il)) & 256)));
-    const uint8_t * grid2 = (const uint8_t *)(iq3s_grid + (qs[2*il+1] | ((x[i].qh[ib] << (7-2*il)) & 256)));
-    const float d = (float)x[i].d * (1 + 2*((x[i].scales[ib/2] >> 4*(ib%2)) & 0xf));
-    const uint8_t signs = x[i].signs[4*ib + il];
-    for (int j = 0; j < 4; ++j) {
-        y[j+0] = ggml_cuda_cast<dst_t>(d * grid1[j] * (signs & kmask_iq2xs[j+0] ? -1.f : 1.f));
-        y[j+4] = ggml_cuda_cast<dst_t>(d * grid2[j] * (signs & kmask_iq2xs[j+4] ? -1.f : 1.f));
-    }
+    dequantize_iq3_s(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_iq1_s(const void * __restrict__ vx, dst_t * __restrict__ yy) {
+    const int64_t i = blockIdx.x;
 
-    const int64_t i   = blockIdx.x;
-    const block_iq1_s * x = (const block_iq1_s  *) vx;
-
-    const int64_t tid = threadIdx.x;
-    const int64_t il = tid/8; // 0...3
-    const int64_t ib = tid%8; // 0...7
-    dst_t * y = yy + i*QK_K + 32*ib + 8*il;
-    const float delta = x[i].qh[ib] & 0x8000 ? -1 - IQ1S_DELTA : -1 + IQ1S_DELTA;
-    const float d = (float)x[i].d * (2*((x[i].qh[ib] >> 12) & 7) + 1);
-    uint32_t grid32[2]; const int8_t * q = (const int8_t *)grid32;
-    grid32[0] = iq1s_grid_gpu[x[i].qs[4*ib+il] | (((x[i].qh[ib] >> 3*il) & 7) << 8)];
-    grid32[1] = (grid32[0] >> 4) & 0x0f0f0f0f;
-    grid32[0] &= 0x0f0f0f0f;
-    for (int j = 0; j < 8; ++j) {
-        y[j] = ggml_cuda_cast<dst_t>(d * (q[j] + delta));
-    }
+    dequantize_iq1_s(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_iq1_m(const void * __restrict__ vx, dst_t * __restrict__ yy) {
+    const int64_t i = blockIdx.x;
 
-    const int64_t i   = blockIdx.x;
-    const block_iq1_m * x = (const block_iq1_m  *) vx;
-
-    const int64_t tid = threadIdx.x;
-    const int64_t il = tid/8; // 0...3
-    const int64_t ib = tid%8; // 0...7
-    dst_t * y = yy + i*QK_K + 32*ib + 8*il;
-    const uint16_t * sc = (const uint16_t *)x[i].scales;
-    iq1m_scale_t scale;
-    scale.u16 = (sc[0] >> 12) | ((sc[1] >> 8) & 0x00f0) | ((sc[2] >> 4) & 0x0f00) | (sc[3] & 0xf000);
-    const int64_t ib16 = 2*ib + il/2; // sc[ib16/4] >> 3*(ib16%4) -> sc[ib/2] >> 3*((2*ib+il/2)%4);
-    const float d = (float)scale.f16 * (2*((sc[ib16/4] >> 3*(ib16%4)) & 0x7) + 1);
-    const float delta = x[i].qh[2*ib+il/2] & (0x08 << 4*(il%2)) ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA;
-    uint32_t grid32[2]; const int8_t * q = (const int8_t *)grid32;
-    grid32[0] = iq1s_grid_gpu[x[i].qs[4*ib+il] | (((x[i].qh[2*ib+il/2] >> 4*(il%2)) & 7) << 8)];
-    grid32[1] = (grid32[0] >> 4) & 0x0f0f0f0f;
-    grid32[0] &= 0x0f0f0f0f;
-    for (int j = 0; j < 8; ++j) {
-        y[j] = ggml_cuda_cast<dst_t>(d * (q[j] + delta));
-    }
+    dequantize_iq1_m(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_iq4_nl(const void * __restrict__ vx, dst_t * __restrict__ yy) {
+    const int64_t i = blockIdx.x;
 
-    const int64_t i   = blockIdx.x;
-    const block_iq4_nl * x = (const block_iq4_nl *) vx + i*(QK_K/QK4_NL);
-
-    const int64_t tid = threadIdx.x;
-    const int64_t il = tid/8; // 0...3
-    const int64_t ib = tid%8; // 0...7
-    dst_t * y = yy + i*QK_K + 32*ib + 4*il;
-    const uint8_t  * q4 = x[ib].qs + 4*il;
-    const float d = (float)x[ib].d;
-    for (int j = 0; j < 4; ++j) {
-        y[j+ 0] = ggml_cuda_cast<dst_t>(d * kvalues_iq4nl[q4[j] & 0xf]);
-        y[j+16] = ggml_cuda_cast<dst_t>(d * kvalues_iq4nl[q4[j] >>  4]);
-    }
+    dequantize_iq4_nl(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_iq4_xs(const void * __restrict__ vx, dst_t * __restrict__ yy) {
-    const int64_t i   = blockIdx.x;
-    const block_iq4_xs * x = (const block_iq4_xs *)vx;
+    const int64_t i = blockIdx.x;
 
-    const int64_t tid = threadIdx.x;
-    const int64_t il = tid/8; // 0...3
-    const int64_t ib = tid%8; // 0...7
-    dst_t * y = yy + i*QK_K + 32*ib + 4*il;
-    const uint8_t  * q4 = x[i].qs + 16*ib + 4*il;
-    const float d = (float)x[i].d * ((((x[i].scales_l[ib/2] >> 4*(ib%2)) & 0xf) | (((x[i].scales_h >> 2*ib) & 3) << 4)) - 32);
-    for (int j = 0; j < 4; ++j) {
-        y[j+ 0] = ggml_cuda_cast<dst_t>(d * kvalues_iq4nl[q4[j] & 0xf]);
-        y[j+16] = ggml_cuda_cast<dst_t>(d * kvalues_iq4nl[q4[j] >>  4]);
-    }
+    dequantize_iq4_xs(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template<typename dst_t>
 static __global__ void dequantize_block_mxfp4(const void * __restrict__ vx, dst_t * __restrict__ yy) {
+    const int64_t i = blockIdx.x;
 
-    const int64_t i   = blockIdx.x;
-    const block_mxfp4 * x = (const block_mxfp4 *) vx + i*(QK_K/QK_MXFP4);
-
-    const int64_t tid = threadIdx.x;
-    const int64_t il = tid/8; // 0...3
-    const int64_t ib = tid%8; // 0...7
-    dst_t * y = yy + i*QK_K + 32*ib + 4*il;
-    const uint8_t  * q4 = x[ib].qs + 4*il;
-    const float d = ggml_cuda_e8m0_to_fp32(x[ib].e);
-    for (int j = 0; j < 4; ++j) {
-        y[j+ 0] = ggml_cuda_cast<dst_t>(d * kvalues_mxfp4[q4[j] & 0xf]*0.5f);
-        y[j+16] = ggml_cuda_cast<dst_t>(d * kvalues_mxfp4[q4[j] >>  4]*0.5f);
-    }
+    dequantize_mxfp4(vx, i, yy + i*QK_K, threadIdx.x);
 }
 
 template <int qk, int qr, dequantize_kernel_t dequantize_kernel, typename dst_t>

ggml/src/ggml-cuda/dequantize.cuh

@@ -1,4 +1,5 @@
 #include "common.cuh"
+#include "convert.cuh"
 
 static __device__ __forceinline__ void dequantize_q1_0(const void * vx, const int64_t ib, const int iqs, float2 & v){
     const block_q1_0 * x = (const block_q1_0 *) vx;
@@ -97,3 +98,335 @@ static __device__ __forceinline__ void dequantize_q8_0(const void * vx, const in
     v.x *= d;
     v.y *= d;
 }
+
+//================================== k-quants
+
+// Each call dequantizes one super-block of QK_K values into y using the
+// thread layout of the caller: 32 threads for q4_K, 64 threads otherwise.
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_q2_K(const void * vx, const int64_t ib, dst_t * yy, const int tid) {
+    const block_q2_K * x = (const block_q2_K *) vx;
+
+    const int64_t n   = tid/32;
+    const int64_t l   = tid - 32*n;
+    const int64_t is  = 8*n + l/16;
+
+    const uint8_t q = x[ib].qs[32*n + l];
+    dst_t * y = yy + 128*n;
+
+    float dall = __low2half(x[ib].dm);
+    float dmin = __high2half(x[ib].dm);
+    y[l+ 0] = ggml_cuda_cast<dst_t>(dall * (x[ib].scales[is+0] & 0xF) * ((q >> 0) & 3) - dmin * (x[ib].scales[is+0] >> 4));
+    y[l+32] = ggml_cuda_cast<dst_t>(dall * (x[ib].scales[is+2] & 0xF) * ((q >> 2) & 3) - dmin * (x[ib].scales[is+2] >> 4));
+    y[l+64] = ggml_cuda_cast<dst_t>(dall * (x[ib].scales[is+4] & 0xF) * ((q >> 4) & 3) - dmin * (x[ib].scales[is+4] >> 4));
+    y[l+96] = ggml_cuda_cast<dst_t>(dall * (x[ib].scales[is+6] & 0xF) * ((q >> 6) & 3) - dmin * (x[ib].scales[is+6] >> 4));
+}
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_q3_K(const void * vx, const int64_t ib, dst_t * yy, const int tid) {
+    const block_q3_K * x = (const block_q3_K *) vx;
+
+    const int64_t r = tid/4;
+    const int64_t t = r/2;
+    const int64_t is0 = r%2;
+    const int64_t l0 = 16*is0 + 4*(tid%4);
+    const int64_t n = t / 4;
+    const int64_t j = t - 4*n;
+
+    uint8_t m = 1 << (4*n + j);
+    int64_t is = 8*n + 2*j + is0;
+    int shift = 2*j;
+
+    int8_t us = is <  4 ? (x[ib].scales[is-0] & 0xF) | (((x[ib].scales[is+8] >> 0) & 3) << 4) :
+                is <  8 ? (x[ib].scales[is-0] & 0xF) | (((x[ib].scales[is+4] >> 2) & 3) << 4) :
+                is < 12 ? (x[ib].scales[is-8] >>  4) | (((x[ib].scales[is+0] >> 4) & 3) << 4) :
+                          (x[ib].scales[is-8] >>  4) | (((x[ib].scales[is-4] >> 6) & 3) << 4);
+    float d_all = x[ib].d;
+    float dl = d_all * (us - 32);
+
+    dst_t * y = yy + 128*n + 32*j;
+    const uint8_t * q = x[ib].qs + 32*n;
+    const uint8_t * hm = x[ib].hmask;
+
+    for (int l = l0; l < l0+4; ++l) {
+        y[l] = ggml_cuda_cast<dst_t>(dl * ((int8_t)((q[l] >> shift) & 3) - ((hm[l] & m) ? 0 : 4)));
+    }
+}
+
+static inline __device__ void get_scale_min_k4(int j, const uint8_t * q, uint8_t & d, uint8_t & m) {
+    if (j < 4) {
+        d = q[j] & 63; m = q[j + 4] & 63;
+    } else {
+        d = (q[j+4] & 0xF) | ((q[j-4] >> 6) << 4);
+        m = (q[j+4] >>  4) | ((q[j-0] >> 6) << 4);
+    }
+}
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_q4_K(const void * vx, const int64_t ib, dst_t * yy, const int tid) {
+    const block_q4_K * x = (const block_q4_K *) vx;
+
+    // assume 32 threads
+    const int64_t il  = tid/8;
+    const int64_t ir  = tid%8;
+    const int64_t is  = 2*il;
+    const int64_t n   = 4;
+
+    dst_t * y = yy + 64*il + n*ir;
+
+    const float dall = __low2half(x[ib].dm);
+    const float dmin = __high2half(x[ib].dm);
+
+    const uint8_t * q = x[ib].qs + 32*il + n*ir;
+
+    uint8_t sc, m;
+    get_scale_min_k4(is + 0, x[ib].scales, sc, m);
+    const float d1 = dall * sc; const float m1 = dmin * m;
+    get_scale_min_k4(is + 1, x[ib].scales, sc, m);
+    const float d2 = dall * sc; const float m2 = dmin * m;
+    for (int l = 0; l < n; ++l) {
+        y[l + 0] = ggml_cuda_cast<dst_t>(d1 * (q[l] & 0xF) - m1);
+        y[l +32] = ggml_cuda_cast<dst_t>(d2 * (q[l] >>  4) - m2);
+    }
+}
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_q5_K(const void * vx, const int64_t ib, dst_t * yy, const int tid) {
+    const block_q5_K * x = (const block_q5_K *) vx;
+
+    // assume 64 threads - this is very slightly better than the one below
+    const int64_t il  = tid/16;   // il is in 0...3
+    const int64_t ir  = tid%16;   // ir is in 0...15
+    const int64_t is  = 2*il;     // is is in 0...6
+
+    dst_t * y = yy + 64*il + 2*ir;
+
+    const float dall = __low2half(x[ib].dm);
+    const float dmin = __high2half(x[ib].dm);
+
+    const uint8_t * ql = x[ib].qs + 32*il + 2*ir;
+    const uint8_t * qh = x[ib].qh + 2*ir;
+
+    uint8_t sc, m;
+    get_scale_min_k4(is + 0, x[ib].scales, sc, m);
+    const float d1 = dall * sc; const float m1 = dmin * m;
+    get_scale_min_k4(is + 1, x[ib].scales, sc, m);
+    const float d2 = dall * sc; const float m2 = dmin * m;
+
+    uint8_t   hm  = 1 << (2*il);
+    y[ 0] = ggml_cuda_cast<dst_t>(d1 * ((ql[ 0] & 0xF) + (qh[ 0] & hm ? 16 : 0)) - m1);
+    y[ 1] = ggml_cuda_cast<dst_t>(d1 * ((ql[ 1] & 0xF) + (qh[ 1] & hm ? 16 : 0)) - m1);
+    hm <<= 1;
+    y[32] = ggml_cuda_cast<dst_t>(d2 * ((ql[ 0] >>  4) + (qh[ 0] & hm ? 16 : 0)) - m2);
+    y[33] = ggml_cuda_cast<dst_t>(d2 * ((ql[ 1] >>  4) + (qh[ 1] & hm ? 16 : 0)) - m2);
+}
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_q6_K(const void * vx, const int64_t ib, dst_t * yy, const int tid) {
+    const block_q6_K * x = (const block_q6_K *) vx;
+
+    // assume 64 threads - this is very slightly better than the one below
+    const int64_t ip  = tid/32;   // ip is 0 or 1
+    const int64_t il  = tid - 32*ip; // 0...32
+    const int64_t is  = 8*ip + il/16;
+
+    dst_t * y = yy + 128*ip + il;
+
+    const float d = x[ib].d;
+
+    const uint8_t * ql = x[ib].ql + 64*ip + il;
+    const uint8_t   qh = x[ib].qh[32*ip + il];
+    const int8_t  * sc = x[ib].scales + is;
+
+    y[ 0] = ggml_cuda_cast<dst_t>(d * sc[0] * ((int8_t)((ql[ 0] & 0xF) | (((qh >> 0) & 3) << 4)) - 32));
+    y[32] = ggml_cuda_cast<dst_t>(d * sc[2] * ((int8_t)((ql[32] & 0xF) | (((qh >> 2) & 3) << 4)) - 32));
+    y[64] = ggml_cuda_cast<dst_t>(d * sc[4] * ((int8_t)((ql[ 0]  >> 4) | (((qh >> 4) & 3) << 4)) - 32));
+    y[96] = ggml_cuda_cast<dst_t>(d * sc[6] * ((int8_t)((ql[32]  >> 4) | (((qh >> 6) & 3) << 4)) - 32));
+}
+
+//================================== i-quants
+
+// Each call dequantizes one super-block of QK_K values into y with 32
+// threads; iq4_nl packs QK_K/QK4_NL sub-blocks per super-block.
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_iq2_xxs(const void * vx, const int64_t ibs, dst_t * yy, const int tid) {
+
+    const block_iq2_xxs * x = (const block_iq2_xxs  *) vx;
+
+    const int64_t il = tid/8; // 0...3
+    const int64_t ib = tid%8; // 0...7
+    dst_t * y = yy + 32*ib + 8*il;
+    const uint16_t * q2 = x[ibs].qs + 4*ib;
+    const uint8_t  * aux8 = (const uint8_t *)q2;
+    const uint8_t  * grid = (const uint8_t *)(iq2xxs_grid + aux8[il]);
+    const uint32_t aux32 = q2[2] | (q2[3] << 16);
+    const float d = (float)x[ibs].d * (0.5f + (aux32 >> 28)) * 0.25f;
+    const uint8_t signs = ksigns_iq2xs[(aux32 >> 7*il) & 127];
+    for (int j = 0; j < 8; ++j) {
+        y[j] = ggml_cuda_cast<dst_t>(d * grid[j] * (signs & kmask_iq2xs[j] ? -1.f : 1.f));
+    }
+}
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_iq2_xs(const void * vx, const int64_t ibs, dst_t * yy, const int tid) {
+
+    const block_iq2_xs * x = (const block_iq2_xs *) vx;
+
+    const int64_t il = tid/8; // 0...3
+    const int64_t ib = tid%8; // 0...7
+    dst_t * y = yy + 32*ib + 8*il;
+    const uint16_t * q2 = x[ibs].qs + 4*ib;
+    const uint8_t  * grid = (const uint8_t *)(iq2xs_grid + (q2[il] & 511));
+    const float d = (float)x[ibs].d * (0.5f + ((x[ibs].scales[ib] >> 4*(il/2)) & 0xf)) * 0.25f;
+    const uint8_t signs = ksigns_iq2xs[q2[il] >> 9];
+    for (int j = 0; j < 8; ++j) {
+        y[j] = ggml_cuda_cast<dst_t>(d * grid[j] * (signs & kmask_iq2xs[j] ? -1.f : 1.f));
+    }
+}
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_iq2_s(const void * vx, const int64_t ibs, dst_t * yy, const int tid) {
+
+    const block_iq2_s * x = (const block_iq2_s *) vx;
+
+    const int64_t il = tid/8; // 0...3
+    const int64_t ib = tid%8; // 0...7
+    dst_t * y = yy + 32*ib + 8*il;
+    const uint8_t * grid = (const uint8_t *)(iq2s_grid + (x[ibs].qs[4*ib+il] | ((x[ibs].qh[ib] << (8-2*il)) & 0x300)));
+    const float d = (float)x[ibs].d * (0.5f + ((x[ibs].scales[ib] >> 4*(il/2)) & 0xf)) * 0.25f;
+    const uint8_t signs = x[ibs].qs[QK_K/8+4*ib+il];
+    for (int j = 0; j < 8; ++j) {
+        y[j] = ggml_cuda_cast<dst_t>(d * grid[j] * (signs & kmask_iq2xs[j] ? -1.f : 1.f));
+    }
+}
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_iq3_xxs(const void * vx, const int64_t ibs, dst_t * yy, const int tid) {
+
+    const block_iq3_xxs * x = (const block_iq3_xxs  *) vx;
+
+    const int64_t il = tid/8; // 0...3
+    const int64_t ib = tid%8; // 0...7
+    dst_t * y = yy + 32*ib + 8*il;
+    const uint8_t  * q3 = x[ibs].qs + 8*ib;
+    const uint16_t * gas = (const uint16_t *)(x[ibs].qs + QK_K/4) + 2*ib;
+    const uint8_t  * grid1 = (const uint8_t *)(iq3xxs_grid + q3[2*il+0]);
+    const uint8_t  * grid2 = (const uint8_t *)(iq3xxs_grid + q3[2*il+1]);
+    const uint32_t aux32 = gas[0] | (gas[1] << 16);
+    const float d = (float)x[ibs].d * (0.5f + (aux32 >> 28)) * 0.5f;
+    const uint8_t signs = ksigns_iq2xs[(aux32 >> 7*il) & 127];
+    for (int j = 0; j < 4; ++j) {
+        y[j+0] = ggml_cuda_cast<dst_t>(d * grid1[j] * (signs & kmask_iq2xs[j+0] ? -1.f : 1.f));
+        y[j+4] = ggml_cuda_cast<dst_t>(d * grid2[j] * (signs & kmask_iq2xs[j+4] ? -1.f : 1.f));
+    }
+}
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_iq3_s(const void * vx, const int64_t ibs, dst_t * yy, const int tid) {
+
+    const block_iq3_s * x = (const block_iq3_s *) vx;
+
+    const int64_t il = tid/8; // 0...3
+    const int64_t ib = tid%8; // 0...7
+    dst_t * y = yy + 32*ib + 8*il;
+    const uint8_t * qs = x[ibs].qs + 8*ib;
+    const uint8_t * grid1 = (const uint8_t *)(iq3s_grid + (qs[2*il+0] | ((x[ibs].qh[ib] << (8-2*il)) & 256)));
+    const uint8_t * grid2 = (const uint8_t *)(iq3s_grid + (qs[2*il+1] | ((x[ibs].qh[ib] << (7-2*il)) & 256)));
+    const float d = (float)x[ibs].d * (1 + 2*((x[ibs].scales[ib/2] >> 4*(ib%2)) & 0xf));
+    const uint8_t signs = x[ibs].signs[4*ib + il];
+    for (int j = 0; j < 4; ++j) {
+        y[j+0] = ggml_cuda_cast<dst_t>(d * grid1[j] * (signs & kmask_iq2xs[j+0] ? -1.f : 1.f));
+        y[j+4] = ggml_cuda_cast<dst_t>(d * grid2[j] * (signs & kmask_iq2xs[j+4] ? -1.f : 1.f));
+    }
+}
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_iq1_s(const void * vx, const int64_t ibs, dst_t * yy, const int tid) {
+
+    const block_iq1_s * x = (const block_iq1_s  *) vx;
+
+    const int64_t il = tid/8; // 0...3
+    const int64_t ib = tid%8; // 0...7
+    dst_t * y = yy + 32*ib + 8*il;
+    const float delta = x[ibs].qh[ib] & 0x8000 ? -1 - IQ1S_DELTA : -1 + IQ1S_DELTA;
+    const float d = (float)x[ibs].d * (2*((x[ibs].qh[ib] >> 12) & 7) + 1);
+    uint32_t grid32[2]; const int8_t * q = (const int8_t *)grid32;
+    grid32[0] = iq1s_grid_gpu[x[ibs].qs[4*ib+il] | (((x[ibs].qh[ib] >> 3*il) & 7) << 8)];
+    grid32[1] = (grid32[0] >> 4) & 0x0f0f0f0f;
+    grid32[0] &= 0x0f0f0f0f;
+    for (int j = 0; j < 8; ++j) {
+        y[j] = ggml_cuda_cast<dst_t>(d * (q[j] + delta));
+    }
+}
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_iq1_m(const void * vx, const int64_t ibs, dst_t * yy, const int tid) {
+
+    const block_iq1_m * x = (const block_iq1_m  *) vx;
+
+    const int64_t il = tid/8; // 0...3
+    const int64_t ib = tid%8; // 0...7
+    dst_t * y = yy + 32*ib + 8*il;
+    const uint16_t * sc = (const uint16_t *)x[ibs].scales;
+    iq1m_scale_t scale;
+    scale.u16 = (sc[0] >> 12) | ((sc[1] >> 8) & 0x00f0) | ((sc[2] >> 4) & 0x0f00) | (sc[3] & 0xf000);
+    const int64_t ib16 = 2*ib + il/2; // sc[ib16/4] >> 3*(ib16%4) -> sc[ib/2] >> 3*((2*ib+il/2)%4);
+    const float d = (float)scale.f16 * (2*((sc[ib16/4] >> 3*(ib16%4)) & 0x7) + 1);
+    const float delta = x[ibs].qh[2*ib+il/2] & (0x08 << 4*(il%2)) ? -1 - IQ1M_DELTA : -1 + IQ1M_DELTA;
+    uint32_t grid32[2]; const int8_t * q = (const int8_t *)grid32;
+    grid32[0] = iq1s_grid_gpu[x[ibs].qs[4*ib+il] | (((x[ibs].qh[2*ib+il/2] >> 4*(il%2)) & 7) << 8)];
+    grid32[1] = (grid32[0] >> 4) & 0x0f0f0f0f;
+    grid32[0] &= 0x0f0f0f0f;
+    for (int j = 0; j < 8; ++j) {
+        y[j] = ggml_cuda_cast<dst_t>(d * (q[j] + delta));
+    }
+}
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_iq4_nl(const void * vx, const int64_t ibs, dst_t * yy, const int tid) {
+
+    const block_iq4_nl * x = (const block_iq4_nl *) vx + ibs*(QK_K/QK4_NL);
+
+    const int64_t il = tid/8; // 0...3
+    const int64_t ib = tid%8; // 0...7
+    dst_t * y = yy + 32*ib + 4*il;
+    const uint8_t  * q4 = x[ib].qs + 4*il;
+    const float d = (float)x[ib].d;
+    for (int j = 0; j < 4; ++j) {
+        y[j+ 0] = ggml_cuda_cast<dst_t>(d * kvalues_iq4nl[q4[j] & 0xf]);
+        y[j+16] = ggml_cuda_cast<dst_t>(d * kvalues_iq4nl[q4[j] >>  4]);
+    }
+}
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_iq4_xs(const void * vx, const int64_t ibs, dst_t * yy, const int tid) {
+    const block_iq4_xs * x = (const block_iq4_xs *)vx;
+
+    const int64_t il = tid/8; // 0...3
+    const int64_t ib = tid%8; // 0...7
+    dst_t * y = yy + 32*ib + 4*il;
+    const uint8_t  * q4 = x[ibs].qs + 16*ib + 4*il;
+    const float d = (float)x[ibs].d * ((((x[ibs].scales_l[ib/2] >> 4*(ib%2)) & 0xf) | (((x[ibs].scales_h >> 2*ib) & 3) << 4)) - 32);
+    for (int j = 0; j < 4; ++j) {
+        y[j+ 0] = ggml_cuda_cast<dst_t>(d * kvalues_iq4nl[q4[j] & 0xf]);
+        y[j+16] = ggml_cuda_cast<dst_t>(d * kvalues_iq4nl[q4[j] >>  4]);
+    }
+}
+
+template<typename dst_t>
+static __device__ __forceinline__ void dequantize_mxfp4(const void * vx, const int64_t ibs, dst_t * yy, const int tid) {
+
+    const block_mxfp4 * x = (const block_mxfp4 *) vx + ibs*(QK_K/QK_MXFP4);
+
+    const int64_t il = tid/8; // 0...3
+    const int64_t ib = tid%8; // 0...7
+    dst_t * y = yy + 32*ib + 4*il;
+    const uint8_t  * q4 = x[ib].qs + 4*il;
+    const float d = ggml_cuda_e8m0_to_fp32(x[ib].e);
+    for (int j = 0; j < 4; ++j) {
+        y[j+ 0] = ggml_cuda_cast<dst_t>(d * kvalues_mxfp4[q4[j] & 0xf]*0.5f);
+        y[j+16] = ggml_cuda_cast<dst_t>(d * kvalues_mxfp4[q4[j] >>  4]*0.5f);
+    }
+}

ggml/src/ggml-cuda/getrows.cu

@@ -40,6 +40,35 @@ static __global__ void k_get_rows(
     }
 }
 
+template<typename dst_t, dequantize_kq_t<dst_t> dequantize_kq>
+static __global__ void k_get_rows_kq(
+        const void * __restrict__ src0, const int32_t * __restrict__ src1, dst_t * __restrict__ dst,
+        const int64_t ne00, /*const int64_t ne01, const int64_t ne02, const int64_t ne03,*/
+        /*const int64_t ne10,*/ const int64_t ne11, const uint3 ne12_fdv, /*const int64_t ne13,*/
+        /*const size_t s0,*/ const size_t s1, const size_t s2, const size_t s3,
+        /*const size_t nb00,*/ const size_t nb01, const size_t nb02, const size_t nb03,
+        const size_t s10, const size_t s11, const size_t s12/*, const size_t s13*/) {
+
+    ggml_cuda_pdl_sync();
+    const int64_t nsb = ne00/QK_K; // super-blocks per row
+    for (int64_t z = blockIdx.z; z < ne11*(int64_t)ne12_fdv.z; z += gridDim.z) {
+        // The x and y dimensions of the grid are swapped because the maximum allowed grid size for x is higher.
+        const int i10 = blockIdx.x;
+        const uint2 dm  = fast_div_modulo((uint32_t)z, ne12_fdv);
+        const int i11 = dm.x;
+        const int i12 = dm.y;
+
+        const int i01 = src1[i10*s10 + i11*s11 + i12*s12];
+
+        dst_t * dst_row = dst + i10*s1 + i11*s2 + i12*s3;
+        const void * src0_row = (const char *) src0 + i01*nb01 + i11*nb02 + i12*nb03;
+
+        for (int64_t ib = blockIdx.y; ib < nsb; ib += gridDim.y) {
+            dequantize_kq(src0_row, ib, dst_row + ib*QK_K, threadIdx.x);
+        }
+    }
+}
+
 template<typename src0_t, typename dst_t>
 static __global__ void k_get_rows_float(
         const src0_t * src0_ptr, const int32_t * src1_ptr, dst_t * dst_ptr,
@@ -55,27 +84,51 @@ static __global__ void k_get_rows_float(
     dst_t         * GGML_CUDA_RESTRICT dst  = dst_ptr;
     ggml_cuda_pdl_sync();
     for (int64_t z = blockIdx.z; z < ne11*(int64_t)ne12_fdv.z; z += gridDim.z) {
+        // The x and y dimensions of the grid are swapped because the maximum allowed grid size for x is higher.
+        const int i10 = blockIdx.x;
+        const uint2 dm = fast_div_modulo((uint32_t)z, ne12_fdv);
+        const int i11 = dm.x;
+        const int i12 = dm.y;
+
+        const int i01 = src1[i10*s10 + i11*s11 + i12*s12];
+
+        dst_t * GGML_CUDA_RESTRICT dst_row = dst + i10*s1 + i11*s2 + i12*s3;
+        const src0_t * GGML_CUDA_RESTRICT src0_row = (const src0_t *)((const char *) src0 + i01*nb01 + i11*nb02 + i12*nb03);
+
         for (int64_t i00 = blockIdx.y*blockDim.x + threadIdx.x; i00 < ne00; i00 += gridDim.y*blockDim.x) {
-            // The x and y dimensions of the grid are swapped because the maximum allowed grid size for x is higher.
-            const int i10 = blockIdx.x;
-            const uint2 dm = fast_div_modulo((uint32_t)z, ne12_fdv);
-            const int i11 = dm.x;
-            const int i12 = dm.y;
-
-            if (i00 >= ne00) {
-                return;
-            }
-
-            const int i01 = src1[i10*s10 + i11*s11 + i12*s12];
-
-            dst_t * dst_row = dst + i10*s1 + i11*s2 + i12*s3;
-            const src0_t * src0_row = (const src0_t *)((const char *) src0 + i01*nb01 + i11*nb02 + i12*nb03);
-
             dst_row[i00] = ggml_cuda_cast<dst_t>(src0_row[i00]);
         }
     }
 }
 
+template<typename dst_t>
+static __global__ void k_get_rows_float_vec(
+        const dst_t * src0_ptr, const int32_t * src1_ptr, dst_t * dst_ptr,
+        const int64_t ne00v,
+        const int64_t ne11, const uint3 ne12_fdv,
+        const size_t s1, const size_t s2, const size_t s3,
+        const size_t nb01, const size_t nb02, const size_t nb03,
+        const size_t s10, const size_t s11, const size_t s12) {
+
+    ggml_cuda_pdl_lc();
+    ggml_cuda_pdl_sync();
+    for (int64_t z = blockIdx.z; z < ne11*(int64_t)ne12_fdv.z; z += gridDim.z) {
+        const int i10 = blockIdx.x;
+        const uint2 dm = fast_div_modulo((uint32_t)z, ne12_fdv);
+        const int i11 = dm.x;
+        const int i12 = dm.y;
+
+        const int i01 = src1_ptr[i10*s10 + i11*s11 + i12*s12];
+
+        int4       * GGML_CUDA_RESTRICT dst_row  = (int4 *)      (dst_ptr + i10*s1 + i11*s2 + i12*s3);
+        const int4 * GGML_CUDA_RESTRICT src0_row = (const int4 *)((const char *) src0_ptr + i01*nb01 + i11*nb02 + i12*nb03);
+
+        for (int64_t i = blockIdx.y*blockDim.x + threadIdx.x; i < ne00v; i += gridDim.y*blockDim.x) {
+            dst_row[i] = src0_row[i];
+        }
+    }
+}
+
 template<typename grad_t, typename dst_t>
 static __global__ void k_get_rows_back_float(
         const grad_t * __restrict__ grad, const int32_t * __restrict__ rows, dst_t * __restrict__ dst,
@@ -140,16 +193,18 @@ static void get_rows_cuda_q(
         s10, s11, s12/*, s13*/);
 }
 
-template<typename src0_t, typename dst_t>
-static void get_rows_cuda_float(
-        const src0_t * src0_d, const int32_t * src1_d, dst_t * dst_d,
+template<int block_dim, typename dst_t, dequantize_kq_t<dst_t> dequantize_kq>
+static void get_rows_cuda_kq(
+        const void * src0_d, const int32_t * src1_d, dst_t * dst_d,
         const int64_t ne00, const size_t nb01, const size_t nb02, const size_t nb03,
         const int64_t ne10, const int64_t ne11, const int64_t ne12, const size_t nb10, const size_t nb11, const size_t nb12,
         const size_t nb1, const size_t nb2, const size_t nb3,
         cudaStream_t stream) {
-    const dim3 block_dims(CUDA_GET_ROWS_BLOCK_SIZE, 1, 1);
-    const int block_num_y = (ne00 + CUDA_GET_ROWS_BLOCK_SIZE - 1) / CUDA_GET_ROWS_BLOCK_SIZE;
-    const dim3 block_nums(ne10, MIN(block_num_y, UINT16_MAX), MIN(ne11*ne12, UINT16_MAX));
+    GGML_ASSERT(ne00 % QK_K == 0);
+    const int64_t nsb = ne00/QK_K;
+
+    const dim3 block_dims(block_dim, 1, 1);
+    const dim3 block_nums(ne10, MIN(nsb, UINT16_MAX), MIN(ne11*ne12, UINT16_MAX));
 
     // strides in elements
     // const size_t s0 = nb0 / sizeof(dst_t);
@@ -166,6 +221,67 @@ static void get_rows_cuda_float(
     GGML_ASSERT(ne11 <= std::numeric_limits<uint32_t>::max() / ne12);
     const uint3 ne12_fdv = init_fastdiv_values(ne12);
 
+    k_get_rows_kq<dst_t, dequantize_kq><<<block_nums, block_dims, 0, stream>>>(
+        src0_d, src1_d, dst_d,
+        ne00, /*ne01, ne02, ne03,*/
+        /*ne10,*/ ne11, ne12_fdv, /*ne13,*/
+        /* s0,*/ s1, s2, s3,
+        /* nb00,*/ nb01, nb02, nb03,
+        s10, s11, s12/*, s13*/);
+}
+
+template<typename src0_t, typename dst_t>
+static void get_rows_cuda_float(
+        const src0_t * src0_d, const int32_t * src1_d, dst_t * dst_d,
+        const int64_t ne00, const size_t nb01, const size_t nb02, const size_t nb03,
+        const int64_t ne10, const int64_t ne11, const int64_t ne12, const size_t nb10, const size_t nb11, const size_t nb12,
+        const size_t nb1, const size_t nb2, const size_t nb3,
+        cudaStream_t stream) {
+    const dim3 block_dims(CUDA_GET_ROWS_BLOCK_SIZE, 1, 1);
+
+    // strides in elements
+    // const size_t s0 = nb0 / sizeof(dst_t);
+    const size_t s1 = nb1 / sizeof(dst_t);
+    const size_t s2 = nb2 / sizeof(dst_t);
+    const size_t s3 = nb3 / sizeof(dst_t);
+
+    const size_t s10 = nb10 / sizeof(int32_t);
+    const size_t s11 = nb11 / sizeof(int32_t);
+    const size_t s12 = nb12 / sizeof(int32_t);
+    // const size_t s13 = nb13 / sizeof(int32_t);
+
+    GGML_ASSERT(ne12 > 0);
+    GGML_ASSERT(ne11 <= std::numeric_limits<uint32_t>::max() / ne12);
+    const uint3 ne12_fdv = init_fastdiv_values(ne12);
+
+    if constexpr (std::is_same<src0_t, dst_t>::value) {
+        constexpr int VEC = 16 / sizeof(dst_t);
+        const int64_t ne00v = ne00 / VEC;
+        const int64_t vec_block_num_y = (ne00v + CUDA_GET_ROWS_BLOCK_SIZE - 1) / CUDA_GET_ROWS_BLOCK_SIZE;
+        const bool enough_blocks = vec_block_num_y * ne10 * ne11 * ne12 >= 128;
+        const bool can_vec = VEC > 1 && enough_blocks &&
+            (ne00 % VEC == 0) &&
+            (nb01 % 16 == 0) && (nb02 % 16 == 0) && (nb03 % 16 == 0) &&
+            (nb1  % 16 == 0) && (nb2  % 16 == 0) && (nb3  % 16 == 0) &&
+            (((uintptr_t) src0_d) % 16 == 0) && (((uintptr_t) dst_d) % 16 == 0);
+
+        if (can_vec) {
+            const int block_num_y = vec_block_num_y;
+            const dim3 block_nums(ne10, MIN(block_num_y, UINT16_MAX), MIN(ne11*ne12, UINT16_MAX));
+            const ggml_cuda_kernel_launch_params launch_params = ggml_cuda_kernel_launch_params{block_nums, block_dims, 0, stream};
+            ggml_cuda_kernel_launch(k_get_rows_float_vec<dst_t>, launch_params,
+                (const dst_t *) src0_d, src1_d, dst_d,
+                ne00v, ne11, ne12_fdv,
+                s1, s2, s3,
+                nb01, nb02, nb03,
+                s10, s11, s12);
+            return;
+        }
+    }
+
+    const int block_num_y = (ne00 + CUDA_GET_ROWS_BLOCK_SIZE - 1) / CUDA_GET_ROWS_BLOCK_SIZE;
+    const dim3 block_nums(ne10, MIN(block_num_y, UINT16_MAX), MIN(ne11*ne12, UINT16_MAX));
+
     const ggml_cuda_kernel_launch_params launch_params = ggml_cuda_kernel_launch_params{block_nums, block_dims, 0, stream};
     ggml_cuda_kernel_launch(k_get_rows_float<src0_t, dst_t>, launch_params,
         src0_d, src1_d, dst_d,
@@ -224,8 +340,67 @@ static void ggml_cuda_get_rows_switch_src0_type(
             get_rows_cuda_q<QK8_0, QR8_0, dequantize_q8_0>(src0_d, src1_d, dst_d,
                 ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
             break;
+        case GGML_TYPE_Q2_K:
+            get_rows_cuda_kq<64, dst_t, dequantize_q2_K<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_Q3_K:
+            get_rows_cuda_kq<64, dst_t, dequantize_q3_K<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_Q4_K:
+            get_rows_cuda_kq<32, dst_t, dequantize_q4_K<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_Q5_K:
+            get_rows_cuda_kq<64, dst_t, dequantize_q5_K<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_Q6_K:
+            get_rows_cuda_kq<64, dst_t, dequantize_q6_K<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_IQ2_XXS:
+            get_rows_cuda_kq<32, dst_t, dequantize_iq2_xxs<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_IQ2_XS:
+            get_rows_cuda_kq<32, dst_t, dequantize_iq2_xs<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_IQ2_S:
+            get_rows_cuda_kq<32, dst_t, dequantize_iq2_s<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_IQ3_XXS:
+            get_rows_cuda_kq<32, dst_t, dequantize_iq3_xxs<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_IQ3_S:
+            get_rows_cuda_kq<32, dst_t, dequantize_iq3_s<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_IQ1_S:
+            get_rows_cuda_kq<32, dst_t, dequantize_iq1_s<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_IQ1_M:
+            get_rows_cuda_kq<32, dst_t, dequantize_iq1_m<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_IQ4_NL:
+            get_rows_cuda_kq<32, dst_t, dequantize_iq4_nl<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_IQ4_XS:
+            get_rows_cuda_kq<32, dst_t, dequantize_iq4_xs<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
+        case GGML_TYPE_MXFP4:
+            get_rows_cuda_kq<32, dst_t, dequantize_mxfp4<dst_t>>(src0_d, src1_d, dst_d,
+                ne00, nb01, nb02, nb03, ne10, ne11, ne12, nb10, nb11, nb12, nb1, nb2, nb3, stream);
+            break;
         default:
-            // TODO: k-quants
             GGML_ABORT("%s: unsupported src0 type: %s\n", __func__, ggml_type_name(src0_type));
             break;
     }

ggml/src/ggml-cuda/ggml-cuda.cu

@@ -2703,6 +2703,7 @@ static int ggml_cuda_try_gdn_cache_fusion(
 
 static bool ggml_cuda_topk_moe_fusion(const struct ggml_cgraph * cgraph, int node_idx, ggml_cuda_topk_moe_args & args) {
     args.sigmoid         = false;
+    args.sqrt_softplus   = false;
     args.softmax         = false;
     args.delayed_softmax = false;
     args.prob_bias       = false;
@@ -2716,10 +2717,17 @@ static bool ggml_cuda_topk_moe_fusion(const struct ggml_cgraph * cgraph, int nod
     }
 
     if (nodes[node_idx]->op == GGML_OP_UNARY) {
-        if (ggml_get_unary_op(nodes[node_idx]) != GGML_UNARY_OP_SIGMOID) {
+        const ggml_unary_op unary_op = ggml_get_unary_op(nodes[node_idx]);
+        if (unary_op == GGML_UNARY_OP_SIGMOID) {
+            args.sigmoid = true;
+        } else if (unary_op == GGML_UNARY_OP_SOFTPLUS && node_idx + 1 < n_nodes &&
+                   nodes[node_idx + 1]->op == GGML_OP_SQRT && nodes[node_idx + 1]->src[0] == nodes[node_idx]) {
+            // sqrt(softplus(x)) scoring (DeepSeek-V4)
+            args.sqrt_softplus = true;
+            node_idx++;
+        } else {
             return false;
         }
-        args.sigmoid = true;
     }
 
     if (nodes[node_idx]->op == GGML_OP_ARGSORT) {
@@ -2728,7 +2736,7 @@ static bool ggml_cuda_topk_moe_fusion(const struct ggml_cgraph * cgraph, int nod
 
     node_idx++;
 
-    if (args.sigmoid || args.softmax) {
+    if (args.sigmoid || args.sqrt_softplus || args.softmax) {
         // SOFTMAX -> RESHAPE
         if (node_idx >= n_nodes || nodes[node_idx]->op != GGML_OP_RESHAPE ||
                 nodes[node_idx]->src[0] != nodes[node_idx - 1]) {
@@ -3172,21 +3180,27 @@ static int ggml_cuda_try_fuse(ggml_backend_cuda_context * cuda_ctx, ggml_cgraph
             const ggml_tensor * scale   = nullptr;
 
             if (!args.delayed_softmax) {
-                ggml_op gating_op = args.sigmoid ? GGML_OP_UNARY : GGML_OP_SOFT_MAX;
-                int     out_nodes[2];  // nodes which can't be elided
+                int out_nodes[2];  // nodes which can't be elided
+
+                if (args.sigmoid) {
+                    ops.insert(ops.end(), { GGML_OP_UNARY });
+                } else if (args.sqrt_softplus) {
+                    ops.insert(ops.end(), { GGML_OP_UNARY, GGML_OP_SQRT });
+                } else {
+                    ops.insert(ops.end(), { GGML_OP_SOFT_MAX });
+                }
+                const int i_probs = i + (int) ops.size() - 1;  // last node of the gating activation
 
                 if (args.prob_bias) {
-                    bias = cgraph->nodes[i + 2]->src[1];
-                    ops.insert(ops.end(), { gating_op, GGML_OP_RESHAPE, GGML_OP_ADD, GGML_OP_ARGSORT, GGML_OP_VIEW,
+                    bias = cgraph->nodes[i_probs + 2]->src[1];
+                    ops.insert(ops.end(), { GGML_OP_RESHAPE, GGML_OP_ADD, GGML_OP_ARGSORT, GGML_OP_VIEW,
                                             GGML_OP_GET_ROWS });
-                    out_nodes[0] = i + 4;
-                    ids          = cgraph->nodes[i + 4];
+                    out_nodes[0] = i_probs + 4;
                 } else {
-                    ops.insert(ops.end(),
-                               { gating_op, GGML_OP_RESHAPE, GGML_OP_ARGSORT, GGML_OP_VIEW, GGML_OP_GET_ROWS });
-                    out_nodes[0] = i + 3;
-                    ids          = cgraph->nodes[i + 3];
+                    ops.insert(ops.end(), { GGML_OP_RESHAPE, GGML_OP_ARGSORT, GGML_OP_VIEW, GGML_OP_GET_ROWS });
+                    out_nodes[0] = i_probs + 3;
                 }
+                ids = cgraph->nodes[out_nodes[0]];
 
                 if (args.norm) {
                     ops.insert(ops.end(),
@@ -4831,7 +4845,25 @@ static bool ggml_backend_cuda_device_supports_op(ggml_backend_dev_t dev, const g
                     case GGML_TYPE_Q5_0:
                     case GGML_TYPE_Q5_1:
                     case GGML_TYPE_Q8_0:
+                    case GGML_TYPE_Q2_K:
+                    case GGML_TYPE_Q3_K:
+                    case GGML_TYPE_Q4_K:
+                    case GGML_TYPE_Q5_K:
+                    case GGML_TYPE_Q6_K:
+                    case GGML_TYPE_IQ2_XXS:
+                    case GGML_TYPE_IQ2_XS:
+                    case GGML_TYPE_IQ2_S:
+                    case GGML_TYPE_IQ3_XXS:
+                    case GGML_TYPE_IQ3_S:
+                    case GGML_TYPE_IQ1_S:
+                    case GGML_TYPE_IQ1_M:
+                    case GGML_TYPE_IQ4_XS:
                         return true;
+                    case GGML_TYPE_IQ4_NL:
+                    case GGML_TYPE_MXFP4:
+                        // 32-value sub-blocks, the row size does not guarantee
+                        // the QK_K super-blocks the get_rows kernel iterates on
+                        return op->src[0]->ne[0] % QK_K == 0;
                     default:
                         return false;
                 }

ggml/src/ggml-cuda/mmq.cu

@@ -130,14 +130,20 @@ void ggml_cuda_mul_mat_q(
         const size_t nbytes_src1_q8_1 = ne13*ne12 * ne11*ne10_padded * y_block_size/y_values_per_block +
             ggml_cuda_mmq_get_J_max(src0->type, fallback, cc, ne11) * sizeof(block_q8_1_mmq);
         ggml_cuda_pool_alloc<char> src1_q8_1(ctx.pool(), nbytes_src1_q8_1);
+        ggml_cuda_pool_alloc<float> src1_scale(ctx.pool());
+        if (src0->type == GGML_TYPE_NVFP4 && use_native_fp4) {
+            src1_scale.alloc(ne13*ne12*ne11);
+        }
 
         {
             const int64_t s11 = src1->nb[1] / ts_src1;
             const int64_t s12 = src1->nb[2] / ts_src1;
             const int64_t s13 = src1->nb[3] / ts_src1;
             if (use_native_fp4) {
+                static constexpr size_t align_float8 = 32;
+                const bool use_aligned_float8 = ggml_cuda_is_aligned(src1, align_float8);
                 static_assert(sizeof(block_fp4_mmq) == 4 * sizeof(block_q8_1));
-                quantize_mmq_fp4_cuda(src1_d, nullptr, src1_q8_1.get(), src0->type, ne10, s11, s12, s13, ne10_padded,
+                quantize_mmq_fp4_cuda(src1_d, nullptr, src1_q8_1.get(), src1_scale.ptr, src0->type, use_aligned_float8, ne10, s11, s12, s13, ne10_padded,
                                         ne11, ne12, ne13, stream);
 
             } else {
@@ -155,6 +161,7 @@ void ggml_cuda_mul_mat_q(
 
         const mmq_args args = {
             src0_d, src0->type, (const int *) src1_q8_1.ptr, nullptr, nullptr, dst_d,
+            src0->type == GGML_TYPE_NVFP4 && use_native_fp4 ? src1_scale.ptr : nullptr,
             ne00, ne01, ne1, s01, ne11, s1,
             ne02, ne12, s02, s12, s2,
             ne03, ne13, s03, s13, s3,
@@ -192,6 +199,10 @@ void ggml_cuda_mul_mat_q(
     const size_t nbytes_src1_q8_1 = ne12*n_expert_used*ne10_padded * y_block_size/y_values_per_block +
         ggml_cuda_mmq_get_J_max(src0->type, fallback, cc, ne11) * sizeof(block_q8_1_mmq);
     ggml_cuda_pool_alloc<char> src1_q8_1(ctx.pool(), nbytes_src1_q8_1);
+    ggml_cuda_pool_alloc<float> src1_scale(ctx.pool());
+    if (src0->type == GGML_TYPE_NVFP4 && use_native_fp4) {
+        src1_scale.alloc(ne12*n_expert_used);
+    }
 
     const int64_t ne11_flat = ne12*n_expert_used;
     const int64_t ne12_flat = 1;
@@ -202,18 +213,19 @@ void ggml_cuda_mul_mat_q(
         const int64_t s12 = src1->nb[2] / ts_src1;
         const int64_t s13 = src1->nb[3] / ts_src1;
 
-        if (dedup_bcast) {
-            // quantize each token once, scatter its block to all n_expert_used slots
-            if (use_native_fp4) {
-                quantize_scatter_mmq_fp4_cuda(src1_d, ids_src1.get(), src1_q8_1.get(), src0->type, ne10,
+        if (use_native_fp4) {
+            static constexpr size_t align_float8 = 32;
+            const bool use_aligned_float8 = ggml_cuda_is_aligned(src1, align_float8);
+            if (dedup_bcast) {
+                quantize_scatter_mmq_fp4_cuda(src1_d, ids_src1.get(), src1_q8_1.get(), src1_scale.ptr, src0->type, use_aligned_float8, ne10,
                                         /*stride_token=*/s12, ne10_padded, ne12, ne11_flat, n_expert_used, stream);
             } else {
-                quantize_scatter_mmq_q8_1_cuda(src1_d, ids_src1.get(), src1_q8_1.get(), src0->type, ne10,
-                                        /*stride_token=*/s12, ne10_padded, ne12, ne11_flat, n_expert_used, stream);
+                quantize_mmq_fp4_cuda(src1_d, ids_src1.get(), src1_q8_1.get(), src1_scale.ptr, src0->type, use_aligned_float8, ne10, s11, s12, s13,
+                                        ne10_padded, ne11_flat, ne12_flat, ne13_flat, stream);
             }
-        } else if (use_native_fp4) {
-            quantize_mmq_fp4_cuda(src1_d, ids_src1.get(), src1_q8_1.get(), src0->type, ne10, s11, s12, s13,
-                                    ne10_padded, ne11_flat, ne12_flat, ne13_flat, stream);
+        } else if (dedup_bcast) {
+            quantize_scatter_mmq_q8_1_cuda(src1_d, ids_src1.get(), src1_q8_1.get(), src0->type, ne10,
+                                    /*stride_token=*/s12, ne10_padded, ne12, ne11_flat, n_expert_used, stream);
         } else {
             quantize_mmq_q8_1_cuda(src1_d, ids_src1.get(), src1_q8_1.get(), src0->type, ne10, s11, s12, s13,
                                    ne10_padded, ne11_flat, ne12_flat, ne13_flat, stream);
@@ -229,6 +241,7 @@ void ggml_cuda_mul_mat_q(
     // Note that ne02 is used instead of ne12 because the number of y channels determines the z dimension of the CUDA grid.
     const mmq_args args = {
         src0_d, src0->type, (const int *) src1_q8_1.get(), ids_dst.get(), expert_bounds.get(), dst_d,
+        src1_scale.ptr,
         ne00, ne01, ne_get_rows, s01, ne_get_rows, s1,
         ne02, ne02, s02, s12, s2,
         ne03, ne13, s03, s13, s3,

ggml/src/ggml-cuda/mmq.cuh

@@ -13,7 +13,7 @@
 typedef void (*ggml_cuda_mmq_load_tiles_t)(const char * __restrict__ x, int * x_tile, const int kbx0, const int i_max, const int stride);
 typedef void (*ggml_cuda_mmq_vec_dot_t)(const int * __restrict__ x, const int * __restrict__ y, float * __restrict__ sum, const int k00);
 typedef void (*ggml_cuda_mmq_write_back_t)(const float * __restrict__ sum, const int32_t * __restrict__ get_rows_to_sorted,
-    float * __restrict__ dst, const int stride, const int i_max, const int j_max);
+    float * __restrict__ dst, const float * __restrict__ y_scale, const int stride, const int i_max, const int j_max);
 
 enum mmq_q8_1_ds_layout {
     MMQ_Q8_1_DS_LAYOUT_D4,
@@ -413,11 +413,13 @@ static __host__ int ggml_cuda_mmq_get_nbytes_shared_x(const ggml_cuda_mmq_config
 
 template <ggml_type type, int J, bool fallback> static __device__ __forceinline__ void ggml_cuda_mmq_write_back_dp4a(
         const float * __restrict__ sum, const int32_t * __restrict__ ids_dst, float * __restrict__ dst,
-        const int stride, const int i_max, const int j_max) {
+        const float * __restrict__ y_scale, const int stride, const int i_max, const int j_max) {
     constexpr int warp_size = ggml_cuda_get_physical_warp_size();
     constexpr int nwarps    = ggml_cuda_mmq_get_nthreads(type, J, fallback) / warp_size;
     constexpr int I         = ggml_cuda_mmq_get_I(type, J, fallback);
 
+    const bool y_scale_used = y_scale != nullptr;
+
 #pragma unroll
     for (int j0 = 0; j0 < J; j0 += nwarps) {
         const int j = j0 + threadIdx.y;
@@ -434,7 +436,16 @@ template <ggml_type type, int J, bool fallback> static __device__ __forceinline_
                 continue;
             }
 
-            dst[ids_dst[j]*stride + i] = sum[(j0/nwarps) * (I/warp_size) + i0/warp_size];
+            if constexpr (type == GGML_TYPE_NVFP4) {
+                if (y_scale_used) {
+                    dst[ids_dst[j]*stride + i] = y_scale[j] * sum[(j0/nwarps) * (I/warp_size) + i0/warp_size];
+                } else {
+                    dst[ids_dst[j]*stride + i] = sum[(j0/nwarps) * (I/warp_size) + i0/warp_size];
+                }
+            } else {
+                dst[ids_dst[j]*stride + i] = sum[(j0/nwarps) * (I/warp_size) + i0/warp_size];
+                GGML_UNUSED(y_scale_used);
+            }
         }
     }
 }
@@ -442,7 +453,8 @@ template <ggml_type type, int J, bool fallback> static __device__ __forceinline_
 template<ggml_type type, int J, bool fallback>
 static __device__ __forceinline__ void ggml_cuda_mmq_write_back_mma(
             const float * __restrict__ sum, const int * __restrict__ ids_dst, float * __restrict__ dst,
-            const int stride, const int i_max, const int j_max) {
+            const float * __restrict__ y_scale, const int stride, const int i_max, const int j_max) {
+
 #if defined(AMD_MFMA_AVAILABLE) || defined(AMD_WMMA_AVAILABLE)
     typedef tile<16, 16, int, DATA_LAYOUT_J_MAJOR> tile_C;
 #else
@@ -457,6 +469,8 @@ static __device__ __forceinline__ void ggml_cuda_mmq_write_back_mma(
 
     const int i0 = (threadIdx.y / ntx) * (ntx*tile_C::I);
 
+    const bool y_scale_used = y_scale != nullptr;
+
 #pragma unroll
     for (int j0 = 0; j0 < J; j0 += ntx*tile_C::J) {
 #pragma unroll
@@ -475,7 +489,16 @@ static __device__ __forceinline__ void ggml_cuda_mmq_write_back_mma(
                     continue;
                 }
 
-                dst[ids_dst[j]*stride + i] = sum[(j0/tile_C::J + n)*tile_C::ne + l];
+                if constexpr (type == GGML_TYPE_NVFP4) {
+                    if (y_scale_used) {
+                        dst[ids_dst[j]*stride + i] = y_scale[j] * sum[(j0/tile_C::J + n)*tile_C::ne + l];
+                    } else {
+                        dst[ids_dst[j]*stride + i] = sum[(j0/tile_C::J + n)*tile_C::ne + l];
+                    }
+                } else {
+                    dst[ids_dst[j]*stride + i] = sum[(j0/tile_C::J + n)*tile_C::ne + l];
+                    GGML_UNUSED(y_scale_used);
+                }
             }
         }
     }
@@ -819,6 +842,7 @@ template <ggml_type type, int J, bool fallback, bool fixup>
 static __device__ __forceinline__ void mul_mat_q_process_tile(
         const char * __restrict__ x, const int offset_x, const int * __restrict__ y,
         const int * __restrict__ ids_dst, float * __restrict__ dst, float * __restrict__ tmp_fixup,
+        const float * __restrict__ y_scale,
         const int stride_row_x, const int ncols_y, const int stride_col_dst,
         const int tile_x_max_i, const int tile_y_max_j, const int kb0_start, const int kb0_stop) {
 
@@ -884,9 +908,9 @@ static __device__ __forceinline__ void mul_mat_q_process_tile(
     }
 
     if (fixup) {
-        write_back(sum, ids_dst, tmp_fixup + blockIdx.x*(J*I), I, I, J);
+        write_back(sum, ids_dst, tmp_fixup + blockIdx.x*(J*I), y_scale, I, I, J);
     } else {
-        write_back(sum, ids_dst, dst, stride_col_dst, tile_x_max_i, tile_y_max_j);
+        write_back(sum, ids_dst, dst, y_scale, stride_col_dst, tile_x_max_i, tile_y_max_j);
     }
 }
 
@@ -898,6 +922,7 @@ __launch_bounds__(ggml_cuda_mmq_get_nthreads(type, J, fallback), ggml_cuda_mmq_g
 static __global__ void mul_mat_q(
         const char * __restrict__ x, const int * __restrict__ y, const int32_t * __restrict__ ids_dst,
         const int32_t * __restrict__ expert_bounds, float * __restrict__ dst, float * __restrict__ tmp_fixup,
+        const float * __restrict__ y_scale,
         const uint3 blocks_per_ne00, const int nrows_x, const int ncols_dst, const int stride_row_x, const int ncols_y, const int stride_col_dst,
         const uint3 channel_ratio, const uint3 nchannels_y, const int stride_channel_x, const int stride_channel_y, const int stride_channel_dst,
         const uint3 sample_ratio, const uint3 nsamples_y, const int stride_sample_x, const int stride_sample_y, const int stride_sample_dst,
@@ -943,8 +968,14 @@ static __global__ void mul_mat_q(
         int col_low    = 0;
         int col_high   = ncols_dst;
         int col_diff   = ncols_dst;
-        int offset_y   = wt*stride_sample_y   + zt*stride_channel_y;
-        int offset_dst = wt*stride_sample_dst + zt*stride_channel_dst + jt*J*stride_col_dst;
+        int offset_y       = wt*stride_sample_y   + zt*stride_channel_y;
+        int offset_dst     = wt*stride_sample_dst + zt*stride_channel_dst + jt*J*stride_col_dst;
+        int offset_y_scale;
+        if constexpr (type == GGML_TYPE_NVFP4) {
+            offset_y_scale = wt*nchannels_y.z*ncols_y + zt*ncols_y;
+        } else {
+            GGML_UNUSED(offset_y_scale);
+        }
 
         if (ids_dst) {
             col_low  = expert_bounds[zt + 0];
@@ -953,6 +984,9 @@ static __global__ void mul_mat_q(
 
             offset_y   = 0;
             offset_dst = 0;
+            if constexpr (type == GGML_TYPE_NVFP4) {
+                offset_y_scale = 0;
+            }
 
             if (jt*J >= col_diff) {
                 return;
@@ -974,6 +1008,11 @@ static __global__ void mul_mat_q(
 
         offset_y   += (col_low + jt*J)*(sizeof(block_q8_1_mmq)/sizeof(int));
         offset_dst += it*I;
+        const float * y_scale_tile = nullptr;
+        if constexpr (type == GGML_TYPE_NVFP4) {
+            offset_y_scale += col_low + jt*J;
+            y_scale_tile = y_scale ? y_scale + offset_y_scale : nullptr;
+        }
 
         const int tile_x_max_i = nrows_x  - it*I - 1;
         const int tile_y_max_j = col_diff - jt*J - 1;
@@ -982,7 +1021,8 @@ static __global__ void mul_mat_q(
 
         constexpr bool fixup = false;
         mul_mat_q_process_tile<type, J, fallback, fixup>
-            (x, offset_x, y + offset_y, ids_dst_shared, dst + offset_dst, tmp_fixup, stride_row_x, ncols_y, stride_col_dst,
+            (x, offset_x, y + offset_y, ids_dst_shared, dst + offset_dst, tmp_fixup, y_scale_tile,
+             stride_row_x, ncols_y, stride_col_dst,
              tile_x_max_i, tile_y_max_j, 0, blocks_per_ne00.z);
         return;
     }
@@ -1016,8 +1056,14 @@ static __global__ void mul_mat_q(
         int col_low    = 0;
         int col_high   = ncols_dst;
         int col_diff   = ncols_dst;
-        int offset_y   = wt*stride_sample_y   + zt*stride_channel_y;
-        int offset_dst = wt*stride_sample_dst + zt*stride_channel_dst + jt*J*stride_col_dst;
+        int offset_y       = wt*stride_sample_y   + zt*stride_channel_y;
+        int offset_dst     = wt*stride_sample_dst + zt*stride_channel_dst + jt*J*stride_col_dst;
+        int offset_y_scale;
+        if constexpr (type == GGML_TYPE_NVFP4) {
+            offset_y_scale = wt*nchannels_y.z*ncols_y + zt*ncols_y;
+        } else {
+            GGML_UNUSED(offset_y_scale);
+        }
 
         if (ids_dst) {
             col_low  = expert_bounds[zt + 0];
@@ -1026,6 +1072,9 @@ static __global__ void mul_mat_q(
 
             offset_y   = 0;
             offset_dst = 0;
+            if constexpr (type == GGML_TYPE_NVFP4) {
+                offset_y_scale = 0;
+            }
 
             if (jt*J >= col_diff) {
                 kbc += blocks_per_ne00.z;
@@ -1053,6 +1102,11 @@ static __global__ void mul_mat_q(
 
         offset_y += (col_low + jt * J) * (sizeof(block_q8_1_mmq) / sizeof(int));
         offset_dst += it*I;
+        const float * y_scale_tile = nullptr;
+        if constexpr (type == GGML_TYPE_NVFP4) {
+            offset_y_scale += col_low + jt * J;
+            y_scale_tile = y_scale ? y_scale + offset_y_scale : nullptr;
+        }
 
         const int tile_x_max_i = nrows_x  - it*I - 1;
         const int tile_y_max_j = col_diff - jt*J - 1;
@@ -1061,7 +1115,8 @@ static __global__ void mul_mat_q(
 
         constexpr bool fixup = false; // All but (potentially) the last iterations write their data to dst rather than the fixup buffer.
         mul_mat_q_process_tile<type, J, fallback, fixup>
-            (x, offset_x, y + offset_y, ids_dst_shared, dst + offset_dst, tmp_fixup, stride_row_x, ncols_y, stride_col_dst,
+            (x, offset_x, y + offset_y, ids_dst_shared, dst + offset_dst, tmp_fixup, y_scale_tile,
+             stride_row_x, ncols_y, stride_col_dst,
              tile_x_max_i, tile_y_max_j, kb0_start, kb0_stop);
 
         kbc += blocks_per_ne00.z;
@@ -1090,8 +1145,14 @@ static __global__ void mul_mat_q(
     int col_low    = 0;
     int col_high   = ncols_dst;
     int col_diff   = ncols_dst;
-    int offset_y   = wt*stride_sample_y   + zt*stride_channel_y;
-    int offset_dst = wt*stride_sample_dst + zt*stride_channel_dst + jt*J*stride_col_dst;
+    int offset_y       = wt*stride_sample_y   + zt*stride_channel_y;
+    int offset_dst     = wt*stride_sample_dst + zt*stride_channel_dst + jt*J*stride_col_dst;
+    int offset_y_scale;
+    if constexpr (type == GGML_TYPE_NVFP4) {
+        offset_y_scale = wt*nchannels_y.z*ncols_y + zt*ncols_y;
+    } else {
+        GGML_UNUSED(offset_y_scale);
+    }
 
     if (ids_dst) {
         col_low  = expert_bounds[zt + 0];
@@ -1100,6 +1161,9 @@ static __global__ void mul_mat_q(
 
         offset_y   = 0;
         offset_dst = 0;
+        if constexpr (type == GGML_TYPE_NVFP4) {
+            offset_y_scale = 0;
+        }
 
         if (jt*J >= col_diff) {
             return;
@@ -1122,6 +1186,11 @@ static __global__ void mul_mat_q(
 
     offset_y += (col_low + jt * J) * (sizeof(block_q8_1_mmq) / sizeof(int));
     offset_dst += it*I;
+    const float * y_scale_tile = nullptr;
+    if constexpr (type == GGML_TYPE_NVFP4) {
+        offset_y_scale += col_low + jt * J;
+        y_scale_tile = y_scale ? y_scale + offset_y_scale : nullptr;
+    }
 
     const int tile_x_max_i = nrows_x  - it*I - 1;
     const int tile_y_max_j = col_diff - jt*J - 1;
@@ -1130,7 +1199,8 @@ static __global__ void mul_mat_q(
 
     constexpr bool fixup = true; // Last index writes its data to fixup buffer to avoid data races with other blocks.
     mul_mat_q_process_tile<type, J, fallback, fixup>
-        (x, offset_x, y + offset_y, ids_dst_shared, dst + offset_dst, tmp_fixup, stride_row_x, ncols_y, stride_col_dst,
+        (x, offset_x, y + offset_y, ids_dst_shared, dst + offset_dst, tmp_fixup, y_scale_tile,
+         stride_row_x, ncols_y, stride_col_dst,
          tile_x_max_i, tile_y_max_j, kb0_start, kb0_stop);
 }
 
@@ -1274,6 +1344,7 @@ static __global__ void mul_mat_q_stream_k_fixup(
 
 struct mmq_args {
     const char * x; ggml_type type_x; const int * y; const int32_t * ids_dst; const int32_t * expert_bounds; float * dst;
+    const float * y_scale;
     int64_t ncols_x; int64_t nrows_x; int64_t ncols_dst; int64_t stride_row_x; int64_t ncols_y; int64_t nrows_dst;
     int64_t nchannels_x; int64_t nchannels_y; int64_t stride_channel_x; int64_t stride_channel_y; int64_t stride_channel_dst;
     int64_t nsamples_x; int64_t nsamples_y; int64_t stride_sample_x; int64_t stride_sample_y; int64_t stride_sample_dst;
@@ -1323,7 +1394,7 @@ static void launch_mul_mat_q(ggml_backend_cuda_context & ctx, const mmq_args & a
 
     if (!ggml_cuda_mmq_get_stream_k(type, J, fallback, cc)) {
         mul_mat_q<type, J, fallback><<<block_nums_xy_tiling, block_dims, nbytes_shared, stream>>>
-            (args.x, args.y, args.ids_dst, args.expert_bounds, args.dst, nullptr,
+            (args.x, args.y, args.ids_dst, args.expert_bounds, args.dst, nullptr, args.y_scale,
              blocks_per_ne00_fd, args.nrows_x, args.ncols_dst, args.stride_row_x, args.ncols_y, args.nrows_dst,
              channel_ratio_fd, nchannels_y_fd, args.stride_channel_x, args.stride_channel_y, args.stride_channel_dst,
              sample_ratio_fd, nsamples_y_fd, args.stride_sample_x, args.stride_sample_y, args.stride_sample_dst,
@@ -1352,7 +1423,7 @@ static void launch_mul_mat_q(ggml_backend_cuda_context & ctx, const mmq_args & a
     const dim3 block_dims_fixup(block_dims.x, block_dims.y/2, block_dims.z);
 
     mul_mat_q<type, J, fallback><<<block_nums_stream_k, block_dims, nbytes_shared, stream>>>
-        (args.x, args.y, args.ids_dst, args.expert_bounds, args.dst, tmp_fixup.ptr,
+        (args.x, args.y, args.ids_dst, args.expert_bounds, args.dst, tmp_fixup.ptr, args.y_scale,
          blocks_per_ne00_fd, args.nrows_x, args.ncols_dst, args.stride_row_x, args.ncols_y, args.nrows_dst,
          channel_ratio_fd, nchannels_y_fd, args.stride_channel_x, args.stride_channel_y, args.stride_channel_dst,
          sample_ratio_fd, nsamples_y_fd, args.stride_sample_x, args.stride_sample_y, args.stride_sample_dst,

ggml/src/ggml-cuda/quantize.cu

@@ -1,6 +1,55 @@
 #include "quantize.cuh"
 #include <cstdint>
 
+#if defined(BLACKWELL_MMA_AVAILABLE)
+// this maps to 256-bit loads in PTX on supported devices,
+// and otherwise falls back to 2 128-bit loads
+struct __builtin_align__(32) float8 {
+    float x; float y; float z; float w;
+    float p; float q; float r; float s;
+};
+#endif
+
+#if CUDART_VERSION >= 12080
+static __device__ __forceinline__ float nvfp4_native_scale_error(
+        const float vals[QK_NVFP4_SUB], const float inv_col_scale, const float inv_scale, const float scale) {
+    const float scale_dequant = 2.0f * scale;
+    float err = 0.0f;
+
+#pragma unroll
+    for (int k = 0; k < QK_NVFP4_SUB; k += 4) {
+        const float v0 = vals[k + 0] * inv_col_scale;
+        const float v1 = vals[k + 1] * inv_col_scale;
+        const float v2 = vals[k + 2] * inv_col_scale;
+        const float v3 = vals[k + 3] * inv_col_scale;
+
+        const __nv_fp4x4_e2m1 q(make_float4(v0 * inv_scale, v1 * inv_scale, v2 * inv_scale, v3 * inv_scale));
+        const __nv_fp4x4_storage_t q_storage = q.__x;
+        const __nv_fp4x2_storage_t q_lo = static_cast<__nv_fp4x2_storage_t>(q_storage);
+        const __nv_fp4x2_storage_t q_hi = static_cast<__nv_fp4x2_storage_t>(q_storage >> 8U);
+
+        const __half2_raw hraw2_lo = __nv_cvt_fp4x2_to_halfraw2(q_lo, __NV_E2M1);
+        const __half2_raw hraw2_hi = __nv_cvt_fp4x2_to_halfraw2(q_hi, __NV_E2M1);
+        const __half2 h2_lo = static_cast<__half2>(hraw2_lo);
+        const __half2 h2_hi = static_cast<__half2>(hraw2_hi);
+        const float2 dq_lo = __half22float2(h2_lo);
+        const float2 dq_hi = __half22float2(h2_hi);
+
+        const float err0 = fabsf(v0) - fabsf(dq_lo.x) * scale_dequant;
+        const float err1 = fabsf(v1) - fabsf(dq_lo.y) * scale_dequant;
+        const float err2 = fabsf(v2) - fabsf(dq_hi.x) * scale_dequant;
+        const float err3 = fabsf(v3) - fabsf(dq_hi.y) * scale_dequant;
+
+        err = fmaf(err0, err0, err);
+        err = fmaf(err1, err1, err);
+        err = fmaf(err2, err2, err);
+        err = fmaf(err3, err3, err);
+    }
+
+    return err;
+}
+#endif // CUDART_VERSION >= 12080
+
 __launch_bounds__(CUDA_QUANTIZE_BLOCK_SIZE, 1)
 static __global__ void quantize_q8_1(
         const float * x_ptr, void * vy_ptr,
@@ -74,115 +123,209 @@ __device__ __forceinline__ uint8_t compute_e8m0_scale(float amax) {
     return static_cast<uint8_t>(biased);
 }
 
-
 // scatter: grid over tokens, quantize once, write to all the token's compact rows
-template <bool scatter>
+template <bool scatter, bool use_aligned_float8>
 static __global__ void quantize_mmq_nvfp4(
-        const float * __restrict__ x, const int32_t * __restrict__ ids, void * __restrict__ vy,
+        const float * __restrict__ x, const int32_t * __restrict__ ids, void * __restrict__ vy, float * __restrict__ scale,
         const int64_t ne00, const int64_t s01, const int64_t s02, const int64_t s03,
         const int64_t ne0, const int64_t ne1, const int64_t ne2, const int n_expert_used) {
 #if defined(BLACKWELL_MMA_AVAILABLE)
 
-    const int64_t i0_base = ((int64_t) blockDim.x * blockIdx.y + threadIdx.x) * QK_NVFP4_SUB;
-    if (i0_base >= ne0) {
-        return;
-    }
-
-    const int64_t k_block = i0_base / QK_FP4_MMQ;
     const int64_t blocks_per_col = (ne0 + QK_FP4_MMQ - 1) / QK_FP4_MMQ;
-    if (k_block >= blocks_per_col) {
-        return;
-    }
-    const int sub = (i0_base % QK_FP4_MMQ) / QK_NVFP4_SUB;
 
     int64_t base_idx;
     if constexpr (scatter) {
         base_idx = (int64_t) blockIdx.x * s02; // one physical row per token
     } else {
-        const int64_t i2  = blockIdx.z % ne2;
-        const int64_t i3  = blockIdx.z / ne2;
+        const int64_t i2  = blockIdx.y % ne2;
+        const int64_t i3  = blockIdx.y / ne2;
         const int64_t i01 = ids ? ids[blockIdx.x] : blockIdx.x;
         base_idx = i3 * s03 + i2 * s02 + i01 * s01;
     }
+    const float * __restrict__  x_row = x + base_idx;
 
-    float vals_raw[QK_NVFP4_SUB];
-    float amax_raw = 0.0f;
+    float amax = 0.0f;
+    if constexpr (use_aligned_float8) {
+        for (int64_t i0 = 8 * threadIdx.x; i0 < ne00; i0 += 8 * blockDim.x) {
+            const float * x_base = x_row + i0;
+            const float8 v = reinterpret_cast<const float8 *>(x_base)[0];
+            amax = fmaxf(amax, fabsf(v.x));
+            amax = fmaxf(amax, fabsf(v.y));
+            amax = fmaxf(amax, fabsf(v.z));
+            amax = fmaxf(amax, fabsf(v.w));
+            amax = fmaxf(amax, fabsf(v.p));
+            amax = fmaxf(amax, fabsf(v.q));
+            amax = fmaxf(amax, fabsf(v.r));
+            amax = fmaxf(amax, fabsf(v.s));
+        }
+    } else {
+        for (int64_t i0 = threadIdx.x; i0 < ne00; i0 += blockDim.x) {
+            amax = fmaxf(amax, fabsf(x_row[i0]));
+        }
+    }
+
+    amax = warp_reduce_max<WARP_SIZE>(amax);
+
+    __shared__ float warp_amax[CUDA_QUANTIZE_BLOCK_SIZE_MMQ / WARP_SIZE];
+    const int lane = threadIdx.x % WARP_SIZE;
+    const int warp = threadIdx.x / WARP_SIZE;
+
+    if (lane == 0) {
+        warp_amax[warp] = amax;
+    }
+    __syncthreads();
+
+    if (warp == 0) {
+        amax = threadIdx.x < int(CUDA_QUANTIZE_BLOCK_SIZE_MMQ / WARP_SIZE) ? warp_amax[lane] : 0.0f;
+        amax = warp_reduce_max<WARP_SIZE>(amax);
+        if (lane == 0) {
+            warp_amax[0] = amax / (6.0f * 448.0f);
+            if constexpr (scatter) {
 #pragma unroll
-    for (int k = 0; k < QK_NVFP4_SUB; k++) {
-        const int64_t i00 = i0_base + k;
-        if (i00 < ne00) {
-            const float v = x[base_idx + i00];
-            vals_raw[k] = v;
-            amax_raw = fmaxf(amax_raw, fabsf(v));
-        } else {
-            vals_raw[k] = 0.0f;
+                for (int slot = 0; slot < n_expert_used; ++slot) {
+                    const int64_t i = ids[(int64_t) blockIdx.x * n_expert_used + slot];
+                    scale[i] = warp_amax[0];
+                }
+            } else {
+                scale[blockIdx.y * ne1 + blockIdx.x] = warp_amax[0];
+            }
         }
     }
-
-    static constexpr int test_offsets[5] = { 0, -1, 1, -2, 2};
-    const int first_fp8_code = (int) ggml_cuda_fp32_to_ue4m3(amax_raw / 6.0f);
-
-    float best_err = FLT_MAX;
-    uint8_t fp8_code = 0;
-    float subblock_scale = 0.0f;
-
-#pragma unroll // Check +/- 2 to find best code to reduce NVFP4 activation loss. Negligible overhead on Blackwell.
-    for (int i = 0; i < 5; i++) {
-        const int test_code = first_fp8_code + test_offsets[i];
-        if (test_code < 0 || test_code > 0x7e) {
-            continue;
-        }
-        const uint8_t code = (uint8_t) test_code;
-        const float test_scale = ggml_cuda_ue4m3_to_fp32(code);
-        const float test_inv_scale = test_scale > 0.0f ? 0.5f / test_scale : 0.0f;
-        float cur_err = 0.0f;
-#pragma unroll
-        for (int k = 0; k < QK_NVFP4_SUB; ++k) {
-            const float v = vals_raw[k];
-            const uint8_t q = ggml_cuda_float_to_fp4_e2m1(v, test_inv_scale);
-            const float err_diff = fabsf(v) - fabsf(kvalues_mxfp4[q & 0x7]) * test_scale;
-            cur_err = fmaf(err_diff, err_diff, cur_err);
-        }
-
-        if (cur_err < best_err) {
-            best_err = cur_err;
-            fp8_code = test_code;
-            subblock_scale = test_scale;
-        }
-    }
-
-    const float inv_scale = subblock_scale > 0.0f ? 0.5f / subblock_scale : 0.0f;
-    uint32_t q0 = 0;
-    uint32_t q1 = 0;
-#pragma unroll // this is faster than the previous __nv_fp4x4_e2m1
-    for (int k = 0; k < QK_NVFP4_SUB / 4; ++k) {
-        q0 |= (uint32_t) ggml_cuda_float_to_fp4_e2m1(vals_raw[k +  0], inv_scale) << (8 * k);
-        q0 |= (uint32_t) ggml_cuda_float_to_fp4_e2m1(vals_raw[k +  8], inv_scale) << (8 * k + 4);
-        q1 |= (uint32_t) ggml_cuda_float_to_fp4_e2m1(vals_raw[k +  4], inv_scale) << (8 * k);
-        q1 |= (uint32_t) ggml_cuda_float_to_fp4_e2m1(vals_raw[k + 12], inv_scale) << (8 * k + 4);
-    }
+    __syncthreads();
 
     block_fp4_mmq * y = (block_fp4_mmq *) vy;
-    if constexpr (scatter) {
+    const int64_t n_subblocks = (ne0 + QK_NVFP4_SUB - 1) / QK_NVFP4_SUB;
+
+    for (int64_t isb = threadIdx.x; isb < n_subblocks; isb += blockDim.x) {
+        const int64_t i0_base = isb * QK_NVFP4_SUB;
+        const int64_t k_block = i0_base / QK_FP4_MMQ;
+        const int sub = (i0_base % QK_FP4_MMQ) / QK_NVFP4_SUB;
+
+        const float row_scale = warp_amax[0];
+        const float inv_col_scale = row_scale > 0.0f ? 1.0f / row_scale : 0.0f;
+
+        float vals[QK_NVFP4_SUB];
+        if constexpr (use_aligned_float8) {
+            const float * x_base = x_row + i0_base;
+            const float8 v0 = i0_base +  7 < ne00 ? reinterpret_cast<const float8 *>(x_base)[0]     : float8{0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f};
+            const float8 v1 = i0_base + 15 < ne00 ? reinterpret_cast<const float8 *>(x_base + 8)[0] : float8{0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f, 0.0f};
+            vals[0] = v0.x; vals[1] = v0.y; vals[2] = v0.z; vals[3] = v0.w;
+            vals[4] = v0.p; vals[5] = v0.q; vals[6] = v0.r; vals[7] = v0.s;
+            vals[8] = v1.x; vals[9] = v1.y; vals[10] = v1.z; vals[11] = v1.w;
+            vals[12] = v1.p; vals[13] = v1.q; vals[14] = v1.r; vals[15] = v1.s;
+        } else {
 #pragma unroll
-        for (int slot = 0; slot < n_expert_used; ++slot) {
-            const int64_t i = ids[(int64_t) blockIdx.x * n_expert_used + slot];
-            block_fp4_mmq * yb = y + (k_block * ne1 + i);
+            for (int k = 0; k < QK_NVFP4_SUB; ++k) {
+                const int64_t i00 = i0_base + k;
+                vals[k] = i00 < ne00 ? x_row[i00] : 0.0f;
+            }
+        }
+
+        uint32_t q0 = 0;
+        uint32_t q1 = 0;
+
+        float amax_sub = 0.0f;
+#pragma unroll
+        for (int k = 0; k < QK_NVFP4_SUB; ++k) {
+            amax_sub = fmaxf(amax_sub, fabsf(vals[k] * inv_col_scale));
+        }
+
+        static constexpr int test_offsets[5] = { 0, -1, 1, -2, 2 };
+        const int first_fp8_code = (int) ggml_cuda_fp32_to_ue4m3(amax_sub / 6.0f);
+
+        uint8_t fp8_code = (uint8_t) first_fp8_code;
+        float subblock_scale = ggml_cuda_ue4m3_to_fp32(fp8_code);
+        float inv_scale_err = subblock_scale > 0.0f ? 0.5f / subblock_scale : 0.0f;
+#if CUDART_VERSION >= 12080
+        float best_err = nvfp4_native_scale_error(vals, inv_col_scale, inv_scale_err, subblock_scale);
+#else
+        float best_err = 0.0f;
+#pragma unroll
+        for (int k = 0; k < QK_NVFP4_SUB; ++k) {
+            const float v = vals[k] * inv_col_scale;
+            const uint8_t q = ggml_cuda_float_to_fp4_e2m1(v, inv_scale_err);
+            const float err_diff = fabsf(v) - fabsf(kvalues_fp4[q & 0x7]) * subblock_scale;
+            best_err = fmaf(err_diff, err_diff, best_err);
+        }
+#endif // CUDART_VERSION >= 12080
+
+#pragma unroll
+        for (int i = 1; i < 5; ++i) {
+            const int test_code = first_fp8_code + test_offsets[i];
+            if (test_code < 0 || test_code > 0x7e) {
+                continue;
+            }
+
+            const float test_scale = ggml_cuda_ue4m3_to_fp32((uint8_t) test_code);
+            const float test_inv_scale = test_scale > 0.0f ? 0.5f / test_scale : 0.0f;
+#if CUDART_VERSION >= 12080
+            const float cur_err = nvfp4_native_scale_error(vals, inv_col_scale, test_inv_scale, test_scale);
+#else
+            float cur_err = 0.0f;
+#pragma unroll
+            for (int k = 0; k < QK_NVFP4_SUB; ++k) {
+                const float v = vals[k] * inv_col_scale;
+                const uint8_t q = ggml_cuda_float_to_fp4_e2m1(v, test_inv_scale);
+                const float err_diff = fabsf(v) - fabsf(kvalues_fp4[q & 0x7]) * test_scale;
+                cur_err = fmaf(err_diff, err_diff, cur_err);
+            }
+#endif // CUDART_VERSION >= 12080
+
+            if (cur_err < best_err) {
+                best_err = cur_err;
+                fp8_code = (uint8_t) test_code;
+                subblock_scale = test_scale;
+            }
+        }
+#if CUDART_VERSION >= 12080
+        const float inv_scale = subblock_scale > 0.0f ? 0.5f / subblock_scale : 0.0f;
+        const float s = inv_col_scale * inv_scale;
+
+        __nv_fp4x4_e2m1 q0_lo(make_float4(vals[0] * s, vals[8]  * s, vals[1] * s, vals[9]  * s));
+        __nv_fp4x4_e2m1 q0_hi(make_float4(vals[2] * s, vals[10] * s, vals[3] * s, vals[11] * s));
+        __nv_fp4x4_e2m1 q1_lo(make_float4(vals[4] * s, vals[12] * s, vals[5] * s, vals[13] * s));
+        __nv_fp4x4_e2m1 q1_hi(make_float4(vals[6] * s, vals[14] * s, vals[7] * s, vals[15] * s));
+
+        const char2 q0_lo_c = *reinterpret_cast<char2 *>(&q0_lo);
+        const char2 q0_hi_c = *reinterpret_cast<char2 *>(&q0_hi);
+        const char2 q1_lo_c = *reinterpret_cast<char2 *>(&q1_lo);
+        const char2 q1_hi_c = *reinterpret_cast<char2 *>(&q1_hi);
+
+        q0 = uint32_t(uint8_t(q0_lo_c.x)) | (uint32_t(uint8_t(q0_lo_c.y)) <<  8) |
+            (uint32_t(uint8_t(q0_hi_c.x)) << 16) | (uint32_t(uint8_t(q0_hi_c.y)) << 24);
+        q1 = uint32_t(uint8_t(q1_lo_c.x)) | (uint32_t(uint8_t(q1_lo_c.y)) <<  8) |
+            (uint32_t(uint8_t(q1_hi_c.x)) << 16) | (uint32_t(uint8_t(q1_hi_c.y)) << 24);
+#else
+        const float inv_scale = subblock_scale > 0.0f ? 0.5f / subblock_scale : 0.0f;
+#pragma unroll
+        for (int k = 0; k < QK_NVFP4_SUB / 4; ++k) {
+            q0 |= uint32_t(ggml_cuda_float_to_fp4_e2m1(vals[k + 0] * inv_col_scale, inv_scale)) << (8 * k);
+            q0 |= uint32_t(ggml_cuda_float_to_fp4_e2m1(vals[k + 8] * inv_col_scale, inv_scale)) << (8 * k + 4);
+            q1 |= uint32_t(ggml_cuda_float_to_fp4_e2m1(vals[k + 4] * inv_col_scale, inv_scale)) << (8 * k);
+            q1 |= uint32_t(ggml_cuda_float_to_fp4_e2m1(vals[k + 12] * inv_col_scale, inv_scale)) << (8 * k + 4);
+        }
+#endif // CUDART_VERSION >= 12080
+
+        if constexpr (scatter) {
+#pragma unroll
+            for (int slot = 0; slot < n_expert_used; ++slot) {
+                const int64_t i = ids[(int64_t) blockIdx.x * n_expert_used + slot];
+                block_fp4_mmq * yb = y + (k_block * ne1 + i);
+                uint32_t * yqs = reinterpret_cast<uint32_t *>(yb->qs);
+                yqs[2 * sub + 0] = q0;
+                yqs[2 * sub + 1] = q1;
+                reinterpret_cast<uint8_t *>(yb->d4)[sub] = fp8_code;
+            }
+        } else {
+            block_fp4_mmq * yb = y + (blockIdx.y * ((int64_t) blocks_per_col * ne1) + k_block * ne1 + blockIdx.x);
             uint32_t * yqs = reinterpret_cast<uint32_t *>(yb->qs);
             yqs[2 * sub + 0] = q0;
             yqs[2 * sub + 1] = q1;
             reinterpret_cast<uint8_t *>(yb->d4)[sub] = fp8_code;
         }
-    } else {
-        block_fp4_mmq * yb = y + (blockIdx.z * ((int64_t) blocks_per_col * ne1) + k_block * ne1 + blockIdx.x);
-        uint32_t * yqs = reinterpret_cast<uint32_t *>(yb->qs);
-        yqs[2 * sub + 0] = q0;
-        yqs[2 * sub + 1] = q1;
-        reinterpret_cast<uint8_t *>(yb->d4)[sub] = fp8_code;
     }
-    GGML_UNUSED(n_expert_used);
 #else
-    GGML_UNUSED(n_expert_used);
+    GGML_UNUSED_VARS(x, ids, vy, scale, ne00, s01, s02, s03, ne0, ne1, ne2, n_expert_used);
     NO_DEVICE_CODE; // This is for Blackwell NVFP4 activations only.
 #endif // defined(BLACKWELL_MMA_AVAILABLE)
 
@@ -491,18 +634,22 @@ void quantize_scatter_mmq_q8_1_cuda(
 
 // scatter=true reuses the quant kernels: grid over tokens, ids = inverse map (token slot -> compact row)
 void quantize_scatter_mmq_fp4_cuda(
-        const float * x, const int32_t * ids_src1_inv, void * vy, const ggml_type type_src0,
+        const float * x, const int32_t * ids_src1_inv, void * vy, float * scale, const ggml_type type_src0, const bool use_aligned_float8,
         const int64_t ne00, const int64_t stride_token, const int64_t ne0,
         const int64_t n_tokens, const int64_t nrows_dst, const int n_expert_used, cudaStream_t stream) {
     GGML_ASSERT(ne0 > 0);
     if (type_src0 == GGML_TYPE_NVFP4) {
+        GGML_ASSERT(scale);
         GGML_ASSERT(ne00 % QK_NVFP4 == 0);
-        constexpr int nvfp4_block_size = 128;
-        const int64_t block_num_y = (ne0 + QK_NVFP4_SUB * nvfp4_block_size - 1) / (QK_NVFP4_SUB * nvfp4_block_size);
-        const dim3 block_size(nvfp4_block_size, 1, 1);
-        const dim3 num_blocks(n_tokens, block_num_y, 1);
-        quantize_mmq_nvfp4<true><<<num_blocks, block_size, 0, stream>>>(
-            x, ids_src1_inv, vy, ne00, /*s01=*/0, /*s02=*/stride_token, /*s03=*/0, ne0, /*ne1=*/nrows_dst, /*ne2=*/1, n_expert_used);
+        const dim3 block_size(CUDA_QUANTIZE_BLOCK_SIZE_MMQ, 1, 1);
+        const dim3 num_blocks(n_tokens, 1, 1);
+        if (use_aligned_float8) {
+            quantize_mmq_nvfp4<true, true><<<num_blocks, block_size, 0, stream>>>(
+                x, ids_src1_inv, vy, scale, ne00, /*s01=*/0, /*s02=*/stride_token, /*s03=*/0, ne0, /*ne1=*/nrows_dst, /*ne2=*/1, n_expert_used);
+        } else {
+            quantize_mmq_nvfp4<true, false><<<num_blocks, block_size, 0, stream>>>(
+                x, ids_src1_inv, vy, scale, ne00, /*s01=*/0, /*s02=*/stride_token, /*s03=*/0, ne0, /*ne1=*/nrows_dst, /*ne2=*/1, n_expert_used);
+        }
     } else {
         GGML_ASSERT(type_src0 == GGML_TYPE_MXFP4);
         constexpr int nwarps = 8;
@@ -516,20 +663,24 @@ void quantize_scatter_mmq_fp4_cuda(
 }
 
 void quantize_mmq_fp4_cuda(
-        const float * x, const int32_t * ids, void * vy, const ggml_type type_src0,
+        const float * x, const int32_t * ids, void * vy, float * scale, const ggml_type type_src0, const bool use_aligned_float8,
         const int64_t ne00, const int64_t s01, const int64_t s02, const int64_t s03,
         const int64_t ne0, const int64_t ne1, const int64_t ne2, const int64_t ne3, cudaStream_t stream) {
     GGML_ASSERT(type_src0 == GGML_TYPE_MXFP4 || type_src0 == GGML_TYPE_NVFP4);
     GGML_ASSERT(ne0 > 0);
 
     if (type_src0 == GGML_TYPE_NVFP4) {
+        GGML_ASSERT(scale);
         GGML_ASSERT(ne00 % QK_NVFP4 == 0);
-        constexpr int nvfp4_block_size = 128;
-        const int64_t block_num_y = (ne0 + QK_NVFP4_SUB * nvfp4_block_size - 1) / (QK_NVFP4_SUB * nvfp4_block_size);
-        const dim3 block_size(nvfp4_block_size, 1, 1);
-        const dim3 num_blocks(ne1, block_num_y, ne2 * ne3);
-        quantize_mmq_nvfp4<false><<<num_blocks, block_size, 0, stream>>>(
-            x, ids, vy, ne00, s01, s02, s03, ne0, ne1, ne2, /*n_expert_used=*/0);
+        const dim3 block_size(CUDA_QUANTIZE_BLOCK_SIZE_MMQ, 1, 1);
+        const dim3 num_blocks(ne1, ne2 * ne3, 1);
+        if (use_aligned_float8) {
+            quantize_mmq_nvfp4<false, true><<<num_blocks, block_size, 0, stream>>>(
+                x, ids, vy, scale, ne00, s01, s02, s03, ne0, ne1, ne2, /*n_expert_used=*/0);
+        } else {
+            quantize_mmq_nvfp4<false, false><<<num_blocks, block_size, 0, stream>>>(
+                x, ids, vy, scale, ne00, s01, s02, s03, ne0, ne1, ne2, /*n_expert_used=*/0);
+        }
     } else {
         GGML_ASSERT(ne0 % (2 * QK_MXFP4) == 0);
 

ggml/src/ggml-cuda/quantize.cuh

@@ -29,7 +29,9 @@ void quantize_mmq_q8_1_cuda(
 void quantize_mmq_fp4_cuda(const float *   x,
                              const int32_t * ids,
                              void *          vy,
+                             float *         scale,
                              ggml_type       type_src0,
+                             bool            use_aligned_float8,
                              int64_t         ne00,
                              int64_t         s01,
                              int64_t         s02,
@@ -44,7 +46,9 @@ void quantize_mmq_fp4_cuda(const float *   x,
 void quantize_scatter_mmq_fp4_cuda(const float *   x,
                                    const int32_t * ids_src1_inv,
                                    void *          vy,
+                                   float *         scale,
                                    ggml_type       type_src0,
+                                   bool            use_aligned_float8,
                                    int64_t         ne00,
                                    int64_t         stride_token,
                                    int64_t         ne0,

ggml/src/ggml-cuda/topk-moe.cu

@@ -8,6 +8,7 @@
 // Kernel config struct - passed by value to CUDA kernel
 struct topk_moe_config {
     bool use_sigmoid;
+    bool use_sqrt_softplus;
     bool with_norm;
     bool delayed_softmax;
 };
@@ -67,6 +68,16 @@ __device__ void sigmoid_warp_inplace(float (&vals)[experts_per_thread], const in
     }
 }
 
+template <int experts_per_thread, bool use_limit>
+__device__ void sqrt_softplus_warp_inplace(float (&vals)[experts_per_thread], const int limit, const int lane) {
+#pragma unroll
+    for (int i = 0; i < experts_per_thread; i++) {
+        const int  idx    = lane + i * WARP_SIZE;
+        const bool active = !use_limit || (idx < limit);
+        vals[i]           = active ? sqrtf(vals[i] > 20.0f ? vals[i] : logf(1.0f + expf(vals[i]))) : -INFINITY;
+    }
+}
+
 /*
     This kernel does the following:
     1. optionally softmax over the logits per token [n_experts, n_tokens]
@@ -115,6 +126,8 @@ __launch_bounds__(4 * WARP_SIZE, 1) __global__ void topk_moe_cuda(const float *
     if (!config.delayed_softmax) {
         if (config.use_sigmoid) {
            sigmoid_warp_inplace<experts_per_thread, false>(wt, n_experts, threadIdx.x);
+        } else if (config.use_sqrt_softplus) {
+           sqrt_softplus_warp_inplace<experts_per_thread, false>(wt, n_experts, threadIdx.x);
         } else {
            softmax_warp_inplace<experts_per_thread, false>(wt, n_experts, threadIdx.x);
         }
@@ -364,9 +377,10 @@ void ggml_cuda_op_topk_moe(ggml_backend_cuda_context &     ctx,
     }
 
     topk_moe_config config;
-    config.use_sigmoid     = args.sigmoid;
-    config.with_norm       = with_norm;
-    config.delayed_softmax = args.delayed_softmax;
+    config.use_sigmoid       = args.sigmoid;
+    config.use_sqrt_softplus = args.sqrt_softplus;
+    config.with_norm         = with_norm;
+    config.delayed_softmax   = args.delayed_softmax;
 
     if (bias) {
         launch_topk_moe_cuda<true>(ctx, logits_d, weights_d, ids_d, bias_d, n_rows, n_experts, n_expert_used, clamp_val,
@@ -415,7 +429,7 @@ bool ggml_cuda_should_use_topk_moe(const ggml_tensor * gating_op,
     } else if (gating_op->op == GGML_OP_UNARY) {
         ggml_unary_op op = ggml_get_unary_op(gating_op);
 
-        if (op != GGML_UNARY_OP_SIGMOID) {
+        if (op != GGML_UNARY_OP_SIGMOID && op != GGML_UNARY_OP_SOFTPLUS) {
             return false;
         }
     }

ggml/src/ggml-cuda/topk-moe.cuh

@@ -5,6 +5,7 @@
 
 struct ggml_cuda_topk_moe_args {
     bool sigmoid{};
+    bool sqrt_softplus{};
     bool softmax{};
     bool delayed_softmax{};
     bool prob_bias{};

ggml/src/ggml-hexagon/ggml-hexagon.cpp

@@ -1286,7 +1286,8 @@ struct ggml_hexagon_opbatch {
         int64_t nb2 = is_repack ? nb1 * ne1 : t->nb[2];
         int64_t nb3 = is_repack ? nb2 * t->ne[2] : t->nb[3];
 
-        return (h->ne[0] == ne0) && (h->ne[1] == ne1) && (h->ne[2] == t->ne[2]) && (h->ne[3] == t->ne[3]) &&
+        return (h->type == t->type) &&
+               (h->ne[0] == ne0) && (h->ne[1] == ne1) && (h->ne[2] == t->ne[2]) && (h->ne[3] == t->ne[3]) &&
                (h->nb[0] == t->nb[0]) && (h->nb[1] == nb1) && (h->nb[2] == nb2) && (h->nb[3] == nb3);
     }
 
@@ -3083,7 +3084,10 @@ static bool ggml_hexagon_supported_activations(const struct ggml_hexagon_session
         return false;
     }
 
-    if (!ggml_is_contiguous(src0) || !ggml_is_contiguous(dst)) {
+    if (!ggml_is_contiguous_1(src0)) {
+        return false;
+    }
+    if (!ggml_is_contiguous(dst)) {
         return false;
     }
 
@@ -3094,7 +3098,7 @@ static bool ggml_hexagon_supported_activations(const struct ggml_hexagon_session
         if (!ggml_are_same_shape(src0, src1)) {
             return false;
         }
-        if (!ggml_is_contiguous(src1)) {
+        if (!ggml_is_contiguous_1(src1)) {
             return false;
         }
     }
@@ -4151,12 +4155,10 @@ static bool ggml_backend_hexagon_device_supports_op(ggml_backend_dev_t dev, cons
                 case GGML_UNARY_OP_SIGMOID:
                 case GGML_UNARY_OP_SOFTPLUS:
                 case GGML_UNARY_OP_TANH:
-                    supp = ggml_hexagon_supported_unary(sess, op);
-                    break;
                 case GGML_UNARY_OP_SILU:
                 case GGML_UNARY_OP_GELU:
                 case GGML_UNARY_OP_GELU_QUICK:
-                    supp = ggml_hexagon_supported_activations(sess, op);
+                    supp = ggml_hexagon_supported_unary(sess, op);
                     break;
                 default:
                     break;

ggml/src/ggml-hexagon/htp/main.c

@@ -723,14 +723,14 @@ static int execute_op(struct htp_ops_context * octx) {
         case HTP_OP_SQRT:
         case HTP_OP_UNARY_SOFTPLUS:
         case HTP_OP_UNARY_SIGMOID:
+        case HTP_OP_UNARY_SILU:
+        case HTP_OP_UNARY_GELU:
         case HTP_OP_UNARY_NEG:
         case HTP_OP_UNARY_EXP:
         case HTP_OP_UNARY_TANH:
         case HTP_OP_L2_NORM:
             return op_unary(octx);
 
-        case HTP_OP_UNARY_SILU:
-        case HTP_OP_UNARY_GELU:
         case HTP_OP_GLU_SWIGLU:
         case HTP_OP_GLU_SWIGLU_OAI:
         case HTP_OP_GLU_GEGLU:

ggml/src/ggml-hexagon/htp/unary-ops.c

@@ -276,6 +276,39 @@ static void sigmoid_f32(const float * restrict src,
     }
 }
 
+// silu(x) = x * sigmoid(x)
+static void silu_f32(const float * restrict src,
+                     float * restrict dst,
+                     const uint32_t num_rows,
+                     const struct htp_unary_context * uctx) {
+    htp_unary_op_preamble;
+
+    for (uint32_t ir = 0; ir < num_rows; ir++) {
+        const uint8_t * restrict src_local = (const uint8_t *)src + (ir * src0_row_size_aligned);
+        uint8_t * restrict dst_local       = (uint8_t *)dst + (ir * dst_row_size_aligned);
+
+        hvx_sigmoid_f32_aa(dst_local, src_local, ne0);
+        hvx_mul_f32_aaa(dst_local, src_local, dst_local, ne0);
+    }
+}
+
+// gelu(x) = x * sigmoid(1.702 * x)  (quick/sigmoid approximation, matches CPU GELU_QUICK reference)
+static void gelu_f32(const float * restrict src,
+                     float * restrict dst,
+                     const uint32_t num_rows,
+                     const struct htp_unary_context * uctx) {
+    htp_unary_op_preamble;
+
+    for (uint32_t ir = 0; ir < num_rows; ir++) {
+        const uint8_t * restrict src_local = (const uint8_t *)src + (ir * src0_row_size_aligned);
+        uint8_t * restrict dst_local       = (uint8_t *)dst + (ir * dst_row_size_aligned);
+
+        hvx_mul_scalar_f32(dst_local, src_local, 1.702f, ne0);
+        hvx_sigmoid_f32_aa(dst_local, dst_local, ne0);
+        hvx_mul_f32_aaa(dst_local, src_local, dst_local, ne0);
+    }
+}
+
 static void tri_f32(const float * restrict src,
                     float * restrict dst,
                     const uint32_t num_rows,
@@ -566,6 +599,8 @@ DEFINE_UNARY_TASK(sqrt,           false, false, sqrt_f32(src0_vtcm, dst_vtcm, bl
 DEFINE_UNARY_TASK(unary_neg,      false, false, neg_f32(src0_vtcm, dst_vtcm, block_size, uctx))
 DEFINE_UNARY_TASK(unary_exp,      false, false, exp_f32(src0_vtcm, dst_vtcm, block_size, uctx))
 DEFINE_UNARY_TASK(unary_sigmoid,  false, false, sigmoid_f32(src0_vtcm, dst_vtcm, block_size, uctx))
+DEFINE_UNARY_TASK(unary_silu,     false, false, silu_f32(src0_vtcm, dst_vtcm, block_size, uctx))
+DEFINE_UNARY_TASK(unary_gelu,     false, false, gelu_f32(src0_vtcm, dst_vtcm, block_size, uctx))
 DEFINE_UNARY_TASK(unary_softplus, false, false, softplus_f32(src0_vtcm, dst_vtcm, block_size, uctx))
 DEFINE_UNARY_TASK(unary_tanh,     false, false, tanh_f32(src0_vtcm, dst_vtcm, block_size, uctx))
 DEFINE_UNARY_TASK(l2_norm,        false, false, l2_norm_f32(src0_vtcm, dst_vtcm, block_size, uctx))
@@ -717,6 +752,19 @@ static inline void tile_unary_softplus_f32(uint8_t * dst_vtcm, const uint8_t * s
     }
 }
 
+// silu(x) = x * sigmoid(x)
+static inline void tile_silu_f32(uint8_t * dst_vtcm, const uint8_t * src_vtcm, uint32_t tw) {
+    hvx_sigmoid_f32_aa(dst_vtcm, src_vtcm, tw);
+    hvx_mul_f32_aaa(dst_vtcm, src_vtcm, dst_vtcm, tw);
+}
+
+// gelu(x) = x * sigmoid(1.702 * x)  (quick/sigmoid approximation, matches CPU GELU_QUICK reference)
+static inline void tile_gelu_f32(uint8_t * dst_vtcm, const uint8_t * src_vtcm, uint32_t tw) {
+    hvx_mul_scalar_f32(dst_vtcm, src_vtcm, 1.702f, tw);
+    hvx_sigmoid_f32_aa(dst_vtcm, dst_vtcm, tw);
+    hvx_mul_f32_aaa(dst_vtcm, src_vtcm, dst_vtcm, tw);
+}
+
 // Triangular mask applied to one column tile. Boundary is an absolute column index, so
 // each vector compares against its absolute column position (col_start + i*VLEN_FP32).
 static inline void tri_apply_tile_f32(const uint8_t * restrict src, uint8_t * restrict dst,
@@ -798,6 +846,8 @@ DEFINE_UNARY_TILED_TASK(sqrt,           false, hvx_sqrt_f32_aa(dst_vtcm, src_vtc
 DEFINE_UNARY_TILED_TASK(unary_neg,      false, hvx_scale_f32_aa(dst_vtcm, src_vtcm, tw, -1.0f))
 DEFINE_UNARY_TILED_TASK(unary_exp,      false, hvx_exp_f32(dst_vtcm, src_vtcm, tw, false))
 DEFINE_UNARY_TILED_TASK(unary_sigmoid,  false, hvx_sigmoid_f32_aa(dst_vtcm, src_vtcm, tw))
+DEFINE_UNARY_TILED_TASK(unary_silu,     false, tile_silu_f32(dst_vtcm, src_vtcm, tw))
+DEFINE_UNARY_TILED_TASK(unary_gelu,     false, tile_gelu_f32(dst_vtcm, src_vtcm, tw))
 DEFINE_UNARY_TILED_TASK(unary_softplus, false, tile_unary_softplus_f32(dst_vtcm, src_vtcm, tw))
 DEFINE_UNARY_TILED_TASK(unary_tanh,     false, hvx_tanh_f32_aa(dst_vtcm, src_vtcm, tw))
 DEFINE_UNARY_TILED_TASK(tri,            true,  tri_apply_tile_f32(src_vtcm, dst_vtcm, tw, col, i01, ne0, tri_ttype))
@@ -821,6 +871,8 @@ static int execute_op_unary_f32(struct htp_ops_context * octx) {
         case HTP_OP_UNARY_NEG:       op_type = "neg-f32";          break;
         case HTP_OP_UNARY_EXP:       op_type = "exp-f32";          break;
         case HTP_OP_UNARY_SIGMOID:   op_type = "sigmoid-f32";      break;
+        case HTP_OP_UNARY_SILU:      op_type = "silu-f32";         break;
+        case HTP_OP_UNARY_GELU:      op_type = "gelu-f32";         break;
         case HTP_OP_UNARY_SOFTPLUS:  op_type = "softplus-f32";     break;
         case HTP_OP_UNARY_TANH:      op_type = "tanh-f32";         break;
         case HTP_OP_L2_NORM:         op_type = "l2norm-f32";       break;
@@ -917,6 +969,8 @@ static int execute_op_unary_f32(struct htp_ops_context * octx) {
                 case HTP_OP_UNARY_NEG:       task_func = unary_task_f32_tiled_unary_neg;      break;
                 case HTP_OP_UNARY_EXP:       task_func = unary_task_f32_tiled_unary_exp;      break;
                 case HTP_OP_UNARY_SIGMOID:   task_func = unary_task_f32_tiled_unary_sigmoid;  break;
+                case HTP_OP_UNARY_SILU:      task_func = unary_task_f32_tiled_unary_silu;     break;
+                case HTP_OP_UNARY_GELU:      task_func = unary_task_f32_tiled_unary_gelu;     break;
                 case HTP_OP_UNARY_SOFTPLUS:  task_func = unary_task_f32_tiled_unary_softplus; break;
                 case HTP_OP_UNARY_TANH:      task_func = unary_task_f32_tiled_unary_tanh;     break;
                 case HTP_OP_TRI:             task_func = unary_task_f32_tiled_tri;            break;
@@ -934,6 +988,8 @@ static int execute_op_unary_f32(struct htp_ops_context * octx) {
                 case HTP_OP_UNARY_NEG:       task_func = unary_task_f32_unary_neg;            break;
                 case HTP_OP_UNARY_EXP:       task_func = unary_task_f32_unary_exp;            break;
                 case HTP_OP_UNARY_SIGMOID:   task_func = unary_task_f32_unary_sigmoid;        break;
+                case HTP_OP_UNARY_SILU:      task_func = unary_task_f32_unary_silu;           break;
+                case HTP_OP_UNARY_GELU:      task_func = unary_task_f32_unary_gelu;           break;
                 case HTP_OP_UNARY_SOFTPLUS:  task_func = unary_task_f32_unary_softplus;       break;
                 case HTP_OP_UNARY_TANH:      task_func = unary_task_f32_unary_tanh;           break;
                 case HTP_OP_L2_NORM:         task_func = unary_task_f32_l2_norm;              break;

ggml/src/ggml-hexagon/htp/unary-ops.h

@@ -51,6 +51,8 @@ static inline bool htp_op_is_unary(uint32_t opcode) {
         case HTP_OP_UNARY_NEG:
         case HTP_OP_UNARY_EXP:
         case HTP_OP_UNARY_SIGMOID:
+        case HTP_OP_UNARY_SILU:
+        case HTP_OP_UNARY_GELU:
         case HTP_OP_UNARY_SOFTPLUS:
         case HTP_OP_UNARY_TANH:
         case HTP_OP_L2_NORM:

ggml/src/ggml-metal/ggml-metal-device.m

@@ -1218,8 +1218,9 @@ bool ggml_metal_device_supports_op(ggml_metal_device_t dev, const struct ggml_te
                    (ggml_get_op_params_i32(op, 4) == 0) && (ggml_get_op_params_i32(op, 6) == 0);
         case GGML_OP_PAD_REFLECT_1D:
         case GGML_OP_TIMESTEP_EMBEDDING:
-        case GGML_OP_LEAKY_RELU:
             return op->src[0]->type == GGML_TYPE_F32;
+        case GGML_OP_LEAKY_RELU:
+            return op->src[0]->type == GGML_TYPE_F32 || op->src[0]->type == GGML_TYPE_F16;
         case GGML_OP_ARGSORT:
         case GGML_OP_TOP_K:
         case GGML_OP_ARANGE:

ggml/src/ggml-openvino/ggml-openvino.cpp

@@ -1378,3 +1378,5 @@ GGML_BACKEND_API ggml_backend_reg_t ggml_backend_openvino_reg(void) {
 
     return ®
 }
+
+GGML_BACKEND_DL_IMPL(ggml_backend_openvino_reg)

ggml/src/ggml-vulkan/ggml-vulkan.cpp

@@ -156,6 +156,24 @@ typedef struct VkPhysicalDeviceShaderFloat8FeaturesEXT {
 } VkPhysicalDeviceShaderFloat8FeaturesEXT;
 #endif
 
+#ifndef VK_KHR_INTERNALLY_SYNCHRONIZED_QUEUES_EXTENSION_NAME
+#define VK_KHR_INTERNALLY_SYNCHRONIZED_QUEUES_EXTENSION_NAME "VK_KHR_internally_synchronized_queues"
+#define VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_INTERNALLY_SYNCHRONIZED_QUEUES_FEATURES_KHR ((VkStructureType)1000504000)
+#define VK_DEVICE_QUEUE_CREATE_INTERNALLY_SYNCHRONIZED_BIT_KHR ((VkDeviceQueueCreateFlagBits)0x00000004)
+
+// Compile-time constant guaranteed; no runtime initialization overhead
+static constexpr vk::DeviceQueueCreateFlagBits eInternallySynchronizedKHR =
+    static_cast<vk::DeviceQueueCreateFlagBits>(0x00000004);
+
+typedef struct VkPhysicalDeviceInternallySynchronizedQueuesFeaturesKHR {
+    VkStructureType    sType;
+    void* pNext;
+    VkBool32           internallySynchronizedQueues;
+} VkPhysicalDeviceInternallySynchronizedQueuesFeaturesKHR;
+#else
+static constexpr vk::DeviceQueueCreateFlagBits eInternallySynchronizedKHR = vk::DeviceQueueCreateFlagBits::eInternallySynchronizedKHR;
+#endif
+
 #define ROUNDUP_POW2(M, N) (((M) + (N) - 1) & ~((N) - 1))
 #define CEIL_DIV(M, N) (((M) / (N)) + (((M) % (N)) != 0))
 static bool is_pow2(uint32_t x) { return x > 1 && (x & (x-1)) == 0; }
@@ -285,27 +303,41 @@ struct vk_command_pool {
 };
 
 // Prevent simultaneous submissions to the same queue.
-// This could be per vk_queue if we stopped having two vk_queue structures
-// sharing the same vk::Queue.
-static std::mutex queue_mutex;
+struct vk_queue_handle {
+    vk::Queue queue;
+    virtual void submit(vk::ArrayProxy<const vk::SubmitInfo> submits, vk::Fence fence) = 0;
+    virtual void lock()   {}   // no-op by default (internally synchronized case)
+    virtual void unlock() {}
+    virtual ~vk_queue_handle() = default;
+};
+
+struct vk_queue_handle_synchronized : vk_queue_handle {
+    std::mutex mutex;
+    void submit(vk::ArrayProxy<const vk::SubmitInfo> submits, vk::Fence fence) override {
+        std::lock_guard<std::mutex> guard(mutex);
+        queue.submit(submits, fence);
+    }
+    void lock()   override { mutex.lock(); }
+    void unlock() override { mutex.unlock(); }
+};
+
+struct vk_queue_handle_unsynchronized : vk_queue_handle {
+    void submit(vk::ArrayProxy<const vk::SubmitInfo> submits, vk::Fence fence) override {
+        // Driver guarantees internal synchronization via VK_KHR_internally_synchronized_queues
+        queue.submit(submits, fence);
+    }
+    // lock()/unlock() inherited no-ops
+};
 
 struct vk_queue {
     uint32_t queue_family_index;
-    vk::Queue queue;
+    std::shared_ptr<vk_queue_handle> handle;
 
     vk_command_pool cmd_pool;
 
     vk::PipelineStageFlags stage_flags;
 
     bool transfer_only;
-
-    // copy everything except the cmd_pool
-    void copyFrom(vk_queue &other) {
-        queue_family_index = other.queue_family_index;
-        queue = other.queue;
-        stage_flags = other.stage_flags;
-        transfer_only = other.transfer_only;
-    }
 };
 
 static const char * ggml_backend_vk_buffer_type_name(ggml_backend_buffer_type_t buft);
@@ -712,11 +744,12 @@ struct vk_device_struct {
     uint32_t vendor_id;
     vk::DriverId driver_id;
     vk_device_architecture architecture;
-    vk_queue compute_queue;
-    vk_queue transfer_queue;
+    std::unique_ptr<vk_queue> compute_queue;
+    std::unique_ptr<vk_queue> transfer_queue;
     bool single_queue;
     bool support_async;
     bool async_use_transfer_queue;
+    bool has_internally_synchronized_queues = false;
     uint32_t subgroup_size;
     uint32_t subgroup_size_log2;
     uint32_t shader_core_count;
@@ -1019,8 +1052,13 @@ struct vk_device_struct {
 
         ggml_vk_destroy_buffer(sync_staging);
 
-        compute_queue.cmd_pool.destroy(device);
-        transfer_queue.cmd_pool.destroy(device);
+        if (compute_queue) compute_queue->cmd_pool.destroy(device);
+        if (transfer_queue) transfer_queue->cmd_pool.destroy(device);
+
+        // Explicitly clear to ensure queues drop their shared_ptrs to handles
+        // before the Vulkan logical device instance is destroyed
+        compute_queue.reset();
+        transfer_queue.reset();
 
         for (auto& pipeline : all_pipelines) {
             if (pipeline.expired()) {
@@ -2157,6 +2195,8 @@ struct ggml_backend_vk_context {
     // and set to true after the buffer contents are consumed.
     bool prealloc_x_need_sync, prealloc_y_need_sync, prealloc_split_k_need_sync;
 
+    bool compute_ctx_has_async_transfers {};
+
     vk_context_ref compute_ctx;
 
     vk_context_ref transfer_ctx;
@@ -2909,8 +2949,7 @@ static vk_command_buffer* ggml_vk_create_cmd_buffer(vk_device& device, vk_comman
 static void ggml_vk_submit(vk_context& ctx, vk::Fence fence) {
     if (ctx->seqs.empty()) {
         if (fence) {
-            std::lock_guard<std::mutex> guard(queue_mutex);
-            ctx->p->q->queue.submit({}, fence);
+            ctx->p->q->handle->submit({}, fence);
         }
         return;
     }
@@ -2979,8 +3018,7 @@ static void ggml_vk_submit(vk_context& ctx, vk::Fence fence) {
         }
     }
 
-    std::lock_guard<std::mutex> guard(queue_mutex);
-    ctx->p->q->queue.submit(submit_infos, fence);
+    ctx->p->q->handle->submit(submit_infos, fence);
 
     ctx->seqs.clear();
 }
@@ -3031,18 +3069,44 @@ static uint32_t ggml_vk_find_queue_family_index(std::vector<vk::QueueFamilyPrope
     abort();
 }
 
-static void ggml_vk_create_queue(vk_device& device, vk_queue& q, uint32_t queue_family_index, uint32_t queue_index, vk::PipelineStageFlags&& stage_flags, bool transfer_only) {
+static std::unique_ptr<vk_queue> ggml_vk_create_queue(vk_device& device, uint32_t queue_family_index, uint32_t queue_index, vk::PipelineStageFlags&& stage_flags, bool transfer_only) {
     VK_LOG_DEBUG("ggml_vk_create_queue()");
     std::lock_guard<std::recursive_mutex> guard(device->mutex);
 
-    q.queue_family_index = queue_family_index;
-    q.transfer_only = transfer_only;
+    auto q = std::make_unique<vk_queue>();
+    q->queue_family_index = queue_family_index;
+    q->transfer_only = transfer_only;
 
-    q.cmd_pool.init(device, &q);
+    std::shared_ptr<vk_queue_handle> h;
+    vk::DeviceQueueInfo2 queue_info2{};
+    queue_info2.queueFamilyIndex = queue_family_index;
+    queue_info2.queueIndex = queue_index;
 
-    q.queue = device->device.getQueue(queue_family_index, queue_index);
+    if (device->has_internally_synchronized_queues) {
+        h = std::make_shared<vk_queue_handle_unsynchronized>();
+        queue_info2.flags = eInternallySynchronizedKHR;
+    } else {
+        h = std::make_shared<vk_queue_handle_synchronized>();
+    }
 
-    q.stage_flags = stage_flags;
+    h->queue = device->device.getQueue2(queue_info2);
+    q->handle = h;
+
+    q->cmd_pool.init(device, q.get());
+
+    q->stage_flags = stage_flags;
+    return q;
+}
+
+static std::unique_ptr<vk_queue> ggml_vk_create_aliased_queue(vk_device& device, const std::unique_ptr<vk_queue>& source) {
+    std::lock_guard<std::recursive_mutex> guard(device->mutex);
+    auto q = std::make_unique<vk_queue>();
+    q->handle = source->handle;
+    q->queue_family_index = source->queue_family_index;
+    q->stage_flags = source->stage_flags;
+    q->transfer_only = source->transfer_only;
+    q->cmd_pool.init(device, q.get());
+    return q;
 }
 
 static vk_context ggml_vk_create_context(ggml_backend_vk_context * ctx, vk_command_pool& p) {
@@ -3107,11 +3171,11 @@ static void ggml_vk_queue_command_pools_cleanup(vk_device& device) {
     // Arbitrary frequency to cleanup/reuse command buffers
     static constexpr uint32_t cleanup_frequency = 10;
 
-    if (device->compute_queue.cmd_pool.buffers_in_use() >= cleanup_frequency) {
-        ggml_vk_command_pool_cleanup(device, device->compute_queue.cmd_pool);
+    if (device->compute_queue->cmd_pool.buffers_in_use() >= cleanup_frequency) {
+        ggml_vk_command_pool_cleanup(device, device->compute_queue->cmd_pool);
     }
-    if (device->transfer_queue.cmd_pool.buffers_in_use() >= cleanup_frequency) {
-        ggml_vk_command_pool_cleanup(device, device->transfer_queue.cmd_pool);
+    if (device->transfer_queue->cmd_pool.buffers_in_use() >= cleanup_frequency) {
+        ggml_vk_command_pool_cleanup(device, device->transfer_queue->cmd_pool);
     }
 }
 
@@ -3358,6 +3422,7 @@ static void ggml_vk_sync_buffers(ggml_backend_vk_context* ctx, vk_context& subct
 
     if (ctx) {
         ctx->prealloc_x_need_sync = ctx->prealloc_y_need_sync = ctx->prealloc_split_k_need_sync = false;
+        ctx->compute_ctx_has_async_transfers = false;
     }
 
     subctx->s->buffer->buf.pipelineBarrier(
@@ -3397,11 +3462,16 @@ static void ggml_vk_wait_events(vk_context& ctx, std::vector<vk::Event>&& events
         return;
     }
 
+    const bool transfer_queue = ctx->p->q->transfer_only;
+
     ctx->s->buffer->buf.waitEvents(
         events,
         ctx->p->q->stage_flags,
         ctx->p->q->stage_flags,
-        {},
+        { {
+          { !transfer_queue ? (vk::AccessFlagBits::eShaderRead | vk::AccessFlagBits::eShaderWrite | vk::AccessFlagBits::eTransferRead | vk::AccessFlagBits::eTransferWrite) : (vk::AccessFlagBits::eTransferRead | vk::AccessFlagBits::eTransferWrite) },
+          { !transfer_queue ? (vk::AccessFlagBits::eShaderRead | vk::AccessFlagBits::eShaderWrite | vk::AccessFlagBits::eTransferRead | vk::AccessFlagBits::eTransferWrite) : (vk::AccessFlagBits::eTransferRead | vk::AccessFlagBits::eTransferWrite) }
+        } },
         {},
         {}
     );
@@ -5886,6 +5956,7 @@ static vk_device ggml_vk_get_device(size_t idx) {
         bool coopmat2_support = false;
         bool coopmat2_decode_vector_support = false;
         bool pipeline_executable_properties_support = false;
+        bool internally_sync_support = false;
         device->coopmat_support = false;
         device->integer_dot_product = false;
         device->shader_64b_indexing = false;
@@ -5957,6 +6028,8 @@ static vk_device ggml_vk_get_device(size_t idx) {
             } else if (strcmp("VK_EXT_shader_64bit_indexing", properties.extensionName) == 0) {
                 device->shader_64b_indexing = true;
 #endif
+            } else if (strcmp(VK_KHR_INTERNALLY_SYNCHRONIZED_QUEUES_EXTENSION_NAME, properties.extensionName) == 0) {
+                internally_sync_support = true;
             }
         }
 
@@ -6143,14 +6216,6 @@ static vk_device ggml_vk_get_device(size_t idx) {
         device->single_queue = compute_queue_family_index == transfer_queue_family_index && queue_family_props[compute_queue_family_index].queueCount == 1;
 
         std::vector<vk::DeviceQueueCreateInfo> device_queue_create_infos;
-        if (compute_queue_family_index != transfer_queue_family_index) {
-            device_queue_create_infos.push_back({vk::DeviceQueueCreateFlags(), compute_queue_family_index, 1, priorities});
-            device_queue_create_infos.push_back({vk::DeviceQueueCreateFlags(), transfer_queue_family_index, 1, priorities + 1});
-        } else if(!device->single_queue) {
-            device_queue_create_infos.push_back({vk::DeviceQueueCreateFlags(), compute_queue_family_index, 2, priorities});
-        } else {
-            device_queue_create_infos.push_back({vk::DeviceQueueCreateFlags(), compute_queue_family_index, 1, priorities});
-        }
         vk::DeviceCreateInfo device_create_info{};
         std::vector<const char *> device_extensions;
         vk::PhysicalDeviceFeatures device_features = device->physical_device.getFeatures();
@@ -6172,6 +6237,17 @@ static vk_device ggml_vk_get_device(size_t idx) {
 
         last_struct = (VkBaseOutStructure *)&vk12_features;
 
+        VkPhysicalDeviceInternallySynchronizedQueuesFeaturesKHR internally_synchronized_queues_features{};
+        internally_synchronized_queues_features.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_INTERNALLY_SYNCHRONIZED_QUEUES_FEATURES_KHR;
+        internally_synchronized_queues_features.pNext = nullptr;
+        internally_synchronized_queues_features.internallySynchronizedQueues = VK_FALSE;
+
+        if (internally_sync_support) {
+            last_struct->pNext = (VkBaseOutStructure *)&internally_synchronized_queues_features;
+            last_struct = (VkBaseOutStructure *)&internally_synchronized_queues_features;
+            device_extensions.push_back(VK_KHR_INTERNALLY_SYNCHRONIZED_QUEUES_EXTENSION_NAME);
+        }
+
         VkPhysicalDevicePipelineRobustnessFeaturesEXT pl_robustness_features;
         pl_robustness_features.pNext = nullptr;
         pl_robustness_features.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PIPELINE_ROBUSTNESS_FEATURES_EXT;
@@ -6310,6 +6386,23 @@ static vk_device ggml_vk_get_device(size_t idx) {
 
         vkGetPhysicalDeviceFeatures2(device->physical_device, &device_features2);
 
+        device->has_internally_synchronized_queues = internally_synchronized_queues_features.internallySynchronizedQueues;
+
+        // Build queue create infos only after querying whether internally synchronized queues are enabled.
+        // getQueue2() later uses the same flag, so creation/retrieval must stay consistent.
+        vk::DeviceQueueCreateFlags queue_flags = device->has_internally_synchronized_queues ?
+                                                eInternallySynchronizedKHR :
+                                                vk::DeviceQueueCreateFlags();
+
+        if (compute_queue_family_index != transfer_queue_family_index) {
+            device_queue_create_infos.push_back({queue_flags, compute_queue_family_index, 1, priorities});
+            device_queue_create_infos.push_back({queue_flags, transfer_queue_family_index, 1, priorities + 1});
+        } else if(!device->single_queue) {
+            device_queue_create_infos.push_back({queue_flags, compute_queue_family_index, 2, priorities});
+        } else {
+            device_queue_create_infos.push_back({queue_flags, compute_queue_family_index, 1, priorities});
+        }
+
         device->pipeline_executable_properties_support = pipeline_executable_properties_support;
 
         device->fp16 = device->fp16 && vk12_features.shaderFloat16;
@@ -6592,7 +6685,7 @@ static vk_device ggml_vk_get_device(size_t idx) {
         device->device = device->physical_device.createDevice(device_create_info);
 
         // Queues
-        ggml_vk_create_queue(device, device->compute_queue, compute_queue_family_index, 0, { vk::PipelineStageFlagBits::eComputeShader | vk::PipelineStageFlagBits::eTransfer }, false);
+        device->compute_queue = ggml_vk_create_queue(device, compute_queue_family_index, 0, { vk::PipelineStageFlagBits::eComputeShader | vk::PipelineStageFlagBits::eTransfer }, false);
 
         // Shaders
         // Disable matmul tile sizes early if performance low or not supported
@@ -6694,13 +6787,11 @@ static vk_device ggml_vk_get_device(size_t idx) {
 
         if (!device->single_queue) {
             const uint32_t transfer_queue_index = compute_queue_family_index == transfer_queue_family_index ? 1 : 0;
-            ggml_vk_create_queue(device, device->transfer_queue, transfer_queue_family_index, transfer_queue_index, { vk::PipelineStageFlagBits::eTransfer }, true);
+            device->transfer_queue = ggml_vk_create_queue(device, transfer_queue_family_index, transfer_queue_index, { vk::PipelineStageFlagBits::eTransfer }, true);
 
             device->async_use_transfer_queue = prefers_transfer_queue || (getenv("GGML_VK_ASYNC_USE_TRANSFER_QUEUE") != nullptr);
         } else {
-            // TODO: Use pointer or reference to avoid copy
-            device->transfer_queue.copyFrom(device->compute_queue);
-            device->transfer_queue.cmd_pool.init(device, &device->transfer_queue);
+            device->transfer_queue = ggml_vk_create_aliased_queue(device, device->compute_queue);
 
             device->async_use_transfer_queue = false;
         }
@@ -7034,6 +7125,10 @@ static void ggml_vk_instance_init() {
         extensions.push_back("VK_EXT_debug_utils");
     }
     VkBool32 enable_best_practice = layer_settings;
+    VkBool32 enable_sync_validation = layer_settings && getenv("GGML_VK_SYNC_VALIDATE") != nullptr;
+    if (enable_sync_validation) {
+        std::cerr << "ggml_vulkan: Synchronization validation enabled" << std::endl;
+    }
     std::vector<vk::LayerSettingEXT> settings = {
         {
             "VK_LAYER_KHRONOS_validation",
@@ -7042,6 +7137,13 @@ static void ggml_vk_instance_init() {
             1,
             &enable_best_practice
         },
+        {
+            "VK_LAYER_KHRONOS_validation",
+            "validate_sync",
+            vk::LayerSettingTypeEXT::eBool32,
+            1,
+            &enable_sync_validation
+        },
     };
     vk::LayerSettingsCreateInfoEXT layer_setting_info(settings);
     vk::InstanceCreateInfo instance_create_info(vk::InstanceCreateFlags{}, &app_info, layers, extensions, &layer_setting_info);
@@ -7263,7 +7365,7 @@ static void ggml_vk_init(ggml_backend_vk_context * ctx, size_t idx) {
     ctx->fence = ctx->device->device.createFence({});
     ctx->almost_ready_fence = ctx->device->device.createFence({});
 
-    ctx->compute_cmd_pool.init(ctx->device, &ctx->device->compute_queue);
+    ctx->compute_cmd_pool.init(ctx->device, ctx->device->compute_queue.get());
     if (ctx->device->async_use_transfer_queue) {
         vk::SemaphoreTypeCreateInfo tci{ vk::SemaphoreType::eTimeline, 0 };
         vk::SemaphoreCreateInfo ci{};
@@ -7271,7 +7373,7 @@ static void ggml_vk_init(ggml_backend_vk_context * ctx, size_t idx) {
         ctx->transfer_semaphore.s = ctx->device->device.createSemaphore(ci);
         ctx->transfer_semaphore.value = 0;
 
-        ctx->transfer_cmd_pool.init(ctx->device, &ctx->device->transfer_queue);
+        ctx->transfer_cmd_pool.init(ctx->device, ctx->device->transfer_queue.get());
     }
 
     if (vk_perf_logger_enabled) {
@@ -8126,7 +8228,7 @@ static void ggml_vk_buffer_write_2d(vk_buffer& dst, size_t offset, const void *
     } else {
         std::lock_guard<std::recursive_mutex> guard(dst->device->mutex);
 
-        vk_context subctx = ggml_vk_create_temporary_context(dst->device->transfer_queue.cmd_pool);
+        vk_context subctx = ggml_vk_create_temporary_context(dst->device->transfer_queue->cmd_pool);
         ggml_vk_ctx_begin(dst->device, subctx);
         bool ret = ggml_vk_buffer_write_2d_async(subctx, dst, offset, src, spitch, dpitch, width, height, true);
         GGML_ASSERT(ret);
@@ -8241,7 +8343,7 @@ static void ggml_vk_buffer_read_2d(vk_buffer& src, size_t offset, void * dst, si
         GGML_ASSERT(src->memory_property_flags & vk::MemoryPropertyFlagBits::eHostCoherent);
 
         std::lock_guard<std::recursive_mutex> guard(src->device->mutex);
-        vk_context subctx = ggml_vk_create_temporary_context(src->device->compute_queue.cmd_pool);
+        vk_context subctx = ggml_vk_create_temporary_context(src->device->compute_queue->cmd_pool);
         ggml_vk_ctx_begin(src->device, subctx);
         subctx->s->buffer->buf.pipelineBarrier(
             vk::PipelineStageFlagBits::eComputeShader | vk::PipelineStageFlagBits::eTransfer,
@@ -8267,7 +8369,7 @@ static void ggml_vk_buffer_read_2d(vk_buffer& src, size_t offset, void * dst, si
     } else {
         std::lock_guard<std::recursive_mutex> guard(src->device->mutex);
 
-        vk_context subctx = ggml_vk_create_temporary_context(src->device->transfer_queue.cmd_pool);
+        vk_context subctx = ggml_vk_create_temporary_context(src->device->transfer_queue->cmd_pool);
         ggml_vk_ctx_begin(src->device, subctx);
         bool ret = ggml_vk_buffer_read_2d_async(subctx, src, offset, dst, spitch, dpitch, width, height, true);
         GGML_ASSERT(ret);
@@ -8304,7 +8406,7 @@ static void ggml_vk_buffer_copy(vk_buffer& dst, size_t dst_offset, vk_buffer& sr
         std::lock_guard<std::recursive_mutex> guard(src->device->mutex);
         VK_LOG_DEBUG("ggml_vk_buffer_copy(SINGLE_DEVICE, " << size << ")");
         // Copy within the device
-        vk_context subctx = ggml_vk_create_temporary_context(src->device->transfer_queue.cmd_pool);
+        vk_context subctx = ggml_vk_create_temporary_context(src->device->transfer_queue->cmd_pool);
         ggml_vk_ctx_begin(src->device, subctx);
         ggml_vk_buffer_copy_async(subctx, dst, dst_offset, src, src_offset, size);
         ggml_vk_ctx_end(subctx);
@@ -8347,7 +8449,7 @@ static void ggml_vk_buffer_memset(vk_buffer& dst, size_t offset, uint32_t c, siz
     }
 
     std::lock_guard<std::recursive_mutex> guard(dst->device->mutex);
-    vk_context subctx = ggml_vk_create_temporary_context(dst->device->transfer_queue.cmd_pool);
+    vk_context subctx = ggml_vk_create_temporary_context(dst->device->transfer_queue->cmd_pool);
     ggml_vk_ctx_begin(dst->device, subctx);
     subctx->s->buffer->buf.fillBuffer(dst->buffer, offset, size, c);
     ggml_vk_ctx_end(subctx);
@@ -15693,6 +15795,10 @@ static void ggml_backend_vk_set_tensor_2d_async(ggml_backend_t backend, ggml_ten
 
     bool ret = ggml_vk_buffer_write_2d_async(cpy_ctx, buf, dst_offset, data, stride_data, stride_tensor, size, n_copies);
 
+    if (ret && !ctx->device->async_use_transfer_queue) {
+        ctx->compute_ctx_has_async_transfers = true;
+    }
+
     if (!ret) {
         const size_t staging_size = size * n_copies;
         ggml_vk_ensure_sync_staging_buffer(ctx, staging_size);
@@ -15815,6 +15921,7 @@ static bool ggml_backend_vk_cpy_tensor_async(ggml_backend_t backend_src, ggml_ba
         ggml_vk_buffer_copy_async(compute_ctx, dst_buf, vk_tensor_offset(dst) + dst->view_offs,
                                    src_buf_ctx->dev_buffer, vk_tensor_offset(src) + src->view_offs,
                                    ggml_nbytes(src));
+        ctx->compute_ctx_has_async_transfers = true;
         return true;
     }
 
@@ -15831,6 +15938,7 @@ static bool ggml_backend_vk_cpy_tensor_async(ggml_backend_t backend_src, ggml_ba
             cpy_ctx = ggml_vk_get_transfer_ctx(ctx);
         } else {
             cpy_ctx = ggml_vk_get_compute_ctx(ctx);
+            ctx->compute_ctx_has_async_transfers = true;
         }
 
         return ggml_vk_buffer_write_async(cpy_ctx, dst_buf,
@@ -15875,19 +15983,17 @@ static void ggml_vk_synchronize(ggml_backend_vk_context * ctx) {
                 1, &ctx->transfer_semaphore.value,
                 0, nullptr,
             };
-            vk::PipelineStageFlags stage = ctx->device->transfer_queue.stage_flags;
+            vk::PipelineStageFlags stage = ctx->device->transfer_queue->stage_flags;
             vk::SubmitInfo si{
                 1, &ctx->transfer_semaphore.s, &stage,
                 0, nullptr,
                 0, nullptr,
             };
             si.setPNext(&tl_info);
-            std::lock_guard<std::mutex> guard(queue_mutex);
-            ctx->device->compute_queue.queue.submit({ si }, ctx->fence);
+            ctx->device->compute_queue->handle->submit({ si }, ctx->fence);
             ctx->transfer_semaphore_last_submitted = ctx->transfer_semaphore.value;
         } else {
-            std::lock_guard<std::mutex> guard(queue_mutex);
-            ctx->device->compute_queue.queue.submit({}, ctx->fence);
+            ctx->device->compute_queue->handle->submit({}, ctx->fence);
         }
         ggml_vk_wait_for_fence(ctx);
         ctx->submit_pending = false;
@@ -16449,7 +16555,9 @@ static ggml_status ggml_backend_vk_graph_compute(ggml_backend_t backend, ggml_cg
         vk::DebugUtilsLabelEXT dul = {};
         dul.pLabelName = "ggml_backend_vk_graph_compute";
         dul.color = std::array<float,4>{1.0f, 1.0f, 1.0f, 1.0f};
-        vk_instance.pfn_vkQueueBeginDebugUtilsLabelEXT(ctx->device->compute_queue.queue, reinterpret_cast<VkDebugUtilsLabelEXT*>(&dul));
+
+        std::lock_guard<vk_queue_handle> guard(*ctx->device->compute_queue->handle);
+        vk_instance.pfn_vkQueueBeginDebugUtilsLabelEXT(ctx->device->compute_queue->handle->queue, reinterpret_cast<VkDebugUtilsLabelEXT*>(&dul));
     }
 
     ctx->prealloc_size_add_rms_partials_offset = 0;
@@ -16471,6 +16579,20 @@ static ggml_status ggml_backend_vk_graph_compute(ggml_backend_t backend, ggml_cg
 
     ggml_vk_submit_transfer_ctx(ctx);
 
+    if (ctx->compute_ctx_has_async_transfers) {
+        vk_context compute_ctx = ggml_vk_get_compute_ctx(ctx);
+        ggml_vk_sync_buffers(ctx, compute_ctx);
+    }
+
+    {
+        vk_context compute_ctx = ggml_vk_get_compute_ctx(ctx);
+        ggml_vk_sync_buffers(nullptr, compute_ctx);
+    }
+
+#ifdef GGML_VULKAN_CHECK_RESULTS
+    ggml_vk_synchronize(ctx);
+#endif
+
     vk_context compute_ctx;
     if (vk_perf_logger_enabled) {
         // allocate/resize the query pool
@@ -17207,7 +17329,11 @@ ggml_backend_t ggml_backend_vk_init(size_t dev_num) {
     };
 
     if (!ctx->device->support_async) {
-        vk_backend->iface.get_tensor_async = nullptr;
+        vk_backend->iface.set_tensor_async    = nullptr;
+        vk_backend->iface.get_tensor_async    = nullptr;
+        vk_backend->iface.set_tensor_2d_async = nullptr;
+        vk_backend->iface.get_tensor_2d_async = nullptr;
+        vk_backend->iface.cpy_tensor_async    = nullptr;
     }
 
     return vk_backend;
@@ -17359,8 +17485,9 @@ static void ggml_backend_vk_device_get_props(ggml_backend_dev_t dev, struct ggml
     props->type        = ggml_backend_vk_device_get_type(dev);
     props->device_id   = ctx->pci_bus_id.empty() ? nullptr : ctx->pci_bus_id.c_str();
     ggml_backend_vk_device_get_memory(dev, &props->memory_free, &props->memory_total);
+    const vk_device& device = ggml_vk_get_device(ctx->device);
     props->caps = {
-        /* .async                 = */ true,
+        /* .async                 = */ device->support_async,
         /* .host_buffer           = */ true,
         /* .buffer_from_host_ptr  = */ false,
         /* .events                = */ true,
@@ -19020,7 +19147,7 @@ static void ggml_vk_check_results_1(ggml_backend_vk_context * ctx, ggml_cgraph *
         ggml_vk_print_graph_origin(tensor, done);
     }
 
-    if (avg_err > 0.01 || std::isnan(avg_err)) {
+    if (avg_err > 0.1 || std::isnan(avg_err)) {
         std::cerr << "ERROR: avg_err=" << avg_err << " in " << ggml_op_name(tensor->op) << " (check " << check_counter << ")" << std::endl;
         std::cerr << "tensor=" << tensor << " tensor->name=" << tensor->name << " tensor->type: " << ggml_type_name(tensor->type) << " ne0=" << tensor->ne[0] << " nb0=" << tensor->nb[0] << " ne1=" << tensor->ne[1] << " nb1=" << tensor->nb[1] << " ne2=" << tensor->ne[2] << " nb2=" << tensor->nb[2] << " ne3=" << tensor->ne[3] << " nb3=" << tensor->nb[3] << " offset=" << tensor->view_offs << std::endl;
         if (src0 != nullptr) {

ggml/src/ggml-webgpu/ggml-webgpu-shader-lib.hpp

@@ -355,6 +355,30 @@ struct ggml_webgpu_conv2d_pipeline_key_hash {
     }
 };
 
+// Same type fields as conv2d plus the input layout (WHCN vs CWHN).
+struct ggml_webgpu_conv2d_dw_pipeline_key {
+    ggml_type weight_type;
+    ggml_type input_type;
+    ggml_type output_type;
+    bool      whcn;
+
+    bool operator==(const ggml_webgpu_conv2d_dw_pipeline_key & other) const {
+        return weight_type == other.weight_type && input_type == other.input_type && output_type == other.output_type &&
+               whcn == other.whcn;
+    }
+};
+
+struct ggml_webgpu_conv2d_dw_pipeline_key_hash {
+    size_t operator()(const ggml_webgpu_conv2d_dw_pipeline_key & key) const {
+        size_t seed = 0;
+        ggml_webgpu_hash_combine(seed, key.weight_type);
+        ggml_webgpu_hash_combine(seed, key.input_type);
+        ggml_webgpu_hash_combine(seed, key.output_type);
+        ggml_webgpu_hash_combine(seed, key.whcn);
+        return seed;
+    }
+};
+
 /** Im2Col **/
 struct ggml_webgpu_im2col_pipeline_key {
     ggml_type input_type;
@@ -1210,6 +1234,8 @@ class ggml_webgpu_shader_lib {
         soft_max_pipelines;
     std::unordered_map<ggml_webgpu_conv2d_pipeline_key, webgpu_pipeline, ggml_webgpu_conv2d_pipeline_key_hash>
         conv2d_pipelines;
+    std::unordered_map<ggml_webgpu_conv2d_dw_pipeline_key, webgpu_pipeline, ggml_webgpu_conv2d_dw_pipeline_key_hash>
+        conv2d_dw_pipelines;
     std::unordered_map<ggml_webgpu_im2col_pipeline_key, webgpu_pipeline, ggml_webgpu_im2col_pipeline_key_hash>
         im2col_pipelines;
 
@@ -3172,6 +3198,50 @@ class ggml_webgpu_shader_lib {
         return conv2d_pipelines[key];
     }
 
+    // whcn selects the input layout: contiguous WHCN vs contiguous-channels CWHN
+    webgpu_pipeline get_conv2d_dw_pipeline(const ggml_webgpu_shader_lib_context & context, bool whcn) {
+        ggml_webgpu_conv2d_dw_pipeline_key key = {};
+        key.weight_type                        = context.src0->type;
+        key.input_type                         = context.src1->type;
+        key.output_type                        = context.dst->type;
+        key.whcn                               = whcn;
+
+        auto it = conv2d_dw_pipelines.find(key);
+        if (it != conv2d_dw_pipelines.end()) {
+            return it->second;
+        }
+
+        std::vector<std::string> defines;
+        std::string              variant = whcn ? "conv_2d_dw_whcn" : "conv_2d_dw_cwhn";
+
+        auto push_type_defines = [&](const char * prefix, ggml_type type) {
+            std::string s_prefix = prefix;
+            if (type == GGML_TYPE_F32) {
+                defines.push_back(s_prefix + "_F32");
+            } else if (type == GGML_TYPE_F16) {
+                defines.push_back(s_prefix + "_F16");
+            } else {
+                GGML_ABORT("Unsupported type for CONV_2D_DW shader");
+            }
+        };
+
+        push_type_defines("WEIGHT", key.weight_type);
+        push_type_defines("INPUT", key.input_type);
+        push_type_defines("OUTPUT", key.output_type);
+        if (whcn) {
+            defines.push_back("WHCN");
+        }
+        defines.push_back(std::string("WG_SIZE=") + std::to_string(context.max_wg_size));
+
+        auto processed           = preprocessor.preprocess(wgsl_conv2d_dw, defines);
+        auto decisions           = std::make_shared<ggml_webgpu_generic_shader_decisions>();
+        decisions->wg_size       = context.max_wg_size;
+        webgpu_pipeline pipeline = ggml_webgpu_create_pipeline(device, processed, variant);
+        pipeline.context         = decisions;
+        conv2d_dw_pipelines[key] = pipeline;
+        return conv2d_dw_pipelines[key];
+    }
+
     webgpu_pipeline get_im2col_pipeline(const ggml_webgpu_shader_lib_context & context) {
         ggml_webgpu_im2col_pipeline_key key = {};
         key.input_type                      = context.src1->type;

ggml/src/ggml-webgpu/ggml-webgpu.cpp

@@ -978,6 +978,67 @@ static webgpu_encoded_op ggml_webgpu_conv_2d(webgpu_context & ctx,
     return ggml_backend_webgpu_build(ctx, pipeline, params, entries, wg_x, wg_y);
 }
 
+// Same param/binding layout as conv_2d; the shader differs
+static webgpu_encoded_op ggml_webgpu_conv_2d_dw(webgpu_context & ctx,
+                                                ggml_tensor *    src0,
+                                                ggml_tensor *    src1,
+                                                ggml_tensor *    dst) {
+    const int32_t s0 = ggml_get_op_params_i32(dst, 0);
+    const int32_t s1 = ggml_get_op_params_i32(dst, 1);
+    const int32_t p0 = ggml_get_op_params_i32(dst, 2);
+    const int32_t p1 = ggml_get_op_params_i32(dst, 3);
+    const int32_t d0 = ggml_get_op_params_i32(dst, 4);
+    const int32_t d1 = ggml_get_op_params_i32(dst, 5);
+
+    // Scalar params matching conv2d_dw.wgsl (weight src0 [KW,KH,1,C], input src1, output dst).
+    std::vector<uint32_t> params = {
+        (uint32_t) (ggml_webgpu_tensor_misalignment(ctx, src0) / ggml_type_size(src0->type)),
+        (uint32_t) (ggml_webgpu_tensor_misalignment(ctx, src1) / ggml_type_size(src1->type)),
+        (uint32_t) (ggml_webgpu_tensor_misalignment(ctx, dst) / ggml_type_size(dst->type)),
+
+        (uint32_t) ggml_nelements(dst),
+        (uint32_t) dst->ne[2],
+        (uint32_t) dst->ne[3],
+        (uint32_t) dst->ne[0],
+        (uint32_t) dst->ne[1],
+        (uint32_t) src1->ne[0],
+        (uint32_t) src1->ne[1],
+        (uint32_t) src0->ne[0],
+        (uint32_t) src0->ne[1],
+
+        (uint32_t) s0,
+        (uint32_t) s1,
+        (uint32_t) p0,
+        (uint32_t) p1,
+        (uint32_t) d0,
+        (uint32_t) d1,
+    };
+
+    std::vector<wgpu::BindGroupEntry> entries = {
+        ggml_webgpu_make_tensor_bind_group_entry(ctx, 0, src0),
+        ggml_webgpu_make_tensor_bind_group_entry(ctx, 1, src1),
+        ggml_webgpu_make_tensor_bind_group_entry(ctx, 2, dst),
+    };
+
+    ggml_webgpu_shader_lib_context shader_lib_ctx = {};
+    shader_lib_ctx.src0                           = src0;
+    shader_lib_ctx.src1                           = src1;
+    shader_lib_ctx.dst                            = dst;
+    shader_lib_ctx.max_wg_size = ctx->global_ctx->capabilities.limits.maxComputeInvocationsPerWorkgroup;
+
+    // Input layout: contiguous -> WHCN, contiguous-channels -> CWHN
+    const bool      whcn      = ggml_is_contiguous(src1);
+    webgpu_pipeline pipeline  = ctx->shader_lib->get_conv2d_dw_pipeline(shader_lib_ctx, whcn);
+    auto *          decisions = static_cast<ggml_webgpu_generic_shader_decisions *>(pipeline.context.get());
+
+    uint32_t wg_x;
+    uint32_t wg_y;
+    uint32_t total_wg = CEIL_DIV((uint32_t) ggml_nelements(dst), decisions->wg_size);
+    compute_2d_workgroups(total_wg, ctx->global_ctx->capabilities.limits.maxComputeWorkgroupsPerDimension, wg_x, wg_y);
+
+    return ggml_backend_webgpu_build(ctx, pipeline, params, entries, wg_x, wg_y);
+}
+
 static webgpu_encoded_op ggml_webgpu_im2col(webgpu_context & ctx,
                                             ggml_tensor *    src0,
                                             ggml_tensor *    src1,
@@ -3164,6 +3225,8 @@ static std::optional<webgpu_encoded_op> ggml_webgpu_encode(webgpu_context ctx,
             return ggml_webgpu_sum_rows(ctx, src0, node);
         case GGML_OP_CONV_2D:
             return ggml_webgpu_conv_2d(ctx, src0, src1, node);
+        case GGML_OP_CONV_2D_DW:
+            return ggml_webgpu_conv_2d_dw(ctx, src0, src1, node);
         case GGML_OP_IM2COL:
             return ggml_webgpu_im2col(ctx, src0, src1, node);
         case GGML_OP_UPSCALE:
@@ -4349,6 +4412,12 @@ static bool ggml_backend_webgpu_device_supports_op(ggml_backend_dev_t dev, const
                           (src0->type == GGML_TYPE_F32 || src0->type == GGML_TYPE_F16) &&
                           (src1->type == GGML_TYPE_F32 || src1->type == GGML_TYPE_F16);
             break;
+        case GGML_OP_CONV_2D_DW:
+            supports_op = (op->type == GGML_TYPE_F32 || op->type == GGML_TYPE_F16) &&
+                          (src0->type == GGML_TYPE_F32 || src0->type == GGML_TYPE_F16) &&
+                          (src1->type == GGML_TYPE_F32 || src1->type == GGML_TYPE_F16) &&
+                          (ggml_is_contiguous(src1) || ggml_is_contiguous_channels(src1));
+            break;
         case GGML_OP_IM2COL:
             supports_op = (op->type == GGML_TYPE_F32 || op->type == GGML_TYPE_F16) &&
                           (src0->type == GGML_TYPE_F32 || src0->type == GGML_TYPE_F16);

ggml/src/ggml-webgpu/wgsl-shaders/conv2d_dw.wgsl

@@ -0,0 +1,137 @@
+#include "common_decls.tmpl"
+enable f16;
+
+// Ported from the Vulkan backend's conv2d_dw.comp. Two variants (based on WHCN)
+// selected by the input (src1) layout: contiguous -> WHCN, else CWHN.
+// weight (src0) is [KW,KH,1,C]; output matches the input layout.
+
+@group(0) @binding(0)
+#if defined(WEIGHT_F32)
+var<storage, read_write> weights: array<f32>;
+#elif defined(WEIGHT_F16)
+var<storage, read_write> weights: array<f16>;
+#endif
+
+@group(0) @binding(1)
+#if defined(INPUT_F32)
+var<storage, read_write> input: array<f32>;
+#elif defined(INPUT_F16)
+var<storage, read_write> input: array<f16>;
+#endif
+
+@group(0) @binding(2)
+#if defined(OUTPUT_F32)
+var<storage, read_write> output: array<f32>;
+#elif defined(OUTPUT_F16)
+var<storage, read_write> output: array<f16>;
+#endif
+
+struct Params {
+    offset_w: u32,
+    offset_i: u32,
+    offset_o: u32,
+
+    ne: u32,
+    channels: u32,
+    batches: u32,
+    dst_w: u32, dst_h: u32,
+    src_w: u32, src_h: u32,
+    knl_w: u32, knl_h: u32,
+
+    stride_x: i32, stride_y: i32,
+    pad_x: i32, pad_y: i32,
+    dilation_x: i32, dilation_y: i32,
+};
+
+@group(0) @binding(3)
+var<uniform> params: Params;
+
+fn load_weight(idx: u32) -> f32 {
+    #if defined(WEIGHT_F32)
+        return weights[idx];
+    #elif defined(WEIGHT_F16)
+        return f32(weights[idx]);
+    #endif
+}
+fn load_input(idx: u32) -> f32 {
+    #if defined(INPUT_F32)
+        return input[idx];
+    #elif defined(INPUT_F16)
+        return f32(input[idx]);
+    #endif
+}
+fn store_output(idx: u32, val: f32) {
+    #if defined(OUTPUT_F32)
+        output[idx] = val;
+    #elif defined(OUTPUT_F16)
+        output[idx] = f16(val);
+    #endif
+}
+
+#if defined(WHCN)
+// Input/output/kernel contiguous in [W, H, C, N] order (kernel [KW,KH,C]).
+fn conv_2d_dw(idx: u32) -> f32 {
+    let i0    = idx / params.dst_w;
+    let dst_x = idx - i0 * params.dst_w;
+    let i1    = i0 / params.dst_h;
+    let dst_y = i0 - i1 * params.dst_h;
+    let n     = i1 / params.channels;
+    let c     = i1 - n * params.channels;
+
+    let src_i = params.offset_i + n * params.channels * params.src_h * params.src_w
+                                + c * params.src_h * params.src_w;
+    let knl_i = params.offset_w + c * params.knl_h * params.knl_w;
+
+    var sum: f32 = 0.0;
+    for (var ky: u32 = 0u; ky < params.knl_h; ky += 1u) {
+        let src_y = i32(dst_y) * params.stride_y + i32(ky) * params.dilation_y - params.pad_y;
+        if (src_y < 0 || src_y >= i32(params.src_h)) { continue; }
+        for (var kx: u32 = 0u; kx < params.knl_w; kx += 1u) {
+            let src_x = i32(dst_x) * params.stride_x + i32(kx) * params.dilation_x - params.pad_x;
+            if (src_x < 0 || src_x >= i32(params.src_w)) { continue; }
+            let v = load_input(src_i + u32(src_y) * params.src_w + u32(src_x));
+            let k = load_weight(knl_i + ky * params.knl_w + kx);
+            sum += v * k;
+        }
+    }
+    return sum;
+}
+#else
+// Channels contiguous (CWHN): channel is the innermost axis.
+fn conv_2d_dw(idx: u32) -> f32 {
+    let i0    = idx / params.channels;
+    let c     = idx - i0 * params.channels;
+    let i1    = i0 / params.dst_w;
+    let dst_x = i0 - i1 * params.dst_w;
+    let n     = i1 / params.dst_h;
+    let dst_y = i1 - n * params.dst_h;
+
+    let src_i   = params.offset_i + n * params.channels * params.src_h * params.src_w;
+    let src_row = params.src_w * params.channels;
+    let knl_row = params.knl_w * params.channels;
+
+    var sum: f32 = 0.0;
+    for (var ky: u32 = 0u; ky < params.knl_h; ky += 1u) {
+        let src_y = i32(dst_y) * params.stride_y + i32(ky) * params.dilation_y - params.pad_y;
+        if (src_y < 0 || src_y >= i32(params.src_h)) { continue; }
+        for (var kx: u32 = 0u; kx < params.knl_w; kx += 1u) {
+            let src_x = i32(dst_x) * params.stride_x + i32(kx) * params.dilation_x - params.pad_x;
+            if (src_x < 0 || src_x >= i32(params.src_w)) { continue; }
+            let v = load_input(src_i + u32(src_y) * src_row + u32(src_x) * params.channels + c);
+            let k = load_weight(params.offset_w + ky * knl_row + kx * params.channels + c);
+            sum += v * k;
+        }
+    }
+    return sum;
+}
+#endif
+
+@compute @workgroup_size(WG_SIZE)
+fn main(
+    @builtin(global_invocation_id) gid: vec3<u32>,
+    @builtin(num_workgroups) num_wg: vec3<u32>
+) {
+    let idx = gid.x + (num_wg.x * u32(WG_SIZE)) * gid.y;
+    if (idx >= params.ne) { return; }
+    store_output(params.offset_o + idx, conv_2d_dw(idx));
+}

gguf-py/gguf/constants.py

@@ -507,6 +507,7 @@ class MODEL_ARCH(IntEnum):
     DOTS1            = auto()
     ARCEE            = auto()
     AFMOE            = auto()
+    LAGUNA           = auto()
     ERNIE4_5         = auto()
     ERNIE4_5_MOE     = auto()
     HUNYUAN_MOE      = auto()
@@ -1088,6 +1089,7 @@ MODEL_ARCH_NAMES: dict[MODEL_ARCH, str] = {
     MODEL_ARCH.DOTS1:            "dots1",
     MODEL_ARCH.ARCEE:            "arcee",
     MODEL_ARCH.AFMOE:            "afmoe",
+    MODEL_ARCH.LAGUNA:           "laguna",
     MODEL_ARCH.ERNIE4_5:         "ernie4_5",
     MODEL_ARCH.ERNIE4_5_MOE:     "ernie4_5-moe",
     MODEL_ARCH.FALCON_H1:        "falcon-h1",
@@ -3823,6 +3825,31 @@ MODEL_TENSORS: dict[MODEL_ARCH, list[MODEL_TENSOR]] = {
         MODEL_TENSOR.FFN_POST_NORM,
         MODEL_TENSOR.FFN_EXP_PROBS_B,
     ],
+    MODEL_ARCH.LAGUNA: [
+        MODEL_TENSOR.TOKEN_EMBD,
+        MODEL_TENSOR.OUTPUT_NORM,
+        MODEL_TENSOR.OUTPUT,
+        MODEL_TENSOR.ATTN_NORM,
+        MODEL_TENSOR.ATTN_Q,
+        MODEL_TENSOR.ATTN_K,
+        MODEL_TENSOR.ATTN_V,
+        MODEL_TENSOR.ATTN_OUT,
+        MODEL_TENSOR.ATTN_Q_NORM,
+        MODEL_TENSOR.ATTN_K_NORM,
+        MODEL_TENSOR.ATTN_GATE,
+        MODEL_TENSOR.FFN_NORM,
+        MODEL_TENSOR.FFN_GATE,
+        MODEL_TENSOR.FFN_DOWN,
+        MODEL_TENSOR.FFN_UP,
+        MODEL_TENSOR.FFN_GATE_INP,
+        MODEL_TENSOR.FFN_EXP_PROBS_B,
+        MODEL_TENSOR.FFN_GATE_EXP,
+        MODEL_TENSOR.FFN_DOWN_EXP,
+        MODEL_TENSOR.FFN_UP_EXP,
+        MODEL_TENSOR.FFN_GATE_SHEXP,
+        MODEL_TENSOR.FFN_UP_SHEXP,
+        MODEL_TENSOR.FFN_DOWN_SHEXP,
+    ],
     MODEL_ARCH.ERNIE4_5: [
         MODEL_TENSOR.TOKEN_EMBD,
         MODEL_TENSOR.OUTPUT_NORM,

gguf-py/gguf/tensor_mapping.py

@@ -479,6 +479,7 @@ class TensorNameMap:
             "model.layers.{bid}.mlp.e_score_correction",                    # exaone-moe
             "model.layers.{bid}.block_sparse_moe.gate.e_score_correction",  # kimi
             "model.layers.{bid}.moe.router_bias",                           # step3.5 expert selection bias
+            "model.layers.{bid}.mlp.experts.e_score_correction",       # laguna
         ),
 
         # Feed-forward up

models/templates/deepseek-ai-DeepSeek-V4.jinja

@@ -8,12 +8,15 @@
     {%- set thinking = false -%}
   {%- endif -%}
 {%- endif -%}
+{%- if not drop_thinking is defined -%}
+  {%- set drop_thinking = false -%}
+{%- endif -%}
 {%- set dsml_token = '｜DSML｜' -%}
 {%- set thinking_start_token = '<think>' -%}
 {%- set thinking_end_token = '</think>' -%}
 {%- set tools_header = '## Tools\n\nYou have access to a set of tools to help answer the user\'s question. You can invoke tools by writing a "<' + dsml_token + 'tool_calls>" block like the following:\n\n<' + dsml_token + 'tool_calls>\n<' + dsml_token + 'invoke name="$TOOL_NAME">\n<' + dsml_token + 'parameter name="$PARAMETER_NAME" string="true|false">$PARAMETER_VALUE</' + dsml_token + 'parameter>\n...\n</' + dsml_token + 'invoke>\n<' + dsml_token + 'invoke name="$TOOL_NAME2">\n...\n</' + dsml_token + 'invoke>\n</' + dsml_token + 'tool_calls>\n\nString parameters should be specified as is and set `string="true"`. For all other types (numbers, booleans, arrays, objects), pass the value in JSON format and set `string="false"`.\n\nIf thinking_mode is enabled (triggered by ' + thinking_start_token + '), you MUST output your complete reasoning inside ' + thinking_start_token + '...' + thinking_end_token + ' BEFORE any tool calls or final response.\n\nOtherwise, output directly after ' + thinking_end_token + ' with tool calls or final response.\n\n### Available Tool Schemas\n\n' -%}
 {%- set tools_footer = '\nYou MUST strictly follow the above defined tool name and parameter schemas to invoke tool calls.\n' -%}
-{%- set ns = namespace(system_prompt='', is_first_sp=true) -%}
+{%- set ns = namespace(system_prompt='', is_first_sp=true, has_tool_calls=false) -%}
 {%- for message in messages -%}
   {%- if message['role'] == 'system' -%}
     {%- if ns.is_first_sp -%}
@@ -46,6 +49,11 @@
   {%- endif -%}
 {%- endfor -%}
 {%- set state = namespace(in_user=false) -%}
+{%- for message in messages -%}
+  {%- if message['role'] == 'tool' -%}
+    {%- set ns.has_tool_calls = true -%}
+  {%- endif -%}
+{%- endfor -%}
 {%- for message in messages -%}
   {%- if message['role'] == 'user' or message['role'] == 'developer' -%}
     {%- if state.in_user -%}
@@ -67,7 +75,8 @@
     {%- set state.in_user = false -%}
     {{- '<｜Assistant｜>' -}}
     {%- set is_after_last_user = loop.index0 > last_user_idx.value -%}
-    {%- if is_after_last_user and thinking -%}
+    {%- set retain_reasoning = (not drop_thinking) or (is_after_last_user or ns.has_tool_calls) -%}
+    {%- if retain_reasoning and thinking -%}
       {{- thinking_start_token -}}
       {%- if message['reasoning_content'] is defined and message['reasoning_content'] -%}
         {{- message['reasoning_content'] -}}

models/templates/poolside-Laguna-S-2.1.jinja

@@ -0,0 +1,93 @@
+{#- Iteration on laguna_glm_thinking_v8/chat_template.jinja -#}
+{#- No formatting instructions -#}
+{{- "〈|EOS|〉" -}}
+{%- set enable_thinking = enable_thinking | default(false) -%}
+{%- set add_generation_prompt = add_generation_prompt | default(false) -%}
+
+{#- ───── header (system message) ───── -#}
+{#- A caller-supplied system message with empty content opts out of the default below, producing no <system> block — used to train without a system message. -#}
+{%- set system_message = "You are a helpful, conversationally-fluent assistant made by Poolside. You are here to be helpful to users through natural language conversations." -%}
+{%- if messages and messages[0].role == "system" -%}
+  {%- set system_message = messages[0].content -%}
+  {%- set messages = messages[1:] -%}
+{%- endif -%}
+
+{%- set has_sys = system_message and system_message.strip() -%}
+{%- if has_sys or tools or enable_thinking -%}
+  {{- "<system>" -}}
+
+  {%- if has_sys -%}
+    {{- system_message.rstrip() -}}
+    {%- if tools -%}{{- "\n\n" -}}{%- endif -%}
+  {%- endif -%}
+
+  {%- if tools -%}
+    {{- "### Tools\n\n" -}}
+    {{- "You may call functions to assist with the user query.\n" -}}
+    {{- "All available function signatures are listed below:\n" -}}
+    {{- "<available_tools>\n" -}}
+    {%- for tool in tools -%}
+      {{- (tool | tojson) ~ "\n" -}}
+    {%- endfor -%}
+    {{- "</available_tools>" -}}
+  {%- endif -%}
+
+  {{- "</system>\n" -}}
+{%- endif -%}
+
+{#- ───── main loop ───── -#}
+{%- for message in messages -%}
+  {%- set content = message.content if message.content is string else "" -%}
+  {%- if message.role == "user" -%}
+    {{- "<user>" + content + "</user>\n" -}}
+  {%- elif message.role == "assistant" -%}
+    {%- generation -%}
+      {{- "<assistant>" -}}
+      {#- Extract reasoning content from message.reasoning (vLLM field name) or message.reasoning_content -#}
+      {%- set reasoning_content = '' -%}
+      {%- if message.reasoning is string -%}
+        {%- set reasoning_content = message.reasoning -%}
+      {%- elif message.reasoning_content is string -%}
+        {%- set reasoning_content = message.reasoning_content -%}
+      {%- endif -%}
+      {#- Display reasoning content for all messages if enable_thinking -#}
+      {%- if enable_thinking -%}
+        {{- '<think>' + reasoning_content + '</think>' -}}
+      {%- else -%}
+        {{- '</think>' -}}
+      {%- endif -%}
+      {#- Display main content (trailing newline only when no tool_calls follow) -#}
+      {%- if content -%}
+        {{- content -}}
+      {%- endif -%}
+      {%- if message.tool_calls -%}
+        {%- for tool_call in message.tool_calls -%}
+          {%- set function_data = tool_call.function -%}
+          {{- '<tool_call>' + function_data.name -}}
+          {%- set _args = function_data.arguments -%}
+          {%- for k, v in _args.items() -%}
+            {{- "<arg_key>" ~ k ~ "</arg_key>" -}}
+            {{- "<arg_value>" -}}{{- v | tojson(ensure_ascii=False) if v is not string else v -}}{{- "</arg_value>" -}}
+          {%- endfor -%}
+          {{- "</tool_call>" -}}
+        {%- endfor -%}
+      {%- endif -%}
+      {{- "</assistant>\n" -}}
+    {%- endgeneration -%}
+  {%- elif message.role == "tool" -%}
+    {{- "<tool_response>" + content + "</tool_response>\n" -}}
+  {%- elif message.role == "system" -%}
+    {#- Render additional system messages (the first one, if any, is handled separately in the header and was sliced off above) -#}
+    {{- "<system>" + content + "</system>\n" -}}
+  {%- endif -%}
+{%- endfor -%}
+{#- ───── generation prompt ───── -#}
+{%- if add_generation_prompt -%}
+  {{- "<assistant>" -}}
+  {#- ───── Include reasoning mode directive ───── -#}
+  {%- if enable_thinking -%}
+    {{- '<think>' -}}
+  {%- else -%}
+    {{- '</think>' -}}
+  {%- endif -%}
+{%- endif -%}

models/templates/poolside-Laguna-XS-2.1.jinja

@@ -0,0 +1,132 @@
+{#- Copied from laguna_glm_thinking_v4/chat_template.jinja -#}
+{#- Removes prefix that references <think> token, and replaces message.reasoning_content reference with message.reasoning -#}
+{{- "〈|EOS|〉" -}}
+{%- set enable_thinking = enable_thinking | default(false) -%}
+{%- set render_assistant_messages_raw = render_assistant_messages_raw | default(false) -%}
+{%- set add_generation_prompt = add_generation_prompt | default(false) -%}
+
+{#- ───── header (system message) ───── -#}
+{%- set system_message = "" -%}
+{%- if messages and messages[0].role == "system" -%}
+  {%- set system_message = messages[0].content -%}
+{%- endif -%}
+
+{%- if (system_message and system_message.strip()) or tools -%}
+  {{- "<system>\n" -}}
+
+  {%- if system_message and system_message.strip() -%}
+    {{- "\n" -}}
+    {{- system_message.rstrip() -}}
+  {%- endif -%}
+
+  {%- if tools -%}
+    {{- "\n\n### Tools\n\n" -}}
+    {%- set ns = namespace(tool_string="You may call functions to assist with the user query.\n"
+          ~ "All available function signatures are listed below:\n"
+          ~ "<available_tools>\n") -%}
+    {%- for tool in tools -%}
+      {%- set ns.tool_string = ns.tool_string ~ (tool | tojson) ~ "\n" -%}
+    {%- endfor -%}
+    {%- if enable_thinking -%}
+      {%- set tool_string = ns.tool_string + "</available_tools>\n\n" ~
+            "Wrap your thinking in '<think>', '</think>' tags, followed by a function call. For each function call, return an unescaped XML-like object with function name and arguments within '<tool_call>' and '</tool_call>' tags, like here:\n" ~
+            "<think> your thoughts here </think>\n" ~
+            "<tool_call>function-name\n<arg_key>argument-key</arg_key>\n<arg_value>value-of-argument-key</arg_value>\n" ~
+            "</tool_call>" -%}
+    {%- else -%}
+      {%- set tool_string = ns.tool_string + "</available_tools>\n\n" ~
+            "For each function call, return an unescaped XML-like object " ~
+            "with function name and arguments within '<tool_call>' and '</tool_call>' tags, like here:\n" ~
+            "<tool_call>function-name\n<arg_key>argument-key</arg_key>\n<arg_value>value-of-argument-key</arg_value>\n" ~
+            "</tool_call>" -%}
+    {%- endif -%}
+    {{- tool_string -}}
+  {%- endif -%}
+
+  {{- "\n</system>\n" -}}
+{%- endif -%}
+
+{#- ───── main loop ───── -#}
+{%- for message in messages -%}
+  {%- set content = message.content if message.content is string else "" -%}
+  {%- if message.role == "user" -%}
+    {{- "<user>\n" + content + "\n</user>\n" -}}
+  {%- elif message.role == "assistant" -%}
+    {%- generation -%}
+      {{- "<assistant>\n" -}}
+      {%- if render_assistant_messages_raw -%}
+        {#- Raw mode: prepend the generation prompt token, then dump content verbatim. -#}
+        {#- The generation prompt is <think> when enable_thinking, </think> otherwise. -#}
+        {#- Only prepend if content doesn't already start with it. -#}
+        {%- if enable_thinking -%}
+          {%- if not content.startswith('<think>') -%}
+            {{- '<think>' -}}
+          {%- endif -%}
+        {%- else -%}
+          {%- if not content.startswith('</think>') -%}
+            {{- '</think>' -}}
+          {%- endif -%}
+        {%- endif -%}
+        {{- content -}}
+        {#- Append closing tag if content doesn't already end with it. -#}
+        {%- if not content.endswith('</assistant>\n') and not content.endswith('</assistant>') -%}
+          {{- '\n</assistant>' -}}
+        {%- endif -%}
+        {{- "\n" -}}
+      {%- else -%}
+        {#- Extract reasoning content from message.reasoning (vLLM field name) or message.reasoning_content, or from <think> tags -#}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning is string %}
+          {%- set reasoning_content = message.reasoning %}
+        {%- elif message.reasoning_content is string %}
+          {%- set reasoning_content = message.reasoning_content %}
+        {%- endif %}
+        {#- Always strip <think> tags from content if present to avoid duplication -#}
+        {%- if '</think>' in content %}
+          {%- if not reasoning_content %}
+            {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+          {%- endif %}
+          {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+        {%- endif %}
+        {#- Display reasoning content for all messages -#}
+        {%- if reasoning_content -%}
+          {{- '<think>\n' + reasoning_content.strip() + '\n</think>\n' -}}
+        {%- else -%}
+          {{- '</think>\n' -}}
+        {%- endif -%}
+        {#- Display main content -#}
+        {%- if content.strip() -%}
+          {{- content.strip() ~ "\n" -}}
+        {%- endif -%}
+        {%- if message.tool_calls -%}
+          {%- for tool_call in message.tool_calls -%}
+            {%- set function_data = tool_call.function -%}
+            {{- '<tool_call>' + function_data.name }}
+            {% set _args = function_data.arguments %}
+            {%- for k, v in _args.items() -%}
+              {{- "<arg_key>" ~ k ~ "</arg_key>\n" -}}
+              {{- "<arg_value>"}}{{ v | tojson(ensure_ascii=False) if v is not string else v }}{{ "</arg_value>\n" -}}
+            {%- endfor -%}
+            {{- "</tool_call>\n" -}}
+          {%- endfor -%}
+        {%- endif -%}
+        {{- "</assistant>\n" -}}
+      {%- endif -%}
+    {%- endgeneration -%}
+  {%- elif message.role == "tool" -%}
+    {{- "<tool_response>\n" + content + "\n</tool_response>\n" -}}
+  {%- elif message.role == "system" and loop.index0 != 0 -%}
+    {#- Render additional system messages (skip the first one which is handled separately in the header) -#}
+    {{- "<system>\n" + content + "\n</system>\n" -}}
+  {%- endif -%}
+{%- endfor -%}
+{#- ───── generation prompt ───── -#}
+{%- if add_generation_prompt -%}
+  {{- "<assistant>\n" -}}
+  {#- ───── Include reasoning mode directive ───── -#}
+  {%- if not enable_thinking %}
+    {{- '</think>' -}}
+  {%- else %}
+    {{- '<think>' -}}
+  {%- endif %}
+{%- endif -%}

models/templates/poolside-Laguna-XS.2.jinja

@@ -0,0 +1,132 @@
+{#- Iteration on laguna_glm_thinking_v5/chat_template.jinja -#}
+{#- Adds a default system message (used when no system message is provided in `messages`). -#}
+{{- "〈|EOS|〉" -}}
+{%- set enable_thinking = enable_thinking | default(false) -%}
+{%- set render_assistant_messages_raw = render_assistant_messages_raw | default(false) -%}
+{%- set add_generation_prompt = add_generation_prompt | default(false) -%}
+
+{#- ───── header (system message) ───── -#}
+{%- set system_message = "You are a helpful, conversationally-fluent assistant made by Poolside. You are here to be helpful to users through natural language conversations." -%}
+{%- if messages and messages[0].role == "system" -%}
+  {%- set system_message = messages[0].content -%}
+{%- endif -%}
+
+{%- if (system_message and system_message.strip()) or tools -%}
+  {{- "<system>\n" -}}
+
+  {%- if system_message and system_message.strip() -%}
+    {{- "\n" -}}
+    {{- system_message.rstrip() -}}
+  {%- endif -%}
+
+  {%- if tools -%}
+    {{- "\n\n### Tools\n\n" -}}
+    {%- set ns = namespace(tool_string="You may call functions to assist with the user query.\n"
+          ~ "All available function signatures are listed below:\n"
+          ~ "<available_tools>\n") -%}
+    {%- for tool in tools -%}
+      {%- set ns.tool_string = ns.tool_string ~ (tool | tojson) ~ "\n" -%}
+    {%- endfor -%}
+    {%- if enable_thinking -%}
+      {%- set tool_string = ns.tool_string + "</available_tools>\n\n" ~
+            "Wrap your thinking in '<think>', '</think>' tags, followed by a function call. For each function call, return an unescaped XML-like object with function name and arguments within '<tool_call>' and '</tool_call>' tags, like here:\n" ~
+            "<think> your thoughts here </think>\n" ~
+            "<tool_call>function-name\n<arg_key>argument-key</arg_key>\n<arg_value>value-of-argument-key</arg_value>\n" ~
+            "</tool_call>" -%}
+    {%- else -%}
+      {%- set tool_string = ns.tool_string + "</available_tools>\n\n" ~
+            "For each function call, return an unescaped XML-like object " ~
+            "with function name and arguments within '<tool_call>' and '</tool_call>' tags, like here:\n" ~
+            "<tool_call>function-name\n<arg_key>argument-key</arg_key>\n<arg_value>value-of-argument-key</arg_value>\n" ~
+            "</tool_call>" -%}
+    {%- endif -%}
+    {{- tool_string -}}
+  {%- endif -%}
+
+  {{- "\n</system>\n" -}}
+{%- endif -%}
+
+{#- ───── main loop ───── -#}
+{%- for message in messages -%}
+  {%- set content = message.content if message.content is string else "" -%}
+  {%- if message.role == "user" -%}
+    {{- "<user>\n" + content + "\n</user>\n" -}}
+  {%- elif message.role == "assistant" -%}
+    {%- generation -%}
+      {{- "<assistant>\n" -}}
+      {%- if render_assistant_messages_raw -%}
+        {#- Raw mode: prepend the generation prompt token, then dump content verbatim. -#}
+        {#- The generation prompt is <think> when enable_thinking, </think> otherwise. -#}
+        {#- Only prepend if content doesn't already start with it. -#}
+        {%- if enable_thinking -%}
+          {%- if not content.startswith('<think>') -%}
+            {{- '<think>' -}}
+          {%- endif -%}
+        {%- else -%}
+          {%- if not content.startswith('</think>') -%}
+            {{- '</think>' -}}
+          {%- endif -%}
+        {%- endif -%}
+        {{- content -}}
+        {#- Append closing tag if content doesn't already end with it. -#}
+        {%- if not content.endswith('</assistant>\n') and not content.endswith('</assistant>') -%}
+          {{- '\n</assistant>' -}}
+        {%- endif -%}
+        {{- "\n" -}}
+      {%- else -%}
+        {#- Extract reasoning content from message.reasoning (vLLM field name) or message.reasoning_content, or from <think> tags -#}
+        {%- set reasoning_content = '' %}
+        {%- if message.reasoning is string %}
+          {%- set reasoning_content = message.reasoning %}
+        {%- elif message.reasoning_content is string %}
+          {%- set reasoning_content = message.reasoning_content %}
+        {%- endif %}
+        {#- Always strip <think> tags from content if present to avoid duplication -#}
+        {%- if '</think>' in content %}
+          {%- if not reasoning_content %}
+            {%- set reasoning_content = content.split('</think>')[0].rstrip('\n').split('<think>')[-1].lstrip('\n') %}
+          {%- endif %}
+          {%- set content = content.split('</think>')[-1].lstrip('\n') %}
+        {%- endif %}
+        {#- Display reasoning content for all messages -#}
+        {%- if reasoning_content -%}
+          {{- '<think>\n' + reasoning_content.strip() + '\n</think>\n' -}}
+        {%- else -%}
+          {{- '</think>\n' -}}
+        {%- endif -%}
+        {#- Display main content -#}
+        {%- if content.strip() -%}
+          {{- content.strip() ~ "\n" -}}
+        {%- endif -%}
+        {%- if message.tool_calls -%}
+          {%- for tool_call in message.tool_calls -%}
+            {%- set function_data = tool_call.function -%}
+            {{- '<tool_call>' + function_data.name }}
+            {% set _args = function_data.arguments %}
+            {%- for k, v in _args.items() -%}
+              {{- "<arg_key>" ~ k ~ "</arg_key>\n" -}}
+              {{- "<arg_value>"}}{{ v | tojson(ensure_ascii=False) if v is not string else v }}{{ "</arg_value>\n" -}}
+            {%- endfor -%}
+            {{- "</tool_call>\n" -}}
+          {%- endfor -%}
+        {%- endif -%}
+        {{- "</assistant>\n" -}}
+      {%- endif -%}
+    {%- endgeneration -%}
+  {%- elif message.role == "tool" -%}
+    {{- "<tool_response>\n" + content + "\n</tool_response>\n" -}}
+  {%- elif message.role == "system" and loop.index0 != 0 -%}
+    {#- Render additional system messages (skip the first one which is handled separately in the header) -#}
+    {{- "<system>\n" + content + "\n</system>\n" -}}
+  {%- endif -%}
+{%- endfor -%}
+{#- ───── generation prompt ───── -#}
+{%- if add_generation_prompt -%}
+  {{- "<assistant>\n" -}}
+  {#- ───── Include reasoning mode directive ───── -#}
+  {%- if not enable_thinking %}
+    {{- '</think>' -}}
+  {%- else %}
+    {{- '<think>' -}}
+  {%- endif %}
+{%- endif -%}

src/llama-arch.cpp

@@ -108,6 +108,7 @@ static const std::map<llm_arch, const char *> LLM_ARCH_NAMES = {
     { LLM_ARCH_DOTS1,            "dots1"            },
     { LLM_ARCH_ARCEE,            "arcee"            },
     { LLM_ARCH_AFMOE,            "afmoe"            },
+    { LLM_ARCH_LAGUNA,           "laguna"           },
     { LLM_ARCH_ERNIE4_5,         "ernie4_5"         },
     { LLM_ARCH_ERNIE4_5_MOE,     "ernie4_5-moe"     },
     { LLM_ARCH_HUNYUAN_MOE,      "hunyuan-moe"      },
@@ -665,7 +666,7 @@ static const std::map<llm_tensor, llm_tensor_info> LLM_TENSOR_INFOS = {
     {LLM_TENSOR_HC_FFN_SCALE,               {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL}},
     {LLM_TENSOR_ATTN_COMPRESSOR_WKV,        {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL_MAT}},
     {LLM_TENSOR_ATTN_COMPRESSOR_WGATE,      {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL_MAT}},
-    {LLM_TENSOR_ATTN_COMPRESSOR_APE,        {LLM_TENSOR_LAYER_REPEATING, GGML_OP_ADD}},
+    {LLM_TENSOR_ATTN_COMPRESSOR_APE,        {LLM_TENSOR_LAYER_REPEATING, GGML_OP_GET_ROWS}},
     {LLM_TENSOR_ATTN_COMPRESSOR_NORM,       {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL}},
     {LLM_TENSOR_ATTN_K_B,                   {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL_MAT}},
     {LLM_TENSOR_ATTN_V_B,                   {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL_MAT}},
@@ -832,7 +833,7 @@ static const std::map<llm_tensor, llm_tensor_info> LLM_TENSOR_INFOS = {
     {LLM_TENSOR_INDEXER_ATTN_Q_B,           {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL_MAT}},
     {LLM_TENSOR_INDEXER_COMPRESSOR_WKV,     {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL_MAT}},
     {LLM_TENSOR_INDEXER_COMPRESSOR_WGATE,   {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL_MAT}},
-    {LLM_TENSOR_INDEXER_COMPRESSOR_APE,     {LLM_TENSOR_LAYER_REPEATING, GGML_OP_ADD}},
+    {LLM_TENSOR_INDEXER_COMPRESSOR_APE,     {LLM_TENSOR_LAYER_REPEATING, GGML_OP_GET_ROWS}},
     {LLM_TENSOR_INDEXER_COMPRESSOR_NORM,    {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL}},
     {LLM_TENSOR_FFN_GATE_TID2EID,           {LLM_TENSOR_LAYER_REPEATING, GGML_OP_GET_ROWS}},
     {LLM_TENSOR_NEXTN_PROJ_PRE,             {LLM_TENSOR_LAYER_REPEATING, GGML_OP_MUL_MAT}},

src/llama-arch.h

@@ -113,6 +113,7 @@ enum llm_arch {
     LLM_ARCH_DOTS1,
     LLM_ARCH_ARCEE,
     LLM_ARCH_AFMOE,
+    LLM_ARCH_LAGUNA,
     LLM_ARCH_ERNIE4_5,
     LLM_ARCH_ERNIE4_5_MOE,
     LLM_ARCH_HUNYUAN_MOE,

src/llama-model-saver.cpp

@@ -28,6 +28,7 @@ bool llama_model_saver_supports_arch(llm_arch arch) {
         case LLM_ARCH_MIMO2:
         case LLM_ARCH_STEP35:
         case LLM_ARCH_MELLUM:
+        case LLM_ARCH_LAGUNA:
             return false;
         default:
             return true;

src/llama-model.cpp

@@ -250,6 +250,8 @@ static llama_model * llama_model_mapping(llm_arch arch, const llama_model_params
             return new llama_model_arcee(params);
         case LLM_ARCH_AFMOE:
             return new llama_model_afmoe(params);
+        case LLM_ARCH_LAGUNA:
+            return new llama_model_laguna(params);
         case LLM_ARCH_ERNIE4_5:
             return new llama_model_ernie4_5(params);
         case LLM_ARCH_ERNIE4_5_MOE:
@@ -2549,6 +2551,7 @@ llama_rope_type llama_model_rope_type(const llama_model * model) {
         case LLM_ARCH_COGVLM:
         case LLM_ARCH_PANGU_EMBED:
         case LLM_ARCH_AFMOE:
+        case LLM_ARCH_LAGUNA:
         case LLM_ARCH_QWEN3NEXT:
         case LLM_ARCH_MIMO2:
         case LLM_ARCH_STEP35:

src/llama-vocab.cpp

@@ -496,6 +496,12 @@ struct llm_tokenizer_bpe : llm_tokenizer {
                     "[!\"#$%&'()*+,\\-./:;<=>?@\\[\\\\\\]^_`{|}~][A-Za-z]+|[^\\r\\n\\p{L}\\p{P}\\p{S}]?[\\p{L}\\p{M}]+| ?[\\p{P}\\p{S}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
                 };
                 break;
+            case LLAMA_VOCAB_PRE_TYPE_LAGUNA:
+                regex_exprs = {
+                    "[^\\n]+|[\\n]+",
+                    "(?:'[sS]|'[tT]|'[rR][eE]|'[vV][eE]|'[mM]|'[lL][lL]|'[dD])|[^\\r\\n\\p{L}\\p{N}]?\\p{L}+|\\p{N}| ?[^\\s\\p{L}\\p{N}]+[\\r\\n]*|\\s*[\\r\\n]+|\\s+(?!\\S)|\\s+",
+                };
+                break;
             case LLAMA_VOCAB_PRE_TYPE_EXAONE_MOE:
                 regex_exprs = {
                     // original regex from tokenizer.json
@@ -2342,6 +2348,10 @@ void llama_vocab::impl::load(llama_model_loader & ml, const LLM_KV & kv) {
                 tokenizer_pre == "afmoe") {
                 pre_type = LLAMA_VOCAB_PRE_TYPE_AFMOE;
                 clean_spaces = false;
+            } else if (
+                tokenizer_pre == "laguna") {
+                pre_type = LLAMA_VOCAB_PRE_TYPE_LAGUNA;
+                clean_spaces = false;
             } else if (
                 tokenizer_pre == "minimax-m2") {
                 pre_type = LLAMA_VOCAB_PRE_TYPE_MINIMAX_M2;

src/llama-vocab.h

@@ -64,6 +64,7 @@ enum llama_vocab_pre_type {
     LLAMA_VOCAB_PRE_TYPE_WHITESPACE        = 53,
     LLAMA_VOCAB_PRE_TYPE_GRANITE_EMB_MULTI = 54,
     LLAMA_VOCAB_PRE_TYPE_MELLUM2           = 55,
+    LLAMA_VOCAB_PRE_TYPE_LAGUNA            = 56,
 };
 
 struct LLM_KV;

src/models/laguna.cpp

@@ -0,0 +1,332 @@
+// Laguna (poolside): sigmoid-routed MoE with a score-correction bias, one shared
+// expert, a softplus attention output gate, QK-norm, and per-layer-type RoPE
+// (YaRN on full-attention layers, plain RoPE on sliding-window layers). XS.2 is
+// hybrid full/SWA with a per-head gate; M.1 is full-attention with a per-element
+// gate. Shares the MoE/gate structure with afmoe.
+
+#include "models.h"
+
+void llama_model_laguna::load_arch_hparams(llama_model_loader & ml) {
+    ml.get_key(LLM_KV_ATTENTION_LAYERNORM_RMS_EPS, hparams.f_norm_rms_eps);
+    ml.get_key(LLM_KV_LEADING_DENSE_BLOCK_COUNT,   hparams.n_layer_dense_lead);
+    ml.get_key(LLM_KV_EXPERT_FEED_FORWARD_LENGTH,  hparams.n_ff_exp);
+    ml.get_key(LLM_KV_EXPERT_GATING_FUNC,          hparams.expert_gating_func, false);
+    ml.get_key(LLM_KV_EXPERT_WEIGHTS_SCALE,        hparams.expert_weights_scale, false);
+    ml.get_key(LLM_KV_EXPERT_WEIGHTS_NORM,         hparams.expert_weights_norm, false);
+
+    // Laguna ships one shared expert and stores its size directly (routed and
+    // shared experts may differ), so read the size from expert_shared_feed_forward_length.
+    // The count is not in the config; default to 1 but read the key if present.
+    hparams.n_expert_shared = 1;
+    ml.get_key(LLM_KV_EXPERT_SHARED_COUNT,               hparams.n_expert_shared, false);
+    ml.get_key(LLM_KV_EXPERT_SHARED_FEED_FORWARD_LENGTH, hparams.n_ff_shexp, false);
+    if (hparams.n_ff_shexp == 0) {
+        // Weightless fixtures (test-llama-archs) omit this key; derive a nonzero
+        // size so the shared expert is still built. Real GGUFs always carry the
+        // exact value (routed and shared FF lengths may differ).
+        hparams.n_ff_shexp = hparams.n_ff_exp * hparams.n_expert_shared;
+    }
+
+    // Sliding-window attention is OPTIONAL. XS.2 is hybrid (full / SWA / SWA /
+    // SWA repeating, period 4 starting with full); M.1 has no sliding window
+    // (all layers full attention). When sliding_window is absent or zero we
+    // leave swa_type = NONE and skip the SWA-specific per-layer-type RoPE.
+    hparams.n_swa = 0;
+    ml.get_key(LLM_KV_ATTENTION_SLIDING_WINDOW, hparams.n_swa, false);
+    if (hparams.n_swa > 0) {
+        hparams.swa_type = LLAMA_SWA_TYPE_STANDARD;
+
+        uint32_t swa_period = 4;
+        ml.get_key_or_arr(LLM_KV_ATTENTION_SLIDING_WINDOW_PATTERN, swa_period, false);
+        hparams.set_swa_pattern(swa_period, /*dense_first=*/true);  // XS.2: FULL at il%4==0
+
+        // Per-layer-type RoPE: full layers use YaRN θ=500000 over 64 dims;
+        // SWA layers use default RoPE θ=10000 over 128 dims. Base load_hparams
+        // already reads ROPE_FREQ_BASE and ROPE_DIMENSION_COUNT into the
+        // non-SWA fields; we explicitly pull the SWA mirrors here.
+        hparams.rope_freq_base_train_swa  = hparams.rope_freq_base_train;
+        hparams.rope_freq_scale_train_swa = 1.0f;  // SWA uses plain RoPE (no YaRN scaling); do NOT inherit full layers 1/factor
+        ml.get_key(LLM_KV_ROPE_FREQ_BASE_SWA, hparams.rope_freq_base_train_swa, false);
+        ml.get_key(LLM_KV_ROPE_DIMENSION_COUNT_SWA, hparams.n_rot_swa, false);
+    }
+
+    // Default the expert gating function to SIGMOID when the key is absent
+    // (matches the HF reference).
+    if (hparams.expert_gating_func == LLAMA_EXPERT_GATING_FUNC_TYPE_NONE) {
+        hparams.expert_gating_func = LLAMA_EXPERT_GATING_FUNC_TYPE_SIGMOID;
+    }
+
+    switch (hparams.n_layer()) {
+        case 40: type = LLM_TYPE_30B_A3B;   break;  // Laguna-XS.2
+        case 70: type = LLM_TYPE_230B_A10B; break;  // Laguna-M.1
+        default: type = LLM_TYPE_UNKNOWN;
+    }
+}
+
+void llama_model_laguna::load_arch_tensors(llama_model_loader & ml) {
+    LLAMA_LOAD_LOCALS;
+
+    tok_embd = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, 0);
+
+    output_norm = create_tensor(tn(LLM_TENSOR_OUTPUT_NORM, "weight"), {n_embd}, 0);
+    output      = create_tensor(tn(LLM_TENSOR_OUTPUT,      "weight"), {n_embd, n_vocab}, TENSOR_NOT_REQUIRED);
+    if (output == NULL) {
+        // tied embeddings fallback
+        output = create_tensor(tn(LLM_TENSOR_TOKEN_EMBD, "weight"), {n_embd, n_vocab}, TENSOR_DUPLICATED);
+    }
+
+    const int64_t n_ff_exp   = hparams.n_ff_exp;
+    const int64_t n_ff_shexp = hparams.n_ff_shexp;
+
+    for (int i = 0; i < n_layer; ++i) {
+        auto & layer = layers[i];
+
+        // Per-layer head count — Laguna varies n_head between full and SWA
+        // layers (48 vs 64 in XS.2). KV head count is uniform.
+        const int64_t n_head_il    = hparams.n_head(i);
+        const int64_t n_head_kv_il = hparams.n_head_kv(i);
+        const int64_t n_embd_q_il  = n_embd_head_k * n_head_il;
+        const int64_t n_embd_k_il  = n_embd_head_k * n_head_kv_il;
+        const int64_t n_embd_v_il  = n_embd_head_v * n_head_kv_il;
+
+        layer.attn_norm = create_tensor(tn(LLM_TENSOR_ATTN_NORM, "weight", i), {n_embd}, 0);
+
+        create_tensor_qkv(layer, i, n_embd, n_embd_q_il, n_embd_k_il, n_embd_v_il, 0);
+        layer.wo = create_tensor(tn(LLM_TENSOR_ATTN_OUT, "weight", i), {n_embd_q_il, n_embd}, 0);
+
+        layer.attn_q_norm = create_tensor(tn(LLM_TENSOR_ATTN_Q_NORM, "weight", i), {n_embd_head_k}, 0);
+        layer.attn_k_norm = create_tensor(tn(LLM_TENSOR_ATTN_K_NORM, "weight", i), {n_embd_head_k}, 0);
+
+        // Attention output gate. XS.2 is per-head (g_proj -> n_head, one scalar
+        // per head broadcast over head_dim at multiply time); M.1 is per-element
+        // (g_proj -> n_head*head_dim, like afmoe). Detect from the stored tensor
+        // shape so a single arch handles both; the graph mirrors this check.
+        // Gate width selects per-head vs per-element. Real GGUFs always carry the
+        // gate tensor, so read the width from it and require EXACTLY one of the two
+        // valid widths -- never guess between them. Weightless fixtures
+        // (test-llama-archs) have no gate tensor; fall back to the per-head layout so
+        // the per-head reshape path is still exercised.
+        const int64_t n_gate_per_head = n_head_il;
+        const int64_t n_gate_per_elem = n_embd_head_k * n_head_il;
+        const ggml_tensor * gate_meta = ml.get_tensor_meta(tn(LLM_TENSOR_ATTN_GATE, "weight", i).str().c_str());
+        int64_t n_gate_out;
+        if (gate_meta != nullptr) {
+            n_gate_out = gate_meta->ne[1];
+            if (n_gate_out != n_gate_per_head && n_gate_out != n_gate_per_elem) {
+                GGML_ABORT("Laguna: unexpected attention gate width %lld at layer %d "
+                           "(expected %lld per-head or %lld per-element)",
+                           (long long) n_gate_out, i, (long long) n_gate_per_head, (long long) n_gate_per_elem);
+            }
+        } else {
+            n_gate_out = n_gate_per_head;
+        }
+        layer.wqkv_gate = create_tensor(tn(LLM_TENSOR_ATTN_GATE, "weight", i), {n_embd, n_gate_out}, 0);
+
+        layer.ffn_norm = create_tensor(tn(LLM_TENSOR_FFN_NORM, "weight", i), {n_embd}, 0);
+
+        if ((uint32_t)i >= hparams.n_layer_dense_lead) {
+            // MoE layer
+            layer.ffn_gate_inp    = create_tensor(tn(LLM_TENSOR_FFN_GATE_INP,    "weight", i), {n_embd, n_expert}, 0);
+            layer.ffn_exp_probs_b = create_tensor(tn(LLM_TENSOR_FFN_EXP_PROBS_B, "bias",   i), {n_expert}, 0);
+
+            layer.ffn_gate_exps = create_tensor(tn(LLM_TENSOR_FFN_GATE_EXPS, "weight", i), {n_embd,   n_ff_exp, n_expert}, 0);
+            layer.ffn_up_exps   = create_tensor(tn(LLM_TENSOR_FFN_UP_EXPS,   "weight", i), {n_embd,   n_ff_exp, n_expert}, 0);
+            layer.ffn_down_exps = create_tensor(tn(LLM_TENSOR_FFN_DOWN_EXPS, "weight", i), {n_ff_exp, n_embd,   n_expert}, 0);
+
+            // Always-on shared expert.
+            layer.ffn_gate_shexp = create_tensor(tn(LLM_TENSOR_FFN_GATE_SHEXP, "weight", i), {n_embd,    n_ff_shexp}, 0);
+            layer.ffn_up_shexp   = create_tensor(tn(LLM_TENSOR_FFN_UP_SHEXP,   "weight", i), {n_embd,    n_ff_shexp}, 0);
+            layer.ffn_down_shexp = create_tensor(tn(LLM_TENSOR_FFN_DOWN_SHEXP, "weight", i), {n_ff_shexp, n_embd},    0);
+        } else {
+            // Dense layer (the leading n_layer_dense_lead layers)
+            layer.ffn_gate = create_tensor(tn(LLM_TENSOR_FFN_GATE, "weight", i), {n_embd, n_ff}, 0);
+            layer.ffn_up   = create_tensor(tn(LLM_TENSOR_FFN_UP,   "weight", i), {n_embd, n_ff}, 0);
+            layer.ffn_down = create_tensor(tn(LLM_TENSOR_FFN_DOWN, "weight", i), {n_ff,   n_embd}, 0);
+        }
+    }
+}
+
+std::unique_ptr<llm_graph_context> llama_model_laguna::build_arch_graph(const llm_graph_params & params) const {
+    return std::make_unique<graph>(*this, params);
+}
+
+llama_model_laguna::graph::graph(const llama_model & model, const llm_graph_params & params) : llm_graph_context(params) {
+    const int64_t n_embd_head = hparams.n_embd_head_v();
+    GGML_ASSERT(n_embd_head == hparams.n_embd_head_k());
+
+    ggml_tensor * cur;
+    ggml_tensor * inpL;
+
+    inpL = build_inp_embd(model.tok_embd);
+    // No MuP embedding scale (laguna omits this; afmoe scales by sqrt(hidden)).
+
+    ggml_tensor * inp_pos = build_inp_pos();
+    // XS.2 is hybrid SWA -> interleaved-SWA KV input; M.1 is all-full -> plain
+    // KV input. Pick the matching input (and build_attn overload) per swa_type.
+    const bool has_swa = hparams.swa_type != LLAMA_SWA_TYPE_NONE;
+    llm_graph_input_attn_kv      * inp_attn_kv   = has_swa ? nullptr : build_attn_inp_kv();
+    llm_graph_input_attn_kv_iswa * inp_attn_iswa = has_swa ? build_attn_inp_kv_iswa() : nullptr;
+    ggml_tensor * inp_out_ids = build_inp_out_ids();
+
+    const float kq_scale = 1.0f / sqrtf(float(n_embd_head));
+
+    for (int il = 0; il < n_layer; ++il) {
+        const bool    is_swa_il   = hparams.is_swa(il);
+        const int64_t n_head_il   = hparams.n_head(il);
+        const int64_t n_head_kv_il = hparams.n_head_kv(il);
+
+        // Per-layer-type RoPE config. SWA layers run plain rope (no YaRN),
+        // achieved by zeroing the YaRN ext/beta params for those layers.
+        const int   n_rot_l       = is_swa_il ? hparams.n_rot_swa : n_rot;
+        const float freq_base_l   = is_swa_il ? hparams.rope_freq_base_train_swa : freq_base;
+        const float freq_scale_l  = is_swa_il ? hparams.rope_freq_scale_train_swa : freq_scale;
+        const float ext_factor_l  = is_swa_il ? 0.0f : ext_factor;
+        // YaRN magnitude scaling (mscale) is already handled by the framework:
+        // llama_context pre-divides cparams.yarn_attn_factor by (1 + 0.1*ln(factor))
+        // to cancel ggml rope_yarn's internal mscale *= 1 + 0.1*ln(1/freq_scale).
+        // Pass attn_factor straight through (like every other arch); SWA layers run
+        // plain RoPE (ext_factor 0, no mscale) so force 1.0 there.
+        const float attn_factor_l = is_swa_il ? 1.0f : attn_factor;
+        const float beta_fast_l   = is_swa_il ? 0.0f : beta_fast;
+        const float beta_slow_l   = is_swa_il ? 0.0f : beta_slow;
+        const int   n_ctx_orig_l  = is_swa_il ? hparams.n_ctx_train : n_ctx_orig;
+
+        ggml_tensor * inpSA = inpL;
+
+        // Pre-norm
+        cur = build_norm(inpL, model.layers[il].attn_norm, NULL, LLM_NORM_RMS, il);
+        cb(cur, "attn_norm", il);
+
+        // Self-attention
+        {
+            ggml_tensor * attn_inp = cur;  // saved for the gate projection
+
+            auto [Qcur, Kcur, Vcur] = build_qkv(model.layers[il], cur,
+                    n_embd_head, n_head_il, n_head_kv_il, il);
+
+            // g_proj on the *pre-attention* hidden state (matches HF
+            // reference: gate is computed from the same `hidden_states`
+            // input as q/k/v, not from the attn output).
+            ggml_tensor * gate = build_lora_mm(model.layers[il].wqkv_gate, attn_inp);
+            cb(gate, "attn_gate_proj", il);
+
+            // QK RMSNorm at head_dim level (Qwen3 style)
+            Qcur = build_norm(Qcur, model.layers[il].attn_q_norm, NULL, LLM_NORM_RMS, il);
+            Kcur = build_norm(Kcur, model.layers[il].attn_k_norm, NULL, LLM_NORM_RMS, il);
+            cb(Qcur, "Qcur_normed", il);
+            cb(Kcur, "Kcur_normed", il);
+
+            Qcur = ggml_rope_ext(ctx0, Qcur, inp_pos, nullptr,
+                    n_rot_l, rope_type, n_ctx_orig_l, freq_base_l, freq_scale_l,
+                    ext_factor_l, attn_factor_l, beta_fast_l, beta_slow_l);
+            Kcur = ggml_rope_ext(ctx0, Kcur, inp_pos, nullptr,
+                    n_rot_l, rope_type, n_ctx_orig_l, freq_base_l, freq_scale_l,
+                    ext_factor_l, attn_factor_l, beta_fast_l, beta_slow_l);
+            cb(Qcur, "Qcur_rope", il);
+            cb(Kcur, "Kcur_rope", il);
+
+            cur = has_swa
+                ? build_attn(inp_attn_iswa,
+                        NULL, NULL, NULL,    // o_proj deferred until after gating
+                        Qcur, Kcur, Vcur, nullptr, nullptr, nullptr, kq_scale, il)
+                : build_attn(inp_attn_kv,
+                        NULL, NULL, NULL,
+                        Qcur, Kcur, Vcur, nullptr, nullptr, nullptr, kq_scale, il);
+            cb(cur, "attn_out", il);
+
+            // Softplus output gate (the unary kernel computes softplus in fp32
+            // and casts back). Two shapes, distinguished by the g_proj output
+            // dim (matching the load-time detection):
+            //   XS.2 per-head     : gate [n_head_il, n_tokens] -> reshape to
+            //                       [1, n_head_il, n_tokens] and broadcast over
+            //                       head_dim against cur [head_dim, n_head, T].
+            //   M.1  per-element  : gate [n_head_il*head_dim, n_tokens] spans the
+            //                       full attention output -> direct ggml_mul.
+            gate = ggml_softplus(ctx0, gate);
+            cb(gate, "attn_gate_softplus", il);
+
+            const int64_t n_tokens = cur->ne[1];
+            if (model.layers[il].wqkv_gate->ne[1] == n_head_il) {
+                cur  = ggml_reshape_3d(ctx0, cur,  n_embd_head, n_head_il, n_tokens);
+                gate = ggml_reshape_3d(ctx0, gate, 1,           n_head_il, n_tokens);
+                cur  = ggml_mul(ctx0, cur, gate);
+                cur  = ggml_reshape_2d(ctx0, cur, n_embd_head * n_head_il, n_tokens);
+            } else {
+                cur = ggml_mul(ctx0, cur, gate);
+            }
+            cb(cur, "attn_gated", il);
+
+            cur = build_lora_mm(model.layers[il].wo, cur, model.layers[il].wo_s);
+            cb(cur, "attn_o_proj", il);
+        }
+
+        if (il == n_layer - 1 && inp_out_ids) {
+            cur   = ggml_get_rows(ctx0,   cur, inp_out_ids);
+            inpSA = ggml_get_rows(ctx0, inpSA, inp_out_ids);
+        }
+
+        ggml_tensor * ffn_inp = ggml_add(ctx0, cur, inpSA);
+        cb(ffn_inp, "ffn_inp", il);
+
+        // Pre-norm only (no post-attn norm)
+        cur = build_norm(ffn_inp, model.layers[il].ffn_norm, NULL, LLM_NORM_RMS, il);
+        cb(cur, "ffn_norm", il);
+
+        if ((uint32_t)il >= hparams.n_layer_dense_lead) {
+            // MoE: sigmoid routing + score-correction bias + sum-norm +
+            // routed_scaling_factor (all handled by build_moe_ffn).
+            ggml_tensor * moe_out = build_moe_ffn(cur,
+                    model.layers[il].ffn_gate_inp,
+                    model.layers[il].ffn_up_exps,
+                    model.layers[il].ffn_gate_exps,
+                    model.layers[il].ffn_down_exps,
+                    model.layers[il].ffn_exp_probs_b,
+                    n_expert, n_expert_used,
+                    LLM_FFN_SILU,
+                    hparams.expert_weights_norm,
+                    hparams.expert_weights_scale,
+                    (llama_expert_gating_func_type) hparams.expert_gating_func,
+                    il);
+            cb(moe_out, "ffn_moe_out", il);
+
+            // Always-on shared expert, summed in parallel.
+            ggml_tensor * ffn_shexp = build_ffn(cur,
+                    model.layers[il].ffn_up_shexp,   NULL, NULL,
+                    model.layers[il].ffn_gate_shexp, NULL, NULL,
+                    model.layers[il].ffn_down_shexp, NULL, NULL,
+                    NULL,
+                    LLM_FFN_SILU, LLM_FFN_PAR, il);
+            cb(ffn_shexp, "ffn_shexp", il);
+
+            cur = ggml_add(ctx0, moe_out, ffn_shexp);
+            cb(cur, "ffn_out", il);
+        } else {
+            // Dense FFN for the leading n_layer_dense_lead layers (XS.2: 1, M.1: 3)
+            cur = build_ffn(cur,
+                    model.layers[il].ffn_up,   NULL, NULL,
+                    model.layers[il].ffn_gate, NULL, NULL,
+                    model.layers[il].ffn_down, NULL, NULL,
+                    NULL,
+                    LLM_FFN_SILU, LLM_FFN_PAR, il);
+            cb(cur, "ffn_out", il);
+        }
+
+        // No post-ffn norm
+        cur = ggml_add(ctx0, cur, ffn_inp);
+        cur = build_cvec(cur, il);
+        cb(cur, "l_out", il);
+
+        inpL = cur;
+    }
+
+    cur = inpL;
+    cur = build_norm(cur, model.output_norm, NULL, LLM_NORM_RMS, -1);
+    cb(cur, "result_norm", -1);
+    res->t_embd = cur;
+
+    cur = build_lora_mm(model.output, cur);
+    cb(cur, "result_output", -1);
+    res->t_logits = cur;
+
+    ggml_build_forward_expand(gf, cur);
+}

src/models/models.h

@@ -1681,6 +1681,19 @@ struct llama_model_afmoe : public llama_model_base {
 };
 
 
+struct llama_model_laguna : public llama_model_base {
+    llama_model_laguna(const struct llama_model_params & params) : llama_model_base(params) {}
+    void load_arch_hparams(llama_model_loader & ml) override;
+    void load_arch_tensors(llama_model_loader & ml) override;
+
+    struct graph : public llm_graph_context {
+        graph(const llama_model & model, const llm_graph_params & params);
+    };
+
+    std::unique_ptr<llm_graph_context> build_arch_graph(const llm_graph_params & params) const override;
+};
+
+
 struct llama_model_ernie4_5 : public llama_model_base {
     llama_model_ernie4_5(const struct llama_model_params & params) : llama_model_base(params) {}
     void load_arch_hparams(llama_model_loader & ml) override;

tests/test-backend-ops.cpp

@@ -5960,6 +5960,7 @@ enum MoeGatingFunc {
     GATING_FUNC_SOFTMAX,
     GATING_FUNC_SIGMOID,
     GATING_FUNC_SOFTMAX_WEIGHT,
+    GATING_FUNC_SQRT_SOFTPLUS,
 };
 
 struct test_topk_moe : public test_case {
@@ -6003,7 +6004,8 @@ struct test_topk_moe : public test_case {
         ggml_tensor * logits = ggml_new_tensor(ctx, GGML_TYPE_F32, 4, ne.data());
         ggml_tensor * probs            =
             (gating_func == GATING_FUNC_SOFTMAX) ? ggml_soft_max(ctx, logits) :
-            (gating_func == GATING_FUNC_SIGMOID) ? ggml_sigmoid(ctx, logits) : logits;
+            (gating_func == GATING_FUNC_SIGMOID) ? ggml_sigmoid(ctx, logits) :
+            (gating_func == GATING_FUNC_SQRT_SOFTPLUS) ? ggml_sqrt(ctx, ggml_softplus(ctx, logits)) : logits;
         ggml_set_name(probs, "probs");
 
         ggml_tensor * selection_probs = probs;
@@ -9584,7 +9586,7 @@ static std::vector<std::unique_ptr<test_case>> make_test_cases_eval() {
         }
     }
 
-    for (auto gate : {GATING_FUNC_SOFTMAX, GATING_FUNC_SIGMOID, GATING_FUNC_SOFTMAX_WEIGHT}) {
+    for (auto gate : {GATING_FUNC_SOFTMAX, GATING_FUNC_SIGMOID, GATING_FUNC_SOFTMAX_WEIGHT, GATING_FUNC_SQRT_SOFTPLUS}) {
         for (bool with_norm : {false, true}) {
             for (bool bias_probs : {false, true}) {
                 for (float scale_w : {0.0f, 2.0f}) {
@@ -9596,6 +9598,7 @@ static std::vector<std::unique_ptr<test_case>> make_test_cases_eval() {
                     test_cases.emplace_back(new test_topk_moe({128, 1, 1, 1}, 128, with_norm, bias_probs, gate, scale_w));
                     test_cases.emplace_back(new test_topk_moe({129, 1, 1, 1}, 128, with_norm, bias_probs, gate, scale_w));
                     test_cases.emplace_back(new test_topk_moe({160, 4, 1, 1}, 160, with_norm, bias_probs, gate, scale_w));
+                    test_cases.emplace_back(new test_topk_moe({256, 22, 1, 1}, 6, with_norm, bias_probs, gate, scale_w)); // Used by DeepSeek-V4
                     test_cases.emplace_back(new test_topk_moe({288, 22, 1, 1}, 8, with_norm, bias_probs, gate, scale_w)); // Used by StepFun 3.7
                 }
             }

tests/test-chat-auto-parser.cpp

@@ -57,6 +57,15 @@ static void test_seed_oss_tool_with_reasoning(testing & t);
 static void test_nemotron_analysis(testing & t);
 static void test_nemotron_reasoning_detection(testing & t);
 static void test_nemotron_tool_format(testing & t);
+static void test_laguna_analysis(testing & t);
+static void test_laguna_reasoning_detection(testing & t);
+static void test_laguna_tool_format(testing & t);
+static void test_laguna_s_analysis(testing & t);
+static void test_laguna_s_reasoning_detection(testing & t);
+static void test_laguna_s_tool_format(testing & t);
+static void test_laguna_xs2_analysis(testing & t);
+static void test_laguna_xs2_reasoning_detection(testing & t);
+static void test_laguna_xs2_tool_format(testing & t);
 
 // CohereForAI template analysis tests
 static void test_cohere_reasoning_detection(testing & t);
@@ -101,6 +110,9 @@ int main(int argc, char * argv[]) {
     t.test("seed_oss_diffs", test_seed_oss_tool_analysis);
     t.test("cohere", test_cohere_analysis);
     t.test("nemotron", test_nemotron_analysis);
+    t.test("laguna", test_laguna_analysis);
+    t.test("laguna-s", test_laguna_s_analysis);
+    t.test("laguna-xs2", test_laguna_xs2_analysis);
     t.test("smollm3", test_smollm3_analysis);
     t.test("standard_json_tools", test_standard_json_tools_formats);
     t.test("normalize_quotes_to_json", test_normalize_quotes_to_json);
@@ -1378,6 +1390,94 @@ static void test_nemotron_tool_format(testing & t) {
     t.assert_true("should support tools", analysis.jinja_caps.supports_tools);
 }
 
+// ============================================================================
+// Laguna Template Analysis Tests
+// ============================================================================
+static common_chat_template load_laguna_template(testing & t) {
+    return load_template(t, "models/templates/poolside-Laguna-XS-2.1.jinja");
+}
+
+static void test_laguna_reasoning_detection(testing & t) {
+    common_chat_template tmpl = load_laguna_template(t);
+    struct autoparser analysis;
+    analysis.analyze_template(tmpl);
+    // Laguna's template renders reasoning delimiters with formatting whitespace
+    // ("<think>\n") that the model does not emit; the Laguna patch trims them.
+    t.assert_equal("reasoning_start should be '<think>'", "<think>", analysis.reasoning.start);
+    t.assert_equal("reasoning_end should be '</think>'", "</think>", analysis.reasoning.end);
+    t.assert_equal("reasoning should be TAG_BASED", reasoning_mode::TAG_BASED, analysis.reasoning.mode);
+}
+
+static void test_laguna_tool_format(testing & t) {
+    common_chat_template tmpl = load_laguna_template(t);
+    struct autoparser analysis;
+    analysis.analyze_template(tmpl);
+    t.assert_equal("arg_value_suffix should be '</arg_value>'", "</arg_value>", analysis.tools.arguments.value_suffix);
+}
+
+static void test_laguna_stop_string(testing & t) {
+    // The </assistant> turn terminator can be emitted as ordinary text tokens
+    // (not the single eot token), so it must also be a literal stop string.
+    common_chat_template tmpl = load_laguna_template(t);
+    struct autoparser analysis;
+    analysis.analyze_template(tmpl);
+    bool has_stop = false;
+    for (const auto & stop : analysis.additional_stops) {
+        if (stop == "</assistant>") { has_stop = true; break; }
+    }
+    t.assert_true("Laguna additional_stops contains </assistant>", has_stop);
+}
+
+static void test_laguna_analysis(testing & t) {
+    t.test("Laguna reasoning detection", test_laguna_reasoning_detection);
+    t.test("Laguna tool format", test_laguna_tool_format);
+    t.test("Laguna stop string", test_laguna_stop_string);
+}
+
+static common_chat_template load_laguna_s_template(testing & t) {
+    return load_template(t, "models/templates/poolside-Laguna-S-2.1.jinja");
+}
+static void test_laguna_s_reasoning_detection(testing & t) {
+    common_chat_template tmpl = load_laguna_s_template(t);
+    struct autoparser analysis;
+    analysis.analyze_template(tmpl);
+    t.assert_equal("Laguna-S(v8) reasoning_start should be '<think>'", "<think>", analysis.reasoning.start);
+    t.assert_equal("Laguna-S(v8) reasoning_end should be '</think>'", "</think>", analysis.reasoning.end);
+    t.assert_equal("Laguna-S(v8) reasoning should be TAG_BASED", reasoning_mode::TAG_BASED, analysis.reasoning.mode);
+}
+static void test_laguna_s_tool_format(testing & t) {
+    common_chat_template tmpl = load_laguna_s_template(t);
+    struct autoparser analysis;
+    analysis.analyze_template(tmpl);
+    t.assert_equal("Laguna-S(v8) arg_value_suffix should be '</arg_value>'", "</arg_value>", analysis.tools.arguments.value_suffix);
+}
+static void test_laguna_s_analysis(testing & t) {
+    t.test("Laguna-S(v8) reasoning detection", test_laguna_s_reasoning_detection);
+    t.test("Laguna-S(v8) tool format", test_laguna_s_tool_format);
+}
+
+static common_chat_template load_laguna_xs2_template(testing & t) {
+    return load_template(t, "models/templates/poolside-Laguna-XS.2.jinja");
+}
+static void test_laguna_xs2_reasoning_detection(testing & t) {
+    common_chat_template tmpl = load_laguna_xs2_template(t);
+    struct autoparser analysis;
+    analysis.analyze_template(tmpl);
+    t.assert_equal("Laguna-XS.2(v5) reasoning_start should be '<think>'", "<think>", analysis.reasoning.start);
+    t.assert_equal("Laguna-XS.2(v5) reasoning_end should be '</think>'", "</think>", analysis.reasoning.end);
+    t.assert_equal("Laguna-XS.2(v5) reasoning should be TAG_BASED", reasoning_mode::TAG_BASED, analysis.reasoning.mode);
+}
+static void test_laguna_xs2_tool_format(testing & t) {
+    common_chat_template tmpl = load_laguna_xs2_template(t);
+    struct autoparser analysis;
+    analysis.analyze_template(tmpl);
+    t.assert_equal("Laguna-XS.2(v5) arg_value_suffix should be '</arg_value>'", "</arg_value>", analysis.tools.arguments.value_suffix);
+}
+static void test_laguna_xs2_analysis(testing & t) {
+    t.test("Laguna-XS.2(v5) reasoning detection", test_laguna_xs2_reasoning_detection);
+    t.test("Laguna-XS.2(v5) tool format", test_laguna_xs2_tool_format);
+}
+
 static common_chat_template load_cohere_template(testing & t) {
     return load_template(t, "models/templates/CohereForAI-c4ai-command-r7b-12-2024-tool_use.jinja");
 }

tests/test-chat.cpp

@@ -109,6 +109,15 @@ static void assert_contains(const std::string & haystack, const std::string & ne
     }
 }
 
+static void assert_not_contains(const std::string & haystack, const std::string & needle) {
+    if (haystack.find(needle) != std::string::npos) {
+        LOG_ERR("Expected NOT to contain: %s\n", needle.c_str());
+        LOG_ERR("Actual: %s\n", haystack.c_str());
+        common_log_flush(common_log_main());
+        throw std::runtime_error("Test failed");
+    }
+}
+
 static void assert_ends_with(const std::string & str, const std::string & suffix) {
     if (str.size() < suffix.size() ||
         str.compare(str.size() - suffix.size(), suffix.size(), suffix) != 0) {
@@ -4016,6 +4025,132 @@ static void test_template_output_peg_parsers(bool detailed_debug) {
             .run();
     }
 
+    // DeepSeek V4 tests - same DSML markup as V3.2, but the tool call block is named
+    // "tool_calls" and the non-thinking generation prompt ends in a bare </think>
+    // instead of an empty <think></think> pair.
+    {
+        auto tst = peg_tester("models/templates/deepseek-ai-DeepSeek-V4.jinja", detailed_debug);
+
+        // Pure content (non-thinking mode; generation prompt ends with </think>)
+        tst.test("Hello, world!\nWhat's up?")
+            .enable_thinking(false)
+            .reasoning_format(COMMON_REASONING_FORMAT_DEEPSEEK)
+            .expect(message_assist)
+            .run();
+
+        // Thinking + content
+        tst.test("I'm\nthinking</think>Hello, world!\nWhat's up?")
+            .enable_thinking(true)
+            .reasoning_format(COMMON_REASONING_FORMAT_DEEPSEEK)
+            .expect(message_assist_thoughts)
+            .run();
+
+        // Thinking + tool call (single, string param)
+        tst.test(
+               "Let me check the time</think>\n\n"
+               "<｜DSML｜tool_calls>\n"
+               "<｜DSML｜invoke name=\"get_time\">\n"
+               "<｜DSML｜parameter name=\"city\" string=\"true\">Tokyo</｜DSML｜parameter>\n"
+               "</｜DSML｜invoke>\n"
+               "</｜DSML｜tool_calls>")
+            .enable_thinking(true)
+            .reasoning_format(COMMON_REASONING_FORMAT_DEEPSEEK)
+            .tools({ get_time_tool })
+            .expect(message_with_tool_calls_and_reasoning("get_time", R"({"city": "Tokyo"})", "Let me check the time"))
+            .run();
+
+        // Tool call without reasoning (non-thinking mode), integer param (string="false")
+        tst.test(
+               "<｜DSML｜tool_calls>\n"
+               "<｜DSML｜invoke name=\"special_function\">\n"
+               "<｜DSML｜parameter name=\"arg1\" string=\"false\">1</｜DSML｜parameter>\n"
+               "</｜DSML｜invoke>\n"
+               "</｜DSML｜tool_calls>")
+            .enable_thinking(false)
+            .reasoning_format(COMMON_REASONING_FORMAT_DEEPSEEK)
+            .tools({ special_function_tool })
+            .expect(message_assist_call)
+            .run();
+
+        // Multiple parallel tool calls with reasoning
+        tst.test(
+               "Calling both</think>\n\n"
+               "<｜DSML｜tool_calls>\n"
+               "<｜DSML｜invoke name=\"get_time\">\n"
+               "<｜DSML｜parameter name=\"city\" string=\"true\">Paris</｜DSML｜parameter>\n"
+               "</｜DSML｜invoke>\n"
+               "<｜DSML｜invoke name=\"get_weather\">\n"
+               "<｜DSML｜parameter name=\"city\" string=\"true\">Paris</｜DSML｜parameter>\n"
+               "</｜DSML｜invoke>\n"
+               "</｜DSML｜tool_calls>")
+            .enable_thinking(true)
+            .reasoning_format(COMMON_REASONING_FORMAT_DEEPSEEK)
+            .parallel_tool_calls(true)
+            .tools({ get_time_tool, get_weather_tool })
+            .expect(message_with_reasoning_content_and_multiple_tool_calls(
+                "Calling both", "",
+                { { "get_time", R"({"city": "Paris"})" }, { "get_weather", R"({"city": "Paris"})" } }))
+            .run();
+
+        // Tool call with content before tool calls
+        tst.test(
+               "Thinking about it</think>"
+               "Let me call the function.\n\n"
+               "<｜DSML｜tool_calls>\n"
+               "<｜DSML｜invoke name=\"special_function\">\n"
+               "<｜DSML｜parameter name=\"arg1\" string=\"false\">1</｜DSML｜parameter>\n"
+               "</｜DSML｜invoke>\n"
+               "</｜DSML｜tool_calls>")
+            .enable_thinking(true)
+            .reasoning_format(COMMON_REASONING_FORMAT_DEEPSEEK)
+            .tools({ special_function_tool })
+            .expect_reasoning("Thinking about it")
+            .expect_content("Let me call the function.")
+            .expect_tool_calls({
+                { "special_function", R"({"arg1": 1})", {} },
+            })
+            .run();
+
+        // Tool call with multiple params (mixed types)
+        tst.test(
+               "Multi-arg call</think>\n\n"
+               "<｜DSML｜tool_calls>\n"
+               "<｜DSML｜invoke name=\"magic_int\">\n"
+               "<｜DSML｜parameter name=\"ref\" string=\"false\">42</｜DSML｜parameter>\n"
+               "<｜DSML｜parameter name=\"name\" string=\"true\">foo bar</｜DSML｜parameter>\n"
+               "</｜DSML｜invoke>\n"
+               "</｜DSML｜tool_calls>")
+            .enable_thinking(true)
+            .reasoning_format(COMMON_REASONING_FORMAT_DEEPSEEK)
+            .tools({ magic_int_tool })
+            .expect_reasoning("Multi-arg call")
+            .expect_tool_calls({
+                { "magic_int", R"({"ref": 42, "name": "foo bar"})", {} },
+            })
+            .run();
+
+        // Continuation tests
+        tst.test("world!\nWhat's up?")
+            .reasoning_format(COMMON_REASONING_FORMAT_DEEPSEEK)
+            .enable_thinking(true)
+            .messages({ message_user, message_assist_prefill_content })
+            .add_generation_prompt(false)
+            .continue_final_message(COMMON_CHAT_CONTINUATION_CONTENT)
+            .expect_reasoning("I'm thinking")
+            .expect_content("Hello, world!\nWhat's up?")
+            .run();
+
+        tst.test(" thinking</think>Hello, world!\nWhat's up?")
+            .reasoning_format(COMMON_REASONING_FORMAT_DEEPSEEK)
+            .enable_thinking(true)
+            .messages({ message_user, message_assist_prefill_reasoning })
+            .add_generation_prompt(false)
+            .continue_final_message(COMMON_CHAT_CONTINUATION_REASONING)
+            .expect_reasoning("I'm thinking")
+            .expect_content("Hello, world!\nWhat's up?")
+            .run();
+    }
+
     // GLM-4.6 tests - format: <tool_call>function_name\n<arg_key>...</arg_key>\n<arg_value>...</arg_value>\n</tool_call>
     {
         auto tst = peg_tester("models/templates/GLM-4.6.jinja", detailed_debug);
@@ -5918,6 +6053,144 @@ static void test_developer_role_to_system_workaround() {
     }
 }
 
+// Verify reasoning-trace retention rules in the DeepSeek-V4 template:
+// all traces are retained unless drop_thinking is true AND the conversation
+// has no tool calls, in which case only the last (after-final-user) trace is
+// kept and earlier ones are dropped.
+static void test_deepseek_v4_thinking_retention() {
+    LOG_DBG("%s\n", __func__);
+
+    auto tmpls = read_templates("models/templates/deepseek-ai-DeepSeek-V4.jinja");
+
+    common_chat_msg user_q1; user_q1.role = "user"; user_q1.content = "Question 1";
+    common_chat_msg user_q2; user_q2.role = "user"; user_q2.content = "Question 2";
+    common_chat_msg asst_a1 = simple_assist_msg("Answer 1", "thinking A1");
+    common_chat_msg asst_a2 = simple_assist_msg("Answer 2", "thinking A2");
+
+    common_chat_msg tool_assist = message_with_tool_calls("special_function", "{\"arg1\": 1}");
+    common_chat_msg tool_result; tool_result.role = "tool";
+    tool_result.tool_name = "special_function"; tool_result.tool_call_id = "0"; tool_result.content = "result";
+
+    // The template uses U+FF5C as the role separator and literal think tags
+    // for the reasoning block.
+    const std::string asst_marker   = "<\xef\xbd\x9c" "Assistant" "\xef\xbd\x9c>";
+    // Built via concatenation so the thinking tokens are not interpreted by
+    // tooling processing this source file.
+    const std::string think_start   = "<" "think" ">";
+    const std::string think_end     = "</" "think" ">";
+
+    const std::string think_a1      = asst_marker + think_start + "thinking A1" + think_end;
+    const std::string think_a2      = asst_marker + think_start + "thinking A2" + think_end;
+    const std::string asst_no_think = asst_marker + think_end;
+
+    auto render = [&](const std::vector<common_chat_msg> & messages, bool drop_thinking) {
+        common_chat_templates_inputs inputs;
+        inputs.messages = messages;
+        inputs.add_generation_prompt = false;
+        inputs.chat_template_kwargs["thinking"]     = "true";
+        inputs.chat_template_kwargs["drop_thinking"] = drop_thinking ? "true" : "false";
+        return common_chat_templates_apply(tmpls.get(), inputs).prompt;
+    };
+
+    // No tools, drop_thinking=false: all reasoning is retained.
+    {
+        auto prompt = render({ user_q1, asst_a1, user_q2, asst_a2 }, /* drop_thinking = */ false);
+        assert_contains(prompt, think_a1);
+        assert_contains(prompt, think_a2);
+    }
+
+    // No tools, drop_thinking=true: only the last reasoning trace is kept,
+    // earlier ones are dropped (the assistant block emits just the end token).
+    {
+        auto prompt = render({ user_q1, asst_a1, user_q2, asst_a2 }, /* drop_thinking = */ true);
+        assert_not_contains(prompt, think_a1);
+        assert_contains(prompt, think_a2);
+        // The dropped assistant turn still opens with the marker + bare end token.
+        assert_contains(prompt, asst_no_think + "Answer 1");
+    }
+
+    // Single assistant turn, drop_thinking=true: the only trace is the last
+    // one, so it must be retained even with drop_thinking set.
+    {
+        auto prompt = render({ user_q1, asst_a1 }, /* drop_thinking = */ true);
+        assert_contains(prompt, think_a1);
+    }
+
+    // Single assistant turn, drop_thinking=false: reasoning is retained.
+    {
+        auto prompt = render({ user_q1, asst_a1 }, /* drop_thinking = */ false);
+        assert_contains(prompt, think_a1);
+    }
+
+    // With tool calls, drop_thinking=true: tool presence forces all reasoning
+    // to be retained, including the pre-tool-call trace.
+    {
+        auto prompt = render({ user_q1, asst_a1, user_q2, tool_assist, tool_result, asst_a2 },
+                             /* drop_thinking = */ true);
+        assert_contains(prompt, think_a1);
+        assert_contains(prompt, think_a2);
+    }
+
+    // With tool calls, drop_thinking=false: all reasoning retained.
+    {
+        auto prompt = render({ user_q1, asst_a1, user_q2, tool_assist, tool_result, asst_a2 },
+                             /* drop_thinking = */ false);
+        assert_contains(prompt, think_a1);
+        assert_contains(prompt, think_a2);
+    }
+}
+
+// Verify that consecutive tool results are rendered in the tool call order of the
+// preceding assistant message (matched by tool call id), as required by the reference
+// DeepSeek-V4 implementation.
+static void test_deepseek_v4_tool_result_ordering() {
+    LOG_DBG("%s\n", __func__);
+
+    auto tmpls = read_templates("models/templates/deepseek-ai-DeepSeek-V4.jinja");
+
+    common_chat_msg user_q; user_q.role = "user"; user_q.content = "Question";
+
+    common_chat_msg assist_calls;
+    assist_calls.role = "assistant";
+    assist_calls.tool_calls.push_back({ "get_time",    "{\"city\": \"Paris\"}", "call_1" });
+    assist_calls.tool_calls.push_back({ "get_weather", "{\"city\": \"Paris\"}", "call_2" });
+
+    common_chat_msg time_result; time_result.role = "tool";
+    time_result.tool_name = "get_time"; time_result.tool_call_id = "call_1"; time_result.content = "12:00";
+    common_chat_msg weather_result; weather_result.role = "tool";
+    weather_result.tool_name = "get_weather"; weather_result.tool_call_id = "call_2"; weather_result.content = "sunny";
+
+    auto render = [&](const std::vector<common_chat_msg> & messages) {
+        common_chat_templates_inputs inputs;
+        inputs.messages = messages;
+        inputs.add_generation_prompt = false;
+        return common_chat_templates_apply(tmpls.get(), inputs).prompt;
+    };
+
+    // Results sent out of order are reordered to match the tool call order.
+    {
+        auto prompt = render({ user_q, assist_calls, weather_result, time_result });
+        assert_contains(prompt, "<tool_result>12:00</tool_result>\n\n<tool_result>sunny</tool_result>");
+    }
+
+    // Results already in call order stay put.
+    {
+        auto prompt = render({ user_q, assist_calls, time_result, weather_result });
+        assert_contains(prompt, "<tool_result>12:00</tool_result>\n\n<tool_result>sunny</tool_result>");
+    }
+
+    // Without tool call ids there is nothing to match against; order is preserved.
+    {
+        auto no_id_calls = assist_calls;
+        no_id_calls.tool_calls[0].id = "";
+        no_id_calls.tool_calls[1].id = "";
+        auto no_id_weather = weather_result; no_id_weather.tool_call_id = "";
+        auto no_id_time    = time_result;    no_id_time.tool_call_id = "";
+        auto prompt = render({ user_q, no_id_calls, no_id_weather, no_id_time });
+        assert_contains(prompt, "<tool_result>sunny</tool_result>\n\n<tool_result>12:00</tool_result>");
+    }
+}
+
 static void test_reasoning_budget_tokens_per_request() {
     LOG_DBG("%s\n", __func__);
     // Use Qwen3 template which has <think>...</think> reasoning markers.
@@ -6139,6 +6412,8 @@ int main(int argc, char ** argv) {
         test_tools_oaicompat_json_conversion();
         test_convert_responses_to_chatcmpl();
         test_developer_role_to_system_workaround();
+        test_deepseek_v4_thinking_retention();
+        test_deepseek_v4_tool_result_ordering();
         test_template_generation_prompt();
         test_reasoning_budget_tokens_per_request();
         test_reasoning_budget_message_per_request();

tests/test-llama-archs.cpp

@@ -362,6 +362,7 @@ static bool moe_mandatory(const llm_arch arch) {
         case LLM_ARCH_STEP35:
         case LLM_ARCH_MISTRAL4:
         case LLM_ARCH_MELLUM:
+        case LLM_ARCH_LAGUNA:
             return true;
         default:
             return false;

tools/mtmd/models/qwen3vl.cpp

@@ -37,7 +37,7 @@ ggml_cgraph * clip_graph_qwen3vl::build() {
     }
 
     // calculate absolute position embedding and apply
-    ggml_tensor * learned_pos_embd = resize_position_embeddings();
+    ggml_tensor * learned_pos_embd = resize_position_embeddings(GGML_SCALE_MODE_BILINEAR | GGML_SCALE_FLAG_ALIGN_CORNERS);
     learned_pos_embd = ggml_cont_4d(
         ctx0, learned_pos_embd,
         n_embd * 2, n_patches_x / 2, n_patches_y, batch_size);

tools/mtmd/mtmd-helper.cpp

@@ -238,6 +238,29 @@ struct decode_embd_batch {
     }
 };
 
+// Helper class to set non-causal attention via RAII
+class scope_non_causal {
+public:
+    scope_non_causal(llama_context * context, bool enabled) : context_(context), enabled_(enabled) {
+        if (enabled_) {
+            // TODO @ngxson : need to make sure only one image is processed at a time, and n_ubatch must be enough to hold the image
+            llama_set_causal_attn(context_, false);
+        }
+    }
+    ~scope_non_causal() {
+        if (enabled_) {
+            llama_set_causal_attn(context_, true);
+        }
+    }
+
+    scope_non_causal(const scope_non_causal &) = delete;
+    scope_non_causal & operator=(const scope_non_causal &) = delete;
+
+private:
+    llama_context * context_;
+    bool enabled_;
+};
+
 // Helper function for decoding an image whose embeddings have already been calculated
 int32_t mtmd_helper_decode_image_chunk(
         mtmd_context * ctx,
@@ -288,10 +311,7 @@ int32_t mtmd_helper_decode_image_chunk(
     }
 
     const bool use_non_causal = mtmd_decode_use_non_causal(ctx, chunk);
-    if (use_non_causal) {
-        llama_set_causal_attn(lctx, false);
-        // TODO @ngxson : need to make sure only one image is processed at a time, and n_ubatch must be enough to hold the image
-    }
+    const scope_non_causal non_causal(lctx, use_non_causal);
 
     while (i_batch < n_img_batches) { // split into batches
         int pos_offset = i_batch*n_batch;
@@ -304,9 +324,6 @@ int32_t mtmd_helper_decode_image_chunk(
         int32_t ret = llama_decode(lctx, batch_embd_view);
         if (ret != 0) {
             LOG_ERR("failed to decode %s\n", name);
-            if (use_non_causal) {
-                llama_set_causal_attn(lctx, true);
-            }
             return ret;
         }
 
@@ -314,9 +331,6 @@ int32_t mtmd_helper_decode_image_chunk(
             ret = callback(batch_embd_view, user_data);
             if (ret != 0) {
                 LOG_ERR("post-decode callback failed\n");
-                if (use_non_causal) {
-                    llama_set_causal_attn(lctx, true);
-                }
                 return ret;
             }
         }
@@ -329,9 +343,6 @@ int32_t mtmd_helper_decode_image_chunk(
     n_past += mtmd_input_chunk_get_n_pos(chunk);
     *new_n_past = n_past;
 
-    if (use_non_causal) {
-        llama_set_causal_attn(lctx, true);
-    }
     return 0;
 }
 

tools/server/server-context.cpp

@@ -1152,6 +1152,11 @@ private:
             return false;
         }
 
+        if (ctx_tgt == nullptr) {
+            SRV_ERR("failed to create_context with model '%s'\n", params_base.model.path.c_str());
+            return false;
+        }
+
         vocab = llama_model_get_vocab(model_tgt);
 
         n_ctx = llama_n_ctx(ctx_tgt);

tools/server/server-stream.cpp

@@ -632,6 +632,13 @@ void server_res_spipe::on_complete() {
     if (!spipe || next_finished) {
         return;
     }
+    // an empty next_orig means set_next() never ran: the request failed before streaming
+    // started, typically a params validation throw. evict the session installed by set_req()
+    // so the failed request leaves nothing behind for discovery or replay
+    if (!next_orig) {
+        g_stream_sessions.evict(server_stream_conv_id_from_headers(req->headers));
+        return;
+    }
     std::string chunk;
     while (!spipe->is_cancelled()) {
         chunk.clear();

tools/ui/.gitignore

@@ -36,3 +36,7 @@ static/favicon*
 *storybook.log
 storybook-static
 *.code-workspace
+
+# Vitest browser mode failure artifacts
+.vitest-attachments/
+tests/**/__screenshots__/

tools/ui/.prettierignore

@@ -16,3 +16,6 @@ build/
 /build/
 /.svelte-kit/
 test-results
+
+# Vendored third party sources, kept byte identical to upstream
+src/lib/vendors/

tools/ui/eslint.config.js

@@ -59,7 +59,8 @@ export default ts.config(
 			'.svelte-kit/**',
 			'test-results/**',
 			'.storybook/**/*',
-			'src/lib/services/sandbox-worker.js'
+			'src/lib/services/sandbox-worker.js',
+			'src/lib/vendors/**'
 		]
 	},
 	storybook.configs['flat/recommended']

tools/ui/scripts/vite-plugin-nerdamer.ts

@@ -0,0 +1,49 @@
+import { build } from 'esbuild';
+import { dirname, resolve } from 'path';
+import { fileURLToPath } from 'url';
+import type { Plugin } from 'vite';
+
+const __dirname = dirname(fileURLToPath(import.meta.url));
+
+const VENDORS_DIR = resolve(__dirname, '../src/lib/vendors');
+const VIRTUAL_ID = 'virtual:nerdamer';
+const RESOLVED_ID = '\0' + VIRTUAL_ID;
+
+/**
+ * Bundle the vendored nerdamer-prime source into a minified IIFE string,
+ * exposed as the `virtual:nerdamer` module. Flags mirror the upstream
+ * build (esbuild --bundle --minify --format=iife --global-name=nerdamer),
+ * so only human readable source lives in the repo and minification is a
+ * build artifact. Vendored under src/lib/vendors/, upstream snapshot:
+ * https://github.com/together-science/nerdamer-prime/commit/1936145f8af306ec0d883b9bfd7730aedd175c24
+ */
+export function nerdamerPlugin(): Plugin {
+	let bundled: string | null = null;
+
+	return {
+		name: 'llamacpp:nerdamer',
+		resolveId(id) {
+			return id === VIRTUAL_ID ? RESOLVED_ID : undefined;
+		},
+		async load(id) {
+			if (id !== RESOLVED_ID) return undefined;
+			if (bundled === null) {
+				const result = await build({
+					entryPoints: [resolve(VENDORS_DIR, 'nerdamer-prime/all.js')],
+					bundle: true,
+					minify: true,
+					format: 'iife',
+					globalName: 'nerdamer',
+					alias: {
+						'big-integer': resolve(VENDORS_DIR, 'big-integer/BigInteger.js'),
+						'decimal.js': resolve(VENDORS_DIR, 'decimal.js/decimal.js')
+					},
+					write: false,
+					logLevel: 'silent'
+				});
+				bundled = result.outputFiles[0].text;
+			}
+			return `export default ${JSON.stringify(bundled)};`;
+		}
+	};
+}

tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionAdd/ChatFormActionAddDropdown.svelte

@@ -71,7 +71,9 @@
 
 <div class="flex items-center gap-1 {className}">
 	<DropdownMenu.Root bind:open={dropdownOpen}>
-		<Tooltip.Root>
+		<!-- ignoreNonKeyboardFocus prevents the tooltip from flashing when the
+		     menu closes and focus returns to the trigger -->
+		<Tooltip.Root ignoreNonKeyboardFocus>
 			<Tooltip.Trigger>
 				{#snippet child({ props })}
 					<DropdownMenu.Trigger

tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionAdd/ChatFormActionAddReasoningSubmenu.svelte

@@ -5,18 +5,18 @@
 	import * as Tooltip from '$lib/components/ui/tooltip';
 	import { useReasoningMenu } from '$lib/hooks/use-reasoning-menu.svelte';
 
-	let subOpen = $state(false);
-
 	const reasoning = useReasoningMenu();
 </script>
 
 {#if reasoning.modelSupportsThinking}
-	<DropdownMenu.Sub bind:open={subOpen}>
+	<DropdownMenu.Sub>
 		<DropdownMenu.SubTrigger class="flex cursor-pointer items-center gap-2">
 			{#if reasoning.thinkingEnabled}
 				<Lightbulb class="{ICON_CLASS_DEFAULT} shrink-0 text-amber-400" />
-			{:else}
+			{:else if reasoning.isOff}
 				<LightbulbOff class="{ICON_CLASS_DEFAULT} shrink-0 text-muted-foreground" />
+			{:else}
+				<Lightbulb class="{ICON_CLASS_DEFAULT} shrink-0 text-muted-foreground" />
 			{/if}
 
 			<span
@@ -27,7 +27,7 @@
 				Reasoning
 
 				<span class="capitalize text-muted-foreground">
-					{reasoning.thinkingEnabled ? reasoning.currentEffort : 'off'}
+					{reasoning.currentEffort}
 				</span>
 			</span>
 		</DropdownMenu.SubTrigger>
@@ -37,14 +37,13 @@
 		>
 			{#each reasoning.levels as level (level.value)}
 				{@const tokenLabel = reasoning.tokenLabel(level)}
-				<button
-					type="button"
-					class="flex w-full cursor-pointer items-center gap-3 rounded-md px-2 py-1.75 text-left text-sm transition-colors hover:bg-accent"
-					class:bg-accent={reasoning.isSelected(level)}
-					onclick={() => {
-						reasoning.select(level);
-						subOpen = false;
-					}}
+				<DropdownMenu.Item
+					class="flex w-full cursor-pointer items-center gap-3 rounded-md px-2 py-1.75 text-left text-sm transition-colors hover:bg-accent {reasoning.isSelected(
+						level
+					)
+						? 'bg-accent'
+						: ''}"
+					onclick={() => reasoning.select(level)}
 				>
 					{#if reasoning.isSelected(level)}
 						<Check class="{ICON_CLASS_DEFAULT} shrink-0 text-foreground" />
@@ -70,7 +69,7 @@
 							</Tooltip.Content>
 						</Tooltip.Root>
 					{/if}
-				</button>
+				</DropdownMenu.Item>
 			{/each}
 		</DropdownMenu.SubContent>
 	</DropdownMenu.Sub>

tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormActionAdd/ChatFormActionAddSheet.svelte

@@ -116,14 +116,16 @@
 
 							{#if reasoning.thinkingEnabled}
 								<Lightbulb class="{ICON_CLASS_DEFAULT} shrink-0 text-amber-400" />
-							{:else}
+							{:else if reasoning.isOff}
 								<LightbulbOff class="{ICON_CLASS_DEFAULT} shrink-0 text-muted-foreground" />
+							{:else}
+								<Lightbulb class="{ICON_CLASS_DEFAULT} shrink-0 text-muted-foreground" />
 							{/if}
 
 							<span class="flex-1">Reasoning</span>
 
 							<span class="text-xs capitalize text-muted-foreground">
-								{reasoning.thinkingEnabled ? reasoning.currentEffort : 'off'}
+								{reasoning.currentEffort}
 							</span>
 						</Collapsible.Trigger>
 

tools/ui/src/lib/components/app/chat/ChatForm/ChatFormActions/ChatFormReasoningEffortSubmenu.svelte

@@ -1,127 +0,0 @@
-<script lang="ts">
-	import { Check, Info, Lightbulb, LightbulbOff } from '@lucide/svelte';
-	import * as DropdownMenu from '$lib/components/ui/dropdown-menu';
-	import * as Tooltip from '$lib/components/ui/tooltip';
-	import { ReasoningEffort, MessageRole } from '$lib/enums';
-	import { REASONING_EFFORT_TOKENS } from '$lib/constants/reasoning-effort-tokens';
-	import { REASONING_EFFORT_LEVELS } from '$lib/constants/reasoning-effort';
-	import type { ReasoningEffortLevel } from '$lib/types';
-	import { DIALOG_SUBMENU_CONTENT, ICON_CLASS_DEFAULT } from '$lib/constants/css-classes';
-	import {
-		modelsStore,
-		checkModelSupportsThinking,
-		supportsThinking,
-		propsCacheVersion,
-		loadedModelIds
-	} from '$lib/stores/models.svelte';
-	import { chatStore } from '$lib/stores/chat.svelte';
-	import { conversationsStore, activeMessages } from '$lib/stores/conversations.svelte';
-	import { isRouterMode } from '$lib/stores/server.svelte';
-	import type { DatabaseMessage } from '$lib/types/database';
-
-	let thinkingEnabled = $derived(conversationsStore.getThinkingEnabled());
-	let currentEffort = $derived(conversationsStore.getReasoningEffort());
-	let isOff = $derived(!thinkingEnabled);
-	let subOpen = $state(false);
-
-	// Get conversation model from message history
-	let conversationModel = $derived(
-		chatStore.getConversationModel(activeMessages() as DatabaseMessage[])
-	);
-
-	let modelSupportsThinkingFromMessages = $derived.by(() => {
-		const modelId = isRouterMode() ? modelsStore.selectedModelName || conversationModel : null;
-		if (!modelId) return false;
-
-		const messages = conversationsStore.activeMessages;
-
-		return messages.some(
-			(m: DatabaseMessage) =>
-				m.role === MessageRole.ASSISTANT && m.model === modelId && !!m.reasoningContent
-		);
-	});
-
-	let modelSupportsThinking = $derived.by(() => {
-		loadedModelIds();
-		propsCacheVersion();
-
-		if (isRouterMode()) {
-			const modelId = modelsStore.selectedModelName || conversationModel;
-			return checkModelSupportsThinking(modelId ?? '') || modelSupportsThinkingFromMessages;
-		}
-
-		return supportsThinking() || modelSupportsThinkingFromMessages;
-	});
-
-	function isSelected(item: ReasoningEffortLevel): boolean {
-		if (item.isOff) return isOff;
-
-		return thinkingEnabled && currentEffort === item.value;
-	}
-
-	function handleSelection(item: ReasoningEffortLevel) {
-		if (item.isOff) {
-			conversationsStore.setThinkingEnabled(false);
-		} else {
-			conversationsStore.setThinkingEnabled(true);
-			conversationsStore.setReasoningEffort(item.value as ReasoningEffort);
-		}
-		subOpen = false;
-	}
-</script>
-
-{#if modelSupportsThinking}
-	<DropdownMenu.Sub bind:open={subOpen}>
-		<DropdownMenu.SubTrigger class="flex cursor-pointer items-center gap-2">
-			{#if thinkingEnabled}
-				<Lightbulb class="{ICON_CLASS_DEFAULT} shrink-0 text-amber-400" />
-			{:else}
-				<LightbulbOff class="{ICON_CLASS_DEFAULT} shrink-0 text-muted-foreground" />
-			{/if}
-
-			<span class="flex-1">Thinking</span>
-
-			{#if thinkingEnabled}
-				<span class="text-xs text-muted-foreground">{currentEffort}</span>
-			{:else}
-				<span class="text-xs text-muted-foreground">off</span>
-			{/if}
-		</DropdownMenu.SubTrigger>
-
-		<DropdownMenu.SubContent class={DIALOG_SUBMENU_CONTENT}>
-			{#each REASONING_EFFORT_LEVELS as level (level.value)}
-				<button
-					type="button"
-					class="flex w-full cursor-pointer items-center gap-2"
-					class:bg-accent={isSelected(level)}
-					onclick={() => handleSelection(level)}
-				>
-					<span class="flex-1 text-left">{level.label}</span>
-
-					{#if !level.isOff}
-						<span class="text-[11px] text-muted-foreground opacity-60">
-							{REASONING_EFFORT_TOKENS[level.value] === -1
-								? 'Unlimited'
-								: `Max ${REASONING_EFFORT_TOKENS[level.value].toLocaleString()} tokens`}
-						</span>
-					{/if}
-
-					{#if level.hasInfo}
-						<Tooltip.Root>
-							<Tooltip.Trigger>
-								<Info class="h-3.5 w-3.5 shrink-0 text-muted-foreground" />
-							</Tooltip.Trigger>
-							<Tooltip.Content side="left">
-								<p>Maximum thinking effort with extended context usage</p>
-							</Tooltip.Content>
-						</Tooltip.Root>
-					{/if}
-
-					{#if isSelected(level)}
-						<Check class="{ICON_CLASS_DEFAULT} shrink-0 text-foreground" />
-					{/if}
-				</button>
-			{/each}
-		</DropdownMenu.SubContent>
-	</DropdownMenu.Sub>
-{/if}

tools/ui/src/lib/constants/reasoning-effort.ts

@@ -6,6 +6,7 @@ import type { ReasoningEffortLevel } from '$lib/types';
  * Keys match the ReasoningEffort enum values for type-safe lookups.
  */
 export const REASONING_EFFORT_LABELS: Record<string, string> = {
+	[ReasoningEffort.DEFAULT]: 'Default',
 	[ReasoningEffort.OFF]: 'Off',
 	[ReasoningEffort.LOW]: 'Low',
 	[ReasoningEffort.MEDIUM]: 'Medium',
@@ -14,7 +15,8 @@ export const REASONING_EFFORT_LABELS: Record<string, string> = {
 };
 
 export const REASONING_EFFORT_LEVELS: ReasoningEffortLevel[] = [
-	{ value: ReasoningEffort.OFF, label: 'Off', isOff: true },
+	{ value: ReasoningEffort.DEFAULT, label: 'Default' },
+	{ value: ReasoningEffort.OFF, label: 'Off' },
 	{ value: ReasoningEffort.LOW, label: 'Low' },
 	{ value: ReasoningEffort.MEDIUM, label: 'Medium' },
 	{ value: ReasoningEffort.HIGH, label: 'High' },

tools/ui/src/lib/constants/sandbox.ts

@@ -13,27 +13,44 @@ export const SANDBOX_EMPTY_OUTPUT = '(no output)';
 
 export const SANDBOX_TRUNCATION_NOTICE = '[output truncated]';
 
-export const SANDBOX_TOOL_DEFINITION: OpenAIToolDefinition = {
-	type: ToolCallType.FUNCTION,
-	function: {
-		name: SANDBOX_TOOL_NAME,
-		description:
-			'Execute JavaScript in a sandboxed browser worker (no DOM, no page access). ' +
-			'Top level await is supported. Use console.log to print intermediate values; ' +
-			'a top level return statement is captured as the result.',
-		parameters: {
-			type: JsonSchemaType.OBJECT,
-			properties: {
-				code: {
-					type: JsonSchemaType.STRING,
-					description: 'JavaScript source to execute'
+const NERDAMER_DESCRIPTION = `
+Symbolic/numeric math via \`nerdamer\` (pre-loaded, do not require, use it directly).
+nerdamer('diff(sin(x)/x,x)') or nerdamer.diff('sin(x)/x','x') → Expression; convert with .toString()/.text()/.toTeX(), or .evaluate() (→ still Expression, then .toString()).
+nerdamer(expr,{x:2}) substitutes only; chain .evaluate() or pass 'numer' for numeric result.
+solve(expr,var)→Symbol[]; solveEquations([eq1,..])→[[var,val],..] pairs.
+Functions: simplify/expand/factor(expr), diff(expr,var[,n]), integrate(expr,var), defint(expr,from,to,var), limit(expr,var,to), laplace(expr,t,s), ilt(expr,s,t), gcd/lcm(a,b), roots/coeffs/partfrac(expr,var), pfactor(n), numer/decimals/erf(expr), product/sum(expr,var,from,to), mean/median/stdev/variance(...vals).
+Object.keys(nerdamer).filter(k=>typeof nerdamer[k]==='function') lists all available functions. If you need a function not documented above, list them first — do not guess function names.`;
+
+/**
+ * Build the sandbox tool definition. When `includeSymbolicMath` is true,
+ * the description includes nerdamer API documentation; otherwise it
+ * describes a plain JavaScript sandbox.
+ */
+export function buildSandboxToolDefinition(includeSymbolicMath: boolean): OpenAIToolDefinition {
+	return {
+		type: ToolCallType.FUNCTION,
+		function: {
+			name: SANDBOX_TOOL_NAME,
+			description: includeSymbolicMath
+				? `Execute JS in a sandboxed browser worker (no DOM/page access). Top-level await ok; console.log for intermediates; top-level return is captured as result.${NERDAMER_DESCRIPTION}`
+				: 'Execute JS in a sandboxed browser worker (no DOM/page access). Top-level await ok; console.log for intermediates; top-level return is captured as result.',
+			parameters: {
+				type: JsonSchemaType.OBJECT,
+				properties: {
+					code: {
+						type: JsonSchemaType.STRING,
+						description: 'JavaScript source to execute'
+					},
+					timeout_ms: {
+						type: JsonSchemaType.NUMBER,
+						description: `Execution timeout in milliseconds, default ${SANDBOX_TIMEOUT_MS_DEFAULT}, max ${SANDBOX_TIMEOUT_MS_MAX}`
+					}
 				},
-				timeout_ms: {
-					type: JsonSchemaType.NUMBER,
-					description: `Execution timeout in milliseconds, default ${SANDBOX_TIMEOUT_MS_DEFAULT}, max ${SANDBOX_TIMEOUT_MS_MAX}`
-				}
-			},
-			required: ['code']
+				required: ['code']
+			}
 		}
-	}
-};
+	};
+}
+
+/** @deprecated Use {@link buildSandboxToolDefinition} instead. Kept for backward compatibility. */
+export const SANDBOX_TOOL_DEFINITION = buildSandboxToolDefinition(true);

tools/ui/src/lib/constants/settings-keys.ts

@@ -67,6 +67,7 @@ export const SETTINGS_KEYS = {
 	EXCLUDE_REASONING_FROM_CONTEXT: 'excludeReasoningFromContext',
 	SHOW_RAW_OUTPUT_SWITCH: 'showRawOutputSwitch',
 	JS_SANDBOX_ENABLED: 'jsSandboxEnabled',
+	SYMBOLIC_MATH_ENABLED: 'symbolicMathEnabled',
 	// PY_INTERPRETER_ENABLED: 'pyInterpreterEnabled',
 	CUSTOM_JSON: 'customJson',
 	CUSTOM_CSS: 'customCss'

tools/ui/src/lib/constants/settings-registry.ts

@@ -724,6 +724,15 @@ const SETTINGS_REGISTRY: Record<string, SettingsSectionEntry> = {
 					paramType: SyncableParameterType.BOOLEAN
 				}
 			},
+			{
+				key: SETTINGS_KEYS.SYMBOLIC_MATH_ENABLED,
+				label: 'Symbolic math (nerdamer)',
+				help: 'Pre-load nerdamer in the sandbox for symbolic computation: simplify, diff, integrate, solve, and more. Requires "JavaScript sandbox tool" to be enabled.',
+				defaultValue: false,
+				type: SettingsFieldType.CHECKBOX,
+				section: SETTINGS_SECTION_SLUGS.DEVELOPER,
+				dependsOn: SETTINGS_KEYS.JS_SANDBOX_ENABLED
+			},
 			{
 				key: SETTINGS_KEYS.CUSTOM_JSON,
 				label: 'Custom JSON',

tools/ui/src/lib/enums/reasoning-effort.enums.ts

@@ -3,6 +3,7 @@
  * These values are sent to the server and mapped to token budgets.
  */
 export enum ReasoningEffort {
+	DEFAULT = 'default',
 	OFF = 'off',
 	LOW = 'low',
 	MEDIUM = 'medium',

tools/ui/src/lib/hooks/use-reasoning-menu.svelte.ts

@@ -17,6 +17,7 @@ import { isRouterMode } from '$lib/stores/server.svelte';
 export interface UseReasoningMenuReturn {
 	readonly modelSupportsThinking: boolean;
 	readonly thinkingEnabled: boolean;
+	readonly isOff: boolean;
 	readonly currentEffort: ReasoningEffort;
 	readonly levels: ReasoningEffortLevel[];
 	isSelected(level: ReasoningEffortLevel): boolean;
@@ -59,8 +60,10 @@ export function useReasoningMenu(): UseReasoningMenuReturn {
 		return supportsThinking() || modelSupportsThinkingFromMessages;
 	});
 
-	const thinkingEnabled = $derived(conversationsStore.getThinkingEnabled());
 	const currentEffort = $derived(conversationsStore.getReasoningEffort());
+	const thinkingEnabled = $derived(
+		currentEffort !== ReasoningEffort.OFF && currentEffort !== ReasoningEffort.DEFAULT
+	);
 
 	return {
 		get modelSupportsThinking() {
@@ -69,6 +72,9 @@ export function useReasoningMenu(): UseReasoningMenuReturn {
 		get thinkingEnabled() {
 			return thinkingEnabled;
 		},
+		get isOff() {
+			return currentEffort === ReasoningEffort.OFF;
+		},
 		get currentEffort() {
 			return currentEffort;
 		},
@@ -76,20 +82,15 @@ export function useReasoningMenu(): UseReasoningMenuReturn {
 			return REASONING_EFFORT_LEVELS;
 		},
 		isSelected(level: ReasoningEffortLevel): boolean {
-			if (level.isOff) return !thinkingEnabled;
-			return thinkingEnabled && currentEffort === level.value;
+			return currentEffort === level.value;
 		},
 		tokenLabel(level: ReasoningEffortLevel): string | null {
-			if (level.isOff) return null;
+			if (level.value === ReasoningEffort.DEFAULT) return 'Model default';
 			const tokens = REASONING_EFFORT_TOKENS[level.value];
+			if (tokens === undefined) return null;
 			return tokens === -1 ? 'Unlimited' : `Max ${tokens.toLocaleString()} tokens`;
 		},
 		select(level: ReasoningEffortLevel): void {
-			if (level.isOff) {
-				conversationsStore.setThinkingEnabled(false);
-				return;
-			}
-			conversationsStore.setThinkingEnabled(true);
 			conversationsStore.setReasoningEffort(level.value as ReasoningEffort);
 		}
 	};

tools/ui/src/lib/services/chat.service.ts

@@ -271,10 +271,14 @@ export class ChatService {
 		const reasoningBudgetTokens =
 			enableThinking && reasoningEffort ? (REASONING_EFFORT_TOKENS[reasoningEffort] ?? -1) : -1;
 
-		requestBody.chat_template_kwargs = {
-			...(requestBody.chat_template_kwargs ?? {}),
-			enable_thinking: enableThinking
-		};
+		// an explicit user choice injects the kwarg, otherwise it is omitted so
+		// the server default applies (--reasoning flag or chat template)
+		if (enableThinking !== undefined) {
+			requestBody.chat_template_kwargs = {
+				...(requestBody.chat_template_kwargs ?? {}),
+				enable_thinking: enableThinking
+			};
+		}
 
 		if (reasoningBudgetTokens >= 0) {
 			requestBody.thinking_budget_tokens = reasoningBudgetTokens;

tools/ui/src/lib/services/index.ts

@@ -276,7 +276,7 @@ export { MCPService } from './mcp.service';
  * - **toolsStore**: Exposes the tool definition when the sandbox is enabled
  * - **agenticStore**: Dispatches ToolSource.FRONTEND calls here
  *
- * @see SANDBOX_TOOL_DEFINITION in constants/sandbox.ts - tool schema sent to the LLM
+ * @see buildSandboxToolDefinition in constants/sandbox.ts - tool schema sent to the LLM
  * @see agenticStore in stores/agentic.svelte.ts - tool dispatch
  */
 export { SandboxService } from './sandbox.service';

tools/ui/src/lib/services/sandbox-harness.ts

@@ -1,14 +1,25 @@
+import { NEWLINE } from '$lib/constants';
 import WORKER_SHIM from './sandbox-worker.js?raw';
 
+/**
+ * CSP for the harness document, inherited by the blob worker. connect-src
+ * falls back to default-src, removing network egress for model and vendored
+ * code. 'unsafe-eval' is required by the worker's AsyncFunction constructor,
+ * 'unsafe-inline' by the inline script below, worker-src by the blob worker.
+ */
+const HARNESS_CSP = `default-src 'none'; script-src 'unsafe-inline' 'unsafe-eval'; worker-src blob:`;
+
 /**
  * Harness loaded as srcdoc into a sandboxed iframe (allow-scripts only).
  * The opaque origin is the security boundary: no access to the app origin,
  * its storage or its API. The harness spawns a worker so model code never
  * runs on a main thread, which makes the parent timeout enforceable by
- * removing the iframe.
+ * removing the iframe. The prelude runs in the worker before the shim,
+ * exposing globals such as `nerdamer` to model code.
  */
-export const SANDBOX_HARNESS_HTML = `<!doctype html><script>
-const SHIM = ${JSON.stringify(WORKER_SHIM)};
+export function buildSandboxHarness(preludeJs: string): string {
+	return `<!doctype html><meta http-equiv="Content-Security-Policy" content="${HARNESS_CSP}"><script>
+const SHIM = ${JSON.stringify(preludeJs + NEWLINE + WORKER_SHIM)};
 addEventListener('message', (event) => {
 	const respond = (payload) => parent.postMessage(payload, '*');
 	let worker;
@@ -23,3 +34,4 @@ addEventListener('message', (event) => {
 	worker.postMessage({ code: event.data.code });
 });
 </script>`;
+}

tools/ui/src/lib/services/sandbox-worker.js

@@ -21,7 +21,9 @@ self.onmessage = async (event) => {
 	const reply = { logs, result: null, error: null };
 	try {
 		const AsyncFunction = Object.getPrototypeOf(async function () {}).constructor;
-		const value = await new AsyncFunction(event.data.code)();
+		// The prelude bundled ahead of this shim defines self.nerdamer,
+		// passed into the execution scope as the `nerdamer` parameter.
+		const value = await new AsyncFunction('nerdamer', event.data.code)(self.nerdamer);
 		if (value !== undefined) reply.result = fmt(value);
 	} catch (err) {
 		reply.error = err instanceof Error ? err.stack || err.message : String(err);

tools/ui/src/lib/services/sandbox.service.ts

@@ -7,9 +7,32 @@ import {
 	SANDBOX_TOOL_NAME,
 	SANDBOX_TRUNCATION_NOTICE
 } from '$lib/constants';
-import { SANDBOX_HARNESS_HTML } from './sandbox-harness';
+import { buildSandboxHarness } from './sandbox-harness';
+import { config } from '$lib/stores/settings.svelte';
 import type { ToolExecutionResult } from '$lib/types';
 
+/** Cached harnesses keyed by whether nerdamer is included. */
+const harnessCache: Record<string, string> = {};
+
+/**
+ * Build the sandbox harness. When symbolic math is enabled, loads the
+ * nerdamer prelude lazily; otherwise builds a plain harness with an empty
+ * prelude. Cached per variant so toggling the setting is instant.
+ */
+async function getHarness(): Promise<string> {
+	const enabled = !!config().symbolicMathEnabled;
+	const key = enabled ? 'nerdamer' : 'plain';
+	if (!harnessCache[key]) {
+		if (enabled) {
+			const { default: nerdamerJs } = await import('virtual:nerdamer');
+			harnessCache[key] = buildSandboxHarness(nerdamerJs);
+		} else {
+			harnessCache[key] = buildSandboxHarness('');
+		}
+	}
+	return harnessCache[key];
+}
+
 interface SandboxReply {
 	logs?: unknown;
 	result?: unknown;
@@ -45,20 +68,22 @@ export class SandboxService {
 	 * timeout or abort. Removing the iframe terminates the worker
 	 * at the browser level, so runaway code cannot outlive it.
 	 */
-	static executeTool(
+	static async executeTool(
 		toolName: string,
 		params: Record<string, unknown>,
 		signal?: AbortSignal
 	): Promise<ToolExecutionResult> {
 		if (toolName !== SANDBOX_TOOL_NAME) {
-			return Promise.resolve({ content: `Unknown frontend tool: ${toolName}`, isError: true });
+			return { content: `Unknown frontend tool: ${toolName}`, isError: true };
 		}
 
 		const code = typeof params.code === 'string' ? params.code : '';
 		if (!code) {
-			return Promise.resolve({ content: 'Missing required parameter: code', isError: true });
+			return { content: 'Missing required parameter: code', isError: true };
 		}
 
+		const harness = await getHarness();
+
 		const requested = Number(params.timeout_ms);
 		const timeoutMs =
 			Number.isFinite(requested) && requested > 0
@@ -69,7 +94,7 @@ export class SandboxService {
 			const iframe = document.createElement('iframe');
 			iframe.setAttribute('sandbox', 'allow-scripts');
 			iframe.style.display = 'none';
-			iframe.srcdoc = SANDBOX_HARNESS_HTML;
+			iframe.srcdoc = harness;
 
 			let settled = false;
 

tools/ui/src/lib/stores/chat.svelte.ts

@@ -2373,9 +2373,12 @@ class ChatStore {
 
 		if (currentConfig.excludeReasoningFromContext) apiOptions.excludeReasoningFromContext = true;
 
-		apiOptions.enableThinking = conversationsStore.getThinkingEnabled();
+		// an explicit reasoning choice overrides the server default, DEFAULT sends nothing
 		const effort = conversationsStore.getReasoningEffort();
-		if (effort !== ReasoningEffort.OFF) apiOptions.reasoningEffort = effort;
+		if (effort !== ReasoningEffort.DEFAULT) {
+			apiOptions.enableThinking = effort !== ReasoningEffort.OFF;
+			if (effort !== ReasoningEffort.OFF) apiOptions.reasoningEffort = effort;
+		}
 
 		if (hasValue(currentConfig.temperature))
 			apiOptions.temperature = Number(currentConfig.temperature);
@@ -2428,7 +2431,8 @@ class ChatStore {
 
 		if (currentConfig.samplers) apiOptions.samplers = currentConfig.samplers;
 
-		apiOptions.backend_sampling = currentConfig.backend_sampling;
+		if (hasValue(currentConfig.backend_sampling))
+			apiOptions.backend_sampling = currentConfig.backend_sampling;
 
 		if (currentConfig.customJson) apiOptions.custom = currentConfig.customJson;
 

tools/ui/src/lib/stores/conversations.svelte.ts

@@ -80,25 +80,17 @@ class ConversationsStore {
 	/** Whether the store has been initialized */
 	isInitialized = $state(false);
 
-	/** Global (non-conversation-specific) thinking toggle default, derived from reasoning effort */
-	pendingThinkingEnabled = $state(false);
-
 	/** Global (non-conversation-specific) reasoning effort default */
-	pendingReasoningEffort = $state<ReasoningEffort | ReasoningEffort.OFF>(
-		ConversationsStore.loadReasoningEffortDefault()
-	);
+	pendingReasoningEffort = $state<ReasoningEffort>(ConversationsStore.loadReasoningEffortDefault());
 
-	/** Last non-off reasoning effort, restored when re-enabling thinking globally */
-	private lastNonOffEffort: ReasoningEffort | null = null;
-
-	/** Load reasoning effort default from localStorage */
-	private static loadReasoningEffortDefault(): ReasoningEffort | ReasoningEffort.OFF {
-		if (typeof globalThis.localStorage === 'undefined') return ReasoningEffort.OFF;
+	/** Load reasoning effort default from localStorage, DEFAULT defers to the server */
+	private static loadReasoningEffortDefault(): ReasoningEffort {
+		if (typeof globalThis.localStorage === 'undefined') return ReasoningEffort.DEFAULT;
 		try {
 			const raw = localStorage.getItem(REASONING_EFFORT_DEFAULT_LOCALSTORAGE_KEY);
-			return (raw as ReasoningEffort | ReasoningEffort.OFF) || ReasoningEffort.OFF;
+			return (raw as ReasoningEffort) || ReasoningEffort.DEFAULT;
 		} catch {
-			return ReasoningEffort.OFF;
+			return ReasoningEffort.DEFAULT;
 		}
 	}
 
@@ -235,17 +227,10 @@ class ConversationsStore {
 		// servers without a per-conversation override to `mcpServers[i].enabled`,
 		// and only explicit toggles are stored on the conversation.
 
-		// Inherit global thinking/reasoning defaults into the new conversation
-		const thinkingEnabled = this.getThinkingEnabled();
-		conversation.thinkingEnabled = thinkingEnabled;
-		conversation.reasoningEffort =
-			this.pendingReasoningEffort === ReasoningEffort.OFF ? undefined : this.pendingReasoningEffort;
+		// Inherit the global reasoning default into the new conversation
+		conversation.reasoningEffort = this.pendingReasoningEffort;
 		await DatabaseService.updateConversation(conversation.id, {
-			thinkingEnabled,
-			reasoningEffort:
-				this.pendingReasoningEffort === ReasoningEffort.OFF
-					? undefined
-					: this.pendingReasoningEffort
+			reasoningEffort: this.pendingReasoningEffort
 		});
 
 		this.conversations = [conversation, ...this.conversations];
@@ -793,63 +778,21 @@ class ConversationsStore {
 		await this.setMcpServerOverride(serverId, undefined);
 	}
 
-	/**
-	 * Gets the effective thinking-enabled state for the active conversation.
-	 * Returns the conversation override if set, otherwise the global default.
-	 */
-	getThinkingEnabled(): boolean {
-		if (this.activeConversation) {
-			if (this.activeConversation.thinkingEnabled !== undefined) {
-				return this.activeConversation.thinkingEnabled;
-			}
-		}
-		return this.getReasoningEffort() !== ReasoningEffort.OFF;
-	}
-
-	/**
-	 * Sets the thinking-enabled state for the active conversation.
-	 * If no conversation exists, stores the global default.
-	 * @param enabled - The enabled state
-	 */
-	async setThinkingEnabled(enabled: boolean): Promise<void> {
-		if (!this.activeConversation) {
-			if (enabled) {
-				const effort = this.lastNonOffEffort ?? ReasoningEffort.LOW;
-				this.pendingReasoningEffort = effort;
-				this.saveReasoningEffortDefaults();
-			} else {
-				if (this.pendingReasoningEffort !== ReasoningEffort.OFF) {
-					this.lastNonOffEffort = this.pendingReasoningEffort;
-				}
-				this.pendingReasoningEffort = ReasoningEffort.OFF;
-				this.saveReasoningEffortDefaults();
-			}
-			return;
-		}
-
-		this.activeConversation = {
-			...this.activeConversation,
-			thinkingEnabled: enabled
-		};
-
-		await DatabaseService.updateConversation(this.activeConversation.id, {
-			thinkingEnabled: enabled
-		});
-
-		const convIndex = this.conversations.findIndex((c) => c.id === this.activeConversation!.id);
-		if (convIndex !== -1) {
-			this.conversations[convIndex].thinkingEnabled = enabled;
-			this.conversations = [...this.conversations];
-		}
-	}
-
 	/**
 	 * Gets the effective reasoning effort for the active conversation.
 	 * Returns the conversation override if set, otherwise the global default.
+	 * DEFAULT means no override is sent and the server decides.
 	 */
-	getReasoningEffort(): ReasoningEffort | ReasoningEffort.OFF {
+	getReasoningEffort(): ReasoningEffort {
 		if (this.activeConversation) {
-			return this.activeConversation.reasoningEffort ?? this.pendingReasoningEffort;
+			if (this.activeConversation.reasoningEffort !== undefined) {
+				return this.activeConversation.reasoningEffort;
+			}
+			// conversations created before the tri-state store an explicit
+			// opt-out only as thinkingEnabled = false
+			if (this.activeConversation.thinkingEnabled === false) {
+				return ReasoningEffort.OFF;
+			}
 		}
 		return this.pendingReasoningEffort;
 	}
@@ -857,7 +800,7 @@ class ConversationsStore {
 	/**
 	 * Sets the reasoning effort for the active conversation.
 	 * If no conversation exists, stores the global default.
-	 * @param effort - The effort level ('low' | 'medium' | 'high' | 'max')
+	 * @param effort - The effort level ('default' | 'off' | 'low' | 'medium' | 'high' | 'max')
 	 */
 	async setReasoningEffort(effort: ReasoningEffort): Promise<void> {
 		if (!this.activeConversation) {

tools/ui/src/lib/stores/tools.svelte.ts

@@ -5,7 +5,7 @@ import { HealthCheckStatus, JsonSchemaType, ToolCallType, ToolSource } from '$li
 import { config } from '$lib/stores/settings.svelte';
 import {
 	DISABLED_TOOL_KEYS_LOCALSTORAGE_KEY,
-	SANDBOX_TOOL_DEFINITION,
+	buildSandboxToolDefinition,
 	TOOL_GROUP_LABELS,
 	TOOL_SERVER_LABELS
 } from '$lib/constants';
@@ -143,7 +143,9 @@ class ToolsStore {
 	}
 
 	get frontendTools(): OpenAIToolDefinition[] {
-		return config().jsSandboxEnabled ? [SANDBOX_TOOL_DEFINITION] : [];
+		return config().jsSandboxEnabled
+			? [buildSandboxToolDefinition(!!config().symbolicMathEnabled)]
+			: [];
 	}
 
 	get customTools(): OpenAIToolDefinition[] {

tools/ui/src/lib/types/reasoning.ts

@@ -1,6 +1,5 @@
 export interface ReasoningEffortLevel {
 	value: string;
 	label: string;
-	isOff?: boolean;
 	hasInfo?: boolean;
 }

tools/ui/src/lib/vendors/big-integer/LICENSE

@@ -0,0 +1,24 @@
+This is free and unencumbered software released into the public domain.
+
+Anyone is free to copy, modify, publish, use, compile, sell, or
+distribute this software, either in source code form or as a compiled
+binary, for any purpose, commercial or non-commercial, and by any
+means.
+
+In jurisdictions that recognize copyright laws, the author or authors
+of this software dedicate any and all copyright interest in the
+software to the public domain. We make this dedication for the benefit
+of the public at large and to the detriment of our heirs and
+successors. We intend this dedication to be an overt act of
+relinquishment in perpetuity of all present and future rights to this
+software under copyright law.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS BE LIABLE FOR ANY CLAIM, DAMAGES OR
+OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+OTHER DEALINGS IN THE SOFTWARE.
+
+For more information, please refer to <http://unlicense.org>

tools/ui/src/lib/vendors/decimal.js/LICENCE.md

@@ -0,0 +1,23 @@
+The MIT Licence.
+
+Copyright (c) 2025 Michael Mclaughlin
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+

tools/ui/src/lib/vendors/nerdamer-prime/Extra.js

@@ -0,0 +1,926 @@
+/*
+ * Author : Martin Donk
+ * Website : http://www.nerdamer.com
+ * Email : martin.r.donk@gmail.com
+ * License : MIT
+ * Source : https://github.com/jiggzson/nerdamer
+ */
+
+// Type imports for JSDoc ======================================================
+// These typedefs provide type aliases for the interfaces defined in index.d.ts.
+// They enable proper type checking when working with the classes defined in this file.
+//
+// Usage patterns:
+// - For return types: @returns {NerdamerSymbolType}
+// - For parameters: @param {NerdamerSymbolType} symbol
+// - For variable declarations: /** @type {NerdamerSymbolType} */
+//
+// Note: When casting local class instances to interface types, use the pattern:
+//   /** @type {InterfaceType} */ (/** @type {unknown} */ (localInstance))
+// This is needed because TypeScript sees local classes and interfaces as separate types.
+
+/**
+ * Core type aliases from index.d.ts
+ *
+ * @typedef {import('./index').NerdamerCore.NerdamerSymbol} NerdamerSymbolType
+ *
+ * @typedef {import('./index').NerdamerCore.Frac} FracType
+ *
+ * @typedef {import('./index').NerdamerCore.Vector} VectorType
+ *
+ * @typedef {import('./index').NerdamerCore.Matrix} MatrixType
+ *
+ * @typedef {import('./index').NerdamerCore.Parser} ParserType
+ *
+ * @typedef {import('./index').NerdamerCore.Settings} SettingsType
+ *
+ * @typedef {import('./index').NerdamerExpression} ExpressionType
+ *
+ * @typedef {typeof import('./index')} NerdamerType
+ *
+ * @typedef {import('./index').NerdamerCore.Utils} UtilsInterface
+ *
+ * @typedef {import('./index').NerdamerCore.Math2} Math2Interface
+ *
+ * @typedef {import('./index').NerdamerCore.Core} CoreType
+ *
+ * @typedef {import('./index').ExpressionParam} ExpressionParam
+ *
+ * @typedef {import('./index').ArithmeticOperand} ArithmeticOperand
+ *
+ * @typedef {import('./index').NerdamerCore.AlgebraModule} AlgebraModuleType
+ *
+ * @typedef {import('./index').NerdamerCore.PartFracSubModule} PartFracSubModuleType
+ *
+ * @typedef {import('./index').NerdamerCore.CalculusModule} CalculusModuleType
+ *
+ * @typedef {import('./index').NerdamerCore.ExtraModule} ExtraModuleType
+ *
+ * @typedef {import('./index').NerdamerCore.LaPlaceSubModule} LaPlaceSubModuleType
+ *
+ * @typedef {import('./index').NerdamerCore.StatisticsSubModule} StatisticsSubModuleType
+ *
+ * @typedef {import('./index').NerdamerCore.UnitsSubModule} UnitsSubModuleType
+ *
+ * @typedef {import('./index').NerdamerCore.DecomposeResultObject} DecomposeResultType
+ */
+
+// Check if nerdamer exists globally (browser) or needs to be required (Node.js)
+let nerdamer = typeof globalThis !== 'undefined' && globalThis.nerdamer ? globalThis.nerdamer : undefined;
+if (typeof module !== 'undefined' && nerdamer === undefined) {
+    nerdamer = require('./nerdamer.core.js');
+    require('./Calculus');
+    require('./Algebra');
+}
+
+/** @returns {ExtraModuleType} */
+(function initExtraModule() {
+    /** @type {CoreType} */
+    const core = nerdamer.getCore();
+    /** @type {ParserType} */
+    const _ = core.PARSER;
+    const {
+        NerdamerSymbol,
+        Vector: _Vector,
+        /** @type {AlgebraModuleType} */
+        Algebra,
+        /** @type {CalculusModuleType} */
+        Calculus,
+    } = core;
+    const { format, isVector, isArray, isSymbol } = core.Utils;
+    const { S, EX: _EX, CP, PL, CB, FN } = core.groups;
+    core.Settings.Laplace_integration_depth = 40;
+
+    /**
+     * Check if a symbol's power is itself a symbol with group S or CB
+     *
+     * @param {NerdamerSymbolType} sym
+     * @returns {boolean}
+     */
+    function hasPowerGroupSOrCB(sym) {
+        return isSymbol(sym.power) && (sym.power.group === S || sym.power.group === CB);
+    }
+
+    /**
+     * Finds a function by name within this symbol's tree.
+     *
+     * @this {NerdamerSymbolType}
+     * @param {string} fname The function name to search for
+     * @returns {NerdamerSymbolType | undefined} The found function symbol clone, or undefined if not found
+     */
+    NerdamerSymbol.prototype.findFunction = function findFunction(fname) {
+        // This is what we're looking for
+        if (this.group === FN && this.fname === fname) {
+            return this.clone();
+        }
+        let found;
+        if (this.symbols) {
+            for (const x in this.symbols) {
+                if (!Object.hasOwn(this.symbols, x)) {
+                    continue;
+                }
+                found = this.symbols[x].findFunction(fname);
+                if (found) {
+                    break;
+                }
+            }
+        }
+
+        return found;
+    };
+
+    /** @type {ExtraModuleType} */
+    const __ = (core.Extra = {
+        version: '1.4.2',
+        // http://integral-table.com/downloads/LaplaceTable.pdf
+        // Laplace assumes all coefficients to be positive
+        LaPlace: {
+            // Using: integral_0^oo f(t)*e^(-s*t) dt
+            /**
+             * @param {NerdamerSymbolType} symbol
+             * @param {NerdamerSymbolType | string} t
+             * @param {NerdamerSymbolType | string} s
+             * @returns {NerdamerSymbolType}
+             */
+            transform(symbol, t, s) {
+                /** @type {NerdamerSymbolType} */
+                symbol = symbol.clone();
+
+                t = t.toString();
+                // First try a lookup for a speed boost
+                symbol = NerdamerSymbol.unwrapSQRT(symbol, true);
+                /** @type {NerdamerSymbolType} */
+                let retval;
+                const coeff = symbol.stripVar(t);
+                const g = symbol.group;
+
+                symbol = /** @type {NerdamerSymbolType} */ (_.divide(symbol, coeff.clone()));
+
+                if (symbol.isConstant() || !symbol.contains(t, true)) {
+                    retval = _.parse(format('({0})/({1})', symbol, s));
+                } else if (g === S && core.Utils.isInt(symbol.power)) {
+                    const n = String(symbol.power);
+                    retval = _.parse(format('factorial({0})/({1})^({0}+1)', n, s));
+                } else if (symbol.group === S && symbol.power.equals(1 / 2)) {
+                    retval = _.parse(format('sqrt(pi)/(2*({0})^(3/2))', s));
+                } else if (symbol.isComposite()) {
+                    retval = new NerdamerSymbol(0);
+                    symbol.each(x => {
+                        retval = /** @type {NerdamerSymbolType} */ (_.add(retval, __.LaPlace.transform(x, t, s)));
+                    }, true);
+                } else if (symbol.isE() && hasPowerGroupSOrCB(symbol)) {
+                    const a = /** @type {NerdamerSymbolType} */ (symbol.power).stripVar(t);
+                    retval = _.parse(format('1/(({1})-({0}))', a, s));
+                } else {
+                    const fns = ['sin', 'cos', 'sinh', 'cosh'];
+                    // Support for symbols in fns with arguments in the form a*t or n*t where a = symbolic and n = Number
+                    if (
+                        symbol.group === FN &&
+                        fns.indexOf(symbol.fname) !== -1 &&
+                        (symbol.args[0].group === S || symbol.args[0].group === CB)
+                    ) {
+                        const a = symbol.args[0].stripVar(t);
+
+                        switch (symbol.fname) {
+                            case 'sin':
+                                retval = _.parse(format('({0})/(({1})^2+({0})^2)', a, s));
+                                break;
+                            case 'cos':
+                                retval = _.parse(format('({1})/(({1})^2+({0})^2)', a, s));
+                                break;
+                            case 'sinh':
+                                retval = _.parse(format('({0})/(({1})^2-({0})^2)', a, s));
+                                break;
+                            case 'cosh':
+                                retval = _.parse(format('({1})/(({1})^2-({0})^2)', a, s));
+                                break;
+                        }
+                    } else {
+                        // Try to integrate for a solution
+                        // we need at least the Laplace integration depth
+                        const depthIsLower = core.Settings.integration_depth < core.Settings.Laplace_integration_depth;
+
+                        let savedIntegrationDepth;
+                        if (depthIsLower) {
+                            savedIntegrationDepth = core.Settings.integration_depth; // Save the depth
+                            core.Settings.integration_depth = core.Settings.Laplace_integration_depth; // Transforms need a little more room
+                        }
+
+                        core.Utils.block(
+                            'PARSE2NUMBER',
+                            () => {
+                                const u = t;
+                                const sym = symbol.sub(t, u);
+                                const integrationExpr = _.parse(`e^(-${s}*${u})*${sym}`);
+                                retval = Calculus.integrate(integrationExpr, u);
+                                if (retval.hasIntegral?.()) {
+                                    retval = _.symfunction('laplace', [symbol, _.parse(String(t)), _.parse(String(s))]);
+                                    return;
+                                }
+                                //                                _.error('Unable to compute transform');
+                                retval = retval.sub(t, 0);
+                                retval = /** @type {NerdamerSymbolType} */ (
+                                    _.expand(_.multiply(retval, new NerdamerSymbol(-1)))
+                                );
+                                retval = retval.sub(u, t);
+                            },
+                            false
+                        );
+
+                        retval = /** @type {NerdamerSymbolType} */ (
+                            core.Utils.block('PARSE2NUMBER', () => _.parse(retval), true)
+                        );
+
+                        if (depthIsLower) // Put the integration depth as it was
+                        {
+                            core.Settings.integration_depth = savedIntegrationDepth;
+                        }
+                    }
+                }
+
+                return /** @type {NerdamerSymbolType} */ (_.multiply(retval, coeff));
+            },
+            /**
+             * @param {NerdamerSymbolType} symbol
+             * @param {NerdamerSymbolType | string} s_
+             * @param {NerdamerSymbolType | string} t
+             * @returns {NerdamerSymbolType}
+             */
+            inverse(symbol, s_, t) {
+                const inputSymbol = symbol.clone();
+                return core.Utils.block(
+                    'POSITIVE_MULTIPLIERS',
+                    () => {
+                        /** @type {NerdamerSymbolType | undefined} */
+                        let retval;
+                        // Expand and get partial fractions
+                        if (symbol.group === CB) {
+                            symbol = /** @type {NerdamerSymbolType} */ (
+                                /** @type {PartFracSubModuleType} */ (Algebra.PartFrac).partfrac(
+                                    /** @type {NerdamerSymbolType} */ (_.expand(symbol)),
+                                    s_
+                                )
+                            );
+                        }
+
+                        if (symbol.group === S || symbol.group === CB || symbol.isComposite()) {
+                            /** @type {number | FracType} */
+                            let p;
+                            /** @type {FracType} */
+                            let denP;
+                            /** @type {NerdamerSymbolType} */
+                            let a;
+                            /** @type {NerdamerSymbolType | string} */
+                            let b;
+                            /** @type {NerdamerSymbolType} */
+                            let d;
+                            /** @type {string} */
+                            let exp;
+                            /** @type {DecomposeResultType} */
+                            let f2;
+                            /** @type {string | number} */
+                            let fact;
+                            // Remove the multiplier
+                            const m = symbol.multiplier.clone();
+                            symbol.toUnitMultiplier();
+                            // Get the numerator and denominator
+                            let num = symbol.getNum();
+                            const den = symbol.getDenom().toUnitMultiplier();
+
+                            // TODO: Make it so factor doesn't destroy pi
+                            // num = core.Algebra.Factor.factor(symbol.getNum());
+                            // den = core.Algebra.Factor.factor(symbol.getDenom().invert(null, true));
+
+                            if (den.group === CP || den.group === PL) {
+                                denP = /** @type {FracType} */ (den.power.clone());
+                                den.toLinear();
+                            } else {
+                                denP = new core.Frac(1);
+                            }
+
+                            // Convert s to a string
+                            const s = s_.toString();
+                            // Split up the denominator if in the form ax+b
+                            /** @type {DecomposeResultType} */
+                            const f = core.Utils.decompose_fn(den, s, true);
+                            // Move the multiplier to the numerator
+                            /** @type {DecomposeResultType} */
+                            const _fe = core.Utils.decompose_fn(
+                                /** @type {NerdamerSymbolType} */ (_.expand(num.clone())),
+                                s,
+                                true
+                            );
+                            num.multiplier = num.multiplier.multiply(m);
+
+                            const finalize = function () {
+                                // Put back the numerator
+                                retval = /** @type {NerdamerSymbolType} */ (_.multiply(retval, num));
+                                retval.multiplier = retval.multiplier.multiply(symbol.multiplier);
+                                // Put back a
+                                retval = /** @type {NerdamerSymbolType} */ (_.divide(retval, f.a));
+                            };
+
+                            // Store the parts in variables for easy recognition
+                            // check if in the form t^n where n = integer
+                            if (
+                                (den.group === S || den.group === CB) &&
+                                f.x.value === s &&
+                                f.b.equals(0) &&
+                                core.Utils.isInt(f.x.power)
+                            ) {
+                                p = /** @type {number} */ (/** @type {unknown} */ (f.x.power)) - 1;
+                                fact = core.Math2.factorial(p);
+                                //  N!/s^(n-1)
+                                retval = /** @type {NerdamerSymbolType} */ (
+                                    _.divide(_.pow(_.parse(String(t)), new NerdamerSymbol(p)), new NerdamerSymbol(fact))
+                                );
+                                // Wrap it up
+                                finalize();
+                            } else if (den.group === CP && denP.equals(1)) {
+                                if (f.x.group === core.groups.PL && Algebra.degree(den).equals(2)) {
+                                    // Possibly in the form 1/(s^2+2*s+1)
+                                    // Try factoring to get it in a more familiar form{
+                                    // Apply inverse of F(s-a)
+                                    /**
+                                     * @type {{
+                                     *     f: NerdamerSymbolType;
+                                     *     a: NerdamerSymbolType;
+                                     *     h: NerdamerSymbolType;
+                                     *     c?: NerdamerSymbolType;
+                                     * }}
+                                     */
+                                    const completed = Algebra.sqComplete(den, s);
+                                    const u = core.Utils.getU(den);
+                                    // Get a for the function above
+                                    a = core.Utils.decompose_fn(completed.a, s, true).b;
+                                    const tf = __.LaPlace.inverse(
+                                        _.parse(`1/((${u})^2+(${completed.c}))`),
+                                        u,
+                                        String(t)
+                                    );
+                                    retval = /** @type {NerdamerSymbolType} */ (
+                                        _.multiply(tf, _.parse(`(${m})*e^(-(${a})*(${t}))`))
+                                    );
+                                    // A/(b*s-c) -> ae^(-bt)
+                                } else if (f.x.isLinear() && !num.contains(s)) {
+                                    t = /** @type {NerdamerSymbolType | string} */ (
+                                        _.divide(_.parse(String(t)), f.a.clone())
+                                    );
+
+                                    // Don't add factorial of one or zero
+                                    p = /** @type {number} */ (/** @type {unknown} */ (denP)) - 1;
+                                    fact = p === 0 || p === 1 ? '1' : `(${denP}-1)!`;
+                                    retval = _.parse(
+                                        format(
+                                            '(({0})^({3}-1)*e^(-(({2})*({0}))/({1})))/(({4})*({1})^({3}))',
+                                            t,
+                                            f.a,
+                                            f.b,
+                                            denP,
+                                            fact
+                                        )
+                                    );
+                                    // Wrap it up
+                                    finalize();
+                                } else if (f.x.group === S && f.x.power.equals(2)) {
+                                    if (num.contains(s)) {
+                                        // A*s/(b*s^2+c^2)
+                                        a = new NerdamerSymbol(1);
+                                        if (num.group === CB) {
+                                            /** @type {NerdamerSymbolType} */
+                                            let newNum = new NerdamerSymbol(1);
+                                            num.each(x => {
+                                                if (x.contains(s)) {
+                                                    newNum = /** @type {NerdamerSymbolType} */ (_.multiply(newNum, x));
+                                                } else {
+                                                    a = /** @type {NerdamerSymbolType} */ (_.multiply(a, x));
+                                                }
+                                            });
+                                            num = newNum;
+                                        }
+
+                                        // We need more information about the denominator to decide
+                                        f2 = core.Utils.decompose_fn(num, s, true);
+                                        const fn1 = f2.a;
+                                        const fn2 = f2.b;
+                                        const aHasSin = fn1.containsFunction('sin');
+                                        const aHasCos = fn1.containsFunction('cos');
+                                        const bHasCos = fn2.containsFunction('cos');
+                                        const bHasSin = fn2.containsFunction('sin');
+                                        if (
+                                            f2.x.value === s &&
+                                            f2.x.isLinear() &&
+                                            !((aHasSin && bHasCos) || aHasCos || bHasSin)
+                                        ) {
+                                            retval = _.parse(
+                                                format(
+                                                    '(({1})*cos((sqrt(({2})*({3}))*({0}))/({2})))/({2})',
+                                                    t,
+                                                    f2.a,
+                                                    f.a,
+                                                    f.b
+                                                )
+                                            );
+                                        } else if (aHasSin && bHasCos) {
+                                            const sin = /** @type {NerdamerSymbolType} */ (fn1.findFunction?.('sin'));
+                                            const cos = /** @type {NerdamerSymbolType} */ (fn2.findFunction?.('cos'));
+                                            // Who has the s?
+                                            if (sin?.args?.[0].equals(cos?.args?.[0]) && !sin?.args?.[0].contains(s)) {
+                                                b = /** @type {NerdamerSymbolType} */ (
+                                                    _.divide(fn2, cos.toUnitMultiplier())
+                                                ).toString();
+                                                const c = sin.args[0].toString();
+                                                d = f.b;
+                                                const e = _.divide(fn1, sin.toUnitMultiplier());
+                                                exp =
+                                                    '(({1})*({2})*cos({3})*sin(sqrt({4})*({0})))/sqrt({4})+({1})*sin({3})*({5})*cos(sqrt({4})*({0}))';
+                                                retval = _.parse(format(exp, t, a, b, c, d, e));
+                                            }
+                                        }
+                                    } else {
+                                        retval = _.parse(
+                                            format(
+                                                '(({1})*sin((sqrt(({2})*({3}))*({0}))/({2})))/sqrt(({2})*({3}))',
+                                                t,
+                                                num,
+                                                f.a,
+                                                f.b
+                                            )
+                                        );
+                                    }
+                                }
+                            } else if (
+                                /** @type {FracType} */ (f.x.power).num &&
+                                /** @type {FracType} */ (f.x.power).num.equals(3) &&
+                                /** @type {FracType} */ (f.x.power).den.equals(2) &&
+                                num.contains('sqrt(pi)') &&
+                                !num.contains(s) &&
+                                num.isLinear()
+                            ) {
+                                b = /** @type {NerdamerSymbolType} */ (_.divide(num.clone(), _.parse('sqrt(pi)')));
+                                retval = _.parse(format('(2*({2})*sqrt({0}))/({1})', t, f.a, b, num));
+                            } else if (denP.equals(2) && f.x.power.equals(2)) {
+                                if (num.contains(s)) {
+                                    // Decompose the numerator to check value of s
+                                    f2 = core.Utils.decompose_fn(
+                                        /** @type {NerdamerSymbolType} */ (_.expand(num.clone())),
+                                        s,
+                                        true
+                                    );
+                                    if (f2.x.isComposite()) {
+                                        /** @type {DecomposeResultType[]} */
+                                        const sTerms = [];
+                                        // First collect the factors e.g. (a)(bx)(cx^2+d)
+                                        /** @type {DecomposeResultType[]} */
+                                        const symbols = /** @type {DecomposeResultType[]} */ (
+                                            num
+                                                .collectSymbols(x => {
+                                                    x = NerdamerSymbol.unwrapPARENS(x);
+                                                    /** @type {DecomposeResultType} */
+                                                    const decomp = core.Utils.decompose_fn(x, s, true);
+                                                    decomp.symbol = x;
+                                                    return decomp;
+                                                })
+                                                // Then sort them by power hightest to lowest
+                                                .sort((x1, x2) => {
+                                                    const p1 =
+                                                        /** @type {DecomposeResultType} */ (x1).x.value === s
+                                                            ? /** @type {number} */ (
+                                                                  /** @type {unknown} */ (
+                                                                      /** @type {DecomposeResultType} */ (x1).x.power
+                                                                  )
+                                                              )
+                                                            : 0;
+                                                    const p2 =
+                                                        /** @type {DecomposeResultType} */ (x2).x.value === s
+                                                            ? /** @type {number} */ (
+                                                                  /** @type {unknown} */ (
+                                                                      /** @type {DecomposeResultType} */ (x2).x.power
+                                                                  )
+                                                              )
+                                                            : 0;
+                                                    return p2 - p1;
+                                                })
+                                        );
+                                        a = new NerdamerSymbol(-1);
+                                        // Grab only the ones which have s
+                                        for (let i = 0; i < symbols.length; i++) {
+                                            const fc = symbols[i];
+                                            if (fc.x.value === s) {
+                                                sTerms.push(fc);
+                                            } else {
+                                                a = /** @type {NerdamerSymbolType} */ (_.multiply(a, fc.symbol));
+                                            }
+                                        }
+                                        // The following 2 assumptions are made
+                                        // 1. since the numerator was factored above then each s_term has a unique power
+                                        // 2. because the terms are sorted by descending powers then the first item
+                                        //    has the highest power
+                                        // We can now check for the next type s(s^2-a^2)/(s^2+a^2)^2
+                                        if (
+                                            sTerms[0].x.power.equals(2) &&
+                                            sTerms[1].x.power.equals(1) &&
+                                            sTerms[1].b.equals(0) &&
+                                            !sTerms[0].b.equals(0)
+                                        ) {
+                                            b = sTerms[0].a.negate();
+                                            exp =
+                                                '-(({1})*({2})*({5})*({0})*sin((sqrt(({4})*({5}))*({0}))/({4})))/' +
+                                                '(2*({4})^2*sqrt(({4})*({5})))-(({1})*({3})*({0})*sin((sqrt(({4})*({5}))*({0}))/({4})))' +
+                                                '/(2*({4})*sqrt(({4})*({5})))+(({1})*({2})*cos((sqrt(({4})*({5}))*({0}))/({4})))/({4})^2';
+                                            retval = _.parse(format(exp, t, a, b, sTerms[0].b, f.a, f.b));
+                                        }
+                                    } else if (f2.x.isLinear()) {
+                                        a = /** @type {NerdamerSymbolType} */ (_.divide(f2.a, new NerdamerSymbol(2)));
+                                        exp =
+                                            '(({1})*({0})*sin((sqrt(({2})*({3}))*({0}))/({2})))/(({2})*sqrt(({2})*({3})))';
+                                        retval = _.parse(format(exp, t, a, f.a, f.b));
+                                    } else if (f2.x.power.equals(2)) {
+                                        if (f2.b.equals(0)) {
+                                            a = /** @type {NerdamerSymbolType} */ (
+                                                _.divide(f2.a, new NerdamerSymbol(2))
+                                            );
+                                            exp =
+                                                '(({1})*sin((sqrt(({2})*({3}))*({0}))/({2})))/(({2})*sqrt(({2})*({3})))+(({1})*({0})*cos((sqrt(({2})*({3}))*({0}))/({2})))/({2})^2';
+                                            retval = _.parse(format(exp, t, a, f.a, f.b));
+                                        } else {
+                                            a = /** @type {NerdamerSymbolType} */ (
+                                                _.divide(f2.a, new NerdamerSymbol(2))
+                                            );
+                                            d = f2.b.negate();
+                                            exp =
+                                                '-((({2})*({4})-2*({1})*({3}))*sin((sqrt(({2})*({3}))*({0}))/({2})))/(2*({2})*({3})*sqrt(({2})*({3})))+' +
+                                                '(({4})*({0})*cos((sqrt(({2})*({3}))*({0}))/({2})))/(2*({2})*({3}))+(({1})*({0})*cos((sqrt(({2})*({3}))*({0}))/({2})))/({2})^2';
+                                            retval = _.parse(format(exp, t, a, f.a, f.b, d));
+                                        }
+                                    }
+                                } else {
+                                    a = /** @type {NerdamerSymbolType} */ (_.divide(num, new NerdamerSymbol(2)));
+                                    exp =
+                                        '(({1})*sin((sqrt(({2})*({3}))*({0}))/({2})))/(({3})*sqrt(({2})*({3})))-(({1})*({0})*cos((sqrt(({2})*({3}))*({0}))/({2})))/(({2})*({3}))';
+                                    retval = _.parse(format(exp, t, a, f.a, f.b));
+                                }
+                            } else if (symbol.isComposite()) {
+                                // 1/(s+1)^2
+                                if (denP.equals(2) && f.x.group === S) {
+                                    retval = _.parse(`(${m})*(${t})*e^(-(${f.b})*(${t}))`);
+                                } else {
+                                    retval = new NerdamerSymbol(0);
+
+                                    symbol = /** @type {NerdamerSymbolType} */ (
+                                        /** @type {PartFracSubModuleType} */ (Algebra.PartFrac).partfrac(
+                                            /** @type {NerdamerSymbolType} */ (_.expand(symbol)),
+                                            s_
+                                        )
+                                    );
+
+                                    symbol.each(x => {
+                                        retval = /** @type {NerdamerSymbolType} */ (
+                                            _.add(retval, __.LaPlace.inverse(x, s_, t))
+                                        );
+                                    }, true);
+                                }
+                            }
+                        }
+
+                        retval ||= _.symfunction('ilt', [inputSymbol, _.parse(String(s_)), _.parse(String(t))]);
+
+                        return /** @type {NerdamerSymbolType} */ (retval);
+                    },
+                    true
+                );
+            },
+        },
+        Statistics: {
+            /**
+             * @param {NerdamerSymbolType[]} arr
+             * @returns {Record<string, number>}
+             */
+            frequencyMap(arr) {
+                /** @type {Record<string, number>} */
+                const map = {};
+                // Get the frequency map
+                for (let i = 0, l = arr.length; i < l; i++) {
+                    const e = arr[i];
+                    const key = e.toString();
+                    map[key] ||= 0; // Default it to zero
+                    map[key]++; // Increment
+                }
+                return map;
+            },
+            /**
+             * @param {NerdamerSymbolType[]} arr
+             * @returns {NerdamerSymbolType[]}
+             */
+            sort(arr) {
+                return arr.sort((a, b) => {
+                    if (!a.isConstant() || !b.isConstant()) {
+                        _.error('Unable to sort! All values must be numeric');
+                    }
+                    return /** @type {number} */ (/** @type {unknown} */ (a.multiplier.subtract(b.multiplier)));
+                });
+            },
+            /**
+             * @param {NerdamerSymbolType[]} arr
+             * @returns {NerdamerSymbolType}
+             */
+            count(arr) {
+                return new NerdamerSymbol(arr.length);
+            },
+            /**
+             * @param {NerdamerSymbolType[]} arr
+             * @param {NerdamerSymbolType} [x_]
+             * @returns {NerdamerSymbolType}
+             */
+            sum(arr, x_) {
+                /** @type {NerdamerSymbolType} */
+                let sum = new NerdamerSymbol(0);
+                for (let i = 0, l = arr.length; i < l; i++) {
+                    const xi = arr[i].clone();
+                    if (x_) {
+                        sum = /** @type {NerdamerSymbolType} */ (
+                            _.add(_.pow(_.subtract(xi, x_.clone()), new NerdamerSymbol(2)), sum)
+                        );
+                    } else {
+                        sum = /** @type {NerdamerSymbolType} */ (_.add(xi, sum));
+                    }
+                }
+
+                return sum;
+            },
+            /**
+             * @param {...NerdamerSymbolType} args
+             * @returns {NerdamerSymbolType}
+             */
+            mean(...args) {
+                // Handle arrays
+                if (isVector(args[0])) {
+                    return __.Statistics.mean(.../** @type {NerdamerSymbolType[]} */ (args[0].elements));
+                }
+                return /** @type {NerdamerSymbolType} */ (_.divide(__.Statistics.sum(args), __.Statistics.count(args)));
+            },
+            /**
+             * @param {...NerdamerSymbolType} args
+             * @returns {NerdamerSymbolType}
+             */
+            median(...args) {
+                /** @type {NerdamerSymbolType} */
+                let retval;
+                // Handle arrays
+                if (isVector(args[0])) {
+                    return __.Statistics.median(.../** @type {NerdamerSymbolType[]} */ (args[0].elements));
+                }
+                try {
+                    const sorted = __.Statistics.sort(args);
+                    const l = args.length;
+                    if (core.Utils.even(l)) {
+                        const mid = l / 2;
+                        retval = __.Statistics.mean(sorted[mid - 1], sorted[mid]);
+                    } else {
+                        retval = sorted[Math.floor(l / 2)];
+                    }
+                } catch (e) {
+                    if (/** @type {Error} */ (e).message === 'timeout') {
+                        throw e;
+                    }
+                    retval = _.symfunction('median', args);
+                }
+                return retval;
+            },
+            /**
+             * @param {...NerdamerSymbolType} args
+             * @returns {NerdamerSymbolType}
+             */
+            mode(...args) {
+                /** @type {NerdamerSymbolType} */
+                let retval;
+                // Handle arrays
+                if (isVector(args[0])) {
+                    return __.Statistics.mode(.../** @type {NerdamerSymbolType[]} */ (args[0].elements));
+                }
+
+                const map = __.Statistics.frequencyMap(args);
+
+                // The mode of 1 item is that item as per issue #310 (verified by Happypig375).
+                if (core.Utils.keys(map).length === 1) {
+                    retval = args[0];
+                } else {
+                    // Invert by arraning them according to their frequency
+                    /** @type {Record<number, string | string[]>} */
+                    const inverse = {};
+                    for (const x in map) {
+                        if (!Object.hasOwn(map, x)) {
+                            continue;
+                        }
+                        const freq = map[x];
+                        // Check if it's in the inverse already
+                        if (freq in inverse) {
+                            const e = inverse[freq];
+                            // If it's already an array then just add it
+                            if (isArray(e)) {
+                                e.push(x);
+                            }
+                            // Convert it to and array
+                            else {
+                                inverse[freq] = [x, /** @type {string} */ (inverse[freq])];
+                            }
+                        } else {
+                            inverse[freq] = x;
+                        }
+                    }
+                    // The keys now represent the maxes. We want the max of those keys
+                    const keyNums = core.Utils.keys(inverse).map(k => Number(k));
+                    const maxKey = Math.max.apply(null, keyNums);
+                    const max = inverse[maxKey];
+                    // Check it's an array. If it is then map over the results and convert
+                    // them to NerdamerSymbol
+                    if (isArray(max)) {
+                        retval = _.symfunction(
+                            'mode',
+                            max.sort().map(v => _.parse(v))
+                        );
+                    } else {
+                        retval = _.parse(/** @type {string} */ (max));
+                    }
+                }
+
+                return retval;
+            },
+            /**
+             * @param {NerdamerSymbolType} k
+             * @param {NerdamerSymbolType[]} args
+             * @returns {NerdamerSymbolType}
+             */
+            gVariance(k, args) {
+                const x_ = __.Statistics.mean(...args);
+                const sum = __.Statistics.sum(args, x_);
+                return /** @type {NerdamerSymbolType} */ (_.multiply(k, sum));
+            },
+            /**
+             * @param {...NerdamerSymbolType} args
+             * @returns {NerdamerSymbolType}
+             */
+            variance(...args) {
+                // Handle arrays
+                if (isVector(args[0])) {
+                    return __.Statistics.variance(.../** @type {NerdamerSymbolType[]} */ (args[0].elements));
+                }
+                const k = /** @type {NerdamerSymbolType} */ (
+                    _.divide(new NerdamerSymbol(1), __.Statistics.count(args))
+                );
+                return __.Statistics.gVariance(k, args);
+            },
+            /**
+             * @param {...NerdamerSymbolType} args
+             * @returns {NerdamerSymbolType}
+             */
+            sampleVariance(...args) {
+                // Handle arrays
+                if (isVector(args[0])) {
+                    return __.Statistics.sampleVariance(.../** @type {NerdamerSymbolType[]} */ (args[0].elements));
+                }
+
+                const k = /** @type {NerdamerSymbolType} */ (
+                    _.divide(new NerdamerSymbol(1), _.subtract(__.Statistics.count(args), new NerdamerSymbol(1)))
+                );
+                return __.Statistics.gVariance(k, args);
+            },
+            /**
+             * @param {...NerdamerSymbolType} args
+             * @returns {NerdamerSymbolType}
+             */
+            standardDeviation(...args) {
+                // Handle arrays
+                if (isVector(args[0])) {
+                    return __.Statistics.standardDeviation(.../** @type {NerdamerSymbolType[]} */ (args[0].elements));
+                }
+                return /** @type {NerdamerSymbolType} */ (
+                    _.pow(__.Statistics.variance(...args), new NerdamerSymbol(1 / 2))
+                );
+            },
+            /**
+             * @param {...NerdamerSymbolType} args
+             * @returns {NerdamerSymbolType}
+             */
+            sampleStandardDeviation(...args) {
+                // Handle arrays
+                if (isVector(args[0])) {
+                    return __.Statistics.sampleStandardDeviation(
+                        .../** @type {NerdamerSymbolType[]} */ (args[0].elements)
+                    );
+                }
+                return /** @type {NerdamerSymbolType} */ (
+                    _.pow(__.Statistics.sampleVariance(...args), new NerdamerSymbol(1 / 2))
+                );
+            },
+            /**
+             * @param {NerdamerSymbolType} x
+             * @param {NerdamerSymbolType} mean
+             * @param {NerdamerSymbolType} stdev
+             * @returns {NerdamerSymbolType}
+             */
+            zScore(x, mean, stdev) {
+                return /** @type {NerdamerSymbolType} */ (_.divide(_.subtract(x, mean), stdev));
+            },
+        },
+        Units: {
+            table: {
+                foot: '12 inch',
+                meter: '100 cm',
+                decimeter: '10 cm',
+            },
+        },
+    });
+
+    nerdamer.register([
+        {
+            name: 'laplace',
+            visible: true,
+            numargs: 3,
+            build() {
+                return __.LaPlace.transform;
+            },
+        },
+        {
+            name: 'ilt',
+            visible: true,
+            numargs: 3,
+            build() {
+                return __.LaPlace.inverse;
+            },
+        },
+        // Statistical
+        {
+            name: 'mean',
+            visible: true,
+            numargs: -1,
+            build() {
+                return __.Statistics.mean;
+            },
+        },
+        {
+            name: 'median',
+            visible: true,
+            numargs: -1,
+            build() {
+                return __.Statistics.median;
+            },
+        },
+        {
+            name: 'mode',
+            visible: true,
+            numargs: -1,
+            build() {
+                return __.Statistics.mode;
+            },
+        },
+        {
+            name: 'smpvar',
+            visible: true,
+            numargs: -1,
+            build() {
+                return __.Statistics.sampleVariance;
+            },
+        },
+        {
+            name: 'variance',
+            visible: true,
+            numargs: -1,
+            build() {
+                return __.Statistics.variance;
+            },
+        },
+        {
+            name: 'smpstdev',
+            visible: true,
+            numargs: -1,
+            build() {
+                return __.Statistics.sampleStandardDeviation;
+            },
+        },
+        {
+            name: 'stdev',
+            visible: true,
+            numargs: -1,
+            build() {
+                return __.Statistics.standardDeviation;
+            },
+        },
+        {
+            name: 'zscore',
+            visible: true,
+            numargs: 3,
+            build() {
+                return __.Statistics.zScore;
+            },
+        },
+    ]);
+
+    // Link registered functions externally
+    nerdamer.updateAPI();
+})();
+
+// Added for all.min.js
+if (typeof module !== 'undefined') {
+    module.exports = nerdamer;
+}

tools/ui/src/lib/vendors/nerdamer-prime/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 together-science
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

tools/ui/src/lib/vendors/nerdamer-prime/all.js

@@ -0,0 +1,16 @@
+/*
+ * Author : Martin Donk
+ * Website : http://www.nerdamer.com
+ * Email : martin.r.donk@gmail.com
+ * Source : https://github.com/jiggzson/nerdamer
+ * Can be used to load all add-ons with one require
+ */
+
+const nerdamer = require('./nerdamer.core.js');
+require('./Algebra.js');
+require('./Calculus.js');
+require('./Solve.js');
+require('./Extra.js');
+
+// Export nerdamer
+module.exports = nerdamer;

tools/ui/src/lib/vendors/nerdamer-prime/constants.js

@@ -0,0 +1,261 @@
+/*
+ * Mathematical constants for nerdamer
+ * This file contains precomputed values and mathematical constants
+ * used throughout the library.
+ */
+
+/**
+ * Container of pregenerated prime numbers up to 2083 This array is used as a cache and can be extended at runtime by
+ * functions like generatePrimes()
+ */
+const PRIMES = [
+    2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43, 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103, 107, 109,
+    113, 127, 131, 137, 139, 149, 151, 157, 163, 167, 173, 179, 181, 191, 193, 197, 199, 211, 223, 227, 229, 233, 239,
+    241, 251, 257, 263, 269, 271, 277, 281, 283, 293, 307, 311, 313, 317, 331, 337, 347, 349, 353, 359, 367, 373, 379,
+    383, 389, 397, 401, 409, 419, 421, 431, 433, 439, 443, 449, 457, 461, 463, 467, 479, 487, 491, 499, 503, 509, 521,
+    523, 541, 547, 557, 563, 569, 571, 577, 587, 593, 599, 601, 607, 613, 617, 619, 631, 641, 643, 647, 653, 659, 661,
+    673, 677, 683, 691, 701, 709, 719, 727, 733, 739, 743, 751, 757, 761, 769, 773, 787, 797, 809, 811, 821, 823, 827,
+    829, 839, 853, 857, 859, 863, 877, 881, 883, 887, 907, 911, 919, 929, 937, 941, 947, 953, 967, 971, 977, 983, 991,
+    997, 1009, 1013, 1019, 1021, 1031, 1033, 1039, 1049, 1051, 1061, 1063, 1069, 1087, 1091, 1093, 1097, 1103, 1109,
+    1117, 1123, 1129, 1151, 1153, 1163, 1171, 1181, 1187, 1193, 1201, 1213, 1217, 1223, 1229, 1231, 1237, 1249, 1259,
+    1277, 1279, 1283, 1289, 1291, 1297, 1301, 1303, 1307, 1319, 1321, 1327, 1361, 1367, 1373, 1381, 1399, 1409, 1423,
+    1427, 1429, 1433, 1439, 1447, 1451, 1453, 1459, 1471, 1481, 1483, 1487, 1489, 1493, 1499, 1511, 1523, 1531, 1543,
+    1549, 1553, 1559, 1567, 1571, 1579, 1583, 1597, 1601, 1607, 1609, 1613, 1619, 1621, 1627, 1637, 1657, 1663, 1667,
+    1669, 1693, 1697, 1699, 1709, 1721, 1723, 1733, 1741, 1747, 1753, 1759, 1777, 1783, 1787, 1789, 1801, 1811, 1823,
+    1831, 1847, 1861, 1867, 1871, 1873, 1877, 1879, 1889, 1901, 1907, 1913, 1931, 1933, 1949, 1951, 1973, 1979, 1987,
+    1993, 1997, 1999, 2003, 2011, 2017, 2027, 2029, 2039, 2053, 2063, 2069, 2081, 2083,
+];
+
+/** Set representation of PRIMES for O(1) lookup This object is used as a cache and can be extended at runtime */
+/** @type {Record<number, boolean>} */
+const PRIMES_SET = {};
+for (const p of PRIMES) {
+    PRIMES_SET[p] = true;
+}
+
+/** High precision value of Pi (200 decimal places) Used for high-precision calculations */
+const LONG_PI =
+    '3.14159265358979323846264338327950288419716939937510582097494459230781640628620899862803482534211706798214' +
+    '808651328230664709384460955058223172535940812848111745028410270193852110555964462294895493038196';
+
+/** High precision value of Euler's number e (200 decimal places) Used for high-precision calculations */
+const LONG_E =
+    '2.718281828459045235360287471352662497757247093699959574966967627724076630353547594571382178525166427427466' +
+    '39193200305992181741359662904357290033429526059563073813232862794349076323382988075319525101901';
+
+/**
+ * Precomputed high-precision fraction values for the bigLog function. These are used for arbitrary-precision logarithm
+ * calculations. Each entry is a string representation of a high-precision rational number.
+ */
+const BIG_LOG_CACHE = [
+    '-253631954333118718762629409109262279926288908775918712466601196032/39970093576053625963957478139049824030906352922262642968060706375',
+    '0',
+    '24553090145869607172412918483124184864289170814122579923404694986469653261608528681589949629750677407356463601998534945057511664951799678336/35422621391945757431676178435630229283255250779216421054188228659061954317501699707236864189383591478024245495110561124597124995986978302375',
+    '369017335340917140706044240090243368728616279239227943871048759140274862131699550043150713059889196223917527172547/335894053932612728969975338549993764554481173661218585876475837409922537622385232776657791604345125227005476864000',
+    '24606853025626737903121303930100462245506322607985779603220820323211395607931699126390918477501325805513849611930008427268176602460462988972957593458726734897129954728102144/17750092415977639787139561330326170936321452137635322313122938207611787444311735251389066106937796085669460151963285086542745859461943369606018450213014148175716400146484375',
+    '399073568781976806715759409052286641738926636328983929439450824555613704676637191564699164303012247386095942144825603522401740680808466858044/247958349743620302021733249049411604982786755454514947379317600613433680222511897950658049325685140346169718465773927872179874971908848116625',
+    '1468102989495846944084741146947295378041808701256909016224309866143294556551407470861354311593351276612463858816796714569499021375899793849136855085849133702029337910502448189055357182595424959360/819363879309286303497217527375463120404739098260200279520788950777458900438307356738082930586032462601215802636320993648007907724899611296693997216938989854861043298494990214825163523387600982777',
+    '5896704855274661767824574093605344871722790278354431422729640950821239030785642943033153793245906863203822369276271050164634206965056233097479117980782641839669/3030306850569309344013726745100070601277982132543905537366562638553198167007159067544789592089960911065181606283478843359856123992707598685058297067179343872000',
+    '76631772943534985713873427262830314617912556928476573358548256872141516989538374761909611879922349479420014771499018155447198112155515453671128814488139633810493264352294560043912066253026059140653027326566801398784/36852092933388988649396042883218509607503204211148493545892849595498822817623842579026942621098851631842754395231561679671400197056377380063233740202370686144673585955581403046886083948450136247134308381940165804875',
+    '3159076083816399509754948610929467278257473888282947311280653574634802580912280940686954763313882823327077171624015737719617373932318151594325834524000275847475866299387913048/1437757485694188822758304467756419845842037623148461107362957994816554782989250555362514354661961482939226272309026092009962414616417412938087494467254146002233028411865234375',
+    '22266067259907364984531611601870291368272674573653403965630628996687370994139884833897773468149149664829922302484782423514167405397665098388400450149078982462318781750661005833037235183394221496186539779712428265837926417581952/9670030144664428565128962309657100138096047028794689249320859276197340398920725569428532293373676415359965773460364494998334259893079003125373872108770534788283842907318071170285038777091588292539102269617376180390982915567375',
+    '14604654564989239958569331443385369522850975185358647132770022716433280072271007767111036877803328768910274400515590151934676819262085211828028638417329558229123989556376108454497813055/6090614019162516693013973409650613208227889078878781039105047015752493519149314227721984436973374032279421344818329285207124280297611253861173835238379831004010748379874393292231671808',
+    '1901241885407696031217292877862925220917660047127261026827869027159993239567933534052663335498281439239753018507182016153657409777749792228538380379703411298411623469292891476969894084838876001545818141543890273256985768690847587711270930688/765116019778838839812655402103512685695769161212360553099732689795578904762091216998790589926057819838537805856579109910198553330075924857419395160755642371550113347465300208422126945265887065434116781678702741657275181694851670325469434625',
+    '139459806786604751793737926146840623607010208216289543036026206208962059593900745886202214788747453279179283344350478734275973878932538430194363355795823581315329311220701640235653288975569812161436/54371368534412517053056101353618694718215711767266376573138772968257303578467926450212293233332401067673270853953399269852376592855992724934941173346260129257754416412476202526978443681584633116375',
+    '1045669091124493070709683241190022970908640501171378776604126771144008324358233819560649021940145166254659028524319517244711645162132513416238958170819347361185944945680269442845829390112062101255500836072082817820950448463314034677353723256969344/396228259004446234921310936915931611736815598535963504660076315228798989932959459406702091180060429080345146735173591749448509810270759531977278642135591672189002006272326131885315743181289970885337574780897529347356567086535505950450897216796875',
+    '9912919238915437302006264477931031611447467070103973106567538528951878797932559935860738745374437522819124347510590800370471910492338584284092534264608801221235029062881964101996762011296996851893455828946521/3660537472668264151218961634689665210933936249986285290553357254224360417386515311493310199319523687171757653216994741150377508234317025158302057758196429623723072084157928224798322861732880034847243894784000',
+    '9263710175433181746575186369318246002919895649622127410824041370079225200282403368319370743363303164313395723904510539050157032684710468364067204876434546848634842333436957245275217583248805993142227630297924119330553308466662488683624783307023014909360640/3341177182697517248552428837661919299725031035849865632511882688786226888137634168024976033652753689210700218163621739078534353578510364301481093730054725078138658805025014615651043313990684347632166030359086885561104034510990826655289288319840595753002771',
+    '5116082230713622171832327542439052727465114322479570603905499496221224653983960598946033081212909066917137546065542953865612718836914393275681318667667521726785633638189373998191090501201427906618075889744489190209584/1805752553736060443820406101277706970767657006346276183748749630179442318063568286372320188433843729960294965366346522303898609655762491623098453269916163621089005711823488749297418113474056676109581110715068124438875',
+    '246569125619713282434448566970352231845414317018379160824176638351574938993535464763890962336882760882398479702237564384291290459961036068916857265499633061660562532011248501476114401629839742058389195725393702000011860799793778295606988057303225493814005789533570432/85307063020836305797178273029353623060860009152114361453434032434699636078115114412588719432277441055049132559782203988387794711585368296817222565434951256788867244687081233632650953850383220864394261763844194948389861147622944651546912394593164406926489862036343375',
+    '133672026303452911046163998480860917119290576658330909785707604886881155606725822685088929236266583416708668502760907677019598002175122453170574729028452721476464728566191464897928696630979863154661704374206171469014225143/45398130975270785045482567762871405072140548998125471025451666500000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
+    '6041693953360002800224091673336562508913199995987479264605216252220579740134601435770085920869376641180763419907442721705887169884230643795126568815123647603047739799302562095542459344811429882053086550900803768964612193941424128649976704727183797495759082741166938351872/2016766992122395667828553277997478570503475626107286343497917705446132017125079612756035254750822860815515899557855166824523851779156336235294914777307802256439645525835223691751931866188957324792276149549076500784191791380803500156776088683900346065830066370370083309875',
+    '705868391597244582764749229356331441978820024796066870551110486625729826111158236686696326058778874201639006234449557592353247542995871491078308187261304930042019640830629526023972693107193897009168955674240659026247094657679060/231848642748474339277532000336338632910990823562381469441716922006107433404523316252618490265927265734670539384485699132080062215196462178933963957679882342083893417545858074378754089719547920901917516016346211301054206383643383',
+    '101832160604157943093944673541651013907278188571533075311673249923948856034633446617630054761681006062910980371900782781226979391765818325065031889334563981235894369036439929651260587335544056975715076598739977065390678221999918899003881778449092038750712969437519295878491018112/32944277910571666002449086492515464541550138004002141571670657643770713783329063548790202120805341989608877739811787937782240802963962520261844114327432160788193314874913687387269408387417806176202979244637915812905426565263196954203487934225589622864145960079736633434831996625',
+    '10655703119271468913597640479490594180964700448340778168715956712130636958373270202484276402718566314881119559090842449610957974112230306343486091910217340665146602598568991520563987490686996746558858366002301982443029430290679385551/3398412687366638541233365137084722368200311117891192348532156645374786104142009695796409107380345795998400850838706661851176885183144928701608654514812261697598380070746520197171576610572921007069104300695592751543563472456384512000',
+    '1903039332876763837419920240543738799531131775028971323439870868730321221615515008394327723508670975623498588291298064320786970626232668956372004004897872810230178526101184611242511193415796638694370503100219710864543168952682617801833318493436174387568067811938490953495819438108686336/598806534367503338307287246320963280558134937382149405305466709787179429317914803617527827862441615350396864359976273212272586892074799651088317544101755361439294687323233086696182687664637422796995789967075271448560870681210580691574924544896656175563265378514188341796398162841796875',
+    '525573915563826130963525826191411949262846916750432019596028344808298471293378917508549164993368392834023782480702893643486699787870059946429810070222126260200026332874480239090370088123833491499400991181659445914352500247596757005142623368/163278727324937389095822405034435687776345799835442022795533783889356344755225815267819508608559076191292900367982490827396869405536484846115955581043091229202052407483776587687967125885665493681707461345895999542381476164157058393971431375',
+    '9263815657177858787273494705338516861045771674838057329170239610953039987023429736752079544014780707408666628475997291124805562998227296677616204140605356257712022384368492575381355563976330347792504605666631512343447560301417325154003481040250148561839861837778597346623630046623751094400/2843321709948499955095590862256744532227698001408929142548057792217790532624003190447363578048562448168721539177458065482170148482375585867230123873178100117094533143052886527452665480614620123764036974180917207421482431983407742154634391264619615289225747664532332469783301704643254076601',
+    '407959339726114455622180187758753007349209016396248763075759257357925636039752474207685682218422721827857994768023399625060206708378433960993946156803948655098667156937949174400873748557248801874735834957795040139401560494087476967548060208243867/123780218751812156744401121690996305978134694678934447237402511116731459214498784497436358160964198336874043702652746834763131444030185151143987331404604087778514863973633941401826334750268416015224906056576641018962863645043976537664227639296000',
+    '2547676391598917379516698439971914695230548782904479778605691338364453606537643088857116141939170899135026552016969320061900926954008522781162186995856580955090548471448276736878300717869625651893741316530109438876067419826217901657017506157997588944233677467357220316084583383623602865379325184/764562034757392298786420374672266498815021229519853724850874576419885380830752931701831256959159800764672605004880389358601658343203513177084389490286723240185146570925957286083025676875197029662038213216541352875570101363668917766225709569356861275434470568767077844675593176178611021135573625',
+    '186545352286463730559933346565311535598243666022232037054735807289501173444103692309735768703898330430135399033529355360391658728987379385732098960609744313878477967971557204207043802935782878745271859468248704012618254203101767841517569443555143252/55399179641621656233589820996143825959365789093262978988289445625153099592463372579496245442338653053662134699646413817866770218574795378644415019944304868289119443774932782235638737888469746745621382139263856603239588594078668393194675445556640625',
+    '664884440164786473344854955309049113269357314957985265728106924238588705533437169796551912202931185746193155801905841712503407258166135075966280435780812714252670362202091663287095423712596462690753468682634261029392794173636943978404002804413009590005984736612421172979101972556772005594499779860608/195485517776407145286424460448995460754674039560651791192647586550615878988380153730602665795647187884543361218962125172808792176382956599256188706636727418572541254480798303566840010217729386905041217793614214518363859058348249961790104618910877813067510758225302884815410347238200133693756493703875',
+    '2614957283934314904315471338485451166053664494383241929385424599389309215073267052860464009981063483440201193771607520572077231889699858482582363845275452280606276949653970992719332472370351170732899676316967244504534154616036371979031399425846100527685/761493664432749089312665480773496290658029971027686543404885407644062485746072719559288231362060149626237939029641098328278650939665665969011529293869562636656650999759724704272743235210867676873525147820749560155294022488994426729939894753293900972032',
+    '124843380518493746761140367283007507854364503961156704095198010255465940085534099747297600085903814014415830785663764373057896014399822131175202342399536439284123918855893825207202244831315575594886675813256448846863723093240955901916229136393454605455444105444987028391748121054399538064686074523506176/36022228212051654395480210378626648518430280334458144892889271272122662467638331091863215146548048144675657239846337165813938424387499358852301016926312083940212100001220180762189978024821166744964908871443681332664798940660421469519997746775275873085770018269706847741064037876137315001228315806659875',
+    '827992369063043155578730871896750570951766628472810506926098505028264552046829097082095665194000002802661600196840639204300804225352337632259980703832713031790922485730615305441309917696044954289187837653933158950774246017223571461858939407386087081525130831392/236805932823686534991153393869288530368011574665859226704279685567723830696754821658770176385138917722808377962346690757191122309876922069867472518117628639913077442806147910884267694879089753138429767401700283014143248445966474839193628309668702223994071394625',
+    '17347276886878323736540051321582548724378497839789943634071026331001588645519865992773157565595886250230140452154269197770615097377486013097979087647774513500701793885978192218455687078883766086309728287172567466406449372659680040183273634701092561727514713494914793425407149186041796935055187281744386432/4919325621804683623339606849970832094714371903709195539440424738973575902329797546592497378000858196173718145883783709223158260700365224756081275272021856393735663399552166737690038832550853145831185979094979556715294990257315369124065787473707136464772247917156232366320267601622617803514003753662109375',
+    '137984231830526866236186357461458917020538108058615632801298091031540729111527734872044790487396302545910108285921421417358113055522725197998483383380192391312304647004240060970929072498293210057120617332323445379424867965764749534125081131327565507524502163460761/38810445792642817561168950890315210470940006613819790543653745327778579787694809782601777514116858514049585074667085399925278459138508514838268321349069481334967221455722811414399738756151414906092225265355449011152267068726417045644222323488445626292574879744000',
+    '746567120547823334914136339633766098626636643449144032626270358619125402826113269699709721071135471625588981126637674402048519990010499180844665151971356149292818375448504122545400227696621572263621729512461528550588108384619064912224884465737417596190735966915167530332762203074440688676123756162572829692160/208334337057923929636884170505570363171441147899816815785150954417598643614152856767186132467069365605496210036171429712485182162940460120834349006784956522600679357307849981862006710239311750261522832996877712350330290831638640913932265004107623954913155144975252743257846945609734368518424172846119306643431',
+    '64649371728330695076928013661001819989330953381731372450140483779536126948957993261299287753791770622512248630224724990234903928056275080682537641377393210728546364176267034339221558641084730052304770498929958838997239635790469536857863963589118888238069738647239076/17903951498200212327802847425913723358452100686246224008745414214690047078122925247086521362329833307849817944645647750649290248110509395628305970523384831671737569872597295947593410067364379687588919135621621162007748635920864926867870502568935739725312687094047375',
+    '2454918942158003099688922026016393688092399295166304634317616773083386087532869193458590448918958337530406410803840837646465522656670050113548208618655070231274778592766244282964463702354872753657766121825196898916725498553882689210280080206627916046484942827487726300822318764058084323314109595329304407466188383616/674880185931325925966586583820010578979699141814417326552629206140252348822939845006845669570885271576698771404162512001549922909048916000017837898649100825976232784446638776021483802989797501705685620612986771521390439936066527738682396560462899753657942715306792783283782238662155922082005591512296007820682995125',
+    '74018558041066162916454010680594042518462756234254788158141115244349044958441521749277686851928706433556285971088455226217644009628399441967508838553345152310730562224910795446341601049647392069373970101491741830623078126344928804029524181578945586663110848142571149861/20204153620006780689923328634586091101021423979622170579036140596085566172775051595588438592742563923428900864000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
+    '2127032036758045513335690185608563023954009095206088224487365541995326714285119384743928987635752931664240752323937321097955456543854943206092931247498833001499955456190701695430459583885125382086777607021670447795321669948733328973350279846928613949120929250312666393359442423066212311060931469017737106028339882830848/576612418511902928757340062840968526862381326698309578771238715462180282212422302261044980131594522407066369222998903808960617461164985318633518680304995784614308979881735537678182134128319596636920719106506829571072447362052319438091347699720147003209417806230149598345068078717948025207635448205253184540936478445125',
+    '35289653975561083576641954928762116897061274899517309102784750384002335187117263273488751066569234386120759866204372398611196356888479036949053282301027789530999737306501029700128744408015642211359442183943916106790666114870974212159410284751571905275610921784716184508440/9503006066880728386808143045924119024212377150217533250562188228062174064693375135306438120385877320162710918716613546077156389583384656340709638430674364232343609717735574035535102953482366914421205216675248471695111720986346092738728929878538430662191272737183832556131',
+    '102633551023964794485575491065909467125458972250222581133681080524371507544152979467328048718122409841060527545925136196267751819689935599599321090571687632103850847605493223603751038996548520557330016046032671961857623066292962260173840972332108111505971231021442896036760967107060309991355545554631003681544611731245475968/27459658121882266328752886605529964804078316737648012166874496015808620265471203512606463219297059547428855195782384236337998738233668399173746663289852416697917397644234441300570212555870401420579737973722145663287124151049692290432756231390864184491891697469874600345958989433125942336757049639797225309327019275689074625',
+    '10034393558388390065766795008210457368713365491566387292163814915435906649268119060550511145023450790393353937124495488860451123302412204483570913557762460385297770427946219119911920640306914453207097103853766023934602534502476962159682750262143380527529536498215384467975023/2667919902603322771586358077760621955455470781865624844984169443739075976572061827709528710108877015489050369589117491611045518221354793418884447632063538994046714401229510497599783726376490260140723032102883617341970952663947646017489439179953454964374887388652792446976000',
+    '248528145263843375390386172800048509380966183384567983242213959113927668429802237067505890436957693495616107089384741585283620097982859345081736730899912519273262934785992235852866637878831878448348444611412764161078458068549719800733237024285525816723480868704742804077255242682077291713092790250511567621735004237450946304/65676865669148624809340872151906045781446981664561196686217551358486802274698228825404698950974939545099727242259547145392352658637333562345477931951890984276718673618736565926663528625796412420753961231404680876558659735251469326707567479071881966875336951133475135427640218972722939427821842173216282390058040618896484375',
+    '7805448718805635696495809414501206964843262114470109146341305656318015059743127114324245035489577134938579856003956861881125856595981500593426840968087618241785931128978516340812066502964561231235073012672356530509663384739132686548934288703179479011016719045530855033205271548/2050525178024039744126592505352202216905491833360272553169520915020715464206141942151086176509423406413311520838568324134077402841030113427309725873344806030836314500267104070131451720947531994814710189000076651895520222646974590481497382830325485174899169093049299764813276375',
+    '3355325071293197839434119105039673324264765809771192815982246040415580387729382404624613875653005261578877047405365032178619450963731719777167015959920645055600439987161800547901539269321100559393048973255388860193948274255340335876890491746900991668165565729269698196233805991206691196045182214641935483083662356666996922240/876402579119117579582569839757462461050855174353108858954282915644790659429341853404829661899850841645529640454766173209897510988090318303454542547519850473808789222552969933222203420847859171250332350076509996295844203965564448154484566493395403967626596213792922784509892086361572955175655987334882030766001799867659814117',
+    '218871061991045868372866381545267589365410350294028138778572466235486397478028823720846191998825628156716190463263492304639890659254282445466806224943413446008645087186307985343574807361972238230520975439736199291019544576443791916302825193643774360055545186783819367378492631806297/56849560726416896431557940314760680962653658127458002233782028041537121216487790008085876994020812492987733987414743604239935223783349870516284048368761617736127892160849065895223288023531930411718807065209903593668117085505482007061969339237404945180379460053180570404846043136000',
+    '5008685108365226931582937964451700746853986170633433728409171904803795018146152804690759530990140552460596075588463394200510044617816085275660078502126507209302951286606953039953843685800941558212440519542602092919776366067720586295390886070120828199562643208637974347390938772070049344991272621102622931576339988103674070876518912/1293888539680354282541277646947380627241979967611883341823378331667976045287311988103163380651334828012840330710760757271860219584371109472132211215957402251594055009937397184768184517621978947384029376766290498101728971145633139541827544539988344772578184316843734267915665730981857376872622787627370859411909330227080697966353375',
+    '15388340113525711660227566446101909585796746979396093776960989868457211684028149502578116456785221720682202816140911944661051001675127262774824593420825587319436537346311831003212424497488485098543512314062112948777572038731823948224734505930748371522309451168088057190162878224801232/3954220582960831691377435160890656173654063611768428458807273708040518769541211737927975894584024448193835165167801976423275767590502552964407494549049777006346189436817215329891530811451811864579644894987864267389290848598289794977382504890216219362031324635609053075313568115234375',
+    '5099039333987561374222193551155323470675617979816941646196895589439391685938046865391119484510329634015275893520725135141878751153360264368353595348921951280561029028912953500944814771064409611917475818956659775131751121312316084465321917769679881052144364834485866477379437705913911371481828140817759401117780199246301705600020671104/1303503600297679371136943454060319958680553228879031326679449263682048703103464872914972900105569835004878963701599765030590097739639045890060548760692125546754294514068052902543220382104483822438283040090444827980927544440984823535260277595466339403795403200720622852069244768910603820007632395190204569927612348189089161551951106625',
+    '4902837141334073026145827027361937996261324349722726869116185158777439337041263482852376194988371853413467559557923410949898048139830183335197992754748294810838187068126867611615800383834975563313220497573778480109264178673389149671194149749735833378557143135481387904961537942569904075/1247045310545991266291285730016853118981099516935251861146038369985109288084420528171217942065832292739130145780833406014673689119563698528225048800794718789218267628507713621235056538202070171596177775095071513194885568843375526804796016261173388452184505503341132236719484809714335744',
+    '38114743522716832107917466438257616720476488812538316101658139632867788464381862291240727309611460187159930652186486096300862388591521625093237019662273764387591494074792574929490381910446287947994150655077877204446864004067956087975012773988833339521775463977233068498404144221045837190392670308437391686081418318624745039402145439223552/9647001083383999453668111809775451078976046488746916070976218645431946648087171586252172936600115032316383427265217993193444199863138429602138841976586190525451324093772097241349417938578878934577091671046050326087898259692917931230974174799815198493279413438192301437068820185757869608523761456160341754512329264442115351926967120404125',
+    '573695055225225727008803730767518906490704995929177617646275646884555707960986625481944101622708415415988844740028718027554452662358957933526173824325955904005404113684003841990198157072540659184995738719040024647370869010473254071681533880576462368600901824622431045529064651675640055917092/144509482511118816399089096021290587489594541280398871255876563615464628718527634679330291741479135415168539765887291789615790513527330600394937614433502341116068305347468133950204152174094704092402978083370792135432486240914953928188835819767755172666693219213868545854371103120604946200875',
+    '23876960329653589647925126180903391687666378233201794403339630995420215267415575142266707357255726330536094448314199602616026935251126469221925945960901748679919435908556550271504767784553484434363646489174587463466333864577705745452492395785557425904735048180164697040313528831173448025400634629163795223739061661461986923675833880378496/5986312408594306954013526197465608559068621248896320652512228238115589875514604632230098997609482248000888567135685167138762172475788060284232459813998201719590208742091697294562538265829954186149162974972471533202880368317237508987477069872431064075005305838801862900501819963793062041081601844759452202282545840716920793056488037109375',
+    '58168289917567723171226992383559866214094157894992327555495441698028867727845766488121900626912848698952863438654895252811583144479300382761129433911280049009362667380001406579175563745824368613319103673817094498117944856004415812877213722455299491145649879676787079744410765053845551958756701/14517067289347903655500020160671113450349743650636953726251191692074385521975132268313263723831804150872238173602847065423463131917373356798750100313145228608894881457107689499956903046984443545789053438946050974567665049237414588435796381674590098629779384355275820782532479708807512981504000',
+    '728621890568281859295409481422447012528302594365693410763821707074444799793690738137592101239862736313347273167450056625929591960610208335290882047413011571781161008296084630072829079783328937418641417642857196346026366370059522990813537731394823630207433267854616768658990289454635793326766697884798538576055949457122067828153655416688640/181030730759516991863708593747964787874073354051675597050399087612142539517308720603687322924426591889179726492403913356461908748733972707460063017057809060190437917851790767968877215795679844983288935075688219234885360839984681619084834228226744165610073685719017596630302462070188937998558312507638434329299017584329479516410907786681093',
+    '86855946923438322218622470067224691860808273886184997065663554841573982963995340977083049132518812923329423480393306918856650577072525633920456721265953575424233701929892019410099166322511413146891121248381648145391642571638857576890568882512129960291171866772665863159474602604647289052079991768/21485753507365901947528588896402264670781310878547726104482740647554738151100954835784115119035980523529677083504495839730499664052882400915208251594384038810917282207449860876251558307288700200910747338758723324686939379138206117634546981163355060740270734146780942696291669461182599512320099625',
+    '2158989152301022938148680102142188531448821359505188055264665167313418619665693092337665573150374231484840948447637297247277576415460889296724813940128955070240137590073233263168835678714131062764247434144994737610229909964847568491446606012581370840699582055341626266533733744293929658949697805855362114229666626620766245630122333733703618176/531794915405164005613733454597931482878479882704956110685223892325074211694837836221759995948610212818642789132749082430059593652854659130217225506942675608692701447738732031302987802196501895840510235161825501235133794449421919927396142470196961877376701957829921152848178076410141813926924749057304222282687697297216661687583257901415465125',
+    '139432548574396829074586704387656697097760057897628994548358619815052936481650396157428747411173567801047221928593253479330480454469358220685854351236980383914223693722868233819483137401339800304943891968050399345430243790898955416907228948287367356990263740207046902209563417267686591994743547621/34201151688775214071963206765436083445901621442002061707492082843232231754829227303539041286301398668437202547003300396162741375435703188500000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
+    '11008517174872833286150985180322584448162884832099344969609291070844193524816852920942383850580217443209402836100467940651581092350600329145627967515818684442171571156446321228596914355704205623857871497315955269266498229823278800717909321269179839084452384509142712677235552103459737790674103994445173074670347080506698168482564009465276165824768/2689223396936080856855299215659204161946704205931885125148201643087176556822542895325191478283706585400237901215485150928036895428721912118467760766508162631903585126377676412573187912443878232521444786090510891599171741773242011017926658231638022943018461086517502584854390836347781674626615709751386455292026775663545470794167629144456268750125',
+    '16108638074211260588800537540680707641986073914251424878121255234668558067988171568946079848860335948991834525552515669040163026131919804987340113244760738846884911038097907756220945883750502673899084880578229601870882631165510396775126850307838505063922101682333806284668762825609556049426829531780/3919363961344261777100658318137884299575193089462944554282218278496298610828757650104922583359642384253066896538203596057302203635134833545580869871333892935330950583664400555463557735723364497947986885146043017010159347046389604172186788902608216894094289769850517098027486468084407618748895626853',
+    '34420755849180279597302103726180110022640946692592540634353734157479505420320000324260530767186132260970572450489530034440214259559325114511265075416512316229177952140217732655405289808326341696986755141965043719344169685611217958619102774617224847284122901023774956887687026904767714958090256282893003000752947427857703259704682455375442735857024/8342030311716679826889917494957593165464748884572298173556257652389845294530325764837124998293398445804458613956489096007564811101361266196542129764287084823604897187311540561857741285793447174119667215803837719660675298308873496219385226998078648428368061868944322478384684509466965129972030932418920415308276430355882329457342937549162000252625',
+    '345888075261020004071220843714060353763382280664960929903544964118831237876694384053904571498830068831026644303797377762345709976595360421502594656308937649239978525713471393570536680412814805076323426256584504251728507416368609420882442293831684681071553766603478479006495757222912500012444787804577811/83505703731469734628961395063481893801938371516752417759131774530720075262459158384433785006689548434701904106312038822969658455364219435022841597243178757423598248565463985786213156556523685666430799283870548238467817226915680747412191245046634279766450629886904716776719219698922088211154187845632000',
+    '1061717830619177527082296723099890392273896386613997004874669053445943252046748251883532634529759169500795452576392700472771365240996842610207274128102329096619028487369622001737128463631016494371635687841733644339636164570819431573829173533941056258744442930643735587780907310433371453992062647737259587563398111688659657406089003293576961475848704/255359631537215747979895955806995352799574790340218399351168178555478073997876110889483456972687438702262017800167048243754141722496276537685853311434069991222324039005160057724073156957530106623908696241268268096879569794431919729620178375212905203484165745866913773304319069321426245521467122472046370356725530914587807274074293673038482666015625',
+    '274122944106300296738399632684955400761495830361663966466225652918683099779465438024846903286816813856490888796372134557295699980528187779624865098445756013563535339056233912394908544185885547842235097677765325396255649207317018754967666450708249125316192200151505568416495274671679500594656671785202496/65687592621976546250581560102201535533608158256953087745856906437400149205693427285162333502528793675585022025602144243543064185647792948495372442630333800126269123531636800213405254045262127593759539706750242430153456891792533267948231185296091297979933562727112487057234422009426868531651634706262125',
+    '7842680480716516803148821198697967237136721860017131244266974996267074742248599085253569637183007740566941125452215834642683053334607896723447140851344501084122965014242091312411884985569341166545074688756440728922408743841592658677792796881188604773469108807869960161395759837407978596679911066586626885830991556090978327508459276025943279064965688960/1872528612245648675720382138045071131304652050696842872529163720558126655075937845539792108048310219395746259570506175902206215101518698490144716531697689534559827422735649881381597761684154409796315455445459537515308174919488497154409643876490472215352056502193150125644288086294418253309947229151074464928874881827227706992859640236086417889990541889',
+    '5300824422251242070074569186825929119848111723012841627275830216301188228660779008353049603527567784119877706984722171178137272986345560485784907345500893648715341273841147320288851034078863843374665850852481747000237834238703248634174397792745914847774297223176674917912406659831206869442510948965571661/1261140476013707338477604677428573831791396352814802149994640617701773078174882455512668089072441176857892331468691160991310474734143842336092636848492066592397892638052212250229129355009939118431643425836944282456647571558383755315238500832868535816144280088644939696339160092963629012001958205063168000',
+    '2220223718762215584659309059880106334425515875615107369399767892051551634000614327272260081056973863669004224981561870246078120862256383581012183852291444462730018546753183156982897386563561418424093883164027305254176874653780425452987066512563140531367766900610414277825262239199580925879453806414860409441845631158680721091621460775043562065815179617536/526383206607841251253861841374779803798480623722760367843070466043030228662340154304405180907941079883976168609082254331465595267209149963786388600028701073430773581228212441424400748220833542964971495005714483235359479470452593264280645360131482713147116366500300066771223383007216182988263355451923333319170174755334598973202740108032097242475554128875',
+    '521891797109626296684891455959263713257353500867652268541535940159815152120871142196535233326890353914761242025931373491906127275561002910157909306979093246574207104081108188995072105948138299097848175016082947174156278439986705241571619793059501724269644447572323501261424770743329858038040685313621446524/123310256826873923765604825413207481739886340225713108649758575106598510022338480189649787216845041382860899099250547657534972156328080736149239332330143771138115695598493059325064119176038137294863053148618656356436332991079150723235214278848602671333076219529535123842212129829931654967367649078369140625',
+    '66086044538329677372986118727999622900471937619891337714357792768200341519193500393739322894033303245376225584865369486696276607060432449792893028061817203932068085863800494054274423512956136695211796751845295921015953538329385253280866669403169919614982155350899648626481405781514434761541281229159396787287553493046927448595964103589100429722948913403008/15562137339474350565671240515273666798063901504051979980452491653975250630723677279081058884163396938548780856293034775459223871281049026140999055923743471466471830572672766633086347312178711643724485955576579988182546105048041649947277672869613992334541438784737993706482731696809943027528882927942967419447250586964258807454003775693567366165507144866375',
+    '335377615394100148751647837967017467711612297170079949298328061159559939969228226474615711044891085626519877634842694983669611974807129333052471799687426665556738316626171408219730853872410792831871526174987402129691897433888027072807302411474690613948951673562473758814664346259109886876538510453475290967835/78715592752271462306588358880337347638000605031000575876214116610339827495261512281635361568951675037834544811575026718101166562072917855004822606752296233435017284127594847656529606648345533195437635894948829857913798336356647286032372695130461573940500785137424365840081503133157308796505622439791698116608',
+    '20090879701618729602554170716780970848925039917987945471322994867171660307998603515745066411687983450400412739285577269751603921163835619296822801840348319742203974023505186187060251544248644338412667631232247108675504629538319425769464277309915502144443973397371136256151336255138506001292355330875114245901820438821732843540725116728866301271466614762497024/4700223519410528857298732096729483544820841497820611795617923063440946097326817340637303431283005509904481323205480729806879570430868897342398783028649633951362398196137429076844504529051072393709154483678349272930361110568616112723747726853614661953537957117231900032044221535502745676310313569997665352252492568100075191900969170979460298189170486601502625',
+    '2904778979985524171206573028445379872240558084236464200857594814631031581387804621371822074061289363372523364167184697785570324832815972970658633551879143187709707164796663015180877412717910872234647704536817108676736661804878068078543241390828229923424191204586313620612539678930999769543756218765870513049986792/677408099044823641581658869221044375312077929976719183424865834811543737800956896926637625166844372424044003929341361734886232742770909683021563822987505236295727478159938135467975522336774471915167606673489722102077041330652185811196423400701795791669780695158730756241178262962515917389382302757366325768069625',
+    '31270155809329751863885224732454397292230969002004953832354065319735530624996254695453061851449600345977646455072512400760539747054003851289540339425848681804190284451253462663731135337775088379954403740058084949675460445909826322297817535400604180338201322667139062500269285493417563095365899631360901732684124930296643108551710704785906431324876072470231424/7269578038000504017073007978844992319987411732848567116655821196644382777088703228960020894756722675887473977480537577509061256138261063926845643360849217556370868752909531088361229374467207196928745673402380473721018157327193509586295879051411183657185176812738231456253321187419224704301236205478184115996135940848503487199394612616742961108684539794921875',
+    '432538822079707760382094121020421735679118830363764570640789368235407853152380328891350816400541189148550353337874309885334920995713154225799660601389784410911658967499100610376065640785585342035058364676314084595283850213942576431310823836792440218271879354669291052589804956435743500204185107215929849054782893113/100246574739326291035824954677502591279343311051719151327066341370995390423713403739043396503785261917771859220535505691760472395306543276314938287868734009582906895763073519374099272340577921671298878837400921045252035507925021904954445172372479744465666760762909731237634082051855588025732494461939980856983552000',
+    '1914333673689206389116942789116917579088664511118582610293383428712902211612554212779880638065888518488492298586641997844141510832940409501694726851666478650414191249534733087933879981733561565249818572204932715347752949087800778646065986244657260832234479202201129845117955957242616947361016603702640821256099895469088229339240402478576285854783063612307200/442333801076281757298117784528962837115323835962460661601905131618341609371649347131724700192551574625400701487125984359494804828935577124602622582550626336986871791407398609915208114339012374456785705161994343348351473385887949809051796407340988735853810174589261300681030826392672282630613354461927208579993042968520650313621522933214063366558703422757071',
+    '5704691626402072213006354545292364761246893919997531024861408248746241619817955824682639582830486790618668221530365426203424888737658778881721063941495350237419723855000515747725926735319471480027293210991869255971365522170749568996651406002311020883635577590045650037569906001924971041810547543163363976464780729932/1314240362076792592671773873754757443276256223533339004339330559325754574023619698171225777585408160438834449576526997055649849875516310105297894855264038450585295422343454458568152668980131977005808840141079502436391909349182185596381509091427752151860204235071122788499996991078935216686010888734471173124487937875',
+    '451219364084386208718456142329444023337343409261545444643031014769484085278440612677813682892926852469540118625738238137458321112005189595703619178533263706811689687213128887082197330137502064260105387500552856851972124172206996205919556553246133218441769325133725631665593372188755090094541462474970525820334385058333756591222492801647110594428922046641945259392/103648250172203340865458115839764297558925693061798169434516001775068769911768281084188883278842546791612199025413573394167639925287068809631958006622842716869211374513136766899877504136929177267362862319748507372147243911522667591375015611312165464514308451222180272935398828092646393830572838772085366567154646425598388620105539619174837489536378605144891769625',
+    '1591076564577634575701791393842535460875733974464805197283632670013516183281542903377750304419996681222758401497321278555686661981435637461350320471258386388843198706277657208526372100698700615835733712519332548607115875288787602084336341594576426630670911478276101702119972195558314357975365863803265163991961173/364433108410193393847203348728981296285742202617988970384277162225847256295865554341611171460436362780497179090329831885853324392923449579538286804729856000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
+    '5307507148709435807261229345132535134848030343901300324623409721698217134932866488937772986386501538026693956121121771089740095760486362654754229904633476234090792827930433598453200536551779789049352131005222659995845427680686677324963223653949277037520711609343166926627713758990384011274777087968799793410506391884872456420041494598273703914689586637995139580794368/1212244290381524115082005575105703496583315188540177702780216570265159923654197746388568151706509243638302707511238539845531608313334248788980188054651601536067740709977678222790481348003085034888244447626347312094586389347535167903408519024105748523384932942441843118813819781347089702286481924493361058339731949844620763272778960860509570622673048786968660129650125',
+    '3046577098843580578619955603029073328361298436129791931304665777036152915858575282362742008136721284817899542475666054101957899858138590963789072530710775790433466847100422875906866496318804986202089528198912098377828580031852152997907433335176267300286466072997014023120087988748396598176622765923059385876855303940720/693911859077752895978833241338902695755528613729508555938416419864772385336777924892434701804078893822446727762077537955240605927550548046309823051841326355655848406646248595628715185413852040295631448250459076043816328082561248420925930431777502622148019371383254316636979070731427737764160145097293260938978413488819',
+    '155310926743873343426312607182060072939030765297630534544899230213054714482456829198485999782086944271490229412707980997432528653509945301574794242118433868747172279224118534460563400440670015323324602117517342201640016853927923976796453577048995243600967202462466681112804476672348206491490513419336308575442086511015101999847896197546900512764233674747415844045184/35278162986589659300679088538176070649273991865663041616088851923111175518405117391134863644540911005782372831496121355954470744169969774540892096320978686548284501139861783292226140413769665461494668479430833892857992401391262903582109993838728281915806394747833588629467613308837196269424421131934859079260185052081536487462257809987002198437182539441349474426375',
+    '46729591025621874782758519074451728476386657576036360734358719976400940301493939192083339293779149127132651616972817165172116269307276487158069293114860391502484125554406945249728802484128756924044633825692779783425628292959170597009935305429239702926898931975023598456207165067568919757902764815108751735753431153581849/10585868084079030838651390738371141142245086465033459640458366146849314274285871375459898014414833295804139979016362796357043372316321872357817727821559232353993714062535883074661734509440994664726425399880995133711038483607773782532430879196405793694658185175583691180757783597895616920432527125993118171361116684288000',
+    '3736372348124144720852190769710129461145889011598636925228657393934132828633132357232883470466940330848177425542748100211498184494252714616379450272611850068867816250209867530921278645286769418080018709947826876461419654782341309127709703626401211996255743831998918894661053669189312375159058718767082163156988766821194002596331826150321864927832618126580509732359424/844176851007504003627016945212023239308348428094023437269532743221937069345682900884618378849283002998220513307273333096775669556093005637615012750733216460458689282791632437851364674879152695438589692227159038555722539345972833161146108367329370564838671791241431125003401861887477969954740544092178721327724620450947646209914621007186497081420384347438812255859375',
+    '1258774755828991281578968023382624723772927642002016270484090409043454336040857926581316994594109169123354553321469500848146015719851609220423736153365139804086413284787598253618361769125996755159571523632747129480387254164008968993734442164892486441152227433281625391753702577143985047832519062595123255569172968685060844/283659859661671181526547833415653453506477950678651675193210969173130116121017723360258249430884213011988678011357458727603413521688184521573094783291496368005697481333739504490647194454695504722542014845706216224432373442438242355188461951883454664693262684873988061018976711201351382163527093784753398257094429403691625',
+    '84932063355292829988908961192574710493098897148701473172754949846455626381329456661808566365329266898990829247446356970454502007127269708487563279536825277374133681167235811080298134899629580318813382668399644553111080625918213250223197440426147821225593304993621451053135332451997633132772608233430131400186571793929377129211228689703376067763625568623535588709576320/19089978133324852910950469658566458037096027722326716800113107848115231563787455584278193954518442601810776347091253561956877155673550458955562102935555510392954425196165785410319126098393353878286400877305164869548380670204577544630353859009177051698096136470072137579698473017257397949994722015089768745013713383769765609613514021200888647472804720456494757423299627',
+    '802638881530832431828249604040579750916118423833791608589560402449036920165704012070349537114920882938466635598602387718300074733476150548724726460209016834416094317724261857969955414000155807312852092720310159572547644569797512233899495300028159721348599816083166712365215075728968005941610056018023633235372936903015771583/179951197386119079732438617407921535065140503043429174394605652913879982486051627760652197484142547447000508189455126493868229565647284332735552462525598465192073558793335913005459266977086104359621022691931002488052727597513413492393525660272900161375677499228252863529934576881596384036401784035248649026076581302370304000',
+    '25208742399375362881099811032135575360109715964024747212026245529087599633280142314962581193303683759605084995818253124445773115574470717199218828756449187055537877478033129862600982068782249943150019637186466260707552416433010545437321814115233841687700051830170191107127799355485920046505591193770164750886037885397478191534797655616745528343172318318678405576430544896/5637713398995569614196397857525646325234056219513202928587580534159596897880731043336790273040813044077153051260989730372846713618900145444802234629922717464041261370803598799826604841654608724727320798324006129524610666235998113655193642594744544226880944882342204407750193512869672849910003246504052298022468012594459974209940607450480609190841893267203392880360823875',
+    '732332637178584560220688900268566130246820235956768724845747830959547501950765063982943061181526237061809052444110437930100210105274824607344902764184151030827266142225894655531497849161692760385938786736436977268616029862577293984376170905024712952813234949508186306774257035535805330366742322777611063402455261391021097128/163377988152179636922409938163005948596822656868040662831003991395905185823836089887990201522673196719628247106777881741843786365701135555917263285268753605514812568258179391272113405920369587922702002732667842511108732068683407168090725712401954314293193572654347237716691784386690948494003094992876867763698101043701171875',
+    '22391374854299462107923583267570593886002658786775211597896252879708753450794332301142909715845151746786016535157797023153041007263258732991465037648536702217273897876864351559736449481285518249846264600935543582562018874574655740660800634883403597181876364712521253650744197321080655028374602772696770227741610874399454362583980465225235674816785988164157351243193265401728/4983275997188967758382167867656806681565521964135131710678447245984239733544941881418957694612909599261843693554043504855207170782744518943778096768859337980470219783210191765678908537645270392500777411134164918803949155037581108678968701961544979451081836872425574123683574475790997414488641534074873248802450108535183413970057319247033379016606809226547805269752123746375',
+    '21252344995592269775107236774689012760750850598567799560343535731251766780016530978663079386453430272698006977668063208502408607227544322945446079708000304179073184745623740466334127867464883762843346619853325415963062181018736410570113171387891518398846617197097407612852037905687899800932343156979591740310928257766081697645/4718515378484509142377558412184183991357747235892194234377932213613746008373918923611236346330189287336493794499188640380975364991548794341177060325509698404571318037298112293926833877103554546466055612498927474225619680186119313129604319096374924037870803554153458104225741255753314955115645304948564151765516183663634546688',
+    '76703085666560609319365659209445044957052359500745196718269665234646121134787807938680787341023203786904308047981099228357850016602203539979246579454229078497537148799349353250167621935384193502383187510928609818011142642954550330069991570983902841318203794113898871515702445720207144863877026526025447819537396146822201819435536435835285098485990260677737192230985069824/16989892821104122916312992616665764943723222199277412857053896319814438961475117951904867066845412639236790762432996309717924037467863024360211163971190006272168845197000304328480920483651558672879967350982199768255256753110375467976246339260326422891913564151453729285191351273342206198366624882195135056176502724912849575385576554208844814085228696838834665339811027625',
+    '40153831166521391225489894857551838468150576827129133168972859441122728710165233865309084872159583787083130713473702296122465319276461991457173085312407612757280915853584698420083436946976844240063731333638150599017886937783470942007376523947840104246927580254612991191040951001539641947153847776050138532219595746056076776809812/8873576113581065493273519627544307418139908640325227196912114520903505426722086265723900326735989771660987609604507503750212298754414472822477243293638486047313990878348820573874809636084574108909615044524663712495422565151340037564156235745628465172219286664653343193162657374410926171153516871765680561505768223660055617934625',
+    '200736374277835272485186523480177159453030082779872429648449412313247640312479214261681899286513818912991945046297334418102711629933437654377760028476767482162031641781499114859553677502002901248124750539270235098492377826240319766677856465093871367817683734079621359186477306173620331748496369122143019303740252461824523590096923561928354020960176605008889579578495073283712/44259440065125442964014453739391594153290923937737136823355312892557975399035370631521014554980129291582486968083228888083079118116286312583989433037097397252506140224404130371160608365777625538148303917306340620019882928744151294738308047800711028158277500317134537540772759486365658024194428796846364486187105343266398338060080611643110071184992193593643605709075927734375',
+    '488459049325494693259159444507437983381645757291858092983371672334043029615965882574409808932509285079401239403272414271652617474184321852388397021836909585659327974611273820676128650810907598106588433939541654215766888212287311943387232664417347883717203611092450971786083806323404432266197250919880225892099146722474124874256523/107455855466267410923480140898552598306699341366032095904938430084768624148829473848763761936703072253841751966906142283640379336131083461646777390874405323298406517250231389493084918065243079819898691146841695350589992171102939046740169198394671965069747042621265775948217054519855346617967866614303497258042439052681849864192000',
+    '8430374068596413768975326329313648683222744787097373111477788794692418932056901235174019333392845529821722488982447683794746518712070635063397464904489452840169472596494433874412726723065560358405027764015273789053064830528919091414531400589850704395887250756457780868677553068164933299802944396576360356896758745030955907258232257358056931680617626606715393885378887657377280/1850480358582748412767893656294669486390769246349349706200869446679212812264400876685764762211659843658541567071977813110496329850010288179702589501255367345935389050373021591572539260951149696801665953123597521983257852207280970577287472932167362521740230973547070264273850381896786822127290397075758780174083415923482640313277341120938690594859118603017930500014640169211239',
+    '58273615882491925540881784328370957720539797010816130007322211512070634295154626813477498607761260613353633996451797686143131893884559909668805628269959901044961666535533876340116728982072626875010391195372759135469446048394835148654407846242721443845351855234347422256700864370273092971804207514736924787577569355949601385705517152/12763172138328432984573837529764902730448666594097466544652830703737195804747682450548488162227364538052585484418491914323658215406327363805353540266048943197822838925184942600367647278764952826664684839953902567694832690683467635727409976388280092262839126779730168889686628630302519063225007497707028600865624133968363921915016625',
+    '2685278694947152969468407055109959900160804835228466214479525891103128459065131221736713585038130737099067040196326815283171857977706979990467444622550497565521863441617001097354649972660183038850033950647594754644997436624623325003537308428828750008305464174443210079059253920383411910911594636045517662796811001285984447671371943267419151220518726940455756833040934506414133888/586864816044968996825907488721678304211296267371936463272955488077666796280028127333229453342355554450708284934221461946991556216095213627059047984199091625921432110913096253352065238765240063555347292393427726758010888546427415276046238297382474708612526770055488323003698641493939145624184943388159614111350839555141202024991876221874110644879676602924730506882384603409121625',
+    '27700345710264347957758638741952394530538598225904772664391173119594616680996031886665218221392453628824570256438960349220263575741495285780845399187013582169907518462437966962923592601721119258663490655013419675469864809004562272799012227293974254329929404036071055528766397079743200179924243479370594973626764330855493789112414191/6040997839051213541001279276287478343874107660287651711609067205111574718442229224212245187655300348484144001298428958025052162253854676210451657425437588244676589965820312500000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
+    '80160269787574270953020489212619791839643207793889009503234879683494928396231769167082355143564723274576166638869430572977442020236299319718643894871736539109822173538266278165276591962215927816541165548699380129014840956895931285215480350321259054359835472320394089646764361802445068798451796525897304221529850133401768027469126408090577121172238802281079916258951770135485696/17444957084936455555074876721808241311651668802091828589911310253709517226715880722948267977560381436307152342821304525073651379763144281678911047276389231713860392794975453518107055568765009486973017341479560154792340888780337029481646046233967923289468141293699236815196553819373828280398488966665994092877680860643266914861837639322607761804740256920429518434353839878280875',
+    '2352816897072623416220002134476921108405735955266703519252095150412419264317091785317803024028565851487861978477208755211589867803009700996828082850796103789009194767813611798769297617674631277847910238088040257037678126316866517601388289837575778165301828137259948173292658462765645645868094197736382683775415645307647043205988394700/510973411316690313485681833991645423802776865720580280058344111363363091616558230182669376841383375321912553771027107891991318313938505988903735114191313214327683449514676737796942389784957658395806951715454372568523597420866050825822028817422805046287798054682484158011143949412956697442991898914560191411624040490122951328364833797',
+    '6403295584873165688372907494046202150046769667837790834896334486679541887567517050446119511695248926941383207478170182650927368177009669717288184903306689332746127845953193587519575304974203099873732502605739219028995266139383163062837007982999189114810534856227848568800302527760100163350814120545587074865568436789021082619398126713943637898657861949091545516403987546145915409024/1387794272010111535893205703999712437783041553900341112488641528986385462810448493444968969845352401058333929711237978223214186693177251566069419805757440174840170213159651962686467523533938145629444468366235554597245713128812532716616087753947246800626006504878203666972651384731498770435755225220796872155249202960801768048854869001310722927230237083418017482134588865244642658875',
+    '82993247683514419570466529457059660634483860665557779709153549045427987672829778520201315148149878525274005978368939092115193636113741972236218502664881450367443614971109677363668874484696543982239492409231870942414193419634675024621942196087473557914167832058111113476295926250739099284241826553737074679953551191767148712684157318697/17950946423927357725787689855263532224005643859095168852729513034456208872420513601894508438640531171097082516559962755244698695622824386001219435651555513795509616906355363573638916895074349491513539093024980575852693293474288638209680085037286354050958859425647536735341886663074581909148323105020337857959651624760873736590065664000',
+    '584891611376763781852144397260140844977346305541197362434227194779766612939978629636198589818106137319267243431810481928639442343946346034433828599323416877248326356345631611148749005937144684862502198147087702668524450709118588741606955966569427636630159793409544653944608958808602551061186799401212712216156799273254257486955348236914237644151956226336750212957092204830385441792/126256710861549838395499078249922986417488985866522660521294610280820007499099176190820291214370699289977888813109514854178180265382471972583921477022411657285850911270389720508719377071949050253246021921916685716353393018411683757067093259101151481189635423399611625009617884983318141463140199839724797412514785751549277231259345923662499623640886881048572831787168979644775390625',
+    '4743054867460856425399742072925732465660626340183690464743217147109403130730445842673866624947360862438925194786600531200056131309608642363389833474026007798643235346104937733349791667694862514383520689594596660275306247615314272223660862212527346572811422915223417783887717426641317921972456913534338708745549252254865615725859881457906376/1021840037832289788284691535543138164288462770384961802287250236519983887262771944174557087207566030730743075334169628971336358708580124427857321953981475719503574867471090607795242192417162791762511940848493176847925838738242153177311649130718886794249673254183907621449520415060660496225939963471753122766421338241291756892571824984344625',
+    '3784664074155769467702999785016514468281913375341134899878893061325465790589101335015569840325786070795267055386681356241209412947116340524588831510768864231937929260236754881829005065056310226407358204278658699999612596866156294195316867934035877283950841910726224355461522065773816109849107487214275801829843762482082803559183694631856772777313673086715631547326400170962722842240/813783281473223559981291694175087508812520505931454895884442580280342455516154674683217348039336712901850738745276445107477919518905155156380620466135946654952837573797479076027688866326359448543065305071605591497778941561986579230698384305536224430794233462949056326864499827444363206162251104552740175503996670997705423888382328014210171324376593090187604268086334796245552762333',
+    '1946671258536842642381655747294621776070051525209940130115769153666368932042152311477746728678182920842238801763565726086459485050237860693394471644023999467670017470376746609892484192072297938254898880354014176373253875722410186683852253828299669530022166361304397631667133689128358365296701757782382475692465977794960109690362462330857/417774039698408581013003883929127512062321623871486379101498968145670269174833505080260389860863417408848209525427705249526516766731427603641806256289098209429110794311660844125377702016056984573671024035213045475132134896835814746425864304907974005950155657789157496389234919107772201305672421982198984613601511388413381215220924416000',
+    '328578487723377153600821813410631465225159589727320893988991729306688663770697528156625398176929288628930496338036815910669579019719633124832162035588583242320537435554101841406566009219059042888412893352417446437227440076869427577701706713564212185163005644118869496869980925214424591264870388237079776503547314587137721633918824664946974571838634860386893404364499977966899068777016064/70382695927096628347637455030970644630942872888311870124896575877585692281624151103204551947776906510366521972834357031777287506848887607457751667816514216907338309372900327919413372044625602555866294337672022362865161523684052940126509854895320132828009551012639156341878073682501185688461720664801829933689280752623773789373199210473995823367199793407048741191786377993390965297664875',
+    '2617201476921368517857942326432090876874414269689140439151907982631768946799200744678055980827789859579196832718849393046147656447672531861353456343821196812881882023188898815179947651274131166835133965629115749368441605680383605331300030886676081418867305170012032824011912531673468215561506423833911621270680104083533917562622600478548/559567121085534865189976875600841717161617153776904752351231210970240323460800785728739412474960421770806162360125774000625035522428160837855944215417821324758873127567654244465281354654299068546570789547691342609793050301258532586131013585959164719533123826201937801986942606844854123769532447570107525081084531848318874835968017578125',
+    '636225736038986537559880265988431731529837451289737542395494683393492040808565905376235074534207026537145408562785279823547657299565440309510931336394031904920056464959974743525662459433889398003683078967642651812081450227654478095420306880762753401111260630654049389197602389949892636943971690212484981672010398108426002137114819838399222096538921225458913266993881960732663394566024064/135776510176793971074115131648637508758953050390591773574951317807919051619690313331192027871176160424663811116849856489187562728496099757910540362703888937768555824513740118941387831822900198029266206334350448626733139136083404404120210893986654422850183837974770675600952078956326317698998103770833069712616832650406225828969036781514645731022616236082175582937900731419575337473384125',
+    '266695771933124633677367149389643417608461366874310588884377151539325854547826373711099517873721616543570605935954334944030816383858485296542260152894035979141266909050267414072982042090341712035518685997484257326212454742816979806460287972757626105526907510197321350895873473656215941034605746494172316089636216915825022339855304925515685/56811706665210352283362623728191218698295056176625217939528332247537278605113496147630185544004654583441448319260578659922931798845493756189402805173037491645434052737405379674607517658118427614090338938517963215812444779184193933749520313676564187507594274551791388039139409235056119788261689087831209441779870873305232021728002651979776',
+    '3511806683161697708497547617957719390189982761002154386881580160856792742952365159764830433511949678304281539875366378131195670004345568047690216126001067194904446295336734931691743477531830892015690816210752795806120303198745685394015161323982229908105397857791180342330098240702332072396030780386362735967021055250450666535422528637737695053315137238368787607412459874094164027214953984/746741569878639983491390741637813989978804202898438708743258000150996080386381281307609038830698579358879333079215327202911977568726258527646560497079622703052765164031089590965199628534477381843079892123440214378949632707668935001371616475282883095939750704292616758568964786737752325652839226013335092148777788733453702438432279149298482004122593243860829060557386699231448957232420125',
+    '499543951252504651717279461487337168721376180441322735807713499521878076780205427598086756686061009718016175215146305489885835839673419698751530207404115002383180054704927695269921072232395828258826213554806570423602966743337801741633869267620843626804016742412046844770601322513184123515405692795346790813502805238635003099976693786012816/106032986203682550514602969462803214831559442358811656484036951559991322084935692953211120289352600484987931812632505499612220494311324679562152078883141464406938524087376964935494245010636163705698220308243980789514815579319533341929989455211613890905485981353837321342730307819060825125980500975023479813757024395422041501208805339176375',
+    '8540849722242122835873311629952985285477986765819584558233324868363310302206006773828897816547299245727087876984857726652932481899766510637403577175623031467933486823994040576431755172220921921877192006685053572215922347418846423419683723609799729359551828522978186208983138345476801247634773975311405913274552616037005854836040162212761099347518242200807692224460514057530656658616850816/1809694575992816440924165741094996511361288430727981159314194000585536832192004274726651828760263522962868944612215633976203596150606503849595633388319338251977160257979846235995654357082321611719654136058257458753105122024859078420173515189641515736029725847315662215778251370398007468665986604529263071981040014321717598469319370651877296085250322599247141397427185438573360443115234375',
+    '51377057693118720457387330519321684810411289582445982078756989158447711080414032055843805733348558621949380554029375294451303430269197115810494758213980833379214402452992657502777067850631862453239349835215260705131864911194740832694498014048717871429797341104227759557199716944554347879767916801106279081864127192611546757627038037780244683/10867246748205139797826516105458406878398263495890048637741584969144406206902037615496936724683237700974333817527350121106320991054058406504571347680049945745432432384570515723033245430148177097144850158758462968940525568041926860856763884474605057056550146491001458649244094211354153171337463406192651150028767064600270112838159624568832000',
+    '2517199821548153657910904242290029026229621935918771922146425373057248090467388430999870036608278542449661971830076845113363443421757278612394167195431759807908413353743377586893872930828548256216510734912027450642648789925142482023713336937258932978503310551945630359646777080073329099268529064437120002567758664419443374110941824106727218341795100740316967386497091058915066147604481280/531524260324016969370728057738851340792702640911631807178654723224511108065633433027773388768889035083734041289308124943130099580765995149865288328550751289346866467755881013217287175392101334538392596956248952430438673292186779006015549928829953979591250274457658042926539541422697852390294886719770034058916886894408015624288115732154492554105512936468520265305162358665491880968652687',
+    '937159485027553069020805864547349554547661693803137696125140510201297417077233061462068930061915381468160677500445535971818609631674361074587732183297775728389124966713749450996643183965795829780345678634356741756033039293590264355732129789110416943789326482663579413229893878422643110852069225671783017610387570935863228248464396615988222968188/197552642195166614912991396771845374656891854357948006221100585931656016640982345159207601253599235265353060942313995345483862152489766993442419668492877390774016299878943454249209917002169480134131612530977452579520564400124908392339263545911443803619972765231094502745980083350567509014991735227390978850048088378721210219321877150756855372125',
+    '5119346675109082499980756672052066247676641510661024682574296075659671743397880591881419516307292610449161239233139088348510165442597278965718490070997173182184417648987611422446347235659085092530927312760229499083348217392125906851921500360553758600756698605047197529446594606336369425040236883171542367713951500007600355406492128741649090501989105696576697387613035366083536629358444229248/1077349190930018641197987339365056771667455373500846354778079878558146320193384166982231171317419356136237346389570422332278447217033773023556383420372685049620361481309067017697564760747047839930422553324681218253359586320952896352245366885171583221864112040038531116947187482413031483309534553461613717398523132176948321964066833901215878133594739833910960843185464241840039390204295097875',
+    '58469980853327028628854378052548435225264353747414624620847476036979290604604988637469977624105451959796142112200285366199247075503424204044884512269071061176877225315884188402954249683107639777479663275428465242905314418930053869461904934138752031124685541853599407727446896558064252696813370692668496452783003306386669962110734257851955001481/12284458784412533668960387046583440199646081362484321078872891438551639470826582933879934711861551014258493898870867541349962531447724393381011633785884743889445407749238183080941846528000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
+    '632694338036291086868292541736272151517212971647680989553388199374739841494700315711180961760384484042159533979873604395621016272772109619401112452105475906812313548587880899058905419220497456502543971131226430775692744506047039958551641270070996110661036371362526401525207493270590999387476275629412687473362310414634469788979236895300129227082421021999210841384596617452097273320509348159488/132710874087075221626157136110978536488168313127228703025408943153782802265472978945703742152954881438058373686972477195846162708986982742647038014147903817539418442869011995988024638278746664230046452545499722614320284478925059527086663455478374200352375121910000967044366831275681254530914306064535110528723482282501358749001657608122307686482720926189537227492041135055090784059994118192625',
+    '2022422468242151190347511883185841333968390991430363660248687284021524206079162212007263606819387821055183192871951076644707111802972354128011904720586729919826758651167984507804867954970064938022106484656560866447290934136610214175516487579122328055482392137619829739560923807880779865442016530712986750711199998190148885726144596663143651240/423528009127070458603837580844559347493775236970253513203186912091140251858605354218715508091394564000304400858761758838182654201653901408046974668599216804223721114331085507752600832390227068985693006017591304916519540245902645712532993889000048992026931475754508248265356181099817467608863068068953756055334213584649136541490530849577363647',
+    '150542201776735231618709653388506394887777837646651406023174643241783006776019388336318216053045300417086929061565868663031237502137150880313922488106751241006715449458736723129545400133450376339156062387859536289694727635468338235084533552501618659272047966040838509286942825414463511175531001410923014751124732748099355476332649458471135027773371541572374016038525336728276024676601778299264/31475618522527975728853391146702172347290819252268551219771201846701658848754300838383637247014705706066050673096278897933831272607774110183488615338021656923433335661890414740664480671162459271850877855521869741833030384682549648626038314686632195158507693321505500888988183270276134667071014075017317692121986144742473139078857816578133693776368882596780120407949076129685719314703426278875',
+    '162664351499283182546788129866259011542529017155637405328005084357078048074065548349602626724265307733005745244338982661090506394708170105916177679714664432754153795772811389612700051002336719842784098507498286260450939289109123759859710248621544296008356071415713931721102824434964336667980062728653562369697289761913925215639430496443820411811651/33956433298509232632327667206936379248172049619640091186698677954874297837764235300806836350019565970550533206216590639008637925266405524715041372407701947953283803854153876844320283735281209725651036513491246238622967031979233265136280687995181375582844780830517825595245506321879177965868747546127685256867456319112247572348416655367667187712000',
+    '27228583713734183629451808889147559240254570554044667607519238865891084165382075658707675405673276615039702916443152954157535195508033284355580742761569101934821088036084480297442018005972653124248838420021440899368869744141698556359295489470499243402799483082406238128875386939389123336303631993557844565927960009744780823166462314116039469622216263221278402673028225214534859076857413557466368/5675134787340359276387085910732530678163604187009567576895331134521132071092293139167824957253050637085885426100206370974479279292804959049999647539684766632278449357893560561608633329561385765035738071726214510832066885156767060324427330401967034889966727638276371320401053302413136551142103678412692111973028587922399670456670380189570979581674699102934733563330382821732200682163238525390625',
+    '1340937726818688387636018677960518622106989311577445614347061615417831160296369500704524022869612973331444242000788110133157689786976834913297544087249593601420972111789552905846168933014570399562460789075871112224533723259660967481288525220689577290386781169316008987012566216235876842757183233207943071523447855623001920854571597886491163165150556/279054368814522483241446594911968519760869180429079780837994812436395792594449929617293503975169489945136244602350941695478861584261665582123038340824700224583170270427163469762789603657025958695551448477050958557681299495041994050951692916032501884523150972841782658389836059236040091605838583126400660344344601879969354053137568588338096589095875',
+    '8910951660339249479517731530773509037034977353457185453617569046885132434552771722727442649095007828468878374139001808825741829728375370140505088133244952288239619141544661695065318568074863317054989982620944602245967155400712702259180443238473885690297281525960618905625084639870407113029712510345262279507055197024677292523214467306898573956604928730377318210588837126450942086527893437954864000/1851573255256476362977604759655284263358595254392797078730401113550522214182030016660458780180636930591149034499142464357511158868625309442843792189394292787657774627425816694628839987456477799348874853555586931850788645677998004186437300173080088839930384424797706381282430208961565555953254145002318893188197561315855286173276564036402546798318351229978103276843822345004757661005876638005274037',
+    '84997663352987536417038496128111159210386455419431137931308301989414462311135708114322046234708053448098538629166672456703651524480738219822858469100454933583774404032968304328859365484249897913676382909050450855222567147661842184538302991526635974158862033287154321908483007506326807469239745851379559259262425675035790574846229470041596801343087041/17634687575122715507915388096857069366146981156897230620371714518707518105157483283253374797127075689236298114335734874553183291806268149747826050838089931351158483980735336772879045027213982661634865556129548356487722537193914986994283729746578044621115589195066212043187229606921062007047940293972381121200723292297716621231320254309060321476608000',
+    '1648849257486312935416274009474520589799750757877252390543329851029437452988499485179026845292068377957697884739052491455807688429630524210823560524122596610276957537756142278585091315141096146341773982302888007751632401090642472506538817998043326415693840862261701364114923327286462650797963969289743571514890723145251583881752876340628073395708716735192690808352188867334842743895738955762067968/341580411902962140529547622267381834425264893226447658723654120920549166416114849731588552578626041597847002946105459812230979900480135423787717825697955152193526724029890829615845314765605770060975538542976523177976182401820926854406945392167140347444433885786340415437006775075196031433365827874254477351374236932624514469241634422312990555534789607377023339236417434574755566856489013828206625',
+    '4899626659231633406569362199187192352933567344960498139854460545644210119722856870634652562532083546344965166340063297647520613455037240144734014575196764021102614748218834054157165781057405100578668000290493431869797883155764963238760612333048336953434663867580862692267574574465371615023999531030446250351305046572267686566999209229202587258144318592/1013522377571209303732207048597964555261512342860326827024786500686692475697570284120359840724596444209277948648863014479460188496587721578075151793081823795161404110811480269106846359964436993190293615381593756076515012514100289406862487971220020752628653845345780766244842386136740768474703009091552043945843632199999007070800871588289737701416015625',
+    '362903847358088423032098939589019598036593525722040887883164115981767777873860799955715587436566233010543937498922435963385091400670303956612881974450548904906949032005878241638546734597308786514086678561900713741746905742866635779432216761799371793260025549698865319760786921444505323758235164807966277024031605673481480590193724980659448692487108892154809490199414370023696404259691299614147712/74959603173756091922435708504751461448318819548943586676159545028063979695880930926805574951275602951047845904770828900645717477098988124553474616764040938121171776556999069351772286262529729056288451037717198273521810585868307402693261479298923209587143468206907278220909447483738730657438961435587752873879244136801168836767273934997417156543866722171766794171623224855358640156858606843576125',
+    '8521348154958613550574095745596657249924328336262127244827955886474742319277424381360677655239017583606851483318951440691757875270001283229585075137091617073509041644097526047127618954293881264113694362598157456878784814696577083997902588926421675843291157049153701100218784481406086810264243143337016042301806376864424632290226377881206765051896745/1757584673254145109614914862134217345456065873908262753169404848658913774363650863904627183104777337461445063704655164325930891269843669860409763569362863600932267319335889540453302785099532688264425988677082044905948179153286276609910158854349711054144968131934713419772852663860893241364433872853338630216232313607969790083448433666334742108176384',
+    '1802461420562646993856730082999823508145602238125054717836501201545920604020389361370931345491160549787411668288359013059160331370751496329806488246135100776263777863399096485894306306621852596694700845918608199329091852956315870664531614358379176680326508877329862713333336188556181470928613423972314983964578645688876556351433429494008513812136152576866650152623510296911708111085518974142728903424/371238439252064016214448115231139360835481920731625321084601566992497057371416121407293286027832357816257507287295719261141426130159269433199862002841433235955835172774430690352481806933860390015899973299268361513643524088242973968537595085600335021120523852420135081643310663869327371821064862449426404044951571585406092523478861879148545678639697679215289523234342337347529225543852787599016265125',
+    '88795280670112240977945082069219541902481768504536167808816453021962616596410396813316064685579412429940019071114917828928080181638058444302439626425155946562362550070187433083541414569447612195370911498321149367969974762244140788494955146280201020345849385865084095189982291190135293114489407801749533448443222584092575861096946605418438181411489276/18262596223069549313969288693970246663726147886364354584825624311486557881239003216971630599389642752648815377795018130449384513824927138566203714891090017787258920529990523519479383032564209629983810866231367438270859421572641494420831435563580050954867101292568399107965982704216434467243223964964535464624988344342084084237484262612100980519602625',
+    '752911409358158070688133336918078236438086521781731735123294741731669530734909563155732092406099003582749182967246236657021875488130304108082404134004381196925042617909108654109138702993903561036987511410396567636331465803949049946947888490287210013788802287247422993384874670368649616782837096233384279817294778916419067404863458318451722971117669642134894906235066332205740872938406767767019265664/154635332883086377656813458754577288834216641577439356891585125701020579768315520324293435008561626926046988056302745604248220408154454267643222328696606367409715369481567213640103076112146615938180089303294709009813772509602954514391675152597458428326644988225555987883250642389550531163153338541870029415077982539614784206119074702022117790679351778993541426110436276530890609137713909149169921875',
+    '15286089077439918584953144558775765002061832952090847117286473868694909713791678395221438112006475047633028543501632631679316850512797978594476229228325228403849089079675681042224057234415951253954044235099139983386056610384489202589484570852048157272551098909455253037561994944665563291526597323199530997923272894119350512727802414680772513760081360291/3135161418037836259442831302977219750614726139657718770206960776514822107155928800694375522572523712277389679035632531727155957120566617736817225541673148354829212969778664144907503411589126111888917929495106029890532891919001138770210977012708107496187227751496980000137223047079127096028939624830654227758737339725345681745847598157538483308068864000',
+    '17723482381737693269787076798246423310802126092500438681864673375704464394105734049099094818738115345706100191580982712146882034618103161268453808819471603805345990162762722549964406042295355916458624836894291267553138041035528315839108252422478610879301656207772491548742981990469454529537049123948604931661837539943536878260665617524824604291547373872716194154198824107266275716083848262625068958720/3630091565725887087605600771358216927099748013831912080380983545872077197366172619225751217619097657405660172801849926220948413919823038425519006413005998509677472226729295278652284861834178791332630676772284971762683579047508016687969040658592436230062049051143001484384767836353233324570686249824980508520186643477313941054900197268584495128610811273834192099099592687485176400539748616360799518357',
+    '39160514032490258389003214587901781721548011632821053230881239001436341012396166857515197251499460842954863342641015306304854226194551189224721257664420778416733889030680295206179109679627522113592373938602737416822981698362363553492610842201005420727750505681358595340127099855823746410663750133019743170700888375554918213182862789028531284364877435896/8009919337434786244380818390213546866794958389603666737562053423775386173149395943310276429895991903753441584557917565343673605019989336264748588735390182130334278887642569515202100870042209261561001627386525182096090194076839370589780431326097965428311786635366819094633651297179987340498215947144870164066409654464551017761658775207392803825996394875',
+    '18188388167811476762477659006849121912679763597132233588170406666718758511478154418948855160838212151370453213943784067796172102916618102024199200086478021533171984254799008541948973652219533819511807681148179806051835318645591102296256347927018942348574774821478499092054497390790734798143983945096982240517058861467440217558586822663985584775920517755909444876443211404222400335500608733284461685888/3715278241795087610941547133490827711133909145414878217965273061493740719983191775105550559905283733134189195190321643965858771461560130864714477448011225816210219450040558581571206711791272284197614810026220745160693898421068168630870850392739817340370146186703867097982858560347971281142766235850971837330222970900661653800041795020382847588362774677568059355238174402415384788280852059532958121625',
+    '46353430636874284402376008361176880938798775506236714098123916668545331718677407145199311191108199195405139575147933283523305343027297808443653460575141799729554209181454676876263582758919969948546918471128299522715691694855904535448703118221685887195383025133924852224568922999317583667985459047407473419196367812949579858081181660610155317911318937349/9455890179897829052705408931064120820559254220394152447502395298827357144081378513080398823188243663943177539817982481713456798757563952162598464929784370305696420245648035222083207596797327498500000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
+    '13607414211126412689435920170832779523235279308039712294681570127241403406455725064605210473923965685858012114332021572088551127762628892611459785625407107300355287623761046438724706918300976102581928643740596336865698485838431972050267114940079353996100620311097561405192365496385584164725035842307146351211183443267583872067352881387294766804298681841402861626441205037637282007810811602289501477350144/2772205947240095684743358245950494400934136973845543049938212403703005150954425087866758842625041409060217562878157900197951382737615822655863548356735553650755342108183641353142157728364469706919993519071329069805875834057743793434164720370026073143626055293058601124362130539040404491770346043977106988100969901297669523170508321703496903672333781306743488573086901583022550356345156457496396749797875',
+    '23264350093315690507131500726914401771633259602904894387312742275462601574788485574913195967854009886824436237740789910851843045306673524324611130536719802717912615609109285261163220007241492964833933975789257681755002054890453747803418465119567389315754801791950553304034666399677630030683759466627797123971742061866928213780452325847848901682792886840980/4733430508316851445525799359992627142007028326865116523033957869609125212251136452396410236930025876982265915963089597825393317909648054808274488356887197933393799241941120691826916741647173843634076299630884655997401370400734954098249519111198509015387869462198674657171808841005212058822254513429836457539629035675822719720295352508916898476137111981397',
+    '538960934807826161927564251064896730663918579817797131801326848839524373936888402762331254883505157846659830555799909822861853145306547699817986010353774500545305596630924815657016569444824389893996399550343022444937690798126189162954188680257730893424775303801562327757655350002001080811866451425665301674688928515041429654624970707803336800125648742168380676815769457467038849213225206824671025803392/109517448833211208909213522579666376007937648875042608355810687327424621469323788799169718267715808580775296460944943658428295481347915131091297305260674731651980547668762317138753250458639043477453950774629862540017034566407286963562574188172753764978963685535201028286359818524519980885827473904574840285411260037264057197155527809916623487144493019982419606308144576939532055057697567269576800542875',
+    '24451481554350251656598350124163983489544445991992661049829014268366250968046589127810466483349378560701491445658725080497078590986743512397927747885285095846398852574219367580068049729894667374214861939897557930527646930793431512683750733092448862551580494245486308114129295467976874843852138144539250235632985676953662288536669286915983261820949386845806781/4962240683851186225340479316959605207278948397833849899236383273145853045488131674290332609876751222112917095096150531825255611911377071887145019702433883343849119017643117746273523770961304609989779231048095665388209125939879595509516490230961558353074751312876252070889983946768175488341370453053495319823381943793679003096504200440997908071027962806272000',
+    '166591964685609920525429719961758335887941461052357297159523350448241484106515158310298216150030768812873781356326923214637847449893575074813991141092681107583183275003570804134461187213143832123628701876041315736048451374824101256980832991214945762266748755723551899001458245164317581617731332466039757786839492869211248064662857987111284518155462148044812626055842061283907063784492325693994911811072/33765943466859756347231737285768516240749714264564410814839966011490233598006227246396923594069488496368994857718734549787000213944990695920130591750377483911116845697231074543526982580529937081378303555262933800375576494067959129682770065443667091668104132884375840812635624862495548888467575848428463895756394050252457592540570490061561616914443009224917652440212112452400106121785938739776611328125',
+    '142464729519206642943985715465196065680101634264787375740459792619754842396453535646509170066091478069799080172299369994438836484938378991283020978342879057271045033903518255546705211647900945143546293360496661466657600494787228476009216508124736254967761901224547269496682078470378676865981855568197705507111970901389105723589666709580851611656838285777566736/28839643075889294346635430272525349651555039807615358795334663734940126053968249608601725991946342449660829847322517564370606746964714098508298705743689729306507554363954348780144675937510900983622836770223945538844314468550644543502501484521561567788992156632892028591678742014134130878101553633195551605796158103033794695984958987367799607407626319024034625',
+    '380457154857422076784416625436182734840012277442496600178664323821064977485271288502806124740204295811759917468598315932331219906040253463960961995849761235911145436552991677050565022181516596464968316292789048809559129516930574340710998611088996508613850407294743457197519091160170604426002924022923388338999324182261954685794522778833029950214443271526469607532589775916381786492089432407550224137344640/76922142291839859196033215395526448408989799722090330866256930898223012427332209054699494033511478050536463051459450844520020426942553107946282832072826069342120835351421637970462054707566481086202287686453371483126873188020538661871147778312986131920021929455276187425940666580158173676284545586944833678819622895550345083175685846944633982680908505900099388037050135210936852210263717185637166725605577',
+    '79999800708369666670693340813609265988600461251844239982904928629083975986385235816587497657678297663151298709076884516168798694292862559083730259839524968813932151813233366175091317046758260934357096757322369138747731224414176608247605314668045838769433811781490030635374856042786947109101585097035794969502111354126773165391577217071956928171336060247633607/16154850265413610119312596299635391433380319311682208755390341659152160578366244774029302281068833963848811596545791104919556346810193385001196395489740571188884873246131813864782581356887110499639544662322008824704830818136660544673293865921754569356583698951612797219776290835161780326945625621758204931298948625885081325268123344406261740843197232316416000',
+    '26517014004241498798848003208133111307300592620947908933072661812816683085261359696887064436611747292468717739609776825664344064024535028242737616786238200200498291226984541942836459188328346609549239724653950302145388877472274074665166631509490297630344351573245265440164082334092997307729007429218385933622623935704482371360985836659823269511115937443161532987660855490906546130131716829909388620539153152/5348257529530073524280989257577050878033239486885664777340947102417077875628238501791711336803418503616042922880218860266611595594400020206192599559925169589023339295256084579966533260861249529670141854632965231468533448709363477120437277477668173149958267956654411822360053335695956270719680086524322530514249891157439878407417971839744392044064720476417844056346174795601710447583861127956455406902139625',
+    '71197348290771978405602732839537012310626009318133570890482364371056186523989330157966045289127980687764218363064941391483781008265266963537623770685390148810455244349859354857421474277963009857011653865476081389968122418691934354122230780300833008753686234433642072919923583176333668289998661499146617277981691372057042930131717336250309680764289256080895092/14342680993892886303954853579883363225921109635079605392320944523109187641283117181461662846411912306480691901515655220490850586434947504504317080144378185989596578890875561336496479576651516902048606328604696610314441229321219627021610788398730173161473456954861916116391687748151443335513249810808309201830214153565966339609616397865465842187404632568359375',
+    '342401018754023891059352629509715357053454506037284788725127073168445189422181540408896452651232759821766616850264122591225331120588918865882061344322996688880669665450231982002597398732201789017495427566260651180470635467002453249479483900268320570004293679361696197905823690145326531180324429318916092471863347575533714871355651558610515931373080997536792242278529575700808878786732127630024064418468916608/68894647682461956089382461406499759613691148650738957016015844830333168307020284281532026442272618372290646283794122772122665531108484704223311663270836645399554940741928358317884620420886609705267248656765357348477857852684140561714900614303731786621656041881975328169543884661780531057279582784036689401206706069028229798913082362331334839997133771880514515972024899408556517159759907883793675672065360125',
+    '190286571838805495686895463752492174798162827967490343625540320679760700551286395956241059969689020885714190883323266375067607282890063444652926151862650426323339187596278760962390404656342309737428214493422028545912504503974763216230927196187743439226243880933947417760147785405444976177822143722569696182520826218120595298059020305548681823408268556482981325/38242608291912408815838081993045123864617058390583394828737640036167893859507494091864963383596867701436244007905770284939991912884465101575490548813894148831966132970730263562113961685364216989999844961688441914455435934703797359360780356482629495214859917249465567846934284121032247250332634283689409883499295469118485912489058982596725212975311140618764288',
+    '15345379701153714436938289567639102307887273898834051456693981514956329731572600026866162286097662958712922836383881840211072389867463987895894067288699955858337488812343674818565432780224654423872618380928462939482247900055572079914753652915184313685270722583926261499687728814560680092231526501970431280783177357735398580700300402036873780385926225005561766822479878280891130232036389707452005183370152749056/3080426623087819678602049171756961824196425548529619576972793653585089380165862889092832375934535464160425580449193072125955685099151500755278763306599307218302635233595329082702148677300945556130106197481236122437978375963261172524503625593066102601042060541617791008722569799735202779005407381267021641545644264334109052258081225159748043792288699674153722046659880085850980843460715080965607434114275256125',
+    '36443601662144279337033484452272105547257065463167427418525882851507989449375452076227652634440476502702584889057376829320548464677623038648843251719168810701744338735202853412651160681259533015293997501525430222870696235141468274274064390273214484922080219643095978640300874086585386991338440415203863901714217818222666715642467237183267303214471854472286478344/7307237990148312451155090506264642657872224684879860008436735430879314349554023096366846856470375025555139219961666754780864835035902856314981482268319886395881809659368275720375576152008289949085453326722156553603358879420820012964006978808113307340321824660484441618487292138631579616351777191144073820686595597986548374588839763719082323114663844043046087375',
+    '1575294930860663922843149113097927168024893739962464327016763741314068438447271326302129681931564068796631337571133444163024224071270063174829664848259482066723104251718808936762938249028582992557822551950212098797718627418782376027133576209718376784834456568694813577764300716139817031545408608110880973380162736536313173329973171844528083756524804574267826691156368696436273087171467254161930265910972517504/315498546654512047637461608750018349421492986465373840347519942494904200733840189483934245529561231604050629653992944140351106887747105945984325466452136415067807666316202346895657914393512001212656936450425268507235863153258769360296536050028695266363553141327896064944506216017933135251875087069031098099192325817395958019524956320840925176589616439477134480262100237057953933117460110224783420562744140625',
+    '7692057599553133417225997786980128299372439542315125030077404519679450372805989755158000864378924877036225189784045171045139667691524763609840822628706422918926410377185494048355128046783294954373197082851501927410350313585640237550107070936498164597231431242858771457898152684298047446530220873884022213114819556822050178579707210264243087257050468000092842761/1538814487241112760739561704846381585063002784795057066438330325566843108114787083741788078680861495340026117682606689156065008672344707615420501964123121620401073675430690541617930520947170306763541251152854304975131871537939618684186766168376514664494488691311397307418298556185724993104760692216619178358268934740746932420562664234738576394742559349407744000',
+    '15190836470550557926140012263310230378455340797095100474720776113765300081463960235506950799869073894127884189857505951314428810423734978025409205413789025720642609035117121385453856877706229778165533034683130064865256448444765902482146676668197546098809055341050873494446231793312407516463576380102073171090898973046611572020564659042146465345334273595086941562425006763960197801018796906114925005810069684480/3035556598829526968124942916297025416522606357010637652635799096498208643970273828496655227572100159414352020909846298817521082538898702393916708804775798552206401718323929157863173504795127180177622667376867235806208021329981136637175570409791442328468473111768140746895005561471574356889047981458002257872092408771959478275534098789982899056118498417649167444480913552548249005268380838457505944462494874581',
+    '2923754549090941424546974281011770689534961442264350984393235983189168024079617848190236639328676436485460311093391982825027512234467853098757486086492333410753919927528867817406389678461354257841007722472106464785557179844602277350411593959136520732862935740433163383885526249897183805981219416420390137516812715245569037561966929572448247077578419534651990827044/583599691615378545100950777601104007812792435102945248411439102112424251478512952087517463400054567832353967731887062349496616445419782499817548928078665212077449420216892583997505908858360011413290038645573504472422610780073975450391707084016931661068694024023083417126286468909117254531307085545768412513447197205482285022417611018126886491416133665625891075125',
+    '441922815213568908489589193556560586318864326425397702047965372289295962228254658331201274291697626694859414786292266008476137667843874780506886929181835469302142365772763129838217316953835831059616268288399119642314758261677639945004547833810080700127847214666763863338091523753898362699222880772298183760217731228628442782431573977490761665091005756534420789258276075174436091798475838370152192077494553263232/88113527373573049332749294663402406454524640221866407165839162576979477138694653404281889227416977618913249275712917582743426231744980461918772793673202077716234026090308791893409200158969108910565467273725064769890443408324386897017825489615600787634985397885230455168591240328585041836771921929567914891379235573982442311269392541694564675053636078990836491104116280492254795616055818375774141667319774630875',
+    '91197133767962483852278456285591810579974783077874307589555559715577599604626819245000672929518645004771085106770784762994476973842713035791487204122653245797126906189270204029735872508462106527240320932458036231707311992868347221176460407003487373754014365881209649632547685657666691589606571865740595829983849941049450471212387808617247938941417000965198123059467/18163679611214677813643455199979098261849405189805364986036911945012016248297645596270602508015864371788974684317239728239847888960155365342460843559658233069613218442910226128113196984801126248862273503232000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
+    '733396724434554481470656964167233690851279923085197144879644750437402627936723531635898542414136943003236478432848965516925891956615111678444586384493883113385335463639674699443808084827715994137410743992130712903997815471369201406683862985677774155704267039224546352414417632475628136460919900347977754644214725932476856018335195927604771408949980634142255497812566866795799230501026066145434755838336990365184/145912495644159810313026420025053989609942265686388630266226856650796088768164358961644113076809511566125145395699271530802891523741636707532676065833327082092521989958971154798465784847092870457944857590623097781956884297103961713362268264159015928554707158239398721416218953003215109202771651993622244788505649098445109438104466256408060944298437033058380804126574878377874459371251056800895394904459469832875',
+    '167606423853658713615749101123336326908381619586838606922565378505211386185130407423587264776519589295420758009764678759213587906216543160801059029133457062067529468464636376639307690657945597847080706144898920305379653754920571797771178341550808041652360635584401789901525709166172283390177673009483341826911179861584213314395893323035527116283758445038413646757280/33310364848744488727470761889861069993071965180756187311178124887286133554324042422503263994717993129539448310743598138364172889305532401505890384806078790034557049074169245078416704866867380089103064821732673967120808611636618526737138435792818828990403374766228037003927569643972203993263329403469747342204808360947840103843317533210523853631711601450617095736473',
+    '22466735478643771803624909352181817912387732117378538284107485621001246177152644141853604633676939694208131243081227668237716194316732125445041260568403752325416203858329469477774720998849334961751541191761521224644520742691981372782009394826739249604257098248751626957528543558542443122470657230835176462612530464044652140581367666271771882198747182114685116336512604038833327707163107207664705120728068987996560512/4460342260599110071819142535903990817893424192972341872394014748315090430999224107274777695306032783105944721337053156074851240201179902835353097683269298594190456286888326067932777720561181760369783486872666974623910515795208353605697065447942804414120030678603538761538385793829787734830129764152263691296719726909633580318897787729004972323599773548689272008712427926734258913271101899780538635213826239542884125',
+    '1972862002066252798837977155272535342727415329632252213508267781390870646909686724955861608046325765650407340102472833428905279473185818898693854344138403744757857928883601637909604082588338869612221777263582365745419719805587946558326048087138197694016224613118304623864709183681172362693106460535117459192992832969130895829916210933710820035479169562970982784580861/391262854245936198649509275719624152305206807261382621390946060636032376057385723427601113233050260359684153963520782285097664628279596618521963444574543409076428660874564026651742234728997339844161514273669735686516761239076529088145208958938384732800586298943281728435207452253170935715239392534815683367114290676180839217919539381651841151674859715992309727232000',
+    '25408691426286653276004556426513401767718328640048945917745783225459709487487356095164164652218100925888847689099153320777389381851598236046402124684511968385325107497047025485302333503075085562533472482645023963717347867696861885363417595298751490212495932766872748125111115039311507374363000062513977436921772096443948159289707996749061007112458126301618769295162649600673247799961118662602756889448152517364992/5033870770817104909025512331238419776407331281944565644457345507262881100618908405924951559853352841154730802006750688581603270849880756026262217651685813584900300357685874044482329601894134443542753690048962260914642678445058891387041919581376195543193199307524572671344323361282633828144708683536721118910549933336800461248338059209853766305058943888965279863852107507075916981165164543199352920055389404296875',
+    '616565940481577765011174617619124841631603944267256178093555235591208387588894068005162766513720546626399308168923904625210748051788652328072241494243040185611373035059619136712559505023703153194815611268503291295059648269772663939137446253437248541496631852986499688052678961495116210133458596866684413931188348798244969797973846568160999564540652111599082181871436/122025682149627484528651507368051367620578469392233875493556828623437869650284356703295836998178256875766319508797903923638967073687991229708051285605187358616531926112736402511567721889845693991963610192161413692928378708223712791930113386840109729368772998347036090902364801014045328348830748307179590755723160142468563977011700764517880467116766150322736003147125',
+    '979882056834583241477188686550561625289337025747787132773853690143359657519481397575561812727394470125172272246774578404543257210124412718725081536309818383310896102958167337333430555168671327992395916735022711297435263157670501117734412587691271084981140167017421900852550822579410636184333142944411060788723653596377015359036101170054828623669169047118551010102258802465554033173968468183492058271365801755986560/193732066620504775981759812703028757608600125742477178751189607602983198559778039141192492589734277739017909889924488699372940787977794555519474807920198944109549023547259912324279741006975129518191081373554817584214941629021088206977532599055014150433581544599721903881284581257812850697423350805715586436114842183295730680490110793225514602133304729484547181740290534056445482186430729398701311563285494964645419',
+    '1138059441890428903336718712159907364975959482163004276901676063186939296217569849990951930955270849692775077813359573394435924934540269310629731801376134549169322946310756738187834128956671195358413024062230774111935109203207768898525882417078119314003162592087920755572050993355843681931139257303484526223461840939880759908449181135029327442039666513325916729112853/224777393704985303260433439401503507705148576727364388720387174977154204629336985298082893443205223017966145098064861566754244624732671470935725326043491911142467178027013112527668183783450932992534517827863293825061296257420183669383719043873666284378647952385401303733274771042385666436214517205678674213118374889121542369996042313879258622144464565403138392064000',
+    '614262617634904509410555695398092190166323289787703954329113412647342230855826568873946233136303566178604275296873234926652080462259552775254755109931312516616838091852915397846984381183732773411263828600108443070778861986849272794442105107285234729077818614332155890946804257724112143631642663084660012362691100872465148864413343598067567142977134846266425075195546265101720326272502009234689388355365794889444257557504/121200854655310345878069155190734381476914416893031941761756379148622591310152793328910334357724762079139509813381997287954319983165971628429295803162563856219211627796200599196263868781937234015821367575512548522790158101892507996780675863234181700253545912643548466256938936835469646114659598727592390628209362096738343417775009361678599075671154405137663077798419897932404104579113056038271753988669813988076515476125',
+    '106893445531995430131837569895703697604978445859412739331042151546083234373443601923983714252801103551874986033841637079479794895452077803182698291822629713500021995241940133037200268381793282318691568234728329378829301282121562652452936297422136961147432559766896143081059518238234777055931999714973999553456547746825617168005650221542417216291412324174772893635832/21070287546583154895102927029203125577385528350171457634203261197137238923923245382493868134869009844094435068284335909266152042903157700510534904847257436666980671067329084409346149729385625108233234542133838924281113454526471855573271066865534480974484196644173270141056470857666632902827625475719738076524047736123182255306429677688129231682978570461273193359375',
+    '1112742798776748551928744983308584493817112400536084744850969025063360679785378789764875076529575583792571198510306186417773101346425704969671256564494135750783642910394213727189639661184379341961737135764570942961692159177189753139209638929302928699705042832922774012622017270172331722295881412111812933593370743210214549038212395941675257558442878012136484052827439007483777859391215222647499741572327725689840850048/219121905514652575155579185322367175400669386131891843382447432851122462422992147756333729893028362013318264155489537138133372703295804002840920402482110260590055749843170064921843460498824128274420925357547023876488939786863662801268798987685503391058250902634999352652063668663245829496836853509616873246554292796823779397317819134891873583857621623143253788854530146303641400986028503534408870194989478421253057625',
+    '983995170259637497940121707857673268620603982844439610076334749792119502557380361624096960070148627663305171311646851566811683351232268240169848810658089698606409195744471343416197434926513684882439596141368794175162315633192028977211909698595778839497844847281262559899914504080092844816319298286511436690429175757804604673752838786852277255064434365401871903307643705/193579499709739777577893639665643417521924510242119561765932078835121983190401762561436208758210519231103767232899529457440602166918701550386152070813526760823513884391055054076485295945972185402977095815609448651316824455250005505410192101148149284756928250258632031981439345164037318129987999546683113483700660879431801728391911161900686707231543216283304182063038464',
+    '30691013076836380319652884848585431917439358375118437035944950498156836885409110087616602043242747413728866747975631727566613593257623099549583001362246649928075859944695060947434885707102684568632559339383409378997568182445019183928058902445738702610436187907199482604867148445903921474846103586253320950728989228898324829144701782792767763794895137313050381112872972806712570559942583906619373159053654481250809168128/6031936190594376585238407905469035598668211679521553144958053099463931217856780171872253258738635531385711702176347575597387389685526098889278278029314982721284691198967204664771178761416891317672705719303088746953644978471289001577698554025410820041497108311270431130911021653860874238212040640737171103823631718483572464361276703378587106311153455557896527384426327053165214814954154328483257657424873929502112904625',
+    '33633002641218571027682690377239300567277297301791434890431645909447040830457253186262970747405514897340315490970846553666201190386457221846694126637998462873786334837901001833771510353365825087187582807843940479192544772559118293851131234910638705258664901225494129023244916125545283621607920983287799648992632814461828998517949449887522866259724675318233522060729654796/6603800575190864815125307817934327053530311915572323544225173935830559756077637251538096164499112411672913910767032482188570876526606118068929913515497400397562722669603861813453418621246538114872330536044519242663538662076583511078523573320035084744399535581216815034837553165853576616422592790103728141443697362408727056813977234078214386771115586436985105038873998875',
+    '9090550743395453237608373761422705922989356346696677314277463900103283296889488490217663400349361962695279888433677856471792856853109469433355422802663947305503479561493033851081993247556828544308258821557994512133708656174864544457143793106703578476914924328960109390350040954158797845288786321215010324565010420734044059968931558099077932776525974187717655105951506223649947806568984848883258144616897526257038550656/1783219994432430133950472014582130883873515063182143523712063859964133099770763874744121895885645346884371273930727450069430704145915261127061538288702855011220545530326588799412076879985026006593893648911637986337776174555198865379578424838210466703683642106934154603710587408512357351455891557594977191641529451749437270208258072920514602243985184744955476557012910446023911358037761232253615162335336208343505859375',
+    '396226087129511787616470716244996965529391571019620342657409150252952330537366537352182763477820160355931003174946109480036915624974500187179367618457008802667089004678925595775249725629495848146669773574395699032785024522882892752987825063933573399691764866816284097232869198405981830106568826923407996367321353003569052419999880089180418759265274331299787468134974811/77651132902382550540201641628906752945078964002822585000619340261128555048628863996784740469908853602590877726845475670110127527874088625214912598995012971630852593597027649154487264595100300180591016581254965295280089902497540043916128232871685159131211213497239568483141031137909208031392464815187505714066235868748950184065923236670418822911614780061752905170944000',
+    '1288523130900767412477856952035690867153496584747510253984686336382980221603917538996138294311274357713893956725672062832264871415025750136690143411884111580335458539824573239771267043414225271729141508235839982033182605445173050590151912297720800168687369852280323356691345317649445843078291369394324862039143247087442309135074453265658730028145500871559458519282433012494054724678302957276371418936282902434203225600/252284450515080370603949213084529993395050575951188044032908270470514999880554862449223347761507020354321422785760279154913310488296485989979706208254206974075438570521939826124860483842758137231373182496438568202110337751291106165654945352183167578675508602832478752400011567482434301614269579368882858973850914513870297071866544875650189106576672289734326492492059816669288907018095832842249738081022227281715919357',
+    '559506782175368748187145067236900803925823792088293851453005882621173124590368101213652371965740365119375069053747415415684193092607523211636912074219215053809881814859129515591479780669511533619258835199715146655774919418434080807837587396115955105126806762961292429904380132116986984435528307613054828929294602185927322238461491059740342387266084431988727321711184308176/109446188216864405154936682374609448082436347600301765944233278303643744736544109262794383069197271409398615846380701459882784547857821831816863216080069944818453908760476475874319791512782445408136169878207358663250499628002971644430164185578966399908622640732411885568638054229180673755690889471388585521113862492432946110902070139679164884208988489955874131206343597375',
+    '365842109776085256207829452385781129402656684569922320812625438296958174120084800064669048017227033185969596458778213306152397766619692644694641677223213185977626940193107608138827771259334379895020377700759789774994800008933137569543592829220611543060189849601828342989023692876182148818363689293009174358168826340010491728715549904988196347316703985894355217235339548078187164176255405596310602921647203428200269666176/71497266937783826306774367225149833789788300609530680399994804704196212252148324552193149332755057982662844988588678159610516634667597707204818296457616315484973288944637139939912166034691155272078609876888919170894915224022982785138928008133003473320018151370486895637936102520353951511525996197267572355782608352075229547764732602557133494648237258900273354364050662921889937355682486287799239363976556848320046933875',
+    '59378684894605773397070783936113244261297966863668546282887298999811950599772732377979382620766180662159460666220894896896564357533604962977968843474062092083279165837424210591042655271577974272394203253875965589611493384384257117088518521598379301169916060951350650621757237206805037261878212424267780312614942173870651684587166691539909068452932891649554575072205080763/11593916877920131260484148322810317629326850763057953025674154928920977965095141485536566217873796019006709247938019404840440511433467040918770165456925609124166970862430104329357419669175323763143377917913760028500000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
+    '440706629842402075137926797732405268015915199334341565218443748798479563505125605478061664327658137598204192554352688830679475840386882350034150077506804799859586772904929402683276898098601814091430598259347980573084686932949502809068565420937301632494387691284902026088950093888426872222813857928092871267130505744095697172494119689779474840037266295586359244899477781345249657455186847452249040895151288114085352974933708544/85971876318334010542313048056773419567285701852816309711738644074302467763075671531248793080425349570584102078764464196653843973936119037322000695949209283327606280423406302517302874806092835694196353778161663338667091195199032831587897410329509487165187753418615356836636352319310103639079123698118997500068644500893282438137614420288839230827096109835206605385673197123994357393257304049093683934224826006704198171592304125',
+    '13220371178312284248107863849287558696210741535280184003301555188086761767176560823174196915070378763091889333203635784037294769747641959860810039816533609140662043493962707445564057197150603226797446933532133956054671103942520142568674339411250001585991743608723150284270101326419464088290151483823061613403673956918458456165552482913079283537536214967294211244361353060/2576683013797055224377707713704899315377379548630511095136568220145808780162671473907260623503552385384482723173913562423316997528913651044277418227616447815609873991028073011738113927961824632868956802271260292075825561646156680740323686126811928372672670917332613490241615850953660246271541592056143611581223295045062157404996958100136358649109693659623847024011265147',
+    '1538372578072246705266078247950089219881197139399081643650271348770110178217959856819170325400476655944115553238940541695152110949081675985299189913515975242840473616169775880272489805123124548256666658584129873597192213080038937409046114908719298170078476325423466734429870048697383304265342125425087105698152643145531760853127801123272389134514333752173173161562273487752064850461823726948588221913540799816489566883798912/299566102446745552813544808169323270315327122249327394279467999308669285416179486075255023523726449428151807672528368606071047812907935426406886253284580296349349558797128593446316786243666290649442956134448480598303527795996222703837663605600624271807461605295956674925277658331760000117761013381199880905748188151556184620779667670278558839116983636972680749836497346863752662521197700717702790359162307617024566689271625',
+    '281696937222363685333688097915539146272117417024070728031063821781398985795813333650547982965341298894634762289669256129805545549280355643122501181852136332529515756074502902204877664395456278205770206019663082511098956468724829475220996427130295010430751942126263620206886404244432409164651524408198286831329682605959881993845189585640874220032515465014097345759490750307207/54806274927935495596891312029173862473687610099325998583092091915101265393157631244775387832621032230434007486602816972765210205596482942091518459849775458089575743699797096697542306049628939919466378189656065461708686421130956996965004776108499273871055615413566043083634090083096967567965373203683231171594922347812966905149873265654962896456451681332216735396249206784000',
+    '228506238533410805238253122961471637501498406104430046727793420497895852412513560203475440156216063574654372726194215013407325207763369075250819386897086702757125418327084312933061738392406321003231853349865003322697360659712771167066453430309997639548320673279964210730701373707765051246595806673865646607841876463608039111252607452843372970797479959213447237540830563469297144446069689149549515457318899981763489792/44418759813188470246596822974281932219088697005123285598909329328419879212164564561558656547678211265529803086702843380128457744445765859393426675396393053094588240457569380155526179040891009717323059641035189017918495485630657360525410506453554255734005459614858972904374739673834145083512160883402151636664208253116362089211658645312778404886460203600762546810897889629330022309705583438699250109493732452392578125',
+    '2396110297187570506257836426251930521245117960980667897194680897478996322590925380310917400763581787981992131769075961259393685331523600287377841990678117113566991100720996059187144934569703045966210306282348527729533806882408323961336166190162075154579344185244957673138372347997134388801085788353482276257401005734929059700346687783400772331280644647089520646562669857652648/465370154262863030800729180763091967573522779260407615567563274096037297593764955820626213827962697601334935773802201919483657310282236100250543992334865461800976838791035335861420892967699843608855185710753433852279737724446294443337363478679601049432404435672469098850676302728116369979460222148206944323730586138087862719386327396334468302608499235112520318322763857676375',
+    '1188272888075247520453568790651819187486389546755745961952033150576243575862120569399808467067800830890874391487822723879717525264875722720416173276963170672622214121308038530968291758445510982186014289745026439477197148445087614813262170975056866881909056450023324743409114115141145340480846088869807540798073417035168139476153897066325127540609756310068558610112731607522096096821938163134080426356894899245575410494080/230586762395914863625709048789913840511134004337945700368117795195769576856434588144361459740833007125516905017415184787078710001969815416281381427387949263792200577791644278234661952852149216540491852757174842146376788280677956350901082838474705227642420261641783722891948710833898248566893322144435457967452449965772543077421407816225704952270697016152452032506898311260979594550143344971416786595888053149375287120733',
+    '1072184299258468249004114536023934869518152618369895613271618698908405514630154819453979477077738526673716264830106745031385951736108376866253955319711026042389282009348594845497252059906901748860111986811345742998968284077119165082435598381810306691078037018432766116865792677955520465319184311529667642995217214507658693259694586107847099911341219913576038605872218680562047/207882129535607118520818565811162643811811563516006836924341123976848022970024793004700710992938987476728640943834012473687892481059618823520721737479364352887178017866425027953255856440099446112508663603319745066750522936408781591578321031173620445865113295407603090322283539974563690157418931060565698315406021569491751612761888641141446406531645379264720684248398299136000',
+    '158094023904374464057702010126993231151433474361820266584020400101533651587072432728050776506672527718975042616651177034635188178967462982926261516748842073587981953038445977514446838733557926696939591069712397947055261777416704028506441173540338327591514650398349069348381309688677335132100441905603664282735310299686268721634369502296867409514777359935022301425165750780988231639828675424239931533849896744198012747670272/30626384366513923671063955469537776291182594080083494737341604579575446319190916924682809546024369330706012653799783122328775875399363759308193432327358837772825609356553955350401725689682838701996446404559005896479618311357573203081153855961484398360670837599853294627153740224543175497655414585918231215639906924313779933211774291565425360415684556655021031131349317390382320286092667852173815259422737956767922154798375',
+    '12216278924286401897470540169822542417611356797676160128176706721343511598951866107858729405615631246003458062949723290192242594609994613705183460814797527214549142043573863947658832694800184691222426521703728394203722420336773310908687071670107794543022567943478843065246011544888390757810534211554666736239506503613750756246364621509359804902243487113673014588325806249988332/2364584127734680580481708368927921125934330000162935118371174269642831409084634375654239735054363928748162229294829526339927907778510530444646754603322373712788147193691037688552857183116332292048731726305451516954679277320777955342295564000586923227775208580772868590742136150431152495718132142742153643081187691849828217592954519485869135930755646768375299870967864990234375',
+    '578939195017447864385837341975558679292903716580114421030538728778491606282153588930983966100514147976321305616196463174070345884292672842497715966933722088536619333912687108737657068701352918582656105418491672377271304970407730786932913490869797471081081431072836814581099271832696990064828717287258860166842791984158634684078346453202928137987930328826212011774670396356151265781053547412633310327737900786639842674288716416/111966291649651600701875186827360807558960817589719096407887999108024557254221008849510152469119211567152616745769256123968832250104006560061907741567034087109512483357851919402017950812795426393415409198268246337542639470834168040295134130657073201020340432236723824670803498949071870597039309004562024033656488150406105175401372709011082725359382691270659245312133919518760328422827067578306005644825927050994860723405473375',
+    '8041514821545833865350873212853873042112343327573364523948780030849543753072688526717886590013028227714585965095216381829914996539132359247488639692828639170075726301411777307168737667077730792892326979446426349813642680537699905651125068091362135055902152994216233622733424128982011248923051410141033146604688083721413402395290842738021365861670415833356403364696759583009635/1553937833499559104910551811592138833189892464446761247731330672284191313590708439680460540329836344767557599746234213006493842842879398684745864919973051635127080738612248150108793397855113122368507483429209127035263504194847798809612278864438287815470711036548695032421148125849669175729331459149833945183228413961985381338162155596008680906676498435954178226973272371625984',
+    '12059659399309169167301353842123191409306995788633965680414620302285606896146667751472818541537183787935352700725982053536032930866708413010971856856768384974849695774437178790890015242323873769697407411558706626105140596160049735605389301414460825674631122462760014451298494875244754354054447697502521388381253293175367558014453865905104467382184997528879171136492277222127399012368677604910497789360212571238146627048080568400384/2328494054002703067219004484782931471919751585969487160602269605916676223315055450845942763690581877771159615428854802721943114463834992106300185328876910271612375509166975628540839858771078495267566008071532876816826751069408153323840515037497664871689085370699991860738368371510578076683395101062439493831773194498579620513073693022191702398601471237166387045790480861932005012807234071429628725981935147069738988682858417182625',
+    '683009426705008850682549700382901603742691705123356866338951397347368059105140826655486518416578697931244347554322654428162554903861783227680935341690740579867651545205562690172425016836559678937794312287639193745517811234257062356826622207475918293015140472942579713388671782202262396642735640315948612572908444501108579457520714632371056288874351015666995549139003946314085696/131769214246522869780429817236150406548549044658474416909806179734873399109656411633488099249198882165279745627038984487365589625562901914030850178330133080409420340227179447548044572594184601425691821617928130423410997853384425373999967759499571890326740433081063424903098252502791477854302204200941376983295622430177445507316706609802292121282957519571162158620971952084486375',
+    '1418116259749091420309271913909337143756589314597557128553117870573928055109322864794169592355058370891251425125187770724828108988587888087023982451205475980371440423992838481547236638456717520835668859093413956339261802305597694403381021863258753441889536361061407579332570493634254542013337999844407181346815549594836374497512447969751514466481466717590255369810506813543577735080228718660747940956248427356785784199073569408/273368882980516556922532339653303822104510203916024760371844491705599093435133738774975538364241630079468712981056336655766117378160822393876348755370711942273876284944422610974149712212944211513462921649872232656449940987552227525504529710373144228818606127541937831103642415534318303187393593236209298242574677293352798419428712873685798636474667204083938875672934235816396371477146089168375198141802684403955936431884765625',
+    '546411077581845999248238069684194831621691193366466972898628141511422317954477211865167402284211971296286346492698354214043840080181702652132835681239302798339667835439189275457116051279366194551632712819028177402221815140045210690908631767786711675029194359998826252332434573121740830827417293557234891406021102083555472155479148332697839972702631641910213821827820374586459319/105246945889799140338072754365901705544530661781295362563214285953881890233987516230568663143342952799036341701952267011236663118111226623815975847494372076420199418712941018288738590264475520229651370015377608638075866459684692488079223686296541020897904588347083045723167762682959422858996673894180805702299907478685374065593161226849447131100462022089783265346597553176576000',
+    '2398330640958841474772606439916070050977544535580605737383995160447105736276950196885906408317628083110923322157113892928963237845914017845444295040924101784423382681801754191301860383927129006953354739240926643562987838836997453985855576402628166875869041032631651591871962852884189548538272285387092843044669499688035134181859376665409767886188304314888753894905317929877238322615838524354191263502347881033855441181420399360/461588070868590122892265681879734295007029130965626060552783760068897000195207878227714842617470320231527222074701444349530952699708435668339712860464533455345665068841333232359698449088497137068713309811942968433868609329301082001752617420002377892756821532220676085014874112083615054550278903960627185675459015343606391094523511117705747842645927349130302549554534056269331809016770715819934970200483161548527932617036185253',
+    '6041015879424725383006424536130409209607854044642113747266098198777011981328765528361630516108680392500990580908509403483891763219659726090675140672989657743882183951954294745396417829943469201306594018454995862321821016087416840247422350906412007336103086620396467456181771583200365740253389107968122850063607085957109965406634738740996318415514360956028575560979203447735121436/1161752799109428422288020947061281540989708937450568100764830251908850596717606701047413407636907934320789870175907792017513896999208892282137299070761467096211814586909598705615312819596495636017728313513520193786266452836805291464826226833593878504804389728477191170027729963773716267868284479768397603444919008915279522376004326398403851684761808785381609370767169521034383625',
+    '13240077436443988749179508462267267187169441948722358165090554769250505713747934643200804819418670147225695324432684266924694524337920816452346599774452681831320005286326986675907899608537972384924882996757503264622991355949039882526389342174307168805166215838138277557052303430492669193939212362638263582899713198716541723383138016564027766560215944409353427176135895982596327685665844815618402881202645610620284792793420780517248/2544223084468158291883698813309541801455311468982232546872485444308211415529998472787377800559884210837213042932180479090277285630234238711851480232520137856848809986631784843528381778520727465146661792797924458540957133423665746229799675650290296217658444899605236550972043549278128087645211909479009099766619355677984218929672461506691980442071860591767266913041147587815452007726513853820116629482732060593116624596368806566625',
+    '1953999166296955830935495158735359200362904181792947794529339487489730042568305997099959302322956898299616194932283060554261566410988618045107398092345476532371402134206635235570281738377188438407703089325315446371127042537576093536896282955524842632708645655481028161471313608974238110718242273935956977555610147714316158486553633871312187084618154014921190595222799283957140353/375191165084882521037046014569185165885459082629136124177286500000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000',
+];
+
+if (typeof module !== 'undefined') {
+    module.exports = {
+        PRIMES,
+        PRIMES_SET,
+        LONG_PI,
+        LONG_E,
+        BIG_LOG_CACHE,
+    };
+}