9 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
ynankaniandGitHub	e4b9af007b	CUDA: XOR swizzle flash attn K,V smem fp16 tiles (#25635) ... * CUDA: XOR swizzle flash attn K,V smem fp16 tiles Signed-off-by: ynankani <ynankani@nvidia.com> * Fix use 64bit generic pointer instead of 32bit shared pointer Signed-off-by: ynankani <ynankani@nvidia.com> * fix shared memory race in FA on DGX Spark * Handle corener case Signed-off-by: ynankani <ynankani@nvidia.com> * Add swizzle test cases and gate sync for swizzled path only Signed-off-by: ynankani <ynankani@nvidia.com> * gate CUDA PTX Signed-off-by: ynankani <ynankani@nvidia.com> * offset calculation specific for swizzle branch Signed-off-by: ynankani <ynankani@nvidia.com> * Reafctor code Signed-off-by: ynankani <ynankani@nvidia.com> * Refactor FA swizzle ldmatrix if/else into helpers (K row/col, V offset) Signed-off-by: ynankani <ynankani@nvidia.com> * rebase and update test case args Signed-off-by: ynankani <ynankani@nvidia.com> * Allow swizzle for non-pow2 shapes, for which nbatch_2%32==0 Signed-off-by: ynankani <ynankani@nvidia.com> --------- Signed-off-by: ynankani <ynankani@nvidia.com>	2026-08-31 22:18:01 +02:00
ynankaniandGitHub	41ef91f7c8	CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion were restricted to 1 token (#27621) ... * CUDA: extend MOE fusion to specdec, earlier MOE glu fusion and topk-router fusion were resticted to 1 token Signed-off-by: ynankani <ynankani@nvidia.com> * Address review comments Signed-off-by: ynankani <ynankani@nvidia.com> * Add SWIGLU_CLAMP case to multi-token moe fusion Signed-off-by: ynankani <ynankani@nvidia.com> --------- Signed-off-by: ynankani <ynankani@nvidia.com>	2026-08-31 19:22:28 +08:00
ynankaniandGitHub	25ae3a9b33	CUDA: MMVQ nwarps=8 for bs=1 for dense models on DGX Spark (#26843) ... * CUDA: MMVQ nwarps=8 for bs=1 for dense models on DGX Spark Signed-off-by: ynankani <ynankani@nvidia.com> * skip moe experts and allow others based on k geometry (allow only small idle tail) Signed-off-by: ynankani <ynankani@nvidia.com> * rename MMVQ DGX Spark params to GB10 and fix MSVC constexpr lambda capture Signed-off-by: ynankani <ynankani@nvidia.com> --------- Signed-off-by: ynankani <ynankani@nvidia.com>	2026-08-18 09:45:53 +05:30
ynankaniandGitHub	5d16e81dd9	convert : keep quantization scales for nemotron --mtp export (#26903) ... Signed-off-by: ynankani <ynankani@nvidia.com>	2026-08-11 15:19:05 +02:00
ynankaniandGitHub	c5229087a5	convert : add FP8 to Q8 conversion (#23250) ... Signed-off-by: ynankani <ynankani@nvidia.com>	2026-05-28 10:16:17 +03:00
ynankaniandGitHub	42928bc14d	model : NvFP4 quantized LM head support (#23046) ... * NvFP4 quantized LM head support Signed-off-by: ynankani <ynankani@nvidia.com> * Address review commnets Signed-off-by: ynankani <ynankani@nvidia.com> * Add assert for NvFp4 lm head and tied embeddings Signed-off-by: ynankani <ynankani@nvidia.com> * Address review commnets Signed-off-by: ynankani <ynankani@nvidia.com> * Create output_s tensor only when LM head NvFp4 Signed-off-by: ynankani <ynankani@nvidia.com> --------- Signed-off-by: ynankani <ynankani@nvidia.com>	2026-05-16 11:09:27 +02:00
3 people	9f5f0e689c	model : support Gemma4_26B_A4B_NVFP4 (#22804) ... * Gemma4_26B_A4B_NvFp4 hf checkpoint convert to gguf format fixes Signed-off-by: ynankani <ynankani@nvidia.com> * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Address review comments Signed-off-by: ynankani <ynankani@nvidia.com> * fix CRLF Signed-off-by: ynankani <ynankani@nvidia.com> * Lint error fix Signed-off-by: ynankani <ynankani@nvidia.com> --------- Signed-off-by: ynankani <ynankani@nvidia.com> Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>	2026-05-08 20:42:09 +02:00
ynankaniandGitHub	0f1bb602dd	model : remove duplicate wo_s scale after build_attn (Qwen3, LLaMA) (#22421) ... Signed-off-by: Yash Nankani <ynankani@nvidia.com>	2026-04-27 09:58:48 +02:00
3 people	5eaee65384	convert : Handle ModelOpt produced mixed precision model during convert to GGUF (#22247) ... * Handle ModelOpt produced mixed precision model during convert to GGUF * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Apply suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>	2026-04-23 08:19:51 +03:00