ggml: add cross-backend profiler

Add an optional per-op / per-copy profiler to the ggml scheduler that
records timed events across all backends of a split graph, so a single
run can be inspected end to end (compute kernels, host<->device copies,
fusion names, tensor shapes/strides/types, op params).

- ggml-profiler.h/.cpp: ggml_profile_record, per-backend profiler
  interface (enable/reset/get_records), JSON export
- ggml-backend.cpp: scheduler-level collection, copy events, backend
  attribution, mul_mat_id stats, throughput stat, concurrent-mode fix,
  auto-export via GGML_PROFILE env var
- Backend profilers: CPU, CUDA/HIP/MUSA (event-based timing), Vulkan
  (timestamp queries), BLAS, Metal (tentative); stubs for the remaining
  backends
- llama: expose profiler enable/export; --profile, --profile-output,
  --with-backends args in common; hooks in server, completion and the
  debug example
- tools/profiler/profiler.py: analysis tool (per-op / per-backend
  summaries, Chrome trace export)
- test-backend-ops / test-export-graph-ops: run perf tests with exactly
  the tensor shapes recorded in a profile (converged with
  export-graph-ops)
- docs/cross-profiler.md
- ggml-cuda: avoid ROCm_Host compute on HIP integrated GPUs

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ney1sm8n1bSjeA3DrrW5ah
This commit is contained in:
Piotr Wilkin
2026-09-07 13:42:53 +02:00
co-authored by Claude Fable 5.1
parent 5202104b59
commit 16db737a1c
45 changed files with 3890 additions and 87 deletions
+23
View File
@@ -252,6 +252,29 @@ int main(int argc, char ** argv) {
return 1;
}
// Export profiling data if profiling was enabled
if (params.profiling) {
ggml_backend_sched_t sched = llama_context_get_sched(ctx);
if (sched != nullptr) {
if (params.profiling_output.empty()) {
ggml_backend_sched_print_profiling(sched);
} else {
const std::string & path = params.profiling_output;
int ret;
if (path.size() >= 4 && path.compare(path.size() - 4, 4, ".txt") == 0) {
ret = ggml_backend_sched_export_profiling_text(sched, path.c_str());
} else {
ret = ggml_backend_sched_export_profiling_json(sched, path.c_str());
}
if (ret == 0) {
LOG("\nProfiling data exported to: %s\n", path.c_str());
} else {
LOG_ERR("\nFailed to export profiling data to: %s\n", path.c_str());
}
}
}
}
LOG("\n");
llama_perf_context_print(ctx);