Live round on WYL-72 exposed that my debate-round comment produced
sycophancy, not debate. Judge-Gemini moved from a 4.0 ACCEPT stance to
"I agree my initial score was too lenient... I have adjusted my scores
to 3s and 2s" after reading Judge-GPT's 2.92 report — without defending
its original scores or challenging GPT's evidence. Classic social-pressure
convergence.
Root cause: my debate comment said "You may hold your position if you
have new evidence; you may move if you find the other reasoning more
grounded. Do not split the difference to compromise." That phrasing
is both weaker than CEK's intent AND it dropped every structural
anti-sycophancy instruction CEK spelled out in judge-with-debate/SKILL.md:
Missing: "Identify disagreements (where your scores differ by >1 point)"
Missing: "Defend your position with evidence from the specification"
Missing: "Challenge the other judge's position with counter-evidence"
Missing: "Only revise if you find their evidence compelling"
Missing: "Defend your original scores if you still believe them"
Also: I asked judges to post a REVISED report (implicitly retracting
their prior position). CEK asks them to APPEND a debate round section
to their prior report, keeping both visible so the revision is a change
ON TOP OF the original rather than a replacement.
Fixed by porting CEK's instruction block verbatim into _build_debate_round_comment.
Added a regression test that fails if any future edit removes these exact
clauses.
Tests: 72 passed (+1 regression test).
Judge-Gemini (on github-copilot/gemini-3.1-pro-preview) emitted reports
with \` (backslash-backtick) inside double-quoted YAML strings,
imitating markdown escaping (e.g. evidence: "see \`foo.py\`"). \`
is not a valid YAML escape sequence, so yaml.safe_load rejected the entire
report. Judge-GPT did not make this mistake, so the consensus degenerated
from 3-to-2 parseable reports to 1-to-2 (and therefore produced a spurious
single-judge convergence).
The fix is a targeted cleanup in _extract_yaml: replace \` with literal
` before parsing. No other interpretation of \` exists in YAML, so
this does not mask semantics.
Tests: 71 passed (added 2 for the new cleanup path).
Separately surfaced (not yet addressed in this commit):
- Judge-Claude on github-copilot/claude-sonnet-4.6 and
github-copilot/claude-opus-4.5 returned model_not_supported. Per GitHub
Copilot Student plan docs (2026-03-13 update), Claude Opus/Sonnet are no
longer student-selectable. Tower-Copilot-Claude now runs
claude-haiku-4.5 (which is student-accessible).
Multica's comment REST API returns content with HTML entities escaped
(`"` for `"`, `>` for `>`, etc.). Agent replies are plain UTF-8,
so we unescape before extracting and parsing.
Caught by the first live test against WYL-72: Meta-Judge produced a
perfectly valid YAML evaluation specification with CK-001..CK-010 checklist
items, but my parser reported it as malformed because `"` is not a
valid YAML token. The model was fine; the plumbing was wrong.
- src/coordinator/orchestrator.py: _extract_yaml now calls html.unescape first
- tests/test_orchestrator.py: +2 tests covering entity decoding
Tests: 69 passed.
The prior pipeline (4 hand-written debater prompts + 1 judge with my prompt
template) kept missing scope drift because every prompt was mine and the
reviewers were all on the same model tier with correlated priors.
This commit replaces the whole review step with CEK's judge-with-debate
pattern translated to multica-native execution:
pending → awaiting_rubric (meta-judge writes YAML spec from issue alone)
→ awaiting_judges (3 judges on 3 copilot models score independently)
→ consensus check (overall within 0.5, criteria within 1.0)
→ accept or reject OR awaiting_debate rounds up to 3
→ error on malformed YAML or cap hit
Per higher-management direction, we do not deal with a model that cannot
produce YAML: malformed rubric or all-unparseable judge reports fail the
round immediately (no retries, no fallback to hand-written prompts).
The anchor retrigger on REJECT (WYL-51 behaviour) is preserved verbatim.
Agent prompts for meta-judge and the 3 judges come from the CEK agents
themselves (Meta-Judge / Judge-GPT / Judge-Claude / Judge-Gemini) whose
`instructions` field is the CEK meta-judge.md / judge.md files uploaded
byte-for-byte. No prompts are authored in this coordinator's source.
Adds pyyaml dependency.
- src/coordinator/orchestrator.py: rewritten for the new phase machine
- src/coordinator/queue.py: Round extended with rubric_yaml, judge_report_comment_ids, debate_round
- tests/test_orchestrator.py: 40 tests for new pipeline (helpers, parsers, consensus math, phase handlers, race fix, retrigger)
- tests/test_integration.py: removed (tested old debater pipeline)
- pyproject.toml: adds pyyaml
Tests: 67 passed in 0.20s (40 orchestrator + 15 queue + 7 watcher + 5 other).
Minimum viable structure for the Tool-MAD coordinator:
- coordinator.config: env-loaded Config dataclass, writes state to ~/.coordinator/
- coordinator.multica_client: thin requests wrapper for issues/comments/agents
- coordinator.state: flat-json SeenState tracking issue_id -> last_seen_updated_at
- coordinator.__main__: watch_loop() that polls in_review and logs candidates
- README.md: why this exists + how to run
v0 only detects in_review transitions; convening debate rounds is WYL-45.
Dependencies: stdlib + requests (nothing else until a working v1 ships).
m-platform-admin
