Skip to content

Loop Engineering Checklist

Operational checklist for creating and promoting loops. For design rationale, see Loop Engineering Design.

New Loop Creation Checklist

Use when implementing a new on-loop-*.yaml workflow. Add docs/explanation/loop-engineering/workflows/loop-<name>-workflow-design.md for domain specifics.

Design Invariants

All must be true. See Loop Engineering Design — Design Invariants.

  • [ ] Agent never writes to integration branches during Execute (isolated worktree only). L3 integration push in Finalize only, with promotion gate
  • [ ] Checker never modifies the repository (read-only phase)
  • [ ] Detect never writes state; detect script invoked once per run (no caller re-run)
  • [ ] Finalize never changes source under repair; .loop/* persistence allowed
  • [ ] State advances only through Finalize (and merge-gated promote for L2 open_pr — see Finalize below)
  • [ ] Phases communicate via GitHub Actions outputs/inputs only
  • [ ] Checkout is performed by the caller workflow
  • [ ] Every decision is traceable (skip_reason, reject reason, outcome)

Detect vs Execute (semantic boundary)

See CONTEXT — Semantic Findings.

  • [ ] Detect script emits mechanical facts only (failures[], changed_files, commits[], …) inside the common envelope (skip, result, verifier_context)
  • [ ] Detect does not emit semantic findings[], triage prose, or repair decisions
  • [ ] Entry skill builds semantic output (findings[], Fix/Watch/Escalate) in Execute from detect facts
  • [ ] verifier_context carries fact summary or log excerpt for verify — not skill triage report
  • [ ] agent_checker_skill_name set on caller (default loop-verifier); domain rubric only in agent_checker_instructions

Phase Contract Compliance

Detect

  • [ ] Outputs should_run, skip_reason, target_matrix via loop-detect
  • [ ] Read-only — does not modify repository or state
  • [ ] LOOP_DETECT_SCRIPT under the domain skill package (<agent-root>/skills/<skill-name>/scripts/detect_*.sh)
  • [ ] Budget / circuit breaker guards via loop-detect; cross-loop serialization via workflow concurrency
  • [ ] No domain vocabulary in loop-* actions

Agent (Execute)

  • [ ] L2/L3 outputs branch, has_changes, verdict, reason, attempts, open_rejections, usage_json, notify_context_json, failure_stage, failure_message
  • [ ] Worktree from target.from (Phase 1+)
  • [ ] Respects Skill allowed paths and denylist
  • [ ] Uses ci-loop-agent.yaml

Verify

  • [ ] Separate agent session from maker (inside loop-execute)
  • [ ] Read-only; outputs verdict, reason, open_rejections
  • [ ] Semantic quality + fit against verifier_context (always wired; may be empty)
  • [ ] verifier_context wired on every execute matrix cell

Finalize

  • [ ] Behavior matches finalize strategy matrix for target.finalize + DEFAULT_LEVEL
  • [ ] L2 open_pr: merge-gated cursor — loop-finalize writes pending; last_sha advances only when fix PR merges via on-loop-state-promote.yaml (label loop-automation). Fix PR is domain-only — see State delivery philosophy
  • [ ] open_pr: create PR on APPROVE; push / push_head: push on APPROVE; delete agent branch on REJECT
  • [ ] L3 auto_merge only when finalize=open_pr — not for push / push_head
  • [ ] State, run-log, domain ledger via domain_persistence_script in finalize job (no caller git push for .loop/*)
  • [ ] outcome: watch for Skill Watch (no consecutive_failures increment)
  • [ ] loop-finalize + if: always() with appropriate guards
  • [ ] loop-notify-pr when target_json.to.pr_number is set (spec); includes bot fix PR link for pull_request + open_pr
  • [ ] pr_exclude for PR-head watch; no pr_require label opt-in
  • [ ] GitHub-entity loops (issue/stale): caller grants issues: write / pull-requests: write; skill performs API actions in Execute; Finalize advances state cursor (no file PR at L1)

Skill

  • [ ] SKILL.md: allowed paths, behavioral rules, generic orchestration
  • [ ] No named consumer domain skills in distributable skill references/ (caller agent_maker_instructions owns stack routing A')
  • [ ] CI failure loops: caller agent_checker_instructions appendix for failure-kind defer (B) where needed

Caller (on-loop-*.yaml)

  • [ ] agent_maker_instructions includes repo-specific overlay (stack routing table for ci-sweeper-type loops)
  • [ ] agent_checker_instructions matches observation trigger (CI log fit, doc factual accuracy, changelog version rules, …)
  • [ ] Fix PRs labeled loop-automation for on-loop-state-promote matching

Multi-Branch Targets (Phase 1+)

See Multi-Branch Loops Design and Loop Caller Workflows.

  • [ ] LOOP_INTEGRATION_BRANCHES and/or LOOP_PULL_REQUESTS (pr_enabled) in caller with:
  • [ ] target_matrix with mode, from, to, stable key per cell
  • [ ] Matrix execute/finalize; capped by LOOP_MAX_TARGETS_PER_SCHEDULE
  • [ ] LOOP_PR_EXCLUDE; bots excluded unless LOOP_PR_INCLUDE_BOTS
  • [ ] DEFAULT_LEVEL=L2 unless L3 gate passed; L3 = auto-merge on bot fix PR when finalize=open_pr
  • [ ] Per-target concurrency.group when using matrix fan-out

Workflow Structure

  • [ ] Follows Loop Caller Workflows Design
  • [ ] timeout-minutes on all jobs; least-privilege permissions per job
  • [ ] Map keys alphabetically ordered per companion github-actions-workflow rules ORD-01 (env, permissions, with, secrets, workflow_call inputs/secrets, action inputs/outputs)
  • [ ] Unique state file (.loop/state-<loop>.json)
  • [ ] Denylist includes standard paths

Retry Policy

  • [ ] State cursor (general): L2 open_pr → merge-gated pending (last_sha on fix PR merge); REJECT / no-op → metadata only (last_sha unchanged; circuit breaker at 3+); L3 push → cursor in same finalize run
  • [ ] consecutive_failures + attempt_fingerprint on target.key
  • [ ] Pause at 3+ consecutive failures (circuit_breaker)

L1 → L2 Promotion Checklist

  • [ ] Loop operated at L1 for 2+ weeks
  • [ ] State schema documented in workflow design doc
  • [ ] SKILL.md includes build/test commands (or GitHub-entity deliverable rules for non-file loops)
  • [ ] Maker and checker separate sessions
  • [ ] Denylist includes auth, payments, secrets, infrastructure
  • [ ] Daily token cap configured
  • [ ] .loop/loop-run-log.md in use
  • [ ] Run-log JSONL may include redacted failure_stage / failure_message from execute/finalize

L2 → L3 Promotion Checklist

  • [ ] Approval Rate > 80%, PR Merge Rate > 90%, Human Override Rate < 10% (2 weeks)
  • [ ] cobusgreyling Pre-Flight Safety complete
  • [ ] L3 enables GitHub auto-merge on bot fix PRs (finalize=open_pr); never default DEFAULT_LEVEL=L3 for new adopters
  • [ ] Stop conditions and escalation path defined