PR Flow Assessor
Every merged PR is meta-reviewed post-hoc. Patterns that recur auto-promote in priority. Fixes are closed-loop-validated against the next N PRs of similar shape. The SDLC that reviews its own SDLC.
The problem
Static SDLCs don't notice when they're wrong.
Dark Factory reviews every commit, gates every push, audits every bypass. But nothing reviews Dark Factory itself. There is no machine-readable record of which kinds of PRs went well or badly, which DF components contributed to the outcome, which patterns recur — or whether prior DF improvements actually fixed what they targeted.
The consequence is a one-way mirror. Iteration traps recur invisibly across PRs because nobody is counting. Fixes ship with stated goals — reduce thrash, preserve plan-PR throughput, make cycle attribution traceable — and no machine asks whether the next N PRs of the targeted shape actually showed the improvement. Agent process quality has no aggregate ledger; cycle scoping defaults to uniform supervision because there's no per-agent data to do better.
A self-improving SDLC closes that loop by construction — not by adding a dashboard nobody reads, but by making every merged PR a structured signal that improves the platform for every subsequent PR.
How it works
Async post-merge assessor; pattern catalog; closed-loop validation.
The assessor runs once per merged PR, asynchronously, on
pull_request: closed with merged == true. It is out of the hot
path. It never blocks a merge. It never amends history. It never re-runs critics.
It collects six classes of input: the PR's final diff vs main, the full
commit history, the per-push critic findings, the bot-review threads with their resolution
outcomes, the linked cycle doc and issues, and the timing data (push count, time-to-merge,
iteration count). A two-tier model routing pass — Haiku triage filtering for substantive
PRs, Opus 4.7 deep assessment on the ones that warrant it — produces a structured verdict:
outcome quality, input quality, process quality, patterns detected, root causes, and
proposed improvement actions.
Outputs are atomic and idempotent. One structured JSON artifact in the assessment store, one terse markdown comment on the merged PR, one GitHub issue per improvement action with a recurrence counter that increments rather than duplicating. When an action issue is closed by a PR, the assessor's next runs check whether the pattern actually stopped recurring — closed-loop, not open-loop.
- Runs once per merged PR — on the pull_request: closed + merged == true event, never in the hot path
- Inputs: PR diff, full commit history, per-push critic findings, bot reviews, cycle doc, timing data
- Pattern recurrence detection from a curated catalog — recurring patterns increment a counter on the existing issue, not file duplicates
- Auto-promotes priority when recurrence ≥ N — the patterns that bite repeatedly bubble up automatically
- Closed-loop validation — when an improvement-action issue is closed, the next N similar PRs check whether the pattern actually stopped
- Agent-trust ledger keyed by PR-author agent — process quality, iteration count, regressions, bypass usage
# .github/workflows/pr-flow-assessor.yml
on:
pull_request:
types: [closed]
jobs:
assess:
if: github.event.pull_request.merged == true
runs-on: ubuntu-latest
concurrency:
group: pr-flow-assessor-${{ github.event.pull_request.number }}
cancel-in-progress: false
steps:
- uses: actions/checkout@v4
with: { fetch-depth: 0 }
- run: pr-flow-assessor assess --pr ${{ github.event.pull_request.number }}
# patterns.yaml — curated catalog (excerpt)
- id: iteration-trap-large-doc
shape: { push_count: ">=4", touches_paths: ["docs/**"] }
signal: critic-finding-then-self-introduced-finding-loop
promote_at: 3
- id: bypass-without-issue-link
shape: { bypass_used: true, issue_in_reason: false }
signal: untracked-override
promote_at: 1
# assessments table — one row per merged PR
# {
# "pr": 1503, "merged_sha": "5d8e1a3",
# "outcome": "GOOD", "input": "OK", "process": "THRASH",
# "patterns": ["iteration-trap-large-doc"],
# "improvement_actions": [{ "issue": 1612, "recurrence": 3, "auto_promoted": true }],
# "validates_prior": [{ "action_issue": 1494, "stopped": true }]
# }Get Started
Static review pipelines learn nothing. Yours should.
Every merged PR teaches Dark Factory how to ship the next one. Install via the GitHub App for the hosted runtime, or wire the workflow into any DF-installed repo.