Pre-Deploy Risk Score: Six Signals That Gate Deploys on Context, Not Gut Feel

Every team has the same incident in its postmortem archive. The Friday push that cascaded across three services. The release that landed while two P1s were already burning. The config tweak nobody realized was wired into fourteen microservices through a shared dependency.

None of those failures were code failures. They were context failures. CI was green. The diff looked surgical. The system around the deploy was the part that broke.

The pre-deploy risk score closes that gap. The agent fires the moment a deploy enters the queue, pulls signals across infrastructure and team state, and returns one of three verdicts — HOLD, PROCEED, or WATCH — with a numerical confidence and a plain-English breakdown of which signals moved the needle. Gut feel becomes a measurement. Tribal knowledge becomes a check the pipeline enforces.

Most deployment failures in mature systems are not bad-code failures. They are context collisions — a technically valid change meeting an environment that was not ready for it. A migration that runs cleanly in staging, then deadlocks against an active A/B test that doubled write traffic on the affected table. A flag rollout that touches the same API surface as the incident remediation an on-call engineer started forty minutes earlier.

The 2024 DORA State of DevOps Report found that the high-performance cluster shrank from 31% of respondents in 2023 to 22% in 2024, while the low cluster grew from 17% to 25%.^[6] That regression happened during a period of increased AI-assisted coding — which improved throughput but increased delivery instability. Teams were shipping faster into worse conditions with no mechanism to read those conditions.

Overmind, which maps dependencies across 100+ AWS resource types and Kubernetes objects to calculate blast radius, raised $6M in 2025 specifically because the problem is not the artifact — it's the infrastructure state surrounding it.^[1] The pre-deploy risk score scores the deployment context, not the artifact.

Blast radius is the heaviest signal and the most technically involved component of the score. It earns its own treatment.

terraform plan tells you which resources will be modified, replaced, or destroyed. It does not tell you which downstream services depend on those resources at runtime. A security group rule can be legal to change and still knock over four services that ingress through it — services that don't appear anywhere in the plan output because Terraform only knows about the resources it manages directly.

The gap between "what will change" and "what will break" is exactly what blast radius analysis closes. Overmind's approach — querying AWS APIs in real time to build a complete dependency map rather than relying solely on IaC state — catches cross-stack references and runtime dependencies that static plan analysis misses entirely.^[4]

For Terraform, the entry point is terraform graph or parsing the state file directly. The open-source blast-radius project pioneered interactive visualization of these dependency graphs in d3.js.^[2] For production scoring you don't need the visualization — pipe terraform graph -type=plan into a parser that extracts nodes and edges, then run a breadth-first traversal from the changed resources outward.^[3]

CloudFormation behaves differently. Stacks expose DependsOn relationships explicitly, and the AWS API returns the stack's resource list on demand. The trap is cross-stack references: when one stack exports a value another imports, the dependency is implicit but the blast radius is real. The parser has to follow Fn::ImportValue references across stack boundaries, or the score under-reads coupling that production absolutely respects.^[5]

The scoring formula weights direct dependents more heavily than transitive ones, with a decay factor at each hop. A change that hits 3 services directly and reaches 12 more transitively scores differently from one that hits 12 services directly with no transitive reach. Same node count, very different blast radius.

The final score is a weighted sum of normalized sub-scores between 0 and 10. The verdict mapping below is a starting point — calibrate the thresholds against your own incident history after 60-90 days of outcome data:

• PROCEED (score 0.0 – 3.9, confidence ≥ 70%): All signals inside acceptable ranges. The deploy goes with standard monitoring.
• WATCH (score 4.0 – 6.4, or confidence 50–69%): Elevated risk on at least one axis. The deploy goes, but the agent shortens canary windows, tightens rollback thresholds, and posts a Slack alert into the on-call channel.
• HOLD (score 6.5 – 10.0, or any single signal at 9+): The agent halts the pipeline and pages the deploy author with which signals fired. A manual override requires two approvals, both logged.

Confidence reflects data completeness. If the experimentation API timed out or the incident tracker rate-limited, confidence drops — and a low-confidence PROCEED is auto-promoted to WATCH on uncertainty alone. Treat every threshold as an initial heuristic. Calibrate against your false-positive and false-negative rates.

Worked example: A payment service deploy on a Thursday at 3 PM. Blast radius: 5 direct dependents → sub-score 5.5 (medium). Seven-day incidents: 1 P2 → sub-score 3.0. On-call load: 1 open → sub-score 4.0. Active A/B tests: 0 overlap → sub-score 1.0. Outstanding P2 bugs: 2 → sub-score 4.5. Time-to-Friday: 26 hours → sub-score 2.0. Weighted sum: (5.5×0.25) + (3.0×0.20) + (4.0×0.15) + (1.0×0.15) + (4.5×0.10) + (2.0×0.15) = 1.375 + 0.60 + 0.60 + 0.15 + 0.45 + 0.30 = 3.475 → PROCEED. Same deploy at 4 PM Friday with 2 open incidents: time-to-Friday → sub-score 8.0, on-call → sub-score 7.5. Weighted sum jumps to 5.8 → WATCH.

The gate runs as a blocking step between "build succeeded" and "deploy to production." It's not a separate process or a Slack bot that posts a warning someone ignores. The verdict is enforced by the pipeline runner — an exit code of 1 is the only mechanism that reliably stops a deploy in CI.

For ArgoCD, the pattern is a pre-sync hook: a Kubernetes Job that runs the scoring agent against the target application before ArgoCD applies the sync. If the Job exits non-zero, the sync is blocked. For Spinnaker, the equivalent is a Precondition stage with a script condition. For GitHub Actions, the gate runs as its own job that the deploy job depends on — the needs field does the enforcement.

Monorepos: The single worst mistake is scoring the entire repository on every PR. One team initially evaluated their full monorepo on every commit and produced HOLDs on documentation typo fixes. The blast radius analyzer read thousands of resource dependencies that had nothing to do with the change. Scoping to Nx's affected command — which returns the precise set of projects a commit range touches — dropped their false-positive rate from roughly 40% to under 8%. Bazel and Turborepo expose equivalent affected-graph APIs. The rule: score the deployable unit, not the repository.

Progressive delivery: Feature flags shrink blast radius by limiting exposure. A deploy behind a 1% canary flag has an effective blast radius of 1% of the calculated value — the risk is real but bounded. The scoring agent should query the feature flag platform and apply a rollout multiplier before scoring. The structural point: progressive delivery is itself a deploy safety mechanism; the score should compose with it rather than double-count risk.^[7]

Degraded signal sources: APIs time out. Rate limits hit at the worst moments. A scoring system that returns PROCEED on any signal failure is a liability — it trains engineers to trust a verdict that was assembled from incomplete data. The correct behavior: a timed-out or errored signal source gets a neutral-high score of 5 (not 0), and overall confidence drops proportionally. A sub-50% confidence PROCEED auto-promotes to HOLD. Make the degradation visible in the PR comment so engineers understand they're operating with partial information.

A risk scoring system is only as good as its calibration. After every deploy — PROCEED, WATCH, or overridden HOLD — log the outcome. Did the deploy cause an incident inside 24 hours? Was a rollback required? Did any experiment results get invalidated by the change?

Store those outcomes alongside the original scores in a tracking table. After 60-90 days of outcome data, run a logistic regression against the weights and see what is actually predictive. Findings that surface consistently across teams that have done this:

• Blast radius is almost always underweighted at the start. Most teams end up moving it from 0.25 to 0.30-0.35.^[4]
• Time-to-Friday is overweighted for teams whose incidents cluster on Mondays or mid-week. Tune the weight against your actual day-of-week incident distribution, not the cultural assumption.
• On-call load gets sharper when the input includes the responder's recent activity timestamps — a proxy for sleep deprivation — and not just open incident count.

The counterintuitive finding: teams that get the most out of the score are not the ones deploying dozens of times a day. They're the ones deploying monthly or quarterly. High-velocity teams build deploy intuition organically — every push is feedback. Low-velocity teams have no feedback loop, so every deploy is a high-stakes event with no pattern recognition behind it. The risk score replaces the intuition the cadence never built.

The pre-deploy risk score is not a brake on velocity. Teams that implement it consistently report higher deploy frequency — engineers ship more often when they trust the system to flag bad timing. The score swaps anxiety for a measurement, and trades the post-incident "we should have known" for the pre-incident "the gate caught it."

Start with blast radius and incident rate. Those two signals catch most contextual deploy failures.^[4] Add the rest as the API integrations land. Within 90 days of outcome calibration, the score reflects the specific shape of your infrastructure and the team that lives inside it.

The code was never the problem. The conditions were. Score them.