An assessment is one bounded investigation of one verified target. It moves through a fixed state machine, produces candidate findings, and only promotes a candidate to confirmed when a fresh sandbox reproduces it.

States

queued → provisioning → preflight → mapping → testing → verifying → completed

Terminal alternatives: blocked, failed, cancelled, budget_exhausted.

Coverage is tracked separately from status. A completed assessment can have partial coverage, and the UI says so rather than calling the run clean.

What the runner does

  1. Creates a Boat sandbox from the saved offense-worker-v0-4-0 snapshot with noEnv: true and a finite TTL, so it inherits no owner environment.
  2. Starts the runtime services: egress proxy, purpose-scoped model gateway, nested Docker, Strix image.
  3. Allowlists only the job target's host and port. The login host is added only when it differs from the target.
  4. Preflights the target through the proxy.
  5. Runs the selected engine:
    • Nuclei for the default scanner path, then the two-identity BOLA probe.
    • Strix through the per-job model gateway. The gateway holds the provider key; the investigator gets only a short-lived bearer token, a request cap and a model-spend cap.
    • Shannon through the same gateway plus its nested Docker Compose stack (Temporal and an ephemeral pentest worker). It runs in no-source mode: an empty repository is mounted, and only live-validated findings whose SARIF status is exploited are reported.
    • Verification / retest through the probe in a brand-new sandbox.
  6. Uploads a checksummed evidence bundle and records provider cost, tokens and compute usage with idempotent external IDs.
  7. Posts terminal status and deletes the sandbox in finally.

The runner heartbeats every 30 seconds. If it dies, the durable job lease expires, the old attempt is cancelled and a new attempt is queued — up to JOB_MAX_ATTEMPTS (default 5), after which the assessment fails and settles.

Candidate versus confirmed

Findings from Nuclei, the BOLA probe or Strix are candidates. A separate verify job creates a fresh sandbox, replays the same request with both identities, and only then promotes the finding to confirmed. An agent assertion alone never marks a finding verified.

A finding carries: project and run ID, target revision when known, route and method, tested identities and roles, expected boundary, actual result, sanitized evidence references, reproduction artifact, severity rationale, verifier outcome, limitations, timestamps and tool versions.

Evidence

  • The runner uploads files to POST /api/assessments/:id/evidence before it reports terminal status. Text is redacted, every file is SHA-256 hashed, and a manifest checksum is stored with the bundle.
  • Production uses the private EVIDENCE R2 binding. Without it, evidence falls back to database objects. Both enforce workspace ownership and the workspace's retention window.
  • Evidence stays downloadable after the sandbox is deleted, and is removed by the retention cron with the assessment.

Retests and regression checks

  • POST /api/v1/assessments/:id/findings/:findingId/retest (assessments:create) checks the patched target: identity A must still get 200 while identity B and anonymous access are denied. The finding records retestStatus as fixed, still_vulnerable or inconclusive.
  • Every finding has a downloadable regression check at /app/assessments/:id/findings/:findingId/regression, and the API equivalent under /api/v1.
  • POST /api/v1/assessments/:id/findings/:findingId/verify runs the fresh verification.

Reading findings over the API

curl -s https://app.offense.sh/api/v1/assessments/$ID/findings \
  -H "Authorization: Bearer $OFFENSE_API_KEY"      # findings:read
curl -s https://app.offense.sh/api/v1/assessments/$ID/evidence \
  -H "Authorization: Bearer $OFFENSE_API_KEY"
curl -s https://app.offense.sh/api/v1/assessments/$ID/usage \
  -H "Authorization: Bearer $OFFENSE_API_KEY"

POST /api/v1/assessments/:id/cancel (assessments:cancel) stops the work and refunds the unused budget.

Budgets and cancellation

Budget is reserved before model calls. Global concurrency is capped by MAX_GLOBAL_ACTIVE (default 20) in addition to the per-workspace MAX_ACTIVE_SANDBOXES (default 10). Browser and API cancellation both call the same stop-and-settle flow; reservation, usage, settlement and refund are atomic and duplicate-safe.

Partial coverage, honestly

These produce an actionable setup error or a coverage gap, not a fake pass:

  • unsupported SSO, MFA or CAPTCHA without supplied session state,
  • unreachable or unverifiable targets,
  • an investigator that cannot reach the target through the egress proxy,
  • a run that exhausts its budget mid-flight.

A completed assessment may have partial coverage. There is no "all tests passed" claim anywhere in the product, because the product cannot prove it.