Abuse Monitoring Runbook

Use this page for BDT-317 on the M5 Mainnet Safety milestone.

The goal is to catch suspicious self-serve activity before it turns into support noise, webhook storms, or live-traffic abuse.

The thresholds below are starter values. Tune them to your production baseline before treating them as paging thresholds.

Detection sources

Use the built-in metrics and audit events together:

Metric: abuse_signal_total
Registration audit events: merchant.registered
Webhook audit events: payment_session.webhook.failed, payment_session.webhook.delivered, payment_session.webhook.skipped
Session diagnostics: merchant-safe payment_session.diagnostics output for stuck, mismatched, expired, and confirmation-lag cases

Alert catalog

Signal	Default severity	Starter threshold	Labels	What it means
Registration spike	P2, upgrade to P1 if live or cross-tenant	More than 10 successful merchant registrations in 10 minutes from one `source_ip_hash` or `agent_id`, or more than 5 failed registrations in 5 minutes	`abuse`, `onboarding`, `security`	A scripted agent or user is creating accounts faster than a normal onboarding flow.
Failed webhook flood	P1	More than 20 failed deliveries in 10 minutes for one merchant, or a failure rate above 50% for 15 minutes	`abuse`, `webhooks`, `security`	A callback target is failing hard enough to create a retry storm or hide real payment state changes.
Simulation abuse	P2, upgrade to P1 if non-demo or repeated	Any `simulate` call against a non-demo session, or more than 10 simulate calls in 10 minutes for one merchant/session	`abuse`, `sandbox`, `onboarding`	A tenant is using the sandbox path as a high-volume automation loop or probing for unsafe behavior.
Limit violation	P1	Any request that exceeds per-merchant or per-agent caps, or any attempt to reach mainnet while the account is still gated	`abuse`, `limits`, `security`	An actor is probing or bypassing the safety gates that keep agent-native onboarding bounded.

Metric labels

abuse_signal_total uses low-cardinality labels only:

Label	Example values
`signal`	`registration_spike`, `simulation_abuse`, `limit_violation`
`outcome`	`registered`, `rate_limited`, `invalid_webhook_url`, `blocked_non_demo`, `blocked_non_pending`, `mainnet_kyc_required`, `amount_limit_exceeded`, `daily_volume_limit_exceeded`
`environment`	`live`, `sandbox`, `demo`
`chain`	`solana`, `ethereum`
`network`	`testnet`, `mainnet`
`token`	`USDC`, `ETH`, `SOL`

Do not add merchant_id, raw IPs, session ids, or webhook URLs as metric labels.

Payload contract

Keep alert payloads small and redacted. Include identifiers, counters, hashes, and route context only.

Include

Field	Purpose
`alert_id`	Stable incident reference
`alert_type`	`registration_spike`, `failed_webhook_flood`, `simulation_abuse`, or `limit_violation`
`severity`	`P1`, `P2`, or `P3`
`merchant_id`	Merchant context
`merchant_name`	Human-readable merchant context
`agent_id`	Agent or automation context
`environment`	`sandbox` or `live`
`chain`	Chain involved, when relevant
`network`	`testnet` or `mainnet`
`route`	Affected API route or webhook path
`window_start` / `window_end`	Alert window
`count`	Observed event volume
`threshold`	Threshold that fired
`request_id`	Request correlation
`session_id`	Payment session correlation
`delivery_id`	Webhook delivery correlation
`source_ip_hash`	Source fingerprint without storing raw IPs in the alert
`user_agent_hash`	Client fingerprint without storing the raw string
`body_hash`	Digest of the redacted payload, if needed
`last_error_code`	Short failure reason, such as `timeout` or `5xx`
`redacted_fields`	Explicit list of omitted secret-bearing fields

Never include

X-API-Key values
webhook secrets
HMAC signature values
private keys
raw request or response bodies
access tokens
seed phrases
full unredacted wallet or payment secrets

Example payload

{
  "alert_id": "abt_01J...",
  "alert_type": "failed_webhook_flood",
  "severity": "P1",
  "merchant_id": "mer_123",
  "merchant_name": "Acme Test Co",
  "agent_id": "agt_456",
  "environment": "live",
  "network": "mainnet",
  "chain": "solana",
  "route": "POST /webhooks/plaidly",
  "window_start": "2026-06-14T12:00:00Z",
  "window_end": "2026-06-14T12:15:00Z",
  "count": 84,
  "threshold": 20,
  "request_id": "req_789",
  "delivery_id": "wd_012",
  "source_ip_hash": "sha256:4e5f...",
  "user_agent_hash": "sha256:9abc...",
  "body_hash": "sha256:def0...",
  "last_error_code": "timeout",
  "redacted_fields": ["X-API-Key", "X-Plaidly-Signature", "webhook_secret", "raw_body"]
}

First response runbook

Classify the blast radius.
- Is this sandbox or live?
- Is the spike isolated to one merchant_id or agent_id?
- Is the same request_id, session_id, or delivery_id repeating?
- Is the problem limited to one route, or are multiple routes involved?
Preserve evidence.
- Copy the alert payload into the incident record.
- Keep the IDs, hashes, and timestamps.
- Do not paste secrets or raw bodies into chat, tickets, or status updates.
Contain the signal that fired.
- Registration spike: pause the offending registration source, lower the onboarding rate limit if needed, and stop auto-provisioning for that tenant until the source is identified.
- Failed webhook flood: pause retries for the affected callback target, confirm whether the endpoint is timing out or returning 5xx, and disable the merchant webhook if the retry loop is burning capacity.
- Simulation abuse: disable simulate for that merchant or session, keep the account in sandbox-only mode, and block repeated demo completion loops.
- Limit violation: freeze the merchant or agent, block mainnet unlock, and reduce the cap to zero until the review is complete.
Decide whether this is abuse or a misconfiguration.
- Abuse usually shows repeated attempts, multiple failures from one source, or attempts against a gated route.
- Misconfiguration usually shows one tenant, one endpoint, and a clean technical failure such as timeout, bad URL, or signature mismatch.
Escalate with the right severity.
- Use P1 for live-traffic impact, cross-tenant activity, secret exposure risk, or a limit bypass attempt.
- Use P2 for single-tenant anomalies that are contained but still suspicious.
- Use P3 for sandbox-only noise that does not affect live merchants.
- Tag the follow-up issue with abuse, security, and the relevant surface label: onboarding, webhooks, sandbox, or limits.

Abuse Monitoring Runbook

On this page