Skip to main content

Ingesting Alerts from Your Tools

Send alerts into Phoenix Alerts from Sentry, Datadog, or any tool via webhook.

D
Written by Dave Rochwerger

Phoenix Alerts can receive alerts from any monitoring tool. When an alert arrives, Phoenix Alerts pages the on-call for the service you choose, deduplicates repeat firings, and resolves automatically when the source recovers.

Before you start, create an ingestion key in Configuration → API Keys and copy it. See API Keys.

⚙️ How it works

Every tool sends alerts to the same endpoint, authenticated with your ingestion key:

POST https://api.phoenixalerts.com/v1/alerts
x-api-key: pi_your_key_here
  • Phoenix Alerts uses alert_key to deduplicate. Sending the same alert_key again updates the existing alert instead of opening a new one.

  • To close an alert, send the same alert_key with "event_action": "resolve". Resolving an alert that is already closed, or never existed, returns a 200, so retries are safe.

  • If you send the same alert_key with a higher severity, the existing alert is upgraded and re-paged when your escalation policy is severity-sensitive. It is not duplicated.

Sentry Sentry

Phoenix Alerts has a Sentry app that connects in a few clicks.

One-time install (each customer):

  1. In your Phoenix Alerts admin panel, go to Configuration → API Keys and create an ingestion key. Copy it.

  2. Open the install link your Phoenix Alerts contact shared with you (it looks like https://sentry.io/sentry-apps/phoenix-alerts/external-install/).

  3. Approve the OAuth consent screen. Sentry sends you back to Phoenix Alerts.

  4. On the "Connect Phoenix Alerts" screen, paste the ingestion key and submit.

Create an alert rule (per Sentry project):

  1. In Sentry, go to Alerts → Create Alert Rule.

  2. Add the action Send a notification via Phoenix Alerts.

  3. Pick a service. The dropdown lists services from your Phoenix Alerts tenant.

  4. Pick a severity (SEV1–SEV5).

  5. Save.

What happens:

  • A Sentry issue fires, and Phoenix Alerts pages the on-call for the chosen service at the chosen severity.

  • The Sentry issue resolves, and Phoenix Alerts auto-resolves the matching alert.

  • You can also resolve from the Phoenix Alerts side, and it pushes the resolve back to Sentry.

Datadog

One webhook works for all your services. Each monitor tells Phoenix Alerts which service to page through a tag, so you set the webhook up once and reuse it across your whole org.

One-time setup:

  1. In Phoenix Alerts, go to Configuration → API Keys and create an ingestion key.

  2. In Datadog, go to Integrations → Webhooks → New Webhook and set the URL, custom headers, and payload below.

URL

https://api.phoenixalerts.com/v1/alerts

Custom Headers

{"x-api-key": "pi_YOUR_KEY_HERE"}

Payload

{
"alert_key": "$ALERT_ID",
"summary": "$EVENT_TITLE",
"source": "datadog",
"severity_level": {{#is_alert}}1{{/is_alert}}{{#is_warning}}2{{/is_warning}},
"service_key": "$TAGS[service]",
"external_reference": "$LINK",
"event_action": {{#is_recovery}}"resolve"{{/is_recovery}}{{^is_recovery}}"trigger"{{/is_recovery}}
}

Per-monitor setup: tag each monitor with service:<your-pa-service-key>. The tag is either inherited automatically when the monitored resource already carries it, or added on the monitor itself under the Tags field. That is the only per-monitor work. Then reference the webhook in the monitor message with @webhook-phoenix-alerts.

If a monitor is missing the service: tag, the alert posts with an empty service_key and falls back to your tenant's default routing. The page still goes out, just not to a specific service's on-call. These show up in the unrouted-alerts view, and you fix them by adding the tag to the monitor.

Optional escape hatch: to route one specific monitor differently, tag it with phoenix-alerts-service:<key> and add a second variable, $TAGS[phoenix-alerts-service], to the payload, using that when it is set and falling back to the service: tag otherwise. This is rarely needed.

What happens:

  • A monitor crosses into Warn, and Phoenix Alerts pages at SEV2.

  • The monitor escalates from Warn to Alert, and Phoenix Alerts bumps the same alert to SEV1 and re-pages if your escalation policy is severity-sensitive. The alert is not duplicated.

  • The monitor recovers, and Phoenix Alerts auto-resolves the alert.

  • The monitor flaps (Warn → OK → Warn), and Phoenix Alerts resolves and re-opens accordingly.

🔌 Any other tool (generic webhook)

Use the same POST https://api.phoenixalerts.com/v1/alerts with the header x-api-key: pi_…. The payload fields are:

Field

Required

Notes

alert_key

Yes

Unique key for the firing condition. Used to deduplicate and to match resolves.

summary

Yes

Short headline shown in the page.

severity_level

No

1 (highest) to 5 (lowest).

service_key

One of these

Phoenix Alerts service slug.

team_key

One of these

Phoenix Alerts team slug.

source

No

Free-text tag for filtering.

external_reference

No

URL back to the source, rendered as a link.

event_action

No

"trigger" (default) or "resolve". Send "resolve" with the same alert_key to close an alert.

Severity escalation: if you send the same alert_key again with a higher severity, the existing alert is upgraded, with no duplicate page.

Resolution: send the same alert_key with "event_action": "resolve" to close it. If the alert is already closed or never existed, you get a 200 (idempotent).

Did this answer your question?