Run a request on every new alert

A Schedule can fire on the alerts of your monitoring tools. You connect Sentry or Alertmanager to devboxes as an alert source, then set up a Schedule with a request such as "Find the cause of this alert and prepare a fix". Each new alert starts that request in its thread, and the work it starts uses one Blueprint in one Project. You decide what that work does with a fix: a Blueprint that publishes a pull request hands it on to your own automation, up to merging it after Review when the Project merges on its own.

You connect an alert source on the tool's page in Integrations, and set up its Schedule in Settings > Schedules. Chat can set up the Schedule, and the CLI and the API can do both. You need:

• Chat turned on, because each alert runs as a Chat request.
• An owner or admin to connect or disconnect a source. If they signed in more than 5 minutes ago, the dashboard asks them to sign in again. From the CLI, they sign in again with devboxes login, or the command fails with SESSION_NOT_FRESH.
• A Blueprint available in the Project.

Connect Sentry

devboxes takes Sentry's issue alerts through an internal integration, which signs every delivery with its Client Secret.

1. In Sentry, open Settings > Developer Settings, create an internal integration, and copy its Client Secret.

2. In devboxes, open Integrations > Sentry. Under Alert sources, enter a name your team recognizes and the Client Secret, then choose Connect. The source appears with its webhook URL.

   From the CLI, connect it with alerts.connect:

   {
     "organizationId": "<organization id>",
     "tool": "sentry",
     "name": "Sentry production",
     "clientSecret": "<client secret>"
   }

   devboxes invoke alerts.connect --input @sentry.json

   The result holds the source's sourceId and its webhookUrl.

3. In Sentry, paste the webhook URL into the integration's Webhook URL, turn on Alert Action, and save.

4. In each issue alert rule that should start work, add the action Send a notification via your integration.

devboxes refuses a delivery that the Client Secret doesn't sign, or that holds more than 2 MiB. Sentry's other webhooks, such as installation events, start nothing.

Connect Alertmanager

Alertmanager doesn't sign its webhooks, so devboxes makes a bearer token for the receiver to send.

1. Open Integrations > Alertmanager. Under Alert sources, enter a name your team recognizes, then choose Connect. The source appears with its webhook URL, and devboxes shows its bearer token.

   From the CLI, connect it with alerts.connect:

   {
     "organizationId": "<organization id>",
     "tool": "alertmanager",
     "name": "Production cluster"
   }

   devboxes invoke alerts.connect --input @alertmanager.json

   The result holds sourceId, webhookUrl and bearerToken.

   devboxes shows the bearer token only this once, so copy it before you leave the page. If you lose it, connect a new source and set up its Schedules again with the new source's id. Then disconnect the old source. Disconnecting pauses the old source's Schedules for good, because they can't resume, so delete them.

2. Add a devboxes receiver to your Alertmanager configuration. Then add a route to it as the first entry under your existing route.routes, matching the alerts that should start work, and keep a route to your default receiver after it. In this example, pager is the team's existing default receiver:

   route:
     receiver: pager
     routes:
       - receiver: devboxes
         matchers:
           - severity="page"
         continue: true
       # Your existing routes follow here.
       - receiver: pager
   
   receivers:
     - name: pager
       # Your existing pager configuration.
     - name: devboxes
       webhook_configs:
         - url: <webhook URL>
           http_config:
             authorization:
               credentials: <bearer token>

   continue: true lets an alert that matches the devboxes route go on to the routes after it. An alert that matches a child route never falls back to the parent's receiver, so the last route sends it to pager too. Without that route, these alerts would reach devboxes and no longer page anyone. authorization sends the token under the Bearer scheme, which is Alertmanager's default type.

   To check the result, run amtool config routes test --config.file=alertmanager.yml severity=page. It should print devboxes,pager.

devboxes refuses a delivery without that token, or one that holds more than 2 MiB.

Set up a Schedule on alerts

In Settings > Schedules, choose Add. Under Runs, choose For each new alert, then the Source, the Project and a Blueprint, as Set up a Schedule describes.

From the CLI, find the source's id with alerts.sources, then create the Schedule with alerts in place of cron and timezone:

{
  "organizationId": "<organization id>",
  "title": "Fix production alerts",
  "request": "Find the cause of this alert and prepare a fix.",
  "alerts": {
    "source": "<source id>",
    "projectId": "<project id>",
    "blueprint": "Prepare a fix"
  }
}

devboxes invoke schedules.create --input @schedule.json

Without conversationId, the Schedule posts in a new web conversation of yours, as Set up a Schedule from the CLI describes. With a chat thread's conversationId, that thread's channel has to be assigned to the Project, or the command fails with SCHEDULE_PROJECT_UNROUTED. A source that isn't connected fails with ALERT_SOURCE_NOT_FOUND. In Chat, devboxes sets up a Schedule on alerts only while the Chat policy grants alerts; otherwise it fails with ALERT_SOURCE_NOT_AUTHORIZED.

When a Schedule on alerts fires

For each delivery with new alerts, devboxes treats your request as a new message from you in the Schedule's thread. The message ends with each new alert's link and its text, quoted as words from outside that give devboxes no instructions:

A Sentry issue fires once, so a later alert of the same issue, including one after it regresses, starts nothing. An Alertmanager alert that repeat_interval sends again keeps its start time and starts nothing more; when it resolves and fires again, it starts again. A resolved alert starts nothing. A notification that groups several firing alerts starts one request for those that are new.

devboxes sees the work that the Schedule's earlier alerts started. When a new alert belongs to work that still runs, devboxes can send the alert to that work instead of starting it again, and it starts new work only for an alert that nothing running covers.

Each alert's work runs as you and is billed like any other Run. Managed work only starts while your Organization has money left, as How managed work is charged describes. To start fewer, narrow the alert rules or Alertmanager routes that reach devboxes.

What work an alert started may do

The work an alert starts, along with every reply, continuation and follow-up that comes from it, follows these rules, even when a person continues it from the CLI:

• It may post the code and files your request asks for, such as a patch or a reproduction script.
• It may start only the Blueprint the Schedule names, in the Schedule's Project. A different Blueprint fails with BLUEPRINT_NOT_AUTHORIZED, and a different Project with PROJECT_NOT_AUTHORIZED.
• It publishes a pull request when its Blueprint does and the Chat policy grants publication, and the Project's merging setting decides whether devboxes merges it after Review.
• It may use the integrations its Blueprint uses, such as Sentry's MCP server to read the alert's events, while the Chat policy allows them, under the Schedule's person: their chat account must be linked to an owner, admin, member or reviewer. Another connection is refused with ISSUE_WORK_FORBIDDEN, and in a Run with EXECUTION_AUTHORITY_REVOKED.
• It writes to no issue: issues.write fails with ISSUE_NOT_AUTHORIZED.
• It may follow up only on the work that this Schedule's alerts started. Another Session fails with SESSION_NOT_AUTHORIZED.
• It may read issues and reply in its thread, and post files only there. Any other command, such as schedules.create, fails with ISSUE_WORK_FORBIDDEN.
• Its replies start fresh: they don't read the thread's earlier conversation, and later replies in the thread don't read theirs or report their results.

schedules.run refuses a Schedule on alerts with SCHEDULE_FIRES_ON_ALERTS, because only a new alert fires it.

Manage alert sources

The tool's page in Integrations lists its sources under Alert sources, each with its webhook URL. An owner or admin disconnects a source there with Disconnect. From the CLI:

Any member can list sources. Only owners and admins connect and disconnect them. A Schedule whose source was disconnected can't resume: schedules.resume fails with ALERT_SOURCE_NOT_FOUND. Connect the tool again and set up the Schedule again. Work that earlier alerts started goes on.