superagent_

// use cases

Validate dependency updates end to end with Devin

[ view markdown ]

Publish policy-cleared vulnerability updates, trigger Devin from the pull request webhook, and test the running application before merge.

Make every vulnerability update prove that the application still works. Superagent generates and evaluates the dependency change, publishes only a policy-cleared pull request, and sends dependency_update.published to Devin. Devin checks out that pull request, starts the application, runs its integration tests, and returns the result to the pull request.

Prerequisites

  • A repository connected through the Superagent Security GitHub App
  • A Devin workspace connected to the repository
  • Application setup, environment variables, and test commands configured in Devin
  • A Devin Automation with an incoming webhook trigger
  • A repository reviewer for the final merge decision

1. Enable vulnerability updates

Open Repository, select the repository, and find Secure Dependency Updates in its settings.

  1. Turn on Secure Dependency Updates.
  2. Set Scope to Vulnerabilities.
  3. Choose a daily, weekly, or monthly schedule.
  4. Keep Supply Chain Scan enabled so the published pull request is checked again before merge.

Repository settings with vulnerability-scoped Secure Dependency Updates enabled on a weekly schedule

On each scheduled run, Superagent:

  1. Reads open Dependabot alerts or resolves vulnerable lockfile versions against the GitHub Advisory Database.
  2. Generates the manifest and lockfile change with pinned Dependabot tooling in an isolated sandbox.
  3. Evaluates the complete generated dependency delta.
  4. Withholds proposals that violate package policy.
  5. Publishes an approved proposal as a pull request.

2. Create the Devin Automation

In Devin, create an Automation with:

  • An incoming Webhook trigger
  • A Start session action
  • The repository and application environment already configured
  • Permission to read and update the dependency pull request
  • No permission to merge without the normal repository controls

Copy the Automation webhook URL and its one-time webhook secret.

Use instructions like these for the Devin session:

Validate the Secure Dependency Update pull request in this webhook payload.

1. Require event type `dependency_update.published`.
2. Read the repository name and pull request URL from `data.object`.
3. Check out the pull request head without changing the selected dependency
   version or regenerating unrelated lockfile entries.
4. Install the exact committed lockfile with the repository package manager.
5. Build and start the application using the existing Devin environment.
6. Run the repository's unit, integration, and end-to-end tests.
7. Exercise the critical user flows and health checks configured for this app.
8. If the bump causes a compatibility failure, diagnose it and apply only the
   smallest necessary compatibility fix on the same pull request branch.
9. Re-run every failed check after a fix.
10. Comment on the pull request with the commands run, results, and any
    remaining blocker.

Do not merge the pull request. Do not use production credentials or include
secret values in comments or logs.

3. Send published updates to Devin

Open Webhooks in Superagent and create a target:

  1. Enter the Devin Automation webhook URL.
  2. Add X-Webhook-Secret as a custom header with Devin's webhook secret.
  3. Subscribe only to dependency_update.published.
  4. Open Sources, expand Repositories, and select the exact repository.
  5. Save the webhook and send a test event.

Devin accepts the event body directly and appends it to the Start session prompt. The exact Repository source prevents dependency updates from unrelated repositories from starting this application test environment.

dependency_update.published fires once after a proposal passes policy and reaches GitHub as a pull request. Withheld, superseded, and failed proposals do not trigger Devin.

4. Test the running application

When the webhook starts a session, Devin should:

  1. Open the repository and pull request from the event payload.
  2. Install the exact dependency graph committed by Superagent.
  3. Build and start the application in its configured environment.
  4. Run the full integration suite and critical browser or API flows.
  5. Add a focused compatibility fix only when the update caused the failure.
  6. Post the final result to the existing pull request.

Keep application secrets in Devin's environment. Use test services and test accounts for integration flows, and keep production deployment outside the Automation.

5. Keep merge approval human

Require the normal Supply Chain Scan, CI, and code review checks in branch protection. Devin validates whether the application still works; a repository reviewer decides whether and when the update merges.

Next steps