E2E Testing Workflow — Developer Guide

This guide explains how the automated E2E testing pipeline works from your first push to receiving results.

How It Works (End-to-End)

Developer pushes to GitHub
        │
        ▼
GitHub sends push event → POST /api/webhooks/github/:projectId
        │
        ▼
Signature verified (HMAC-SHA256)
        │
        ▼
Trigger conditions checked:
  - Branch in allowed list?
  - Changed files match patterns?
  - Dedup: same commit already running?
        │
        ▼
Test run record created (status: PENDING)
GitHub commit status posted: "pending"
        │
        ▼
DockerPlaywrightExecutor runs tests
in official Playwright Docker container
        │
        ▼
Results collected by ResultCollector
Report formatted by ReportFormatter
        │
        ▼
Test run record updated (PASSED/FAILED/ERROR)
GitHub commit status updated: success/failure/error

Step 1: Connect Your Repository

1. Log in and navigate to your project
2. Go to Settings → Webhooks
3. Call webhooks.configure with your filtering preferences:
   • branches: which branches trigger tests (empty = all)
   • filePatterns: glob patterns for relevant files (empty = any change)
4. Copy the webhook URL and secret — the secret is shown only once
5. In your GitHub repo: Settings → Webhooks → Add webhook
   • Payload URL: the URL from step 4
   • Content type: application/json
   • Secret: the secret from step 4
   • Events: select "Pushes" (and optionally "Pull requests")

Step 2: Write Your Playwright Tests

Tests run inside a Docker container. Your test files must be compatible with the official Playwright image (mcr.microsoft.com/playwright:v1.42.0-jammy).

Structure your tests in a directory, e.g.:

tests/
  e2e/
    login.spec.ts
    checkout.spec.ts
  playwright.config.ts

The executor mounts this directory as read-only at /tests inside the container.

Step 3: Push and Watch

1. Push a commit to a configured branch
2. GitHub sends a webhook event to the platform
3. The platform verifies the signature and checks trigger conditions
4. If conditions pass, a test run is created and tests execute in Docker
5. When complete, your commit gets a GitHub status check (green ✓ or red ✗)

Step 4: Get AI Test Suggestions

The platform can analyze your repo and suggest additional tests using AI.

1. The repo-analyzer package scans your codebase
2. The analysis is sent to the AI service (POST /suggest-tests)
3. Claude or GPT-4 returns suggestions: file paths, test types, rationale, and example code

Access suggestions via the analysis.suggestTests tRPC procedure.

Checking Test Run Status

Use testRuns.list or testRuns.get (tRPC) to view:

• status: PENDING → PASSED / FAILED / ERROR
• passedTests, failedTests, totalTests
• commitSha: which commit triggered this run
• executorType: always docker for automated runs
• results: full structured report (JSON)
• error: error message if status is ERROR

GitHub Commit Status Checks

When a GitHub token is configured in your project settings, the platform posts commit status checks:

Deduplication and Debouncing

• Dedup: If you push the same commit SHA twice (e.g. force-push), only one test run is created per hour
• Debounce: Rapid pushes to the same branch are coalesced — only the latest commit triggers a run (configurable delay)

These prevent wasted test runs on noisy branches.