Test Execution

Docker-based test execution engine for E2E testing MVP

📋 Overview

Test Execution enables running AI-generated E2E tests in isolated Docker environments, collecting results, computing coverage, and storing test run history. Built specifically for Playwright-based E2E testing with staging URL support.

🎯 Current Status

Status: � IMPLEMENTATION IN PROGRESS (Phase 1)

Test execution infrastructure is being built with Docker-based isolation, Playwright support, and staging environment integration. Ready for E2E testing MVP.

🏗️ Architecture

Database Schema

testRuns table (Already implemented):

{
  id: string
  projectId: string
  userId: string
  type: 'unit' | 'integration' | 'e2e' | 'all'
  status: 'PENDING' | 'RUNNING' | 'COMPLETED' | 'FAILED' | 'CANCELLED'
  startedAt: Date?
  completedAt: Date?
  duration: number?        // milliseconds
  totalTests: number
  passedTests: number
  failedTests: number
  skippedTests: number
  coverage: number?        // percentage
  results: object          // JSONB with detailed results
  logs: string?            // Execution logs
  error: string?           // Error message if failed
  createdAt: Date
}

🔌 API Reference (Planned)

Run Tests

const run = await trpc.tests.runTests.mutate({
  projectId: 'proj_123',
  type: 'unit',
  files: ['src/**/*.test.ts'], // Optional: specific files
  options: {
    coverage: true,
    bail: false,
    timeout: 30000,
    environment: 'node'
  }
})

// Response
{
  runId: 'run_456',
  status: 'RUNNING',
  message: 'Tests started'
}

Get Test Run Status

const status = await trpc.tests.getRunStatus.query({
  runId: 'run_456'
})

// Response
{
  run: {
    id: 'run_456',
    status: 'RUNNING',
    totalTests: 50,
    passedTests: 42,
    failedTests: 3,
    skippedTests: 5,
    duration: 15000,
    coverage: 85.5
  }
}

List Test Runs

const runs = await trpc.tests.listRuns.query({
  projectId: 'proj_123',
  limit: 20
})

🚀 Execution Engine (Planned)

Docker-based Execution

class TestExecutor {
  async runTests(project: Project, options: RunOptions) {
    // 1. Create isolated Docker container
    const container = await this.createContainer(project)
    
    // 2. Clone repository
    await this.cloneRepo(container, project.repository)
    
    // 3. Install dependencies
    await this.installDependencies(container, project)
    
    // 4. Run tests
    const results = await this.executeTests(container, options)
    
    // 5. Collect coverage
    const coverage = await this.collectCoverage(container)
    
    // 6. Clean up
    await container.remove()
    
    return { results, coverage }
  }
}

Test Runner Configuration

interface TestRunnerConfig {
  framework: 'vitest' | 'jest' | 'playwright' | 'pytest'
  command: string
  configFile: string
  coverageDirectory: string
  reportFormat: 'json' | 'xml' | 'html'
}

📊 Results Format

Test Result Structure

interface TestRunResults {
  summary: {
    totalTests: number
    passedTests: number
    failedTests: number
    skippedTests: number
    duration: number
    coverage?: {
      lines: number
      functions: number
      branches: number
      statements: number
    }
  }
  
  suites: Array<{
    file: string
    duration: number
    tests: Array<{
      name: string
      status: 'passed' | 'failed' | 'skipped'
      duration: number
      error?: {
        message: string
        stack: string
      }
    }>
  }>
  
  coverage?: {
    coveredFiles: number
    totalFiles: number
    uncoveredFiles: string[]
  }
}

🎨 Frontend Implementation (Planned)

Run Tests Button

export function RunTestsButton({ projectId }: Props) {
  const runTests = trpc.tests.runTests.useMutation({
    onSuccess: (data) => {
      navigate(`/dashboard/projects/${projectId}/runs/${data.runId}`)
    }
  })

  return (
    <Button onClick={() => runTests.mutate({ projectId, type: 'all' })}>
      <PlayIcon /> Run All Tests
    </Button>
  )
}

Test Run Status Display

export function TestRunStatus({ runId }: Props) {
  const { data } = trpc.tests.getRunStatus.useQuery(
    { runId },
    { refetchInterval: 2000 } // Poll every 2s while running
  )

  if (!data) return <Skeleton />

  return (
    <Card>
      <CardHeader>
        <div className="flex items-center justify-between">
          <CardTitle>Test Run #{runId}</CardTitle>
          <StatusBadge status={data.run.status} />
        </div>
      </CardHeader>
      <CardContent>
        <div className="grid gap-4 md:grid-cols-4">
          <StatCard label="Total" value={data.run.totalTests} />
          <StatCard label="Passed" value={data.run.passedTests} color="green" />
          <StatCard label="Failed" value={data.run.failedTests} color="red" />
          <StatCard label="Skipped" value={data.run.skippedTests} color="gray" />
        </div>
        
        {data.run.coverage && (
          <div className="mt-4">
            <div className="flex justify-between text-sm mb-2">
              <span>Coverage</span>
              <span>{data.run.coverage.toFixed(1)}%</span>
            </div>
            <Progress value={data.run.coverage} />
          </div>
        )}
      </CardContent>
    </Card>
  )
}

🔧 Configuration

Environment Variables

# Test execution
TEST_TIMEOUT=30000
TEST_RETRIES=2
MAX_PARALLEL_RUNS=5

# Docker
DOCKER_IMAGE=node:20-alpine
DOCKER_MEMORY_LIMIT=2g
DOCKER_CPU_LIMIT=2

✅ Testing

Simulated Test Run

describe('Test Execution', () => {
  it('creates a test run record', async () => {
    const run = await createTestRun({
      projectId: 'proj_123',
      userId: 'user_123',
      type: 'unit'
    })
    
    expect(run.id).toBeDefined()
    expect(run.status).toBe('PENDING')
  })
})

🔗 Related Documentation

• Test Generation: Generating tests
• Test Reporting: Viewing results
• Projects: Project management

Next: Test Reporting →