# @izri/repo-analyzer

> Hybrid repository analysis tooling that pairs a traditional plugin pipeline with AI-ready context preparation.

## 📋 Overview

The Repo Analyzer package provides two analyzer implementations that share an intake and plugin system:

- **EnhancedRepositoryAnalyzer** – orchestrates the modern three-phase workflow (intake → traditional analysis → AI context preparation) and powers future-facing features.
- **RepositoryAnalyzer** – retains the legacy analysis contract for downstream consumers that still expect the original result shape.

> ℹ️ **Current server usage**: the tRPC analysis router still instantiates `RepositoryAnalyzer`. Migrating to the enhanced analyzer only requires swapping the class and updating TypeScript expectations; persisted JSONB payloads remain compatible.

### When to reach for this package

- Platform engineers who run cross-repository scans or feed repos into the AI service.
- Contributors building or maintaining analysis plugins.
- Teams preparing to adopt the enhanced analyzer in API layers (for example, tRPC routers).

## 📦 Package structure (high level)

- **Entry points**: `src/index.ts` exports analyzers, pipelines, and plugins for consumers.
- **Analyzer implementations**: `src/analyzer.ts` (legacy) and `src/enhanced-analyzer.ts` (hybrid workflow).
- **Intake pipeline**: `src/intake/repository-intake.ts` handles cloning, indexing, and structure detection.
- **Analysis pipeline**: `src/analysis/traditional-pipeline.ts` wires default plugins and telemetry summarization.
- **Built-in plugins**: `src/analysis/traditional-pipeline.ts` defines dependency, code-structure, test, and configuration analyzers.
- **Type definitions**: `src/types/intake.ts` (enhanced pipeline) and `src/types.ts` (legacy compatibility).

## 🧭 Analysis workflow at a glance

1. **Repository Intake** – clones a repository into a managed temp directory, gathers git metadata, indexes files, and detects structure.
2. **Traditional Analysis Pipeline** – runs the registered plugins sequentially, collecting metrics, issues, and dependency metadata.
3. **AI Context Preparation** (enhanced analyzer) – condenses analysis output into an AI-optimized payload for downstream consumers.

*Diagram description*: Enhanced analyzer coordinates Repository Intake → Traditional Analysis Pipeline → AI Context preparation, with optional additional plugins feeding into the pipeline stage.

## 🚀 Getting started: common workflows

| Scenario | Inputs | Result | Notes |
| --- | --- | --- | --- |
| Baseline repository scan | Public repo URL with defaults | Temp clone, built-in plugins executed, `ComprehensiveAnalysis` returned, cleanup performed | Use when you need a quick health snapshot.
| Private repository | Repo URL plus `githubToken`, optional branch | Same as baseline with authenticated clone | Provide a token scoped for read access only; logs avoid printing token values.
| Custom plugin instrumentation | Repo URL and array of additional plugin instances | Built-in plugins plus your custom plugin run in order of registration | Plugins must implement the `AnalysisPlugin` interface; see "Extending with custom plugins".
| AI-ready analysis | Repo URL, optional intake overrides | Returns the AI-optimized context structure in addition to traditional metrics | Ideal for feeding the AI service or other LLM tooling.

### Expectations for callers

- The enhanced analyzer invokes `RepositoryIntake.cleanup` automatically after each run (success or failure).
- When you instantiate `RepositoryIntake` directly, you are responsible for calling `cleanup` once you finish consuming the context.
- Plugin failures are logged and skipped; the pipeline continues with remaining plugins.

## 🔍 Phase 1 – Repository intake

The intake pipeline (`RepositoryIntake`) is the authoritative source for repository metadata.

### Intake options

| Option | Default | Purpose | Typical adjustments |
| --- | --- | --- | --- |
| `branch` | `main` | Selects the branch to clone. | Target release branches or feature branches for focused scans. |
| `githubToken` | _none_ | Enables authenticated clones for private repositories. | Use tokens with read-only scopes; never store them in configs. |
| `maxFiles` | `10000` | Caps file indexing to protect memory usage. | Lower for very large repos to speed up analysis; align with AI token limits. |
| `maxFileSizeBytes` | `1 MB` | Skips overly large files (per file). | Increase when binary configs are small but exceed default size. |
| `includeHashes` | `true` | Computes SHA1 hashes for indexed files. | Disable to speed up large analyses where hashing is unnecessary. |
| `depth` | `1` | Performs a shallow clone. | Raise to capture deeper commit history, at the cost of clone time. |

### Intake output summary

| Structure | Description | Downstream usage |
| --- | --- | --- |
| `repoDir` | Absolute path to the temporary clone. | Input to the traditional pipeline and custom plugins. |
| `gitMetadata` | Branch list, current commit, remotes, top contributors, recent commits. | Telemetry, migration audits, reporting in AI context. |
| `fileIndex` | Indexed files with language, size, hashes, aggregate totals, language stats. | Plugin heuristics, AI summaries, size budgeting. |
| `projectStructure` | Detected directories, package managers, frameworks, testing tools, root files. | Plugin defaults, enhanced analyzer summaries, migration planning. |
| `processedAt` | Timestamp for intake completion. | Inclusion in final analysis payloads. |

### Operational notes

- Intake writes to OS-managed temp directories with a unique prefix (`izri-intake-*`).
- Skips heavyweight directories (`node_modules`, build outputs, caches) by default.
- Hashing is best-effort; unreadable files are skipped without failing the run.
- Framework and tool detection relies primarily on manifest files (package.json, requirements.txt, etc.).

### Troubleshooting intake

| Symptom | Likely cause | Resolution |
| --- | --- | --- |
| "Permission denied" during clone | Missing or expired token | Regenerate a personal access token with repo read scope and pass as `githubToken`. |
| Analysis stops early with very large repos | `maxFiles` or `maxFileSizeBytes` cap reached | Increase the limits deliberately or filter the repository to the sub-directory of interest. |
| Frameworks list is empty for known frameworks | Manifest lacks explicit dependency entries | Verify that package manifests list the framework or add custom plugin detection logic. |

## 🧪 Phase 2 – Traditional analysis pipeline

`TraditionalAnalysisPipeline` registers four built-in plugins and processes any additional plugins you provide.

### Built-in plugins

| Plugin ID | Purpose | Key metrics & metadata | Typical consumers |
| --- | --- | --- | --- |
| `dependency-analysis` | Enumerates dependencies across ecosystems, flags potential issues. | `metrics.totalDependencies`, `metrics.devDependencies`, `metadata.dependencies[]`. | Security reviews, dependency health dashboards, AI dependency briefs. |
| `code-structure` | Evaluates project layout, naming conventions, directory depth. | `metrics.hasSourceDirectory`, `metrics.namingConventions`, `metrics.maxDirectoryDepth`. | Architecture reviews, refactoring candidates, onboarding guides. |
| `test-analysis` | Locates test files, estimates coverage heuristics, inspects organization. | `metrics.testFileCount`, `metrics.testToSourceRatio`, `metadata.testFiles`, `metrics.testsInSeparateDirectory`. | QA readiness assessments, CI quality checks, AI recommendations. |
| `configuration-analysis` | Captures configuration files, environment setup, CI/CD signals. | `metrics.configurationFiles`, `metrics.hasLinting`, `metrics.hasEnvironmentConfig`, `metrics.hasDockerfile`. | Deployment readiness, compliance audits, AI environment summaries. |

### Plugin lifecycle

1. Pipeline registers default plugins, then additional plugins in provided order.
2. `analyze(context)` is called sequentially for each plugin.
3. Each plugin returns metrics, metadata, and issues; failures are caught and logged while remaining plugins continue.
4. The pipeline aggregates:
   - `results`: array of individual plugin outputs (with `pluginId`).
   - `dependencies`: flattened list from dependency analysis metadata.
   - `summary`: repository-wide totals (files, sizes, languages, frameworks, testing frameworks, issue counts).

### Extending with custom plugins

- Implement the `AnalysisPlugin` interface (`id`, `name`, `description`, `supportedExtensions`, `analyze`).
- Prefer deterministic output; structure your `metrics`, `metadata`, and `issues` objects for stable downstream consumption.
- Avoid reading huge files into memory; leverage the intake `fileIndex` to pre-filter paths.
- Surface actionable issues (severity, message, optional file/line metadata) to integrate cleanly with enhanced analyzer recommendations.
- Register the plugin via `new EnhancedRepositoryAnalyzer([myPlugin])` or call `registerPlugin` on an existing analyzer instance.

## 🧠 Enhanced analyzer APIs

The enhanced analyzer exposes a concise public surface:

| Method | Description | Key parameters | Returns |
| --- | --- | --- | --- |
| `analyzeRepository(repoUrl, options?)` | Executes intake, traditional analysis, and returns the comprehensive payload. | `repoUrl` (string), `options` (subset of intake options). | `ComprehensiveAnalysis` containing `context`, `results`, `dependencies`, `summary`, `completedAt`. |
| `analyzeForAI(repoUrl, options?)` | Runs `analyzeRepository`, then converts the output into an AI-optimized structure. | Same as above. | `AIReadyContext` with repository, project, codebase, testing, dependencies, issues, recommendations. |
| `registerPlugin(plugin)` | Adds a plugin to the pipeline for subsequent runs. | `plugin` implementing `AnalysisPlugin`. | `void` (mutates analyzer instance). |
| `getPlugins()` | Lists registered plugins in execution order. | _none_ | `AnalysisPlugin[]` including defaults and custom entries. |

### ComprehensiveAnalysis essentials

- `context.repoDir` is cleaned automatically once `analyzeRepository` completes.
- `summary` mirrors key telemetry (file counts, language distribution, frameworks, testing frameworks, issue counts).
- `results[n].issues` aggregates plugin-reported issues with severity tags (`error`, `warning`, `info`).
- Optional fields (`security`, `coverage`, `quality`) are reserved for future plugins; current built-ins populate `dependencies`, `results`, and summary metrics.

## 🤖 AI-ready context preparation

`analyzeForAI` converts the comprehensive analysis into a structure designed for LLM consumers:

- **Repository**: branch, commit hash, source URL.
- **Project**: type, frameworks, testing frameworks, build tools, package managers (derived from `projectStructure`).
- **Codebase**: directories, root files, files grouped by language, entry points, detected API endpoints.
- **Testing**: presence of tests, files, organization (separate vs. co-located), ratio to source files.
- **Dependencies**: production vs. development counts, major frameworks, testing libraries.
- **Issues**: totals, grouped by category and severity, up to 10 critical highlights.
- **Recommendations**: curated suggestions (e.g., add linting, introduce testing) inferred from summary and plugin metrics.

Call `analyzeForAI` when feeding the FastAPI AI service or any feature that needs condensed context. See [`docs/packages/ai-service.md`](./ai-service.md) for how the AI pipeline expects this payload.

Example recommendations you might observe:

- “Consider adding a testing framework like Jest, Vitest, or PyTest.”
- “Organize source files under a dedicated directory (src/, lib/, etc.).”
- “Add ESLint configuration to improve code consistency.”
- “Address critical issues reported by the plugin pipeline before generating AI responses.”

## 🔄 Migration: RepositoryAnalyzer → EnhancedRepositoryAnalyzer

1. **Swap imports**: replace `RepositoryAnalyzer` references with `EnhancedRepositoryAnalyzer`.
2. **Update instantiation**: optional plugins are passed into the constructor (`new EnhancedRepositoryAnalyzer([myPlugin])`).
3. **Adjust TypeScript types**: expect `ComprehensiveAnalysis` instead of the legacy summary format; update router contracts accordingly.
4. **Handle telemetry**: align any logging or analytics with the new `summary` structure (language counts, framework arrays, issue counts).
5. **Run integration tests**: verify that cleanup occurs as expected and downstream persistence still accepts JSONB payloads.
6. **Coordinate with API consumers**: tRPC router updates should be reflected in the documentation within `docs/packages/trpc-package.md`.

No database schema changes are required because both analyzers emit JSON-compatible payloads.

## ⚙️ Operational guidelines

### Resource planning

- Clone depth of 1 keeps network usage minimal; increase only if historical data is essential.
- Large repositories can generate sizable `fileIndex` objects—adjust `maxFiles` and `maxFileSizeBytes` to stay within process memory limits.
- The enhanced analyzer logs progress for each phase (intake, pipeline, cleanup) to aid observability.

### Security & cleanup

- GitHub tokens are embedded into the clone URL only for the duration of the clone operation.
- Temporary directories are removed automatically in both success and failure paths for the enhanced analyzer.
- Custom plugins should avoid logging file contents or secrets; rely on metadata and hashes when reporting issues.

### Telemetry integration

- Pipeline summary exposes issue counts and language distribution; forward these metrics to monitoring systems as needed.
- Plugin-specific `metrics` objects can be enriched to support dashboards (for example, dependency counts, test ratios).
- Enhanced analyzer console logs use emoji markers (`🔄`, `✅`, `🧹`)—redirect or wrap logging for production contexts if necessary.

## 🧱 Troubleshooting & FAQs

| Question | Guidance |
| --- | --- |
| Why are dependencies missing from `dependencies`? | Only the dependency plugin contributes to this list. Ensure the plugin remains registered and that manifests are accessible. |
| How do I skip default plugins? | Instantiate `TraditionalAnalysisPipeline` directly and register only the plugins you need, or fork the class; the enhanced analyzer always registers the defaults. |
| Can I analyze monorepos with multiple package managers? | Yes. Intake collects root manifests, and the dependency plugin inspects additional files (`package.json`, `requirements.txt`, lockfiles). Consider custom plugins for specialized build systems. |
| What if a plugin throws an error? | The pipeline catches the error, logs it, and continues. Investigate plugin logs to resolve the root cause. |

## 🔗 Related documentation

- [`docs/packages/trpc-package.md`](./trpc-package.md) – how analysis results flow into API routes.
- [`docs/packages/database-package.md`](./database-package.md) – storage expectations for analysis payloads (JSONB fields).
- [`docs/packages/ai-service.md`](./ai-service.md) – consuming the AI-ready context in the FastAPI service.
- Repository source under `packages/repo-analyzer/src` for implementation references.

Focus future edits on maintaining accuracy with the underlying TypeScript modules; avoid reintroducing large code blocks so the documentation stays high-signal and easy to maintain.