@izri/repo-analyzer

Hybrid repository analysis tooling that pairs a traditional plugin pipeline with AI-ready context preparation.

📋 Overview

The Repo Analyzer package provides two analyzer implementations that share an intake and plugin system:

EnhancedRepositoryAnalyzer – orchestrates the modern three-phase workflow (intake → traditional analysis → AI context preparation) and powers future-facing features.
RepositoryAnalyzer – retains the legacy analysis contract for downstream consumers that still expect the original result shape.

ℹ️ Current server usage: the tRPC analysis router still instantiates RepositoryAnalyzer. Migrating to the enhanced analyzer only requires swapping the class and updating TypeScript expectations; persisted JSONB payloads remain compatible.

When to reach for this package

Platform engineers who run cross-repository scans or feed repos into the AI service.
Contributors building or maintaining analysis plugins.
Teams preparing to adopt the enhanced analyzer in API layers (for example, tRPC routers).

📦 Package structure (high level)

Entry points: src/index.ts exports analyzers, pipelines, and plugins for consumers.
Analyzer implementations: src/analyzer.ts (legacy) and src/enhanced-analyzer.ts (hybrid workflow).
Intake pipeline: src/intake/repository-intake.ts handles cloning, indexing, and structure detection.
Analysis pipeline: src/analysis/traditional-pipeline.ts wires default plugins and telemetry summarization.
Built-in plugins: src/analysis/traditional-pipeline.ts defines dependency, code-structure, test, and configuration analyzers.
Type definitions: src/types/intake.ts (enhanced pipeline) and src/types.ts (legacy compatibility).

🧭 Analysis workflow at a glance

Repository Intake – clones a repository into a managed temp directory, gathers git metadata, indexes files, and detects structure.
Traditional Analysis Pipeline – runs the registered plugins sequentially, collecting metrics, issues, and dependency metadata.
AI Context Preparation (enhanced analyzer) – condenses analysis output into an AI-optimized payload for downstream consumers.

Diagram description: Enhanced analyzer coordinates Repository Intake → Traditional Analysis Pipeline → AI Context preparation, with optional additional plugins feeding into the pipeline stage.

🚀 Getting started: common workflows

Scenario	Inputs	Result	Notes
Baseline repository scan	Public repo URL with defaults	Temp clone, built-in plugins executed, `ComprehensiveAnalysis` returned, cleanup performed	Use when you need a quick health snapshot.
Private repository	Repo URL plus `githubToken`, optional branch	Same as baseline with authenticated clone	Provide a token scoped for read access only; logs avoid printing token values.
Custom plugin instrumentation	Repo URL and array of additional plugin instances	Built-in plugins plus your custom plugin run in order of registration	Plugins must implement the `AnalysisPlugin` interface; see "Extending with custom plugins".
AI-ready analysis	Repo URL, optional intake overrides	Returns the AI-optimized context structure in addition to traditional metrics	Ideal for feeding the AI service or other LLM tooling.

Expectations for callers

The enhanced analyzer invokes RepositoryIntake.cleanup automatically after each run (success or failure).
When you instantiate RepositoryIntake directly, you are responsible for calling cleanup once you finish consuming the context.
Plugin failures are logged and skipped; the pipeline continues with remaining plugins.

🔍 Phase 1 – Repository intake

The intake pipeline (RepositoryIntake) is the authoritative source for repository metadata.

Intake options

Option	Default	Purpose	Typical adjustments
`branch`	`main`	Selects the branch to clone.	Target release branches or feature branches for focused scans.
`githubToken`	none	Enables authenticated clones for private repositories.	Use tokens with read-only scopes; never store them in configs.
`maxFiles`	`10000`	Caps file indexing to protect memory usage.	Lower for very large repos to speed up analysis; align with AI token limits.
`maxFileSizeBytes`	`1 MB`	Skips overly large files (per file).	Increase when binary configs are small but exceed default size.
`includeHashes`	`true`	Computes SHA1 hashes for indexed files.	Disable to speed up large analyses where hashing is unnecessary.
`depth`	`1`	Performs a shallow clone.	Raise to capture deeper commit history, at the cost of clone time.

Intake output summary

Structure	Description	Downstream usage
`repoDir`	Absolute path to the temporary clone.	Input to the traditional pipeline and custom plugins.
`gitMetadata`	Branch list, current commit, remotes, top contributors, recent commits.	Telemetry, migration audits, reporting in AI context.
`fileIndex`	Indexed files with language, size, hashes, aggregate totals, language stats.	Plugin heuristics, AI summaries, size budgeting.
`projectStructure`	Detected directories, package managers, frameworks, testing tools, root files.	Plugin defaults, enhanced analyzer summaries, migration planning.
`processedAt`	Timestamp for intake completion.	Inclusion in final analysis payloads.

Operational notes

Intake writes to OS-managed temp directories with a unique prefix (izri-intake-*).
Skips heavyweight directories (node_modules, build outputs, caches) by default.
Hashing is best-effort; unreadable files are skipped without failing the run.
Framework and tool detection relies primarily on manifest files (package.json, requirements.txt, etc.).

Troubleshooting intake

Symptom	Likely cause	Resolution
"Permission denied" during clone	Missing or expired token	Regenerate a personal access token with repo read scope and pass as `githubToken`.
Analysis stops early with very large repos	`maxFiles` or `maxFileSizeBytes` cap reached	Increase the limits deliberately or filter the repository to the sub-directory of interest.
Frameworks list is empty for known frameworks	Manifest lacks explicit dependency entries	Verify that package manifests list the framework or add custom plugin detection logic.

🧪 Phase 2 – Traditional analysis pipeline

TraditionalAnalysisPipeline registers four built-in plugins and processes any additional plugins you provide.

Built-in plugins

Plugin ID	Purpose	Key metrics & metadata	Typical consumers
`dependency-analysis`	Enumerates dependencies across ecosystems, flags potential issues.	`metrics.totalDependencies`, `metrics.devDependencies`, `metadata.dependencies[]`.	Security reviews, dependency health dashboards, AI dependency briefs.
`code-structure`	Evaluates project layout, naming conventions, directory depth.	`metrics.hasSourceDirectory`, `metrics.namingConventions`, `metrics.maxDirectoryDepth`.	Architecture reviews, refactoring candidates, onboarding guides.
`test-analysis`	Locates test files, estimates coverage heuristics, inspects organization.	`metrics.testFileCount`, `metrics.testToSourceRatio`, `metadata.testFiles`, `metrics.testsInSeparateDirectory`.	QA readiness assessments, CI quality checks, AI recommendations.
`configuration-analysis`	Captures configuration files, environment setup, CI/CD signals.	`metrics.configurationFiles`, `metrics.hasLinting`, `metrics.hasEnvironmentConfig`, `metrics.hasDockerfile`.	Deployment readiness, compliance audits, AI environment summaries.

Plugin lifecycle

Pipeline registers default plugins, then additional plugins in provided order.
analyze(context) is called sequentially for each plugin.
Each plugin returns metrics, metadata, and issues; failures are caught and logged while remaining plugins continue.
The pipeline aggregates:
- results: array of individual plugin outputs (with pluginId).
- dependencies: flattened list from dependency analysis metadata.
- summary: repository-wide totals (files, sizes, languages, frameworks, testing frameworks, issue counts).

Extending with custom plugins

Implement the AnalysisPlugin interface (id, name, description, supportedExtensions, analyze).
Prefer deterministic output; structure your metrics, metadata, and issues objects for stable downstream consumption.
Avoid reading huge files into memory; leverage the intake fileIndex to pre-filter paths.
Surface actionable issues (severity, message, optional file/line metadata) to integrate cleanly with enhanced analyzer recommendations.
Register the plugin via new EnhancedRepositoryAnalyzer([myPlugin]) or call registerPlugin on an existing analyzer instance.

🧠 Enhanced analyzer APIs

The enhanced analyzer exposes a concise public surface:

Method	Description	Key parameters	Returns
`analyzeRepository(repoUrl, options?)`	Executes intake, traditional analysis, and returns the comprehensive payload.	`repoUrl` (string), `options` (subset of intake options).	`ComprehensiveAnalysis` containing `context`, `results`, `dependencies`, `summary`, `completedAt`.
`analyzeForAI(repoUrl, options?)`	Runs `analyzeRepository`, then converts the output into an AI-optimized structure.	Same as above.	`AIReadyContext` with repository, project, codebase, testing, dependencies, issues, recommendations.
`registerPlugin(plugin)`	Adds a plugin to the pipeline for subsequent runs.	`plugin` implementing `AnalysisPlugin`.	`void` (mutates analyzer instance).
`getPlugins()`	Lists registered plugins in execution order.	none	`AnalysisPlugin[]` including defaults and custom entries.

ComprehensiveAnalysis essentials

context.repoDir is cleaned automatically once analyzeRepository completes.
summary mirrors key telemetry (file counts, language distribution, frameworks, testing frameworks, issue counts).
results[n].issues aggregates plugin-reported issues with severity tags (error, warning, info).
Optional fields (security, coverage, quality) are reserved for future plugins; current built-ins populate dependencies, results, and summary metrics.

🤖 AI-ready context preparation

analyzeForAI converts the comprehensive analysis into a structure designed for LLM consumers:

Repository: branch, commit hash, source URL.
Project: type, frameworks, testing frameworks, build tools, package managers (derived from projectStructure).
Codebase: directories, root files, files grouped by language, entry points, detected API endpoints.
Testing: presence of tests, files, organization (separate vs. co-located), ratio to source files.
Dependencies: production vs. development counts, major frameworks, testing libraries.
Issues: totals, grouped by category and severity, up to 10 critical highlights.
Recommendations: curated suggestions (e.g., add linting, introduce testing) inferred from summary and plugin metrics.

Call analyzeForAI when feeding the FastAPI AI service or any feature that needs condensed context. See docs/packages/ai-service.md for how the AI pipeline expects this payload.

Example recommendations you might observe:

“Consider adding a testing framework like Jest, Vitest, or PyTest.”
“Organize source files under a dedicated directory (src/, lib/, etc.).”
“Add ESLint configuration to improve code consistency.”
“Address critical issues reported by the plugin pipeline before generating AI responses.”

🔄 Migration: RepositoryAnalyzer → EnhancedRepositoryAnalyzer

Swap imports: replace RepositoryAnalyzer references with EnhancedRepositoryAnalyzer.
Update instantiation: optional plugins are passed into the constructor (new EnhancedRepositoryAnalyzer([myPlugin])).
Adjust TypeScript types: expect ComprehensiveAnalysis instead of the legacy summary format; update router contracts accordingly.
Handle telemetry: align any logging or analytics with the new summary structure (language counts, framework arrays, issue counts).
Run integration tests: verify that cleanup occurs as expected and downstream persistence still accepts JSONB payloads.
Coordinate with API consumers: tRPC router updates should be reflected in the documentation within docs/packages/trpc-package.md.

No database schema changes are required because both analyzers emit JSON-compatible payloads.

⚙️ Operational guidelines

Resource planning

Clone depth of 1 keeps network usage minimal; increase only if historical data is essential.
Large repositories can generate sizable fileIndex objects—adjust maxFiles and maxFileSizeBytes to stay within process memory limits.
The enhanced analyzer logs progress for each phase (intake, pipeline, cleanup) to aid observability.

Security & cleanup

GitHub tokens are embedded into the clone URL only for the duration of the clone operation.
Temporary directories are removed automatically in both success and failure paths for the enhanced analyzer.
Custom plugins should avoid logging file contents or secrets; rely on metadata and hashes when reporting issues.

Telemetry integration

Pipeline summary exposes issue counts and language distribution; forward these metrics to monitoring systems as needed.
Plugin-specific metrics objects can be enriched to support dashboards (for example, dependency counts, test ratios).
Enhanced analyzer console logs use emoji markers (🔄, ✅, 🧹)—redirect or wrap logging for production contexts if necessary.

🧱 Troubleshooting & FAQs

Question	Guidance
Why are dependencies missing from `dependencies`?	Only the dependency plugin contributes to this list. Ensure the plugin remains registered and that manifests are accessible.
How do I skip default plugins?	Instantiate `TraditionalAnalysisPipeline` directly and register only the plugins you need, or fork the class; the enhanced analyzer always registers the defaults.
Can I analyze monorepos with multiple package managers?	Yes. Intake collects root manifests, and the dependency plugin inspects additional files (`package.json`, `requirements.txt`, lockfiles). Consider custom plugins for specialized build systems.
What if a plugin throws an error?	The pipeline catches the error, logs it, and continues. Investigate plugin logs to resolve the root cause.

🔗 Related documentation

docs/packages/trpc-package.md – how analysis results flow into API routes.
docs/packages/database-package.md – storage expectations for analysis payloads (JSONB fields).
docs/packages/ai-service.md – consuming the AI-ready context in the FastAPI service.
Repository source under packages/repo-analyzer/src for implementation references.

Focus future edits on maintaining accuracy with the underlying TypeScript modules; avoid reintroducing large code blocks so the documentation stays high-signal and easy to maintain.