Data Architecture & Metrics Pipeline

How every AI-assisted action becomes a measurable metric — from git commit to executive dashboard

End-to-End Data Flow

Data Sources

1. Git Hooks (Local Machine)

Installed via bootstrapper/metric-hooks/install.sh. Fire automatically on every commit and merge.

2. GitHub Webhooks

The webhook handler at collector/github-webhook-handler/index.ts processes 5 event types with HMAC-SHA256 signature verification.

3. GitHub Actions Workflows

Three workflows in bootstrapper/github-workflows/ emit structured events on specific triggers.

4. CI Metadata Emitter

The shell script at collector/ci-metadata-emitter/emit-metrics.sh runs inside GitHub Actions to emit deploy/PR events. It parses the git log from base ref to HEAD, counts commits by AI origin, calculates AI-to-merge ratio, and emits to EventBridge.

# Environment variables the emitter reads
PRISM_EVENT_TYPE=deploy|pr     PRISM_TEAM_ID=team-alpha
PRISM_EVENT_BUS=prism-d1-metrics   PRISM_REPO=owner/repo
PRISM_BASE_REF=origin/main    PRISM_SHA=abc123
PRISM_PR_CREATED_AT=2026-04-20T10:00:00Z   # for lead time calc

Event Schema

Every event follows this structure on EventBridge:

{
  "source": "prism.d1.velocity",
  "detail-type": "prism.d1.commit | .pr | .deploy | .eval | .incident | .assessment | .agent | .agent.eval",
  "detail": {
    "team_id": "team-alpha",
    "repo": "owner/repo-name",
    "timestamp": "2026-04-22T14:30:00Z",
    "prism_level": 3,
    "metric": {
      "name": "commit.files_changed",     // primary metric name
      "value": 5,                          // numeric value
      "unit": "files"                      // count | percent | seconds | hours | files | lines
    },
    "ai_context": {
      "tool": "claude-code",               // claude-code | kiro | q-developer | none
      "model": "claude-sonnet-4",          // model identifier
      "session_id": "sess_abc123",         // optional session ID
      "origin": "ai-assisted"              // ai-assisted | ai-generated | human
    },
    "dora": {
      "deployment_frequency": 1,           // count per event
      "lead_time_seconds": 3600,           // PR created → merged
      "change_failure_rate": 0.0,          // 0 or 1 per event, averaged over time
      "mttr_seconds": null                 // incident → resolution
    },
    "ai_dora": {
      "ai_acceptance_rate": 0.85,          // approved reviews / total reviews
      "ai_to_merge_ratio": 0.70,           // AI commits / total commits
      "spec_to_code_hours": 1.8,           // spec approved → code ready
      "post_merge_defect_rate": 0.014,     // defects per AI-assisted merge
      "eval_gate_pass_rate": 1.0,          // 1 if pass, 0 if fail
      "ai_test_coverage_delta": 0.123      // coverage change from AI tests
    }
  }
}

Processing & Storage

The Lambda processor (infra/lib/lambda/metrics-processor.ts) does a triple-write for every event:

Write 1: DynamoDB Events Table

Write 2: DynamoDB Metadata Table

Write 3: CloudWatch Metrics

Published to namespace PRISM/D1/Velocity with dimensions:

CloudWatch Metrics Catalog

DORA Metrics

AI-DORA Metrics

Agent Metrics

What Dashboards Read From

Active Alarms

Current Gap: Token Usage & Cost Tracking

What's tracked vs. what's missing

Why it's hard today

• Claude Code doesn't expose token counts via environment variables or files that git hooks can read. Session data lives in ~/.claude/ local storage.
• Bedrock API calls are logged in CloudTrail with token counts, but they're not automatically correlated to specific commits or PRs.
• Kiro and Q Developer session token data is similarly siloed in their respective backends.
• Multi-turn sessions may span multiple commits, making 1:1 correlation non-trivial.

Proposed solution: Bedrock CloudTrail → Token Pipeline

See the Token & Cost Intelligence phase in the Community Roadmap for the full implementation plan.

Bedrock InvokeModel API calls
         |
         v
CloudTrail (logs every call with input_tokens, output_tokens, model_id)
         |
         v
EventBridge Rule (matches bedrock.amazonaws.com InvokeModel events)
         |
         v
Lambda: token-cost-processor
  - Extract: model_id, input_tokens, output_tokens, timestamp, IAM user/role
  - Look up cost: model pricing table (DynamoDB)
  - Correlate: timestamp proximity to commits (within 5-min window)
  - Correlate: IAM role → developer identity
  - Aggregate: per-session, per-commit, per-PR, per-feature
         |
         v
CloudWatch: PRISM/D1/Velocity namespace
  - BedrockTokensInput (by TeamId, Developer, Model)
  - BedrockTokensOutput (by TeamId, Developer, Model)
  - BedrockCostUSD (by TeamId, Developer, Model)
  - CostPerCommit (by TeamId, AITool)
  - CostPerPR (by TeamId)
         |
         v
Dashboards: Token usage trends, cost overlays, ROI calculation

Key Files Reference