35 backlog items across 8 phases — open for contributions — see the gap analysis driving these priorities
Recommended execution order: Phase 1 (Dashboard Gaps) → 2 (Cost & ROI) → 3 (Developer Sentiment) → 7 (Key Integrations) → 4 (Benchmarking) → 5 (Scale) → 6 (Analytics) → 8 (Novel)
| ID | Item | Priority | Size | Labels | Closes Gap vs |
|---|---|---|---|---|---|
| R-101 | Bubble quality metrics to Executive dashboard — Add Defect Rate (AI vs Human), AI Test Coverage Delta, Spec-to-Code Hours widgets | P0 | S | dashboard | Internal gap |
| R-102 | Add PRISM Level widget to Team Velocity — Show current maturity level + dimension breakdown | P0 | S | dashboard | Internal gap |
| R-103 | Cross-team comparison for developers — Let devs see how their team compares on key metrics | P0 | S | dashboard | Swarmia, Jellyfish |
| R-104 | Incident/alert panel on both dashboards — Surface active alarms, recent incidents, MTTR trends | P0 | M | dashboard pipeline | All 3P tools |
| R-105 | Expand alerting — Alarms for deploy frequency drops, lead time spikes, defect anomalies | P1 | M | infra dashboard | All 3P tools |
| ID | Item | Priority | Size | Labels | Closes Gap vs |
|---|---|---|---|---|---|
| R-201 | AI tool cost ingestion pipeline — New EventBridge event type for per-tool, per-seat cost data | P0 | M | pipeline new-metric | Jellyfish, DX, Faros |
| R-202 | True ROI dashboard widget — Calculate (productivity_gain x avg_eng_salary) / tool_cost | P0 | M | dashboard | Jellyfish, DX |
| R-203 | License utilization tracking — Active vs provisioned seats per AI tool, flag underutilization | P1 | M | pipeline dashboard | Swarmia, Faros |
| R-204 | Cost-per-deploy and cost-per-feature metrics — Derived metrics combining deployment events with cost data | P1 | S | dashboard new-metric | Jellyfish |
| ID | Item | Priority | Size | Labels | Details |
|---|---|---|---|---|---|
| R-210 | CloudTrail → EventBridge token pipeline — EventBridge rule matching bedrock.amazonaws.com InvokeModel/Converse calls in CloudTrail. Lambda extracts model_id, input_tokens, output_tokens, timestamp, IAM principal. | P0 | L | pipeline infra | Foundation for all token/cost metrics. Requires CloudTrail data events enabled for Bedrock. |
| R-211 | Model pricing table — DynamoDB table mapping model_id to per-1K-token cost (input/output). Updatable via API or manual config. Covers Claude Opus/Sonnet/Haiku, Amazon Titan, and future models. | P0 | S | infra new-metric | Prerequisite for cost calculation. Include on-demand and provisioned throughput pricing. |
| R-212 | IAM principal → Developer identity mapping — Resolve IAM role/user ARNs from CloudTrail to developer identities. Support mapping table (IAM ARN → developer email → team_id) and SSO/Identity Center federation. | P0 | M | pipeline infra | Critical for per-developer attribution. Without this, tokens are only trackable at the IAM role level. |
| R-213 | Token-to-commit correlation — Match Bedrock API calls to git commits using timestamp proximity (configurable window, default 5 min). When a commit with AI-Tool: claude-code lands, look back at Bedrock calls from the same developer in the preceding window. | P1 | L | pipeline new-metric | Enables cost-per-commit. Challenge: multi-turn sessions may span multiple commits. Use session grouping heuristic (gap > 15 min = new session). |
| R-214 | Token-to-PR aggregation — Sum all correlated token usage across commits in a PR. Link to PR metadata (author, feature branch, labels). Calculate cost-per-PR. | P1 | M | pipeline new-metric | Depends on R-213. Enables feature-level cost visibility when PRs are tagged with feature labels. |
| R-215 | Token-to-feature/epic mapping — Aggregate PR-level costs up to feature/epic level using Jira/Linear issue keys extracted from PR titles, branch names, or commit messages. Optional: label-based grouping. | P1 | M | pipeline integration | Depends on R-214. Answers “how much AI spend went into Feature X?” Requires issue key extraction regex. |
| R-216 | Developer session analytics — Group Bedrock API calls into developer sessions (gap-based: >15 min idle = new session). Track: session duration, turns, total tokens (in/out), models used, cost. Publish as prism.d1.session event. | P1 | L | pipeline new-metric | New event type. Enables “avg session cost”, “tokens per session”, “sessions per developer per day” metrics. |
| R-217 | CloudWatch token & cost metrics — New metrics in PRISM/D1/Velocity namespace: BedrockTokensInput, BedrockTokensOutput, BedrockCostUSD, CostPerCommit, CostPerPR, CostPerSession, TokenEfficiency (output tokens / lines of code). Dimensions: TeamId, Developer, Model, AITool. | P0 | M | pipeline new-metric | Depends on R-210. Core metric publication for all downstream dashboards and alarms. |
| R-218 | Token & cost dashboards — Add to both Team Velocity and Executive Readout: daily/weekly token consumption trends, cost by tool/model/developer, cost-per-commit trend, cost-per-PR distribution, token efficiency (tokens per line of code), budget burn rate with threshold lines. | P0 | L | dashboard | Depends on R-217. Exec dashboard: cost trends + budget. Dev dashboard: per-developer breakdown + efficiency. |
| R-219 | Cost anomaly alarms — CloudWatch alarms for: daily spend exceeds 2x 7-day average, single developer exceeds team P90, token efficiency drops below threshold. SNS notification to team channel. | P1 | S | infra dashboard | Depends on R-217. Prevents runaway costs from unattended agent loops or inefficient prompting. |
| R-220 | Multi-tool cost normalization — Unified cost view across Bedrock (pay-per-token), Copilot (flat subscription), Cursor (subscription + usage), and other tools. Normalize to cost-per-developer-day for apples-to-apples comparison. | P2 | M | pipeline dashboard | Extends R-201. Different pricing models make comparison hard. Cost-per-developer-day is the common unit. |
| R-221 | ROI calculator with token cost input — Enhance the Executive ROI Multiplier widget: (time_saved_hours × avg_hourly_eng_cost) / (token_cost + license_cost). Show break-even point and marginal ROI per additional AI-assisted commit. | P1 | M | dashboard new-metric | Depends on R-218 + R-202. The “is AI worth it?” answer for CFOs. |
| ID | Item | Priority | Size | Labels | Closes Gap vs |
|---|---|---|---|---|---|
| R-301 | Developer sentiment survey framework — Lightweight pulse surveys via CLI or Slack, stored in DynamoDB | P0 | L | pipeline new-metric integration | DX (core) |
| R-302 | Survey results dashboard — Sentiment trends, correlation with DORA, satisfaction by AI tool | P1 | M | dashboard | DX, Faros |
| R-303 | Developer Experience Index — Composite score combining systems metrics + survey responses | P1 | M | new-metric dashboard | DX |
| R-304 | AI tool satisfaction breakdown — Per-tool sentiment (Claude Code vs Copilot vs Kiro) | P2 | S | dashboard | DX |
| ID | Item | Priority | Size | Labels | Closes Gap vs |
|---|---|---|---|---|---|
| R-401 | Anonymous benchmark data submission — Opt-in pipeline for anonymized metrics to shared dataset (S3 + Athena) | P1 | L | pipeline infra community | LinearB, Jellyfish, DX |
| R-402 | Benchmark comparison widgets — Team metrics against community percentiles (P25/P50/P75/P90) | P1 | M | dashboard | LinearB, Jellyfish |
| R-403 | PRISM Level distribution — Where teams fall on L1-L5 relative to community | P2 | S | dashboard community | DX |
| R-404 | Annual PRISM benchmark report — Community-driven equivalent of LinearB's benchmarks | P2 | L | docs community | LinearB |
| ID | Item | Priority | Size | Labels | Closes Gap vs |
|---|---|---|---|---|---|
| R-501 | Org hierarchy model — DynamoDB schema for org > division > team > repo with rollup aggregation | P1 | L | pipeline infra | Jellyfish, Faros |
| R-502 | Executive drill-down dashboards — Org-level summary drilling into division, team, repo views | P1 | L | dashboard | Jellyfish, Faros |
| R-503 | Multi-repo aggregation — Team-level metrics aggregating across all owned repos | P1 | M | pipeline | All 3P tools |
| R-504 | QuickSight row-level security — Role-based access per org hierarchy level | P2 | M | infra dashboard | Jellyfish, Faros |
| ID | Item | Priority | Size | Labels | Closes Gap vs |
|---|---|---|---|---|---|
| R-601 | Cohort A/B analysis — Compare AI-using vs non-AI-using developer cohorts | P1 | L | pipeline dashboard | DX, Faros |
| R-602 | Anomaly detection — ML-based alerts using CloudWatch Anomaly Detection | P1 | M | infra | All 3P tools |
| R-603 | Natural-language metrics query — Claude-powered conversational interface via Bedrock Agent | P1 | XL | infra integration | Jellyfish, DX, Faros |
| R-604 | Automated weekly insights digest — Bedrock-generated summary delivered via SNS/email | P2 | L | infra integration | Jellyfish, Faros |
| R-605 | Predictive PRISM leveling — ML model predicting when team reaches next level | P2 | L | new-metric infra | Novel |
| ID | Item | Priority | Size | Labels | Closes Gap vs |
|---|---|---|---|---|---|
| R-701 | Multi-AI tool breakdown dashboard — Metrics by tool (Claude Code, Copilot, Kiro, Q Dev, Cursor) | P1 | M | dashboard | Jellyfish (10+ tools) |
| R-702 | Slack/Teams notifications — Alert summaries and weekly digests to team channels | P1 | M | integration | Swarmia, LinearB |
| R-703 | Jira/Linear integration — Map issues to deployments for investment allocation | P2 | L | integration pipeline | Jellyfish, LinearB |
| R-704 | R&D capitalization report — Time allocation breakdown for finance teams | P2 | L | dashboard new-metric | Jellyfish, Swarmia |
| R-705 | GitHub Actions marketplace action — One-click PRISM metric emission from any CI/CD | P1 | M | integration community | Swarmia, LinearB |
| R-706 | GitLab CI integration — Equivalent collector for GitLab-based teams | P2 | M | integration community | Swarmia, Faros |
| ID | Item | Priority | Size | Labels | Status |
|---|---|---|---|---|---|
| R-801 | Eval gate marketplace — Community-contributed Bedrock Evaluation templates | P1 | L | community infra | Novel |
| R-802 | Spec quality scoring — Rate spec completeness using Bedrock, correlate with code quality | P1 | L | new-metric pipeline | Novel |
| R-803 | AI pair-programming session analytics — Track session duration, tool calls, iteration count | P2 | L | new-metric pipeline | Novel |
| R-804 | Cross-team knowledge flow — Track AI pattern propagation across teams | P2 | XL | new-metric pipeline | Novel |
| R-805 | Guardrail effectiveness metrics — Bedrock Guardrail trigger rates and false positives | P2 | M | new-metric pipeline | Novel |
| R-806 | PRISM certification program — Automated assessment with badge generation | P2 | L | community docs | Novel |
| R-807 | AI technical debt tracker — Monitor AI code accumulating review comments, fixes, or churn | P1 | L | new-metric pipeline | Novel |
Pick an item, open an issue, submit a PR. Reference the roadmap ID (e.g., [R-301]) in your issue title. Maintainers will respond within 48 hours.
specs/ before codingai-origin: claude-code or ai-origin: kiro in commit trailersscripts/ generators to validate changesHave an idea not on the roadmap? Open an issue with the roadmap-proposal label including: what, why, complexity estimate, and competitive context.