Skip to main content

AI Coding Tool Analytics

AI coding assistants have become the standard workflow for modern software development. But the analytics layer — measuring which tools are used, for what tasks, and with what outcomes — remains almost entirely absent. This pillar page collects everything I've written and built around solving this problem.

The Core Problem​

84% of developers now use AI coding tools, but most teams have zero visibility into usage patterns, productivity impact, or ROI. The tools themselves don't help — Claude Code stores transcripts as raw JSONL, Cursor exports nothing, and GitHub Copilot only shows acceptance rates (a vanity metric).

Engineering leaders are investing real budget in AI tooling with no data to justify the spend or optimize the investment.

My Work in This Space​

PromptConduit​

PromptConduit is the analytics platform I built to close this gap. It captures events from Claude Code, Cursor, and other AI tools in real-time, normalizes them into a common schema, and provides dashboards for understanding usage patterns.

The architecture is designed around one principle: never block the AI tool. Analytics happen asynchronously in the background.

Key components:

  • Go CLI — Captures real-time events via hooks, ships as a single binary via Homebrew
  • macOS App — Native menu bar app for managing AI agent sessions
  • SaaS Platform — Web dashboard for pattern analysis and team-wide insights

Measurement Framework​

In How to Measure AI Coding Productivity, I published a three-category framework:

  1. Usage Patterns — What's happening (prompts per day, tool distribution, session depth)
  2. Quality Signals — Is it working (iteration count, commit attribution, test pass rates)
  3. Impact Metrics — Does it matter (time to first commit, scope expansion, context switching reduction)

The framework includes a tiered implementation approach:

  • Tier 1: Git attribution — Add AI-Tool: trailers to commits (free, start today)
  • Tier 2: Session analytics — Parse Claude Code JSONL transcripts for tool usage patterns
  • Tier 3: Team dashboards — Aggregate across tools and team members with PromptConduit

Key Insights from My Own Data​

After months of tracking through PromptConduit:

  • Short, focused sessions (10-15 min) outperform long exploratory ones
  • File reads dominate early in a feature; edits dominate later
  • Cross-project context switching is the biggest friction point
  • Tool choice matters less than prompting skill

Metrics to Avoid​

  • Lines of code generated — More isn't better
  • Suggestion acceptance rate — High acceptance ≠ high quality
  • Self-reported time savings — Developers are terrible at estimating counterfactuals
  • Raw prompt count — More prompts might mean more friction, not more productivity

Related Content​