dataproduct-management
Metrics Framework
Define product and engineering metrics frameworks — choosing primary, guardrail, and diagnostic metrics with measurement methodology, review cadence, ownership, and dashboard specifications.
metricsKPIsmeasurementdashboardsanalyticsOKRs
Works well with agents
metrics-framework/
SKILL.md
Markdown| 1 | |
| 2 | # Metrics Framework |
| 3 | |
| 4 | ## Before you start |
| 5 | |
| 6 | Gather the following from the user. If anything is missing, ask before proceeding: |
| 7 | |
| 8 | 1. **Business objective** — What outcome are you trying to drive? (e.g., increase retention, reduce cost-to-serve, grow revenue per user) |
| 9 | 2. **Product or team scope** — Which product, feature area, or team does this framework cover? |
| 10 | 3. **Current state** — Do metrics exist today? What is being tracked, and what is missing? |
| 11 | 4. **Audience** — Who will consume these metrics? (executives, PMs, engineers, cross-functional reviews) |
| 12 | 5. **Data infrastructure** — What tools are in place? (warehouse, event tracking, BI layer, alerting) |
| 13 | 6. **Review cadence** — How often does the team review metrics today? How often should they? |
| 14 | |
| 15 | If the user says "we need a dashboard," push back and ask what decisions the dashboard should inform. Dashboards without decision context become decoration. |
| 16 | |
| 17 | ## Metrics framework template |
| 18 | |
| 19 | ### 1. Business Objective |
| 20 | |
| 21 | State the business objective in one sentence. This anchors every metric choice — if a metric does not connect to this objective, it does not belong in the framework. |
| 22 | |
| 23 | ``` |
| 24 | Objective: Increase 90-day user retention from 34% to 45% by Q3, |
| 25 | driving ARR expansion through reduced churn. |
| 26 | ``` |
| 27 | |
| 28 | ### 2. Metric Types |
| 29 | |
| 30 | Every framework needs exactly three tiers. No more, no fewer. |
| 31 | |
| 32 | | Type | Purpose | Count | Example | |
| 33 | |------|---------|-------|---------| |
| 34 | | **Primary** | The single metric that defines success for this objective | 1 | 90-day retention rate | |
| 35 | | **Guardrail** | Metrics that must not degrade while pursuing the primary metric | 2-4 | Support ticket volume, NPS, p95 latency | |
| 36 | | **Diagnostic** | Metrics that explain *why* the primary metric is moving | 4-8 | Feature adoption rate, onboarding completion, time-to-value | |
| 37 | |
| 38 | **Primary metric rules:** |
| 39 | - Exactly one. If you have two primary metrics, you have zero — pick one or combine them into a composite. |
| 40 | - Must be directly measurable, not derived from surveys or estimates. |
| 41 | - Must move on the timescale of your review cadence. A quarterly metric reviewed weekly creates noise, not signal. |
| 42 | |
| 43 | **Guardrail metric rules:** |
| 44 | - Protect against perverse incentives. If your primary metric is retention, a guardrail should catch cases where you retain users by making cancellation harder rather than making the product better. |
| 45 | - Each guardrail has a threshold, not a target. You are not optimizing guardrails — you are ensuring they stay within acceptable bounds. |
| 46 | |
| 47 | **Diagnostic metric rules:** |
| 48 | - Diagnostics are investigation tools, not success measures. They answer "why is the primary metric moving?" |
| 49 | - Organize diagnostics in a causal chain: input metrics (actions) lead to output metrics (results). |
| 50 | |
| 51 | ### 3. Metric Definition Card |
| 52 | |
| 53 | Define every metric using this template. Ambiguity in definitions is the #1 reason metrics frameworks fail — two people looking at the same dashboard should never disagree on what a number means. |
| 54 | |
| 55 | ``` |
| 56 | Metric: 90-day retention rate |
| 57 | Formula: Users active on day 90 / Users who completed onboarding 90 days ago |
| 58 | Data source: events.user_activity + events.onboarding_completed (warehouse) |
| 59 | Owner: @product-lead (Growth) |
| 60 | Review cadence: Weekly (trended), Monthly (cohort deep-dive) |
| 61 | Alerting threshold: < 30% (7-day rolling avg) triggers Slack alert to #growth-metrics |
| 62 | Segmentation: By plan tier, signup source, onboarding path |
| 63 | ``` |
| 64 | |
| 65 | Every metric in the framework gets a card. No exceptions. If you cannot fill out the formula and data source fields, the metric is aspirational, not operational — flag it as a gap to close. |
| 66 | |
| 67 | ### 4. Dashboard Specification |
| 68 | |
| 69 | Dashboards serve decisions, not aesthetics. Structure by audience: |
| 70 | |
| 71 | - **Executive dashboard:** 3-5 metrics, trended over time, updated daily. Primary metric front and center, guardrails visible at a glance. No diagnostic metrics — executives do not debug. |
| 72 | - **Team dashboard:** All three tiers. Primary and guardrails at top, diagnostics below. Include filters for key segments. Updated in real-time or hourly. |
| 73 | - **Investigation view:** Diagnostic metrics with drill-down capability. Cohort breakdowns, funnel analysis, event-level detail. Used ad hoc, not on a schedule. |
| 74 | |
| 75 | For each dashboard, specify: tool (Looker, Metabase, Tableau, etc.), refresh frequency, access control, and the one person responsible for keeping it accurate. |
| 76 | |
| 77 | ### 5. Review Process |
| 78 | |
| 79 | Define how metrics are reviewed, not just displayed: |
| 80 | |
| 81 | - **Weekly standup** (15 min): Primary metric trend + any guardrail violations. Action: assign investigation owners for anomalies. |
| 82 | - **Monthly review** (45 min): Cohort analysis on primary metric. Diagnostic deep-dive. Action: update priorities based on what diagnostics reveal. |
| 83 | - **Quarterly calibration** (90 min): Is the primary metric still the right one? Have guardrails caught real problems? Action: revise the framework if objectives have shifted. |
| 84 | |
| 85 | Every review must produce either "no action needed" or a named owner with a deadline. Reviews without outcomes are status theater. |
| 86 | |
| 87 | ### 6. Anti-Metrics: What NOT to Measure |
| 88 | |
| 89 | Explicitly list metrics you considered and rejected. This prevents them from creeping back in. |
| 90 | |
| 91 | - **Vanity metrics** that move in only one direction (total signups, cumulative revenue) — these feel good but inform no decisions. |
| 92 | - **Proxy metrics** where the proxy has diverged from the real outcome (DAU as a proxy for engagement when users open the app but do not complete any action). |
| 93 | - **Lagging-only metrics** that cannot be influenced within your review cadence (annual churn measured weekly). |
| 94 | - **Composite scores** that obscure signal by blending unrelated inputs (a "health score" averaging NPS, usage, and support tickets). |
| 95 | |
| 96 | ## Quality checklist |
| 97 | |
| 98 | Before delivering a metrics framework, verify: |
| 99 | |
| 100 | - [ ] Exactly one primary metric is defined — not two, not a composite |
| 101 | - [ ] Every metric has a complete definition card with formula, data source, and owner |
| 102 | - [ ] Guardrail metrics protect against perverse incentives of the primary metric |
| 103 | - [ ] Diagnostic metrics form a causal chain that explains primary metric movement |
| 104 | - [ ] Every metric moves on a timescale compatible with its review cadence |
| 105 | - [ ] Dashboard spec names the tool, refresh rate, and a single person responsible for accuracy |
| 106 | - [ ] Review process defines outcomes, not just meetings |
| 107 | - [ ] Anti-metrics section documents what was excluded and why |
| 108 | |
| 109 | ## Common mistakes to avoid |
| 110 | |
| 111 | - **Vanity metrics as primary metrics.** Total signups, page views, or "engagement score" feel good but drive no decisions. A primary metric must be something you can act on and would change your priorities if it moved. |
| 112 | - **Too many metrics.** Frameworks with 20+ metrics get ignored. If everything is a KPI, nothing is. Constrain to 1 primary + 3 guardrails + 6 diagnostics maximum. |
| 113 | - **No ownership.** A metric without an owner is a metric nobody acts on. Every metric card must name a person — not a team, not a channel, a person. |
| 114 | - **Missing guardrails.** Optimizing a primary metric without guardrails invites Goodhart's Law. If your primary metric is time-on-site, you need guardrails for task completion and satisfaction — otherwise you are incentivizing confusion, not engagement. |
| 115 | - **Measuring what is easy instead of what matters.** Click counts are easy to track. Whether users achieved their goal is harder but more valuable. Do the hard instrumentation work. |
| 116 | - **No review cadence.** Dashboards without scheduled reviews become wallpaper. Define who looks at what, when, and what decisions they make as a result. |
| 117 |