Thought Prompts

If one test risks a 5% false positive, what happens when you run ten? You don’t get ten times the truth… you get nearly a coin-flip chance of fooling yourself. Multiple testing quietly inflates Type I error… turning noise into “discoveries.” The fix isn’t magical; it’s methodological. Either adjust your alpha (Bonferroni, Holm, FDR) or […]

Prediction Is the Point  Good analysis upgrades your bets. If knowing X helps you predict Y better than chance, you’ve learned something you can use. Frame outputs as improved prediction and cost-aware decisions… not as proofs. Base Rates First  Before patterns, honor context. What’s common? What’s rare? Base rates anchor expectations, prevent over-reaction to noisy spikes, and

Most analysis failures aren’t math errors… they’re hygiene lapses. Build a boring, repeatable checklist: (1) Reconcile counts: do totals match inputs after filters and joins? (2) Confirm definitions: do variables mean what the decision requires, and are time windows aligned? (3) Pre-write the one-sentence takeaway before you compute: then see if the numbers actually support

“Statistically significant” means your data would be unlikely if nothing were happening. Leaders ask a different question: how big is the thing that is happening, and what does it buy us? Always pair claims of rarity with claims of magnitude. Translate the effect into absolute differences, expected counts, percentage points, or risk ratios that tie

Complex questions tempt infinite slicing. Start by defining the core contrast that actually answers the decision question. If extra categories blur the picture, collapse them thoughtfully (with a note explaining how). You’ll reveal the signal that stakeholders can act on. Then, and only then, add nuance back selectively: re-expand categories where stakes are high, where

Evidence is contrast. Every “yes” needs its “no,” every success its failure, every win its baseline of total attempts. Rates require denominators, and patterns require counter-patterns. Drop the “other half” and the tallest bars will always seduce you into over-reading. Keeping both sides also surfaces asymmetries that matter for action: maybe the success rate barely

Data doesn’t make decisions… people do. Flip the order. First: What decision will this inform right now? Next: What specific outcome would make us choose A over B? Then: What evidence would change our mind the other way? When you anchor analysis to a fork in the road, you automatically narrow the variables, shrink the

One-Tailed On Purpose  Sometimes the direction doesn’t matter… distance does. How far from “what we’d expect” are we? That’s the question. Deviation is the story. In a noisy world, clarity is leverage. Measure the gap. Then decide what to do about the gap. Expected ≠ Equal Your baseline isn’t always “all is equal”. Sometimes the

Systems create outcomes on purpose… or by default. Cutoffs, criteria, calendars: quiet levers. Think you’re being neutral? You’re probably cementing the status quo. Start with the base rate. Compare observed to expected. If the mismatch is systematic, adjust the rules. Band by age. Rotate gates. Audit selections. Build ladders, not just leaderboards. Fair doesn’t mean

Cryptic labels waste cognition. Human labels return it. “Group A” is fog. “Early-season cohort” is a map. Same data, different friction. Communication is part of the method, not an afterthought. If you want people to reason well, lower the decoding cost. Name variables so a beginner can argue with them. Because argument is learning. And