Eight Tiny Experiments for Choosing Tests That Tell the Truth

  1. One-sentence test choice  
    Write the test you’ll use in one sentence that a smart non-statistician would accept. If you can’t, you’re not ready to analyze.
  2. Direction audit  
    State your directional hypothesis on paper. Circle the observation that would make you retract it. If you can’t find one, you don’t have a hypothesis… you have a preference.
  3. Ambiguity tally  
    For three days, note every “Wait… what?” moment. End each with a rewrite that would’ve prevented it. Which rewrite repeats?
  4. Ending rehearsal  
    Schedule a five-minute ending for one task you dread. Make it easy, specific, and positive. Re-rate the task tomorrow.
  5. Label like a human  
    Replace “Group A/B” with real labels that capture the design logic. Read your results aloud. Do they suddenly make more sense?
  6. Assumption speed-check  
    Create a 30-second checklist: randomization or not… sample size or normality… correct test value… correct mapping of rows. Run it before every analysis.
  7. Two-analysis trap  
    Analyze a small dataset two ways that answer different questions. Present both to a friend. Which question matters more in the real world?
  8. Power with a conscience  
    If you switch from two-tailed to one-tailed, write why… and what would make you switch back. Keep the note with the output.

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.