Product Feature Adoption & A/B Test Analysis
Designed and analyzed a simulated A/B test on product feature adoption using Python and Power BI, identifying a statistically significant +23.7% lift in Day-7 retention for new users.
Year
2026
Scope
Product Analytics / A/B Testing
Client
Independent Research
Duration
2 weeks
Simulated A/B experiment designed to practice real statistical decision-making the way product teams do it.
Challenge:
Most junior PMs talk about A/B testing without actually running one. The challenge was to simulate a real experiment with genuine statistical rigor — not just a surface-level exercise.
Solution:
Generated a 5,000-user dataset, ran chi-square significance testing with 95% confidence intervals using Python (scipy + pandas), and segmented results by new vs returning users. Identified +23.7% lift in Day-7 retention for new users (p = 0.000046) with no significant effect on returning users. Delivered a 4-page interactive Power BI dashboard and a data-backed ship/iterate decision memo.
