What is Hypothesis testing? +40% revenue. Looks great on your dashboard.

 Until your boss asks: “What caused it?”

 You changed the homepage, pricing, and checkout. 

Sales went up but you have no idea why. Are your customers actually price sensitive or was it just a change in the wind? So you decide to simplify it. You change one thing: the checkout button color.

Conversions go from 10% to 12%. Looks like a win. But maybe those users were just more likely to buy anyway. Maybe it was payday.

The increase might be real, or it might be noise. 

This is why hypothesis testing exists.

In real life, data is messy. Sales go up and down naturally, users behave differently every day, and sometimes things improve even if you do nothing.

If you look at a +40% increase and say “that worked,” you’re not doing data science, you’re just making up a story with numbers. The real question is how you separate actual impact from random noise.

Today we’re going to take a look at hypothesis testing. What it is, how it works and how you can use it to make actual decisions instead of relying on guesswork.

Your Jupyter notebook can’t be production and the biggest reason is that when things you wrong, it’s an absolute nightmare to debug. So let’s start by solving that problem from the start. In this series we talk about how to turn a Jupyter notebook into something that can actually be put into production, starting with the first step: How you write it.

Not many people realise this but your favourite LLM can't actually be your therapist.

Because a therapist understands what you mean. An LLM processes what you wrote.

There's a difference.

You read "I feel lonely" and hear fear, grief, and a need for connection.

A model reads tokens.

Words, to a model, are useless. Until you convert them. That conversion is what today's video is about.

If you get that conversion wrong, you get a model that spouts nonsense or sees the word "and" as the ultimate predictor of customer churn. If you get it right, everything becomes possible.

So how do you actually do it?

How do you take a hundred film synopses and turn them into numbers that capture meaning?

Two tools. TF-IDF and Truncated SVD.

Let's do some Evil Work.