The call an hour before the release
Release call, Tuesday, four o’clock
PM Twenty minutes. Are we going or not.
Engineer The code is in production since Friday, dark. It is behind a flag, so going means flipping it, not deploying anything.
PM Then we start at one percent tomorrow morning and sit on it.
Engineer Staged, one percent, and I want the kill switch tested against the real config before I go to bed, not the staging one.
Analyst If it is at one percent I cannot read the experiment for a week. The minimum detectable effect at that traffic is bigger than the change we are hoping for.
PM I saw a chart this morning that said we were up eleven percent.
Analyst That chart was thirty hours old and I would like it deleted. Looking every morning and stopping on a good day is how a team convinces itself of a coin flip.
Engineer There was also a split problem on the last one. Fifty-fifty came out fifty-three to forty-seven on two million sessions.
Analyst Which means the assignment was broken, not that the variant won. I am not reading a test that fails that check.
PM What about the guardrails.
Analyst Refunds and support contacts, both flat. And activation is what we agreed to move — first shared project, not sign-ups.
Engineer One more thing. We spent most of the error budget on the outage two weeks ago, so if this wobbles, we stop for the month.
PM Understood. Go, at one percent, and triage at ten tomorrow.
Twenty minutes to decide, and almost none of it is about the feature. It is about who can turn it off, who is allowed to read the numbers, and what happens if the numbers are lying.
A release call runs from the decision to the way out of it
It starts with a yes or a no and immediately stops being that: nothing goes to everyone at once any more. Then come the words for measuring, and for the ways measurement goes wrong. And it ends with what the team has left to spend on risk.
Go/no-goGo / no-go
The meeting or decision point at which a team confirms whether a release goes ahead.
Go/no-go is at four, bring the crash rate.
The moment the call exists for, and the shortest item on its agenda.
Feature flag
A switch in the code that turns a feature on or off for chosen users without shipping a new release.
It's behind a feature flag, so we can ship it dark and turn it on Monday.
What turned shipping into a separate thing from releasing. It is why the code has been in production since Friday and nobody outside the team has met it.
Staged rolloutDespliegue por fases
Releasing a change to a growing share of users instead of to everyone at once.
Staged rollout: one percent today, ten tomorrow if the graphs stay boring.
The reason "we launched" now means "one percent of people can see it". Confused with a beta: one is chosen people, this is a fraction of everyone.
Kill switch
A control that turns a feature off immediately when something goes wrong.
Confirm the kill switch works before we go to fifty percent.
The thing that has to be tested when nothing is wrong, because the moment it is needed nobody will be reading documentation.
A/B testingTest A/B
Splitting live traffic between two versions and deciding by measured behaviour rather than by whose argument sounded better.
We A/B tested the empty state and the version with one button won.
How a disagreement about a button becomes a question with an answer. It also turns everyone in the room into an amateur statistician.
Guardrail metricMétrica guardarraíl
A metric watched during an experiment to make sure a win in one place is not doing damage somewhere else.
Conversion is up, but the guardrail metric on refunds moved too.
The number nobody is trying to move, watched precisely for that reason. Without it, a win in one place quietly buys a loss in another.
Peeking
Checking experiment results repeatedly and stopping as soon as they look good, which inflates false positives.
Stop peeking, the test runs until Friday.
The most human way to break an experiment, and the hardest to admit to. Anyone who checks a dashboard daily has done it.
Sample ratio mismatchSRM
A statistically significant gap between the intended traffic split and the actual one, which signals a broken experiment.
On two million sessions the split failed the sample ratio mismatch check, so we're not reading it.
The check that says the experiment itself is broken, not that a variant lost. Skipping it is how a broken assignment gets shipped as a finding.
Tracking planPlan de medición
The agreed document of which events and properties are logged and what each one means.
Check the tracking plan before inventing a new event name.
The agreement that keeps two teams from meaning different things by the same event name. Written once, ignored for a year, rewritten in anger.
ActivationActivación
The point at which a new user completes the action that shows they have got real value.
Activation is 'first project shared', not 'account created'.
The definition every team argues about and few write down. Set it at sign-up and every graph after it is decoration.
Bug triageTriaje de errores
The regular pass in which reported bugs get a severity, an owner and a decision to fix or not.
It came out of triage as a P3, so not this sprint.
The half hour where a bug gets a name, an owner and a verdict — including the verdict "not now", which is a decision and gets recorded as one.
Error budgetPresupuesto de error
The amount of failure an SLO allows, which a team may spend on risk before releases have to pause.
We burned the error budget in one afternoon, so no risky deploys this month.
The reason a team sometimes stops shipping while everything looks fine. It turns reliability from an argument into an amount.
Tomorrow it goes to ten percent
Then fifty, then everyone, and somewhere in there someone asks whether the test can be called early. The one holding the switch will be you.
Questions and answers
What about the rest of the product words?
On the deck page, all 150. These twelve are the ones a single release call uses.
Why is there Spanish on the card?
Release words arrive in English in every office — "feature flag", "rollout" — and the argument around them happens in Spanish. Both forms sit on one card.
Where do the definitions come from?
From the Product & UX deck — the same cards as in the app.
The next release call will run on a different twelve.
The Product & UX deck holds the rest — in the app they come a few a day.
