Use the same shape every time. The shape is the point. A clever note that invents its own sections cannot be compared with the last one, and comparison is the only reason to call the thing an experiment instead of a story. Stories are allowed on this site. They belong in the journal. When you change one thing on purpose and you want to know what happened, use this template. Then file it where the other tests live.
The experiments section is this template with room to write. The experiment log is the idea of making the comparison easier later. Do not wait for the log. Write the note. Waiting for a tool is how a test stays a feeling. Feelings do not have a measure chosen in advance.
If you changed two things, you have two experiments or zero. You do not have a richer one.
The parts
Hypothesis
One sentence. What you believe will happen, and to whom. Name the person. "Engagement will go up" is not a hypothesis. It has no person and no behavior you could fail. "A reader who wanted a course will leave earlier, and a reader who wanted a notebook will reach the next section more often" can fail. Failure is the feature that makes the sentence worth writing down.
Keep it falsifiable in ordinary language. You do not need a statistical costume. You need a predicted direction and a someone. If you cannot say what would count as the hypothesis being wrong, you have a hope. Hopes can open a journal entry. They should not open this template. I would rather a blunt sentence a colleague can argue with than a careful one that survives every outcome. Surviving every outcome is the mark of a sentence that was never a test.
Tie the hypothesis to a belief you already hold in public if you can. A test that arrives from nowhere teaches less than a test that pressures a note. The positioning experiment pressures positioning is a refusal. The public-writing experiment pressures the claim that an audience makes a sentence sharper. Linking the belief gives the result a place to land. A result with nowhere to land becomes a trivia fact about a page.
Change
What you actually altered. If you changed two things, you have two experiments or zero. Two, if you are willing to run them separately. Zero, if you ship them together and narrate a moral anyway. The moral will be fiction. You will not know which change the person reacted to, and you will choose the explanation that flatters the idea you liked more.
Write the change as a diff a stranger could apply. "We improved the headline" is not a diff. "We replaced a broad welcome with one refusing sentence, and we left the rest of the page" is a diff. Include what you held still. Held still is half the change. If the offer, the audience, and the page all moved, say so, and downgrade the note to a launch story. Launch stories are fine. They are not this template. Pretending they are is how a team builds a mythology of testing without ever isolating a cause.
One change also means one owner of the sentence. Someone has to be allowed to say what the before and after were. Committees smuggle extra changes in at the end: a new image, a tweaked verb, a different button. Each tweak feels too small to mention. Together they are a second experiment hiding in the first. List them. If the list is longer than one item, split the work or admit you are shipping a bundle.
Measure
What you will look at, chosen before the result. Include what would count as "nothing happened." The second half is the one people skip, and it is the half that keeps a flat week from being reinterpreted as a subtle win. "Nothing happened" might mean the next section was reached at the same rate, or that no decision you can point to used the new sentence. Write it down while you still hope something will happen. Hope edits the measure if you let it wait.
Pick a measure you can actually see. A private notebook can see a traceable decision more honestly than it can see a market. Writing in public uses that kind of measure on purpose: not traffic, a decision you can link. Vanity counts are a bad fit for a site that will not invent readers. If your measure requires a number you do not have, choose a different measure. Do not invent the number so the template looks grown up. A grown-up lie is still a lie.
Also write the window. A day, a week, a handful of sends. Windows stop you from peeking until the story appears. Peeking is human. The template's job is to make the peek visible, so you can say "we looked early" instead of quietly moving the finish line. If the topics differ between two sends, the measure does not get to pretend they were the same test. That is why the subject-line note stays inconclusive. The template would rather be inconclusive than neat.
Result
What happened, including a result you do not like. Lead with the observation, not with the lesson. "The refusing line lost the first screen and held more of the later reading" is an observation. "Therefore narrow positioning always wins" is a lesson the observation did not earn. Lessons can follow, marked as tentative. They should not replace the result. A replaced result is how a template becomes a sermon.
Include the mess. Small samples. Mixed topics. A week where something else shipped. Readers who were you, refreshing the page. Mess does not ruin the note. Hidden mess does. A later comparison will trust a stained result more than a clean one, because the clean one might be a stain that was edited out. Make marketing inspectable is this idea pointed at campaigns. Point it at your own note. A stranger should be able to see the path from hypothesis to result without inheriting your optimism.
If you are tempted to skip a result because it makes the hypothesis look naive, write it shorter, not gone. Naive hypotheses are how the next one gets better. A template full of only confirmed hopes is a brochure. Brochures belong somewhere else. This site's experiments include an inconclusive one on purpose, so the section does not become a trophy case.
Repeat
Whether you would run it again, and what you would hold still next time. The answers are repeat, stop, or hold one piece still and change a named other piece. "Iterate" is not an answer. Iterate is how a bundle continues without a decision. Stopping is allowed, and it should be common. Most changes do not deserve a sequel. A sequel with three new tweaks is a new bundle wearing the old name.
Say what you will not change next time. That list is the discipline. If the next run changes the audience and the sentence, you have left the experiment. You may still do it. Call it a rewrite. The weekly review can ask which decision uses this note. If the answer is "we will broaden the line because the first screen dipped," check that you are not undoing the refusal to soothe a metric you did not commit to in the measure. The measure is the contract. The repeat section is where people try to renegotiate it.
| Part | A usable line | A line that breaks the template |
|---|---|---|
| Hypothesis | A person, a behavior, a direction you could fail | "This should perform better" |
| Change | One diff, and what stayed put | "We refreshed the page" |
| Measure | Chosen first, including "nothing happened" | A metric picked after the graph looked nice |
| Result | What happened, mess included | A moral with the numbers sanded off |
| Repeat | Repeat, stop, or one named next change | "We'll keep iterating" |
A note you can copy
Hypothesis. Who does what, in one sentence you could be wrong about.
Change. The single diff. What you held still.
Measure. What you will look at, the window, and what "nothing" means.
Result. What happened, including what you dislike and anything that contaminated the comparison.
Repeat. Repeat, stop, or the one thing you will change next while the rest stays.
That is the whole form. If you need a sixth section called "learnings," you are about to smuggle the sermon back in. Put the tentative lesson under result, in a sentence that does not outrun the evidence. Then stop writing. A short honest note can be compared. A long defensive one cannot. Length belongs in the essay that holds the belief. The experiment note stays a record.
How to file it
Put it in the experiments section with a status the site already understands: running, documented, or inconclusive. Running means the window is open and you should not narrate a victory. Documented means you are willing to stand behind the result as written. Inconclusive means the comparison did not hold, and you are not going to launder it with confidence. Status is not a mood. It is a promise to future comparison.
Link the belief it pressures and the decision it informs. A note that links to nothing will not be found when the next test needs a predecessor. The future log, if it ever exists, will only be as good as these links and these stable words. Do not rename the measure between notes because a new name sounds sharper. Sharp names that change are how history breaks. Use the boring word again.
If you do not have a measure yet, you are not late. You are early. Write the hypothesis in a journal line and wait. Shipping the change anyway is allowed as ordinary work. It is not allowed to borrow this template after the fact. The template is small so you will use it before, when it is still uncomfortable, which is the only time it helps.