Experiments are already a section of the site. The log is the stricter version of that section: one shape, every time, so two tests can be read side by side without translating them first. This page is an idea, not a tool. The status says so. I am writing the idea down so I do not accidentally build it early, and so I do not forget the reason if I build it late.
The pain it would solve is comparison. I can already write a hypothesis and a result. I cannot yet lay three tests on a table and see, without rereading the prose, what changed and what was held still. Prose is the right medium for a single test. It is a bad medium for a row. A log is a row. Until I have enough rows to suffer, a log is a fantasy of rigor.
A log is only useful if the names stay stable. A new name for the same measure is how a history becomes incomparable on purpose.
The shape of a note
The shape is the experiment note. Hypothesis, change, measure, result, and whether to repeat it. The pages already carry hypothesis and result. The log would add the missing discipline of a measure you choose before you look at the outcome. Choosing late is how a test becomes a story. Stories are allowed in the journal. They are not allowed to wear the word experiment.
Hypothesis is one sentence about a person, not about a metric. "A refusing line will lose some of the first screen and keep more of the right readers" is a hypothesis. "Engagement will improve" is a wish. Change is one alteration. If the headline and the offer move together, you have a bundle, and a bundle cannot teach. Measure is the thing you promised to look at, including the result that means nothing happened. Result includes the result you dislike. Repeat is a decision, not a vibe. You run it again, you stop, or you hold something still and change one named piece.
I have three experiments on the site already, and they show why the shape matters more than the tool. One-sentence positioning is documented. Plain-language subject lines is inconclusive, on purpose, because the topics differed and the sample was small. Writing in public is running, and its measure is a traceable decision, not traffic. A log that forced all three into a single success score would be a worse artifact than the prose. The log has to be able to say inconclusive without treating it as a blank to fill later.
Why it stays an idea
I am not building a separate tool yet. The section is the prototype. If the notes stay inconsistent, a tool will not fix them. If they stay consistent for a while, a small log might be worth making so the comparison does not depend on memory. Memory is generous to the test that confirmed a hunch and vague about the test that did not. A file with a date is less generous. I want less generosity.
The failure mode I am avoiding has a familiar shape. You design a database of experiments. You add fields for owner, channel, confidence, and impact. You backfill two old tests badly. You stop writing new ones because the form is longer than the learning. The section dies, and the tool remains as evidence that you care about experimentation. Evidence of caring is not a result. Activity is not learning applies to builds. A schema is activity until a comparison changes a decision.
There is a smaller failure too. You build the log and then only record the tests that fit it. A weird, useful observation has no field, so it goes in a chat and disappears. The prose section can hold a weird observation inside a stable shape. A rigid app often cannot, unless someone with time keeps extending it. I do not want a second product to maintain. This notebook is already the build in progress. The log has to earn its place beside it, not instead of the habit.
| Signal | Keep writing notes | Consider a log |
|---|---|---|
| Shape | Each note invents its sections | Five notes use the same five parts |
| Names | "Interest" means a new thing each time | The same measure name survives a month |
| Comparison | You reread to remember the last test | You already want a row and the prose is in the way |
| Status | Everything sounds successful | Inconclusive and running are first-class |
| Decision | The note ends in a moral | The note ends in repeat, stop, or hold still |
What the log is not
It is not a dashboard of marketing performance. Performance dashboards answer "how is the channel." A log answers "what did we change, and what did we learn from that change." Mixing them makes every fluctuation look like a test. Most fluctuations are not tests. They are the world. Calling them tests lets you skip the hypothesis. I will not add a chart to this idea. A chart without a prior measure is decoration.
It is not a backlog of ideas to try. Ideas belong in a sentence until one of them is actually running. A backlog of forty experiments is a wish list with scientific stationery. The running test should be visible because it is rare. If everything is an experiment, nothing has a control, and the word stops meaning a comparison. I would rather three honest notes than a board full of cards named "test."
It is not a place to hide uncertainty with precision. Fake sample sizes, invented conversion lifts, and decimal points on a handful of readers would make the log look mature and make it a lie. The subject-line note stays inconclusive because the evidence is weak. A log that required a percentage would pressure me to type one. I am refusing that pressure in the design, before the design exists. The fields have to allow words like "too small to call" and "topics differed."
Names that have to stay still
Related: make marketing inspectable. A log is only useful if the names stay stable. "Signup" and "joined" and "new lead" cannot rotate across rows and still be a column. The stability is a writing problem first. I need a short glossary I actually obey: what a first screen means, what a right reader means, what a decision means. If those words drift, the log will compare puns.
Stable does not mean eternal. A measure can be retired. Retirement should be a dated note, not a silent rename. Silent renames are how teams win arguments with history. I would rather a column that ends than a column that quietly changes species. The future log, if it exists, should show the end. A blank cell is more honest than a converted one.
This is also why I do not want the log to import numbers automatically from an analytics tool. Automatic imports feel rigorous and smuggle in whatever definition the tool used that week. A hand-copied number, with the definition written beside it, is slower and harder to fudge by accident. When the volume is large enough that hand copying is foolish, the definition should still be written by a person. The pipe can move the digit. It should not invent the meaning.
The smallest version I would accept
If the notes stay consistent, the smallest log is not an application. It is a table that links to the notes. One row per test. Columns for the date, the single change, the measure, the status, and the decision to repeat or stop. The prose remains the source. The table is an index. Indexes are allowed to be dumb. The moment the index becomes the place you edit the truth, you have two sources, and they will disagree.
I might keep that table as a page in this section before I ever touch a database. A page can be wrong in public, which is a useful pressure. A private spreadsheet will drift toward the numbers that flatter the quarter. Public rows have to survive a reader who was not in the meeting. That reader is the design constraint. Write the decision, not the meeting, applies here too. The row is the decision about the test. The narrative of how we felt during the test can stay in the essay or be cut.
Until that table has five real rows, I will not design a screen for it. Five is arbitrary and still better than zero. Arbitrary brakes are how ideas stay ideas long enough to deserve code. If I never reach five consistent notes, the log was a fantasy, and this page can remain the record of a fantasy I did not fund. That outcome would be a successful use of the page.
What would make me delete the idea
I would shelve the idea if the experiments section is doing the comparison well enough that a second artifact adds no decision. I would also shelve it if maintaining the shape requires nagging I will not keep up. A practice you nag yourself into for a month and drop is not a build. It is a mood. Moods do not get repositories.
I would build it if I catch myself misremembering a prior test in a way that changes a new one. That misremembering is the bug. The log is a patch for that bug, and only for that bug. Not for looking scientific. Not for a portfolio of rigor. For a row I can trust more than my memory of last month. Until the bug bites, the section is the log, and this page is the reminder not to skip ahead.