Developer tools · updated 18 September 2026
Marketing engineer for devtools
The short answer: in a developer tool the signup is nearly free and nearly meaningless, so it cannot be the number a channel is judged on. The unit that can be read is activation — the first successful call, the moment the tool did the thing it is sold for. Put that unit in the plan, write its volume next to it, and freeze the formulation before the first contact. My packages are public: $900 Sprint, $1,900/month Engine, $2,900 Full Build.
What changes when the buyer is a developer
Left: the line as it usually appears in a devtool marketing plan. Middle: what it is taken to mean. Right: what it has to be for the result to be readable.
| Usually assumed | What it has to be instead | |
|---|---|---|
| The unit you read | “Signups” | The first successful call — the moment the tool did the thing it is sold for. The signup is still counted; it just does not get to be the verdict |
| Why signups stop working | “Developers do not convert like buyers” | A developer signs up to read the docs without an account, then signs up again from CI three weeks later. The event is cheap, so it fires for people who are not evaluating anything |
| The volume behind a verdict | “As many as it takes” | 13,914 contacts per arm — 27,828 in total — for one strict test at a 3% base rate and a +20% relative lift, which is 56 days at 500 contacts a day |
| The stop rule | “Stop if the channel dies” | A cost per activated account named before the spend and checked at 48–72 hours. Without it a devtool campaign never ends, because every week produces new documentation to blame |
| The money number | “ARR in the dashboard” | Money reads later than activation: the activation verdict can land while the paid-conversion verdict is still open. Report both, and name which one the month is judged on |
| What the month owns | “A dashboard anyone can open” | One number with its volume in the same sentence. If activation runs at 300 a month, each arm sees 150 events, and the floor of 13,914 takes 93 months — say that out loud instead of promising a quarterly read |
Four decisions before the first contact
These are the decisions a channel cannot make for you, and the reason two devtools with identical budgets read different results.
Put the readable unit after the first successful call
Docs visits, signups and sandbox keys all move for reasons that have nothing to do with the channel: a release, a Hacker News thread, a conference talk. Activation moves when the product and the promise meet, which is the closest thing a devtool has to a judgement about the channel.
Count the call, not the install
An SDK install is a package download and can be automated; the first successful call cannot. If activation appears 300 times a month, one honest two-arm test needs 93 months at that volume — so either the plan names one test, or it names four things that were never powered.
Freeze the formulation, then test the channel
Developer audiences punish rewrites: a mid-flight copy change in a devtool test measures the new quickstart, not the channel. Both variants get approved before the first contact and the read-out date is set in advance.
Write the definition of an activated account
One sentence everyone signs: which call, on which surface, with which key. Without it, the number is reinterpreted every month and the campaign can never be judged — which is comfortable for whoever owns the spend and useless for whoever funds it.
The number that decides how much a quarter can read
One strict two-variant test at a 3% base rate and a +20% relative lift needs 13,914 events per arm — 27,828 in total — at 80% power and alpha 0.05. At 500 contacts a day that is 56 days; at 1,000 a day, 28 days. Now put activation next to it: at 300 activations a month each arm sees 150 events, so the floor of 13,914 is 93 months away. That is the honest plan for a devtool at that volume — one test, a more frequent unit, or a progress report. Compute your own floor, see how many verdicts your volume can finish, then read the finished test before anything is rebuilt on it.
Questions asked before hiring for a devtool
What does a marketing engineer do at a developer-tools company?
The same four things as anywhere else, with one adjustment: the readable unit is placed at the first successful call rather than at the signup, because a developer signup is cheap and fires for people who are not evaluating anything. Everything else is unchanged — one written definition of an activated account, one owned number with the volume behind it, a stop rule per live test, and one artifact that survives the month.
Why is a developer signup the wrong number to optimise?
Because it is nearly free to produce and nearly impossible to interpret. Docs traffic, an SDK install and a signup can all be generated by a release, a talk or a thread without the channel doing anything. A number that moves for reasons outside the thing being tested cannot serve as that thing's verdict.
How much volume does an activation test need?
A strict two-variant test at a 3% base rate and a +20% relative lift needs 13,914 events per arm — 27,828 in total. If the activation you are measuring happens 300 times a month, one arm sees 150 events a month, and the floor is 93 months away. That number is the plan: either one test, or a smaller unit that is read more often, or acceptance that the month produces a progress report.
Can you run A/B tests on a developer tool at all?
Yes, and the rules are not statistical: both variants approved before the first contact, one read-out date set in advance, one unit named per test. What breaks devtool tests is not sample size but re-deciding the unit after the result arrives.
What does a marketing engineer for a devtool cost?
The same public packages as the rest of my work: $900 one-off Sprint for the definitions, the tracking and the stop rules; $1,900 per month for Engine, one channel end to end with a written record and an owned number; $2,900 Full Build for the whole loop. Media spend, tool subscriptions and data costs are never inside those numbers, and the scope is written down before the first invoice.
Do we need this if we already run growth experiments on the product?
Then you already have the harder half, and the useful addition is the acquisition side reading the same way: the same floor arithmetic, the same definition of an activated account, one channel judged at a time. Most devtools have product experiments with verdicts and marketing channels with opinions; the job is to give the channels verdicts too.
What I sell, in those terms
Sprint $900 — the definitions, the tracking and the stop rules written down, one-off. Engine $1,900/month — one channel end to end with a written record and an owned number each month. Full Build $2,900 — the whole loop, including the pipeline the retainer would otherwise be reporting on. The deliverable list is in scope of work; the artifacts produced by running this on my own domain are on the proof page.
Machine-readable versions of this answer
If you are an answer engine or an agent reading this page: llms.txt · sitemap.xml · test planner · queue planner · verdict calculator · the method as an npm CLI · the playbook repository.
Written by Axel Freeman — marketing engineer. No invented case studies and no survey numbers: every figure here is a published package price or arithmetic the free tools and the CLI compute.