[{"data":1,"prerenderedAt":540},["ShallowReactive",2],{"kc-/knowledge/sequential-testing-forms":3,"kc-clusters-/knowledge/sequential-testing-forms":225,"kc-related-/knowledge/sequential-testing-forms":226},{"id":4,"title":5,"author":6,"body":7,"date":184,"description":185,"draft":186,"extension":187,"faqs":188,"image":198,"imageAlt":199,"isPillar":186,"meta":200,"navigation":201,"path":202,"pillar":203,"pillarName":204,"seo":205,"sources":206,"stem":214,"tags":215,"takeaways":220,"updated":184,"__hash__":224},"knowledge/knowledge/sequential-testing-forms.md","Sequential Testing: Stop Form A/B Tests Early, Safely","RoundPushPin Team",{"type":8,"value":9,"toc":175},"minimark",[10,19,24,35,39,49,53,97,104,111,115,125,129,161,164],[11,12,13,14,18],"p",{},"Researchers face a constant tension when testing form variants: you want to act on a result the moment it's real, but stopping a normal A/B test early — the instant it looks significant — is one of the most reliable ways to fool yourself. Sequential testing resolves that tension by deciding the stopping rule ",[15,16,17],"em",{},"before"," you collect a single response.",[20,21,23],"h2",{"id":22},"why-is-it-risky-to-stop-an-ab-test-as-soon-as-it-looks-significant","Why is it risky to stop an A/B test as soon as it looks significant?",[11,25,26,30,31,34],{},[27,28,29],"strong",{},"Because checking for significance over and over and stopping at the first \"win\" inflates false positives."," Armitage, McPherson and Rowe (1969) showed that repeating a significance test on accumulating data pushes the chance of a false positive well above the nominal 5%. The more often you peek, the more likely random noise crosses your threshold at ",[15,32,33],{},"some"," point — so the \"winner\" you stop on is frequently a fluke.",[20,36,38],{"id":37},"what-is-sequential-testing","What is sequential testing?",[11,40,41,44,45,48],{},[27,42,43],{},"A test designed up front to be monitored as data arrives, with a stopping boundary built into the design."," Because the rule already accounts for the repeated looks, crossing the boundary is a valid reason to stop — unlike peeking at a fixed-horizon test. You trade a fixed end date for a fixed ",[15,46,47],{},"rule",", and in return you can often end far sooner.",[20,50,52],{"id":51},"how-does-evan-millers-simple-sequential-test-work","How does Evan Miller's simple sequential test work?",[11,54,55,58,59,62,63,67,68,71,72,74,75,78,79,74,82,85,86,89,90,74,93,96],{},[27,56,57],{},"Pick a sample size N from a power calculation, split traffic 50/50, count the successes in each arm, and watch the lead."," In Evan Miller's ",[15,60,61],{},"Simple Sequential A/B Testing",", you track ",[64,65,66],"code",{},"T − C"," (treatment successes minus control successes) and apply two rules: stop and declare the treatment a ",[27,69,70],{},"winner"," the moment ",[64,73,66],{}," reaches ",[27,76,77],{},"2√N","; if ",[64,80,81],{},"T + C",[27,83,84],{},"N"," first, stop and call it ",[27,87,88],{},"no winner",". For a two-sided test, stop when ",[64,91,92],{},"|T − C|",[27,94,95],{},"2.25√N",".",[11,98,99],{},[100,101],"img",{"alt":102,"src":103},"A running success lead climbing until it crosses the winning boundary at two times the square root of N, ending the test before the planned cap of N","/images/knowledge/diagrams/sequential-boundaries.png",[11,105,106,107,110],{},"Notably, the method rests on ",[27,108,109],{},"gambler's ruin"," (a random walk toward one of two boundaries), not the SPRT. It assumes nothing about the distribution of possible effects and ignores the number of failures in each group — which is what makes it simple enough to run with basic arithmetic.",[20,112,114],{"id":113},"how-much-faster-is-it-and-when-does-it-fall-short","How much faster is it — and when does it fall short?",[11,116,117,120,121,124],{},[27,118,119],{},"At low conversion rates it can cut the observations you need by half or more, because you stop the instant the lead is decisive."," The trade-offs Miller notes are real: the advantage shrinks above roughly 10% conversion, a test with ",[15,122,123],{},"no"," true effect can run longer than a fixed-size test, and the savings fade when the real effect is large (you'd have spotted it quickly anyway). The one rule you cannot break: don't peek or extend beyond the N and boundary you committed to.",[20,126,128],{"id":127},"how-to-run-a-sequential-form-test-in-roundpushpin","How to run a sequential form test in RoundPushPin",[11,130,131,134,135,138,139,142,143,145,146,151,152,156,157,160],{},[27,132,133],{},"Define a clear success — a completed form or a qualified lead — split visitors across two form branches, and track each arm's successes in your structured data."," Because every response lands as a typed row, counting ",[64,136,137],{},"T"," and ",[64,140,141],{},"C"," and computing the ",[64,144,77],{}," boundary is a simple query, and you stop the moment the lead crosses it. Pair this with ",[147,148,150],"a",{"href":149},"/knowledge/ab-testing-forms","how to A/B test forms"," for the setup and ",[147,153,155],{"href":154},"/knowledge/form-completion-rate","form completion rate"," for choosing the metric — and use the built-in heatmaps and drop-off analytics to understand ",[15,158,159],{},"why"," a variant wins, not just that it did.",[162,163],"hr",{},[11,165,166],{},[15,167,168,169,174],{},"This guide adapts the method from Evan Miller's ",[147,170,61],{"href":171,"rel":172},"https://www.evanmiller.org/sequential-ab-testing.html",[173],"nofollow"," (2015). See the original for the full derivation and the power-calculation formula for choosing N.",{"title":176,"searchDepth":177,"depth":177,"links":178},"",2,[179,180,181,182,183],{"id":22,"depth":177,"text":23},{"id":37,"depth":177,"text":38},{"id":51,"depth":177,"text":52},{"id":113,"depth":177,"text":114},{"id":127,"depth":177,"text":128},"2026-06-03","Sequential testing lets you end a form A/B test as soon as there's a real winner — without the peeking that inflates false positives. A researcher's guide.",false,"md",[189,192,195],{"q":190,"a":191},"Can I stop an A/B test early when it looks significant?","Not with an ordinary fixed-horizon test — stopping at the first significant result inflates false positives. A sequential test is designed for it: the stopping boundary already accounts for checking the data as it arrives, so an early stop stays valid.",{"q":193,"a":194},"What is the peeking problem?","Peeking is repeatedly checking an A/B test for significance and stopping at the first 'win'. Armitage and colleagues (1969) showed this pushes the false-positive rate well above the nominal level, so apparent winners are often flukes.",{"q":196,"a":197},"When is sequential testing not worth it?","Its savings are largest at low conversion rates. Above roughly 10% conversion the advantage shrinks, a truly null test can take longer than a fixed-size test, and you must not peek or extend beyond the rule you set.","/images/knowledge/sequential-testing-forms.png","A running success lead climbing until it crosses a sequential stopping boundary, ending the test early",{},true,"/knowledge/sequential-testing-forms","form-optimization","A/B testing & optimization",{"title":5,"description":185},[207,210],{"title":208,"url":171,"publisher":209},"Evan Miller — Simple Sequential A/B Testing (2015)","Evan Miller",{"title":211,"url":212,"publisher":213},"Armitage, P., McPherson, C. K. & Rowe, B. C. (1969) — Repeated Significance Tests on Accumulating Data","https://doi.org/10.2307/2343787","Journal of the Royal Statistical Society, Series A","knowledge/sequential-testing-forms",[216,217,218,219],"a/b testing","sequential testing","statistics","research",[221,222,223],"Stopping at the first significant result inflates false positives — the peeking problem (Armitage et al., 1969).","Sequential testing bakes the stopping rule into the design: stop when the success lead reaches 2√N (or 2.25√N two-sided), or call it off at N.","Biggest gains at low conversion rates — and never peek beyond the pre-set rule.","ujqMFo_8bF-JtQrfKgxgcwrXD-VBH-tjxeTXRjPQ-ls",[],[227,403],{"id":228,"title":229,"author":6,"body":230,"date":370,"description":371,"draft":186,"extension":187,"faqs":372,"image":382,"imageAlt":383,"isPillar":186,"meta":384,"navigation":201,"path":149,"pillar":203,"pillarName":204,"seo":385,"sources":386,"stem":394,"tags":395,"takeaways":398,"updated":370,"__hash__":402},"knowledge/knowledge/ab-testing-forms.md","How to A/B Test Forms (and Read the Results)",{"type":8,"value":231,"toc":363},[232,235,241,245,251,255,261,303,310,314,328,332,338,342],[11,233,234],{},"A/B testing a form means showing two versions of it to comparable, randomly assigned groups and measuring which one performs better — usually on completion rate. Done properly it tells you what actually works instead of what you assume works; done sloppily it produces confident, wrong conclusions.",[11,236,237],{},[100,238],{"alt":239,"src":240},"Incoming form traffic split into branch A and branch B, with completion measured on each to pick the winner","/images/knowledge/diagrams/ab-branches.png",[20,242,244],{"id":243},"can-you-ab-test-a-form","Can you A/B test a form?",[11,246,247,250],{},[27,248,249],{},"Yes — a form is well suited to A/B testing because it has a clear, measurable outcome: did the person finish it."," You split incoming respondents randomly between version A and version B, keep everything else equal, and compare completion. Random assignment is the core idea from controlled experiments (Kohavi, Tang & Xu, 2020): it's what lets you credit the difference to your change rather than to chance or to who happened to see which version.",[20,252,254],{"id":253},"what-should-you-ab-test-on-a-form","What should you A/B test on a form?",[11,256,257,260],{},[27,258,259],{},"Test one meaningful change at a time, so you can attribute any difference to it."," High-leverage things to test on a form:",[262,263,264,271,282,292,298],"ol",{},[265,266,267,270],"li",{},[27,268,269],{},"Length"," — fewer fields vs more (the change most likely to move completion).",[265,272,273,276,277,281],{},[27,274,275],{},"One question at a time vs all-on-one-page"," — the ",[147,278,280],{"href":279},"/knowledge/conversational-form-design","conversational format"," vs a classic layout.",[265,283,284,287,288,96],{},[27,285,286],{},"Question wording"," — since ",[147,289,291],{"href":290},"/knowledge/how-to-ask-the-right-questions-in-a-form","wording shapes answers and effort",[265,293,294,297],{},[27,295,296],{},"Question order"," — front-loading easy questions vs sensitive ones.",[265,299,300],{},[27,301,302],{},"The call to action and intro copy.",[11,304,305,306,309],{},"Changing several things at once is fine for shipping, but then you won't know ",[15,307,308],{},"which"," change caused the result.",[20,311,313],{"id":312},"how-do-you-read-ab-test-results","How do you read A/B test results?",[11,315,316,319,320,323,324,327],{},[27,317,318],{},"Compare the primary metric between variants and ask whether the difference is real or noise — using a significance test, not eyeballing."," Compute completion rate for each variant and a confidence interval or p-value; a gap that isn't statistically significant is not yet a result. The most common mistake is ",[15,321,322],{},"peeking"," — repeatedly checking and stopping the moment it looks significant — which dramatically inflates false positives (Evan Miller, \"How Not to Run an A/B Test\"). Decide your metric and stopping rule before you start, and read per-question drop-off too, so you can see ",[15,325,326],{},"where"," a variant helped or hurt.",[20,329,331],{"id":330},"how-long-should-you-run-a-form-ab-test","How long should you run a form A/B test?",[11,333,334,337],{},[27,335,336],{},"Long enough to reach the sample size you set in advance, and across full business cycles — not until it looks good."," Estimate the sample with a power calculation based on your baseline completion rate and the smallest improvement worth detecting; smaller effects need much larger samples. Run for whole weeks to avoid day-of-week bias, and avoid stopping early on an exciting-but-underpowered result.",[20,339,341],{"id":340},"how-roundpushpin-helps-you-test-and-read-forms","How RoundPushPin helps you test and read forms",[11,343,344,347,348,352,353,357,358,362],{},[27,345,346],{},"Because RoundPushPin stores responses relationally, the metrics an A/B test needs are already in the data — no tracking project required."," Completion rate and per-question drop-off come straight from the database with a ",[147,349,351],{"href":350},"/knowledge/query-form-data-with-sql","SQL query",", and because you can run ",[147,354,356],{"href":355},"/knowledge/one-template-many-versions","one master template in many versions",", standing up an A and a B variant is quick — see ",[147,359,361],{"href":360},"/features/ab-testing","RoundPushPin's A/B testing feature",". Structured data is what turns a form test from guesswork into a measurable experiment.",{"title":176,"searchDepth":177,"depth":177,"links":364},[365,366,367,368,369],{"id":243,"depth":177,"text":244},{"id":253,"depth":177,"text":254},{"id":312,"depth":177,"text":313},{"id":330,"depth":177,"text":331},{"id":340,"depth":177,"text":341},"2026-03-08","A/B testing a form means showing two versions to comparable groups to see which converts better. Learn what to test, how to read results, and how long to run.",[373,376,379],{"q":374,"a":375},"What is A/B testing for forms?","It's a controlled experiment: visitors are split randomly between two versions of a form, and you compare a metric — usually completion rate — to see which performs better. Random assignment is what lets you attribute the difference to the change rather than to chance or audience.",{"q":377,"a":378},"How big a sample do I need to A/B test a form?","Enough to detect the effect size you care about — smaller expected improvements need larger samples. Decide the sample size before you start using a calculator, and don't stop early just because a result looks significant; peeking inflates false positives.",{"q":380,"a":381},"What metric should I track for a form A/B test?","Usually completion rate (finishers ÷ starters), plus per-question drop-off to see where a variant helps or hurts. Pick one primary metric before the test so you're not cherry-picking afterward.","/images/knowledge/ab-testing-forms.png",null,{},{"title":229,"description":371},[387,391],{"title":388,"url":389,"publisher":390},"Kohavi, R., Tang, D., & Xu, Y. (2020) — Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing","https://doi.org/10.1017/9781108653985","Cambridge University Press",{"title":392,"url":393,"publisher":209},"Evan Miller — How Not to Run an A/B Test","https://www.evanmiller.org/how-not-to-run-an-ab-test.html","knowledge/ab-testing-forms",[216,396,397],"conversion","analytics",[399,400,401],"A/B testing splits the same traffic randomly between two or more form branches and compares completion.","Read results with a significance test and a sample size set in advance — never stop early on a peek.","Because RoundPushPin stores responses relationally, completion and drop-off come straight from your data.","PijemUrRKD26EpCTa0xaJvZqvKsM8SgCidHM5b4oL_w",{"id":404,"title":405,"author":6,"body":406,"date":503,"description":504,"draft":186,"extension":187,"faqs":505,"image":513,"imageAlt":383,"isPillar":186,"meta":514,"navigation":201,"path":515,"pillar":516,"pillarName":517,"seo":518,"sources":519,"stem":531,"tags":532,"takeaways":535,"updated":503,"__hash__":539},"knowledge/knowledge/building-trust-in-forms.md","How to Build Trust in Your Forms (So People Complete Them)",{"type":8,"value":407,"toc":496},[408,411,415,425,429,447,451,462,466,476,480],[11,409,410],{},"A form asks people to hand over their data, and people only do that for a site they trust. Trust isn't a nice-to-have on a form — it's a precondition for completion, and it's especially fragile the moment you ask for something personal.",[20,412,414],{"id":413},"why-does-trust-matter-for-form-completion","Why does trust matter for form completion?",[11,416,417,420,421,424],{},[27,418,419],{},"Because submitting a form is an act of trust, and doubt converts directly into abandonment."," When credibility is low, people hesitate, skip fields, or leave — and the effect is sharpest on sensitive questions, where distrust drives both refusals and inaccurate answers (Tourangeau & Yan, 2007). Earning trust isn't separate from conversion; it ",[15,422,423],{},"is"," part of conversion.",[20,426,428],{"id":427},"what-makes-a-form-look-trustworthy","What makes a form look trustworthy?",[11,430,431,434,435,438,439,442,443,446],{},[27,432,433],{},"The elements people notice, and the meaning they assign to them."," Fogg's ",[15,436,437],{},"Prominence-Interpretation Theory"," (2003) explains online credibility as a two-step process: a person has to ",[27,440,441],{},"notice"," an element (prominence), then ",[27,444,445],{},"interpret"," it as good or bad. So trust on a form is built from noticeable, positively-interpreted cues — a clean, professional design, a real organization clearly behind the form, plain language, and no jarring or excessive questions (Nielsen Norman Group). Sloppiness and surprises read as risk.",[20,448,450],{"id":449},"how-do-you-reassure-people-about-their-data","How do you reassure people about their data?",[11,452,453,456,457,461],{},[27,454,455],{},"Tell them what you'll do with it, why you're asking, and prove you ask for little."," Concretely: state the purpose in plain language, link a privacy notice near the submit action, keep the form ",[147,458,460],{"href":459},"/knowledge/what-to-ask-on-a-form","minimal",", and when you must ask something sensitive, explain why and place it late — after the person has invested effort. Transparency is what lowers the refusals that distrust causes (Tourangeau & Yan, 2007).",[20,463,465],{"id":464},"do-trust-signals-actually-change-behaviour","Do trust signals actually change behaviour?",[11,467,468,471,472,475],{},[27,469,470],{},"Yes — but only the ones people notice and believe."," Prominence-Interpretation Theory is a useful filter: a trust cue does nothing if it isn't noticed, and backfires if it's interpreted as hollow. Genuine signals (a real company, a clear privacy explanation, a short honest form) beat generic badges. Test which cues move ",[147,473,474],{"href":154},"completion rate"," for your audience rather than assuming.",[20,477,479],{"id":478},"how-roundpushpin-helps-you-earn-trust","How RoundPushPin helps you earn trust",[11,481,482,485,486,490,491,495],{},[27,483,484],{},"RoundPushPin supports trustworthy forms by default: clean conversational design, minimal relevant questions, and self-hosted data you genuinely control."," Because responses live in ",[147,487,489],{"href":488},"/knowledge/self-hosted-forms","your own database",", \"we keep your data private\" isn't a slogan — you decide where it lives and how long you keep it, which is the substance behind ",[147,492,494],{"href":493},"/knowledge/gdpr-compliant-forms","GDPR-compliant"," trust claims.",{"title":176,"searchDepth":177,"depth":177,"links":497},[498,499,500,501,502],{"id":413,"depth":177,"text":414},{"id":427,"depth":177,"text":428},{"id":449,"depth":177,"text":450},{"id":464,"depth":177,"text":465},{"id":478,"depth":177,"text":479},"2026-03-16","People won't hand data to a form they don't trust. A research-backed guide to how visitors judge credibility and the trust signals that matter on forms.",[506,508,510],{"q":414,"a":507},"Filling in a form means handing over data, which people only do when they trust the site. Low credibility raises hesitation and abandonment — and on sensitive questions, distrust increases refusals and misreporting.",{"q":428,"a":509},"Credibility comes from elements people notice and judge positively — clear design, a real organization behind it, plain language about why you ask, visible privacy/security cues, and no surprising or excessive questions.",{"q":511,"a":512},"How do you reassure people about their form data?","Tell them plainly what you'll do with it and why each question is asked, link a privacy notice, keep the form minimal, and place any sensitive question late with an explanation. Transparency reduces refusals.","/images/knowledge/building-trust-in-forms.png",{},"/knowledge/building-trust-in-forms","conversational-form-design","Conversational form design",{"title":405,"description":504},[520,524,527],{"title":521,"url":522,"publisher":523},"Fogg, B. J. (2003) — Prominence-Interpretation Theory: explaining how people assess credibility online","https://doi.org/10.1145/765891.765951","CHI '03 / Stanford Web Credibility Project",{"title":437,"url":525,"publisher":526},"https://www.nngroup.com/articles/prominence-interpretation-theory/","Nielsen Norman Group",{"title":528,"url":529,"publisher":530},"Tourangeau, R. & Yan, T. (2007) — Sensitive questions in surveys","https://doi.org/10.1037/0033-2909.133.5.859","Psychological Bulletin","knowledge/building-trust-in-forms",[533,534,396,219],"trust","credibility",[536,537,538],"People only submit data to a form they trust — low credibility raises hesitation and abandonment.","Credibility is what users notice and how they interpret it (Fogg's Prominence-Interpretation Theory).","Reassure with clear purpose, visible privacy cues, minimal asks, and sensitive questions placed late.","aHzdODBT56NeRa2qKSzusEDJwHzvD7N4RRNlxTbvIt0",1780930373816]