Open Source A/B Testing Tools: GrowthBook and More

An open source A/B testing tool delivers the same core job as any experimentation platform (variation assignment, conversion collection, a statistical verdict), but with source code available for inspection and, in most cases, the option to run the whole stack on your own infrastructure instead of a vendor’s. This guide compares the four names that come up most often in that category, GrowthBook, PostHog, Unleash, and Flagsmith, on the criteria that actually decide fit for a real team: self-hosting weight, license terms, and whether the tool has a genuine statistics engine or just routes traffic. For the two most-cited names head-to-head, see our GrowthBook vs Statsig comparison; for the full landscape including proprietary platforms, see the complete guide to CRO tools compared by category.

Quick overview

Tool	Primary focus	Self-hosted	Managed cloud	Native statistics engine	Core license
GrowthBook	Feature flags plus experimentation	Yes, light stack (app plus MongoDB)	Yes, GrowthBook Cloud	Yes: Bayesian by default, frequentist and sequential available	MIT on the core, separate Enterprise layer
PostHog	Product analytics with experimentation built in	Yes, heavy stack (ClickHouse, Kafka, Postgres, Redis)	Yes, PostHog Cloud	Yes: Bayesian by default, frequentist available	Open source in part, separate paid layer
Unleash	Feature flags first	Yes, light stack (app plus PostgreSQL)	Yes, managed edition	Limited: basic A/B routing, no dedicated statistics engine	AGPL-3.0-or-later on the core
Flagsmith	Feature flags plus remote config	Yes, light stack (app plus PostgreSQL, Docker Compose)	Yes, managed edition	Limited: multivariate flag routing, math left to external analytics	BSD-3-Clause on the core, separate Enterprise Edition

The rest of this guide walks through why a team picks this path at all, what each tool genuinely solves, how heavy each one is to operate, and how to verify statistical rigor yourself before trusting any tool’s verdict.

Why a team chooses open source over proprietary

Four reasons show up repeatedly when a team picks the open source path instead of a closed, proprietary platform.

Data control. When you self-host, visitor behavior data (who saw which variation, who converted) never leaves your own infrastructure to reach a third-party vendor’s server. That matters under LGPD and GDPR: keeping data inside your own environment simplifies control over where it physically resides and reduces the surface shared with outside parties. The caveat is real: this is not automatic compliance, it is one piece of the puzzle. The rest of the legal obligations, lawful basis, minimization, retention, access security, stay your responsibility regardless of where the server sits.

No vendor lock-in. Migrating between two closed, proprietary platforms usually means rebuilding the integration from scratch, because the data format and the API are both proprietary. With an open source tool, the code is there to read, export, or even fork if the vendor changes direction in a way that no longer serves your team.

Code-level customization. Engineering teams with a specific need (a different bucketing algorithm, an internal integration, a non-standard compliance requirement) can change the behavior directly in the code, something impossible on a closed platform, where the only option is requesting a feature and waiting on the vendor’s roadmap.

Zero license cost, not zero cost. The core code costs nothing to run, but it does not run itself: somebody has to provision the server, the database, backups, monitoring, and the engineering time to keep the version current. Trading license cost for operational cost is a genuine trade, not an automatic saving, and the rest of this guide addresses exactly that trade-off.

The tools, one by one

GrowthBook started as a feature flagging and experimentation platform without trying to be anything beyond that. The statistics engine is Bayesian by default, with the option to switch to frequentist or sequential testing, and self-hosting is relatively light: the main application plus a MongoDB database, usually brought up with Docker Compose. It is the option cited most by dev-first teams that already have, or plan to build, their own data warehouse and want a dedicated experimentation engine sitting on top of it.

PostHog started as an open source product analytics suite and grew to fold experimentation in as one module inside a much wider product, alongside funnels, session recording, surveys, and feature flags. Its experimentation engine is also Bayesian by default with a frequentist option, and it runs on the same event pipeline that feeds the rest of the suite. Self-hosting is the heaviest operation of the four tools in this guide: the full stack includes ClickHouse, Kafka, Postgres, and Redis, and the company’s own documentation recommends the cloud version for most production use cases.

Unleash is feature flags first, full stop: the product is built to reduce the risk of shipping new functionality (gradual rollout, canary release, kill switch), not to call A/B test winners. It has some A/B testing support, but that means routing a fraction of traffic to different variations of a flag, with no dedicated statistics engine calculating significance behind it. Self-hosting needs a Node.js server plus a PostgreSQL database, a fairly lean stack. One licensing detail worth flagging: the core repository is under AGPL-3.0-or-later, a stricter copyleft license than the MIT license on GrowthBook’s core or the BSD-3-Clause license on Flagsmith’s core, which can matter depending on how your team plans to modify or redistribute the code.

Flagsmith is also feature flags first, with remote config and multivariate flags as the path to testing variations. Like Unleash, it has no native significance engine: the documented pattern is to use a multivariate flag to route traffic and send the conversion event to an external analytics tool (Amplitude, Mixpanel, or similar) that calculates the result. Self-hosting is light: Docker Compose brings up the application and a PostgreSQL database together, without the streaming infrastructure PostHog requires. The core is licensed under BSD-3-Clause, a permissive license, with Enterprise Edition features (RBAC, SSO/SAML) kept separate.

Self-hosting trade-offs: what the “free” banner leaves out

The core code of any of the four tools above is free to run, but self-hosting trades license cost for operational cost, and that second cost rarely shows up in a quick comparison chart. A genuinely healthy self-hosted deployment needs, at minimum, dedicated time to provision and maintain the server and the database, security patches applied as soon as they ship (exposed databases and proxies are a constant target), version upgrades that do not break experiments already in flight, and uptime monitoring, because an outage in the flagging or experimentation service can freeze variation assignment for real visitors mid-test.

That weight is not equal across the four tools. GrowthBook, Unleash, and Flagsmith need a fairly lean stack (an application plus a single relational database), the kind of thing that comes up via Docker Compose in a few minutes and that a small engineering team can maintain without it becoming a full-time job. PostHog needs a much heavier data operation, because it is built to process high event volume: ClickHouse, Kafka, Postgres, and Redis, among other components, which is why the company itself recommends the cloud version for most teams running it in production.

Tool	Minimum self-hosted stack	Operational weight	Core license
GrowthBook	Application plus MongoDB	Light to medium	MIT (Enterprise layer separate)
PostHog	Application plus ClickHouse plus Kafka plus Postgres plus Redis	Heavy	Open source in part
Unleash	Node.js application plus PostgreSQL 15 or newer	Light	AGPL-3.0-or-later
Flagsmith	Application plus PostgreSQL (Docker Compose)	Light	BSD-3-Clause (Enterprise layer separate)

The managed cloud version of the same tool (GrowthBook Cloud, PostHog Cloud, or the managed editions of Unleash and Flagsmith) removes that operational load from your team, but trades it for a recurring subscription and, in most cases, data leaving your infrastructure for the vendor’s, the same model as any proprietary SaaS. Neither path is absolutely “better”: it is a choice between spending engineering time or spending recurring budget, and between keeping data in-house or accepting that it sits with a third party.

Statistical rigor in open source tools: verify before you trust

On a mature, expensive proprietary platform, the statistics engine tends to be a product tested over years by thousands of paying customers, leaving little room for the interface to diverge from what the math actually computes. In the open source world that guarantee is less uniform: every project has its own team, its own review pace, and its own choice of method, and engine depth varies far more from tool to tool than it typically does between similarly sized proprietary competitors.

That shows clearly across the four tools in this guide. GrowthBook and PostHog have a dedicated experimentation engine built specifically to compute significance, with both Bayesian and frequentist options documented. Unleash and Flagsmith do not have that piece natively: they are excellent at reliably assigning traffic to variations, but the calculation of whether a difference is real or noise typically happens in a connected analytics tool outside the flagging product itself. Using Unleash or Flagsmith to run an A/B test without also having a serious statistical analysis layer behind it is the most common mistake: the flag works perfectly, but nobody confirmed the sample was large enough or that the result is not peeking dressed up as a win.

The practical recommendation, for any of the four: do not blindly trust the verdict the interface shows, especially on a less mature tool or one with a smaller maintenance team. Take the same visitor and conversion counts the tool used and reproduce the calculation independently. The calculator below, from our statistical significance in A/B testing guide, does exactly that: paste the visitors and conversions for each variation and compare the p-value, the lift, and the verdict against what your open source tool reported for the same experiment.

Control (A)

VisitorsConversions

Variation (B)

VisitorsConversions

Confidence90%95%99%

Control (A) · Rate-

Variation (B) · Rate-

Relative lift-

p-value-

95% CI of the difference-

Two-sided two-proportion z-test. "Not significant" almost always means not enough sample, not that the versions are equal.

A worked example, computed end to end

Run the same numbers that ship as the calculator’s own default example: a control (A) with 210 conversions out of 4,200 visitors, and a variation (B) with 273 conversions out of 4,200 visitors. This is the exact math behind the calculator, not an invented result.

• Rate of A: 210 divided by 4,200 equals 5.00 percent. Rate of B: 273 divided by 4,200 equals 6.50 percent.
• Relative lift: (6.5 minus 5.0) divided by 5.0 equals plus 30 percent. That looks great on its own, but the question is whether it is real.
• Pooled rate: (210 plus 273) divided by 8,400 equals 5.75 percent.
• z-score from the two-proportion test: approximately 2.95.
• Two-sided p-value: approximately 0.0031.
• 95 percent confidence interval on the difference: roughly 0.5 to 2.5 percentage points.

With a p-value of about 0.0031, well under the conventional 0.05 threshold, the result is significant and B wins, and even the low end of the confidence interval stays positive, meaning B still comes out ahead in the worst plausible case. If any open source tool reports a materially different p-value or interval for these exact inputs, that is a signal to check its documentation or open an issue, a luxury that exists specifically because the code is open.

How to choose: explicit selection criteria

Skip the marketing copy and pick by criteria that actually predict whether the tool fits your team.

Criterion	Question to ask	Why it matters
Primary job	Are you calling A/B test winners, or rolling out features safely?	GrowthBook and PostHog are built for the former; Unleash and Flagsmith are built for the latter
Statistics engine	Does the tool compute significance natively, or does that live in a separate analytics tool?	A native engine means one less integration to keep in sync and trust
Operational weight	Can your team realistically run this stack at 2 a.m. on a Friday?	PostHog’s data stack is meaningfully heavier than the other three
License terms	Is the core license permissive (MIT, BSD-3-Clause) or copyleft (AGPL)?	Matters if you plan to modify and redistribute, or embed the code in a commercial product
Data residency needs	Does your compliance posture require the data to never leave your own infrastructure?	If yes, self-hosting anything beats any managed cloud option by default

When it makes sense to skip open source

Open source is not the default right answer for every team. The proprietary managed path is worth considering when at least one of these is true:

• No real DevOps bandwidth. If nobody on the team can confidently answer “who patches the database at 2 a.m. on a Friday,” self-hosting any of these four tools means taking on operational risk nobody is prepared to carry.
• The urgency does not allow for an engineering sprint. An A/B test that needs to start tomorrow does not pair well with provisioning a server, configuring a database, testing backups, and validating an SDK before the first experiment runs.
• Traffic volume does not justify running your own infrastructure. For a small or mid-size site, the engineering cost of keeping a stack alive, in salary and attention, frequently exceeds what a managed subscription would charge for the same volume.
• The team would rather not become a database operator. Even the lightest options (GrowthBook, Unleash, Flagsmith) need someone willing to learn to run Docker, PostgreSQL, or MongoDB in production; that is a real choice about where to invest engineering time, not a minor footnote.

In those cases, the managed cloud tier of the same open source tool (GrowthBook Cloud, PostHog Cloud) already solves most of the problem without abandoning the product. And a fully managed proprietary platform remains the most direct option for a team that wants to run a test without thinking about infrastructure at all.

Automate this with Donnu

Choosing open source usually starts from wanting to avoid an expensive or opaque vendor, and stalls on the weight of standing up and maintaining a self-hosted stack: server, database, patching, backups, monitoring. Donnu is worth putting on the same shortlist as one more option: a managed SaaS with no infrastructure of your own to operate, honest Bayesian statistics, and per-account data isolation from day one. It is not a replacement for product-style feature flagging like Unleash or Flagsmith, nor for PostHog’s full analytics suite, and it is not the right pick for a team that genuinely needs to modify source code or keep everything inside its own infrastructure by internal requirement. But for a team that wants to stop weighing Docker Compose against a monthly subscription and just run the test, it is a leaner managed alternative to an enterprise contract.

Start a free trial and see whether skipping self-hosting is worth it for your case, especially if your team’s engineering time is scarcer than its budget.

Read also

See the GrowthBook vs Statsig comparison for a decision framework between two of the most-cited names in this space, the complete guide to CRO tools compared by category for the full landscape including proprietary options, the statistical significance in A/B testing guide for the full mechanics behind any Bayesian or frequentist engine, and the best free A/B testing tools in 2026 to separate open source from a trial disguised as free. Leia em português: ferramentas de teste A/B open-source.

References

• GrowthBook. Official documentation and licensing. docs.growthbook.io and github.com/growthbook/growthbook
• PostHog. Official documentation and self-hosting guide. posthog.com/docs
• Unleash. Official documentation and open source repository. docs.getunleash.io and github.com/unleash/unleash
• Flagsmith. Official documentation, licensing, and self-hosting guide. docs.flagsmith.com and github.com/Flagsmith/flagsmith