Quick answer
Beta testing is the stage where a nearly finished product or feature is released to a limited group of real users outside the development team, to find bugs, usability problems and missing pieces in real conditions before the general release.
What is beta testing?
Beta testing is the stage where a nearly finished product or feature is given to a limited group of real users outside the development team, before the general release. Testers use it on their own devices, with their own data and habits, and the team watches for bugs, confusing flows and missing pieces that internal testing did not reveal.
The word comes from IBM's naming of test phases in the 1950s: "A" for the internal test of a product, "B" for the test with outside users. Today "beta" is used loosely, from a two-week private test with 50 customers to a "public beta" label that stays on a product for years.
Alpha, beta, canary and general release
| Stage | Who uses it | Product state | Main question |
|---|---|---|---|
| Alpha | Internal team, employees (dogfooding) | Feature-incomplete, unstable | Does it basically work? |
| Closed beta | Invited external users | Feature-complete, rough edges | Does it work for real users in real conditions? |
| Open (public) beta | Anyone who opts in | Release candidate | Does it hold up at scale and across devices? |
| Canary release | A small share of normal users, without opting in | Production build | Does the new version break anything in production? |
| General availability | Everyone | Released | Is it delivering the expected outcome? |
Types of beta tests
- Closed beta. A hand-picked group, often existing customers or a waitlist, sometimes under NDA. Easy to talk to; small enough to support individually.
- Open beta. Anyone can join. Useful for load, device diversity and marketing buzz, but feedback is noisier.
- Focused (technical) beta. Tests one feature or one risk, such as a new payment provider or a rewritten sync engine.
- Marketing beta. Mostly about early adopters and word of mouth; the product decision has already been made.
How to run a beta test
- Write the goal and exit criteria first. What must be true to release? Decide this before you see any data.
- Recruit the right testers. Match the target users: platform, plan size, region. A beta full of power users will hide onboarding problems.
- Instrument the build. Make sure the beta build reports its version, crashes and the key events of the new feature, so you can separate testers from everyone else.
- Run it for two to six weeks. Ship fixes as new beta builds; tell testers what changed.
- Collect feedback in two channels. Structured (crash-free share, task completion, activation rate) and unstructured (in-app feedback, chat, short interviews).
- Decide. Release, extend the beta, or cut the feature. Write down why.
Exit criteria: a worked example
Sample data: a mobile app runs a four-week closed beta of a new offline mode with 400 testers on build 3.8.0-beta. The current release (3.7.2) is the comparison.
| Criterion (set in advance) | Target | Beta result | Pass? |
|---|---|---|---|
| Crash-free users (week 4) | ≥ 99.5% (current release: 99.6%) | 7 of 380 active testers crashed: 1 − 7 ÷ 380 = 98.2% | No |
| Testers who used offline mode at least once | ≥ 50% | 232 of 400 = 58% | Yes |
| Used it again in week 2 | ≥ 30% of week-1 users | 81 of 232 = 35% | Yes |
| Open P1 bugs | 0 | 2 | No |
The point of the table is the decision rule: two criteria failed, so the beta is extended by one build instead of being released on schedule, even though adoption of the new feature looks good.
App store testing tracks
- Apple TestFlight lets you invite up to 10,000 external testers by email or public link; external builds go through a beta app review, and each build is available for 90 days.
- Google Play Console offers internal testing (up to 100 testers, available within minutes), closed testing (lists or Google Groups) and open testing (anyone can join from the store listing).
Measuring a beta in TapCub
- Separate testers by build. App analytics › Users › Versions lists each version with its share of active users, crash count and first-seen date; click the beta version to filter the whole page to it. See track app versions.
- Compare stability. App analytics › Quality & attribution › Stability shows the crash-free share (1 − crash users ÷ active users), ANRs, freezes and start times; filter by version to compare the beta with the current release. See crash-free rate.
- Keep internal devices out. Exclusion rules for apps can match device or app version (with a trailing wildcard), so your own QA phones do not inflate the beta numbers.
- Measure activation of the feature with event analysis on the new feature's events, filtered to the beta version.
Sources
- Apple Developer, TestFlight.
- Google Play Console Help, Set up an open, closed or internal test.
Was this explanation clear?
We use your answers to fix and expand this page.
Couldn’t send. Please try again in a minute.
Thanks for the feedback. We’ll use it to improve this page.You’ve already sent feedback on this page. Thanks.
Technical Engineer
Focuses on tracking code, SDKs, and API integrations, with practical guidance on implementation checks and troubleshooting.