Risk-Based Regression for a Release Cycle That Didn’t Fit

Turning a 1,000-case manual regression into evidence, a tiered release strategy, and a capacity model, for a hospitality loyalty app on iOS and Android.

  • Mobile
  • Test Strategy
  • QA Lead
Full regression time, down from 7 days with 4 QAs
4 days · 2 QAs
Manual cases per full regression cycle
~1,000
Release strategy tiers, backed by a capacity model
4 tiers

Context

As QA Lead on a hospitality loyalty app (loyalty, rewards redemption, bookings and in-app engagement, backed by a BFF layer and several third-party integrations), I owned test strategy, regression planning, stakeholder reporting and release sign-off. The full regression suite had about 1,000 manual cases, while the client’s release calendar gave QA about two days per release, and the team was shrinking from four people to two.

The first full regression I led took 7 active days with 4 QAs. More people or longer windows weren’t an option, so the gap had to be closed by changing what we tested and how. Development carried on during release weeks, test data depended on the client, and a third-party integration changed mid-run.

What I did

I ran the first regression as a measurement exercise as much as a quality gate: daily tracking reports built from test-management exports, every blocked case tagged by cause, and in-run scope decisions. Of the 979 executable cases, 962 passed (98.3%), with 17 defects logged and the release approved with 2 minor known issues.

The data showed 10 of 19 sections produced zero defects, while four sections produced 12 of the 17 failures. From that I wrote a four-tier release strategy (build acceptance, change-impacted, core business-critical, and rotating regression) with a capacity model of about 36 cases per QA per day and two agreed staffing profiles. When the next major version triggered a full regression, broader UI automation and earlier handling of test data let 2 QAs finish 1,002 cases in 4 days: 968 passed, 1 defect, and 33 blocked, each with a stated cause.

Tools used:

  • TestRail
  • Jira
  • Maestro
  • Charles Proxy
  • Firebase/GA4
  • Slack/Teams

Key decisions

  • Cut scope where the defect data says it’s safe

    Sections with zero failures across a full run were the first candidates for rotation. Low-risk areas moved into a rotating tier tested once every four releases, while a fixed core of business-critical flows ran every time.

  • Make reduced coverage a shared decision

    The strategy states plainly what is no longer tested under each staffing profile (3 or 2 QAs), so coverage cuts are agreed with the client up front instead of being quietly absorbed by QA every release.

  • Separate a changing integration instead of blocking sign-off

    When a third-party integration changed mid-run, I moved its outdated cases into a dedicated effort after the regression rather than delaying the whole release.

  • Blocked with a cause, never just untested

    Every blocked case was tagged by cause (missing data, API state, API limitation, missing scenario), turning blockers into a list of actions for the client. Pass rates are reported both with and without blocked cases, because a single number hides cases nobody verified.

  • Full regression only on named triggers

    Full regression stays available, but only when one of six named triggers applies, such as a major version or new feature areas. Both platforms are tested only on flows that touch native code.

Outcomes

Full regression with half the team, down from 7 days with 4 QAs
4 days
Pass rate on executable cases in the first cycle, every blocker with a stated cause
98.3%
Risk-based release strategy and capacity model, designed from real defect data
4 tiers
Strategy, capacity model and reporting workflow adopted by the client’s team
Handed over