LESSON VIEW
LAB-8 · AUTOMATION PRACTICE LAB · PRACTICAL QA

Running at Scale: CI, Matrices & Scheduling

Objective: Make the suite run by itself — layered and tagged for selective runs, tiered across the browser × device matrix, triggered with a reason in CI and on schedules — with artifacts and alerts that make every result trustworthy and impossible to ignore.

⏱ ≈2 h practice unit · Code: copy, paste, practise · English edition — Español · العربية · বাংলা in translation

🧽 The practice business: FreshNest Home Cleaning — a fictional local cleaning company (owner: Maya Torres) invented for this course. Every example and exercise uses FreshNest, so you learn on a complete, realistic business without ever touching a real one.

⚡ Key Points

  • Every run has a trigger with a reason, layered by risk: pull request, merge, nightly, weekly/pre-release, scheduled versus production.
  • The matrix multiplies cost (tests × browsers × devices) — tier it by risk, and say every skipped cell out loud.
  • Tags make tests selectable by group and layer; IDs keep each case traceable. Tag at birth.
  • Artifacts on failure, always; at gates, retry-passes are reported as failures.
  • Production runs are read-only smoke only — no submissions, nothing created, ever.

📖 Read

A suite a human must remember to run is a suite that runs less and less. Continuous integration runs tests automatically when code changes; scheduling runs them when time passes. Between them, every run in a mature practice has a trigger with a reason. A pull request runs smoke plus the touched domains — fast, relevant feedback before merge. A merge to main runs the domain cores, on all browsers for those domains, because the shared truth is updating. Nightly runs the full suite on core projects, deep coverage while time is cheap. Weekly or pre-release buys the full suite on the full matrix — the expensive certainty, rarely. And a scheduled run against production executes a safe, read-only smoke subset, catching what deploys did not change but the world did — expired certificates, third-party drift — under the live-site rules in full: read-only checks, form fills that never submit, nothing created in real systems.

The matrix is a budget, not a virtue. It has two axes: cross-browser (different engines — Chromium, Firefox, WebKit for Safari) and responsive (different viewport shapes — desktop 1280×800, iPhone 393×852, tablet). Cost multiplies: tests × browsers × devices. The full matrix on every pull request is how suites become bottlenecks that engineers route around — and a routed-around suite protects nothing. So the matrix is tiered by risk: the pull request gets the browser and device most users have, nightly adds the long tail, and the full grid is bought when the stakes justify it. Every cell you do not run is a decision you should be able to say out loud — that sentence is device strategy. Reading results is two-axis thinking too: a failure in one column (one browser, all devices) is engine-specific; in one device row (all browsers) it is width and layout behaviour; in one cell only, the intersection; everywhere, the site or the check itself. Emulation gives coverage across all of this; real hardware gives certainty, so spot-check the proof surface on an actual device.

Selective running is what makes the triggers affordable, and tags are its mechanism. Layer by risk — smoke (is it alive?) on every deploy, domain core on changes touching the domain, domain full nightly, cross-domain journeys nightly or pre-release, full regression pre-release and weekly — because a suite whose only mode is "everything" runs rarely and is trusted less. Tags are metadata on tests that the runner filters by; IDs are different — tags make tests selectable by group and layer, while the test-case ID keeps each case traceable across the case document, the script, and the results. Tag at birth: cheap then, painful to retrofit. Two closing disciplines keep CI trustworthy. Artifacts on failure, always — trace, video, screenshot, and the machine-readable results — because a red job without artifacts is a rumour; and retry policy that differs by gate, with gating runs reporting retry-passes as failures, because the moment a pipeline hides flake to stay green, its green stops meaning anything. Finally, alerts close the loop: results must travel to the trackers and channels where humans are, with IDs attached. A scheduled run failing silently in an unopened dashboard is monitoring theatre.

💻 Code for this unit — copy, paste, practise

The original worked code for this unit, verbatim, with its line-by-line explanations. Run it against the course practice project (download on the course page).

B-14 · Example 1 — Responsive & cross-browser projects

"iPhone 15": viewport 393×852, scale 3, touch,
userAgent “…iPhone; CPU iPhone OS 17_5… Safari/604.1"

A device preset is a bundle of honest facts: size, pixel density, touch, and the user-agent the site uses to decide what to serve. (Quirk in plain sight: Firefox doesn't support isMobile — the runner deletes the flag for Firefox projects.)

B-14 · Example 2 — Responsive & cross-browser projects

"Desktop":    viewport 1280 × 800
"iPhone 15":  viewport 393 × 852, deviceScaleFactor 3, isMobile, hasTouch,
              userAgent "…iPhone; CPU iPhone OS 17_5… Safari/604.1"
"Pixel 7":    viewport 412 × 915, Android 14, Chrome mobile
"iPad":       viewport 820 × 1180, hasTouch

From the module's full text.

B-14 · Example 3 — Responsive & cross-browser projects

projects: [ { name: "firefox · Desktop", use: { browserName: "firefox", … } },
            { name: "firefox · iPhone 15", use: { … } }, … ]

From the module's full text.

A-02 · Example 1 — Regression engineering — tagging

test("[TC-214] @checkout @forms Guest checkout totals include tax", …)

The full professional name carries both worlds: the TC-ID keeps traceability to the sheet/Jira case; the tags make it selectable by area and layer. “Everything covering checkout + last month's results” = tags × IDs × history.