|
| 1 | +--- |
| 2 | +description: Consolidate one or more agentic test iteration branches into a clean, shippable fix branch — analyzes overlaps, merges intelligently, verifies with flakiness probing, and pushes |
| 3 | +parameters: |
| 4 | + - name: branch-name |
| 5 | + description: "Target branch name (e.g. test/OBSINTA-1290-incident-stability). Created from main." |
| 6 | + required: true |
| 7 | + type: string |
| 8 | + - name: source-branches |
| 9 | + description: "Comma-separated list of iteration branch names or a glob (e.g. 'FIX-PROPOSAL-AGENTIC/*' or 'branch-a,branch-b,branch-c'). Fetched from origin." |
| 10 | + required: true |
| 11 | + type: string |
| 12 | + - name: flakiness-runs |
| 13 | + description: "Number of flakiness probe runs after fixes are applied (default: 3)" |
| 14 | + required: false |
| 15 | + type: string |
| 16 | + - name: test-target |
| 17 | + description: "Cypress test target — 'all', 'regression', a spec file, or grepTags pattern (default: all)" |
| 18 | + required: false |
| 19 | + type: string |
| 20 | +--- |
| 21 | + |
| 22 | +# Productize Agentic Test Iterations |
| 23 | + |
| 24 | +Consolidate one or more agentic iteration branches into a single clean, shippable branch. |
| 25 | +When multiple branches exist, analyze whether their changes are complementary, redundant, |
| 26 | +or conflicting — then merge them intelligently rather than blindly stacking cherry-picks. |
| 27 | + |
| 28 | +## Prerequisites |
| 29 | + |
| 30 | +- `web/cypress/export-env.sh` must exist with cluster credentials. |
| 31 | + If missing, create it from cluster credentials in the conversation (do NOT run `/cypress:setup`). |
| 32 | +- Node modules installed in `web/` (`npm install`). |
| 33 | + |
| 34 | +## Step 1: Discover and Fetch Source Branches |
| 35 | + |
| 36 | +Parse `$2` (source-branches): |
| 37 | +- If it contains `*`, treat as a glob: `git ls-remote origin | grep <pattern>` |
| 38 | +- If comma-separated, split into a list |
| 39 | + |
| 40 | +Fetch all source branches as remote tracking refs: |
| 41 | +```bash |
| 42 | +git fetch origin "<branch>:refs/remotes/origin/<branch>" |
| 43 | +``` |
| 44 | + |
| 45 | +If any source branch has an overview/index document (e.g. `docs/agentic-fix-proposals.md`), |
| 46 | +read it first — it may describe the branches and their relationships. |
| 47 | + |
| 48 | +Report what was found: |
| 49 | +``` |
| 50 | +Found N source branches: |
| 51 | + - origin/<branch-1> — N commits above main |
| 52 | + - origin/<branch-2> — N commits above main |
| 53 | +``` |
| 54 | + |
| 55 | +## Step 2: Analyze Each Branch |
| 56 | + |
| 57 | +For each source branch, extract: |
| 58 | + |
| 59 | +1. **Commits above main**: `git log origin/main..<branch> --oneline` |
| 60 | +2. **Files changed**: `git diff origin/main..<branch> --stat` |
| 61 | +3. **Commit messages and descriptions**: Read each commit message for intent |
| 62 | + |
| 63 | +Build a structured summary per branch: |
| 64 | +``` |
| 65 | +Branch: <name> |
| 66 | +Origin: <how it was created — CI loop, local sandbox, manual> |
| 67 | +Commits: N |
| 68 | +Files touched: <list> |
| 69 | +What it fixes: |
| 70 | + - <description of each fix with classification> |
| 71 | +``` |
| 72 | + |
| 73 | +## Step 3: Evaluate Overlaps |
| 74 | + |
| 75 | +This is the critical step. For each pair of branches that touch the **same file**: |
| 76 | + |
| 77 | +1. **Read both versions** of the overlapping file |
| 78 | +2. **Identify the relationship**: |
| 79 | + - **Complementary**: Different fixes to different parts of the same file (both needed) |
| 80 | + - **Progressive**: One branch's changes are a superset of another's (keep the latest) |
| 81 | + - **Redundant**: Both branches solve the same problem differently (pick the better one) |
| 82 | + - **Conflicting**: Changes that are incompatible (requires manual resolution) |
| 83 | +3. **Decide the merge strategy** for each overlap |
| 84 | + |
| 85 | +Also check for cross-file dependencies: |
| 86 | +- Does branch A add a function that branch B calls? |
| 87 | +- Does branch A change a signature that branch B depends on? |
| 88 | + |
| 89 | +Produce an evaluation report: |
| 90 | +``` |
| 91 | +## Overlap Analysis |
| 92 | +
|
| 93 | +### <file-path> |
| 94 | +Touched by: branch-1, branch-3 |
| 95 | +Relationship: COMPLEMENTARY |
| 96 | + - branch-1: adds OOM prevention (cy.reload, quiet search) |
| 97 | + - branch-3: adds warmUpForPlugin() method |
| 98 | +Strategy: Apply both — they modify different sections |
| 99 | +
|
| 100 | +### <file-path> |
| 101 | +Touched by: branch-1, branch-2 |
| 102 | +Relationship: PROGRESSIVE |
| 103 | + - branch-1: adds cy.reload() to search loop |
| 104 | + - branch-2: rewrites search loop with quiet search (includes reload) |
| 105 | +Strategy: Take branch-2 only — it's a superset |
| 106 | +
|
| 107 | +## Proposed Commit Structure |
| 108 | +1. <logical-group>: <description> — from branches X, Y |
| 109 | +2. <logical-group>: <description> — from branch Z |
| 110 | +``` |
| 111 | + |
| 112 | +**Present this evaluation to the user and wait for confirmation before proceeding.** |
| 113 | + |
| 114 | +## Step 4: Create Clean Branch |
| 115 | + |
| 116 | +```bash |
| 117 | +git checkout -b $1 origin/main |
| 118 | +``` |
| 119 | + |
| 120 | +If the branch already exists, ask the user whether to overwrite or append a suffix. |
| 121 | + |
| 122 | +## Step 5: Apply Changes |
| 123 | + |
| 124 | +Based on the evaluation from Step 3, apply changes in logical order. |
| 125 | + |
| 126 | +**Do NOT blindly cherry-pick individual commits.** Instead: |
| 127 | + |
| 128 | +1. For **progressive** overlaps: only apply the most complete version |
| 129 | +2. For **redundant** overlaps: apply the one chosen in Step 3 |
| 130 | +3. For **complementary** changes: apply in dependency order (e.g., page object before tests that use it) |
| 131 | +4. For **conflicts**: manually merge, reading both versions and combining intent |
| 132 | + |
| 133 | +After applying each logical group: |
| 134 | +- Run `npx prettier --write <changed-files>` immediately |
| 135 | +- Run `npx eslint --fix <changed-files>` |
| 136 | +- Check for remaining lint errors with `npx eslint <changed-files>` |
| 137 | +- Fix any remaining max-len or formatting issues manually |
| 138 | + |
| 139 | +**Do not defer lint fixes to a separate commit.** Each commit must pass the pre-commit hook. |
| 140 | + |
| 141 | +## Step 6: Structure Commits by Logical Concern |
| 142 | + |
| 143 | +Group changes into commits by **what problem they solve**, not by which branch they came from. |
| 144 | +Good commit groupings: |
| 145 | +- One commit per distinct fix category (OOM prevention, plugin warm-up, test hygiene, etc.) |
| 146 | +- Page object changes bundled with the test changes that use them |
| 147 | +- Fixture additions bundled with the tests that reference them |
| 148 | + |
| 149 | +Bad commit groupings: |
| 150 | +- One commit per source branch (archaeology, not intent) |
| 151 | +- Intermediate steps that are immediately superseded |
| 152 | +- Separate "fix lint" commits |
| 153 | + |
| 154 | +Commit message format: |
| 155 | +``` |
| 156 | +fix(tests): <summary of what problem this solves> |
| 157 | +
|
| 158 | +<description of the approach and why> |
| 159 | +
|
| 160 | +Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
| 161 | +``` |
| 162 | + |
| 163 | +Use `--no-gpg-sign` for all commits (sandbox environment). |
| 164 | + |
| 165 | +## Step 7: Run Verification |
| 166 | + |
| 167 | +### 7a: Resolve test target |
| 168 | + |
| 169 | +Based on `$4` (test-target, default: `all`): |
| 170 | + |
| 171 | +| Target | Spec | Grep Tags | |
| 172 | +|--------|------|-----------| |
| 173 | +| `all` | `cypress/e2e/incidents/**/*.cy.ts` | `@incidents --@e2e-real --@xfail --@demo` | |
| 174 | +| `regression` | `cypress/e2e/incidents/regression/**/*.cy.ts` | `@incidents --@e2e-real --@xfail` | |
| 175 | +| specific file | `cypress/e2e/incidents/{target}` | (none) | |
| 176 | + |
| 177 | +### 7b: Run tests once |
| 178 | + |
| 179 | +From `web/`: |
| 180 | +```bash |
| 181 | +bash scripts/clean-test-artifacts.sh |
| 182 | +source cypress/export-env.sh && node --max-old-space-size=4096 \ |
| 183 | + ./node_modules/.bin/cypress run --browser electron \ |
| 184 | + --spec "{SPEC}" --env grepTags="{GREP_TAGS}" |
| 185 | +``` |
| 186 | + |
| 187 | +If there are failures, diagnose using `/cypress:test-iteration:diagnose-test-failure`. |
| 188 | +Apply fixes and re-run. Max 2 retries per test. |
| 189 | + |
| 190 | +### 7c: Run e2e-real (if cluster available) |
| 191 | + |
| 192 | +If `web/cypress/export-env.sh` has cluster credentials and the cluster is reachable: |
| 193 | +```bash |
| 194 | +source cypress/export-env.sh && node --max-old-space-size=4096 \ |
| 195 | + ./node_modules/.bin/cypress run --browser electron \ |
| 196 | + --spec "cypress/e2e/incidents/00.coo_incidents_e2e.cy.ts" |
| 197 | +``` |
| 198 | + |
| 199 | +This test takes 10-25 minutes. Run in background. |
| 200 | +If it fails, diagnose the failure — it may reveal issues the regression suite doesn't cover. |
| 201 | + |
| 202 | +### 7d: Flakiness probe |
| 203 | + |
| 204 | +Run the test target `$3` times (default: 3). For each run: |
| 205 | +1. Clean artifacts |
| 206 | +2. Run tests |
| 207 | +3. Record per-test pass/fail |
| 208 | + |
| 209 | +Compute flakiness: |
| 210 | +``` |
| 211 | +Flakiness Report: |
| 212 | + Total tests: N |
| 213 | + Stable (all runs passed): N |
| 214 | + Flaky (some runs failed): N |
| 215 | + Broken (all runs failed): N |
| 216 | +``` |
| 217 | + |
| 218 | +If any flaky tests are found, diagnose and fix them. Re-run the probe on fixed tests. |
| 219 | + |
| 220 | +## Step 8: Present Results and Confirm Push |
| 221 | + |
| 222 | +Present the final state to the user: |
| 223 | +``` |
| 224 | +# Ready to Push |
| 225 | +
|
| 226 | +## Branch: $1 |
| 227 | +## Commits: N |
| 228 | +
|
| 229 | +| # | SHA | Description | |
| 230 | +|---|-----|-------------| |
| 231 | +
|
| 232 | +## Verification |
| 233 | +- Regression: N/N passed, N runs, 0 flaky |
| 234 | +- e2e-real: passed / skipped / failed |
| 235 | +- Files changed: N |
| 236 | +
|
| 237 | +## Excluded from this branch |
| 238 | +- <list any process artifacts, docs, ledgers not included> |
| 239 | +``` |
| 240 | + |
| 241 | +**Wait for user confirmation before pushing.** |
| 242 | + |
| 243 | +## Step 9: Push |
| 244 | + |
| 245 | +```bash |
| 246 | +git push origin $1 |
| 247 | +``` |
| 248 | + |
| 249 | +If the push fails due to auth, try HTTPS with `gh auth token`: |
| 250 | +```bash |
| 251 | +git remote set-url origin https://$(gh auth token)@github.com/<owner>/<repo>.git |
| 252 | +git push origin $1 |
| 253 | +``` |
| 254 | + |
| 255 | +Report the push result and suggest PR creation if desired. |
| 256 | + |
| 257 | +## Error Handling |
| 258 | + |
| 259 | +- **Lint failures after cherry-pick**: Run prettier + eslint --fix first. If max-len errors |
| 260 | + remain, shorten log messages or wrap comments. Never create a separate "fix lint" commit. |
| 261 | +- **Cherry-pick conflicts**: Read both sides, understand intent, merge manually. Never use |
| 262 | + `--ours` or `--theirs` without understanding what's being dropped. |
| 263 | +- **Test failures after merge**: Diagnose with `/cypress:test-iteration:diagnose-test-failure`. |
| 264 | + If the failure is caused by the merge (not a pre-existing issue), fix it before proceeding. |
| 265 | +- **Cypress crashes**: Check `--max-old-space-size`, missing deps, or config issues. |
| 266 | +- **No Chrome**: Use `--browser electron` as fallback. |
| 267 | + |
| 268 | +## Guardrails |
| 269 | + |
| 270 | +- **Never edit source code** (`src/`) — only test files, page objects, fixtures |
| 271 | +- **Never disable tests** — no `.skip()`, no adding `@flaky` tags |
| 272 | +- **Never push without user confirmation** |
| 273 | +- **Never include process artifacts** (iteration docs, stability ledgers) in the output branch |
| 274 | + unless the user explicitly asks |
| 275 | +- **Preserve commit authorship** where possible — use the original author from cherry-picks |
0 commit comments