Skip to content

Commit e96d764

Browse files
Claude Agentclaude
andcommitted
feat: add productize-iterations skill for consolidating agentic fixes
Skill that takes one or more agentic test iteration branches and produces a clean, shippable fix branch. Key steps: 1. Discover and fetch source branches 2. Analyze each branch's changes and intent 3. Evaluate overlaps (complementary/progressive/redundant/conflicting) 4. Present evaluation and wait for user confirmation 5. Apply changes with inline lint fixing (no separate lint commits) 6. Structure commits by logical concern, not by source branch 7. Verify with regression run + e2e-real + flakiness probe 8. Confirm with user before pushing Codifies the workflow from the OBSINTA-1290 consolidation session. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
1 parent fad1ad6 commit e96d764

1 file changed

Lines changed: 275 additions & 0 deletions

File tree

Lines changed: 275 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,275 @@
1+
---
2+
description: Consolidate one or more agentic test iteration branches into a clean, shippable fix branch — analyzes overlaps, merges intelligently, verifies with flakiness probing, and pushes
3+
parameters:
4+
- name: branch-name
5+
description: "Target branch name (e.g. test/OBSINTA-1290-incident-stability). Created from main."
6+
required: true
7+
type: string
8+
- name: source-branches
9+
description: "Comma-separated list of iteration branch names or a glob (e.g. 'FIX-PROPOSAL-AGENTIC/*' or 'branch-a,branch-b,branch-c'). Fetched from origin."
10+
required: true
11+
type: string
12+
- name: flakiness-runs
13+
description: "Number of flakiness probe runs after fixes are applied (default: 3)"
14+
required: false
15+
type: string
16+
- name: test-target
17+
description: "Cypress test target — 'all', 'regression', a spec file, or grepTags pattern (default: all)"
18+
required: false
19+
type: string
20+
---
21+
22+
# Productize Agentic Test Iterations
23+
24+
Consolidate one or more agentic iteration branches into a single clean, shippable branch.
25+
When multiple branches exist, analyze whether their changes are complementary, redundant,
26+
or conflicting — then merge them intelligently rather than blindly stacking cherry-picks.
27+
28+
## Prerequisites
29+
30+
- `web/cypress/export-env.sh` must exist with cluster credentials.
31+
If missing, create it from cluster credentials in the conversation (do NOT run `/cypress:setup`).
32+
- Node modules installed in `web/` (`npm install`).
33+
34+
## Step 1: Discover and Fetch Source Branches
35+
36+
Parse `$2` (source-branches):
37+
- If it contains `*`, treat as a glob: `git ls-remote origin | grep <pattern>`
38+
- If comma-separated, split into a list
39+
40+
Fetch all source branches as remote tracking refs:
41+
```bash
42+
git fetch origin "<branch>:refs/remotes/origin/<branch>"
43+
```
44+
45+
If any source branch has an overview/index document (e.g. `docs/agentic-fix-proposals.md`),
46+
read it first — it may describe the branches and their relationships.
47+
48+
Report what was found:
49+
```
50+
Found N source branches:
51+
- origin/<branch-1> — N commits above main
52+
- origin/<branch-2> — N commits above main
53+
```
54+
55+
## Step 2: Analyze Each Branch
56+
57+
For each source branch, extract:
58+
59+
1. **Commits above main**: `git log origin/main..<branch> --oneline`
60+
2. **Files changed**: `git diff origin/main..<branch> --stat`
61+
3. **Commit messages and descriptions**: Read each commit message for intent
62+
63+
Build a structured summary per branch:
64+
```
65+
Branch: <name>
66+
Origin: <how it was created — CI loop, local sandbox, manual>
67+
Commits: N
68+
Files touched: <list>
69+
What it fixes:
70+
- <description of each fix with classification>
71+
```
72+
73+
## Step 3: Evaluate Overlaps
74+
75+
This is the critical step. For each pair of branches that touch the **same file**:
76+
77+
1. **Read both versions** of the overlapping file
78+
2. **Identify the relationship**:
79+
- **Complementary**: Different fixes to different parts of the same file (both needed)
80+
- **Progressive**: One branch's changes are a superset of another's (keep the latest)
81+
- **Redundant**: Both branches solve the same problem differently (pick the better one)
82+
- **Conflicting**: Changes that are incompatible (requires manual resolution)
83+
3. **Decide the merge strategy** for each overlap
84+
85+
Also check for cross-file dependencies:
86+
- Does branch A add a function that branch B calls?
87+
- Does branch A change a signature that branch B depends on?
88+
89+
Produce an evaluation report:
90+
```
91+
## Overlap Analysis
92+
93+
### <file-path>
94+
Touched by: branch-1, branch-3
95+
Relationship: COMPLEMENTARY
96+
- branch-1: adds OOM prevention (cy.reload, quiet search)
97+
- branch-3: adds warmUpForPlugin() method
98+
Strategy: Apply both — they modify different sections
99+
100+
### <file-path>
101+
Touched by: branch-1, branch-2
102+
Relationship: PROGRESSIVE
103+
- branch-1: adds cy.reload() to search loop
104+
- branch-2: rewrites search loop with quiet search (includes reload)
105+
Strategy: Take branch-2 only — it's a superset
106+
107+
## Proposed Commit Structure
108+
1. <logical-group>: <description> — from branches X, Y
109+
2. <logical-group>: <description> — from branch Z
110+
```
111+
112+
**Present this evaluation to the user and wait for confirmation before proceeding.**
113+
114+
## Step 4: Create Clean Branch
115+
116+
```bash
117+
git checkout -b $1 origin/main
118+
```
119+
120+
If the branch already exists, ask the user whether to overwrite or append a suffix.
121+
122+
## Step 5: Apply Changes
123+
124+
Based on the evaluation from Step 3, apply changes in logical order.
125+
126+
**Do NOT blindly cherry-pick individual commits.** Instead:
127+
128+
1. For **progressive** overlaps: only apply the most complete version
129+
2. For **redundant** overlaps: apply the one chosen in Step 3
130+
3. For **complementary** changes: apply in dependency order (e.g., page object before tests that use it)
131+
4. For **conflicts**: manually merge, reading both versions and combining intent
132+
133+
After applying each logical group:
134+
- Run `npx prettier --write <changed-files>` immediately
135+
- Run `npx eslint --fix <changed-files>`
136+
- Check for remaining lint errors with `npx eslint <changed-files>`
137+
- Fix any remaining max-len or formatting issues manually
138+
139+
**Do not defer lint fixes to a separate commit.** Each commit must pass the pre-commit hook.
140+
141+
## Step 6: Structure Commits by Logical Concern
142+
143+
Group changes into commits by **what problem they solve**, not by which branch they came from.
144+
Good commit groupings:
145+
- One commit per distinct fix category (OOM prevention, plugin warm-up, test hygiene, etc.)
146+
- Page object changes bundled with the test changes that use them
147+
- Fixture additions bundled with the tests that reference them
148+
149+
Bad commit groupings:
150+
- One commit per source branch (archaeology, not intent)
151+
- Intermediate steps that are immediately superseded
152+
- Separate "fix lint" commits
153+
154+
Commit message format:
155+
```
156+
fix(tests): <summary of what problem this solves>
157+
158+
<description of the approach and why>
159+
160+
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
161+
```
162+
163+
Use `--no-gpg-sign` for all commits (sandbox environment).
164+
165+
## Step 7: Run Verification
166+
167+
### 7a: Resolve test target
168+
169+
Based on `$4` (test-target, default: `all`):
170+
171+
| Target | Spec | Grep Tags |
172+
|--------|------|-----------|
173+
| `all` | `cypress/e2e/incidents/**/*.cy.ts` | `@incidents --@e2e-real --@xfail --@demo` |
174+
| `regression` | `cypress/e2e/incidents/regression/**/*.cy.ts` | `@incidents --@e2e-real --@xfail` |
175+
| specific file | `cypress/e2e/incidents/{target}` | (none) |
176+
177+
### 7b: Run tests once
178+
179+
From `web/`:
180+
```bash
181+
bash scripts/clean-test-artifacts.sh
182+
source cypress/export-env.sh && node --max-old-space-size=4096 \
183+
./node_modules/.bin/cypress run --browser electron \
184+
--spec "{SPEC}" --env grepTags="{GREP_TAGS}"
185+
```
186+
187+
If there are failures, diagnose using `/cypress:test-iteration:diagnose-test-failure`.
188+
Apply fixes and re-run. Max 2 retries per test.
189+
190+
### 7c: Run e2e-real (if cluster available)
191+
192+
If `web/cypress/export-env.sh` has cluster credentials and the cluster is reachable:
193+
```bash
194+
source cypress/export-env.sh && node --max-old-space-size=4096 \
195+
./node_modules/.bin/cypress run --browser electron \
196+
--spec "cypress/e2e/incidents/00.coo_incidents_e2e.cy.ts"
197+
```
198+
199+
This test takes 10-25 minutes. Run in background.
200+
If it fails, diagnose the failure — it may reveal issues the regression suite doesn't cover.
201+
202+
### 7d: Flakiness probe
203+
204+
Run the test target `$3` times (default: 3). For each run:
205+
1. Clean artifacts
206+
2. Run tests
207+
3. Record per-test pass/fail
208+
209+
Compute flakiness:
210+
```
211+
Flakiness Report:
212+
Total tests: N
213+
Stable (all runs passed): N
214+
Flaky (some runs failed): N
215+
Broken (all runs failed): N
216+
```
217+
218+
If any flaky tests are found, diagnose and fix them. Re-run the probe on fixed tests.
219+
220+
## Step 8: Present Results and Confirm Push
221+
222+
Present the final state to the user:
223+
```
224+
# Ready to Push
225+
226+
## Branch: $1
227+
## Commits: N
228+
229+
| # | SHA | Description |
230+
|---|-----|-------------|
231+
232+
## Verification
233+
- Regression: N/N passed, N runs, 0 flaky
234+
- e2e-real: passed / skipped / failed
235+
- Files changed: N
236+
237+
## Excluded from this branch
238+
- <list any process artifacts, docs, ledgers not included>
239+
```
240+
241+
**Wait for user confirmation before pushing.**
242+
243+
## Step 9: Push
244+
245+
```bash
246+
git push origin $1
247+
```
248+
249+
If the push fails due to auth, try HTTPS with `gh auth token`:
250+
```bash
251+
git remote set-url origin https://$(gh auth token)@github.com/<owner>/<repo>.git
252+
git push origin $1
253+
```
254+
255+
Report the push result and suggest PR creation if desired.
256+
257+
## Error Handling
258+
259+
- **Lint failures after cherry-pick**: Run prettier + eslint --fix first. If max-len errors
260+
remain, shorten log messages or wrap comments. Never create a separate "fix lint" commit.
261+
- **Cherry-pick conflicts**: Read both sides, understand intent, merge manually. Never use
262+
`--ours` or `--theirs` without understanding what's being dropped.
263+
- **Test failures after merge**: Diagnose with `/cypress:test-iteration:diagnose-test-failure`.
264+
If the failure is caused by the merge (not a pre-existing issue), fix it before proceeding.
265+
- **Cypress crashes**: Check `--max-old-space-size`, missing deps, or config issues.
266+
- **No Chrome**: Use `--browser electron` as fallback.
267+
268+
## Guardrails
269+
270+
- **Never edit source code** (`src/`) — only test files, page objects, fixtures
271+
- **Never disable tests** — no `.skip()`, no adding `@flaky` tags
272+
- **Never push without user confirmation**
273+
- **Never include process artifacts** (iteration docs, stability ledgers) in the output branch
274+
unless the user explicitly asks
275+
- **Preserve commit authorship** where possible — use the original author from cherry-picks

0 commit comments

Comments
 (0)