Skip to content

Commit f224fd8

Browse files
etmurasakietmurasaki
committed
monitoring testing for admin and virtualization
1 parent 4e1f998 commit f224fd8

30 files changed

Lines changed: 1271 additions & 540 deletions

‎web/cypress/E2E_TEST_SCENARIOS.md‎

Lines changed: 210 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,210 @@
1+
# E2E Test Scenarios Overview
2+
3+
This document provides a comprehensive overview of all End-to-End (E2E) test scenarios for the Monitoring Plugin, including COO (Cluster Observability Operator) and standard Monitoring tests.
4+
5+
## Table of Contents
6+
- [COO (Cluster Observability Operator) Tests](#coo-cluster-observability-operator-tests)
7+
- [Virtualization Tests](#virtualization-tests)
8+
- [Monitoring Tests](#monitoring-tests)
9+
- [Support Module Test Scenarios](#support-module-test-scenarios)
10+
11+
---
12+
13+
## COO (Cluster Observability Operator) Tests
14+
15+
Located in `e2e/coo/`
16+
17+
### Build Verification Tests (BVT)
18+
19+
| File | Test Suite | Test Scenario | Description |
20+
|------|------------|---------------|-------------|
21+
| `01.coo_bvt.cy.ts` | BVT: COO | 1. Admin perspective - Observe Menu | Verifies Observe menu navigation and submenus (Alerting, Silences, Alerting rules, Dashboards (Perses)) |
22+
23+
---
24+
25+
## Virtualization Tests
26+
27+
Located in `e2e/virtualization/`
28+
29+
These tests verify the Monitoring Plugin functionality within the Virtualization perspective, including integration with OpenShift Virtualization (KubeVirt).
30+
31+
### Integration Verification Tests (IVT)
32+
33+
| File | Test Suite | Test Scenario | Perspective | Namespace Scope |
34+
|------|------------|---------------|-------------|-----------------|
35+
| `01.coo_ivt.cy.ts` | Setting up Monitoring Plugin | 1. Setting up Monitoring Plugin | N/A | N/A |
36+
| `01.coo_ivt.cy.ts` | Installation: Virtualization | 1. Virtualization perspective - Observe Menu | Virtualization | N/A |
37+
| `01.coo_ivt.cy.ts` | IVT: Monitoring + Virtualization | *Runs BVT Monitoring Tests* | Virtualization | All Projects |
38+
| `01.coo_ivt.cy.ts` | IVT: Monitoring + Virtualization - Namespaced | *Runs BVT Monitoring Tests (Namespace)* | Virtualization | openshift-monitoring |
39+
| | | | |
40+
| `02.coo_ivt_alerts.cy.ts` | Setting up Monitoring Plugin | 1. Setting up Monitoring Plugin | N/A | N/A |
41+
| `02.coo_ivt_alerts.cy.ts` | IVT: Monitoring UIPlugin + Virtualization | 1. Virtualization perspective - Observe Menu | Virtualization | N/A |
42+
| `02.coo_ivt_alerts.cy.ts` | Regression: Monitoring - Alerts (Virtualization) | *Runs All Regression Alerts Tests* | Virtualization | All Projects |
43+
| `02.coo_ivt_alerts.cy.ts` | Regression: Monitoring - Alerts Namespaced (Virtualization) | *Runs All Regression Alerts Tests (Namespace)* | Virtualization | openshift-monitoring |
44+
| | | | |
45+
| `03.coo_ivt_metrics.cy.ts` | Setting up Monitoring Plugin | 1. Setting up Monitoring Plugin | N/A | N/A |
46+
| `03.coo_ivt_metrics.cy.ts` | IVT: Monitoring UIPlugin + Virtualization | 1. Virtualization perspective - Observe Menu | Virtualization | N/A |
47+
| `03.coo_ivt_metrics.cy.ts` | Regression: Monitoring - Metrics (Virtualization) | *Runs All Regression Metrics Tests* | Virtualization | All Projects |
48+
| `03.coo_ivt_metrics.cy.ts` | Regression: Monitoring - Metrics Namespaced (Virtualization) | *Runs All Regression Metrics Tests (Namespace)* | Virtualization | openshift-monitoring |
49+
| | | | |
50+
| `04.coo_ivt_legacy_dashboards.cy.ts` | Setting up Monitoring Plugin | 1. Setting up Monitoring Plugin | N/A | N/A |
51+
| `04.coo_ivt_legacy_dashboards.cy.ts` | IVT: Monitoring UIPlugin + Virtualization | 1. Virtualization perspective - Observe Menu | Virtualization | N/A |
52+
| `04.coo_ivt_legacy_dashboards.cy.ts` | Regression: Monitoring - Legacy Dashboards (Virtualization) | *Runs All Regression Legacy Dashboards Tests* | Virtualization | All Projects |
53+
| `04.coo_ivt_legacy_dashboards.cy.ts` | Regression: Monitoring - Legacy Dashboards Namespaced (Virtualization) | *Runs All Regression Legacy Dashboards Tests (Namespace)* | Virtualization | openshift-monitoring |
54+
55+
---
56+
57+
## Monitoring Tests
58+
59+
Located in `e2e/monitoring/`
60+
61+
### Basic Verification Tests (BVT)
62+
63+
| File | Test Suite | Test Scenario | Description | Namespace Scope |
64+
|------|------------|---------------|-------------|-----------------|
65+
| `00.bvt_admin.cy.ts` | BVT: Monitoring | 1. Admin perspective - Observe Menu | Verifies all Observe submenus navigation | All Projects |
66+
| `00.bvt_admin.cy.ts` | BVT: Monitoring | 2. Admin perspective - Overview Page > Status - View alerts | Verifies navigation from Overview status card to Alerting page | All Projects |
67+
| `00.bvt_admin.cy.ts` | BVT: Monitoring | 3. Admin perspective - Cluster Utilization - Metrics | Verifies navigation from cluster utilization to Metrics page | All Projects |
68+
| `00.bvt_admin.cy.ts` | BVT: Monitoring | *Runs BVT Monitoring Tests* | Additional monitoring scenarios from support module | All Projects |
69+
| | | | | |
70+
| `00.bvt_admin.cy.ts` | BVT: Monitoring - Namespaced | Admin perspective - Observe Menu | Verifies all Observe submenus navigation with namespace scope | openshift-monitoring |
71+
| `00.bvt_admin.cy.ts` | BVT: Monitoring - Namespaced | Admin perspective - Overview Page > Status - View alerts | Verifies navigation from Overview status card to Alerting page | openshift-monitoring |
72+
| `00.bvt_admin.cy.ts` | BVT: Monitoring - Namespaced | Admin perspective - Cluster Utilization - Metrics | Verifies navigation from cluster utilization to Metrics page | openshift-monitoring |
73+
| `00.bvt_admin.cy.ts` | BVT: Monitoring - Namespaced | *Runs BVT Monitoring Tests (Namespace)* | Additional monitoring scenarios from support module | openshift-monitoring |
74+
75+
### Regression Tests
76+
77+
| File | Test Suite | Test Scenario | Feature Area | Namespace Scope |
78+
|------|------------|---------------|--------------|-----------------|
79+
| `regression/01.reg_alerts_admin.cy.ts` | Regression: Monitoring - Alerts (Administrator) | *Runs All Regression Alerts Tests* | Alerts | All Projects |
80+
| `regression/01.reg_alerts_admin.cy.ts` | Regression: Monitoring - Alerts Namespaced (Administrator) | *Runs All Regression Alerts Tests (Namespace)* | Alerts | openshift-monitoring |
81+
| | | | | |
82+
| `regression/02.reg_metrics_admin.cy.ts` | Regression: Monitoring - Metrics (Administrator) | *Runs All Regression Metrics Tests* | Metrics | All Projects |
83+
| `regression/02.reg_metrics_admin.cy.ts` | Regression: Monitoring - Metrics Namespaced (Administrator) | *Runs All Regression Metrics Tests (Namespace)* | Metrics | openshift-monitoring |
84+
| | | | | |
85+
| `regression/03.reg_legacy_dashboards_admin.cy.ts` | Regression: Monitoring - Legacy Dashboards (Administrator) | *Runs All Regression Legacy Dashboards Tests* | Dashboards | All Projects |
86+
| `regression/03.reg_legacy_dashboards_admin.cy.ts` | Regression: Monitoring - Legacy Dashboards Namespaced (Administrator) | *Runs All Regression Legacy Dashboards Tests (Namespace)* | Dashboards | openshift-monitoring |
87+
88+
---
89+
90+
## Support Module Test Scenarios
91+
92+
These test scenarios are reusable test suites called by the main E2E test files.
93+
94+
### Alerts Tests
95+
96+
#### Non-Namespaced Alerts (`01.reg_alerts.cy.ts`)
97+
98+
| Test Scenario | Description |
99+
|---------------|-------------|
100+
| `{perspective} perspective - Alerting > Alerts page - Filtering` | Tests all filtering options: Alert State, Severity, Source filters; Export CSV; Search by name and label |
101+
| `{perspective} perspective - Alerting > Silences page > Create silence` | Tests silence creation form validation: comment validation, creator validation, label name/value validation |
102+
| `{perspective} perspective - Alerting > Alerts / Silences > Kebab icon on List and Details` | Comprehensive test covering: silence creation, silence expiration, kebab menu actions on alerts and silences, recreate/edit silence functionality |
103+
| `{perspective} perspective - Alerting > Alerting Rules` | Tests alerting rules page: filtering, searching, silence creation from alerting rule, kebab menu validation |
104+
105+
#### Namespaced Alerts (`04.reg_alerts_namespace.cy.ts`)
106+
107+
| Test Scenario | Description |
108+
|---------------|-------------|
109+
| `{perspective} perspective - Alerting > Alerts page - Filtering` | Same as non-namespaced but validates Source filter is not available in namespace scope |
110+
| `{perspective} perspective - Alerting > Silences page > Create silence` | Tests silence creation with namespace scope validation (namespace label should be disabled) |
111+
| `{perspective} perspective - Alerting > Alerts / Silences > Kebab icon on List and Details` | Same comprehensive tests as non-namespaced with namespace context |
112+
| `{perspective} perspective - Alerting > Alerting Rules` | Same alerting rules tests with namespace scope |
113+
| `{perspective} perspective - Alerting > Empty state` | Tests empty state for Alerts, Silences, and Alerting Rules when switching to empty namespace |
114+
115+
### Metrics Tests
116+
117+
#### Non-Namespaced Metrics (`02.reg_metrics.cy.ts`)
118+
119+
| Test Scenario | Description |
120+
|---------------|-------------|
121+
| `{perspective} perspective - Metrics` | Validates Metrics page loading, Units dropdown, Refresh interval, Actions dropdown, Predefined queries, Kebab menu |
122+
| `{perspective} perspective - Metrics > Actions - No query added` | Tests Add query, Collapse/Expand all queries, Delete all queries when no query is entered |
123+
| `{perspective} perspective - Metrics > Actions - One query added` | Tests same actions with an active query loaded |
124+
| `{perspective} perspective - Metrics > Insert Example Query` | Tests example query insertion, graph timespan dropdown/input, Reset zoom, Hide/Show graph, Stacked/Disconnected checkbox |
125+
| `{perspective} perspective - Metrics > Add Query - Run Queries - Kebab icon` | Comprehensive test: Add/run queries, Disable/Enable query via kebab and switch, Hide/Show all series, Select/Unselect series, Delete/Duplicate query |
126+
| `{perspective} perspective - Metrics > Predefined Queries > Export as CSV` | Tests CSV export for all 9 predefined queries (CPU, Memory, Filesystem, Network metrics) |
127+
| `{perspective} perspective - Metrics > Ungraphable results` | Tests ungraphable results error state when too many queries are added |
128+
| `{perspective} perspective - Metrics > No Datapoints` | Tests empty state when query returns no data |
129+
| `{perspective} perspective - Metrics > No Datapoints with alert` | Tests error alert display for invalid queries |
130+
131+
#### Namespaced Metrics (`05.reg_metrics_namespace.cy.ts`)
132+
133+
| Test Scenario | Description |
134+
|---------------|-------------|
135+
| All scenarios from non-namespaced | Same 9 test scenarios as non-namespaced metrics |
136+
| `{perspective} perspective - Metrics > Empty state` | Additional test for empty state when switching to empty namespace |
137+
138+
**Total Metrics Scenarios:** 9 (non-namespaced), 10 (namespaced)
139+
140+
### Legacy Dashboards Tests
141+
142+
#### Non-Namespaced Legacy Dashboards (`03.reg_legacy_dashboards.cy.ts`)
143+
144+
| Test Scenario | Description |
145+
|---------------|-------------|
146+
| `{perspective} perspective - Dashboards (legacy)` | Tests dashboard page loading, time range dropdown, refresh interval, dashboard dropdown, API Performance dashboard panels, Inspect functionality |
147+
| `{perspective} perspective - Dashboards (legacy) - Inspect and Export as CSV` | Tests Export CSV functionality and disabled state for empty data |
148+
| `{perspective} perspective - Dashboards (legacy) - No kebab dropdown` | Validates that Single Stat and Table panels don't show kebab menu |
149+
| `{perspective} perspective - OU-897 - Hide Graph / Show Graph on Metrics, Alert Details and Dashboards` | Tests Hide/Show graph state persistence across Metrics, Dashboards, Alert details, and Alerting rule details pages |
150+
151+
#### Namespaced Legacy Dashboards (`06.reg_legacy_dashboards_namespace.cy.ts`)
152+
153+
| Test Scenario | Description |
154+
|---------------|-------------|
155+
| `{perspective} perspective - Dashboards (legacy)` | Tests Kubernetes Compute Resources Namespace Pods dashboard with namespace scope |
156+
| `{perspective} perspective - Dashboards (legacy) - Export as CSV` | Tests CSV export with namespace scope and empty state validation |
157+
| `{perspective} perspective - Dashboards (legacy) - No kebab dropdown` | Validates kebab menu absence for specific chart types |
158+
| `{perspective} perspective - OU-897 - Hide Graph / Show Graph on Metrics, Alert Details and Dashboards` | Same Hide/Show graph tests with namespace context |
159+
160+
### BVT Monitoring Tests
161+
162+
#### Non-Namespaced BVT (`00.bvt_monitoring.cy.ts`)
163+
164+
| Test Scenario | Description |
165+
|---------------|-------------|
166+
| `{perspective} perspective - Alerting > Alerting Details page > Alerting Rule > Metrics` | Tests full navigation flow: Alerts list → Alert details → Alerting rule details → Metrics page, validates expression query |
167+
| `{perspective} perspective - Creates and expires a Silence` | Tests complete silence lifecycle: creation, validation on multiple pages (Alerts, Silences, Alerting Rules), expiration |
168+
169+
#### Namespaced BVT (`00.bvt_monitoring_namespace.cy.ts`)
170+
171+
| Test Scenario | Description |
172+
|---------------|-------------|
173+
| `{perspective} perspective - Alerting > Alerting Details page > Alerting Rule > Metrics` | Same flow as non-namespaced with namespace scope validation |
174+
| `{perspective} perspective - Creates and expires a Silence` | Same silence lifecycle with namespace scope, validates namespace label is disabled in forms |
175+
176+
---
177+
178+
## Test Statistics Summary
179+
180+
| Category | Test Files | Direct it() Scenarios | Support Module Scenarios | Total Scenarios |
181+
|----------|------------|----------------------|-------------------------|-----------------|
182+
| **COO Tests** | 1 | 1 | 0 | 1 |
183+
| **Virtualization Tests** | 4 | 6 | ~30+ (via support modules) | ~36+ |
184+
| **Monitoring Tests** | 4 | 9 | ~30+ (via support modules) | ~39+ |
185+
| **Support Modules** | 8 | 0 | 39 | 39 |
186+
| **TOTAL** | **20** | **29** | **39** | **~128+** |
187+
188+
---
189+
190+
## Perspectives Tested
191+
192+
- **Administrator** - Standard admin perspective with full cluster access
193+
- **Virtualization** - Virtualization-specific perspective with integrated monitoring
194+
195+
## Namespace Scopes
196+
197+
- **All Projects** - Cluster-wide monitoring across all namespaces
198+
- **openshift-monitoring** - Namespace-scoped monitoring for core monitoring components
199+
- **default** - Used for testing empty state scenarios
200+
201+
---
202+
203+
## Notes
204+
205+
1. **Support Module Tests**: Test scenarios marked with *Runs X Tests* are implemented in support files and called by the main E2E test files with different perspectives.
206+
207+
2. **Perspective Parameter**: `{perspective}` is dynamically replaced with the actual perspective name (Administrator, Virtualization) when tests run.
208+
209+
3. **Empty State Tests**: Namespaced tests include additional empty state validation by switching to empty namespaces like `default`.
210+

‎web/cypress/README.md‎

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,9 @@
11
# Openshift Monitoring Plugin and Monitoring Console Plugin UI Tests
22
These console tests are related to Monitoring Plugin deployed by Cluster Monitoring Operator (CMO) as part of OCP - Observe menu with Alerting, Metrics, Dashboards pages and other related Alerting links.
3-
Besides, Monitoring Console Plugin deployed by Cluster Observability Operator through Monitoring UIPlugin installation.
3+
Besides, Monitoring Console Plugin deployed by Cluster Observability Operator through Monitoring UIPlugin installation.
4+
5+
## Test Documentation
6+
For a comprehensive overview of all E2E test scenarios, including COO (Cluster Observability Operator), Monitoring, and Incidents tests, see [E2E_TEST_SCENARIOS.md](./E2E_TEST_SCENARIOS.md).
47

58
## Prerequisite
69
1. [node.js](https://nodejs.org/) >= 18
@@ -166,5 +169,10 @@ It runs the Monitoring Regression tests
166169
npx cypress run --spec "cypress/e2e/monitoring/regression/**"
167170
```
168171

172+
It runs the Virtualization IVT tests
173+
```bash
174+
npx cypress run --spec "cypress/e2e/virtualization/**"
175+
```
176+
169177
### Testing recording
170178
You can access the recording for your test under monitoring-plugin/web/cypress/videos folder

‎web/cypress/e2e/coo/01.coo_ivt.cy.ts‎

Lines changed: 0 additions & 40 deletions
This file was deleted.

0 commit comments

Comments
 (0)