🌱 Add make targets for running single e2e scenarios by pedjak · Pull Request #2790 · operator-framework/operator-controller

pedjak · 2026-06-30T13:20:04Z

Description

Add developer-facing Makefile targets for running individual Godog e2e scenarios
without full suite setup/teardown on each iteration.

What's new

make e2e/<feature>/<scenario-prefix> — runs e2e scenarios matching a name
prefix against an already-running cluster. The prefix is resolved to godog's
native file.feature:LINE syntax using the Gherkin parser in TestMain.

make e2e/install                      # all scenarios in install.feature
make e2e/install/Install              # scenarios starting with "Install"
make "e2e/install/Install latest"     # prefix with spaces

make e2e-setup / make experimental-e2e-setup — create persistent KIND
clusters with OLM deployed for iterative development (no teardown after tests).

make e2e-teardown / make experimental-e2e-teardown — delete the clusters
when done.

Reviewer Checklist

API Go Documentation
Tests: Unit Tests (and E2E Tests, if appropriate)
Comprehensive Commit Messages
Links to related GitHub Issue(s)

netlify · 2026-06-30T13:20:09Z

✅ Deploy Preview for olmv1 ready!

Name	Link
🔨 Latest commit	`1834117`
🔍 Latest deploy log	https://app.netlify.com/projects/olmv1/deploys/6a4500db75870000070956c6
😎 Deploy Preview	https://deploy-preview-2790--olmv1.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

Copilot

Pull request overview

Adds developer-facing support for running individual Godog e2e scenarios against a persistent KIND cluster, enabling faster iteration without per-run cluster setup/teardown. This fits the operator-controller development workflow by improving local e2e debugging loops while reusing the existing Godog-based suite harness.

Changes:

Introduces persistent cluster lifecycle targets (e2e-setup / experimental-e2e-setup and matching teardown targets).
Adds a new --e2e.scenario flag and Gherkin-based scenario prefix → file.feature:LINE resolution in the e2e TestMain.
Documents the iterative workflow and the new scenario-selection flag.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
test/e2e/README.md	Documents persistent cluster workflow and single-scenario make targets.
test/e2e/features_test.go	Adds `--e2e.scenario` flag and Gherkin parsing to map scenario prefixes to `file:line` filters.
Makefile	Adds setup/teardown targets and an `e2e/%` pattern target for running a single feature/prefix.
go.mod	Promotes gherkin/messages dependencies to direct requirements for the new parsing code.
AGENTS.md	Updates contributor guidance with the iterative e2e workflow and examples.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

pedjak · 2026-06-30T14:13:43Z

+.PHONY: e2e/%
+e2e/%: E2E_TIMEOUT ?= 20m
+e2e/%: KUBECONFIG ?= $(KUBECONFIG_DIR)/operator-controller-e2e.kubeconfig
+e2e/%: #EXHELP Run e2e scenario against a cluster created by e2e-setup or experimental-e2e-setup (tear down with e2e-teardown).


Fixed: added explicit KUBECONFIG override example for the experimental cluster in the help text and documentation.

pedjak · 2026-06-30T14:13:55Z

+make e2e/install E2E_TIMEOUT=30m              # Override timeout
+make e2e/install KUBECONFIG=~/.kube/config    # Override kubeconfig
+


Fixed: added an explicit example showing KUBECONFIG override for the experimental cluster.

codecov · 2026-06-30T13:42:41Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 70.29%. Comparing base (c3caa22) to head (2f2e70d).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2790      +/-   ##
==========================================
- Coverage   70.44%   70.29%   -0.16%     
==========================================
  Files         143      142       -1     
  Lines       10625    10577      -48     
==========================================
- Hits         7485     7435      -50     
- Misses       2579     2580       +1     
- Partials      561      562       +1

Flag	Coverage Δ
e2e	`35.12% <ø> (+<0.01%)`	⬆️
experimental-e2e	`52.53% <ø> (+0.11%)`	⬆️
unit	`59.32% <ø> (-0.19%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

tmshort

Just some comments

tmshort · 2026-06-30T14:45:54Z

 test-e2e: kind-clean-e2e #HELP Run e2e test suite on local kind cluster
 test-experimental-e2e: kind-clean-experimental-e2e #HELP Run experimental e2e test suite on local kind cluster



Should these be with the other test-e2e/test-experimental-e2e lines?

Yes, they are — e2e-setup and experimental-e2e-setup are placed immediately after the test-e2e/test-experimental-e2e lines. They mirror the same variant-specific variable overrides (E2E_SOURCE_MANIFEST, E2E_RELEASE_MANIFEST, KIND_CONFIG) and depend on the same wait-% chain.

I mean should they be moveI mean, should these lines be moved up to ~340

Done — setup variable overrides moved up next to the test-e2e/test-experimental-e2e overrides block.

tmshort · 2026-06-30T14:52:01Z

+.PHONY: e2e-teardown
+e2e-teardown: $(KIND) #EXHELP Delete the standard e2e KIND cluster.
+	$(KIND) delete cluster --name operator-controller-e2e
+
+.PHONY: experimental-e2e-teardown
+experimental-e2e-teardown: $(KIND) #EXHELP Delete the experimental e2e KIND cluster.
+	$(KIND) delete cluster --name operator-controller-experimental-e2e


Any way to combine these with kind-clean-% or parameterize them to avoid duplication?
(Yes, I know you can't depend on kind-clean-% directly, but maybe they can call a common recipe?)

The existing kind-clean-% has a dependency on e2e-coverage-% (which depends on e2e-run-%, etc.), so we can't reuse it directly without pulling in the entire test chain. We could extract a kind-delete-% recipe that both kind-clean-% and e2e-teardown call, but that would touch the existing targets from #2785. Happy to do it if you think it's worth it.

I'd prefer not to have 3 recipes basically doing the same thing, and possibly getting out of sync.

Addressed — with #2792 merged, kind-clean-% is now standalone. Teardown targets are prereq-only delegations to kind-clean-operator-controller-e2e / kind-clean-operator-controller-experimental-e2e, zero recipe duplication.

tmshort · 2026-06-30T14:57:17Z

+.PHONY: e2e-setup
+e2e-setup: E2E_SOURCE_MANIFEST := $(STANDARD_E2E_MANIFEST)
+e2e-setup: E2E_RELEASE_MANIFEST := $(STANDARD_RELEASE_MANIFEST)
+e2e-setup: wait-e2e #EXHELP Create a KIND cluster with standard OLM deployed for iterative e2e testing.


So, let me understand this, since this changed recently (and I didn't get a chance to review):

test-% -> kind-clean-%
kind-clean-% -> e2e-coverage-%
e2e-coverage-% -> e2e-run-%
e2e-run-% -> prometheus-%
prometheus-% -> wait-% << SAME
%-setup -> wait-% << SAME
wait-% -> lint-deployed-%
lint-deployed-% -> kind-deployed-%
kind-deploy-% -> kind-load-% manifests
kind-load-% -> kind-cluster-%
kind-cluster-% -> docker-build

So, basically you strung these out as parameterized recipes, rather than having a long list of dependencies on a single target?

Correct — the chain is exactly as you describe. The pattern rules were introduced in #2785, and the %-setup targets in this PR reuse the same chain, just stopping at wait-% instead of continuing through e2e-run-% → e2e-coverage-% → kind-clean-%.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Add `make e2e/<feature>/<scenario-prefix>` for running individual Godog scenarios by name prefix against an already-running cluster. The prefix is resolved to godog's file:line syntax using the Gherkin parser. Add `e2e-setup` and `experimental-e2e-setup` targets to create persistent KIND clusters for iterative development, with matching `e2e-teardown` and `experimental-e2e-teardown` for cleanup. Co-Authored-By: Claude <noreply@anthropic.com>

tmshort

/approve

tmshort · 2026-07-01T13:51:12Z

/lgtm cancel

Did not mean to do lgtm and approve!

perdasilva · 2026-07-01T13:57:36Z

/lgtm

openshift-ci · 2026-07-01T14:00:18Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rashmigottipati, tmshort

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [tmshort]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copilot AI review requested due to automatic review settings June 30, 2026 13:20

openshift-ci Bot requested review from dtfranz and trgeiger June 30, 2026 13:20

Copilot started reviewing on behalf of pedjak June 30, 2026 13:20 View session

pedjak requested review from perdasilva and removed request for trgeiger June 30, 2026 13:23

Copilot AI reviewed Jun 30, 2026

View reviewed changes

pedjak force-pushed the e2e-single-test-run branch from 447c53c to 2f2e70d Compare June 30, 2026 13:59

tmshort reviewed Jun 30, 2026

View reviewed changes

tmshort mentioned this pull request Jun 30, 2026

🌱 Consolidate kind-* Makefile targets into pattern rules #2792

Merged

4 tasks

Copilot AI review requested due to automatic review settings July 1, 2026 11:47

pedjak force-pushed the e2e-single-test-run branch from 2f2e70d to 0036574 Compare July 1, 2026 11:47

Copilot started reviewing on behalf of pedjak July 1, 2026 11:49 View session

Copilot AI reviewed Jul 1, 2026

View reviewed changes

pedjak force-pushed the e2e-single-test-run branch from 0036574 to 1834117 Compare July 1, 2026 11:58

pedjak requested a review from tmshort July 1, 2026 12:14

tmshort approved these changes Jul 1, 2026

View reviewed changes

openshift-ci Bot assigned tmshort Jul 1, 2026

openshift-ci Bot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jul 1, 2026

openshift-ci Bot removed the lgtm Indicates that a PR is ready to be merged. label Jul 1, 2026

openshift-ci Bot assigned perdasilva Jul 1, 2026

openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jul 1, 2026

rashmigottipati approved these changes Jul 1, 2026

View reviewed changes

openshift-ci Bot assigned rashmigottipati Jul 1, 2026

openshift-merge-bot Bot merged commit 88bd84f into operator-framework:main Jul 1, 2026
23 checks passed

		make e2e/install E2E_TIMEOUT=30m # Override timeout
		make e2e/install KUBECONFIG=~/.kube/config # Override kubeconfig

		test-e2e: kind-clean-e2e #HELP Run e2e test suite on local kind cluster
		test-experimental-e2e: kind-clean-experimental-e2e #HELP Run experimental e2e test suite on local kind cluster

Uh oh!

Conversation

pedjak commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

What's new

Reviewer Checklist

Uh oh!

netlify Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for olmv1 ready!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

codecov Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

tmshort left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

tmshort left a comment

Choose a reason for hiding this comment

Uh oh!

tmshort commented Jul 1, 2026

Uh oh!

perdasilva commented Jul 1, 2026

Uh oh!

openshift-ci Bot commented Jul 1, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pedjak commented Jun 30, 2026 •

edited

Loading

netlify Bot commented Jun 30, 2026 •

edited

Loading

codecov Bot commented Jun 30, 2026 •

edited

Loading