Skip to content

Add examples for local testing and CI/CD lifecycle management#36

Open
Martijn Visser (MartijnVisser) wants to merge 1 commit into
masterfrom
table-api-cuj-examples
Open

Add examples for local testing and CI/CD lifecycle management#36
Martijn Visser (MartijnVisser) wants to merge 1 commit into
masterfrom
table-api-cuj-examples

Conversation

@MartijnVisser

@MartijnVisser Martijn Visser (MartijnVisser) commented Jun 12, 2026

Copy link
Copy Markdown
Member

Restructures the integration and deployment example around a realistic developer workflow and adds local testing and CI/CD lifecycle management:

  • The pipeline logic of Example_08 is now a plugin-free function that is unit tested locally on Apache Flink with mock data from fromValues(), without Confluent Cloud connectivity. The Confluent plugin and the OSS planner both register engine factories under the identifier 'default' and cannot share a runtime classpath, so surefire excludes the plugin and failsafe excludes the OSS planner (documented in the README, including IDE run-configuration setup).
  • New integration tests (*IT) run the same pipeline logic against Confluent Cloud on a bounded mock table. They fail fast with a clear message when the required environment variables are missing, so a CI pipeline (e.g. a deploy pipeline with a misconfigured secret) cannot silently skip its verification step and still report success; builds without credentials skip them explicitly with -DskipITs. The required variable names are taken from ConfluentPluginOptions so the check cannot drift from the plugin contract, and a FLINK_PROPERTIES-based configuration is accepted as an alternative. The fixture has a timeout, validates the row threshold was reached, and always stops and deletes its fill statement.
  • Example_08's main() is reduced to a deployment entrypoint: configuration via environment variables, a deterministic statement name set through the typed ConfluentPluginOptions.CLIENT_STATEMENT_NAME option, idempotent re-runs, and rejection of stale mode arguments.
  • New Example_12_StatementLifecycle performs status/stop/resume/delete on deployed statements by name.
  • New CI workflow (format check, compile, local tests, fat JAR; no credentials required) and example deploy/manage workflows under .github/workflows-examples/ for use in downstream projects. The manage workflow passes inputs via quoted environment variables to avoid script injection.
  • README gains a developer journey overview, a testing section covering the three test tiers and the Apache Flink (local) limitations, and CI/CD guidance.

Verified: ./mvnw clean verify -DskipITs passes offline (4 unit tests green), verify without credentials fails fast with the expected message, the shaded JAR is unaffected by the test dependencies (no OSS planner classes or service entries), and the integration test has been executed successfully against a live Confluent Cloud environment.

@MartijnVisser Martijn Visser (MartijnVisser) requested a review from a team as a code owner June 12, 2026 12:05
@MartijnVisser Martijn Visser (MartijnVisser) marked this pull request as draft June 12, 2026 12:07
@airlock-confluentinc airlock-confluentinc Bot force-pushed the table-api-cuj-examples branch 2 times, most recently from c55ba3c to 54960f8 Compare June 12, 2026 15:08
@MartijnVisser Martijn Visser (MartijnVisser) marked this pull request as ready for review June 12, 2026 15:08
Comment thread .github/workflows-examples/deploy.yml
Comment thread .github/workflows-examples/manage.yml Outdated
@airlock-confluentinc airlock-confluentinc Bot force-pushed the table-api-cuj-examples branch from 54960f8 to b37ae89 Compare June 29, 2026 12:22
Comment thread .github/workflows-examples/manage.yml Outdated
ENV_ID: ${{ secrets.ENV_ID }}
COMPUTE_POOL_ID: ${{ secrets.COMPUTE_POOL_ID }}
# Group lifecycle operations under the same application name used at deploy time.
APPLICATION_NAME: marketplace-analytics

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we move this to action inputs? In a real world scenario you will probably want to re-use this action for all applications you might own

String finalName = ConfluentTools.getStatementName(insertIntoResult);

System.out.println("Statement has been deployed as: " + finalName);
private static String requireEnv(String name) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should not include this function here. All of the other examples have no checks whether the given database and catalog are existent and follow the same pattern of having variables for them at the top. Sticking to the same patterns across examples helps a user to focus on the more important parts.

@airlock-confluentinc airlock-confluentinc Bot force-pushed the table-api-cuj-examples branch 2 times, most recently from 3e6dd1d to 20f491e Compare June 29, 2026 13:22
@airlock-confluentinc airlock-confluentinc Bot force-pushed the table-api-cuj-examples branch from 20f491e to 41b85c7 Compare June 29, 2026 13:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants