BQAA Java preview readiness: parity and safety gaps vs Python plugin

[caohy1988]

Summary

The Java BigQuery Agent Analytics plugin is close to internal dogfood, but it does not look ready to announce as preview parity with the Python BQAA plugin yet. This issue tracks the fixes needed before preview.

Reviewed (pinned SHAs — main has since moved, so all Java line references in this issue and its comments are against the Java SHA below):

• Java google/adk-java at ce6af87e209e9cc718fb987ef1c01665e0be2432
• Python google/adk-python at 38d715cbae2ab160ea1833baa3eebd8c3c93120f (the follow-up verification comment below cross-checked against Python 400f512d812a0c92ebee5d13b5e7448516a55eae; the behaviors cited here match across both)

P1 preview blockers

• Add sensitive-key redaction in Java logging/truncation.

  • Java JsonFormatter.smartTruncate recursively copies object fields without redacting secrets.
  • Python redacts client_secret, access_token, refresh_token, id_token, api_key, password, and temp:* before logging.
  • Add tests for nested maps, tool args/results, session state, A2A/custom metadata, and custom tags.

• Fix HITL and user-message resume handling.

  • Java onUserMessageCallback checks functionCall() and emits HITL_*_COMPLETED.
  • Python handles user-message function_response parts; HITL responses emit HITL_*_COMPLETED, and non-HITL long-running tool resume responses emit TOOL_COMPLETED with pair keys.
  • Add pause_kind and function_call_id parity for long-running tool joins.

• Align event taxonomy with Python before preview.

  • Java currently emits STATE_DELTA for every event, even when there is no state delta.
  • Python only emits STATE_DELTA for non-empty state changes and also emits AGENT_TRANSFER, EVENT_COMPACTION, AGENT_STATE_CHECKPOINT, and TOOL_PAUSED.
  • Update Java view definitions for the added event types.

• Preserve BQAA execution-tree span semantics under ambient OpenTelemetry.

  • Java currently prefers ambient OTel span IDs when present.
  • Python keeps span_id and parent_span_id as the BQAA internal execution tree when the plugin stack exists; ambient OTel is used for trace_id and optional correlation metadata.
  • This is important so parent_span_id points to another BigQuery row rather than a framework span that was never logged.

• Make table bootstrap production-safe.

  • Java creates the table without timestamp partitioning; Python creates day partitioning on timestamp.
  • Java sets tableEnsured = true before table check/create/upgrade succeeds, so a first-run IAM/table race can prevent retries in the same process.
  • Mark ensured only after successful setup; keep retrying after setup failures.

P2 parity and preview-contract gaps

• Decide and document default table/view behavior.

  • Java defaults to dataset agent_analytics, table events, and createViews(false).
  • Python defaults to table agent_events and create_views=True.
  • If Java keeps different defaults, preview docs should call that out clearly.

• Add ADK envelope parity.

  • Python stamps attributes.adk with schema version, app name, source event id, node, branch, scope, route/render/rewind metadata, and long-running pair keys.
  • Java currently has flatter event-specific attributes and lacks the canonical envelope path used by Python views/joins.

• Ensure root agent name is initialized.

  • Java TraceManager.initTrace() sets root agent name but does not appear to be called by the plugin, so attributes.root_agent_name can remain the sentinel value.

• Guard workflow/no-agent callbacks.

  • Python resolves the agent column defensively when InvocationContext.agent is absent and falls back to Event.author for source-event rows.
  • Java directly uses invocationContext.agent().name(), which may drop rows if workflow-driven callbacks have no current agent.

Suggested test checklist

• Secret redaction for nested payloads and all logging paths.
• Empty state_delta produces no STATE_DELTA row.
• User-message functionResponse emits HITL completion or long-running TOOL_COMPLETED as appropriate.
• TOOL_PAUSED/TOOL_COMPLETED rows share function_call_id and pause_kind for joins.
• Ambient OTel trace is preserved while BQAA span_id / parent_span_id remain internally joinable.
• Table creation sets timestamp partitioning and clustering.
• Failed table setup is retried on later events.
• Java view SQL includes the Python-parity event types and pair-key columns.

References

• Java plugin package (pinned to reviewed SHA so line numbers stay stable): https://github.com/google/adk-java/tree/ce6af87e209e9cc718fb987ef1c01665e0be2432/core/src/main/java/com/google/adk/plugins/agentanalytics
• Python reference plugin (pinned to reviewed SHA): https://github.com/google/adk-python/blob/38d715cbae2ab160ea1833baa3eebd8c3c93120f/src/google/adk/plugins/bigquery_agent_analytics_plugin.py

[caohy1988]

Additional reviewed findings (write-path reliability, OTel export, scalability)

Updated after direct line-by-line checks against:

• adk-java issue-pinned SHA ce6af87e209e9cc718fb987ef1c01665e0be2432
• adk-python reference SHA 400f512d812a0c92ebee5d13b5e7448516a55eae

Note: Java main has moved since this issue was opened, so Java line references below are against the pinned SHA above.

These are intended to be non-overlapping with the issue body: the body already covers schema/taxonomy/parity items such as redaction, HITL resume, unconditional STATE_DELTA, table partitioning / tableEnsured, defaults, ADK envelope, root_agent_name, and no-current-agent guard.

Correctness / data-loss (P1-class)

• No drop accounting / health signal for lost rows. Python exposes structured counters (queue_full, retry_exhausted, non_retryable, arrow_prep_failed, unexpected_error) through get_drop_stats(). Java only logs queue-full and write failures (BatchProcessor.append, BatchProcessor.flush), so hosts cannot programmatically detect that analytics rows were dropped.

• Plugin-owned real OTel spans can be exported to the user's trace backend. Java TraceManager.pushSpan() starts a real OTel span and popSpan() ends it. Python now records internal span IDs only and explicitly avoids creating OTel spans, because exporter-enabled hosts can otherwise see duplicate analytics spans. This is separate from the already-filed ambient span tree issue: Java should either stop owning OTel spans for BQAA bookkeeping or make export behavior explicit and tested.

• Append/schema failures can lose a drained batch with no counter, salvage, retry, or DLQ. Java drains rows from the queue before append; if AppendRowsResponse.hasError() or AppendSerializationError happens, the batch is logged and discarded. The default batchSize=1 limits blast radius today, but any real batching setting turns one bad row or append failure into multi-row data loss.

Reliability hardening (P2-class)

• Storage Write regional routing is not explicitly set or tested. The control-plane BigQuery client uses setLocation(config.location()), but the Java write path does not call StreamWriter.Builder.setLocation(config.location()) and does not set x-goog-request-params itself. Python explicitly passes x-goog-request-params: write_stream=... for append routing. Add a regional dataset test, and set the StreamWriter location or otherwise prove non-US/EU appends route correctly.

• No JVM shutdown hook / exit-time warning. Java drains only when the host calls close() or an invocation reaches ensureInvocationCompleted(). Python registers atexit cleanup, but note that Python's handler mainly warns/drains queued items rather than performing a full BigQuery flush at interpreter exit. For Java preview, either register a conservative shutdown hook or document/test the required close/flush lifecycle.

• Per-invocation StreamWriter creation is likely too expensive at scale. PluginState.getBatchProcessor(invocationId) creates a new BatchProcessor, StreamWriter, Arrow allocator, and root per invocation. High invocation throughput means many writers and allocators; createWriter() failures surface as VerifyException from inside async append setup.

Scalability / operational polish (P3)

• batchSize=1 default keeps batching effectively disabled. Python currently shares this default, so this is not Java-only, but it is worth deciding before preview whether the default is for correctness/simplicity or whether Java should ship a throughput-oriented preview default.

• Processed-invocation cache edge should be documented or tested. The cache is maximumSize(10000) with expireAfterWrite(10min). This is not a bug for invocations that merely run longer than 10 minutes; the realistic edge is callbacks that arrive more than 10 minutes after completion, or after heavy completed-invocation churn, getting reprocessed.

• JsonFormatter fallback loses observability. IllegalArgumentException in smartTruncate() / convertToJsonNode() falls back to string conversion without logging. This is survivable, but preview users will have little signal that structured metadata was degraded.

• Hot-path INFO logs are chatty. Examples: duplicate processed invocation, pending-task wait/removal, and table ensure. Most of these should likely be FINE / FINER for preview.

• org.threeten.bp and java.time are mixed in the write path. This looks type-contained today, but it adds avoidable conversion risk and confusion in PluginState / retry / shutdown code.

Suggested additional tests

• Drop counters increment on queue-full, retry-exhausted, non-retryable, serialization, and unexpected-error paths.
• Host with an OTel SDK/exporter enabled does not receive duplicate plugin-owned spans, while BigQuery rows still have a consistent span_id / parent_span_id tree.
• Regional dataset append verifies the Storage Write routing path (setLocation or x-goog-request-params) for non-default locations.
• A batch containing a schema-invalid row records data loss and either salvages good rows, requeues retryable rows, or sends failed rows to a documented dead-letter path.
• JVM close/shutdown lifecycle is tested: explicit close() drains, and shutdown-hook / exit-time behavior is documented and covered.
• LLM success and error paths produce consistent span_id / parent_span_id pairing. This is a regression guard, not a confirmed current bug.

Withdrawn or down-scoped after verification

• Withdrawn: onModelErrorCallback wrong parent_span_id. Current Java appears consistent with Python: success captures the parent before pop, while error pops first and then reads the new current span as the popped span's parent.
• Withdrawn: no retry jitter. GAX RetrySettings defaults to jittered retries; the jitter setter is deprecated with "Retries always jitter."
• Withdrawn: latency_ms / Duration serialization bug. Java converts Duration to millis in a Map before JSON conversion, and JsonFormatter uses ObjectMapper.findAndRegisterModules(). A round-trip test is still fine, but this is not a confirmed defect.
• Withdrawn: part_attributes Java STRING vs Python RECORD. Python also defines part_attributes as STRING and stores serialized JSON.
• Down-scoped: aggregate truncation budget. Java truncation is per-field, but Python is also effectively per-field for this path. Track only if preview requires a top-level row-size/depth guard, not as a Java/Python parity defect.
• Down-scoped: dry-run mode. I did not find a Python dry-run mode either; this is an optional onboarding feature, not a parity blocker.

Net-new findings from 2026-07-01 follow-up

• Schema auto-upgrade can early-return on the version label alone. BigQueryUtils.maybeUpgradeSchema() returns immediately when the stored adk_schema_version label equals the constant, before diffing the actual table schema. A table stamped with the current label but missing columns will never be reconciled until the constant changes. Also, schemaFieldsMatch() only handles missing fields and nested STRUCT additions; non-STRUCT type/mode drift is ignored and later surfaces as Storage Write failures. Fix by always diffing actual schema and logging or failing on incompatible drift.

• EventData.java is missing the Apache license header. Every other file in this package carries the Google Apache header; EventData.java starts directly with the package declaration and also has stale Javadoc saying fields are passed to _log_event. Trivial fix, but should be handled before release/preview.

• ensureInvocationCompleted() timeout can close the writer while parse tasks keep running. On timeout, doFinally() flushes/closes the invocation BatchProcessor, but the timed-out CompletableFutures are not cancelled. A late parse future can then finish and call processor.append(row) against a closed StreamWriter path. Either cancel/ignore late tasks after timeout or make appends after close a no-op with accounting.

• View DDL uses unvalidated identifier interpolation. createAnalyticsViews() splices project, dataset, table, and view prefix into CREATE OR REPLACE VIEW using String.format. These are operator-supplied config values and createViews=false by default, so this is low severity, but a backtick or invalid identifier can break or redirect DDL. Validate BigQuery identifiers before building SQL.

• Numeric config values are not validated. batchSize, queueMaxSize, and maxContentLength accept zero/negative values through the builder. Add builder validation for positive queue/batch sizes and documented behavior for maxContentLength.

• Minor polish bucket: sub-millisecond latencies floor to 0 via Duration.toMillis(); cycle sentinel strings differ from Python; table-create conflict/already-exists is noisy; close() can briefly spin if a scheduled flush holds flushLock; deep nesting still has no recursion-depth guard.

• logEvent runs blocking/synchronous work on the agent callback thread, most acutely on the first event. logEvent() executes its body synchronously and returns an already-completed Completable (only content parse + append are async). That synchronous body calls ensureTableExistsOnce() (BigQueryAgentAnalyticsPlugin.java:232), which on the first event of the process performs blocking BigQuery RPCs — getTable(), then create() and/or maybeUpgradeSchema() — on the callback thread, stalling that invocation until table setup finishes. Every event additionally pays synchronous getResolvedTraceIds() and getAttributes(), including smartTruncate(session.state()) (:352) over potentially-large session state, before returning. This is distinct from the tableEnsured race in the issue body (that is retry-after-failure; this is hot-path latency). Suggested fix: run the first-run table ensure off-thread or at construction, and move ID resolution / attribute building / session-state truncation into the async stage. Related: view creation is submitted to the shared 2-thread flush executor (:176), so a slow DDL can occupy a flush thread.

[hemasekhar-p]

Hi @caohy1988, Thank you for creating this issue and for your contribution. To facilitate the review process, could you please complete the pull request and transition it from a draft to open state? This will allow us to begin reviewing the Issue and the associated changes. We appreciate your work on this.