Skip to content

feat: directory upload for managed table loads (--dir) #187

Description

@zfarrell

Follow-up to the presigned/direct-to-storage upload migration (runtimedb#333).

Add a --dir option to hotdata databases load / databases tables load that uploads every parquet file in a directory in one invocation, using the server's batch endpoint (POST /v1/uploads/batch, already merged).

Scope

  • One file per table — each <name>.parquet loads into a table named <name>. This is explicitly not "many files concatenated into one table" (that's a separate concern; a sharded-dataset → single-table mode can be considered later).
  • Maps to the vectorized batch create (single-PUT or multipart per file), per-file finalize, partial-batch success allowed.
  • Aggregate progress across the batch (single bar, consistent with the single-file UX).

Depends on

  • Presigned single-PUT (Phase 1) and multipart (Phase 2) landing in the SDK + CLI first. This is purely additive CLI surface on top of that work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions