Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .openapi-generator/FILES
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,9 @@ docs/LoadManagedTableRequest.md
docs/LoadManagedTableResponse.md
docs/ManagedSchemaResponse.md
docs/ManagedTableResponse.md
docs/MintUploadPartsRequest.md
docs/MintUploadPartsResponse.md
docs/MintedUploadPartResponse.md
docs/NumericProfileDetail.md
docs/QueryApi.md
docs/QueryRequest.md
Expand Down Expand Up @@ -242,6 +245,9 @@ hotdata/models/load_managed_table_request.py
hotdata/models/load_managed_table_response.py
hotdata/models/managed_schema_response.py
hotdata/models/managed_table_response.py
hotdata/models/mint_upload_parts_request.py
hotdata/models/mint_upload_parts_response.py
hotdata/models/minted_upload_part_response.py
hotdata/models/numeric_profile_detail.py
hotdata/models/query_request.py
hotdata/models/query_response.py
Expand Down Expand Up @@ -371,6 +377,9 @@ test/test_load_managed_table_request.py
test/test_load_managed_table_response.py
test/test_managed_schema_response.py
test/test_managed_table_response.py
test/test_mint_upload_parts_request.py
test/test_mint_upload_parts_response.py
test/test_minted_upload_part_response.py
test/test_numeric_profile_detail.py
test/test_query_api.py
test/test_query_request.py
Expand Down
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Changed

- feat(uploads): support streaming uploads with on-demand part URLs

## [0.5.0] - 2026-06-28

### Added
Expand Down
2 changes: 1 addition & 1 deletion docs/CreateUploadRequest.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Name | Type | Description | Notes
**checksum_value** | **str** | Integrity checksum value, paired with `checksum_algo`. Optional. | [optional]
**content_encoding** | **str** | Content encoding to record for the uploaded file (for example `gzip`). Optional. | [optional]
**content_type** | **str** | Content type to record for the uploaded file (for example the Parquet, CSV, or JSON MIME type). Optional. | [optional]
**declared_size_bytes** | **int** | The exact size, in bytes, of the file you will upload. Validated at create time against the maximum allowed size, and again at finalize against the bytes actually stored — a mismatch fails the finalize. |
**declared_size_bytes** | **int** | The exact size, in bytes, of the file you will upload. Optional. When provided, it is validated at create time against the maximum allowed size, and again at finalize against the bytes actually stored — a mismatch fails the finalize. Omit it to create a streaming (unknown-size) upload: the session is always multi-part and returns no part URLs up front; instead you mint part URLs on demand from `POST /v1/uploads/{upload_id}/parts` as you upload, and finalize validates only that the file is non-empty. | [optional]
**filename** | **str** | Original file name, recorded with the upload for your own bookkeeping. Optional and advisory — it does not affect where the bytes are stored or how they are loaded. | [optional]
**part_size** | **int** | Preferred size, in bytes, of each part for a large (multi-part) upload. Optional hint — the service clamps it to the allowed part-size range and to the maximum number of parts, and ignores it for small files uploaded with a single `PUT`. Omit to let the service choose. | [optional]

Expand Down
30 changes: 30 additions & 0 deletions docs/MintUploadPartsRequest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# MintUploadPartsRequest

Request body for `POST /v1/uploads/{upload_id}/parts`: mint presigned upload URLs for specific parts of a streaming (unknown-size) multi-part upload. Provide the 1-based part numbers you want URLs for. Mint parts as you upload, and re-request a part number if its URL expires before you finish — the parts you have already uploaded are unaffected.

## Properties

Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
**part_numbers** | **List[int]** | The 1-based part numbers to mint URLs for. Must be non-empty; each number must be between 1 and the maximum number of parts allowed. |

## Example

```python
from hotdata.models.mint_upload_parts_request import MintUploadPartsRequest

# TODO update the JSON string below
json = "{}"
# create an instance of MintUploadPartsRequest from a JSON string
mint_upload_parts_request_instance = MintUploadPartsRequest.from_json(json)
# print the JSON string representation of the object
print(MintUploadPartsRequest.to_json())

# convert the object into a dict
mint_upload_parts_request_dict = mint_upload_parts_request_instance.to_dict()
# create an instance of MintUploadPartsRequest from a dict
mint_upload_parts_request_from_dict = MintUploadPartsRequest.from_dict(mint_upload_parts_request_dict)
```
[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)


30 changes: 30 additions & 0 deletions docs/MintUploadPartsResponse.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# MintUploadPartsResponse

Response body for `POST /v1/uploads/{upload_id}/parts`: the minted part URLs, in ascending part-number order.

## Properties

Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
**parts** | [**List[MintedUploadPartResponse]**](MintedUploadPartResponse.md) | The minted part URLs, in ascending part-number order. `PUT` each part's bytes to its URL and keep the response's `ETag` to pass to finalize. |

## Example

```python
from hotdata.models.mint_upload_parts_response import MintUploadPartsResponse

# TODO update the JSON string below
json = "{}"
# create an instance of MintUploadPartsResponse from a JSON string
mint_upload_parts_response_instance = MintUploadPartsResponse.from_json(json)
# print the JSON string representation of the object
print(MintUploadPartsResponse.to_json())

# convert the object into a dict
mint_upload_parts_response_dict = mint_upload_parts_response_instance.to_dict()
# create an instance of MintUploadPartsResponse from a dict
mint_upload_parts_response_from_dict = MintUploadPartsResponse.from_dict(mint_upload_parts_response_dict)
```
[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)


31 changes: 31 additions & 0 deletions docs/MintedUploadPartResponse.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# MintedUploadPartResponse

One minted part URL.

## Properties

Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
**part_number** | **int** | The 1-based part number this URL is for. |
**url** | **str** | Short-lived URL to `PUT` this part's bytes to. Keep the response's `ETag` and pass the `{part_number, e_tag}` pair to finalize. |

## Example

```python
from hotdata.models.minted_upload_part_response import MintedUploadPartResponse

# TODO update the JSON string below
json = "{}"
# create an instance of MintedUploadPartResponse from a JSON string
minted_upload_part_response_instance = MintedUploadPartResponse.from_json(json)
# print the JSON string representation of the object
print(MintedUploadPartResponse.to_json())

# convert the object into a dict
minted_upload_part_response_dict = minted_upload_part_response_instance.to_dict()
# create an instance of MintedUploadPartResponse from a dict
minted_upload_part_response_from_dict = MintedUploadPartResponse.from_dict(minted_upload_part_response_dict)
```
[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)


4 changes: 2 additions & 2 deletions docs/UploadSessionResponse.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,8 @@ Name | Type | Description | Notes
**finalize_token** | **str** | One-time token that authorizes finalizing this upload. Returned exactly once at create time — store it; it cannot be retrieved again. |
**headers** | **Dict[str, str]** | Headers you must send verbatim with each `PUT`. Currently always empty; present so a future mode can require signed headers without changing the response shape. |
**mode** | **str** | Upload mode: `single` (upload the whole file with one `PUT` to `url`) or `multipart` (upload each part with one `PUT` to the matching entry in `part_urls`). Modeled as a string so additional modes can be added later without breaking clients. |
**part_size** | **int** | For a `multipart` upload, the size in bytes to split the file into: send bytes `[(i-1) * part_size, i * part_size)` to `part_urls[i - 1]`, with the last part carrying the remainder. Slice by this value — do **not** divide the file evenly by `part_urls.len()`, which can make a non-final part smaller than the 5 MiB minimum that storage requires (the upload then fails at finalize). Absent for `single` uploads. | [optional]
**part_urls** | **List[str]** | For a `multipart` upload, the per-part URLs in ascending part order: `PUT` your file's part *i* (1-based) to `part_urls[i - 1]` and keep each response's `ETag`, then pass the `{part_number, e_tag}` list to finalize. Absent for `single` uploads. | [optional]
**part_size** | **int** | For a `multipart` upload (both known-size and streaming), the size in bytes to split the file into: send bytes `[(i-1) * part_size, i * part_size)` as part *i*, with the last part carrying the remainder. Slice by this value — do **not** divide the file evenly by the number of parts, which can make a non-final part smaller than the 5 MiB minimum that storage requires (the upload then fails at finalize). Absent for `single` uploads. | [optional]
**part_urls** | **List[str]** | For a known-size `multipart` upload, the per-part URLs in ascending part order: `PUT` your file's part *i* (1-based) to `part_urls[i - 1]` and keep each response's `ETag`, then pass the `{part_number, e_tag}` list to finalize. Absent for `single` uploads, and also absent for a streaming (unknown-size) `multipart` upload — there, mint part URLs on demand via `POST /v1/uploads/{upload_id}/parts`. | [optional]
**upload_id** | **str** | Identifier for this upload. Pass it to the finalize endpoint and to the managed-table load endpoint once finalized. |
**url** | **str** | The URL to `PUT` the raw file bytes to, for a `single` upload. Short-lived — upload promptly and finalize. Absent for `multipart` uploads (use `part_urls`). | [optional]

Expand Down
98 changes: 96 additions & 2 deletions docs/UploadsApi.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ Method | HTTP request | Description
[**create_upload_sessions_batch_handler**](UploadsApi.md#create_upload_sessions_batch_handler) | **POST** /v1/uploads/batch | Create upload sessions in bulk
[**finalize_upload_handler**](UploadsApi.md#finalize_upload_handler) | **POST** /v1/uploads/{upload_id}/finalize | Finalize upload
[**list_uploads**](UploadsApi.md#list_uploads) | **GET** /v1/files | List uploads
[**mint_upload_parts_handler**](UploadsApi.md#mint_upload_parts_handler) | **POST** /v1/uploads/{upload_id}/parts | Mint upload part URLs
[**upload_file**](UploadsApi.md#upload_file) | **POST** /v1/files | Upload file


Expand All @@ -16,7 +17,7 @@ Method | HTTP request | Description

Create upload session

Create an upload session for a file you will send directly to storage. Based on the declared size, the response is one of two shapes. For a small file (`mode: single`) it contains a short-lived `url` to `PUT` the whole file to. For a large file (`mode: multipart`) it contains `part_urls` and `part_size`: split the file into `part_size`-byte chunks (the last is the remainder) and `PUT` chunk *i* (1-based) to `part_urls[i - 1]`, keeping each response's `ETag`. Slice by `part_size`, not by an even division across `part_urls.len()` (which can make a non-final part too small). In both cases the response also includes a one-time `finalize_token`. After uploading, call the finalize endpoint with the token (and, for multipart, the `{part_number, e_tag}` list) to make the upload usable as managed-table contents. The returned upload ID can then be passed to the managed-table load endpoint.
Create an upload session for a file you will send directly to storage. The response is one of three shapes. For a small file (`mode: single`) it contains a short-lived `url` to `PUT` the whole file to. For a large file with a known size (`mode: multipart`) it contains `part_urls` and `part_size`: split the file into `part_size`-byte chunks (the last is the remainder) and `PUT` chunk *i* (1-based) to `part_urls[i - 1]`, keeping each response's `ETag`. Slice by `part_size`, not by an even division across `part_urls.len()` (which can make a non-final part too small). For a file whose size you do not know up front, omit `declared_size_bytes`: the response is `mode: multipart` with a `part_size` but NO `part_urls`. As you stream, call `POST /v1/uploads/{upload_id}/parts` with a batch of `part_numbers` to mint per-part `PUT` URLs, `PUT` each part and keep its `ETag`, then finalize as for any multipart upload. In all cases the response also includes a one-time `finalize_token`. After uploading, call the finalize endpoint with the token (and, for multipart, the `{part_number, e_tag}` list) to make the upload usable as managed-table contents. The returned upload ID can then be passed to the managed-table load endpoint.

You may hint a preferred part size with `part_size`; the service clamps it to the allowed range and ignores it for single-`PUT` uploads.

Expand Down Expand Up @@ -198,7 +199,7 @@ Name | Type | Description | Notes

Finalize upload

Confirm that a file has been uploaded to storage and make it usable as managed-table contents. Supply the `finalize_token` returned when the session was created, in the `X-Upload-Finalize-Token` header. The uploaded file's size is validated against the size declared at create time; a mismatch is rejected. Finalize is exactly-once: a second finalize of the same upload is rejected.
Confirm that a file has been uploaded to storage and make it usable as managed-table contents. Supply the `finalize_token` returned when the session was created, in the `X-Upload-Finalize-Token` header. When you declared a size at create time, the uploaded file's size is validated against it and a mismatch is rejected. An upload created without a declared size is finalized from its uploaded parts; it must be non-empty and is rejected if it exceeds the server's maximum upload size. Finalize is exactly-once: a second finalize of the same upload is rejected.

### Example

Expand Down Expand Up @@ -368,6 +369,99 @@ Name | Type | Description | Notes

[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)

# **mint_upload_parts_handler**
> MintUploadPartsResponse mint_upload_parts_handler(upload_id, x_upload_finalize_token, mint_upload_parts_request)

Mint upload part URLs

Mint short-lived presigned URLs for specific parts of a multi-part upload. This is required for a streaming (unknown-size) upload — created by omitting the declared size — which mints no part URLs up front. It also works for a known-size multi-part upload: use it to re-mint a part whose URL expired before you uploaded that part. Supply the `finalize_token` returned when the session was created, in the `X-Upload-Finalize-Token` header, and the 1-based `part_numbers` you want URLs for. `PUT` each part's bytes to its URL, keep each response's `ETag`, then pass the `{part_number, e_tag}` list to finalize. You may mint parts in batches as you upload, and re-mint a part number whose URL expired before you finished uploading it.

### Example

* Api Key Authentication (WorkspaceId):
* Bearer Authentication (BearerAuth):

```python
import hotdata
from hotdata.models.mint_upload_parts_request import MintUploadPartsRequest
from hotdata.models.mint_upload_parts_response import MintUploadPartsResponse
from hotdata.rest import ApiException
from pprint import pprint

# Defining the host is optional and defaults to https://api.hotdata.dev
# See configuration.py for a list of all supported configuration parameters.
configuration = hotdata.Configuration(
host = "https://api.hotdata.dev"
)

# The client must configure the authentication and authorization parameters
# in accordance with the API server security policy.
# Examples for each auth method are provided below, use the example that
# satisfies your auth use case.

# Configure API key authorization: WorkspaceId
configuration.api_key['WorkspaceId'] = os.environ["API_KEY"]

# Uncomment below to setup prefix (e.g. Bearer) for API key, if needed
# configuration.api_key_prefix['WorkspaceId'] = 'Bearer'

# Configure Bearer authorization: BearerAuth
configuration = hotdata.Configuration(
access_token = os.environ["BEARER_TOKEN"]
)

# Enter a context with an instance of the API client
with hotdata.ApiClient(configuration) as api_client:
# Create an instance of the API class
api_instance = hotdata.UploadsApi(api_client)
upload_id = 'upload_id_example' # str | Upload session ID returned at create time
x_upload_finalize_token = 'x_upload_finalize_token_example' # str | One-time finalize token returned when the session was created
mint_upload_parts_request = hotdata.MintUploadPartsRequest() # MintUploadPartsRequest |

try:
# Mint upload part URLs
api_response = api_instance.mint_upload_parts_handler(upload_id, x_upload_finalize_token, mint_upload_parts_request)
print("The response of UploadsApi->mint_upload_parts_handler:\n")
pprint(api_response)
except Exception as e:
print("Exception when calling UploadsApi->mint_upload_parts_handler: %s\n" % e)
```



### Parameters


Name | Type | Description | Notes
------------- | ------------- | ------------- | -------------
**upload_id** | **str**| Upload session ID returned at create time |
**x_upload_finalize_token** | **str**| One-time finalize token returned when the session was created |
**mint_upload_parts_request** | [**MintUploadPartsRequest**](MintUploadPartsRequest.md)| |

### Return type

[**MintUploadPartsResponse**](MintUploadPartsResponse.md)

### Authorization

[WorkspaceId](../README.md#WorkspaceId), [BearerAuth](../README.md#BearerAuth)

### HTTP request headers

- **Content-Type**: application/json
- **Accept**: application/json

### HTTP response details

| Status code | Description | Response headers |
|-------------|-------------|------------------|
**200** | Minted part URLs | - |
**400** | Invalid finalize token, invalid part numbers, batch too large, or the upload is not a multi-part upload | - |
**404** | Upload session not found | - |
**501** | Storage backend cannot issue upload URLs | - |

[[Back to top]](#) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to Model list]](../README.md#documentation-for-models) [[Back to README]](../README.md)

# **upload_file**
> UploadResponse upload_file(body)

Expand Down
6 changes: 6 additions & 0 deletions hotdata/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,9 @@
"LoadManagedTableResponse",
"ManagedSchemaResponse",
"ManagedTableResponse",
"MintUploadPartsRequest",
"MintUploadPartsResponse",
"MintedUploadPartResponse",
"NumericProfileDetail",
"QueryRequest",
"QueryResponse",
Expand Down Expand Up @@ -283,6 +286,9 @@
from hotdata.models.load_managed_table_response import LoadManagedTableResponse as LoadManagedTableResponse
from hotdata.models.managed_schema_response import ManagedSchemaResponse as ManagedSchemaResponse
from hotdata.models.managed_table_response import ManagedTableResponse as ManagedTableResponse
from hotdata.models.mint_upload_parts_request import MintUploadPartsRequest as MintUploadPartsRequest
from hotdata.models.mint_upload_parts_response import MintUploadPartsResponse as MintUploadPartsResponse
from hotdata.models.minted_upload_part_response import MintedUploadPartResponse as MintedUploadPartResponse
from hotdata.models.numeric_profile_detail import NumericProfileDetail as NumericProfileDetail
from hotdata.models.query_request import QueryRequest as QueryRequest
from hotdata.models.query_response import QueryResponse as QueryResponse
Expand Down
Loading