Cloud Storage

Overview

Clonit can push and pull snapshots to and from object storage, enabling you to share snapshots across machines, teams, and environments. Supported providers include:

• AWS S3
• Cloudflare R2
• MinIO (and any path-style S3-compatible storage endpoint)
• Google Cloud Storage (GCS)
• Azure Blob Storage
• The local filesystem (for air-gapped or on-prem use)

Setting Up Storage Profiles

A storage profile defines the credentials and bucket configuration for a cloud storage provider. You can create multiple profiles for different providers or environments.

These are local storage profiles — the credentials live (encrypted) on this machine. Clonit Cloud’s org-level storage profiles, including the keyless S3 AssumeRole mode that stores no keys at all, are managed separately with clonit cloud storage-profiles — see Keyless Credentials.

Credential sources

Every S3-compatible profile authenticates from one of three sources — in the web UI’s storage profile form, this is the Authentication section’s Credential Source selector:

1. Access keys (stored encrypted) — an access/secret key pair, encrypted at rest on this machine.
2. Shared AWS profile (~/.aws) — a named profile from your AWS CLI configuration (SSO-backed profiles included); only the profile name is stored. On the CLI this is --awscli-profile.
3. Agent credential chain (keyless) — nothing is stored. At push/pull time the agent uses whatever AWS identity its environment already has: environment variables, a shared profile, a container/task role, or an EC2 instance role. When the agent-wide aws.profile config key is set (see Configuration), the keyless chain uses that shared profile instead of the ambient default. Check Settings → AWS to see which identity that is.

Provider caveats: MinIO requires keys or a shared profile (no keyless option); R2 usually needs its own bucket-scoped API tokens as stored keys; Azure always uses stored account keys; GCS can leave the service-account JSON empty to fall back to Application Default Credentials.

Switching an existing profile away from stored keys removes them on save — otherwise the stored keys would keep taking precedence over the source you picked. For the full keyless picture — including the cloud’s role-based AssumeRole profiles (with an ExternalId) for organizations that never issue static keys — see Keyless Credentials.

AWS S3

clonit storage-profile add \
  --name my-s3 \
  --provider s3 \
  --bucket my-snapshots \
  --region us-east-1 \
  --access-key AKIA... \
  --secret-key wJalr...

Cloudflare R2

clonit storage-profile add \
  --name my-r2 \
  --provider r2 \
  --bucket my-snapshots \
  --account-id <cloudflare-account-id> \
  --access-key <r2-access-key> \
  --secret-key <r2-secret-key>

MinIO (path-style S3-compatible)

MinIO and similar self-hosted gateways require path-style addressing and a custom endpoint. Use the minio provider (it forces path-style automatically); credentials use the same access-key / secret-key fields as S3.

clonit storage-profile add \
  --name my-minio \
  --provider minio \
  --bucket my-snapshots \
  --endpoint http://minio.local:9000 \
  --access-key minioadmin \
  --secret-key minioadmin

Required: --bucket, --endpoint, and --access-key + --secret-key.

Google Cloud Storage (GCS)

GCS authenticates with a service-account JSON key. Paste the entire JSON into --secret-key (it is encrypted at rest like every other secret). If you omit it, clonit falls back to Application Default Credentials (GOOGLE_APPLICATION_CREDENTIALS or workload identity).

clonit storage-profile add \
  --name my-gcs \
  --provider gcs \
  --bucket my-snapshots \
  --secret-key "$(cat service-account.json)"

Required: --bucket (plus the service-account JSON in --secret-key, unless using Application Default Credentials).

Azure Blob Storage

Azure uses a shared-key credential: the storage account name goes in --access-key, the account key in --secret-key, and the container name in --bucket. A custom --endpoint (blob service URL) is supported for Azurite or private clouds; otherwise https://<account>.blob.core.windows.net/ is used.

clonit storage-profile add \
  --name my-azure \
  --provider azure \
  --bucket my-container \
  --access-key mystorageaccount \
  --secret-key "<account-key>"

Required: --bucket (container), --access-key (account name), and --secret-key (account key).

Additional Options

Flag	Description
--awscli-profile	Use credentials from a named AWS CLI profile instead of explicit keys
--endpoint	Custom endpoint URL for S3-compatible providers
--prefix	Key prefix for organizing objects within the bucket (e.g., snapshots/prod/)

Using an AWS CLI profile

If you already have AWS credentials configured via ~/.aws/credentials, you can reference them by profile name:

clonit storage-profile add \
  --name my-s3 \
  --provider s3 \
  --bucket my-snapshots \
  --region us-east-1 \
  --awscli-profile my-profile

Custom S3-compatible endpoint

For MinIO and other path-style gateways, prefer the dedicated minio provider (see MinIO above) — it forces path-style addressing for you. For virtual-hosted-style S3-compatible services you can also use --provider s3 with an explicit --endpoint:

clonit storage-profile add \
  --name my-s3-compatible \
  --provider s3 \
  --bucket my-snapshots \
  --endpoint https://s3.example.com \
  --access-key <access-key> \
  --secret-key <secret-key>

Object key prefix

Use a prefix to organize snapshots within the bucket:

clonit storage-profile add \
  --name my-s3 \
  --provider s3 \
  --bucket my-snapshots \
  --region us-east-1 \
  --access-key AKIA... \
  --secret-key wJalr... \
  --prefix snapshots/production/

Linking a Target to a Storage Profile

When creating a target, associate it with a storage profile:

clonit targets add \
  --name mydb \
  --src-url "postgres://..." \
  --storage-profile my-s3

Or update an existing target to add a storage profile:

clonit targets update mydb --storage-profile my-s3

Push and Pull Operations

Push (upload)

Upload a snapshot to cloud storage.

# Push the latest snapshot
clonit push mydb

# Push a specific snapshot by index
clonit push mydb 0

Pull (download)

Download a snapshot from cloud storage.

# Pull the latest snapshot
clonit pull mydb

# Pull a specific snapshot by index
clonit pull mydb 0

Options

Flag	Description
--no-verify	Skip SHA-256 checksum verification after transfer

Transfer Manager

Clonit includes a robust transfer manager that handles uploads and downloads reliably:

• Retry with exponential backoff and jitter – transient network errors are automatically retried with increasing delays and randomized jitter to avoid thundering herd problems.
• SHA-256 checksum verification – after each transfer, the checksum of the local and remote files is compared to ensure data integrity.
• Transient error detection – the transfer manager distinguishes between transient errors (network timeouts, throttling) and permanent errors (access denied, bucket not found) to avoid unnecessary retries.

Managing Storage Profiles

List all storage profiles

clonit storage-profile

Remove a storage profile

clonit storage-profile remove <name>

Editing in the web UI

When editing a storage profile in the web UI, credential fields left blank keep their stored values – each field independently – so you can rotate one key without re-entering the other.

The exception is switching the Credential Source away from stored keys: saving then removes the stored access keys, so the profile really uses the shared profile or agent credential chain you selected. The form warns you before saving.

Example Workflow

A typical cloud storage workflow for sharing sanitized snapshots across a team:

# 1. Build and sanitize a snapshot
clonit build mydb
clonit sanitize mydb

# 2. Push the sanitized snapshot to cloud storage
clonit push mydb 1

# 3. On another machine, pull the snapshot
clonit pull mydb 0

# 4. Load the pulled snapshot into the local dev database
clonit load mydb