Architecture

Clonit Agent follows a clean layered architecture with dependency injection. The CLI layer is kept thin, delegating all business logic to services that coordinate between models, database engines, and cloud storage.

Package Layout

clonit-agent/
├── cmd/                          # Cobra CLI commands (thin layer)
├── internal/
│   ├── app/                      # App struct (dependency container)
│   ├── config/                   # Config struct + Viper defaults
│   ├── cloud/                    # Cloud API client (agent registration, snapshots, org, API keys)
│   ├── database/                 # SQLite + goose migrations
│   │   └── migrations/           # SQL migration files
│   ├── docker/                   # Docker container management for ephemeral sanitization
│   ├── encryption/               # AES-GCM encrypt/decrypt
│   ├── engine/                   # DatabaseTools interface + factory
│   │   └── pgengine/             # PostgreSQL: pg_dump, pg_restore, psql
│   ├── llm/                      # LLM client for sanitization analysis (Anthropic Claude)
│   ├── model/                    # Data models + repos (Target, Snapshot, StorageProfile, Analysis, Query)
│   ├── output/                   # CLI output service (colored status, tables, quiet mode)
│   ├── service/                  # Business logic (build, load, push, pull, sanitize, tool status)
│   ├── storage/                  # Cloud storage (S3, R2) + transfer manager
│   ├── util/                     # File helpers, checksums, URL parsing
│   └── web/                      # HTTP server (chi router), middleware, SPA serving
│       └── handlers/             # REST API handlers
├── ui/                           # React frontend (Bun + shadcn/ui + TanStack Query)
├── cloud/                        # Clonit Cloud backend (separate Go module)
│   └── internal/
│       ├── auth/                 # JWT + API key authentication, SSO/OIDC
│       ├── config/               # Cloud server configuration
│       ├── model/                # Cloud data models (users, orgs, agents, API keys)
│       ├── storage/              # Presigned URL generation
│       └── web/                  # Cloud API routes and handlers
├── shared/                       # Shared types between agent and cloud (API contracts)
├── main.go
├── Makefile
└── go.mod

Key Design Decisions

SQLite for Metadata

Rather than storing everything in a single YAML config file, Clonit Agent uses SQLite for structured data. Targets, snapshots, storage profiles, analyses, and sanitization queries live in ~/.config/clonit/clonit.db. This provides proper relational storage, query capability, and avoids the complexity of managing large nested YAML documents.

YAML for Settings Only

The config file at ~/.config/clonit/config.yaml holds only application settings: the secret key, working directory, log level, cloud connection, analysis API key, and defaults. Target definitions, snapshot records, and storage profiles are not stored in YAML – they live in SQLite where they can be properly queried and related to each other.

Transparent Encryption

Sensitive fields such as database connection URLs (which contain credentials) and storage access keys are encrypted on write and decrypted on read using the $aes$ prefix convention. The encryption key comes from secret_key in the config file. This means sensitive data is never stored in plaintext in the SQLite database. See the Encryption page for details.

Dependency Injection

An App struct is created during CLI initialization in PersistentPreRunE and passed to all commands. This struct acts as a dependency container, holding references to the config, database, logger, output service, and version information. This makes testing straightforward since dependencies can be swapped with mocks.

DatabaseTools Interface

The DatabaseTools interface defines the contract for database engine operations (dump, restore, execute SQL, version detection). A factory function creates the appropriate implementation based on the database type. PostgreSQL is supported now via pgengine, and this pattern allows adding MySQL or other engines later without changing the service layer.

The interface also includes version detection methods: ToolVersions() reports the versions of local tools (e.g., pg_dump, pg_restore, psql), and ServerVersion() queries a remote database server for its version. These are used by ToolStatusService to compute compatibility between local tools and remote servers.

Transfer Manager

Cloud storage operations use a transfer manager that handles retry with exponential backoff and jitter, SHA-256 checksum verification, and transient error detection. This ensures reliable uploads and downloads even when network conditions are imperfect.

Cloud Proxy Architecture

The agent serves as a proxy between the web UI and Clonit Cloud. The web UI calls local /api/v1/cloud/* endpoints on the agent, which authenticate with the cloud using an API key and forward the requests. This means the web UI never needs to know about the cloud API directly – the agent handles all authentication and proxying.

LLM-Powered Analysis

The internal/llm/ package provides an abstraction over LLM providers (currently Anthropic Claude). The analysis service sends database schema information to the LLM, which returns a structured sensitivity analysis identifying columns containing PII, credentials, or other sensitive data. This analysis is used to auto-generate sanitization SQL queries.

Ephemeral Sanitization

The internal/docker/ package manages Docker containers for isolated sanitization testing. An ephemeral sanitize run creates a temporary PostgreSQL container, restores a snapshot into it, applies the sanitization query, and dumps the result – all without touching the user’s databases. Progress is streamed to the UI via server-sent events (SSE).

Web UI

The internal/web/ package provides an embedded web UI and REST API built with chi. The React frontend (in ui/) is compiled and embedded into the Go binary via //go:embed. The server serves the SPA at / and the API at /api/*, with CORS middleware for development. See the serve command for usage.

Data Flow

graph LR
  A[cmd layer] --> B[service layer]
  B --> C[model/repo layer]
  B --> D[engine layer]
  B --> E[storage layer]
  C --> F[SQLite + encryption]
  G[web UI] --> H[web handlers]
  H --> B
  H --> I[cloud client]
  I --> J[Clonit Cloud]
  H --> K[LLM client]

Commands in the cmd/ layer parse flags and arguments, then delegate to service functions. Services orchestrate the work by coordinating between model repositories (for metadata), engine implementations (for database operations), and storage providers (for cloud uploads/downloads). The model layer handles all SQLite access with transparent encryption of sensitive fields.

The web handlers serve the same functionality via HTTP, with additional cloud proxy and LLM analysis capabilities exposed through the REST API.

Key Patterns

Dependency Injection

The App struct is created in cmd/root.go during PersistentPreRunE and made available to all subcommands via cmd.APP. This centralizes initialization and ensures every command has access to the same configured dependencies.

Factory Pattern

engine.NewTools(dbType) returns the appropriate DatabaseTools implementation based on the database type string (e.g., "postgresql"). Similarly, storage.New(ctx, config) creates the appropriate cloud storage client. This decouples command and service code from specific implementations.

Repository Pattern

Model repositories encapsulate database queries and handle transparent encryption. When a target is saved, its connection URL is encrypted before writing to SQLite. When a target is loaded, the $aes$-prefixed fields are automatically decrypted. Callers never need to think about encryption.

Cloud Proxy Pattern

The cloud proxy handler (CloudProxyHandler) validates that the cloud client is configured, then delegates to the cloud client methods. This keeps cloud operations behind a single authentication boundary (the agent’s API key) while exposing them through the standard REST API for the web UI.

Dependencies

Package	Purpose
github.com/spf13/cobra	CLI framework
github.com/spf13/viper	Configuration management
modernc.org/sqlite	Pure Go SQLite driver (no CGO)
github.com/pressly/goose/v3	SQL migration management
github.com/google/uuid	UUID generation
github.com/stretchr/testify	Test assertions
github.com/aws/aws-sdk-go-v2	S3/R2 cloud storage
github.com/fatih/color	Colored terminal output
github.com/olekukonenko/tablewriter	Table formatting
github.com/go-chi/chi/v5	HTTP router (web UI + API)
github.com/anthropics/anthropic-sdk-go	Anthropic Claude LLM client
github.com/docker/docker	Docker container management