bootstrap-state/: one-time project that creates the S3 bucket used for central OpenTofu state.
ami-builder/: submodule that publishes a project-agnostic NixOS EC2 AMI with a matched root snapshot and preseeded ZFS /nix snapshot.
central/: singleton central infrastructure used by runtime deployments. Today this owns the S3 bucket for runtime deployment state/registry, the long-lived encrypted EBS volume for shared state such as Caddy ACME material and SSH host keys, the platform VPC/subnets/security groups, and the optional central app-server singleton. This is also the intended home for future central services such as auth, public site, VPCs, or other shared platform infrastructure.
runtime/: Quixos deployment stack. It consumes a published AMI for the system image and creates app/runtime resources separately. The deploy wrapper can resolve that AMI from the AMI-builder submodule revision and target architecture.

Destroying the runtime stack destroys the EC2/app/runtime resources only. It must not deregister the AMI or delete the AMI root or /nix snapshots. Those belong to the image-builder state. It also must not delete central infrastructure such as the runtime state bucket or shared persistent volume; those belong to central.

Central Infrastructure

Bootstrap the S3 bucket for central state once:

cd tofu/bootstrap-state
cp terraform.tfvars.example terraform.tfvars
$EDITOR terraform.tfvars
tofu init
tofu apply

bucket_name should be a stable, globally unique operator/team value. Commit the example, not your local terraform.tfvars.

Then initialize central against that bucket. OpenTofu backend config cannot read normal Terraform variables or other state directly, so use the wrapper to render backend.hcl from the bootstrap outputs and run tofu init:

cd ../central
./init-backend.sh

backend.hcl.example shows the generated shape if you need to do it manually.

Create central infrastructure before runtime deploys:

tofu apply

The deploy wrapper reads tofu/central outputs unless you pass explicit overrides:

runtime_state_bucket_name: S3 bucket for runtime deployment state and registry.
shared_state_volume_id: EBS volume mounted by runtime deployments at /persist.
platform_vpc_id and platform_runtime_subnet_id: the managed VPC/subnet intended for runtime deployments.
central_proxy_security_group_id: when the central app server is enabled, runtime deployments can use this to accept HTTP/HTTPS only from the proxy while still keeping public IPv4 egress.

The central app server uses the recommended AMI published by the AMI builder:

route53_zone_id = "Z..."

By default, central reads the AMI ID from SSM parameter /quixos/amis/nixos-zfs-ec2/arm64/recommended, which is written by tofu/ami-builder#publish-ami. Set ami_id only to pin or override that recommended artifact. app_server_ami_id remains available only when the app-server needs to differ from other central EC2 services.

Runtime deployments share this bucket but not a state object. Each deployment uses its own key:

runtime/deployments/<deployment-id>/terraform.tfstate

The wrapper initializes tofu/runtime with OpenTofu's S3 backend and use_lockfile=true, so concurrent operations against the same deployment key are locked. The registry is stored as per-deployment JSON objects under:

registry/deployments/<deployment-id>.json

Use the same aws_region, vpc_id, and subnet_id values as the runtime stack if you override them. The central shared state volume must be in the same availability zone as the runtime EC2 instance.

Runtime Deploy

Create tofu/runtime/terraform.tfvars:

aws_region      = "us-west-2"
route53_zone_id = "Z..."

# Optional, if the package tree is private:
package_tree_git_username = "quixos"
package_tree_git_password = "..."

Keep deployment-specific values out of terraform.tfvars. Local tfvars are operator config: AWS region, Route53 zone, VPC/subnet overrides, package git credentials, instance sizing, and similar static environment inputs.

qx-deploy owns deployment choices: deployment id, runtime state key, package-tree ref, Quixos ref, AMI, shared state volume, generated domain, dev mode, and SSH settings. For each run it resolves those values, rejects local tfvars that try to set deployment-owned keys such as subdomain or dev_mode, writes a temporary zz-qx-deploy.auto.tfvars.json, plans to a saved plan, and applies that saved plan. The generated tfvars file is removed after the run.

Then:

cd tofu/runtime
nix run ../..#quixos-deploy -- deploy --packages-ref master

nix run ../..#quixos-deploy runs the Nix-provided deploy command, which supplies aws, git, node, and tofu. The command itself is TypeScript bundled by the root flake. It prompts when an interactive choice is missing and prints the equivalent noninteractive command before planning.

By default, qx-deploy assigns one DNS name per deployment:

prod: <deployment-id>.<base-domain>
dev: <deployment-id>.dev.<base-domain>

Pass --base-domain <name> to override the base domain from the deploy command, or --domain <name> to override the full generated name for one deployment.

Useful registry commands:

nix run ../..#quixos-deploy -- list
nix run ../..#quixos-deploy -- list --include-destroyed
nix run ../..#quixos-deploy -- ssh
nix run ../..#quixos-deploy -- ssh --deployment-id test-deployment
nix run ../..#quixos-deploy -- doctor
nix run ../..#quixos-deploy -- doctor --deployment-id test-deployment
nix run ../..#quixos-deploy -- destroy --deployment-id test-deployment
nix run ../..#quixos-deploy -- destroy --deployment-id test-deployment --detach

list prints deployment id, status, mode (dev, prod, or unknown), URL, and state key. The ssh command defaults to quixos-dev for dev deployments and quixos-admin for prod deployments; override with --ssh-user.

The deploy wrapper resolves AMIs by listing owned EC2 images tagged with the selected project and target architecture. In interactive mode it always asks which AMI to use, ranking images built from the current AMI-builder submodule HEAD first. Noninteractive deploys must pass --ami-id ami-..., or --auto-select-ami to select the newest image that exactly matches the current AMI-builder rev. The selected value is written to resolved-ami.auto.tfvars.json, which is intentionally local state and ignored by Git. The ami_id Terraform variable has a null default so tofu destroy does not prompt for an AMI just to destroy resources already recorded in state.

The deploy wrapper resolves the package-tree ref. If .quixos-package-tree.json contains quixos.rev, that rev is deployed. Use --quixos-ref master to override it.

If the package tree or its submodules are private, provide HTTPS credentials through the wrapper. The token is written to /persist-local/secrets/quixos-git.env and consumed by the package-tree systemd unit through GIT_ASKPASS.

QUIXOS_PACKAGE_TREE_GIT_USERNAME=oauth2 \
QUIXOS_PACKAGE_TREE_GIT_TOKEN=... \
nix run ../..#quixos-deploy -- deploy --packages-ref master

Runtime Shape

The runtime AMI is expected to provide /nix from the AMI-attached nixos-zfs-nix ZFS volume from first boot. Runtime user data creates/imports no filesystems; it is a NixOS flake consumed by the AMI's user-data-flake service.

The runtime /persist-local volume contains deployment-local Quixos workspace, secret, and package state. It uses a ZFS pool named quixos-persist with a quixos-persist/persist dataset. Runtime NixOS initializes that pool on the attached local persist EBS volume if the volume is blank, imports it if it already exists, and fails on unknown existing filesystem signatures.

The /persist mount is reserved for long-lived shared state. It uses the central EBS volume and a ZFS pool named quixos-shared with a quixos-shared/persist dataset. Caddy's data directory is /persist/var/lib/caddy, and SSH host keys live under /persist/etc/ssh, so both survive runtime tofu destroy.

The visualizer static derivation uses project-visualizer/yarn-project.nix, generated by yarn-plugin-nixify.