{ "available": true, "status": "partly-running", "note": "The object storage layer: which endpoint answers when an artifact or a backup is asked for, whether that answer is current, and who is allowed to delete it. The backup buckets are running today. The artifact plane — MinIO on the NAS with a read-through cache on VIN — is designed and not deployed, and this register says so in every number rather than describing the estate it hopes for.", "policy": { "maxRestoreHours": 12, "maxRestoreEur": 5, "maxMonthlyEur": 0, "integrityMaxAgeDays": 45, "verifyMaxAgeDays": 30, "restoreDrillMaxAgeDays": 180, "observationMaxAgeDays": 90, "quotaWarnPct": 75, "adminNodes": [ "nas" ], "notes": [ "Three roles, and a copy is in exactly one of them. The source of truth is where writes land, and there is exactly one per bucket — two authorities for one object name do not conflict loudly, the later sync wins and the earlier write is gone with a 200 on both. A replica is fed on a schedule and read-only, and it is what makes a bucket survive the room. A cache holds whatever was asked for recently: it answers fast, it answers incompletely, and it may never be the last copy.", "Cache versus source of truth is a decision about who is allowed to be wrong. The NAS is the source for everything durable because it is the only endpoint with the whole set. VIN holds a cache because a VPS disk is small, rented and the wrong place to keep the only copy of anything — but it is also the only node that keeps working when the house is dark, so a cache there is the difference between a slow build and no build.", "The recovery point is arithmetic over the replication graph, never a number somebody typed. A replica fed hourly from a replica fed nightly is a day and an hour behind whatever the objective says, so the chain is walked and the distance falls out. The same applies to restore time: bytes over the endpoint's own measured bandwidth, plus the minutes it takes to find the passphrase and decrypt.", "A mirror is not a backup. Any replication edge that propagates deletes gives `rm -rf` a second place to land, so on a copy holding backups that is a finding rather than a configuration detail. Off-site deletes are done by a retention command with a window, from the node that owns the retention, and never by a sync.", "Nothing here answers an unauthenticated read. Every bucket requires a credential, credentials are scoped per node and per direction, and the register names them by variable and by file — never by value. An object store has no 'may put but not delete' scope, so a read-write key is a delete permission and is only issued to a node that actually writes.", "Reachability is derived from the inventory, not asserted. An endpoint is reachable from a node when a declared path exists whose every hop is up, which is what makes 'readable with home offline' a computed answer rather than a hopeful one. The survival matrix is computed on the copies that are running today; the design view shows what it becomes when the planned ones exist." ] }, "summary": { "endpoints": 3, "buckets": 3, "copies": 6, "copiesRunning": 4, "edges": 3, "edgesRunning": 2, "edgesVerified": 2, "durable": 2, "durableWithOffsiteToday": 2, "scenarios": 4, "cellsTotal": 12, "cellsReadable": 8, "cellsServed": 8, "cellsWritable": 6, "worstRestoreHours": 1.4554222222222222, "restoreEur": 0, "monthlyEur": 0, "sizeGbToday": 313, "drilled": 1, "offsiteDrilled": 0, "needProof": 2, "anonymous": 0, "fatal": 0, "gaps": 8, "accepted": 3, "notes": 14 }, "endpoints": [ { "id": "ep-nas-minio", "name": "MinIO on the NAS", "kind": "minio", "purpose": "The source of truth for every durable object in the estate: database backups, configuration archives and, once it exists, the artifact plane. It is on the NAS because that is the only place with room for the whole set, and it is not off-site because it is in the same room as the thing it protects.", "entityId": "nas-backups", "node": "nas", "tier": "t-nas-pool", "domain": "fd-home-rack", "url": "https://s3.nas.home.arpa", "addressing": "path", "region": "home", "tls": true, "anonymousRead": false, "status": "unknown", "capabilities": { "versioning": true, "objectLock": true, "lifecycle": true, "multipart": true, "presign": true, "serverSideReplication": true, "checksums": "sha256" }, "bandwidthMbps": { "read": 940, "write": 720, "measuredOn": "2026-08-14", "note": "Measured with mc from home1 over the rack LAN. Reads saturate the link; writes are held back by the pool's parity calculation, not by the network." }, "cost": { "model": "owned-hardware", "monthlyEur": 0, "quotaGb": null, "egressEurPerGb": 0, "note": "The drives are bought. The recurring cost of this endpoint is electricity, and reading from it costs nothing at all — which is the whole reason the restore that can use it should." }, "reachableFrom": [ { "via": [], "node": "nas", "note": "Loopback on the NAS itself, which is how the ILM and heal administration runs." }, { "via": [ "internal-dns" ], "node": "home1", "note": "Over the rack LAN. Needs internal DNS to resolve s3.nas.home.arpa; the fallback is the literal address, which is in the runbook." }, { "via": [ "wireguard-tunnel", "isp-uplink" ], "node": "vin", "note": "Over the mesh, which crosses the home uplink. That makes this path unavailable in exactly the scenario the off-site copy exists for." } ], "credentials": [ { "id": "cred-nas-rw", "envName": "JARVISOS_S3_NAS_RW", "location": "/etc/jarvisos/s3-nas-rw.env", "scope": "read-write", "usedBy": [ "home1" ] }, { "id": "cred-nas-ro", "envName": "JARVISOS_S3_NAS_RO", "location": "/etc/jarvisos/s3-nas-ro.env", "scope": "read", "usedBy": [ "vin" ] }, { "id": "cred-nas-admin", "envName": "JARVISOS_S3_NAS_ADMIN", "location": "/etc/jarvisos/s3-nas-admin.env", "scope": "admin", "usedBy": [ "nas" ] } ], "design": { "sizeGb": 548, "objects": 262600, "quotaPct": null }, "today": { "sizeGb": 248, "objects": 22600, "placed": [ { "status": "running", "prefix": "postgres/", "objects": 18400, "observedOn": "2026-08-14", "validates": null, "writtenBy": [ "home1" ], "lifecycle": [ { "id": "lc-db-nas-expire", "action": "expire", "afterDays": 190, "appliesTo": "postgres/", "serverSide": true, "enforcedBy": "MinIO ILM (mc ilm rule add), applied on the NAS. Set beyond the 180-day retention on purpose: the retention command decides what to delete, and the lifecycle rule is only the backstop for what it misses." }, { "id": "lc-db-nas-multipart", "action": "abort-multipart", "afterDays": 2, "appliesTo": "", "serverSide": true, "enforcedBy": "MinIO ILM. An interrupted 40 GB base backup leaves parts that no listing shows and that occupy the pool until something aborts them." } ], "integrity": { "method": "mc admin heal on the bucket, then a sha256 manifest compare against the newest base backup", "intervalDays": 30, "coverage": "full", "lastRunOn": "2026-08-03", "outcome": "pass" }, "notes": [], "id": "c-db-nas", "endpoint": "ep-nas-minio", "role": "source", "bucket": "jarvisos-backups", "writable": true, "sizeGb": 240, "measured": true, "versioning": true, "objectLock": false, "encryption": { "inTransit": "tls", "atRest": "dataset-level", "keyLocation": null, "keyOnSourceNode": false, "note": "The pool is encrypted at rest and the hop is TLS inside the rack. Client-side encryption is not used here on purpose: this copy is the one an operator restores from in a hurry, and a passphrase in the path is a passphrase to find first." }, "bucketId": "b-db-backups", "bucketName": "Database backups", "class": "backup", "reproducible": false }, { "status": "running", "prefix": "config/", "objects": 4200, "observedOn": "2026-08-14", "validates": null, "writtenBy": [ "home1" ], "lifecycle": [ { "id": "lc-config-nas-expire", "action": "expire", "afterDays": 400, "appliesTo": "config/", "serverSide": true, "enforcedBy": "MinIO ILM. A year of monthly points is what this archive is for, so the rule sits beyond the retention window rather than inside it." }, { "id": "lc-config-nas-multipart", "action": "abort-multipart", "afterDays": 2, "appliesTo": "", "serverSide": true, "enforcedBy": "MinIO ILM." } ], "integrity": { "method": "sha256 manifest compare over the whole prefix", "intervalDays": 30, "coverage": "full", "lastRunOn": "2026-08-03", "outcome": "pass" }, "notes": [], "id": "c-config-nas", "endpoint": "ep-nas-minio", "role": "source", "bucket": "jarvisos-backups", "writable": true, "sizeGb": 8, "measured": true, "versioning": true, "objectLock": false, "encryption": { "inTransit": "tls", "atRest": "dataset-level", "keyLocation": null, "keyOnSourceNode": false, "note": "Pool encryption and a TLS hop inside the rack, same as the database backups and for the same reason." }, "bucketId": "b-config-archive", "bucketName": "Configuration archive", "class": "backup", "reproducible": false } ], "quotaPct": null, "estimated": false } }, { "id": "ep-vin-cache", "name": "MinIO cache on VIN", "kind": "minio", "purpose": "A read-through cache so a build on VIN does not cross the home uplink for every layer, and so something still answers for artifacts when the house is dark. It is a cache and nothing else: it is never the source of truth for anything, and losing it costs a slower build.", "entityId": "vin", "node": "vin", "tier": "t-local-vin", "domain": "fd-vps", "url": "https://s3.vin.internal", "addressing": "path", "region": "vps", "tls": true, "anonymousRead": false, "status": "unknown", "capabilities": { "versioning": false, "objectLock": false, "lifecycle": true, "multipart": true, "presign": true, "serverSideReplication": false, "checksums": "sha256" }, "bandwidthMbps": { "read": 2000, "write": 2000, "measuredOn": null, "note": "Unmeasured, and deliberately so: this is local NVMe on the node doing the reading, and no restore plan depends on the number." }, "cost": { "model": "included", "monthlyEur": 0, "quotaGb": 40, "egressEurPerGb": 0, "note": "Part of the VPS plan's disk. The quota is self-imposed: the plan's disk is shared with Postgres and the worktrees, and a cache that grows without a ceiling is how a rented disk fills at 3am." }, "reachableFrom": [ { "via": [], "node": "vin", "note": "Loopback on VIN. This is the path that matters: the cache exists for the node it is on." }, { "via": [ "wireguard-tunnel", "isp-uplink" ], "node": "home1", "note": "Over the mesh, for warming it deliberately after a release. Not a path anything depends on." } ], "credentials": [ { "id": "cred-vin-cache-rw", "envName": "JARVISOS_S3_VIN_RW", "location": "/etc/jarvisos/s3-vin-rw.env", "scope": "read-write", "usedBy": [ "vin" ] } ], "design": { "sizeGb": 12, "objects": 9000, "quotaPct": 30 }, "today": { "sizeGb": 0, "objects": 0, "placed": [], "quotaPct": 0, "estimated": false } }, { "id": "ep-gwscli-offsite", "name": "GWSCLI / S3 off-site", "kind": "gwscli", "purpose": "The encrypted subset that survives the house. It holds the backup buckets and nothing reproducible, because paying to move bytes off-site that a build would recreate is buying a slower version of something we already have.", "entityId": "gwscli-s3-offsite", "node": null, "tier": "t-offsite-gwscli", "domain": "fd-offsite", "url": "https://s3.gwscli.example.net", "addressing": "path", "region": "offsite", "tls": true, "anonymousRead": false, "status": "unknown", "capabilities": { "versioning": false, "objectLock": false, "lifecycle": false, "multipart": true, "presign": false, "serverSideReplication": false, "checksums": "none" }, "bandwidthMbps": { "read": 200, "write": 40, "measuredOn": "2026-07-12", "note": "Read measured from VIN during the July restore drill. Write is the home uplink's upstream, which is the real ceiling on how current the off-site copy can be." }, "cost": { "model": "free-tier", "monthlyEur": 0, "quotaGb": 100, "egressEurPerGb": null, "note": "Free up to the quota. Egress has never been priced against an invoice, which means the cost of the restore this endpoint exists for is unknown until it happens. That is the one number here worth going and finding out." }, "reachableFrom": [ { "via": [], "node": "vin", "note": "Straight out of the VPS's own uplink. This is the restore path: it does not touch the home rack, which is the point." }, { "via": [ "isp-uplink" ], "node": "home1", "note": "The write path. Everything pushed off-site crosses the home uplink, which is why an uplink outage silently stops the estate advancing its off-site copy." } ], "credentials": [ { "id": "cred-offsite-rw", "envName": "JARVISOS_S3_OFFSITE_RW", "location": "/etc/jarvisos/s3-offsite-rw.env", "scope": "read-write", "usedBy": [ "home1" ] }, { "id": "cred-offsite-ro", "envName": "JARVISOS_S3_OFFSITE_RO", "location": "/etc/jarvisos/s3-offsite-ro.env", "scope": "read", "usedBy": [ "vin" ] } ], "design": { "sizeGb": 65, "objects": 10900, "quotaPct": 65 }, "today": { "sizeGb": 65, "objects": 10900, "placed": [ { "status": "running", "prefix": "restic/backups/", "objects": 9100, "observedOn": "2026-08-15", "validates": null, "writtenBy": [], "lifecycle": [ { "id": "lc-db-offsite-forget", "action": "expire", "afterDays": 190, "appliesTo": "restic/backups/", "serverSide": false, "enforcedBy": "restic forget --prune, inside the same push that writes it, and its exit status fails the push. A server-side expiry rule here would not be retention, it would be repository corruption: the provider cannot tell a pack file that is still referenced from one that is not." }, { "id": "lc-db-offsite-multipart", "action": "abort-multipart", "afterDays": 3, "appliesTo": "", "serverSide": false, "enforcedBy": "`jarvisos-objectstore.mjs lifecycle` lists this as a rule the far end applies none of; the operator aborts stale parts by hand, quarterly. This endpoint runs no rules of its own and orphaned parts here are billed against the quota." } ], "integrity": { "method": "restic check --read-data-subset=5%", "intervalDays": 30, "coverage": "sampled", "samplePct": 5, "lastRunOn": "2026-08-10", "outcome": "pass" }, "notes": [ "62 GB of restic packs is 240 GB of backups: the difference is deduplication and compression, and it is measured rather than assumed because the repository reports both." ], "id": "c-db-offsite", "endpoint": "ep-gwscli-offsite", "role": "replica", "bucket": "jarvisos-offsite", "writable": false, "sizeGb": 62, "measured": true, "versioning": false, "objectLock": false, "encryption": { "inTransit": "tls", "atRest": "client-side", "keyLocation": "The restic repository passphrase, held offline by the operator. By policy it is on no node in this estate, including the one that writes it.", "keyOnSourceNode": false, "note": null }, "bucketId": "b-db-backups", "bucketName": "Database backups", "class": "backup", "reproducible": false }, { "status": "running", "prefix": "restic/config/", "objects": 1800, "observedOn": "2026-08-15", "validates": null, "writtenBy": [], "lifecycle": [ { "id": "lc-config-offsite-forget", "action": "expire", "afterDays": 400, "appliesTo": "restic/config/", "serverSide": false, "enforcedBy": "restic forget --prune in the same push, exit status included." }, { "id": "lc-config-offsite-multipart", "action": "abort-multipart", "afterDays": 3, "appliesTo": "", "serverSide": false, "enforcedBy": "Listed by `jarvisos-objectstore.mjs lifecycle` and aborted by hand, quarterly, in the same sitting as the database repository." } ], "integrity": { "method": "restic check --read-data (the repository is small enough to read all of it)", "intervalDays": 30, "coverage": "full", "lastRunOn": "2026-08-10", "outcome": "pass" }, "notes": [], "id": "c-config-offsite", "endpoint": "ep-gwscli-offsite", "role": "replica", "bucket": "jarvisos-offsite", "writable": false, "sizeGb": 3, "measured": true, "versioning": false, "objectLock": false, "encryption": { "inTransit": "tls", "atRest": "client-side", "keyLocation": "The same offline restic passphrase as the database backups. One passphrase, held in one place that is not this estate.", "keyOnSourceNode": false, "note": null }, "bucketId": "b-config-archive", "bucketName": "Configuration archive", "class": "backup", "reproducible": false } ], "quotaPct": 65, "estimated": false } } ], "buckets": [ { "id": "b-db-backups", "name": "Database backups", "purpose": "Postgres base backups, WAL archives and the Supabase dumps. When this is gone, the estate's history is gone with it: there is no way to reconstruct a table from anything else that exists.", "owner": "homelab-operator", "class": "backup", "dataset": "db-backups", "reproducible": false, "objectives": { "rpoMinutes": 1440, "rtoHours": 8, "retentionDays": 180, "mustSurviveSite": true, "readableDuring": [ "sc-home-offline", "sc-vin-offline", "sc-uplink-down", "sc-offsite-gone" ], "writableDuring": [ "sc-vin-offline", "sc-uplink-down", "sc-offsite-gone" ] }, "design": { "sizeGb": 240, "objects": 27500, "hasOffsite": true, "rpoMinutes": 1440 }, "today": { "sizeGb": 240, "objects": 27500, "hasOffsite": true, "rpoMinutes": 1440, "copies": [ { "status": "running", "prefix": "postgres/", "objects": 18400, "observedOn": "2026-08-14", "validates": null, "writtenBy": [ "home1" ], "lifecycle": [ { "id": "lc-db-nas-expire", "action": "expire", "afterDays": 190, "appliesTo": "postgres/", "serverSide": true, "enforcedBy": "MinIO ILM (mc ilm rule add), applied on the NAS. Set beyond the 180-day retention on purpose: the retention command decides what to delete, and the lifecycle rule is only the backstop for what it misses." }, { "id": "lc-db-nas-multipart", "action": "abort-multipart", "afterDays": 2, "appliesTo": "", "serverSide": true, "enforcedBy": "MinIO ILM. An interrupted 40 GB base backup leaves parts that no listing shows and that occupy the pool until something aborts them." } ], "integrity": { "method": "mc admin heal on the bucket, then a sha256 manifest compare against the newest base backup", "intervalDays": 30, "coverage": "full", "lastRunOn": "2026-08-03", "outcome": "pass" }, "notes": [], "id": "c-db-nas", "endpoint": "ep-nas-minio", "role": "source", "bucket": "jarvisos-backups", "writable": true, "sizeGb": 240, "measured": true, "versioning": true, "objectLock": false, "encryption": { "inTransit": "tls", "atRest": "dataset-level", "keyLocation": null, "keyOnSourceNode": false, "note": "The pool is encrypted at rest and the hop is TLS inside the rack. Client-side encryption is not used here on purpose: this copy is the one an operator restores from in a hurry, and a passphrase in the path is a passphrase to find first." }, "endpointName": "MinIO on the NAS", "endpointKind": "minio", "tier": "t-nas-pool", "domain": "fd-home-rack", "offsite": false, "lagMinutes": 0, "fed": true }, { "status": "running", "prefix": "restic/backups/", "objects": 9100, "observedOn": "2026-08-15", "validates": null, "writtenBy": [], "lifecycle": [ { "id": "lc-db-offsite-forget", "action": "expire", "afterDays": 190, "appliesTo": "restic/backups/", "serverSide": false, "enforcedBy": "restic forget --prune, inside the same push that writes it, and its exit status fails the push. A server-side expiry rule here would not be retention, it would be repository corruption: the provider cannot tell a pack file that is still referenced from one that is not." }, { "id": "lc-db-offsite-multipart", "action": "abort-multipart", "afterDays": 3, "appliesTo": "", "serverSide": false, "enforcedBy": "`jarvisos-objectstore.mjs lifecycle` lists this as a rule the far end applies none of; the operator aborts stale parts by hand, quarterly. This endpoint runs no rules of its own and orphaned parts here are billed against the quota." } ], "integrity": { "method": "restic check --read-data-subset=5%", "intervalDays": 30, "coverage": "sampled", "samplePct": 5, "lastRunOn": "2026-08-10", "outcome": "pass" }, "notes": [ "62 GB of restic packs is 240 GB of backups: the difference is deduplication and compression, and it is measured rather than assumed because the repository reports both." ], "id": "c-db-offsite", "endpoint": "ep-gwscli-offsite", "role": "replica", "bucket": "jarvisos-offsite", "writable": false, "sizeGb": 62, "measured": true, "versioning": false, "objectLock": false, "encryption": { "inTransit": "tls", "atRest": "client-side", "keyLocation": "The restic repository passphrase, held offline by the operator. By policy it is on no node in this estate, including the one that writes it.", "keyOnSourceNode": false, "note": null }, "endpointName": "GWSCLI / S3 off-site", "endpointKind": "gwscli", "tier": "t-offsite-gwscli", "domain": "fd-offsite", "offsite": true, "lagMinutes": 1440, "fed": true } ] }, "copies": [ { "id": "c-db-nas", "endpoint": "ep-nas-minio", "role": "source", "bucket": "jarvisos-backups", "prefix": "postgres/", "status": "running", "writable": true, "writtenBy": [ "home1" ], "domain": "fd-home-rack", "offsite": false, "sizeGb": 240, "objects": 18400, "measured": true, "observedOn": "2026-08-14", "versioning": true, "objectLock": false, "validates": null, "encryption": { "inTransit": "tls", "atRest": "dataset-level", "keyLocation": null, "keyOnSourceNode": false, "note": "The pool is encrypted at rest and the hop is TLS inside the rack. Client-side encryption is not used here on purpose: this copy is the one an operator restores from in a hurry, and a passphrase in the path is a passphrase to find first." }, "integrity": { "method": "mc admin heal on the bucket, then a sha256 manifest compare against the newest base backup", "intervalDays": 30, "coverage": "full", "lastRunOn": "2026-08-03", "outcome": "pass" }, "lifecycle": [ { "id": "lc-db-nas-expire", "action": "expire", "afterDays": 190, "appliesTo": "postgres/", "serverSide": true, "enforcedBy": "MinIO ILM (mc ilm rule add), applied on the NAS. Set beyond the 180-day retention on purpose: the retention command decides what to delete, and the lifecycle rule is only the backstop for what it misses." }, { "id": "lc-db-nas-multipart", "action": "abort-multipart", "afterDays": 2, "appliesTo": "", "serverSide": true, "enforcedBy": "MinIO ILM. An interrupted 40 GB base backup leaves parts that no listing shows and that occupy the pool until something aborts them." } ], "lagMinutes": 0, "fed": true }, { "id": "c-db-offsite", "endpoint": "ep-gwscli-offsite", "role": "replica", "bucket": "jarvisos-offsite", "prefix": "restic/backups/", "status": "running", "writable": false, "writtenBy": [], "domain": "fd-offsite", "offsite": true, "sizeGb": 62, "objects": 9100, "measured": true, "observedOn": "2026-08-15", "versioning": false, "objectLock": false, "validates": null, "encryption": { "inTransit": "tls", "atRest": "client-side", "keyLocation": "The restic repository passphrase, held offline by the operator. By policy it is on no node in this estate, including the one that writes it.", "keyOnSourceNode": false, "note": null }, "integrity": { "method": "restic check --read-data-subset=5%", "intervalDays": 30, "coverage": "sampled", "samplePct": 5, "lastRunOn": "2026-08-10", "outcome": "pass" }, "lifecycle": [ { "id": "lc-db-offsite-forget", "action": "expire", "afterDays": 190, "appliesTo": "restic/backups/", "serverSide": false, "enforcedBy": "restic forget --prune, inside the same push that writes it, and its exit status fails the push. A server-side expiry rule here would not be retention, it would be repository corruption: the provider cannot tell a pack file that is still referenced from one that is not." }, { "id": "lc-db-offsite-multipart", "action": "abort-multipart", "afterDays": 3, "appliesTo": "", "serverSide": false, "enforcedBy": "`jarvisos-objectstore.mjs lifecycle` lists this as a rule the far end applies none of; the operator aborts stale parts by hand, quarterly. This endpoint runs no rules of its own and orphaned parts here are billed against the quota." } ], "lagMinutes": 1440, "fed": true } ], "replication": [ { "id": "e-db-offsite", "from": "c-db-nas", "to": "c-db-offsite", "mode": "push", "mechanism": "restic backup with the S3 backend, reading the NAS bucket through mc mirror into a staging path", "schedule": "nightly at 02:20 Europe/Amsterdam", "intervalMinutes": 1440, "deleteBehaviour": "retain", "status": "running", "runsOn": "home1", "verifiedBy": "restic check --read-data-subset=5% after the push, and the snapshot count against the expected retention", "lastVerifiedOn": "2026-08-10", "outcome": "pass" } ], "readOrder": [ "c-db-nas", "c-db-offsite" ], "writeFailover": null, "restore": { "overheadMinutes": 45, "procedure": [ "On VIN: source /etc/jarvisos/s3-offsite-ro.env — the read-only off-site key.", "Fetch the repository passphrase from wherever it is held offline. It is on no node, including this one.", "restic -r s3:https://s3.gwscli.example.net/jarvisos-offsite/restic/backups snapshots — confirm the newest snapshot is within the recovery point.", "restic restore latest --target /srv/restore/pg --verify", "Compare the restored base backup's sha256 against the manifest in the same snapshot before starting Postgres against it.", "Record the wall-clock duration in this register. An untimed restore leaves the recovery time an estimate." ], "drills": [ { "date": "2026-07-12", "from": "c-db-nas", "outcome": "pass", "measuredMinutes": 41, "sampleGb": 18, "restoredTo": "home1:/srv/restore/pg-drill", "note": "Restored the newest base backup from the NAS and started Postgres against it. Proves the NAS, the manifest and the procedure — and nothing about the copy that survives the room." } ] }, "restores": [ { "copy": "c-db-nas", "offsite": false, "role": "source", "mbps": 940, "transferMinutes": 34.85957446808511, "overheadMinutes": 45, "hours": 1.3309929078014184, "eur": 0, "egressPriced": true }, { "copy": "c-db-offsite", "offsite": true, "role": "replica", "mbps": 200, "transferMinutes": 42.32533333333333, "overheadMinutes": 45, "hours": 1.4554222222222222, "eur": null, "egressPriced": false } ], "bestOffsiteRestore": { "copy": "c-db-offsite", "offsite": true, "role": "replica", "mbps": 200, "transferMinutes": 42.32533333333333, "overheadMinutes": 45, "hours": 1.4554222222222222, "eur": null, "egressPriced": false }, "lastDrill": { "date": "2026-07-12", "from": "c-db-nas", "outcome": "pass", "measuredMinutes": 41, "sampleGb": 18, "restoredTo": "home1:/srv/restore/pg-drill", "note": "Restored the newest base backup from the NAS and started Postgres against it. Proves the NAS, the manifest and the procedure — and nothing about the copy that survives the room." }, "offsiteDrilled": false }, { "id": "b-config-archive", "name": "Configuration archive", "purpose": "Coolify's own database dumps, the compose and environment files with credentials removed, and the Caddy configuration. It answers the question nobody plans for: what did this look like before the change nobody remembers making.", "owner": "homelab-operator", "class": "backup", "dataset": "config-archive", "reproducible": false, "objectives": { "rpoMinutes": 1440, "rtoHours": 2, "retentionDays": 360, "mustSurviveSite": true, "readableDuring": [ "sc-home-offline", "sc-vin-offline", "sc-uplink-down", "sc-offsite-gone" ], "writableDuring": [ "sc-vin-offline", "sc-uplink-down", "sc-offsite-gone" ] }, "design": { "sizeGb": 8, "objects": 6000, "hasOffsite": true, "rpoMinutes": 1440 }, "today": { "sizeGb": 8, "objects": 6000, "hasOffsite": true, "rpoMinutes": 1440, "copies": [ { "status": "running", "prefix": "config/", "objects": 4200, "observedOn": "2026-08-14", "validates": null, "writtenBy": [ "home1" ], "lifecycle": [ { "id": "lc-config-nas-expire", "action": "expire", "afterDays": 400, "appliesTo": "config/", "serverSide": true, "enforcedBy": "MinIO ILM. A year of monthly points is what this archive is for, so the rule sits beyond the retention window rather than inside it." }, { "id": "lc-config-nas-multipart", "action": "abort-multipart", "afterDays": 2, "appliesTo": "", "serverSide": true, "enforcedBy": "MinIO ILM." } ], "integrity": { "method": "sha256 manifest compare over the whole prefix", "intervalDays": 30, "coverage": "full", "lastRunOn": "2026-08-03", "outcome": "pass" }, "notes": [], "id": "c-config-nas", "endpoint": "ep-nas-minio", "role": "source", "bucket": "jarvisos-backups", "writable": true, "sizeGb": 8, "measured": true, "versioning": true, "objectLock": false, "encryption": { "inTransit": "tls", "atRest": "dataset-level", "keyLocation": null, "keyOnSourceNode": false, "note": "Pool encryption and a TLS hop inside the rack, same as the database backups and for the same reason." }, "endpointName": "MinIO on the NAS", "endpointKind": "minio", "tier": "t-nas-pool", "domain": "fd-home-rack", "offsite": false, "lagMinutes": 0, "fed": true }, { "status": "running", "prefix": "restic/config/", "objects": 1800, "observedOn": "2026-08-15", "validates": null, "writtenBy": [], "lifecycle": [ { "id": "lc-config-offsite-forget", "action": "expire", "afterDays": 400, "appliesTo": "restic/config/", "serverSide": false, "enforcedBy": "restic forget --prune in the same push, exit status included." }, { "id": "lc-config-offsite-multipart", "action": "abort-multipart", "afterDays": 3, "appliesTo": "", "serverSide": false, "enforcedBy": "Listed by `jarvisos-objectstore.mjs lifecycle` and aborted by hand, quarterly, in the same sitting as the database repository." } ], "integrity": { "method": "restic check --read-data (the repository is small enough to read all of it)", "intervalDays": 30, "coverage": "full", "lastRunOn": "2026-08-10", "outcome": "pass" }, "notes": [], "id": "c-config-offsite", "endpoint": "ep-gwscli-offsite", "role": "replica", "bucket": "jarvisos-offsite", "writable": false, "sizeGb": 3, "measured": true, "versioning": false, "objectLock": false, "encryption": { "inTransit": "tls", "atRest": "client-side", "keyLocation": "The same offline restic passphrase as the database backups. One passphrase, held in one place that is not this estate.", "keyOnSourceNode": false, "note": null }, "endpointName": "GWSCLI / S3 off-site", "endpointKind": "gwscli", "tier": "t-offsite-gwscli", "domain": "fd-offsite", "offsite": true, "lagMinutes": 1440, "fed": true } ] }, "copies": [ { "id": "c-config-nas", "endpoint": "ep-nas-minio", "role": "source", "bucket": "jarvisos-backups", "prefix": "config/", "status": "running", "writable": true, "writtenBy": [ "home1" ], "domain": "fd-home-rack", "offsite": false, "sizeGb": 8, "objects": 4200, "measured": true, "observedOn": "2026-08-14", "versioning": true, "objectLock": false, "validates": null, "encryption": { "inTransit": "tls", "atRest": "dataset-level", "keyLocation": null, "keyOnSourceNode": false, "note": "Pool encryption and a TLS hop inside the rack, same as the database backups and for the same reason." }, "integrity": { "method": "sha256 manifest compare over the whole prefix", "intervalDays": 30, "coverage": "full", "lastRunOn": "2026-08-03", "outcome": "pass" }, "lifecycle": [ { "id": "lc-config-nas-expire", "action": "expire", "afterDays": 400, "appliesTo": "config/", "serverSide": true, "enforcedBy": "MinIO ILM. A year of monthly points is what this archive is for, so the rule sits beyond the retention window rather than inside it." }, { "id": "lc-config-nas-multipart", "action": "abort-multipart", "afterDays": 2, "appliesTo": "", "serverSide": true, "enforcedBy": "MinIO ILM." } ], "lagMinutes": 0, "fed": true }, { "id": "c-config-offsite", "endpoint": "ep-gwscli-offsite", "role": "replica", "bucket": "jarvisos-offsite", "prefix": "restic/config/", "status": "running", "writable": false, "writtenBy": [], "domain": "fd-offsite", "offsite": true, "sizeGb": 3, "objects": 1800, "measured": true, "observedOn": "2026-08-15", "versioning": false, "objectLock": false, "validates": null, "encryption": { "inTransit": "tls", "atRest": "client-side", "keyLocation": "The same offline restic passphrase as the database backups. One passphrase, held in one place that is not this estate.", "keyOnSourceNode": false, "note": null }, "integrity": { "method": "restic check --read-data (the repository is small enough to read all of it)", "intervalDays": 30, "coverage": "full", "lastRunOn": "2026-08-10", "outcome": "pass" }, "lifecycle": [ { "id": "lc-config-offsite-forget", "action": "expire", "afterDays": 400, "appliesTo": "restic/config/", "serverSide": false, "enforcedBy": "restic forget --prune in the same push, exit status included." }, { "id": "lc-config-offsite-multipart", "action": "abort-multipart", "afterDays": 3, "appliesTo": "", "serverSide": false, "enforcedBy": "Listed by `jarvisos-objectstore.mjs lifecycle` and aborted by hand, quarterly, in the same sitting as the database repository." } ], "lagMinutes": 1440, "fed": true } ], "replication": [ { "id": "e-config-offsite", "from": "c-config-nas", "to": "c-config-offsite", "mode": "push", "mechanism": "restic backup with the S3 backend, in the same nightly unit as the database push", "schedule": "nightly at 02:40 Europe/Amsterdam", "intervalMinutes": 1440, "deleteBehaviour": "retain", "status": "running", "runsOn": "home1", "verifiedBy": "restic check --read-data after the push, and a diff of the newest snapshot's file list against the source prefix", "lastVerifiedOn": "2026-08-10", "outcome": "pass" } ], "readOrder": [ "c-config-nas", "c-config-offsite" ], "writeFailover": null, "restore": { "overheadMinutes": 20, "procedure": [ "On VIN: source /etc/jarvisos/s3-offsite-ro.env.", "Fetch the restic passphrase from offline storage.", "restic -r s3:https://s3.gwscli.example.net/jarvisos-offsite/restic/config snapshots", "restic restore latest --target /srv/restore/config --verify", "Read the restored env files before using them: credentials were removed on the way in, so anything that looks like one is a scrubbing failure and a finding." ], "drills": [] }, "restores": [ { "copy": "c-config-nas", "offsite": false, "role": "source", "mbps": 940, "transferMinutes": 1.1619858156028369, "overheadMinutes": 20, "hours": 0.3526997635933806, "eur": 0, "egressPriced": true }, { "copy": "c-config-offsite", "offsite": true, "role": "replica", "mbps": 200, "transferMinutes": 2.048, "overheadMinutes": 20, "hours": 0.3674666666666667, "eur": null, "egressPriced": false } ], "bestOffsiteRestore": { "copy": "c-config-offsite", "offsite": true, "role": "replica", "mbps": 200, "transferMinutes": 2.048, "overheadMinutes": 20, "hours": 0.3674666666666667, "eur": null, "egressPriced": false }, "lastDrill": null, "offsiteDrilled": false }, { "id": "b-artifacts", "name": "Build artifacts and layers", "purpose": "Build caches and container layers that a build on either node can pull instead of rebuilding. Losing all of it costs a slow afternoon, which is exactly why none of it is replicated off-site: paying to store bytes a `git clone` and a build would recreate is buying a slower version of what we already have.", "owner": "homelab-operator", "class": "cache", "dataset": "artifact-cache", "reproducible": true, "objectives": { "rpoMinutes": 1440, "rtoHours": 4, "retentionDays": 30, "mustSurviveSite": false, "readableDuring": [ "sc-home-offline" ], "writableDuring": [] }, "design": { "sizeGb": 300, "objects": 249000, "hasOffsite": false, "rpoMinutes": null }, "today": { "sizeGb": 0, "objects": 0, "hasOffsite": false, "rpoMinutes": null, "copies": [] }, "copies": [ { "id": "c-artifacts-nas", "endpoint": "ep-nas-minio", "role": "source", "bucket": "jarvisos-artifacts", "prefix": "", "status": "planned", "writable": true, "writtenBy": [ "home1" ], "domain": "fd-home-rack", "offsite": false, "sizeGb": 300, "objects": 240000, "measured": false, "observedOn": null, "versioning": false, "objectLock": false, "validates": null, "encryption": { "inTransit": "tls", "atRest": "dataset-level", "keyLocation": null, "keyOnSourceNode": false, "note": "Pool encryption is enough: nothing here is confidential and everything here is reproducible." }, "integrity": null, "lifecycle": [ { "id": "lc-artifacts-expire", "action": "expire", "afterDays": 30, "appliesTo": "", "serverSide": true, "enforcedBy": "MinIO ILM. A cache with no eviction is not a cache, it is a disk leak with good intentions." }, { "id": "lc-artifacts-multipart", "action": "abort-multipart", "afterDays": 1, "appliesTo": "", "serverSide": true, "enforcedBy": "MinIO ILM." } ], "lagMinutes": 0, "fed": true }, { "id": "c-artifacts-vin", "endpoint": "ep-vin-cache", "role": "cache", "bucket": "jarvisos-artifacts", "prefix": "", "status": "planned", "writable": false, "writtenBy": [], "domain": "fd-vps", "offsite": false, "sizeGb": 12, "objects": 9000, "measured": false, "observedOn": null, "versioning": false, "objectLock": false, "validates": "etag", "encryption": { "inTransit": "tls", "atRest": "none", "keyLocation": null, "keyOnSourceNode": false, "note": "Nothing confidential, nothing unique, on a disk that is already inside the VPS's own encryption. Encrypting a reproducible cache buys a slower build and no confidentiality anybody needed." }, "integrity": null, "lifecycle": [ { "id": "lc-artifacts-vin-expire", "action": "expire", "afterDays": 14, "appliesTo": "", "serverSide": true, "enforcedBy": "MinIO ILM on VIN. Fourteen days rather than thirty, because this disk is rented and small: the cache's job is the current release, not the quarter." }, { "id": "lc-artifacts-vin-multipart", "action": "abort-multipart", "afterDays": 1, "appliesTo": "", "serverSide": true, "enforcedBy": "MinIO ILM on VIN." } ], "lagMinutes": null, "fed": false } ], "replication": [ { "id": "e-artifacts-warm", "from": "c-artifacts-nas", "to": "c-artifacts-vin", "mode": "pull", "mechanism": "rclone copy with --immutable and --checksum, on demand from the build and once after each release", "schedule": "on demand, and after every release", "intervalMinutes": 1440, "deleteBehaviour": "retain", "status": "planned", "runsOn": "vin", "verifiedBy": "rclone check against the source prefix, comparing checksums rather than sizes", "lastVerifiedOn": null, "outcome": null } ], "readOrder": [ "c-artifacts-vin", "c-artifacts-nas" ], "writeFailover": null, "restore": { "overheadMinutes": 0, "procedure": [ "Nothing is restored. Every object here is rebuilt: `docker buildx build` against the source, and the layers repopulate as they are asked for.", "If the whole bucket is gone, the first build after that is slow and every build after it is not. That is the entire cost, and it is why nothing here is replicated off-site." ], "drills": [] }, "restores": [ { "copy": "c-artifacts-nas", "offsite": false, "role": "source", "mbps": 940, "transferMinutes": 43.57446808510638, "overheadMinutes": 0, "hours": 0.726241134751773, "eur": 0, "egressPriced": true }, { "copy": "c-artifacts-vin", "offsite": false, "role": "cache", "mbps": 2000, "transferMinutes": 0.8192, "overheadMinutes": 0, "hours": 0.013653333333333333, "eur": 0, "egressPriced": true } ], "bestOffsiteRestore": null, "lastDrill": null, "offsiteDrilled": false } ], "clients": [ { "id": "cl-home1-mc", "tool": "mc", "node": "home1", "endpoint": "ep-nas-minio", "credentialId": "cred-nas-rw", "scope": "read-write", "verifyChecksums": true, "purpose": "Writes the nightly database and configuration backups into the NAS buckets, and stages them for the off-site push.", "configPath": "/etc/jarvisos/mc/config.json" }, { "id": "cl-home1-restic", "tool": "restic", "node": "home1", "endpoint": "ep-gwscli-offsite", "credentialId": "cred-offsite-rw", "scope": "read-write", "verifyChecksums": true, "purpose": "The off-site push. Encrypts client-side with a passphrase held offline, deduplicates against what is already there, and runs its own forget --prune so retention and the push share one exit status.", "configPath": "/etc/jarvisos/restic-offsite.env" }, { "id": "cl-vin-restore", "tool": "restic", "node": "vin", "endpoint": "ep-gwscli-offsite", "credentialId": "cred-offsite-ro", "scope": "read", "verifyChecksums": true, "purpose": "The restore path with nothing at home reachable. Read-only by credential, so the node exposed to the public internet cannot delete the copy that survives the room.", "configPath": "/etc/jarvisos/restic-offsite-ro.env" }, { "id": "cl-vin-rclone", "tool": "rclone", "node": "vin", "endpoint": "ep-nas-minio", "credentialId": "cred-nas-ro", "scope": "read", "verifyChecksums": true, "purpose": "Warms the artifact cache from the NAS and, during a restore, reads backups over the mesh when the rack is reachable and faster than off-site.", "configPath": "/etc/jarvisos/rclone.conf" }, { "id": "cl-nas-admin", "tool": "mc", "node": "nas", "endpoint": "ep-nas-minio", "credentialId": "cred-nas-admin", "scope": "admin", "verifyChecksums": true, "purpose": "Bucket creation, lifecycle rules, heal and the integrity sweep. Runs on the NAS itself so the admin credential never leaves the node whose compromise already implies losing these objects.", "configPath": "/etc/jarvisos/mc/config.json" } ], "scenarios": [ { "id": "sc-home-offline", "name": "Home rack offline", "note": "Power cut, fire, flood, theft, or one mistake made as root on both machines in the room. VIN is up, serving, and is the node asking for objects. This is the scenario every off-site copy in this register exists for.", "observer": "vin", "offline": [ "fd-home-rack" ], "offlineEntities": [ "home1", "nas", "isp-uplink", "jarvisos-control-plane", "postgres-primary", "redis", "internal-dns", "nas-backups", "nas-media" ] }, { "id": "sc-vin-offline", "name": "VPS offline", "note": "Provider outage, account action, or a billing failure with the blast radius of a hardware one. The house is up and home1 is asking. Nothing durable should notice.", "observer": "home1", "offline": [ "fd-vps" ], "offlineEntities": [ "vin", "caddy", "jarvisos-auth", "jarvisos-worker", "postgres-replica" ] }, { "id": "sc-uplink-down", "name": "Home uplink down", "note": "The failure that does not look like one: both hosts are healthy, the console is green, and every copy that leaves the house has stopped advancing. Reads inside the rack are unaffected, which is why nobody finds out.", "observer": "home1", "offline": [ "isp-uplink" ], "offlineEntities": [ "isp-uplink" ] }, { "id": "sc-offsite-gone", "name": "Off-site provider gone", "note": "Vendor outage, credential loss or account action. Nothing stops working today; what is gone is the copy that survives the room, and the estate is one failure from having no backup at all.", "observer": "vin", "offline": [ "fd-offsite" ], "offlineEntities": [ "gwscli-s3", "gwscli-s3-offsite" ] } ], "matrix": [ { "bucketId": "b-db-backups", "name": "Database backups", "cells": [ { "scenario": "sc-home-offline", "read": { "outcome": "served", "copy": "c-db-offsite", "detail": "GWSCLI / S3 off-site answers, about 1d behind." }, "write": { "outcome": "lost", "copy": null, "detail": "Nothing reachable accepts writes." } }, { "scenario": "sc-vin-offline", "read": { "outcome": "served", "copy": "c-db-nas", "detail": "MinIO on the NAS answers as the authority." }, "write": { "outcome": "served", "copy": "c-db-nas", "detail": "MinIO on the NAS is reachable and is the authority." } }, { "scenario": "sc-uplink-down", "read": { "outcome": "served", "copy": "c-db-nas", "detail": "MinIO on the NAS answers as the authority." }, "write": { "outcome": "served", "copy": "c-db-nas", "detail": "MinIO on the NAS is reachable and is the authority." } }, { "scenario": "sc-offsite-gone", "read": { "outcome": "served", "copy": "c-db-nas", "detail": "MinIO on the NAS answers as the authority." }, "write": { "outcome": "served", "copy": "c-db-nas", "detail": "MinIO on the NAS is reachable and is the authority." } } ] }, { "bucketId": "b-config-archive", "name": "Configuration archive", "cells": [ { "scenario": "sc-home-offline", "read": { "outcome": "served", "copy": "c-config-offsite", "detail": "GWSCLI / S3 off-site answers, about 1d behind." }, "write": { "outcome": "lost", "copy": null, "detail": "Nothing reachable accepts writes." } }, { "scenario": "sc-vin-offline", "read": { "outcome": "served", "copy": "c-config-nas", "detail": "MinIO on the NAS answers as the authority." }, "write": { "outcome": "served", "copy": "c-config-nas", "detail": "MinIO on the NAS is reachable and is the authority." } }, { "scenario": "sc-uplink-down", "read": { "outcome": "served", "copy": "c-config-nas", "detail": "MinIO on the NAS answers as the authority." }, "write": { "outcome": "served", "copy": "c-config-nas", "detail": "MinIO on the NAS is reachable and is the authority." } }, { "scenario": "sc-offsite-gone", "read": { "outcome": "served", "copy": "c-config-nas", "detail": "MinIO on the NAS answers as the authority." }, "write": { "outcome": "served", "copy": "c-config-nas", "detail": "MinIO on the NAS is reachable and is the authority." } } ] }, { "bucketId": "b-artifacts", "name": "Build artifacts and layers", "cells": [ { "scenario": "sc-home-offline", "read": { "outcome": "lost", "copy": null, "detail": null }, "write": { "outcome": "lost", "copy": null, "detail": "Nothing reachable accepts writes." } }, { "scenario": "sc-vin-offline", "read": { "outcome": "lost", "copy": null, "detail": null }, "write": { "outcome": "lost", "copy": null, "detail": "Nothing reachable accepts writes." } }, { "scenario": "sc-uplink-down", "read": { "outcome": "lost", "copy": null, "detail": null }, "write": { "outcome": "lost", "copy": null, "detail": "Nothing reachable accepts writes." } }, { "scenario": "sc-offsite-gone", "read": { "outcome": "lost", "copy": null, "detail": null }, "write": { "outcome": "lost", "copy": null, "detail": "Nothing reachable accepts writes." } } ] } ], "findings": [ { "code": "checksums-unsupported", "severity": "gap", "fatal": false, "title": "An endpoint cannot checksum what it stores, and something here depends on it doing so.", "endpointId": "ep-gwscli-offsite", "bucketId": null, "copyId": null, "edgeId": null, "detail": "GWSCLI / S3 off-site holds backups and cannot checksum what it stores, so a silently corrupted object reads as a successful GET. Whatever verifies these bytes has to do it from the client side, and has to say so.", "acceptance": null }, { "code": "object-lock-absent", "severity": "gap", "fatal": false, "title": "A backup copy can be deleted by the credential that writes it.", "endpointId": "ep-nas-minio", "bucketId": "b-db-backups", "copyId": "c-db-nas", "edgeId": null, "detail": "MinIO on the NAS holds backups of Database backups with no object lock, so the credential that writes them can also delete them. Ransomware does not need a new capability for that; it needs the key it already has.", "acceptance": null }, { "code": "restore-drill-not-offsite", "severity": "gap", "fatal": false, "title": "Every restore drill read from a copy that does not survive the site.", "endpointId": null, "bucketId": "b-db-backups", "copyId": null, "edgeId": null, "detail": "Every restore drill of Database backups read from c-db-nas. The restore that matters reads from the off-site copy onto a node with nothing at home reachable, and it has never been run.", "acceptance": null }, { "code": "object-lock-absent", "severity": "gap", "fatal": false, "title": "A backup copy can be deleted by the credential that writes it.", "endpointId": "ep-nas-minio", "bucketId": "b-config-archive", "copyId": "c-config-nas", "edgeId": null, "detail": "MinIO on the NAS holds backups of Configuration archive with no object lock, so the credential that writes them can also delete them. Ransomware does not need a new capability for that; it needs the key it already has.", "acceptance": null }, { "code": "restore-drill-never", "severity": "gap", "fatal": false, "title": "Nobody has ever restored anything from this bucket.", "endpointId": null, "bucketId": "b-config-archive", "copyId": null, "edgeId": null, "detail": "Nobody has ever restored anything from Configuration archive.", "acceptance": null }, { "code": "restore-drill-not-offsite", "severity": "gap", "fatal": false, "title": "Every restore drill read from a copy that does not survive the site.", "endpointId": null, "bucketId": "b-config-archive", "copyId": null, "edgeId": null, "detail": "Every restore drill of Configuration archive read from nowhere. The restore that matters reads from the off-site copy onto a node with nothing at home reachable, and it has never been run.", "acceptance": null }, { "code": "source-not-running", "severity": "gap", "fatal": false, "title": "A bucket’s source of truth is declared and not running.", "endpointId": null, "bucketId": "b-artifacts", "copyId": "c-artifacts-nas", "edgeId": null, "detail": "Build artifacts and layers's authority (MinIO on the NAS) is declared and not running, so writes land nowhere it defines.", "acceptance": null }, { "code": "read-objective-missed", "severity": "gap", "fatal": false, "title": "A bucket is unreadable in a scenario it is required to be readable in.", "endpointId": null, "bucketId": "b-artifacts", "copyId": null, "edgeId": null, "detail": "Build artifacts and layers is required to be readable during \"Home rack offline\" and nothing reachable from vin holds it. Read order: c-artifacts-vin → c-artifacts-nas.", "acceptance": null }, { "code": "object-lock-absent", "severity": "accepted", "fatal": false, "title": "A backup copy can be deleted by the credential that writes it.", "endpointId": "ep-gwscli-offsite", "bucketId": "b-db-backups", "copyId": "c-db-offsite", "edgeId": null, "detail": "GWSCLI / S3 off-site holds backups of Database backups with no object lock, so the credential that writes them can also delete them. Ransomware does not need a new capability for that; it needs the key it already has.", "acceptance": { "reason": "GWSCLI/S3 exposes no object lock and no append-only credential scope, so the push key on home1 can delete what it wrote. The compensating controls are that the key exists only on home1, that the edge retains deletes rather than propagating them, and that the retention keeps six monthly points a single bad run cannot reach. A compromise of home1 can still destroy this copy, and that is the risk being accepted until an endpoint with object lock exists.", "acceptedBy": "homelab-operator", "reviewOn": "2026-11-01" } }, { "code": "object-lock-absent", "severity": "accepted", "fatal": false, "title": "A backup copy can be deleted by the credential that writes it.", "endpointId": "ep-gwscli-offsite", "bucketId": "b-config-archive", "copyId": "c-config-offsite", "edgeId": null, "detail": "GWSCLI / S3 off-site holds backups of Configuration archive with no object lock, so the credential that writes them can also delete them. Ransomware does not need a new capability for that; it needs the key it already has.", "acceptance": { "reason": "Same endpoint and same reasoning as the database backups: no object lock exists to enable. Accepted together with that one and reviewed on the same date, because they are one decision about one provider.", "acceptedBy": "homelab-operator", "reviewOn": "2026-11-01" } }, { "code": "lifecycle-shorter-than-retention", "severity": "accepted", "fatal": false, "title": "A lifecycle rule deletes objects sooner than the retention they are kept for.", "endpointId": "ep-vin-cache", "bucketId": "b-artifacts", "copyId": "c-artifacts-vin", "edgeId": null, "detail": "Rule \"lc-artifacts-vin-expire\" on MinIO cache on VIN expires objects after 14 days; Build artifacts and layers is required to be recoverable for 30. The recovery point beyond 14 days does not exist, whatever any other register says.", "acceptance": { "reason": "The VIN cache expires objects after 14 days against a 30-day retention. That is correct for a cache on a rented disk: an object aged out early costs one pull from the NAS, and the alternative is a VPS disk that fills with layers nobody has asked for since the last release. The finding is kept open in this form so the day somebody puts something unique in this bucket, it reads as the data-loss risk it would then be.", "acceptedBy": "homelab-operator", "reviewOn": "2026-12-01" } }, { "code": "credential-absent", "severity": "note", "fatal": false, "title": "A declared credential is not present, so nothing on that node can reach the endpoint today.", "endpointId": "ep-nas-minio", "bucketId": null, "copyId": null, "edgeId": null, "detail": "cred-nas-rw is declared in JARVISOS_S3_NAS_RW (/etc/jarvisos/s3-nas-rw.env) and is not present on this host, so nothing here can reach MinIO on the NAS today. That is the correct state on a node that is not meant to.", "acceptance": null }, { "code": "credential-absent", "severity": "note", "fatal": false, "title": "A declared credential is not present, so nothing on that node can reach the endpoint today.", "endpointId": "ep-nas-minio", "bucketId": null, "copyId": null, "edgeId": null, "detail": "cred-nas-ro is declared in JARVISOS_S3_NAS_RO (/etc/jarvisos/s3-nas-ro.env) and is not present on this host, so nothing here can reach MinIO on the NAS today. That is the correct state on a node that is not meant to.", "acceptance": null }, { "code": "credential-absent", "severity": "note", "fatal": false, "title": "A declared credential is not present, so nothing on that node can reach the endpoint today.", "endpointId": "ep-nas-minio", "bucketId": null, "copyId": null, "edgeId": null, "detail": "cred-nas-admin is declared in JARVISOS_S3_NAS_ADMIN (/etc/jarvisos/s3-nas-admin.env) and is not present on this host, so nothing here can reach MinIO on the NAS today. That is the correct state on a node that is not meant to.", "acceptance": null }, { "code": "credential-absent", "severity": "note", "fatal": false, "title": "A declared credential is not present, so nothing on that node can reach the endpoint today.", "endpointId": "ep-vin-cache", "bucketId": null, "copyId": null, "edgeId": null, "detail": "cred-vin-cache-rw is declared in JARVISOS_S3_VIN_RW (/etc/jarvisos/s3-vin-rw.env) and is not present on this host, so nothing here can reach MinIO cache on VIN today. That is the correct state on a node that is not meant to.", "acceptance": null }, { "code": "credential-absent", "severity": "note", "fatal": false, "title": "A declared credential is not present, so nothing on that node can reach the endpoint today.", "endpointId": "ep-gwscli-offsite", "bucketId": null, "copyId": null, "edgeId": null, "detail": "cred-offsite-rw is declared in JARVISOS_S3_OFFSITE_RW (/etc/jarvisos/s3-offsite-rw.env) and is not present on this host, so nothing here can reach GWSCLI / S3 off-site today. That is the correct state on a node that is not meant to.", "acceptance": null }, { "code": "credential-absent", "severity": "note", "fatal": false, "title": "A declared credential is not present, so nothing on that node can reach the endpoint today.", "endpointId": "ep-gwscli-offsite", "bucketId": null, "copyId": null, "edgeId": null, "detail": "cred-offsite-ro is declared in JARVISOS_S3_OFFSITE_RO (/etc/jarvisos/s3-offsite-ro.env) and is not present on this host, so nothing here can reach GWSCLI / S3 off-site today. That is the correct state on a node that is not meant to.", "acceptance": null }, { "code": "integrity-sampled-only", "severity": "note", "fatal": false, "title": "A copy is verified by sampling, so most of it has never been read.", "endpointId": "ep-gwscli-offsite", "bucketId": "b-db-backups", "copyId": "c-db-offsite", "edgeId": null, "detail": "Database backups on GWSCLI / S3 off-site: \"restic check --read-data-subset=5%\" reads 5% of the objects. Sampling finds systemic corruption cheaply and will never find the one object that matters.", "acceptance": null }, { "code": "lifecycle-not-server-side", "severity": "note", "fatal": false, "title": "A lifecycle rule depends on a client running, because the endpoint applies none.", "endpointId": "ep-gwscli-offsite", "bucketId": "b-db-backups", "copyId": "c-db-offsite", "edgeId": null, "detail": "Rule \"lc-db-offsite-forget\" on GWSCLI / S3 off-site is applied by restic forget --prune, inside the same push that writes it, and its exit status fails the push. A server-side expiry rule here would not be retention, it would be repository corruption: the provider cannot tell a pack file that is still referenced from one that is not. rather than by the endpoint. It stops happening the day that timer stops, and nothing at the far end notices.", "acceptance": null }, { "code": "lifecycle-not-server-side", "severity": "note", "fatal": false, "title": "A lifecycle rule depends on a client running, because the endpoint applies none.", "endpointId": "ep-gwscli-offsite", "bucketId": "b-db-backups", "copyId": "c-db-offsite", "edgeId": null, "detail": "Rule \"lc-db-offsite-multipart\" on GWSCLI / S3 off-site is applied by `jarvisos-objectstore.mjs lifecycle` lists this as a rule the far end applies none of; the operator aborts stale parts by hand, quarterly. This endpoint runs no rules of its own and orphaned parts here are billed against the quota. rather than by the endpoint. It stops happening the day that timer stops, and nothing at the far end notices.", "acceptance": null }, { "code": "restore-egress-unpriced", "severity": "note", "fatal": false, "title": "A restore crosses a metered endpoint whose egress price is unknown.", "endpointId": null, "bucketId": "b-db-backups", "copyId": "c-db-offsite", "edgeId": null, "detail": "Nobody has priced egress from GWSCLI / S3 off-site, so the cost of restoring 240 GB is unknown until the invoice arrives.", "acceptance": null }, { "code": "lifecycle-not-server-side", "severity": "note", "fatal": false, "title": "A lifecycle rule depends on a client running, because the endpoint applies none.", "endpointId": "ep-gwscli-offsite", "bucketId": "b-config-archive", "copyId": "c-config-offsite", "edgeId": null, "detail": "Rule \"lc-config-offsite-forget\" on GWSCLI / S3 off-site is applied by restic forget --prune in the same push, exit status included. rather than by the endpoint. It stops happening the day that timer stops, and nothing at the far end notices.", "acceptance": null }, { "code": "lifecycle-not-server-side", "severity": "note", "fatal": false, "title": "A lifecycle rule depends on a client running, because the endpoint applies none.", "endpointId": "ep-gwscli-offsite", "bucketId": "b-config-archive", "copyId": "c-config-offsite", "edgeId": null, "detail": "Rule \"lc-config-offsite-multipart\" on GWSCLI / S3 off-site is applied by Listed by `jarvisos-objectstore.mjs lifecycle` and aborted by hand, quarterly, in the same sitting as the database repository. rather than by the endpoint. It stops happening the day that timer stops, and nothing at the far end notices.", "acceptance": null }, { "code": "restore-egress-unpriced", "severity": "note", "fatal": false, "title": "A restore crosses a metered endpoint whose egress price is unknown.", "endpointId": null, "bucketId": "b-config-archive", "copyId": "c-config-offsite", "edgeId": null, "detail": "Nobody has priced egress from GWSCLI / S3 off-site, so the cost of restoring 8 GB is unknown until the invoice arrives.", "acceptance": null }, { "code": "replication-not-running", "severity": "note", "fatal": false, "title": "A declared replication edge is not running, so it moves nothing today.", "endpointId": null, "bucketId": "b-artifacts", "copyId": "c-artifacts-vin", "edgeId": "e-artifacts-warm", "detail": "\"e-artifacts-warm\" is planned: nothing reaches MinIO cache on VIN today.", "acceptance": null } ], "source": "/app/config/objectstore.json" }