diff --git a/.agent/skills/bastille-network/references/troubleshooting.md b/.agent/skills/bastille-network/references/troubleshooting.md index 031e040..e8f3b84 100644 --- a/.agent/skills/bastille-network/references/troubleshooting.md +++ b/.agent/skills/bastille-network/references/troubleshooting.md @@ -65,6 +65,20 @@ Response: - recreate or provision with: - `-g 192.168.100.1` +### Archived console log shows `add net default: gateway 51.83.197.1 ... Invalid argument` + +Interpretation: +- the jail boot sequence attempted to install the host uplink gateway +- that is the wrong gateway for the Warden subnet + +Meaning: +- Bastille VNET creation is incomplete or wrong +- this is not a DNS or NAT problem yet + +Response: +- fix the create path to use: + - `-g 192.168.100.1` + ### Successful VNET thick jail creation prints `e0a_*` and `e0b_*` Interpretation: diff --git a/.agent/skills/postgres-memory/SKILL.md b/.agent/skills/postgres-memory/SKILL.md index 45bb700..6ba2477 100644 --- a/.agent/skills/postgres-memory/SKILL.md +++ b/.agent/skills/postgres-memory/SKILL.md @@ -89,6 +89,7 @@ Target future flow: - Install commands: `references/install.md` - Validation sequence: `references/validation.md` - ZFS and service notes: `references/layout.md` +- Failure signatures: `references/troubleshooting.md` ## Scripts diff --git a/.agent/skills/postgres-memory/references/troubleshooting.md b/.agent/skills/postgres-memory/references/troubleshooting.md new file mode 100644 index 0000000..94bb264 --- /dev/null +++ b/.agent/skills/postgres-memory/references/troubleshooting.md @@ -0,0 +1,55 @@ +# PostgreSQL Memory Jail Troubleshooting + +Use this file when the `db` jail is created successfully but still is not ready +for PostgreSQL installation. + +## Common failure signatures + +### `hostname` is still `db` + +Likely cause: +- the jail was created successfully +- the hostname was not yet updated after creation + +Response: +- apply the intended hostname with Bastille config +- restart the jail before treating hostname mismatch as a deeper failure + +### `vnet0` has `192.168.100.2/24` but `netstat -rn` has no `default` + +Likely cause: +- Bastille VNET create path omitted the explicit Warden gateway + +Observed pattern: +- jail creation succeeds +- `vnet0` has the expected address +- the routing table only shows the local subnet + +Response: +- recreate or provision with: + - `-g 192.168.100.1` +- do not rely on a manual `route add default 192.168.100.1` as the final fix + +### Bastille console log shows `add net default: gateway 51.83.197.1 ... Invalid argument` + +Likely cause: +- Bastille inferred the host uplink gateway instead of the Warden gateway +- this is a provisioning defect for the VNET jail, not a PostgreSQL problem + +Interpretation: +- host bridge and jail creation may still be correct +- but the jail routing model is wrong + +Response: +- treat this as evidence the create command is missing: + - `-g 192.168.100.1` + +### Bastille destroy prints `Note: jail console logs archived.` + +Interpretation: +- expected and useful +- archived console logs under `/var/log/bastille/` are a valid troubleshooting source + +Recommended use: +- capture meaningful signatures from the archived logs into skill references +- avoid treating the raw log archive as the long-term knowledge store diff --git a/.agent/skills/warden-bootstrap/references/troubleshooting.md b/.agent/skills/warden-bootstrap/references/troubleshooting.md index 4ab8406..36ff519 100644 --- a/.agent/skills/warden-bootstrap/references/troubleshooting.md +++ b/.agent/skills/warden-bootstrap/references/troubleshooting.md @@ -55,3 +55,14 @@ Likely cause: Response: - use `bastille destroy -y controlplane` in deterministic operator or agent flows - remember that destroy archives console logs under `/var/log/bastille/` + +### Archived Bastille console log shows `add net default: gateway 51.83.197.1 ... Invalid argument` + +Likely cause: +- VNET jail was created without the explicit Warden gateway +- Bastille inferred the host uplink gateway instead + +Response: +- fix the create path to include: + - `-g 192.168.100.1` +- recreate the jail instead of layering manual route fixes over bad provisioning diff --git a/docs/POSTGRES-MEMORY.md b/docs/POSTGRES-MEMORY.md index ea2ddc4..0f428a5 100644 --- a/docs/POSTGRES-MEMORY.md +++ b/docs/POSTGRES-MEMORY.md @@ -95,6 +95,16 @@ If a VNET `db` jail comes up without a `default` route, treat that as a provisio - the create path is missing the explicit `-g 192.168.100.1` flag - fix the create command rather than adding the route manually and forgetting the root cause +Useful real-world signal from the archived Bastille console log: + +- `add net default: gateway 51.83.197.1 ... Invalid argument` + +Interpretation: + +- Bastille tried to use the host uplink gateway +- the correct gateway for the Warden subnet is `192.168.100.1` +- this is evidence of a bad VNET create path, not a PostgreSQL defect + ## Restore path The deployment design should support restore from the start.