Skip to content

Troubleshooting

When something isn't working, start with the built-in diagnostics:

bash
lerd doctor   # full check: podman, systemd, DNS, ports, images, config, sites
lerd status   # quick health snapshot of all running services

lerd doctor reports OK/FAIL/WARN for each check with a hint for every failure.

It finishes with a [Sites] section, one line per linked site, so the broad command is actually broad: an environment that passes everything else while three sites are down is not a healthy machine. Each site runs the cheap half of lerd site:doctor, the file-and-config checks, and the line names the command to run for the detail:

bash
lerd site:doctor acme.test

The expensive checks, composer audit, npm audit, and the response-time lookup, only run there, so the sweep stays quick however many sites you have. lerd site:doctor also validates the project's .lerd.yaml, which is what lerd check used to do on its own; check still works as a deprecated alias for it.

Repairing findings automatically

lerd doctor --fix runs the same diagnostic and then offers to repair the findings it safely can. It confirms each fix before applying it, so you can pick and choose:

bash
lerd doctor --fix             # confirm each repair
lerd doctor --fix --yes       # apply without prompting (heavy fixes still confirm)
lerd doctor --fix --dry-run   # list what would be repaired, change nothing

Run without a terminal, over ssh with no tty or from a script, every prompt takes its own default rather than waiting for an answer that cannot come, and says which way it went. The defaults are the cautious ones, so a confirmation nobody is there to give is declined: lerd uninstall piped from a script stops rather than proceeding, and --yes is how you say you meant it. Reading a prompt from a pipe that stays open used to block until the command was killed.

The fixes fall into three groups. lerd applies the safe ones itself, creating a missing data or config directory, enabling linger so services survive logout, installing the network-online drop-in, rebuilding a missing PHP image, and, after you confirm the heavier ones, reinstalling the services or reclaiming podman disk. Anything that needs sudo lerd never runs for you; it prints the exact command to copy. That covers installing podman, crun, fuse-overlayfs, the rootless network helpers or adding a subuid range, and also lerd dns:repair and lerd wsl:setup, which rewrite the resolver and podman configuration through sudo and so are yours to run even though lerd knows the command. Findings that are external state, a foreign process already holding port 80, a config file with a syntax error, are left untouched with their hint. The same safe, non-heavy repairs are available to AI assistants through the MCP diag tool's doctor_fix action, which therefore never elevates on your behalf.

Reclaimable disk is listed separately as optional, because nothing is wrong when there is disk to reclaim. It runs the same interactive reclaim as lerd cleanup, so it takes the deep scope and can remove an unreferenced catalog image whoever pulled it, and the size doctor quotes is that same deep scope. If you run other podman workloads on the machine, run lerd cleanup --safe yourself instead. Optional fixes never count towards what a re-check reports as still outstanding.

Filing a bug report

If you need help on the issue tracker, run:

bash
lerd bug-report

This writes a single plain-text file (default: ./lerd-bug-report-<timestamp>.txt) containing the full lerd doctor output, your config.yaml, sites.yaml and every linked site's .lerd.yaml, the state of every lerd-* systemd unit, recent journal and container logs for lerd's own infra units, listening sockets on the lerd ports, and a curated set of environment variables.

What gets filtered before it lands on disk:

  • Site .env files are excluded outright.
  • Home paths render as $HOME and the username as $USER.
  • Site names, domains and parked-directory paths are replaced with site-1/site1.<tld>/$PARK_1 placeholders. Pass --show-real-names to keep the raw values for local debugging.
  • Logs are kept only for lerd's own infra (lerd-nginx, lerd-ui, lerd-dns, lerd-watcher, lerd-tray, etc.). Preset services (mysql, redis, meilisearch, gotenberg, …), FPM containers and per-site workers still appear in the unit-state and container tables but their logs are dropped, they were producing repetitive request-shaped noise that didn't help triage.
  • Custom services and per-site custom / FrankenPHP containers are omitted entirely so the report doesn't expose user app identifiers.
  • Nginx structured error lines have their request: / upstream: / referrer: URI fields redacted, and HTTP access lines are dropped.

Skim the file before posting (it's plain text, open it in any editor) and attach it to your GitHub issue.

Override the destination with --output, change how many log lines per service to include with --log-lines, or keep raw site names with --show-real-names:

bash
lerd bug-report --output /tmp/report.txt --log-lines 500
lerd bug-report --show-real-names

Profiling lerd-ui

If lerd-ui itself is using more CPU or memory than it should, you can capture a Go profile from the running process and attach it to an issue. Profiling is off until you turn it on, and lerd-ui only serves it to the local machine.

Create the marker, capture, then remove it:

bash
touch ~/.local/share/lerd/run/pprof.enabled
curl -o /tmp/lerd-cpu.prof 'http://127.0.0.1:7073/debug/pprof/profile?seconds=30'
rm ~/.local/share/lerd/run/pprof.enabled

The marker is read on every request, so nothing needs restarting. That matters when you are chasing a daemon that is busy right now, because restarting it would throw away the state worth capturing. Drive the site or wait for the symptom while the thirty seconds are running, otherwise you profile an idle process.

Other useful captures, once the marker is in place:

bash
curl -o /tmp/lerd-heap.prof http://127.0.0.1:7073/debug/pprof/heap
curl 'http://127.0.0.1:7073/debug/pprof/goroutine?debug=1' | head -50

Read a profile by passing the binary alongside it, or just attach the file to the issue:

bash
go tool pprof -top ~/.local/bin/lerd /tmp/lerd-cpu.prof

Release binaries are stripped, so the plain-text views (?debug=1) print raw addresses rather than function names. Pass the binary as above and go tool pprof resolves them anyway, so a profile captured from an ordinary install is still readable.

Remove the marker when you are done. While it exists, anyone who can reach the machine's loopback interface can read goroutine stacks, the process command line, and heap contents, and can start a profile that occupies a core for as long as they ask for.


.test domains not resolving

First, confirm DNS is actually meant to be managed by lerd. If lerd dns:check reports DNS managed externally, you opted out of dnsmasq during install and your sites should be on *.localhost rather than *.test. See DNS for switching modes.

Otherwise, the fastest way to find the broken rung is lerd doctor. The DNS section walks the chain top to bottom and surfaces exactly where it breaks, with a hint per failure:

[DNS]
  DNS TLD (.test)                     OK
    lerd-dns container                running
    dnsmasq config                    address=/.test/127.0.0.1, port=5300
    port 5300 listening               127.0.0.1:5300
    dig @127.0.0.1 -p 5300            127.0.0.1
    resolver hookup                   NetworkManager dispatcher: /etc/NetworkManager/dispatcher.d/99-lerd-dns
    interface routes .test to 5300    enp14s0
    system DNS lookup                 127.0.0.1

The chain in order:

RungWhat it checksIf it fails
lerd-dns containerThe dnsmasq container is running.lerd start (or podman logs lerd-dns to see why it crashed).
dnsmasq config~/.local/share/lerd/dnsmasq/lerd.conf exists with port=5300 and address=/.<tld>/.lerd start regenerates the config from your registered TLD.
port 5300 listeningTCP/UDP 5300 is reachable on 127.0.0.1.Another process owns the port. Find it with ss -tlnp sport = :5300 on Linux, or lsof -nP -iTCP:5300 -sTCP:LISTEN on macOS.
dig @127.0.0.1 -p 5300A direct query at port 5300 returns 127.0.0.1 for lerd-probe.<tld>, or the host's LAN IP when lan:expose is on.dnsmasq is up but its config drifted. lerd dns:repair.
resolver hookupThe NetworkManager dispatcher script or systemd-resolved drop-in is installed.Rerun lerd install.
interface routes .test to 5300resolvectl status shows 127.0.0.1:5300 and ~<tld> on the active interface.sudo systemctl restart NetworkManager, or set the routing manually with sudo resolvectl domain <iface> ~test ~..
system DNS lookuphost lerd-probe.test (the system resolver) returns 127.0.0.1, or the host's LAN IP under lan:expose.The drop-in is installed but resolved isn't honouring it. Check whether cloud-init or another tool wrote a higher-priority resolver config. Common on EC2 / cloud images. With a VPN connected this rung is reported as a warning rather than a failure, see the VPN section below.

You can also call this programmatically over MCP via the diag tool's dns_diagnose action, useful for AI-driven troubleshooting:

bash
echo '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"diag","arguments":{"action":"dns_diagnose"}}}' | lerd mcp

The response includes a steps array with a status (ok / fail / warn / skip) and hint per rung, plus a first_failure index so an LLM can jump straight to the broken layer.

.test domains stop resolving when offline (no internet)

On systemd-resolved systems, .test used to reach lerd-dns only through a route that depended on your real network being up: per interface (resolvectl domain <iface> ~test) when NetworkManager manages resolved, or a global drop-in otherwise. Either way, systemd-resolved refuses to resolve anything at all, over both resolvectl and the glibc/NSS path a browser uses, once no real link is routable, so a fresh .test lookup failed even though lerd-dns kept answering on 127.0.0.1:5300. A common symptom was a page that still worked while the browser stayed open (cached DNS) but failed the moment you closed and reopened it.

Lerd now keeps an always-up dummy interface, lerd0, that carries the ~test route. Because that link never goes down, systemd-resolved keeps forwarding .test to lerd-dns with no network connection at all. It is created by a small system service, lerd-dns-link.service, which starts on every boot, so the fix survives reboots and applies automatically on your next lerd start, nothing to run by hand. This applies to both systemd-resolved setups: with NetworkManager (Ubuntu, Fedora, CachyOS) and without it (Arch, omarchy).

If you also saw a stall of up to twenty seconds on .test while offline, that was an AAAA (IPv6) lookup. lerd's dnsmasq config answers both address=/.test/127.0.0.1 and address=/.test/::1, but the NetworkManager dispatcher used to regenerate that file from a v4-only template whenever an interface came up, dropping the AAAA record. dnsmasq then forwarded .test AAAA queries to your upstream, which times out once that upstream is unreachable. The dispatcher now leaves the address records alone, so AAAA is answered locally and returns instantly.

What is the lerd0 network interface?

lerd0 is a dummy (virtual) network interface lerd creates on Linux so that .test domains keep resolving when you have no network at all. It carries no traffic and connects to nothing; it exists purely to give systemd-resolved a link that is always up to hang the .test route on. See the offline entry above for why that is necessary.

It is deliberately marked unmanaged in NetworkManager (/etc/NetworkManager/conf.d/lerd-dns-link.conf), so it does not appear as a connection in your desktop's network menu and cannot be switched off by accident. Its only address is 192.0.2.1/32, from the range RFC 5737 reserves for documentation and which never appears on a real network, so it cannot conflict with anything you connect to.

Alongside it, lerd turns off systemd-resolved's fallback DNS servers (/etc/systemd/resolved.conf.d/lerd-fallback.conf). This is the price of lerd0: it stops resolved refusing every lookup when you are offline, which is the point for .test, but the same switch makes resolved willing to try names it cannot reach, so it works through its fallback servers (Quad9, Cloudflare, Google) one at a time and every offline lookup of an ordinary domain hangs for 20 seconds or more instead of failing at once. Debian, Ubuntu and Fedora already ship these fallbacks off, so nothing changes there; on Arch and its derivatives this aligns them with the others. The trade is that if your own DNS server breaks, lookups now fail instead of quietly going to a public resolver. lerd uninstall puts the fallbacks back.

If it ever goes missing, lerd doctor reports it under the offline .test route check and the next lerd start recreates it. To recreate it by hand:

bash
sudo systemctl restart lerd-dns-link.service

lerd uninstall removes the interface, its service, and the NetworkManager rule. To remove it without uninstalling lerd, use lerd dns:disable, which turns off lerd's DNS management entirely.

DNS shows "Degraded" while connected to a VPN

VPN clients such as Cisco AnyConnect, ProtonVPN, Mullvad, and WireGuard take over the system resolver when they connect, rewriting systemd-resolved so .test no longer routes to lerd-dns through the normal path. lerd-dns itself keeps running and answering, so the dashboard shows a yellow Degraded pill rather than a red Failed one, and lerd doctor reports the system DNS lookup rung as a warning instead of a failure. Sites still resolve, because lerd-dns answers directly on 127.0.0.1:5300.

The watcher subscribes to kernel rtnetlink link and address events on Linux, so it reacts to a VPN connect or disconnect within a second of the interface coming up or going down (a poll every 30 seconds covers the rare case of a missed kernel event). When the host resolver environment changes, it re-points the lerd network's aardvark-dns at the current host resolvers and reloads the network so containers pick them up with a fresh cache. This is what previously required a manual lerd restart after connecting the VPN before PHP could reach VPN-internal API endpoints. The re-sync briefly (about a second) interrupts DNS for lerd containers while aardvark-dns restarts.

If you want the system resolver path itself restored while the VPN is up, so the pill goes back to green, move resolve after dns in the hosts: line of /etc/nsswitch.conf:

text
hosts: mymachines mdns_minimal [NOTFOUND=return] files myhostname dns resolve

This makes glibc consult the plain dns module before systemd-resolved's nss-resolve, which the VPN client no longer shadows.

lerd install fails at "Building dnsmasq" with "unable to select packages"

The dnsmasq image is the one thing lerd builds rather than pulls, and the apk add inside that build resolves names from the build's own network namespace, not through the host resolver that just pulled the base image. When that namespace has no working resolver, the build fails with WARNING: fetching https://dl-cdn.alpinelinux.org/...: DNS: transient error followed by ERROR: unable to select packages, even though every pull before it succeeded.

lerd retries the build with your host's real upstream nameservers pinned, which covers the common case of a stub resolver (127.0.0.53) that podman's default handling does not translate correctly on your setup. When that still resolves nothing, it builds once more on the host network, which is the namespace that pulled the base image a moment earlier and so is known to work. A rootless build cannot join the lerd network, so those two are the levers available.

If every attempt fails, lerd says so at the end of the install rather than reporting a clean finish: without the image, lerd-dns cannot start and no .test name resolves. Check what the container network can reach with lerd doctor, in particular the internet DNS from containers line, fix it, then run lerd install again.

php artisan fails with "could not translate host name lerd-postgres" while the site works in the browser

The site's .env names the service container and its internal port, lerd-postgres:5432, which is correct: that is how nginx and PHP-FPM reach the database, and it is why the browser is fine. The name exists only on the lerd network, so a PHP that runs on the host cannot look it up, and every console command fails on the name while nothing else does. Pointing the .env at 127.0.0.1 swaps the symptom rather than fixing it, the CLI starts working and the site stops.

The command is meant to run inside the container, and normally does: lerd install puts a php shim in ~/.local/share/lerd/bin/ ahead of your PATH, and php artisan migrate routes through the project's PHP-FPM container. What breaks it is another PHP arriving in front of the shim. lerd writes its PATH line to your shell rc once, at install, so anything appended below it later, Herd, a Homebrew shellenv, mise, asdf or phpenv, takes php back, and so does an install that wrote to a different rc than your terminal reads.

Check which one you have:

bash
which php        # expect ~/.local/share/lerd/bin/php
lerd doctor      # the Configuration section reports what leads

Doctor's php on PATH line names the binary in front when it is not lerd's. Move lerd's export PATH line to the end of your shell rc and open a new shell, and if the entry is missing entirely, lerd path:enable writes it back. To keep your own PHP in front deliberately, run lerd path:disable and type lerd artisan migrate instead, which always runs in the container whatever your PATH says.

composer or npm fails with "could not resolve host" inside a container

Composer, npm and the framework store all run inside the container, not on the host, so they use the resolver the lerd network hands aardvark-dns rather than yours. Those two can differ: .test domains and container names are answered by aardvark-dns from its own records and keep working regardless, so a broken forwarder shows up only as downloads that fail with could not resolve host or curl error 28 while downloading, with nothing else complaining.

lerd doctor checks this directly. In the DNS section, the internet DNS from containers line asks the running nginx container to resolve the framework store's hostname, which is an ordinary internet name and the same lookup composer makes:

[DNS]
  ...
  internet DNS from containers (raw.githubusercontent.com)   OK

If it fails, the forwarders on the lerd network are stale or unreachable from inside the container network namespace. lerd stop && lerd start re-points them at your current host resolvers, which fixes it in most cases. To see what they are set to:

bash
podman network inspect lerd --format '{{.NetworkDNSServers}}'

An address that is valid on the host but not routable from a rootless network namespace is the usual cause. This is worth checking on WSL2 in particular, where the Windows-side resolver address the WSL VM is given is not always reachable from inside the namespace.

"Secure Connection Failed" after the host wakes from suspend or hibernate

After a long suspend or hibernate, rootless podman networking can come back in a bad state: the lerd-nginx container loses its host port forward (or stops), so nothing listens on 443 and the browser shows a generic "Secure Connection Failed" for your .test sites, or the lerd-dns container stops and names no longer resolve.

On Linux the watcher now restarts nginx automatically. It notices the host has resumed from a real wall-clock gap in its tick loop (the timer is frozen while the machine is suspended), and on that one tick it checks whether lerd-nginx is accepting on its HTTPS port and restarts it if the listener died. Keying off the resume event rather than a continuous poll means it acts exactly once and can never fight a lerd start you ran yourself, since a start does not suspend the machine. DNS resolution is repaired by the same watcher's existing path, so .test names come back on their own too.

A stopped lerd-dns is healed too. Whenever the watcher finds .test broken it now asks lerd's dnsmasq directly on port 5300 whether it is alive, and restarts the container when it is not, instead of only rewriting the host resolver config, which can never bring back a container that is gone. Waking is not the only way to lose it: the NetworkManager dispatcher restarts lerd-dns on every interface change, and a wake that brings wifi, ethernet and a VPN back at once used to fire enough restarts in a few seconds to exhaust systemd's start rate limit, which parks the unit in failed permanently. That limit is now lifted for lerd-dns, and the watcher clears any leftover failed state before it restarts.

One case it still leaves for lerd start rather than acting from a background timer: a host whose IPv6 support changed across the wake, since the lerd network must be recreated and that rebuilds every container. The same applies in the rare case the watcher itself was not running at the moment of resume.

Nginx not serving a site

Check that nginx and the PHP-FPM container are running, then inspect the generated vhost:

bash
lerd status                         # check nginx and FPM are running
podman logs lerd-nginx              # nginx error log
cat ~/.local/share/lerd/nginx/conf.d/my-app.test.conf   # check generated vhost
My custom nginx directive disappeared after an update

Don't edit ~/.local/share/lerd/nginx/conf.d/*.conf directly. Lerd regenerates those files on lerd link, lerd secure, lerd site rebuild, and every lerd install (which lerd update re-execs). Drop your snippet in ~/.local/share/lerd/nginx/custom.d/{domain}.conf instead, the generated vhost ends with an include for that file, and lerd never writes into custom.d/. See Nginx Overrides for examples.

PHP-FPM container not running

Check the systemd unit status and logs:

bash
systemctl --user status lerd-php84-fpm
systemctl --user start lerd-php84-fpm
podman logs lerd-php84-fpm

If the image is missing (e.g. after podman rmi):

bash
lerd php:rebuild
podman exec fails with "chdir: No such file or directory"

This happens when your project is outside your home directory (e.g. /var/www/, /opt/projects/). The PHP-FPM and nginx containers only mount $HOME by default.

Lerd handles this automatically: when you lerd link, lerd park, or run any exec command (lerd php, composer, laravel new) from an outside path, lerd adds the volume mount and restarts the affected containers.

If you see this error on an older lerd version, update to the latest and re-link the site:

bash
lerd update
lerd unlink && lerd link

To verify the mounts are in place:

bash
grep Volume ~/.config/containers/systemd/lerd-nginx.container
grep Volume ~/.config/containers/systemd/lerd-php*-fpm.container

You should see your project path listed alongside the %h:%h mount. The quadlet is only half the answer though, a container keeps the mounts it booted with, so check the running one too:

bash
podman inspect lerd-php84-fpm --format '{{range .Mounts}}{{.Source}}
{{end}}'

If the path is in the quadlet but not in that output, lerd restart picks it up.

nginx fails to start with "statfs /path: no such file or directory"

Podman refuses to start a container whose bind-mount source is gone, so a directory outside $HOME that lerd mounted while it existed, and that has since disappeared, stops the container dead. A Git branch checkout that removes a project subdirectory is the usual cause, and because nginx serves every site, one missing directory takes the whole stack down.

lerd start repairs this for you: it drops the stale mounts before starting anything and tells you which path and site was responsible.

WARN: /var/www/erp/Modules/Accounts no longer exists (site erp), removed from lerd-nginx

The mount comes back on its own once the directory is there again and you run any command from it, or after lerd restart.

Permission denied on port 80/443

Rootless Podman cannot bind to ports below 1024 by default. Allow it:

bash
sudo sysctl -w net.ipv4.ip_unprivileged_port_start=80
# Make permanent:
echo 'net.ipv4.ip_unprivileged_port_start=80' | sudo tee /etc/sysctl.d/99-lerd.conf

lerd install sets this automatically, but it may need to be re-applied after a kernel update.

Watcher service not running

The watcher monitors parked directories, site config files, git worktrees, and DNS health. If sites aren't being auto-registered or queue workers aren't restarting on .env changes:

bash
lerd status                            # shows watcher running/stopped
systemctl --user start lerd-watcher   # start it from the terminal
# or use the Start button in the UI under System > Watcher

The watcher reports itself ready as soon as its watch loops are live and does its boot reconciliation (registering parked projects, provisioning worktrees) after that, in the background. A worktree install that takes minutes therefore delays only that worktree; it can't hold the unit below its start timeout and put systemd in a restart loop.

To see what the watcher is doing:

bash
journalctl --user -u lerd-watcher -f
# or open the live log stream in the UI under System > Watcher

For verbose output (DEBUG level), set LERD_DEBUG=1 in the service environment:

bash
systemctl --user edit lerd-watcher
# Add:
# [Service]
# Environment=LERD_DEBUG=1
systemctl --user restart lerd-watcher
HTTPS certificate warning in browser

The mkcert CA must be installed in your browser's trust store. Ensure certutil / nss-tools is installed, then re-run lerd install:

  • Arch: sudo pacman -S nss
  • Debian/Ubuntu: sudo apt install libnss3-tools
  • Fedora: sudo dnf install nss-tools

After installing the package, run lerd install again to register the CA.

On macOS, this can also show up after a reinstall even though everything worked the first time: a macOS update can drop a certificate's trust settings while leaving the certificate itself in the keychain. lerd install asks whether the keychain still trusts the CA now, rather than only whether the certificate is there, and repairs it (with the usual admin authorization prompt) when it finds that drifted state. Just re-run lerd install.

PHP image build is slow on first run

lerd normally pulls a pre-built base image from ghcr.io and finishes in ~30 seconds. If you see it fall back to a local build instead, the most common cause is being logged into ghcr.io with expired or unrelated credentials; the registry rejects the authenticated request even though the image is public.

lerd handles this automatically since v1.3.4 by always pulling anonymously. If you are on an older version, running podman logout ghcr.io before the build will fix it.

Nginx fails to start (missing certificates)

lerd start automatically detects SSL vhosts that reference missing certificate files and repairs them before starting nginx:

  • Registered sites: the site is switched back to HTTP and the vhost is regenerated. The registry is updated (Secured = false).
  • Orphan SSL vhosts: configs left behind by unlinked sites with missing certs are removed.

Repaired items are printed as warnings during startup:

  WARN: missing TLS certificate for myapp.test, switched to HTTP

To re-enable HTTPS after the automatic repair, run lerd secure <name>.

If nginx still fails to start, check the logs:

bash
journalctl --user -u lerd-nginx -n 30 --no-pager
Port conflicts on lerd start

lerd start checks for port conflicts before starting containers. If another process is already using a required port, you'll see a warning:

Port conflicts detected:
  WARN: port 80 (nginx HTTP) already in use, may fail to start (check: ss -tlnp sport = :80)

Common culprits are Apache, another nginx instance, or a previously running lerd that wasn't stopped cleanly. Find and stop the conflicting process:

bash
# Linux
ss -tlnp sport = :80

# macOS
lsof -nP -iTCP:80 -sTCP:LISTEN

The exact command lerd suggests in lerd doctor and lerd start output is already platform-correct, so you can copy it from there.

lerd doctor also checks for port conflicts as part of its full diagnostic, and adds a dedicated [Stopped service ports] section that flags installed services whose host port is already bound by another process. The same warning is shown next to the inactive status pill in the web UI, so you can spot the conflict without running anything: most often this is a system-installed service (Postgres, MySQL, Redis) listening on the default port. Stop the conflicting process and the warning clears on the next snapshot refresh.

Uninstall leaves the data directory behind

Services write their files as a subuid inside the rootless user namespace, so MySQL, Postgres, MongoDB, Redis and RabbitMQ all leave trees under ~/.local/share/lerd that your own user cannot delete. Both lerd uninstall and the installer's --uninstall remove them through podman unshare, which enters that namespace, so this normally happens without you noticing.

If podman is already gone by then, the uninstall finishes and tells you what survived. Remove it yourself with:

bash
podman unshare rm -rf ~/.local/share/lerd

If podman is no longer installed either, sudo rm -rf ~/.local/share/lerd is the last resort.

An uninstall also takes ~/.cache/lerd, the lerd-tray binary alongside lerd, both PATH entries lerd ever wrote into your shell rc, and the images it built itself (lerd-php*-fpm, lerd-custom-*, lerd-dnsmasq) when you accept the purge. Images it only pulled, your databases and your project files are never touched.

Workers missing after reinstall

If you ran lerd uninstall and then reinstalled, worker units and service quadlets are deleted during uninstall. Running lerd start after reinstalling automatically restores them from the workers list saved in each site's .lerd.yaml. If .lerd.yaml does not exist or was not committed, you will need to start workers again manually (lerd queue:start, etc.).

To check what was restored:

bash
lerd status   # shows all active workers and services
Workers failing or crash-looping

Check lerd status, the Workers section lists all active, restarting, or failed workers. In the web UI, failing workers show a pulsing red toggle and a ! on their log tab.

To inspect the error:

bash
journalctl --user -u lerd-queue-my-app -f    # or lerd-horizon-my-app, lerd-schedule-my-app

Common causes:

  • Missing Redis when QUEUE_CONNECTION=redis, start it with lerd service start redis
  • Missing dependencies after a fresh clone, run lerd setup to install them
  • Bad .env values, run lerd env to reset service connection settings

When you unlink a site, crash-looping workers are automatically detected and stopped.

Error: could not fetch latest version: unexpected release URL format

Symptom: lerd update fails with unexpected release URL format: https://github.com/lerd-env/lerd/releases/latest.

Cause: your binary predates lerd 1.26 and still asks GitHub for releases under the old geodro/lerd path. Since the project moved to the lerd-env organisation, GitHub answers that path with a rename redirect to the new /releases/latest URL instead of the release itself, and older binaries only read the first redirect. The update command is the broken part, so no release can repair it remotely.

Fix: reinstall once. This only replaces the binary; your sites, services, and config are untouched:

bash
curl -fsSL https://lerd.sh/install.sh | bash

Everything from 1.26 onwards resolves the organisation move on its own, so this is a one-time step.

On Homebrew, apt, dnf, or if you'd rather not pipe a script anywhere, Updating from a version before 1.26 has the route for each.

Error: could not fetch latest pre-release: GitHub API rate limit exhausted

Symptom: lerd update --beta stops with GitHub API rate limit exhausted for https://api.github.com/repos/lerd-env/lerd/releases, it resets in 46 min.

Cause: pre-releases are not covered by the /releases/latest redirect the stable channel follows, so the beta check asks the GitHub API instead. An anonymous API call is charged to a bucket of 60 requests an hour shared by everything on your IP, and any other tooling on the machine can empty it before lerd gets there. The stable channel is unaffected.

Fix: wait for the reset the message names, or authenticate the call. Lerd sends GITHUB_TOKEN or GH_TOKEN if either is set in the environment, which raises the ceiling to 5,000 requests an hour:

bash
export GITHUB_TOKEN=$(gh auth token)
lerd update --beta

The token needs no scopes, public release metadata is all lerd reads, and it is only ever sent to api.github.com over https, never to a mirror configured through LERD_RELEASES_API_URL.

A token that has expired or been revoked costs you nothing: GitHub answers it with a 401, and lerd drops the token and asks again anonymously, so the check still works on the 60 requests an hour every IP gets.

Error: NetworkUpdate is not supported for backend CNI: invalid argument

Your system is likely configured to use the older CNI backend, which lacks support for the requested network operation. Edit or create the Podman configuration file at /etc/containers/containers.conf and add or modify the network_backend setting to netavark:

toml
[network]
network_backend = "netavark"

To ensure a clean switch and recreate the networks with the new backend, reset the Podman storage. Warning: this will wipe all existing containers, pods, and networks:

bash
podman system reset
Error: unknown flag: --dns (during lerd install)

Symptom: lerd install aborts at the podman network create step with Error: unknown flag: --dns.

Cause: your podman is older than 4.5. The --dns flag on podman network create was added in podman 4.5 (April 2023), and lerd needs it to write upstream DNS servers into netavark's per-network JSON atomically (otherwise the post-create network update --dns-add path crashes on Ubuntu 24.04's netavark <1.11). Distributions that ship podman older than 4.5: Ubuntu 22.04 / Zorin 17 (3.4.4), Debian 12 (4.3.1), Debian 11 (3.0.1).

Fix: upgrade podman to 4.5 or newer. On Ubuntu 22.04 and Zorin 17 the main archive doesn't ship a new enough podman, but the Kubic libcontainers OBS repo does (it's the path podman's own docs recommend). On Debian 12 enable bookworm-backports and run sudo apt install -t bookworm-backports podman. See the requirements page for the full distro/version table.

Error: unable to parse ip fe80::...%18 specified in AddDNSServer: invalid argument

Your host's DNS configuration includes a zoned link-local IPv6 nameserver, typically advertised by your router via SLAAC + RDNSS. The zone identifier (%18 is a kernel interface index) is meaningless inside a container's network namespace, and netavark refuses to accept it.

Lerd 1.18+ filters these addresses automatically before handing them to podman. If you're still on 1.17 or older, upgrade with lerd update and rerun lerd install. The filter is conservative: only zoned link-local (fe80::...%iface) addresses are dropped; globally routable IPv6 nameservers (e.g. 2606:4700:4700::1111) are preserved.

When filtering empties the entire DNS list, lerd falls back to pasta's standard forwarder (169.254.1.1), which bridges into the host's resolver and preserves .test routing.

Containers can resolve .test over IPv4 but not over IPv6

Lerd 1.18+ creates the lerd podman network as dual-stack (v4 + v6) and writes both A and AAAA records for .test domains. If you upgraded from an older version, the existing v4-only lerd network is migrated automatically the next time you run lerd install: attached containers stop, the network is recreated with the fd00:1e7d::/64 ULA prefix, the previous DNS server list is restored, and the containers restart. Quick check:

bash
podman network inspect lerd --format '{{.Subnets}}'
# expect both an IPv4 subnet and one starting with fd00:1e7d::

If the v6 subnet is missing, run lerd install once to migrate. To verify resolution from inside a container:

bash
podman run --rm --network lerd alpine sh -c 'nslookup laravel.test; nslookup -type=AAAA laravel.test'
Services fail to start with "aardvark-dns failed to bind [fd00:1e7d::1]:53"

Symptom: after lerd install, a subset of service containers (commonly lerd-nginx, lerd-postgres, lerd-meilisearch) fail to start. Journal shows:

Error: netavark: error while applying dns entries: IO error: aardvark-dns failed to start
Error starting server failed to bind udp listener on [fd00:1e7d::1]:53:
IO error: Cannot assign requested address (os error 99)

Cause: the host advertises IPv6 in the kernel but has no routable v6 address on any interface, only ::1 and fe80::, so netavark can't hold the ULA gateway on the rootless bridge, and aardvark-dns bind fails with EADDRNOTAVAIL. Typical in headless QEMU/KVM VMs and networks without v6 DHCP.

Lerd 1.18+ detects this on every lerd install by reading /proc/net/if_inet6 (any non-loopback, non-link-local v6 address counts as usable) and falls back to a v4-only lerd network. An existing dual-stack network on a v6-less host is recreated as v4-only automatically. Force it:

bash
lerd install
# look for: "Recreated lerd network as v4-only (host has no usable IPv6)."

If the host later gains v6 connectivity, the next lerd install will recreate the network as dual-stack again.

If you'd rather skip the dual-stack code path entirely, even on a v6-capable host, opt out:

bash
lerd install --no-ipv6
# or persistently via shell rc:
export LERD_DISABLE_IPV6=1

Either path writes ~/.local/share/lerd/ipv6-probe-failed-lerd, which EnsureNetwork honors on every code path (initial create, migration, recreate). To re-enable dual-stack, delete that marker file and re-run lerd install.

Every service fails with "rootlessport listen tcp [::1]:80: bind: cannot assign requested address"

Symptom: nothing starts. nginx, mysql, redis, mailpit and every other service exit with status 126, and the journal shows a bind failure on an IPv6 loopback address:

Error: rootlessport listen tcp [::1]:3306: bind: cannot assign requested address
lerd-mysql.service: Main process exited, code=exited, status=126/n/a

Cause: the host no longer has ::1. IPv6 was turned off kernel-wide, either at boot with ipv6.disable=1 or at runtime by a VPN client doing leak prevention (the NordVPN Linux client sets net.ipv6.conf.{all,default,lo}.disable_ipv6=1 while connected). Lerd published each service on both 127.0.0.1 and [::1], and podman treats a publish it cannot bind as fatal rather than falling back to the address that would have worked.

Lerd now checks for ::1 before writing a unit and publishes on IPv4 alone when it is gone, so a host in this state comes up normally. Existing units written by an older build are repaired on the next lerd start, which rewrites them and restarts whatever was already running. If IPv6 comes back later, the following start restores the dual-stack publish.

Note that this is a different check from the one the network schema uses. That one wants a routable address; the publish only needs loopback, so a VM with just ::1 and fe80:: still gets its [::1] binds while the network stays v4-only.

Every DNS lookup inside a lerd container stalls ~5 seconds

Symptom: pages that hit the database or any container-to-container hostname feel slow, and time dig <anything> @<container> takes roughly five seconds before returning an answer. The network looks fine in podman network inspect lerd (both IPv4 and IPv6 subnets present), but aardvark-dns's on-disk config has the v6 gateway absent from its listen-ips line.

Cause: podman network rm doesn't clean up $XDG_RUNTIME_DIR/containers/networks/aardvark-dns/<name> between rm and recreate, so a network that was originally v4-only can leave aardvark with a v4-only listen header even after the network is recreated dual-stack. The container's /etc/resolv.conf still lists the v6 gateway as the primary nameserver, queries to it time out (~5s), then glibc falls back to the v4 gateway.

Lerd 1.18+ detects this drift on lerd install (aardvark listen line is v4-only despite the network being dual-stack) and self-heals by recreating the network with the stale aardvark state wiped. If you're on an earlier 1.18 build or the heal didn't fire, force it:

bash
lerd install

Manual verification:

bash
cat "${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/containers/networks/aardvark-dns/lerd" | head -1
# expect both gateways, e.g.: fd00:1e7d::1,10.89.7.1 169.254.1.1
# if only 10.89.7.1 is present, the drift fix didn't run — re-run lerd install
Every container takes 90 seconds to start (Fedora Silverblue and other atomic images)

Symptom: lerd start sits there for a minute and a half per container, and after a reboot nothing is serving until well over a minute in. systemctl --user list-units --state=failed shows podman-user-wait-network-online.service failed with a timeout, and lerd doctor reports the podman network-online wait check as a warning.

Cause: podman's quadlet generator makes every rootless container Wants= and After=podman-user-wait-network-online.service, a unit that polls the system's network-online.target until it gives up after 90 seconds. That target is only reached when some unit pulls it in, and on atomic images (Silverblue, Kinoite, Bazzite, CoreOS) nothing does, so the wait can never succeed. Every container start, and the boot itself, pays the full timeout.

Lerd detects this on lerd start and lerd install and writes a drop-in that turns the wait into a no-op, since lerd publishes on loopback and needs no routable network:

bash
lerd start   # writes ~/.config/systemd/user/podman-user-wait-network-online.service.d/10-lerd-no-network-wait.conf

The override only lands on hosts where network-online.target is genuinely inactive; on an ordinary distro the wait is left alone. To confirm the target is the one at fault:

bash
systemctl is-active network-online.target   # "inactive" here means every quadlet start pays 90s

To go back to podman's stock behaviour, delete the drop-in and run systemctl --user daemon-reload.

System tray missing on Fedora Silverblue and other atomic images

Symptom: no tray icon, and systemctl --user is-system-running reports degraded because lerd-tray.service failed with status 127.

Cause: lerd-tray links libayatana-appindicator3.so.1, which these images don't ship, and an immutable OS can't just install it into /usr on demand.

Lerd checks the helper's libraries at install time and leaves the tray unit stopped and disabled when one is missing, so the failure no longer drags the systemd user session into degraded. Everything else (CLI, dashboard, watcher, containers) is unaffected, the tray is the only thing you lose.

To get the tray back, layer the package and reboot, then re-enable the unit:

bash
rpm-ostree install libayatana-appindicator-gtk3
systemctl reboot
lerd install   # re-enables the tray now that the library resolves
Podman Machine overlay-storage error (macOS)

Symptom: on macOS, lerd start fails and every container start reports a graph-driver / overlay error:

exit status 125: Error: getting graph driver info "<id>":
readlink /var/lib/containers/storage/overlay: invalid argument

Cause: the macOS host was shut down ungracefully (forced power-off, battery death, kernel panic) while the Podman Machine VM was still running. The VM's container storage is left with a stale overlay mount and corrupt container layers, so no container can start until the storage is remounted and the stale containers are rebuilt.

lerd start detects this and self-heals automatically on the first run: it restarts the Podman Machine to remount the storage, force-removes the stale lerd-* containers so they rebuild on fresh storage, and retries the start pass once. Your data is safe throughout: lerd bind-mounts every database and site directory to the host, not into the VM.

If the automatic recovery isn't enough (it prints guidance pointing here), recreate the VM:

bash
lerd machine reset

This stops the VM, removes it, and re-initialises it. Databases and site data are preserved (they live on the host); container images are rebuilt automatically on the next lerd start. See Start, Stop & Autostart → lerd machine reset.

A project on an external drive is created inside the VM (macOS)

Symptom: on macOS, lerd new on a path outside your home directory reports the project as created, then the run warns chdir /Volumes/<drive>/<project>: no such file or directory and the folder is nowhere on the drive.

Cause: on macOS every bind mount is resolved inside the Podman Machine VM, and the VM only sees the host trees it was given when it was created. A drive mounted under /Volumes is often not among them, so the container sees an empty directory at that path, composer writes the whole project into the VM, and it never lands on the disk.

lerd now checks that the container is really looking at your directory before it scaffolds, and stops with an explanation instead of creating a project you cannot find.

Machines lerd creates share /Users, /private, /var/folders and /Volumes with the VM, but a machine created before lerd asked for /Volumes (or one created by hand with podman machine init) keeps Podman's own defaults and never got it. Podman writes the guest mount units once at init, so this cannot be repaired by editing the machine config; recreate the VM instead:

bash
lerd machine reset

lerd start points this out on its own when something is already served from outside your home directory. If the drive is still invisible after a reset, macOS is withholding access to it from the VM process rather than lerd failing to ask; keep the project under your home directory.

An external drive will not eject while lerd is running (macOS)

Symptom: diskutil unmount or the eject button refuses with dissented by PID ... com.apple.Virtualization.VirtualMachine.

Cause: the Podman Machine VM shares /Volumes for as long as it runs, so macOS treats every drive under it as in use. Stopping the containers is not enough, the VM itself has to go down:

bash
lerd stop
podman machine stop

The drive ejects after that, and lerd start brings the VM and your other sites back. Plugging the drive in again is picked up by a running VM on its own, no restart needed.

While the drive is away, lerd starts normally and every other site is unaffected: the missing path is dropped from the container mounts and the site on the drive stops being served. Your files are untouched, but the site is removed from lerd sites, so bring it back with lerd link and, if it was on HTTPS, lerd secure from the project directory.

Released under the MIT License.