Portainer to Docker Compose Migration Guide

Migrating from Portainer to Docker Compose

A Field-Tested Guide with Real Mistakes and How to Fix Them

Context: This tutorial was built from a real migration of a home server (Dell Optiplex 3070 Micros) running multiple services in Portainer, moved to pure Docker Compose under /opt/stacks/. It includes every mistake made along the way and how to recover.

Target audience: Someone who knows basic Linux and Docker but may forget steps, mix up machines, or make assumptions that get them in trouble.

The Big Picture

What we are doing

Moving services managed by Portainer (which has its own hidden config layer) into clean, self-contained Docker Compose stacks stored under /opt/stacks/. Each service gets its own folder with a compose.yml and .env file.

Why this matters for backup

Everything lives under /opt/stacks/ — one folder to back up, one folder to restore. No hunting for Portainer's hidden volumes or config files.

The 3-2-1 backup goal

• Copy 1: Live data on PC03 (primary server)
• Copy 2: Warm backup on PC02 (sleeps, wakes via WoL 3x/day and on failure)
• Copy 3: Synology NAS (cold archive, always on)

Before You Start: Know Your Environment

Check which machine you are on

hostname
# Should print something like PC03dell3050

Common mistake: Running commands on PC02 when you think you are on PC03, or vice versa. Always check hostname before doing anything destructive.

Know your folder structure

/opt/stacks/
├── wordpress/
│   ├── compose.yml
│   ├── .env
│   ├── wordpress_data/    ← webroot files
│   └── db_data/           ← database files
├── mediawiki/
├── zulip/
├── openwebui/
└── ...

All data lives INSIDE the stack folder as bind mounts. No named Docker volumes. This makes backup and restore straightforward.

Know your user situation

Most of these operations require root. If you are logged in as a regular user:

sudo -i
# or
su -
# enter root password when prompted

If you cannot write to /opt/stacks/, you are not root. Do not proceed until you are.

Step 0: Preparation Checklist

Before migrating any service, do these once:

Create the shared Docker network

All services and NPM (your reverse proxy) share this network:

docker network create npm_proxy

Safe to run even if it already exists — it will just say it already exists.

Create the base stacks directory

mkdir -p /opt/stacks

Step 1: Read the Portainer Stack

In Portainer, go to Stacks → [stack name] → Editor and copy the entire compose content.

What to look for:

Field	What it tells you
image:	The Docker image and version
ports:	Host port → container port mapping
volumes:	Where data lives — see below
environment:	Credentials and config — do not lose these
container_name:	The exact name Docker uses — you will need this

Understanding volumes — the most important thing

Portainer stacks can use two types of volumes:

Type 1 — Named volumes (Portainer manages them, hidden location):

volumes:
  - mydata:/var/lib/data
volumes:
  mydata:

The actual files are at /var/lib/docker/volumes/STACKNAME_mydata/_data/

Type 2 — Bind mounts (explicit path on the host):

volumes:
  - /home/user/myapp/data:/var/lib/data

The files are exactly where the path says.

How to tell which type you have: If the volume line has a full path starting with /, it is a bind mount. If it is just a name with no /, it is a named volume.

Step 2: Find Where the Data Actually Is

For named volumes — Portainer renames them

Critical: Portainer adds the stack name as a prefix. If your stack is named wp240927 and your volume is named db, the actual Docker volume name is wp240927_db.

Find the real names:

docker volume ls | grep -i KEYWORD
# Example:
docker volume ls | grep -i wordpress

Find the actual path on disk:

docker volume inspect VOLUMENAME
# Look for "Mountpoint" in the output
# Example output:
# "Mountpoint": "/var/lib/docker/volumes/wp240927_db/_data"

For bind mounts

Just check if the path exists:

ls /home/user/wp250223-8070/

Step 3: Create the New Stack Files

Directory structure

mkdir -p /opt/stacks/SERVICENAME/data_folder_name

Create compose.yml

Use cat > to write the file directly in the terminal. This avoids file transfer issues:

cat > /opt/stacks/SERVICENAME/compose.yml << 'EOF'
services:
  app:
    image: someimage:latest
    restart: unless-stopped
    ports:
      - "PORT:PORT"
    environment:
      SOME_VAR: ${SOME_VAR}
    volumes:
      - ./data:/app/data
    networks:
      - proxy

networks:
  proxy:
    external: true
    name: npm_proxy
EOF

Key changes from the Portainer version:

• Remove version: line (obsolete in modern Compose)
• Change restart: always to restart: unless-stopped
• Move credentials out of environment: into .env using ${VAR_NAME} syntax
• Change named volume paths to ./foldername (relative bind mounts)
• Add the npm_proxy network so NPM can reach the service

Create .env file

cat > /opt/stacks/SERVICENAME/.env << 'EOF'
DB_NAME=mydb
DB_USER=myuser
DB_PASSWORD=change_this_password
EOF

Critical: The values in .env must match what is already inside your database files. If you copy the DB files from the old stack, use the same credentials the old stack had. Changing them will break the connection.

How to find the original credentials: Before stopping the old stack, go to Portainer → Stack → Editor and read the environment variables.

Step 4: Copy the Data

Stop the old Portainer stack first

Go to Portainer → Stacks → [stack name] → Stop.

Do NOT delete it yet. It is your rollback. Only delete after you confirm the new stack works.

For bind mount data (path already known)

rsync -av --progress \
  /old/path/data/ \
  /opt/stacks/SERVICENAME/data/

The trailing slash on the source matters. With a trailing slash, rsync copies the contents. Without it, rsync copies the folder itself (creating an extra nesting level).

For named volume data (path from docker volume inspect)

rsync -av --progress \
  /var/lib/docker/volumes/STACKNAME_VOLUMENAME/_data/ \
  /opt/stacks/SERVICENAME/data/

For data on another machine (SSH)

If rsync is available on both machines:

# Run on the destination machine, pulling from source
rsync -av --progress -e ssh \
  root@SOURCE_IP:/var/lib/docker/volumes/STACKNAME_VOLUMENAME/_data/ \
  /opt/stacks/SERVICENAME/data/

If rsync is NOT installed on the source machine:

# On source machine: create a tar archive
tar -czvf /home/user/service-backup.tar.gz \
  /var/lib/docker/volumes/STACKNAME_vol1/_data \
  /var/lib/docker/volumes/STACKNAME_vol2/_data

# Transfer via SCP from destination
scp root@SOURCE_IP:/home/user/service-backup.tar.gz /opt/stacks/SERVICENAME/

Then extract directly into target folders:

cd /opt/stacks/SERVICENAME

tar -xzvf service-backup.tar.gz \
  --strip-components=6 \
  -C data/subfolder \
  var/lib/docker/volumes/STACKNAME_vol1/_data

The --strip-components number = how many path levels to strip. Count the slashes in var/lib/docker/volumes/STACKNAME_vol1/_data — that is 6 components (var, lib, docker, volumes, STACKNAME_vol1, _data).

If you cannot write to /opt/stacks/ via SFTP: Upload to /home/user/ instead, then move:

mv /home/user/service-backup.tar.gz /opt/stacks/SERVICENAME/

Fix file ownership after copying

Different services run as different users inside their containers:

Service	uid:gid	Why
WordPress (webroot)	33:33	www-data
MariaDB / MySQL	999:999	mysql
PostgreSQL	70:70	postgres
Most Node apps	1000:1000	node
RabbitMQ	999:999	rabbitmq
Redis	999:999	redis

chown -R 33:33 /opt/stacks/wordpress/wordpress_data/
chown -R 999:999 /opt/stacks/wordpress/db_data/

Step 5: Start the New Stack

cd /opt/stacks/SERVICENAME
docker compose up -d

Watch the logs:

docker compose logs -f

Press Ctrl+C to stop watching logs. This does NOT stop the containers.

Test locally before checking the public URL:

curl -I http://localhost:PORT

Expected responses:

• 200 OK — service is up and working
• 301 Moved Permanently — service is up, redirecting (usually HTTP→HTTPS, this is fine)
• 500 Internal Server Error — service is up but something is wrong (usually credentials)
• Connection refused — container is not running or wrong port

Common Mistakes and How to Fix Them

Mistake 1: Old container still running, new one cannot start

Symptom:

Error response from daemon: Conflict. The container name "/myapp" is already in use

Cause: Stopping a Portainer stack does not always stop the containers. Portainer stopped managing them but left them running.

Fix:

docker rm -f CONTAINERNAME
# Then retry:
docker compose up -d

Mistake 2: Wrong credentials in .env — database refuses connection

Symptom: 500 Internal Server Error or "Error establishing database connection"

Cause: The .env credentials do not match what is stored inside the database files. The database files remember what credentials they were initialized with.

Diagnosis — check what the container actually sees:

docker compose exec SERVICENAME env | grep DB_
# or for WordPress specifically:
docker compose exec wordpress env | grep WORDPRESS_DB

Diagnosis — test the credentials directly:

docker compose exec db mariadb -u USERNAME -p'PASSWORD' DBNAME -e "SELECT 1;"
# If it returns "1", those credentials work.

Fix: Edit .env to match the credentials that actually work, then do a FULL restart:

docker compose down
docker compose up -d

Critical: docker compose restart does NOT re-read .env. You must use down + up.

Mistake 3: Portainer volume names are different from what the compose file says

Symptom: docker volume inspect VOLUMENAME returns no such volume

Cause: Portainer prefixes volumes with the stack name. If your stack is wp240927 and your volume is db, the real name is wp240927_db.

Fix:

docker volume ls | grep -i KEYWORD
# Use the full name shown in the output

Mistake 4: Data lands in the wrong place after extraction

Symptom: ls data/postgresql/ shows _data instead of actual database files

Cause: --strip-components count was wrong, or the tar had an unexpected nesting level.

Fix — check what is actually there:

find /opt/stacks/SERVICENAME -name "PG_VERSION" 2>/dev/null
find /opt/stacks/SERVICENAME -name "*.conf" 2>/dev/null | head -5

Then move the contents up:

mv data/postgresql/_data/* data/postgresql/
rmdir data/postgresql/_data

Mistake 5: .env changes not taking effect

Symptom: You edited .env and restarted, but the container still uses old values.

Cause: docker compose restart only restarts the process, it does not rebuild the environment.

Fix:

docker compose down
docker compose up -d

Verify the new values are actually injected:

docker compose exec SERVICENAME env | grep VARNAME

Mistake 6: NPM proxy still pointing to old IP/port after migration

Symptom: Service is running locally (curl -I http://localhost:PORT returns 200) but the public domain gives 502 Bad Gateway.

Cause: NPM's proxy host was pointing to the old machine's IP or old port.

Fix: In NPM → Proxy Hosts → Edit the domain → update Forward Hostname to the new IP or localhost.

Mistake 7: Zulip/service doing HTTP→HTTPS redirect loop through NPM

Symptom: "This page isn't redirecting properly" error in browser.

Cause: Service internally redirects HTTP to HTTPS, but NPM is sending HTTP. The service sends back a 301 to HTTPS, which loops.

Fix: In NPM, change the scheme to https and expose the service's HTTPS port instead of the HTTP port. Also disable SSL verification in NPM since services often use self-signed certs.

Verifying Everything Works

After bringing up each service:

# 1. Check containers are running
docker compose ps

# 2. Check logs for errors
docker compose logs --tail=20

# 3. Test local connectivity
curl -I http://localhost:PORT

# 4. Check the public domain loads
# (Do this in your browser)

# 5. Verify data survived (service-specific)
# For WordPress: log into wp-admin and check posts
# For Wiki: browse to a page you know exists
# For Zulip: log in and check message history

After Confirming a Service Works

Wait 24–48 hours of stable operation, then:

1. In Portainer: Delete the old stack
2. Clean up old data directories if they were bind mounts:

rm -rf /home/user/old-stack-data/

1. Clean up orphaned named volumes:

docker volume ls | grep OLDSTACKNAME
docker volume rm OLDSTACKNAME_volumename

Backup Strategy (What This All Feeds Into)

Once all services are under /opt/stacks/, the backup is simple:

PC03 → Synology (3x daily via Restic):

restic -r sftp:synology:/backups/pc03 backup /opt/stacks/

PC02 warm backup:

• PC02 sleeps most of the time
• Synology watchdog pings PC03 every 60s
• If PC03 fails 3 checks → Synology sends WoL magic packet → PC02 wakes
• PC06 relay (iptables DNAT) switches traffic to PC02
• When PC03 comes back → Restic syncs PC02→PC03 → manual confirm → traffic switches back

PC00 desktop → Synology (daily):

restic -r sftp:synology:/backups/pc00 backup /home/USERNAME/

Quick Reference: Ports Used

Service	Port	Domain
WordPress	8070	blog.gi7b.org
MediaWiki	8191	wiki.gi7b.org
Open WebUI	3000	—
SearXNG	8290	—
Zulip	8010 (HTTP), 8011 (HTTPS)	chat.gi7b.org
Forgejo	3001	—
Gemini sites	27081	—
Jitsi	TBD	meet.gi7b.org
Grafana	TBD	—
Frappe	TBD	—

Giving This Context to a New AI Chat

Paste this block at the start of a new conversation:

I am continuing a home server migration project. Here is the context:

Setup:
- PC03 (192.168.196.76) = primary server, Dell Optiplex 3070, Ubuntu, Docker Compose
- PC02 (192.168.196.189) = warm backup, same hardware, sleeps until needed
- PC06 = Contabo Japan VPS, pure iptables DNAT relay (no NGINX), forwards traffic to PC03
- Synology NAS = always on, cold backup archive, runs watchdog container
- ZeroTier connects all machines

All stacks live under /opt/stacks/SERVICENAME/ on PC03
Each stack has: compose.yml, .env, and data folders as bind mounts
Shared Docker network: npm_proxy (external)
Reverse proxy: NGINX Proxy Manager (NPM) running as its own stack on PC03

Completed migrations (Portainer → Docker Compose):
- WordPress (blog.gi7b.org) → port 8070 ✓
- MediaWiki (wiki.gi7b.org) → port 8191 ✓ (check status)
- Open WebUI → port 3000 ✓
- SearXNG → port 8290 ✓
- Zulip (chat.gi7b.org) → port 8010/8011 ✓ (troubleshooting NPM HTTPS)
- Forgejo → port 3001 ✓ (fresh install, no data migration)
- Gemini-sites → port 27081 ✓

Still to do:
- Jitsi (meet.gi7b.org)
- Grafana + Prometheus
- Frappe / ERPNext
- PC06 iptables watchdog script
- Synology WoL watchdog container
- Restic backup setup (PC03→Synology, PC02 agent, PC00→Synology)
- Clean up abandoned Mastodon volumes on PC03