Portainer: Docker Container Management That Actually Works

I was managing Docker exclusively through CLI for years. Log inspection meant scrolling through thousands of lines, and checking stats across 5 servers was SSH hell. Portainer changed all that. Here's my setup managing containers across a home lab and a VPS.

Portainer Agent Connection Refused on Remote Host

Problem

Had Portainer running on my main server, wanted to add a second VPS to the same dashboard. Deployed the Portainer Agent on the VPS, but Portainer couldn't connect - "Connection refused". The agent was running (verified with docker ps), but no go.

Error: Failed to add environment: Connection refused

What I Tried

Attempt 1: Used the server's private IP - failed (servers on different networks).
Attempt 2: Used public IP with port 9001 - timed out (firewall blocking).
Attempt 3: Exposed port 9001 in Docker compose - still blocked by ufw.

Actual Fix

The agent needs port 9001/tcp exposed AND the firewall must allow it. Also, for security, I set up an SSH tunnel to the agent instead of exposing it publicly. Portainer supports SSH connections. 

# Option 1: Direct connection (expose port 9001)
# On remote server, run agent with exposed port:
docker run -d \
  -p 9001:9001 \
  --name portainer-agent \
  --restart=always \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /var/lib/docker/volumes:/var/lib/docker/volumes \
  portainer/agent:latest

# Allow through firewall (UFW example)
sudo ufw allow 9001/tcp

# Option 2: SSH tunnel (more secure, no port exposure)
# In Portainer UI: Add Environment
# - Environment type: Agent
# - Environment URL: ssh://user@remote-server-ip
# - Port: 9001 (agent's internal port)
# - SSH key: Upload your private key or password

# No firewall changes needed - uses SSH port 22
Container Templates Not Loading

Problem

Clicked on "Templates" in Portainer, the list never loaded. Just spun forever with a loading spinner. Could deploy containers manually, but the template feature (my main use case) was broken.

What I Tried

Attempt 1: Refreshed the page multiple times - no change.
Attempt 2: Restarted the Portainer container - still failed.
Attempt 3: Checked Portainer logs - showed "fetching templates..." but no errors.

Actual Fix

The default template URL (portainer.io templates) was timing out. Portainer allows custom template definitions. I either needed to: 1) Use a local template file, or 2) Host my own template repository, or 3) Define templates manually in Portainer's App Templates settings. 

# Create custom template file (templates.json)
{
  "version": "2",
  "templates": [
    {
      "title": "Nginx",
      "description": "High performance web server",
      "logo": "https://example.com/nginx.png",
      "image": "nginx:latest",
      "ports": ["80:80", "443:443"],
      "volumes": ["/etc/nginx/conf.d", "/var/www/html"]
    },
    {
      "title": "PostgreSQL",
      "description": "Object-relational database",
      "logo": "https://example.com/postgres.png",
      "image": "postgres:15",
      "env": [
        {
          "name": "POSTGRES_PASSWORD",
          "label": "Postgres Password",
          "default": "changeme"
        }
      ]
    }
  ]
}

# In Portainer: Settings → App Templates
# Use custom URL: file:///templates/templates.json
# Or host the file and use: https://your-domain.com/templates.json
Docker Compose Stack Deployment Failed with "Invalid Reference Format"

Problem

Uploading a docker-compose.yml file through Portainer's web editor. Clicked "Deploy the stack", got "Failed to deploy stack: invalid reference format". The same docker-compose.yml worked perfectly when deployed from CLI with docker-compose up.

What I Tried

Attempt 1: Removed comments from docker-compose.yml - didn't help.
Attempt 2: Simplified the compose file to basic service - still failed.
Attempt 3: Tried pasting the content instead of uploading - same error.

Actual Fix

The issue was hidden characters in the web editor - specifically, smart quotes and em-dashes from copying documentation. Portainer's parser is stricter than docker-compose CLI. Fixed by: 1) Using a text editor to ensure plain ASCII, or 2) Uploading the file instead of pasting, or 3) Using Portainer's stack upload from git repository. 

# Instead of pasting into web editor:

# Option 1: Upload file directly
# In Portainer: Stacks → Add stack → Upload from file
# Select your docker-compose.yml

# Option 2: Use Git repository
# In Portainer: Stacks → Add stack → Git repository
# Repository URL: https://github.com/username/repo
# Compose path: docker-compose.yml (in repo)

# Option 3: Fix the text editor issue
# Use VS Code or similar to ensure no smart quotes
# Find: "" Replace: "  (straight quotes)
# Find: — Replace: --  (em dash to hyphens)

What I Learned

Production Setup

Complete setup for Portainer with agents across multiple servers. 

# 1. Main Portainer Server
docker run -d \
  -p 9443:9443 \
  --name portainer \
  --restart=always \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v portainer_data:/data \
  cr.portainer.io/portainer/portainer-ce:latest

# Access at: https://your-server-ip:9443
# Default admin account required on first login

# 2. Portainer Agent (on remote Docker hosts)
docker run -d \
  -p 9001:9001 \
  --name portainer-agent \
  --restart=always \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /var/lib/docker/volumes:/var/lib/docker/volumes \
  portainer/agent:latest

# 3. Add Agent to Portainer
# In Portainer UI: Environments → Add environment
# Select: Agent
# Environment URL: agent-server-ip:9001
# Or use SSH: ssh://user@agent-server-ip

# 4. Edge Agent (for IoT/remote devices with unstable connectivity)
# On edge device:
docker run -d \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v /var/lib/docker/volumes:/var/lib/docker/volumes \
  --name portainer-edge-agent \
  --restart=always \
  portainer/agent:latest

# In Portainer UI: Environments → Add edge environment
# The agent will poll Portainer periodically (no inbound connection needed)
# Docker Compose for Portainer stack
version: '3.8'

services:
  portainer:
    image: portainer/portainer-ce:latest
    container_name: portainer
    restart: unless-stopped
    ports:
      - "9443:9443"
      - "9000:9000"
      - "8000:8000"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - portainer_data:/data
    networks:
      - proxy-network

  agent:
    image: portainer/agent:latest
    container_name: portainer-agent
    restart: unless-stopped
    ports:
      - "9001:9001"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/lib/docker/volumes:/var/lib/docker/volumes
    networks:
      - proxy-network

volumes:
  portainer_data:

networks:
  proxy-network:
    external: true

Monitoring & Debugging

Key commands and monitoring tips for Portainer:

Essential Commands

# View Portainer logs
docker logs portainer -f

# Restart Portainer (preserves data)
docker restart portainer

# Backup Portainer data
docker run --rm \
  -v portainer_data:/data \
  -v $(pwd):/backup \
  alpine tar czf /backup/portainer-backup.tar.gz /data

# Restore from backup
docker run --rm \
  -v portainer_data:/data \
  -v $(pwd):/backup \
  alpine tar xzf /backup/portainer-backup.tar.gz

# Check agent connectivity
# In Portainer UI: Environments → Select environment → Status

Red Flags to Watch For

Related Resources