Jump to content

VPS

From Archive

VPS Documentation

Self-hosted services and applications running on the ejfvps infrastructure

Applications

System Overview

Hardware Specifications

Component Specification
Host System Debian GNU/Linux 12 (bookworm)
Kernel Linux 6.1.0-37-cloud-amd64
Hostname ejfvps
Architecture x86_64
CPU Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz (2 cores)
Memory 3.8GB RAM
Root Storage 7.8GB (/dev/vda1)
Data Storage 20GB (/dev/vdb1)
Data2 Storage 49GB (/dev/vdc)

Port Mapping

Service Internal Port Public URL Purpose
Main Website 3013 ejfox.com Personal site & blog
Personal APIs 3013 ejfox.com/api/* Self-quantification endpoints
MediaWiki 8437 archive.ejfox.com Knowledge base
ArchiveBox 4040 snap.ejfox.com Web archiving
Smallweb 7777 ejfox.tools Development platform
n8n 5678 n8n.tools.ejfox.com Workflow automation
Datasette 8002 datasette.tools.ejfox.com Data exploration
Capabilities 9991 capabilities.tools.ejfox.com Skills tracking
Grafana 3333 grafana.tools.ejfox.com Metrics dashboards
Loki 3100 loki.tools.ejfox.com Log aggregation
Umami 3900 umami.tools.ejfox.com Web analytics
Uptime Kuma 54321 status.tools.ejfox.com Uptime monitoring
Room302 3012 room302.studio Creative projects

Docker Troubleshooting

Emergency Procedures

Kill Runaway Docker Process

When dockerd consumes 100%+ CPU: 

# Force kill Docker
sudo kill -9 $(pgrep dockerd)
sudo systemctl stop docker
sudo systemctl stop docker.socket
sudo systemctl stop containerd

Disable Docker Bridge Networking

Prevents bridge networking conflicts on VPS: 

# Edit /etc/docker/daemon.json
{
  "iptables": false,
  "bridge": "none"
}

# Restart Docker
sudo systemctl restart docker

Prevent Docker Auto-Restart

sudo systemctl stop docker docker.socket containerd
sudo systemctl disable docker docker.socket containerd
sudo systemctl mask docker  # Prevents any activation

Disk Space Management

Quick Cleanup Commands

# Nuclear Docker cleanup (removes ALL unused data)
docker system prune -a --volumes --force

# Clean journal logs
sudo journalctl --vacuum-time=3d

# Clean apt cache
sudo apt clean

Find Large Files Without du

When du hangs or is slow: 

# Find files over 100MB
find /data2 -type f -size +100M -exec ls -lh {} \; 2>/dev/null | sort -k5 -rh

# Sort directory by size
ls -lhS /directory/

# Count Docker veth interfaces
ip link show | grep veth | wc -l

Docker Overlay2 Maintenance

Location: `/var/lib/docker/overlay2/` or `/data2/docker/overlay2/` 

# Check Docker disk usage
docker system df

# Progressive cleanup
docker image prune -a       # Remove unused images
docker container prune       # Remove stopped containers
docker volume prune          # Remove unused volumes
docker builder prune         # Remove build cache

System Diagnostics

Process Monitoring

# System load
top
ps aux | grep -E "(docker|containerd)"

# Memory status
free -h

# Disk status
df -h

Log Investigation

# Check time-specific logs
sudo journalctl --since "2025-09-09 07:00" --until "2025-09-09 08:00"

# Docker logs
sudo journalctl -u docker -n 100

# System errors
sudo journalctl -b -p err

Recovery Methods

Console Keyboard Shortcuts

  • Ctrl+Alt+F2 to F6 - Switch TTY when console is flooded
  • Ctrl+C - Interrupt running process
  • Ctrl+Alt+Del - Force system reboot

Volume Mount Recovery

When instance is inaccessible:

  1. Stop broken instance
  2. Create minimal rescue instance
  3. Detach root volume from broken instance
  4. Attach to rescue instance as secondary drive
  5. Mount and repair:
sudo mount /dev/vdb1 /mnt
sudo chroot /mnt
# Fix issues (disable services, edit configs, etc)
systemctl disable problem-service
exit
sudo umount /mnt

Preventive Maintenance

Weekly Tasks

  • Clean Docker: `docker system prune -a --volumes`
  • Check disk usage: `df -h`
  • Review Docker disk usage: `docker system df`

Monthly Tasks

  • Clean old logs: `sudo journalctl --vacuum-time=30d`
  • Update system packages: `sudo apt update && sudo apt upgrade`
  • Review container resource usage

Critical Files

File/Directory Purpose
/etc/docker/daemon.json Docker daemon configuration
/etc/systemd/system/docker.service.d/ Docker service overrides
/var/lib/docker/ Docker data directory
/etc/caddy/Caddyfile Reverse proxy configuration

Network Configuration

Docker Network Alternatives

When bridge networking fails: 

# Host networking (shares host network)
docker run --network host image:tag

# No network
docker run --network none image:tag

# Custom network
docker network create mynet --driver bridge
docker run --network mynet image:tag

Firewall Management

With Docker iptables disabled:

  • Manage ports via Caddy reverse proxy
  • Use ufw for firewall rules
  • Manual iptables configuration if needed

Monitoring Thresholds

Metric Warning Level Critical Level
Disk Usage 85% 95%
CPU Usage (sustained) 70% 90%
Memory Usage 80% 90%
Docker overlay2 size 20GB 30GB

Service-Specific Documentation

Core Services

Caddy (Reverse Proxy)

  • Port: 80/443
  • Config: /etc/caddy/Caddyfile
  • Features: Automatic SSL, HTTP/2, reverse proxy
  • Restart: `sudo systemctl restart caddy`

Docker & Containerd

  • Config: /etc/docker/daemon.json
  • Data: /data2/docker/
  • Status: `systemctl status docker`
  • Logs: `journalctl -u docker`

PM2 Process Manager

  • Status: `pm2 status`
  • Logs: `pm2 logs`
  • Save config: `pm2 save`
  • Startup script: `pm2 startup`

Container Management

Essential Container Commands

# View all containers
docker ps -a

# Start specific service
docker start container_name

# View logs
docker logs -f container_name

# Restart with docker-compose
cd /path/to/project
docker-compose up -d

# Rebuild container
docker-compose build --no-cache
docker-compose up -d

Backup Procedures

  • MediaWiki: Backup scripts in `/home/debian/mediawiki/`
  • Docker volumes: Located in `/data2/docker/volumes/`
  • Databases: Regular dumps to `/data/backups/`
  • Configuration: Version controlled in git repositories
Retrieved from "https://archive.ejfox.com/index.php?title=VPS&oldid=182"