VPS

VPS Documentation

Self-hosted services and applications running on the ejfvps infrastructure

Applications

• Main Website - ejfox.com - Personal site and blog (Nuxt.js)
• Archive Wiki - archive.ejfox.com - MediaWiki knowledge base
• Smallweb - *.ejfox.tools - Small self-hosted deno micro-apps
• Kuma Uptime Monitoring - status.tools.ejfox.com - Service availability tracking
• Personal APIs - ejfox.com/api/* - Self-quantification data endpoints
• ArchiveBox - snap.ejfox.com - Web archiving and snapshot preservation
• n8n - n8n.tools.ejfox.com - Workflow automation and API orchestration
• Grafana - grafana.tools.ejfox.com - System metrics and personal data dashboards
• Loki - loki.tools.ejfox.com - Log aggregation and analysis
• Umami Analytics - umami.tools.ejfox.com - Privacy-focused web analytics

System Overview

Hardware Specifications

Component	Specification
Host System	Debian GNU/Linux 12 (bookworm)
Kernel	Linux 6.1.0-37-cloud-amd64
Hostname	ejfvps
Architecture	x86_64
CPU	Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz (2 cores)
Memory	3.8GB RAM
Root Storage	7.8GB (/dev/vda1)
Data Storage	20GB (/dev/vdb1)
Data2 Storage	49GB (/dev/vdc)

Port Mapping

Service	Internal Port	Public URL	Purpose
Main Website	3013	ejfox.com	Personal site & blog
Personal APIs	3013	ejfox.com/api/*	Self-quantification endpoints
MediaWiki	8437	archive.ejfox.com	Knowledge base
ArchiveBox	4040	snap.ejfox.com	Web archiving
Smallweb	7777	ejfox.tools	Development platform
n8n	5678	n8n.tools.ejfox.com	Workflow automation
Datasette	8002	datasette.tools.ejfox.com	Data exploration
Capabilities	9991	capabilities.tools.ejfox.com	Skills tracking
Grafana	3333	grafana.tools.ejfox.com	Metrics dashboards
Loki	3100	loki.tools.ejfox.com	Log aggregation
Umami	3900	umami.tools.ejfox.com	Web analytics
Uptime Kuma	54321	status.tools.ejfox.com	Uptime monitoring
Room302	3012	room302.studio	Creative projects

Docker Troubleshooting

Emergency Procedures

Kill Runaway Docker Process

When dockerd consumes 100%+ CPU:

# Force kill Docker
sudo kill -9 $(pgrep dockerd)
sudo systemctl stop docker
sudo systemctl stop docker.socket
sudo systemctl stop containerd

Disable Docker Bridge Networking

Prevents bridge networking conflicts on VPS:

# Edit /etc/docker/daemon.json
{
  "iptables": false,
  "bridge": "none"
}

# Restart Docker
sudo systemctl restart docker

Prevent Docker Auto-Restart

sudo systemctl stop docker docker.socket containerd
sudo systemctl disable docker docker.socket containerd
sudo systemctl mask docker  # Prevents any activation

Disk Space Management

Quick Cleanup Commands

# Nuclear Docker cleanup (removes ALL unused data)
docker system prune -a --volumes --force

# Clean journal logs
sudo journalctl --vacuum-time=3d

# Clean apt cache
sudo apt clean

Find Large Files Without du

When du hangs or is slow:

# Find files over 100MB
find /data2 -type f -size +100M -exec ls -lh {} \; 2>/dev/null | sort -k5 -rh

# Sort directory by size
ls -lhS /directory/

# Count Docker veth interfaces
ip link show | grep veth | wc -l

Docker Overlay2 Maintenance

Location: `/var/lib/docker/overlay2/` or `/data2/docker/overlay2/`

# Check Docker disk usage
docker system df

# Progressive cleanup
docker image prune -a       # Remove unused images
docker container prune       # Remove stopped containers
docker volume prune          # Remove unused volumes
docker builder prune         # Remove build cache

System Diagnostics

Process Monitoring

# System load
top
ps aux | grep -E "(docker|containerd)"

# Memory status
free -h

# Disk status
df -h

Log Investigation

# Check time-specific logs
sudo journalctl --since "2025-09-09 07:00" --until "2025-09-09 08:00"

# Docker logs
sudo journalctl -u docker -n 100

# System errors
sudo journalctl -b -p err

Recovery Methods

Console Keyboard Shortcuts

• Ctrl+Alt+F2 to F6 - Switch TTY when console is flooded
• Ctrl+C - Interrupt running process
• Ctrl+Alt+Del - Force system reboot

Volume Mount Recovery

When instance is inaccessible:

1. Stop broken instance
2. Create minimal rescue instance
3. Detach root volume from broken instance
4. Attach to rescue instance as secondary drive
5. Mount and repair:

sudo mount /dev/vdb1 /mnt
sudo chroot /mnt
# Fix issues (disable services, edit configs, etc)
systemctl disable problem-service
exit
sudo umount /mnt

Preventive Maintenance

Weekly Tasks

• Clean Docker: `docker system prune -a --volumes`
• Check disk usage: `df -h`
• Review Docker disk usage: `docker system df`

Monthly Tasks

• Clean old logs: `sudo journalctl --vacuum-time=30d`
• Update system packages: `sudo apt update && sudo apt upgrade`
• Review container resource usage

Critical Files

File/Directory	Purpose
/etc/docker/daemon.json	Docker daemon configuration
/etc/systemd/system/docker.service.d/	Docker service overrides
/var/lib/docker/	Docker data directory
/etc/caddy/Caddyfile	Reverse proxy configuration

Network Configuration

Docker Network Alternatives

When bridge networking fails:

# Host networking (shares host network)
docker run --network host image:tag

# No network
docker run --network none image:tag

# Custom network
docker network create mynet --driver bridge
docker run --network mynet image:tag

Firewall Management

With Docker iptables disabled:

• Manage ports via Caddy reverse proxy
• Use ufw for firewall rules
• Manual iptables configuration if needed

Monitoring Thresholds

Metric	Warning Level	Critical Level
Disk Usage	85%	95%
CPU Usage (sustained)	70%	90%
Memory Usage	80%	90%
Docker overlay2 size	20GB	30GB

Service-Specific Documentation

Core Services

Caddy (Reverse Proxy)

• Port: 80/443
• Config: /etc/caddy/Caddyfile
• Features: Automatic SSL, HTTP/2, reverse proxy
• Restart: `sudo systemctl restart caddy`

Docker & Containerd

• Config: /etc/docker/daemon.json
• Data: /data2/docker/
• Status: `systemctl status docker`
• Logs: `journalctl -u docker`

PM2 Process Manager

• Status: `pm2 status`
• Logs: `pm2 logs`
• Save config: `pm2 save`
• Startup script: `pm2 startup`

Container Management

Essential Container Commands

# View all containers
docker ps -a

# Start specific service
docker start container_name

# View logs
docker logs -f container_name

# Restart with docker-compose
cd /path/to/project
docker-compose up -d

# Rebuild container
docker-compose build --no-cache
docker-compose up -d

Backup Procedures

• MediaWiki: Backup scripts in `/home/debian/mediawiki/`
• Docker volumes: Located in `/data2/docker/volumes/`
• Databases: Regular dumps to `/data/backups/`
• Configuration: Version controlled in git repositories