VPS Documentation

Self-hosted services and applications running on the ejfvps infrastructure

Applications

Main Website - ejfox.com - Personal site and blog (Nuxt.js)
Archive Wiki - archive.ejfox.com - MediaWiki knowledge base
Smallweb - *.ejfox.tools - Small self-hosted deno micro-apps
Kuma Uptime Monitoring - status.tools.ejfox.com - Service availability tracking
Personal APIs - ejfox.com/api/* - Self-quantification data endpoints
ArchiveBox - snap.ejfox.com - Web archiving and snapshot preservation
Smallweb - ejfox.tools - Personal app platform and development environment
n8n - n8n.tools.ejfox.com - Workflow automation and API orchestration
Grafana - grafana.tools.ejfox.com - System metrics and personal data dashboards
Loki - loki.tools.ejfox.com - Log aggregation and analysis
Umami Analytics - umami.tools.ejfox.com - Privacy-focused web analytics

Architecture

Domain Strategy

ejfox.com - Main personal brand and API endpoints
ejfox.tools - Development tools and utilities subdomain
*.tools.ejfox.com - Individual tool subdomains for organized access, powered by smallweb
archive.ejfox.com - Knowledge management and documentation
room302.studio - Studio work

Port Mapping

Service	Internal Port	Public URL	Purpose
Main Website	3013	ejfox.com	Personal site & blog
Personal APIs	3013	ejfox.com/api/*	Self-quantification endpoints
MediaWiki	8437	archive.ejfox.com	Knowledge base
ArchiveBox	4040	snap.ejfox.com	Web archiving
Smallweb	7777	ejfox.tools	Development platform
n8n	5678	n8n.tools.ejfox.com	Workflow automation
Datasette	8002	datasette.tools.ejfox.com	Data exploration
Capabilities	9991	capabilities.tools.ejfox.com	Skills tracking
Grafana	3333	grafana.tools.ejfox.com	Metrics dashboards
Loki	3100	loki.tools.ejfox.com	Log aggregation
Umami	3900	umami.tools.ejfox.com	Web analytics
Uptime Kuma	54321	status.tools.ejfox.com	Uptime monitoring
Room302	3012	room302.studio	Creative projects

System Overview

Host System: Debian GNU/Linux 12 (bookworm)
Kernel: Linux 6.1.0-37-cloud-amd64
Hostname: ejfvps
Architecture: x86_64
CPU: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz (2 cores)
Memory: 3.8GB RAM
Storage:

Root: 7.8GB (/dev/vda1)
Data: 20GB (/dev/vdb1)
Data2: 49GB (/dev/vdc)

Core Infrastructure Services

Reverse Proxy & SSL

Caddy - Modern web server with automatic HTTPS
- Port: 80/443
- Config: /etc/caddy/Caddyfile
- Features: Automatic SSL certificates, HTTP/2, reverse proxy

Container Orchestration

Docker Engine - Container runtime
Docker Compose - Multi-container application definitions
Containerd - Low-level container runtime

System Monitoring

Cloudflared - Cloudflare tunnel service
Fail2ban - Intrusion prevention system
Monit - System monitoring daemon

Application Stack

Analytics & Monitoring

Umami Analytics

Primary Instance:

Image: ghcr.io/umami-software/umami:postgresql-latest

Grafana Monitoring Stack

Grafana - Visualization dashboard
- Image: grafana/grafana:latest
- Plugins: ClickHouse, GitHub, Loki Explorer, Pyroscope, JSON datasource

Loki - Log aggregation system
- Image: grafana/loki:latest

Uptime Monitoring

Uptime Kuma - Self-hosted monitoring tool
- Image: louislam/uptime-kuma:latest
- Base URL: https://status.tools.ejfox.com
- Features: Website monitoring, notifications, status pages

Content Management

MediaWiki

Image: mediawiki:latest
Port: 8437 → 80
Database: SQLite (multiple databases)
Extensions: AWS S3 integration
Skin: Citizen theme
Volumes:
- Data: /home/debian/mediawiki/data
- Config: /home/debian/mediawiki/LocalSettings.php

ArchiveBox

Image: archivebox/archivebox:latest
Features: Web archiving, full-text search via Sonic
Components:
- Main archiver
- Background scheduler (ofelia-based)
- Sonic search backend
- noVNC remote browser control

Development & Automation

Smallweb

Image: ghcr.io/pomdtr/smallweb:latest
Ports: 7777, 2222
Purpose: Lightweight web app hosting platform
Projects: Multiple TypeScript/Deno applications including:
- Health dashboard and webhooks
- GitHub integrations
- Location tracking
- Screenshot generation
- Weather apps
- Mastodon integration

n8n Workflow Automation

Image: docker.n8n.io/n8nio/n8n

Retroscope

Custom Application - retroscope
Scheduler: mcuadros/ofelia:latest
Status: Healthy, with background job scheduling

Personal Websites

ejfox.com

Custom Build - ejfoxcom_ejfox
Port: 3013 → 3013
Size: 1.84GB image
Status: Production website

room302.studio

Custom Build - website_room302
Port: 3012 → 3012
Size: 1.82GB image
Status: Secondary website

Network Configuration

Backup Strategy

MediaWiki has backup/restore scripts
Database files stored in persistent volumes
Configuration files version controlled

Process Management

PM2

PM2 modules: Auto-pull, log rotation
Stopped processes: Discord bot, capabilities service
Log management: Automated log rotation

---

Docker Troubleshooting Guide

Common Docker Issues and Solutions

High CPU Usage by Docker

Symptoms: dockerd process consuming 100%+ CPU, system unresponsive, SSH slow or failing

Quick Fix: <syntaxhighlight lang="bash">

Force kill Docker

sudo kill -9 $(pgrep dockerd) sudo systemctl stop docker sudo systemctl stop docker.socket sudo systemctl stop containerd </syntaxhighlight>

Docker Bridge Networking Issues

Problem: Default bridge network conflicts with VPS network configuration

Solution: Disable Docker's bridge networking <syntaxhighlight lang="bash">

Create/edit Docker daemon config

sudo nano /etc/docker/daemon.json

Add this configuration:

{

 "iptables": false,
 "bridge": "none"

}

Restart Docker

sudo systemctl restart docker </syntaxhighlight>

Docker Won't Stay Dead

Problem: Docker restarts automatically even after stopping

Solution: Disable all Docker services and sockets <syntaxhighlight lang="bash"> sudo systemctl stop docker docker.socket containerd sudo systemctl disable docker docker.socket containerd sudo systemctl mask docker # Prevents any activation </syntaxhighlight>

Disk Space Management

Quick Space Recovery

Nuclear Docker cleanup (removes ALL unused data)

docker system prune -a --volumes --force

Clean journal logs

sudo journalctl --vacuum-time=3d

Clean apt cache

sudo apt clean

Find large files (when du doesn't work)

find /data2 -type f -size +100M -exec ls -lh {} \; 2>/dev/null | sort -k5 -rh ls -lhS /directory/ # Sort by size without du </syntaxhighlight>

Docker Overlay2 Cleanup

Docker's overlay2 directory accumulates layers over time. Location: `/var/lib/docker/overlay2/` or `/data2/docker/overlay2/`

Regular maintenance: <syntaxhighlight lang="bash">

Check Docker disk usage

docker system df

Remove unused images

docker image prune -a

Remove stopped containers

docker container prune

Remove unused volumes

docker volume prune </syntaxhighlight>

Emergency Recovery Procedures

When Console is Flooded with Errors

Try these keyboard combinations:

`Ctrl+Alt+F2` through `F6` - Switch to different TTY
`Ctrl+C` repeatedly - Interrupt running process
`Ctrl+Alt+Del` - Force system reboot

Creating Recovery Access

Option 1: Volume Mount Method

Stop broken instance
Create minimal rescue instance
Detach root volume from broken instance
Attach to rescue instance as /dev/vdb
Mount and fix:

<syntaxhighlight lang="bash"> sudo mount /dev/vdb1 /mnt sudo chroot /mnt

Fix whatever is broken

systemctl disable problem-service exit sudo umount /mnt </syntaxhighlight>

Option 2: Rebuild Instance

Preserves attached data volumes
Fresh OS installation
Keeps IP address
Requires reinstalling services

System Diagnostics

Essential Diagnostic Commands

System load and processes

top htop ps aux | grep -E "(docker|high-cpu-process)"

Disk I/O issues

iostat -x 1 3 iotop

Network interfaces

ip link show | grep veth | wc -l # Count Docker interfaces

Memory status

free -h

Disk usage (alternatives to du)

df -h ls -lhS /directory/ find /path -type f -size +100M -exec ls -lh {} \; </syntaxhighlight>

Log Investigation

Check system logs for time period

sudo journalctl --since "2025-09-09 07:00" --until "2025-09-09 08:00"

Docker-specific logs

sudo journalctl -u docker -n 100

Check for errors

sudo journalctl -b -p err

When logs show nothing (system too broken to log)
This indicates severe resource exhaustion

</syntaxhighlight>

Preventive Measures

Regular Maintenance Tasks

Weekly: Clean Docker

docker system prune -a --volumes

Monthly: Clean logs

sudo journalctl --vacuum-time=30d

Monitor disk usage

df -h docker system df </syntaxhighlight>

Critical Configuration Files

`/etc/docker/daemon.json` - Docker daemon configuration
`/etc/systemd/system/docker.service.d/` - Docker service overrides
`/var/lib/docker/` - Docker data directory (can be symlinked to larger disk)

Resource Monitoring

Set up alerts for:

Disk usage above 90%
CPU usage sustained above 80%
Memory usage above 85%
Docker overlay2 directory size

Network Configuration for VPS

Alternative Docker Network Modes

When bridge networking causes issues: <syntaxhighlight lang="bash">

Host networking (shares host network stack)

docker run --network host image:tag

None (no network)

docker run --network none image:tag

Custom networks (create your own)

docker network create mynet --driver bridge docker run --network mynet image:tag </syntaxhighlight>

Firewall Considerations

Docker manipulates iptables by default. To prevent this:

Set `"iptables": false` in daemon.json
Manage firewall rules manually with ufw or iptables
Use reverse proxy (Caddy/Nginx) for port management