anchorage_layer_simple/data/restart-services-cron.sh
ncantu 31fd2c3c8b Fix: Improve Docker services automatic startup and restart order
**Motivations:**
- Docker services (bitcoind and mempool) were not starting automatically after reboot
- Mempool API was unhealthy because it tried to connect to bitcoind before it was ready
- No restart policy for bitcoind container
- Restart script started mempool before bitcoind, causing connection errors

**Root causes:**
- Bitcoind container had no restart policy (restart: no)
- Restart script started mempool before bitcoind
- No wait for bitcoind RPC to be ready before starting mempool
- No boot startup script for Docker services

**Correctifs:**
- Configured restart policy for bitcoind container: docker update --restart=always
- Improved restart-services-cron.sh:
  - Start/restart bitcoind BEFORE mempool
  - Wait for bitcoind RPC to be ready (max 60s) before starting mempool
  - Detect if container is stopped (start) or running (restart)
  - Use 'docker compose up -d' for mempool instead of 'restart'
- Created start-docker-services.sh:
  - Start bitcoind first
  - Wait for bitcoind RPC to be ready (max 120s at boot)
  - Start mempool after bitcoind
  - Logging to data/start-docker-services.log
- Created docker-services.service:
  - Systemd service to start Docker services at boot
  - Runs after docker.service and network.target
  - Executes start-docker-services.sh

**Evolutions:**
- Automatic startup at boot: Docker services start automatically after reboot
- Guaranteed startup order: Bitcoind always starts before mempool
- Availability check: Wait for bitcoind to be ready before starting mempool
- Restart policy: Bitcoind container restarts automatically if stopped
- Better observability: Detailed logs for diagnosis

**Pages affectées:**
- data/restart-services-cron.sh: Improved startup order and availability checks
- data/start-docker-services.sh: New boot startup script
- data/docker-services.service: New systemd service for automatic startup
- fixKnowledge/docker-services-boot-startup.md: Documentation of improvements
2026-01-28 11:48:25 +01:00

129 lines
3.9 KiB
Bash
Executable File

#!/bin/bash
# Daily restart of project systemd services (local only).
# Run via cron. Requires passwordless sudo for systemctl restart.
# Log: data/restart-services.log
#
# Local only: no SSH, no remote commands. Manages systemd units and
# local Docker (mempool stack, bitcoind container) on the current machine.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PROJECT_DIR="$(dirname "$SCRIPT_DIR")"
LOG_FILE="$SCRIPT_DIR/restart-services.log"
MEMPOOL_COMPOSE="${PROJECT_DIR}/mempool/docker-compose.signet.yml"
BITCOIND_CONTAINER="bitcoin-signet-instance"
SERVICES=(
bitcoin-signet
bitcoin
anchorage-api
api-relay
clamav-api
faucet-api
filigrane-api
signet-dashboard
userwallet
website-skeleton
)
log() { echo "$(date -Iseconds) $*" | tee -a "$LOG_FILE"; }
cd "$PROJECT_DIR" || exit 1
log "=== Restart services cron (local only, no SSH) ==="
# 1. Verify enabled (warn if not)
for u in "${SERVICES[@]}"; do
if ! systemctl is-enabled "$u" &>/dev/null; then
log "WARN: $u is not enabled (will not start at boot)"
fi
done
# 2. Restart active units; track attempted
restarted=()
for u in "${SERVICES[@]}"; do
if ! systemctl is-active --quiet "$u" 2>/dev/null; then
log "SKIP: $u not active (not installed or not running on this machine)"
continue
fi
log "Restarting $u..."
if sudo systemctl restart "$u"; then
log " $u OK"
restarted+=("$u")
else
log " $u FAILED"
restarted+=("$u")
fi
done
# 3. Verify active for restarted units
for u in "${restarted[@]}"; do
if ! systemctl is-active --quiet "$u" 2>/dev/null; then
log "WARN: $u not active after restart"
fi
done
# 4. Docker: bitcoind container (local only) - START FIRST
# Bitcoind must be running before mempool can connect to it
if docker ps -a -q -f "name=^${BITCOIND_CONTAINER}$" 2>/dev/null | grep -q .; then
log "Starting/restarting $BITCOIND_CONTAINER..."
bitcoind_ok=0
if docker ps -q -f "name=^${BITCOIND_CONTAINER}$" 2>/dev/null | grep -q .; then
# Container is running, just restart it
if docker restart "$BITCOIND_CONTAINER" &>/dev/null; then
bitcoind_ok=1
fi
else
# Container exists but is stopped, start it
if docker start "$BITCOIND_CONTAINER" &>/dev/null; then
bitcoind_ok=1
fi
fi
if [ "$bitcoind_ok" -eq 1 ]; then
log " $BITCOIND_CONTAINER started, waiting for RPC to be ready..."
# Wait for bitcoind RPC to be ready (max 60 seconds)
max_wait=60
wait_count=0
while [ $wait_count -lt $max_wait ]; do
if docker exec "$BITCOIND_CONTAINER" bitcoin-cli -datadir=/root/.bitcoin getblockchaininfo &>/dev/null; then
log " $BITCOIND_CONTAINER RPC ready"
break
fi
sleep 1
wait_count=$((wait_count + 1))
done
if [ $wait_count -ge $max_wait ]; then
log " WARN: $BITCOIND_CONTAINER RPC not ready after ${max_wait}s"
fi
else
log " $BITCOIND_CONTAINER FAILED"
fi
else
log "SKIP: $BITCOIND_CONTAINER container not found (bitcoind may run as systemd)"
fi
# 5. Docker: mempool stack (local only) - START AFTER BITCOIND
# Mempool depends on bitcoind being ready
if [ -f "$MEMPOOL_COMPOSE" ] && command -v docker &>/dev/null; then
log "Starting/restarting mempool (docker)..."
mem_ok=1
(cd "${PROJECT_DIR}/mempool" && docker compose -f docker-compose.signet.yml up -d &>/dev/null) && mem_ok=0
[ "$mem_ok" -ne 0 ] && (cd "${PROJECT_DIR}/mempool" && docker-compose -f docker-compose.signet.yml up -d &>/dev/null) && mem_ok=0
if [ "$mem_ok" -eq 0 ]; then
log " mempool started"
# Wait a bit for mempool to initialize
sleep 5
else
log " mempool FAILED"
fi
else
if [ ! -f "$MEMPOOL_COMPOSE" ]; then
log "SKIP: mempool compose not found (not deployed on this machine)"
else
log "SKIP: mempool requires docker (not available)"
fi
fi
log "=== Done ==="
tail -n 100 "$LOG_FILE" > "$LOG_FILE.tmp" && mv "$LOG_FILE.tmp" "$LOG_FILE"