Nicolas Cantu 58cc2493e5 chore: consolidate ia_dev module, sync tooling, and harden gateways (0.0.5)
Initial state:
- ia_dev was historically referenced as ./ia_dev in docs and integrations, while the vendored module lives under services/ia_dev.
- AnythingLLM sync and hook installation had error masking / weak exit signaling.
- Proxy layers did not validate proxy path segments, allowing path normalization tricks.

Motivation:
- Make the IDE-oriented workflow usable (sync -> act -> deploy/preview) with explicit errors.
- Reduce security footguns in proxying and script automation.

Resolution:
- Standardize IA_DEV_ROOT usage and documentation to services/ia_dev.
- Add SSH remote data mirroring + optional AnythingLLM ingestion.
- Extend AnythingLLM pull sync to support upload-all/prefix and fail on upload errors.
- Harden smart-ide-sso-gateway and smart-ide-global-api proxying with safe-path checks and non-leaking error responses.
- Improve ia-dev-gateway runner validation and reduce sensitive path leakage.
- Add site scaffold tool (Vite/React) with OIDC + chat via sso-gateway -> orchestrator.

Root cause:
- Historical layout changes (submodule -> vendored tree) and missing central contracts for path resolution.
- Missing validation for proxy path traversal patterns.
- Overuse of silent fallbacks (|| true, exit 0 on partial failures) in automation scripts.

Impacted features:
- Project sync: git pull + AnythingLLM sync + remote data mirror ingestion.
- Site frontends: SSO gateway proxy and orchestrator intents (rag.query, chat.local).
- Agent execution: ia-dev-gateway script runner and SSE output.

Code modified:
- scripts/remote-data-ssh-sync.sh
- scripts/anythingllm-pull-sync/sync.mjs
- scripts/install-anythingllm-post-merge-hook.sh
- cron/git-pull-project-clones.sh
- services/smart-ide-sso-gateway/src/server.ts
- services/smart-ide-global-api/src/server.ts
- services/smart-ide-orchestrator/src/server.ts
- services/ia-dev-gateway/src/server.ts
- services/ia_dev/tools/site-generate.sh

Documentation modified:
- docs/** (architecture, API docs, ia_dev module + integration, scripts)

Configurations modified:
- config/services.local.env.example
- services/*/.env.example

Files in deploy modified:
- services/ia_dev/deploy/*

Files in logs impacted:
- logs/ia_dev.log (runtime only)
- .logs/* (runtime only)

Databases and other sources modified:
- None

Off-project modifications:
- None

Files in .smartIde modified:
- .smartIde/agents/*.md
- services/ia_dev/.smartIde/**

Files in .secrets modified:
- None

New patch version in VERSION:
- 0.0.5

CHANGELOG.md updated:
- yes
2026-04-04 18:36:43 +02:00

92 lines
5.4 KiB
Bash
Executable File

#!/usr/bin/env bash
# Agent loop: poll for mails to treat (spooler criterion: from/to in conf.json, not IMAP unread).
# Run from repo root. Runs forever. Do NOT start from the agent-loop agent (use bounded runs only, e.g. agent-loop-chat-iterations.sh N).
#
# Usage (manual only; agent must not launch with nohup/&):
# ./git-issues/agent-loop.sh [interval_seconds]
# AGENT_LOOP_INTERVAL_SEC=120 ./git-issues/agent-loop.sh
#
# Witness file: projects/<id>/logs/git-issues/agent-loop.status (or AGENT_LOOP_STATUS_FILE)
# State file (not a log): updated every iteration. If mtime is older than 2*interval, loop is considered stopped.
# Pending file: projects/<id>/logs/git-issues/agent-loop.pending (or AGENT_LOOP_PENDING_FILE)
# Written when there are .pending mails in the spooler (no matching .response); contains paths to treat.
#
# Optional: set AGENT_LOOP_RUN_AGENT=1 to run the Cursor Agent CLI when mails are detected.
# Requires Cursor Agent CLI (https://cursor.com/docs/cli/using). If "agent" is not in PATH, the loop only updates status/pending.
#
# Optional: AGENT_LOOP_MODEL=<model> to force the model (e.g. sonnet-4.6, gpt-5.4-low). Default: sonnet-4.6 to avoid Opus usage limits when running unattended.
#
set -euo pipefail
# Source user env so PATH includes ~/.local/bin (Cursor Agent CLI, etc.)
if [ -n "${HOME:-}" ] && [ -r "$HOME/.bashrc" ]; then
set +u
# shellcheck source=/dev/null
source "$HOME/.bashrc" 2>/dev/null || true
set -u
fi
[ -n "${HOME:-}" ] && [ -d "$HOME/.local/bin" ] && export PATH="$HOME/.local/bin:$PATH"
GIT_ISSUES_DIR="${GIT_ISSUES_DIR:-$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)}"
ROOT="$(cd "${GIT_ISSUES_DIR}/../.." && pwd)"
export GIT_ISSUES_DIR
export REPO_ROOT="${ROOT}"
cd "$ROOT"
# Per-project logs under projects/<id>/logs (lib.sh sets PROJECT_LOGS_DIR)
# shellcheck source=lib.sh
source "${GIT_ISSUES_DIR}/lib.sh" 2>/dev/null || true
LOGS_GIT_ISSUES="${PROJECT_LOGS_DIR:-$ROOT/logs}/git-issues"
# Load agent-loop parameters from .secrets (optional; .secrets under ia_dev)
AGENT_LOOP_ENV="${GIT_ISSUES_DIR}/../.secrets/git-issues/agent-loop.env"
if [ -r "$AGENT_LOOP_ENV" ]; then
set +u
# shellcheck source=/dev/null
source "$AGENT_LOOP_ENV"
set -u
fi
INTERVAL="${1:-${AGENT_LOOP_INTERVAL_SEC:-60}}"
# STATUS_FILE: state/witness file (not a log) — indicates if the loop is active; under projects/<id>/logs/git-issues/ for per-project state.
STATUS_FILE="${AGENT_LOOP_STATUS_FILE:-$LOGS_GIT_ISSUES/agent-loop.status}"
PENDING_FILE="${AGENT_LOOP_PENDING_FILE:-$LOGS_GIT_ISSUES/agent-loop.pending}"
mkdir -p "$(dirname "$STATUS_FILE")"
write_status() {
local status="$1"
local detail="${2:-}"
printf "%s\n%s\n%s\n" "$(date -Iseconds)" "$status" "$detail" > "$STATUS_FILE"
}
while true; do
write_status "running" "interval=${INTERVAL}s"
# Spooler flow: fetch by from/to (conf.json), then list .pending without .response. No IMAP unread criterion.
"${GIT_ISSUES_DIR}/tickets-fetch-inbox.sh" 2>&1 || true
pending_out=""
pending_out=$("${GIT_ISSUES_DIR}/list-pending-spooler.sh" 2>&1) || true
if [ -n "$pending_out" ] && echo "$pending_out" | grep -q "\.pending"; then
write_status "mails_pending" "Mails en attente dans le spooler (from/to). Lancer l'agent git-issues-process dans Cursor."
printf "%s\n%s\n%s\n%s\n" "$(date -Iseconds)" "mails_pending" "---" "$pending_out" > "$PENDING_FILE"
n=$(echo "$pending_out" | grep -c "\.pending" || true)
echo "[agent-loop] $(date -Iseconds)$n mail(s) en attente dans le spooler (critère from/to). Lancer l'agent git-issues-process dans Cursor."
if [ "${AGENT_LOOP_RUN_AGENT:-0}" = "1" ] && command -v agent >/dev/null 2>&1; then
write_status "running_agent" "Lancement de l'agent Cursor pour traiter les mails du spooler."
echo "[agent-loop] $(date -Iseconds) — Lancement de l'agent Cursor (workflow git-issues-process spooler)."
AGENT_MODEL="${AGENT_LOOP_MODEL:-sonnet-4.6}"
echo "[agent-loop] $(date -Iseconds) — Modèle: $AGENT_MODEL"
AGENT_OPTS=(-p "Exécute le workflow mails du spooler (agent git-issues-process). 1) Lister les mails à traiter avec ./git-issues/list-pending-spooler.sh (ou utiliser les chemins dans agent-loop.pending). 2) Pour chaque fichier .pending : lire le JSON (from, to, subject, body, message_id, base). Répondre à l'expéditeur (--to <from> du JSON), pas à une adresse fixe ; le « to » du mail reçu a déjà déterminé le projet. Rédiger une réponse pertinente (uniquement ton texte, pas de citation ; mail-send-reply.sh refuse si le body contient From:, Message-ID, wrote:, etc.). Envoyer avec ./git-issues/mail-send-reply.sh --to <from> --subject \"Re: ...\" --body \"<ta_réponse>\" --in-reply-to \"<message_id>\". Après envoi réussi : ./git-issues/write-response-spooler.sh --base <base> --to <from> --subject \"Re: ...\" --body \"<ta_réponse>\" --in-reply-to \"<message_id>\". Ne pas appeler mail-mark-read.sh (spooler)." -f --model "$AGENT_MODEL")
if agent "${AGENT_OPTS[@]}" 2>&1; then
write_status "agent_done" "Agent terminé."
else
write_status "mails_pending" "Agent terminé avec erreur ou interruption. Relancer l'agent manuellement si besoin."
fi
fi
else
write_status "idle" "Aucun mail en attente dans le spooler (critère: from/to dans conf.json)."
if [ -f "$PENDING_FILE" ]; then
: > "$PENDING_FILE"
fi
echo "[agent-loop] $(date -Iseconds) — Aucun mail en attente dans le spooler (critère: from/to dans conf.json)."
fi
sleep "$INTERVAL"
done