Nicolas Cantu 58cc2493e5 chore: consolidate ia_dev module, sync tooling, and harden gateways (0.0.5)
Initial state:
- ia_dev was historically referenced as ./ia_dev in docs and integrations, while the vendored module lives under services/ia_dev.
- AnythingLLM sync and hook installation had error masking / weak exit signaling.
- Proxy layers did not validate proxy path segments, allowing path normalization tricks.

Motivation:
- Make the IDE-oriented workflow usable (sync -> act -> deploy/preview) with explicit errors.
- Reduce security footguns in proxying and script automation.

Resolution:
- Standardize IA_DEV_ROOT usage and documentation to services/ia_dev.
- Add SSH remote data mirroring + optional AnythingLLM ingestion.
- Extend AnythingLLM pull sync to support upload-all/prefix and fail on upload errors.
- Harden smart-ide-sso-gateway and smart-ide-global-api proxying with safe-path checks and non-leaking error responses.
- Improve ia-dev-gateway runner validation and reduce sensitive path leakage.
- Add site scaffold tool (Vite/React) with OIDC + chat via sso-gateway -> orchestrator.

Root cause:
- Historical layout changes (submodule -> vendored tree) and missing central contracts for path resolution.
- Missing validation for proxy path traversal patterns.
- Overuse of silent fallbacks (|| true, exit 0 on partial failures) in automation scripts.

Impacted features:
- Project sync: git pull + AnythingLLM sync + remote data mirror ingestion.
- Site frontends: SSO gateway proxy and orchestrator intents (rag.query, chat.local).
- Agent execution: ia-dev-gateway script runner and SSE output.

Code modified:
- scripts/remote-data-ssh-sync.sh
- scripts/anythingllm-pull-sync/sync.mjs
- scripts/install-anythingllm-post-merge-hook.sh
- cron/git-pull-project-clones.sh
- services/smart-ide-sso-gateway/src/server.ts
- services/smart-ide-global-api/src/server.ts
- services/smart-ide-orchestrator/src/server.ts
- services/ia-dev-gateway/src/server.ts
- services/ia_dev/tools/site-generate.sh

Documentation modified:
- docs/** (architecture, API docs, ia_dev module + integration, scripts)

Configurations modified:
- config/services.local.env.example
- services/*/.env.example

Files in deploy modified:
- services/ia_dev/deploy/*

Files in logs impacted:
- logs/ia_dev.log (runtime only)
- .logs/* (runtime only)

Databases and other sources modified:
- None

Off-project modifications:
- None

Files in .smartIde modified:
- .smartIde/agents/*.md
- services/ia_dev/.smartIde/**

Files in .secrets modified:
- None

New patch version in VERSION:
- 0.0.5

CHANGELOG.md updated:
- yes
2026-04-04 18:36:43 +02:00

96 lines
2.1 KiB
Bash

#!/usr/bin/env bash
set -euo pipefail
require_ssh_key() {
local key_path="$1"
if [[ -z "$key_path" ]]; then
echo "SSH key path is required" >&2
return 1
fi
if [[ ! -f "$key_path" ]]; then
echo "SSH key not found: $key_path" >&2
return 1
fi
}
ssh_common_opts() {
local ssh_user="$1"
local ssh_host="$2"
# Keepalive to reduce flakiness through ProxyJump
# (observed: "Connection reset by peer" during scp/ssh).
#
# Notes:
# - Avoid SSH multiplexing here: ControlPath/ControlMaster can be flaky on Windows OpenSSH + MSYS paths.
# - Increased timeouts and keepalive settings to handle network instability
# - Compression disabled to reduce overhead and potential connection issues
echo \
-o BatchMode=yes \
-o StrictHostKeyChecking=accept-new \
-o ConnectTimeout=30 \
-o ServerAliveInterval=10 \
-o ServerAliveCountMax=6 \
-o TCPKeepAlive=yes \
-o Compression=no
}
ssh_run() {
local ssh_key="$1"
local ssh_user="$2"
local ssh_host="$3"
shift 3
require_ssh_key "$ssh_key"
local proxy_host="${DEPLOY_SSH_PROXY_HOST:-}"
local proxy_user="${DEPLOY_SSH_PROXY_USER:-$ssh_user}"
local proxy_args=()
if [[ -n "$proxy_host" ]]; then
proxy_args=(-J "$proxy_user@$proxy_host")
fi
# shellcheck disable=SC2207
local common_opts=($(ssh_common_opts "$ssh_user" "$ssh_host"))
ssh -i "$ssh_key" \
"${common_opts[@]}" \
"${proxy_args[@]}" \
"$ssh_user@$ssh_host" "$@"
}
scp_copy() {
local ssh_key="$1"
local src="$2"
local ssh_user="$3"
local ssh_host="$4"
local dst="$5"
local recursive="${6:-false}"
require_ssh_key "$ssh_key"
local proxy_host="${DEPLOY_SSH_PROXY_HOST:-}"
local proxy_user="${DEPLOY_SSH_PROXY_USER:-$ssh_user}"
local proxy_args=()
if [[ -n "$proxy_host" ]]; then
proxy_args=(-o "ProxyJump=$proxy_user@$proxy_host")
fi
# shellcheck disable=SC2207
local common_opts=($(ssh_common_opts "$ssh_user" "$ssh_host"))
local scp_opts=()
# Add -r for recursive copy if requested or if source is a directory
if [[ "$recursive" == "true" ]] || [[ -d "$src" ]]; then
scp_opts=(-r)
fi
scp -i "$ssh_key" \
"${scp_opts[@]}" \
"${common_opts[@]}" \
"${proxy_args[@]}" \
"$src" "$ssh_user@$ssh_host:$dst"
}