9 Commits

Author SHA1 Message Date
ncantu
8662e9e584 Optimize anchor API performance and fix request abort issues
**Motivations:**
- Anchor API requests were being aborted with 'This operation was aborted' error
- API anchorage had performance issues causing long response times (30-60s)
- Mutex could block indefinitely if a previous request crashed or timed out

**Root causes:**
- No timeout on mutex acquisition, causing indefinite blocking
- Sequential RPC calls (8+ getNewAddress calls) instead of parallel
- Expensive fee calculation making N RPC calls (one per input) instead of using known UTXO amounts
- RPC timeout too short (30s) for slow Bitcoin node operations
- No guarantee of mutex release in error cases

**Correctifs:**
- Added 180s timeout on mutex acquisition with Promise.race() to prevent indefinite blocking
- Parallelized getNewAddress() calls with Promise.all() (9 sequential calls → 1 parallel call)
- Optimized fee calculation to use known UTXO amounts instead of getRawTransaction() per input (saves N RPC calls, up to 20+)
- Increased RPC timeout from 30s to 120s in .env.example
- Added finally block to guarantee mutex release in all cases (success, error, timeout)
- Added timeout and explicit error handling in api-filigrane callAnchorAPI() with AbortController (120s timeout)

**Evolutions:**
- Performance improvement: execution time reduced from ~30-60s to ~10-20s
- RPC calls reduction: from ~15-35 calls to ~8-12 calls per anchor transaction
- Better resilience: mutex cannot block indefinitely anymore
- Improved error messages with explicit timeout/abort information

**Pages affectées:**
- api-anchorage/src/bitcoin-rpc.js: mutex timeout, parallel address generation, optimized fee calculation, finally block
- api-anchorage/.env.example: increased RPC timeout to 120s
- api-filigrane/src/routes/watermark.js: timeout and error handling for anchor API calls
- fixKnowledge/api-filigrane-anchor-request-aborted.md: documentation of issues and fixes
2026-01-28 11:38:43 +01:00
ncantu
9291eab9d5 UserWallet: runCollectLoop options object, root null check, pairing/collect/bip32
**Motivations:**
- Passer runCollectLoop en objet d’options pour lisibilité et évolution
- Éviter non-null assertion sur root dans main.tsx
- Ajustements pairing, collect signatures, bip32

**Root causes:**
- N/A (évolutions + correctifs ciblés)

**Correctifs:**
- main.tsx: vérification root !== null avant createRoot, throw explicite si absent

**Evolutions:**
- LoginScreen: runCollectLoop({ relayEndpoints, hash, ourSigs, path, pairToMembers, pubkeyToPair, opts })
- pairingConfirm.ts, PairingDisplayScreen, PairingSetupBlock: modifications
- collectSignatures.ts, bip32.ts: ajustements
- SyncScreen: modifications mineures

**Pages affectées:**
- userwallet: LoginScreen, main.tsx, PairingDisplayScreen, PairingSetupBlock, SyncScreen, pairingConfirm.ts, bip32.ts, collectSignatures.ts
- data: sync-utxos.log
2026-01-28 09:43:55 +01:00
ncantu
13898d1012 UserWallet: useRelayNotifications, CNIL validation, ServiceSync polling, SyncScreen link
**Motivations:**
- Intégrer les notifications relais (polling, auto-process hashes) dans ServiceSyncScreen
- Valider conformité CNIL des contrats/champs à l’ajout au graphe
- Accéder à la sync par service depuis SyncScreen

**Root causes:**
- N/A (évolutions + petits correctifs)

**Correctifs:**
- relayNotificationService: suppression import inutilisé RelayConfig

**Evolutions:**
- useRelayNotifications: hook (RelayNotificationService, start/stop polling, auto-process hash)
- cnilValidation: validateContractCNIL, validateChampCNIL (valid, errors, warnings)
- graphResolver: addContrat/addChamp appellent validation CNIL (async, logs warnings/errors, non bloquant)
- ServiceSyncScreen: useRelayNotifications, GraphResolver ref, polling selon configs (fréquence min), sync result (hasMessages, « Aucun nouveau message »), useBloom mention
- SyncScreen: bouton « Sync par service » vers /service-sync

**Pages affectées:**
- userwallet: useRelayNotifications, cnilValidation, ServiceSyncScreen, SyncScreen, graphResolver, relayNotificationService
2026-01-28 01:42:26 +01:00
ncantu
f27345e0ba RESTE_A_FAIRE, relay validation, verify proof structure, UserWallet diagnostic/sync
**Motivations:**
- Documenter le reste à faire (userwallet, service-login-verify, api-relay, website-skeleton)
- Renforcer la validation côté api-relay et service-login-verify
- Ajouter écrans diagnostic et sync service, service notifications relais, contrat par défaut

**Root causes:**
- N/A (évolutions + correctifs ciblés)

**Correctifs:**
- api-relay: GET /:hash (keys, messages, signatures) rejette hash vide → 400
- service-login-verify: validation structure preuve (challenge.hash, nonce, timestamp, signatures), reason invalid_proof_structure

**Evolutions:**
- RESTE_A_FAIRE.md: vue d’ensemble et tâches par projet
- UserWallet: DiagnosticScreen, ServiceSyncScreen, relayNotificationService (hash events, fetch, decrypt, graph), defaultContract, loginStateMachine, useChannel, loginPublish, LoginScreen, LoginCollectShare
- website-skeleton: README étendu

**Pages affectées:**
- RESTE_A_FAIRE.md
- api-relay: keys, messages, signatures
- service-login-verify: types, verifyLoginProof
- userwallet: App, DiagnosticScreen, LoginCollectShare, LoginScreen, ServiceSyncScreen, useChannel, loginStateMachine, relayNotificationService, defaultContract, loginPublish
- website-skeleton: README
2026-01-28 01:37:16 +01:00
ncantu
0960e43a45 Optimisation mémoire api-anchorage avec base de données SQLite
**Motivations:**
- Réduction drastique de la consommation mémoire lors des ancrages
- Élimination du chargement de 173k+ UTXOs à chaque requête
- Stabilisation de la mémoire système sous charge élevée (50+ ancrages/minute)

**Root causes:**
- api-anchorage chargeait tous les UTXOs (173k+) via listunspent RPC à chaque ancrage
- Filtrage et tri de 173k+ objets en mémoire pour sélectionner un seul UTXO
- Croissance mémoire de ~16 MB toutes les 12 secondes avec 50 ancrages/minute
- Saturation mémoire système en quelques minutes

**Correctifs:**
- Création du module database.js pour gérer la base de données SQLite partagée
- Remplacement de listunspent RPC par requête SQL directe avec LIMIT 1
- Sélection directe d'un UTXO depuis la DB au lieu de charger/filtrer 173k+ objets
- Marquage des UTXOs comme dépensés dans la DB après utilisation
- Fermeture propre de la base de données lors de l'arrêt

**Evolutions:**
- Utilisation de la base de données SQLite partagée avec signet-dashboard
- Réduction mémoire de 99.999% (173k+ objets → 1 objet par requête)
- Amélioration des performances (requête SQL indexée vs filtrage en mémoire)
- Optimisation mémoire de signet-dashboard (chargement UTXOs seulement si nécessaire)
- Monitoring de lockedUtxos dans api-anchorage pour détecter les fuites
- Nettoyage des intervalles frontend pour éviter les fuites mémoire

**Pages affectées:**
- api-anchorage/src/database.js (nouveau)
- api-anchorage/src/bitcoin-rpc.js
- api-anchorage/src/server.js
- api-anchorage/package.json
- signet-dashboard/src/bitcoin-rpc.js
- signet-dashboard/public/app.js
- features/optimisation-memoire-applications.md (nouveau)
- features/api-anchorage-optimisation-base-donnees.md (nouveau)
2026-01-27 21:12:22 +01:00
ncantu
5de870fa13 Update login state machine and add new login components
**Motivations:**
- Continue implementation of login state machine
- Add new login components for collect, share and sign screens
- Add utilities for login signing, validation, and publishing
- Update documentation for CNIL compliance and login workflow
- Add features for timeouts, backoff, and validator acceptance

**Root causes:**
- N/A (feature additions and improvements)

**Correctifs:**
- Update login state machine implementation
- Update graph resolver and sync service

**Evolutions:**
- Add LoginCollectShare and LoginSignScreen components
- Add useSignAndPostLogin hook
- Add login utilities: loginSign, loginPublish, loginValidation, collectSignatures
- Add sync utilities: syncUpdateGraph, syncValidate
- Add validatorsAccept utility
- Update App.tsx and LoginScreen.tsx
- Update loginBuilder and loginStateMachine services
- Add documentation for CNIL compliance, remote collection, timeouts, and validator acceptance
- Update identity types

**Pages affectées:**
- userwallet/src/App.tsx
- userwallet/src/components/LoginScreen.tsx
- userwallet/src/components/LoginCollectShare.tsx (new)
- userwallet/src/components/LoginSignScreen.tsx (new)
- userwallet/src/hooks/useSignAndPostLogin.ts (new)
- userwallet/src/services/graphResolver.ts
- userwallet/src/services/loginBuilder.ts
- userwallet/src/services/loginStateMachine.ts
- userwallet/src/services/syncService.ts
- userwallet/src/services/syncUpdateGraph.ts (new)
- userwallet/src/services/syncValidate.ts (new)
- userwallet/src/types/identity.ts
- userwallet/src/utils/relay.ts
- userwallet/src/utils/collectSignatures.ts (new)
- userwallet/src/utils/loginPublish.ts (new)
- userwallet/src/utils/loginSign.ts (new)
- userwallet/src/utils/loginValidation.ts (new)
- userwallet/src/utils/validatorsAccept.ts (new)
- userwallet/docs/synthese.md
- userwallet/docs/specs-champs-obligatoires-cnil.md
- userwallet/features/userwallet-acceptation-version-validateurs.md (new)
- userwallet/features/userwallet-dh-systematique-scan-fetch.md (new)
- features/userwallet-contrat-login-reste-a-faire.md
- features/userwallet-login-state-machine.md
- features/userwallet-validation-conformite.md
- features/userwallet-champs-obligatoires-cnil.md (new)
- features/userwallet-collecte-distante-2-devices.md (new)
- features/userwallet-timeouts-backoff.md (new)
- mempool (submodule)
- hash_list.txt
- hash_list_cache.txt
2026-01-26 15:04:07 +01:00
ncantu
c3c11f0ef0 Update userwallet components, services and documentation
**Motivations:**
- Add new features and fixes for userwallet application
- Update documentation for pairing, login state machine, and sync
- Add new utilities for bloom filters, nonce store, and contract versioning
- Fix mempool websocket offline issues

**Root causes:**
- N/A (feature additions and improvements)

**Correctifs:**
- Fix mempool websocket offline handling
- Update ESLint configuration

**Evolutions:**
- Add login state machine service and hook
- Add sync loop service
- Add bloom filter utilities for anti-replay and state visibility
- Add nonce store and contract version utilities
- Update pairing confirmation and graph resolver services
- Add new documentation for features and fixes
- Update userwallet components (LoginScreen, SyncScreen)
- Update types for contract, identity, and messages

**Pages affectées:**
- userwallet/src/components/LoginScreen.tsx
- userwallet/src/components/SyncScreen.tsx
- userwallet/src/hooks/useChannel.ts
- userwallet/src/hooks/useLoginStateMachine.ts (new)
- userwallet/src/services/graphResolver.ts
- userwallet/src/services/pairingConfirm.ts
- userwallet/src/services/syncService.ts
- userwallet/src/services/syncLoop.ts (new)
- userwallet/src/services/loginStateMachine.ts (new)
- userwallet/src/types/contract.ts
- userwallet/src/types/identity.ts
- userwallet/src/types/message.ts
- userwallet/src/utils/canonical.ts
- userwallet/src/utils/identity.ts
- userwallet/src/utils/indexedDbStorage.ts
- userwallet/src/utils/relay.ts
- userwallet/src/utils/verification.ts
- userwallet/src/utils/bloom.ts (new)
- userwallet/src/utils/contractVersion.ts (new)
- userwallet/src/utils/nonceStore.ts (new)
- userwallet/eslint.config.mjs
- userwallet/package.json
- userwallet/package-lock.json
- userwallet/docs/synthese.md
- userwallet/docs/specs-champs-obligatoires-cnil.md (new)
- api-relay/README.md
- features/userwallet-pairing-words-only-finalise.md
- features/userwallet-anti-rejeu-etats-visibles-bloom.md (new)
- features/userwallet-bloom-usage-sync.md (new)
- features/userwallet-contrat-login-reste-a-faire.md (new)
- features/userwallet-ecrans-login-a-valider.md (new)
- features/userwallet-eslint-fix.md (new)
- features/userwallet-login-state-machine.md (new)
- features/userwallet-validation-conformite.md (new)
- fixKnowledge/mempool-websocket-offline-fix.md (new)
- mempool (submodule)
- hash_list.txt
- hash_list_cache.txt
2026-01-26 14:00:32 +01:00
ncantu
5689693507 UserWallet: Multiple feature implementations and updates
**Motivations:**
- Implement BIP39 mnemonic import for identity creation
- Add password-based key protection for enhanced security
- Improve pairing workflow with QR code and URL display
- Migrate hash cache from LocalStorage to IndexedDB for better scalability
- Update signet-dashboard and mempool components

**Root causes:**
- N/A (feature implementations)

**Correctifs:**
- N/A (no bug fixes in this commit)

**Evolutions:**
- BIP39 mnemonic import: Support for 12/24 word English mnemonics with BIP32 derivation path m/44'/0'/0'/0/0
- Key protection: Password-based encryption of private keys at rest with unlock/lock functionality
- Pairing workflow: QR code and URL display for device pairing, form-based word exchange between devices
- IndexedDB migration: Hash cache moved from LocalStorage to IndexedDB to avoid size limitations
- Global action bar: URL parameter support for navigation
- Pairing connection: Enhanced pairing status management

**Pages affectées:**
- userwallet/src/utils/identity.ts
- userwallet/src/utils/keyProtection.ts
- userwallet/src/utils/sessionUnlockedKey.ts
- userwallet/src/utils/indexedDbStorage.ts
- userwallet/src/utils/cache.ts
- userwallet/src/utils/pairing.ts
- userwallet/src/components/UnlockScreen.tsx
- userwallet/src/components/PairingDisplayScreen.tsx
- userwallet/src/components/PairingSetupBlock.tsx
- userwallet/src/components/GlobalActionBar.tsx
- userwallet/src/components/HomeScreen.tsx
- userwallet/src/components/ImportIdentityScreen.tsx
- userwallet/src/components/DataExportImportScreen.tsx
- userwallet/src/hooks/useIdentity.ts
- userwallet/src/hooks/usePairingConnected.ts
- userwallet/src/services/syncService.ts
- userwallet/src/services/pairingConfirm.ts
- userwallet/src/App.tsx
- userwallet/package.json
- userwallet/docs/specs.md
- userwallet/docs/storage.md
- userwallet/docs/synthese.md
- signet-dashboard/public/*.html
- signet-dashboard/public/app.js
- signet-dashboard/public/styles.css
- mempool (submodule updates)
- hash_list.txt, hash_list_cache.txt, utxo_list.txt, utxo_list_cache.txt, fees_list.txt
- features/*.md (documentation files)
2026-01-26 10:23:34 +01:00
ncantu
cad73cb265 UTXO-list: dates/blockTime historiques, récupération frais depuis ancrages, diagnostic Bloc Rewards
**Motivations:**
- Ajouter dates manquantes dans hash_list.txt et compléter historique
- Compléter blockTime manquants dans utxo_list.txt et compléter historique
- Récupérer frais depuis transactions d'ancrage (OP_RETURN) et les stocker
- Bouton UI pour déclencher récupération frais
- Diagnostic Bloc Rewards (pourquoi ~4700 BTC au lieu de 50 BTC)

**Root causes:**
- hash_list.txt sans date (format ancien)
- utxo_list.txt blockTime souvent vide
- Frais absents du fichier (métadonnées OP_RETURN non stockées)
- Pas de moyen de récupérer/compléter frais depuis UI

**Correctifs:**
- hash_list.txt : format étendu avec date (rétrocompatible)
- utxo_list.txt : blockTime complété automatiquement lors écritures
- fees_list.txt : nouveau fichier pour stocker frais
- updateFeesFromAnchors() : récupère frais depuis OP_RETURN ancrages
- Endpoint /api/utxo/fees/update pour déclencher récupération
- Bouton "Récupérer les frais depuis les ancrages" dans section Frais (spinner)
- Scripts batch : complete-hash-list-dates.js, complete-utxo-list-blocktime.js
- Script diagnostic : diagnose-bloc-rewards.js (subsidy, coinbase, listunspent)

**Evolutions:**
- Frais chargés depuis fees_list.txt dans getUtxoList
- Complétion automatique dates/blockTime lors écritures futures

**Pages affectées:**
- signet-dashboard/src/bitcoin-rpc.js
- signet-dashboard/src/server.js
- signet-dashboard/public/utxo-list.html
- scripts/complete-hash-list-dates.js
- scripts/complete-utxo-list-blocktime.js
- scripts/diagnose-bloc-rewards.js
- features/utxo-list-fees-update-and-historical-completion.md
2026-01-26 01:59:46 +01:00