api-anchorage: combine multiple UTXOs, api-relay: storage interface refactor

**Motivations:**
- Résoudre erreur "No UTXO large enough" en combinant plusieurs petits UTXOs
- Refactorer api-relay avec interface StorageServiceInterface pour meilleure abstraction
- Ajouter script restart-bitcoind.sh

**Root causes:**
- api-anchorage: logique cherchait uniquement un UTXO assez grand, ne combinait pas plusieurs petits UTXOs
- api-relay: code dupliqué entre StorageService et DatabaseStorageService

**Correctifs:**
- api-anchorage: combinaison automatique de plusieurs petits UTXOs si aucun assez grand, ajustement frais pour inputs multiples, limite 20 UTXOs, gestion cohérente verrouillage/déverrouillage

**Evolutions:**
- api-relay: StorageServiceInterface (abstraction commune), refactoring routes (keys, messages, signatures, metrics, bloom) pour utiliser interface, eslint config, package updates
- fixKnowledge: api-anchorage-combine-utxos.md (documentation correction)
- restart-bitcoind.sh: script redémarrage bitcoind

**Pages affectées:**
- api-anchorage: bitcoin-rpc.js
- api-relay: eslint.config.mjs, package.json, index.ts, middleware/auth.ts, routes (bloom, keys, messages, metrics, signatures), services (apiKeyService, database, relay, storageAdapter, storageInterface)
- fixKnowledge: api-anchorage-combine-utxos.md
- restart-bitcoind.sh
This commit is contained in:
ncantu 2026-01-28 07:50:56 +01:00
parent 695aff4f85
commit 3c212e56e9
18 changed files with 835 additions and 201 deletions

View File

@ -294,50 +294,101 @@ class BitcoinRPC {
`); `);
let utxoFromDb = utxoQuery.get(totalNeeded); let utxoFromDb = utxoQuery.get(totalNeeded);
// Si aucun UTXO assez grand, essayer de combiner plusieurs petits UTXOs
let selectedUtxos = [];
let totalSelectedAmount = 0;
let estimatedFeeForMultipleInputs = estimatedFee;
if (!utxoFromDb) { if (!utxoFromDb) {
// Si aucun UTXO trouvé avec le montant requis, essayer de trouver le plus grand disponible // Chercher plusieurs petits UTXOs dont la somme est suffisante
const largestUtxoQuery = db.prepare(` const combineUtxosQuery = db.prepare(`
SELECT txid, vout, address, amount, confirmations, block_time SELECT txid, vout, address, amount, confirmations, block_time
FROM utxos FROM utxos
WHERE confirmations > 0 WHERE confirmations > 0
AND is_spent_onchain = 0 AND is_spent_onchain = 0
AND is_locked_in_mutex = 0 AND is_locked_in_mutex = 0
ORDER BY amount DESC ORDER BY amount DESC
LIMIT 1
`); `);
const largestUtxo = largestUtxoQuery.get(); const availableUtxos = combineUtxosQuery.all();
if (!largestUtxo) { if (availableUtxos.length === 0) {
throw new Error('No available UTXOs in database (all are locked, spent, or unconfirmed)'); throw new Error('No available UTXOs in database (all are locked, spent, or unconfirmed)');
} }
throw new Error( // Calculer le montant total nécessaire avec une marge pour les frais supplémentaires
`No UTXO large enough for anchor with provisioning. Required: ${totalNeeded} BTC, ` + // (combiner plusieurs UTXOs augmente la taille de la transaction)
`Largest available: ${largestUtxo.amount} BTC` // Estimation: ~148 bytes par input supplémentaire
); const estimatedBytesPerInput = 148;
} const estimatedFeePerInput = 0.0000001; // Conservateur
const maxUtxosToCombine = 20; // Limite pour éviter des transactions trop grandes
estimatedFeeForMultipleInputs = estimatedFee;
// Convertir l'UTXO de la DB au format attendu // Sélectionner les UTXOs jusqu'à atteindre le montant nécessaire
selectedUtxo = { for (let i = 0; i < availableUtxos.length && i < maxUtxosToCombine; i++) {
txid: utxoFromDb.txid, const utxo = availableUtxos[i];
vout: utxoFromDb.vout, if (totalSelectedAmount >= totalNeeded + estimatedFeeForMultipleInputs) {
address: utxoFromDb.address || '', break;
amount: utxoFromDb.amount, }
confirmations: utxoFromDb.confirmations || 0, selectedUtxos.push({
blockTime: utxoFromDb.block_time, txid: utxo.txid,
}; vout: utxo.vout,
address: utxo.address || '',
amount: utxo.amount,
confirmations: utxo.confirmations || 0,
blockTime: utxo.block_time,
});
totalSelectedAmount += utxo.amount;
// Ajuster l'estimation des frais pour chaque input supplémentaire
if (selectedUtxos.length > 1) {
estimatedFeeForMultipleInputs += estimatedFeePerInput;
}
}
logger.info('Selected UTXO from database', { if (totalSelectedAmount < totalNeeded + estimatedFeeForMultipleInputs) {
txid: selectedUtxo.txid.substring(0, 16) + '...', const largestUtxo = availableUtxos[0];
vout: selectedUtxo.vout, throw new Error(
amount: selectedUtxo.amount, `No UTXO large enough for anchor with provisioning. Required: ${totalNeeded.toFixed(8)} BTC, ` +
confirmations: selectedUtxo.confirmations, `Largest available: ${largestUtxo.amount} BTC. ` +
totalNeeded, `Total from ${selectedUtxos.length} UTXOs: ${totalSelectedAmount.toFixed(8)} BTC`
}); );
}
logger.info('Combining multiple UTXOs for anchor transaction', {
numberOfUtxos: selectedUtxos.length,
totalAmount: totalSelectedAmount,
totalNeeded: totalNeeded + estimatedFeeForMultipleInputs,
});
// Verrouiller l'UTXO sélectionné // Verrouiller tous les UTXOs sélectionnés
this.lockUtxo(selectedUtxo.txid, selectedUtxo.vout); for (const utxo of selectedUtxos) {
this.lockUtxo(utxo.txid, utxo.vout);
}
// Utiliser le premier UTXO comme référence pour la compatibilité avec le code existant
selectedUtxo = selectedUtxos[0];
} else {
// Un seul UTXO assez grand trouvé
selectedUtxos = [{
txid: utxoFromDb.txid,
vout: utxoFromDb.vout,
address: utxoFromDb.address || '',
amount: utxoFromDb.amount,
confirmations: utxoFromDb.confirmations || 0,
blockTime: utxoFromDb.block_time,
}];
totalSelectedAmount = utxoFromDb.amount;
selectedUtxo = selectedUtxos[0];
logger.info('Selected UTXO from database', {
txid: selectedUtxo.txid.substring(0, 16) + '...',
vout: selectedUtxo.vout,
amount: selectedUtxo.amount,
confirmations: selectedUtxo.confirmations,
totalNeeded,
});
// Verrouiller l'UTXO sélectionné
this.lockUtxo(selectedUtxo.txid, selectedUtxo.vout);
// Créer les outputs // Créer les outputs
// Note: Bitcoin Core ne permet qu'un seul OP_RETURN par transaction via 'data' // Note: Bitcoin Core ne permet qu'un seul OP_RETURN par transaction via 'data'
@ -357,7 +408,11 @@ class BitcoinRPC {
} }
// Calculer le change (arrondi à 8 décimales) // Calculer le change (arrondi à 8 décimales)
const change = roundTo8Decimals(selectedUtxo.amount - totalOutputAmount - estimatedFee); // Utiliser totalSelectedAmount si plusieurs UTXOs sont combinés
const totalInputAmount = selectedUtxos.length > 1 ? totalSelectedAmount : selectedUtxo.amount;
// Ajuster les frais si plusieurs inputs
const finalEstimatedFee = selectedUtxos.length > 1 ? estimatedFeeForMultipleInputs : estimatedFee;
const change = roundTo8Decimals(totalInputAmount - totalOutputAmount - finalEstimatedFee);
let changeAddress = null; let changeAddress = null;
if (change > 0.00001) { if (change > 0.00001) {
changeAddress = await this.getNewAddress(); changeAddress = await this.getNewAddress();
@ -401,59 +456,74 @@ class BitcoinRPC {
totalSize: combinedData.length, totalSize: combinedData.length,
}); });
// Vérifier que l'UTXO est toujours disponible avant de l'utiliser // Vérifier que tous les UTXOs sont toujours disponibles avant de les utiliser
// (peut avoir été dépensé entre la sélection et l'utilisation) // (peut avoir été dépensés entre la sélection et l'utilisation)
// Limiter les tentatives pour éviter les boucles infinies // Limiter les tentatives pour éviter les boucles infinies
if (retryCount < 3) { if (retryCount < 3) {
try { try {
const utxoCheck = await this.client.listunspent(0, 9999999, [selectedUtxo.address]); // Récupérer toutes les adresses uniques des UTXOs sélectionnés
const utxoStillAvailable = utxoCheck.some(u => const uniqueAddresses = [...new Set(selectedUtxos.map(u => u.address))];
u.txid === selectedUtxo.txid && u.vout === selectedUtxo.vout const utxoCheck = await this.client.listunspent(0, 9999999, uniqueAddresses);
);
if (!utxoStillAvailable) { // Vérifier que tous les UTXOs sont toujours disponibles
// L'UTXO n'est plus disponible, le marquer comme dépensé et réessayer let allUtxosAvailable = true;
logger.warn('Selected UTXO no longer available, marking as spent and retrying', { for (const utxo of selectedUtxos) {
txid: selectedUtxo.txid.substring(0, 16) + '...', const utxoStillAvailable = utxoCheck.some(u =>
vout: selectedUtxo.vout, u.txid === utxo.txid && u.vout === utxo.vout
);
if (!utxoStillAvailable) {
allUtxosAvailable = false;
logger.warn('Selected UTXO no longer available, marking as spent', {
txid: utxo.txid.substring(0, 16) + '...',
vout: utxo.vout,
});
// Marquer l'UTXO comme dépensé dans la DB
try {
const dbForUpdate = getDatabase();
dbForUpdate.prepare(`
UPDATE utxos
SET is_spent_onchain = 1, is_locked_in_mutex = 0, updated_at = CURRENT_TIMESTAMP
WHERE txid = ? AND vout = ?
`).run(utxo.txid, utxo.vout);
} catch (dbError) {
logger.warn('Error updating UTXO in database', { error: dbError.message });
}
}
}
if (!allUtxosAvailable) {
// Au moins un UTXO n'est plus disponible, déverrouiller tous et réessayer
logger.warn('Some UTXOs no longer available, unlocking all and retrying', {
retryCount, retryCount,
numberOfUtxos: selectedUtxos.length,
}); });
// Déverrouiller l'UTXO avant de le marquer comme dépensé // Déverrouiller tous les UTXOs
this.unlockUtxo(selectedUtxo.txid, selectedUtxo.vout); for (const utxo of selectedUtxos) {
this.unlockUtxo(utxo.txid, utxo.vout);
try {
const dbForUpdate = getDatabase();
dbForUpdate.prepare(`
UPDATE utxos
SET is_spent_onchain = 1, is_locked_in_mutex = 0, updated_at = CURRENT_TIMESTAMP
WHERE txid = ? AND vout = ?
`).run(selectedUtxo.txid, selectedUtxo.vout);
} catch (dbError) {
logger.warn('Error updating UTXO in database', { error: dbError.message });
} }
// Réessayer avec un autre UTXO (récursion limitée à 3 tentatives) // Réessayer (récursion limitée à 3 tentatives)
return this.createAnchorTransaction(hash, recipientAddress, provisioningAddresses, numberOfProvisioningUtxos, retryCount + 1); return this.createAnchorTransaction(hash, recipientAddress, provisioningAddresses, numberOfProvisioningUtxos, retryCount + 1);
} }
} catch (checkError) { } catch (checkError) {
logger.warn('Error checking UTXO availability, proceeding anyway', { logger.warn('Error checking UTXO availability, proceeding anyway', {
error: checkError.message, error: checkError.message,
txid: selectedUtxo.txid.substring(0, 16) + '...', numberOfUtxos: selectedUtxos.length,
vout: selectedUtxo.vout,
}); });
// Continuer même si la vérification échoue (peut être un problème réseau temporaire) // Continuer même si la vérification échoue (peut être un problème réseau temporaire)
} }
} else { } else {
logger.error('Max retry count reached for UTXO selection', { retryCount }); logger.error('Max retry count reached for UTXO selection', { retryCount });
throw new Error('Failed to find available UTXO after multiple attempts'); throw new Error('Failed to find available UTXOs after multiple attempts');
} }
// Créer la transaction // Créer la transaction avec tous les UTXOs sélectionnés
const inputs = [{ const inputs = selectedUtxos.map(utxo => ({
txid: selectedUtxo.txid, txid: utxo.txid,
vout: selectedUtxo.vout, vout: utxo.vout,
}]; }));
let tx; let tx;
try { try {
@ -464,17 +534,20 @@ class BitcoinRPC {
txid: selectedUtxo.txid.substring(0, 16) + '...', txid: selectedUtxo.txid.substring(0, 16) + '...',
vout: selectedUtxo.vout, vout: selectedUtxo.vout,
}); });
// Marquer l'UTXO comme dépensé si l'erreur suggère qu'il n'existe plus // Marquer tous les UTXOs comme dépensés si l'erreur suggère qu'ils n'existent plus
if (error.message.includes('not found') || error.message.includes('does not exist')) { if (error.message.includes('not found') || error.message.includes('does not exist')) {
try { try {
const dbForUpdate = getDatabase(); const dbForUpdate = getDatabase();
dbForUpdate.prepare(` const updateStmt = dbForUpdate.prepare(`
UPDATE utxos UPDATE utxos
SET is_spent_onchain = 1, updated_at = CURRENT_TIMESTAMP SET is_spent_onchain = 1, updated_at = CURRENT_TIMESTAMP
WHERE txid = ? AND vout = ? WHERE txid = ? AND vout = ?
`).run(selectedUtxo.txid, selectedUtxo.vout); `);
for (const utxo of selectedUtxos) {
updateStmt.run(utxo.txid, utxo.vout);
}
} catch (dbError) { } catch (dbError) {
logger.warn('Error updating UTXO in database', { error: dbError.message }); logger.warn('Error updating UTXOs in database', { error: dbError.message });
} }
} }
throw new Error(`Failed to create transaction: ${error.message}`); throw new Error(`Failed to create transaction: ${error.message}`);
@ -513,14 +586,16 @@ class BitcoinRPC {
if (hasUtxoNotFoundError) { if (hasUtxoNotFoundError) {
try { try {
const dbForUpdate = getDatabase(); const dbForUpdate = getDatabase();
dbForUpdate.prepare(` const updateStmt = dbForUpdate.prepare(`
UPDATE utxos UPDATE utxos
SET is_spent_onchain = 1, is_locked_in_mutex = 0, updated_at = CURRENT_TIMESTAMP SET is_spent_onchain = 1, is_locked_in_mutex = 0, updated_at = CURRENT_TIMESTAMP
WHERE txid = ? AND vout = ? WHERE txid = ? AND vout = ?
`).run(selectedUtxo.txid, selectedUtxo.vout); `);
logger.info('UTXO marked as spent due to signing error', { for (const utxo of selectedUtxos) {
txid: selectedUtxo.txid.substring(0, 16) + '...', updateStmt.run(utxo.txid, utxo.vout);
vout: selectedUtxo.vout, }
logger.info('UTXOs marked as spent due to signing error', {
numberOfUtxos: selectedUtxos.length,
error: errorMessages, error: errorMessages,
}); });
} catch (dbError) { } catch (dbError) {
@ -632,16 +707,20 @@ class BitcoinRPC {
} }
} }
// Marquer l'UTXO comme dépensé dans la base de données // Marquer tous les UTXOs comme dépensés dans la base de données
// L'UTXO est dépensé dans une transaction (mempool), mais pas encore confirmé dans un bloc // Les UTXOs sont dépensés dans une transaction (mempool), mais pas encore confirmés dans un bloc
try { try {
const dbForUpdate = getDatabase(); const dbForUpdate = getDatabase();
dbForUpdate.prepare(` const updateStmt = dbForUpdate.prepare(`
UPDATE utxos UPDATE utxos
SET is_spent_onchain = 1, is_locked_in_mutex = 0, updated_at = CURRENT_TIMESTAMP SET is_spent_onchain = 1, is_locked_in_mutex = 0, updated_at = CURRENT_TIMESTAMP
WHERE txid = ? AND vout = ? WHERE txid = ? AND vout = ?
`).run(selectedUtxo.txid, selectedUtxo.vout); `);
logger.debug('UTXO marked as spent in database', { for (const utxo of selectedUtxos) {
updateStmt.run(utxo.txid, utxo.vout);
}
logger.debug('UTXOs marked as spent in database', {
numberOfUtxos: selectedUtxos.length,
txid: selectedUtxo.txid.substring(0, 16) + '...', txid: selectedUtxo.txid.substring(0, 16) + '...',
vout: selectedUtxo.vout, vout: selectedUtxo.vout,
}); });
@ -653,9 +732,11 @@ class BitcoinRPC {
}); });
} }
// Déverrouiller l'UTXO maintenant que la transaction est dans le mempool // Déverrouiller tous les UTXOs maintenant que la transaction est dans le mempool
// (mise à jour DB déjà faite ci-dessus, mais on déverrouille aussi en mémoire) // (mise à jour DB déjà faite ci-dessus, mais on déverrouille aussi en mémoire)
this.unlockUtxo(selectedUtxo.txid, selectedUtxo.vout); for (const utxo of selectedUtxos) {
this.unlockUtxo(utxo.txid, utxo.vout);
}
// Libérer le mutex // Libérer le mutex
releaseMutex(); releaseMutex();
@ -675,10 +756,12 @@ class BitcoinRPC {
hash: hash?.substring(0, 16) + '...', hash: hash?.substring(0, 16) + '...',
}); });
// En cas d'erreur, déverrouiller l'UTXO et libérer le mutex // En cas d'erreur, déverrouiller tous les UTXOs et libérer le mutex
if (selectedUtxo) { if (selectedUtxos && selectedUtxos.length > 0) {
// Déverrouiller l'UTXO (mise à jour DB + mémoire) // Déverrouiller tous les UTXOs (mise à jour DB + mémoire)
this.unlockUtxo(selectedUtxo.txid, selectedUtxo.vout); for (const utxo of selectedUtxos) {
this.unlockUtxo(utxo.txid, utxo.vout);
}
} }
releaseMutex(); releaseMutex();

View File

@ -10,6 +10,11 @@ export default tseslint.config(
...tseslint.configs.recommended, ...tseslint.configs.recommended,
{ {
files: ['**/*.ts'], files: ['**/*.ts'],
languageOptions: {
parserOptions: {
project: './tsconfig.json',
},
},
plugins: { plugins: {
'unused-imports': unusedImports, 'unused-imports': unusedImports,
}, },

View File

@ -29,7 +29,8 @@
"eslint": "^8.55.0", "eslint": "^8.55.0",
"eslint-plugin-unused-imports": "^3.0.0", "eslint-plugin-unused-imports": "^3.0.0",
"tsx": "^4.7.0", "tsx": "^4.7.0",
"typescript": "^5.2.2" "typescript": "^5.2.2",
"typescript-eslint": "^8.54.0"
} }
}, },
"node_modules/@esbuild/aix-ppc64": { "node_modules/@esbuild/aix-ppc64": {
@ -913,6 +914,42 @@
} }
} }
}, },
"node_modules/@typescript-eslint/project-service": {
"version": "8.54.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/project-service/-/project-service-8.54.0.tgz",
"integrity": "sha512-YPf+rvJ1s7MyiWM4uTRhE4DvBXrEV+d8oC3P9Y2eT7S+HBS0clybdMIPnhiATi9vZOYDc7OQ1L/i6ga6NFYK/g==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/tsconfig-utils": "^8.54.0",
"@typescript-eslint/types": "^8.54.0",
"debug": "^4.4.3"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/typescript-eslint"
},
"peerDependencies": {
"typescript": ">=4.8.4 <6.0.0"
}
},
"node_modules/@typescript-eslint/project-service/node_modules/@typescript-eslint/types": {
"version": "8.54.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.54.0.tgz",
"integrity": "sha512-PDUI9R1BVjqu7AUDsRBbKMtwmjWcn4J3le+5LpcFgWULN3LvHC5rkc9gCVxbrsrGmO1jfPybN5s6h4Jy+OnkAA==",
"dev": true,
"license": "MIT",
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/typescript-eslint"
}
},
"node_modules/@typescript-eslint/scope-manager": { "node_modules/@typescript-eslint/scope-manager": {
"version": "6.21.0", "version": "6.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-6.21.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-6.21.0.tgz",
@ -931,6 +968,23 @@
"url": "https://opencollective.com/typescript-eslint" "url": "https://opencollective.com/typescript-eslint"
} }
}, },
"node_modules/@typescript-eslint/tsconfig-utils": {
"version": "8.54.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/tsconfig-utils/-/tsconfig-utils-8.54.0.tgz",
"integrity": "sha512-dRgOyT2hPk/JwxNMZDsIXDgyl9axdJI3ogZ2XWhBPsnZUv+hPesa5iuhdYt2gzwA9t8RE5ytOJ6xB0moV0Ujvw==",
"dev": true,
"license": "MIT",
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/typescript-eslint"
},
"peerDependencies": {
"typescript": ">=4.8.4 <6.0.0"
}
},
"node_modules/@typescript-eslint/type-utils": { "node_modules/@typescript-eslint/type-utils": {
"version": "6.21.0", "version": "6.21.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/type-utils/-/type-utils-6.21.0.tgz", "resolved": "https://registry.npmjs.org/@typescript-eslint/type-utils/-/type-utils-6.21.0.tgz",
@ -3923,6 +3977,54 @@
"node": ">=20" "node": ">=20"
} }
}, },
"node_modules/tinyglobby": {
"version": "0.2.15",
"resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.15.tgz",
"integrity": "sha512-j2Zq4NyQYG5XMST4cbs02Ak8iJUdxRM0XI5QyxXuZOzKOINmWurp3smXu3y5wDcJrptwpSjgXHzIQxR0omXljQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"fdir": "^6.5.0",
"picomatch": "^4.0.3"
},
"engines": {
"node": ">=12.0.0"
},
"funding": {
"url": "https://github.com/sponsors/SuperchupuDev"
}
},
"node_modules/tinyglobby/node_modules/fdir": {
"version": "6.5.0",
"resolved": "https://registry.npmjs.org/fdir/-/fdir-6.5.0.tgz",
"integrity": "sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=12.0.0"
},
"peerDependencies": {
"picomatch": "^3 || ^4"
},
"peerDependenciesMeta": {
"picomatch": {
"optional": true
}
}
},
"node_modules/tinyglobby/node_modules/picomatch": {
"version": "4.0.3",
"resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz",
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=12"
},
"funding": {
"url": "https://github.com/sponsors/jonschlinkert"
}
},
"node_modules/to-regex-range": { "node_modules/to-regex-range": {
"version": "5.0.1", "version": "5.0.1",
"resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz", "resolved": "https://registry.npmjs.org/to-regex-range/-/to-regex-range-5.0.1.tgz",
@ -4043,6 +4145,263 @@
"node": ">=14.17" "node": ">=14.17"
} }
}, },
"node_modules/typescript-eslint": {
"version": "8.54.0",
"resolved": "https://registry.npmjs.org/typescript-eslint/-/typescript-eslint-8.54.0.tgz",
"integrity": "sha512-CKsJ+g53QpsNPqbzUsfKVgd3Lny4yKZ1pP4qN3jdMOg/sisIDLGyDMezycquXLE5JsEU0wp3dGNdzig0/fmSVQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/eslint-plugin": "8.54.0",
"@typescript-eslint/parser": "8.54.0",
"@typescript-eslint/typescript-estree": "8.54.0",
"@typescript-eslint/utils": "8.54.0"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/typescript-eslint"
},
"peerDependencies": {
"eslint": "^8.57.0 || ^9.0.0",
"typescript": ">=4.8.4 <6.0.0"
}
},
"node_modules/typescript-eslint/node_modules/@typescript-eslint/eslint-plugin": {
"version": "8.54.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/eslint-plugin/-/eslint-plugin-8.54.0.tgz",
"integrity": "sha512-hAAP5io/7csFStuOmR782YmTthKBJ9ND3WVL60hcOjvtGFb+HJxH4O5huAcmcZ9v9G8P+JETiZ/G1B8MALnWZQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"@eslint-community/regexpp": "^4.12.2",
"@typescript-eslint/scope-manager": "8.54.0",
"@typescript-eslint/type-utils": "8.54.0",
"@typescript-eslint/utils": "8.54.0",
"@typescript-eslint/visitor-keys": "8.54.0",
"ignore": "^7.0.5",
"natural-compare": "^1.4.0",
"ts-api-utils": "^2.4.0"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/typescript-eslint"
},
"peerDependencies": {
"@typescript-eslint/parser": "^8.54.0",
"eslint": "^8.57.0 || ^9.0.0",
"typescript": ">=4.8.4 <6.0.0"
}
},
"node_modules/typescript-eslint/node_modules/@typescript-eslint/parser": {
"version": "8.54.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/parser/-/parser-8.54.0.tgz",
"integrity": "sha512-BtE0k6cjwjLZoZixN0t5AKP0kSzlGu7FctRXYuPAm//aaiZhmfq1JwdYpYr1brzEspYyFeF+8XF5j2VK6oalrA==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/scope-manager": "8.54.0",
"@typescript-eslint/types": "8.54.0",
"@typescript-eslint/typescript-estree": "8.54.0",
"@typescript-eslint/visitor-keys": "8.54.0",
"debug": "^4.4.3"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/typescript-eslint"
},
"peerDependencies": {
"eslint": "^8.57.0 || ^9.0.0",
"typescript": ">=4.8.4 <6.0.0"
}
},
"node_modules/typescript-eslint/node_modules/@typescript-eslint/scope-manager": {
"version": "8.54.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/scope-manager/-/scope-manager-8.54.0.tgz",
"integrity": "sha512-27rYVQku26j/PbHYcVfRPonmOlVI6gihHtXFbTdB5sb6qA0wdAQAbyXFVarQ5t4HRojIz64IV90YtsjQSSGlQg==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/types": "8.54.0",
"@typescript-eslint/visitor-keys": "8.54.0"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/typescript-eslint"
}
},
"node_modules/typescript-eslint/node_modules/@typescript-eslint/type-utils": {
"version": "8.54.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/type-utils/-/type-utils-8.54.0.tgz",
"integrity": "sha512-hiLguxJWHjjwL6xMBwD903ciAwd7DmK30Y9Axs/etOkftC3ZNN9K44IuRD/EB08amu+Zw6W37x9RecLkOo3pMA==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/types": "8.54.0",
"@typescript-eslint/typescript-estree": "8.54.0",
"@typescript-eslint/utils": "8.54.0",
"debug": "^4.4.3",
"ts-api-utils": "^2.4.0"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/typescript-eslint"
},
"peerDependencies": {
"eslint": "^8.57.0 || ^9.0.0",
"typescript": ">=4.8.4 <6.0.0"
}
},
"node_modules/typescript-eslint/node_modules/@typescript-eslint/types": {
"version": "8.54.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/types/-/types-8.54.0.tgz",
"integrity": "sha512-PDUI9R1BVjqu7AUDsRBbKMtwmjWcn4J3le+5LpcFgWULN3LvHC5rkc9gCVxbrsrGmO1jfPybN5s6h4Jy+OnkAA==",
"dev": true,
"license": "MIT",
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/typescript-eslint"
}
},
"node_modules/typescript-eslint/node_modules/@typescript-eslint/typescript-estree": {
"version": "8.54.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/typescript-estree/-/typescript-estree-8.54.0.tgz",
"integrity": "sha512-BUwcskRaPvTk6fzVWgDPdUndLjB87KYDrN5EYGetnktoeAvPtO4ONHlAZDnj5VFnUANg0Sjm7j4usBlnoVMHwA==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/project-service": "8.54.0",
"@typescript-eslint/tsconfig-utils": "8.54.0",
"@typescript-eslint/types": "8.54.0",
"@typescript-eslint/visitor-keys": "8.54.0",
"debug": "^4.4.3",
"minimatch": "^9.0.5",
"semver": "^7.7.3",
"tinyglobby": "^0.2.15",
"ts-api-utils": "^2.4.0"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/typescript-eslint"
},
"peerDependencies": {
"typescript": ">=4.8.4 <6.0.0"
}
},
"node_modules/typescript-eslint/node_modules/@typescript-eslint/utils": {
"version": "8.54.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/utils/-/utils-8.54.0.tgz",
"integrity": "sha512-9Cnda8GS57AQakvRyG0PTejJNlA2xhvyNtEVIMlDWOOeEyBkYWhGPnfrIAnqxLMTSTo6q8g12XVjjev5l1NvMA==",
"dev": true,
"license": "MIT",
"dependencies": {
"@eslint-community/eslint-utils": "^4.9.1",
"@typescript-eslint/scope-manager": "8.54.0",
"@typescript-eslint/types": "8.54.0",
"@typescript-eslint/typescript-estree": "8.54.0"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/typescript-eslint"
},
"peerDependencies": {
"eslint": "^8.57.0 || ^9.0.0",
"typescript": ">=4.8.4 <6.0.0"
}
},
"node_modules/typescript-eslint/node_modules/@typescript-eslint/visitor-keys": {
"version": "8.54.0",
"resolved": "https://registry.npmjs.org/@typescript-eslint/visitor-keys/-/visitor-keys-8.54.0.tgz",
"integrity": "sha512-VFlhGSl4opC0bprJiItPQ1RfUhGDIBokcPwaFH4yiBCaNPeld/9VeXbiPO1cLyorQi1G1vL+ecBk1x8o1axORA==",
"dev": true,
"license": "MIT",
"dependencies": {
"@typescript-eslint/types": "8.54.0",
"eslint-visitor-keys": "^4.2.1"
},
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/typescript-eslint"
}
},
"node_modules/typescript-eslint/node_modules/eslint-visitor-keys": {
"version": "4.2.1",
"resolved": "https://registry.npmjs.org/eslint-visitor-keys/-/eslint-visitor-keys-4.2.1.tgz",
"integrity": "sha512-Uhdk5sfqcee/9H/rCOJikYz67o0a2Tw2hGRPOG2Y1R2dg7brRe1uG0yaNQDHu+TO/uQPF/5eCapvYSmHUjt7JQ==",
"dev": true,
"license": "Apache-2.0",
"engines": {
"node": "^18.18.0 || ^20.9.0 || >=21.1.0"
},
"funding": {
"url": "https://opencollective.com/eslint"
}
},
"node_modules/typescript-eslint/node_modules/ignore": {
"version": "7.0.5",
"resolved": "https://registry.npmjs.org/ignore/-/ignore-7.0.5.tgz",
"integrity": "sha512-Hs59xBNfUIunMFgWAbGX5cq6893IbWg4KnrjbYwX3tx0ztorVgTDA6B2sxf8ejHJ4wz8BqGUMYlnzNBer5NvGg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">= 4"
}
},
"node_modules/typescript-eslint/node_modules/minimatch": {
"version": "9.0.5",
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-9.0.5.tgz",
"integrity": "sha512-G6T0ZX48xgozx7587koeX9Ys2NYy6Gmv//P89sEte9V9whIapMNF4idKxnW2QtCcLiTWlb/wfCabAtAFWhhBow==",
"dev": true,
"license": "ISC",
"dependencies": {
"brace-expansion": "^2.0.1"
},
"engines": {
"node": ">=16 || 14 >=14.17"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
}
},
"node_modules/typescript-eslint/node_modules/ts-api-utils": {
"version": "2.4.0",
"resolved": "https://registry.npmjs.org/ts-api-utils/-/ts-api-utils-2.4.0.tgz",
"integrity": "sha512-3TaVTaAv2gTiMB35i3FiGJaRfwb3Pyn/j3m/bfAvGe8FB7CF6u+LMYqYlDh7reQf7UNvoTvdfAqHGmPGOSsPmA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=18.12"
},
"peerDependencies": {
"typescript": ">=4.8.4"
}
},
"node_modules/undici-types": { "node_modules/undici-types": {
"version": "6.21.0", "version": "6.21.0",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz", "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-6.21.0.tgz",

View File

@ -8,7 +8,7 @@
"dev": "tsx watch src/index.ts", "dev": "tsx watch src/index.ts",
"build": "tsc", "build": "tsc",
"start": "node dist/index.js", "start": "node dist/index.js",
"lint": "eslint . --ext ts --report-unused-disable-directives --max-warnings 0", "lint": "eslint . --report-unused-disable-directives --max-warnings 0",
"type-check": "tsc --noEmit", "type-check": "tsc --noEmit",
"migrate": "tsx src/scripts/migrate-to-db.ts" "migrate": "tsx src/scripts/migrate-to-db.ts"
}, },
@ -34,6 +34,7 @@
"eslint": "^8.55.0", "eslint": "^8.55.0",
"eslint-plugin-unused-imports": "^3.0.0", "eslint-plugin-unused-imports": "^3.0.0",
"tsx": "^4.7.0", "tsx": "^4.7.0",
"typescript": "^5.2.2" "typescript": "^5.2.2",
"typescript-eslint": "^8.54.0"
} }
} }

View File

@ -24,9 +24,7 @@ const STORAGE_PATH = process.env.STORAGE_PATH ?? './data';
const PEER_RELAYS = process.env.PEER_RELAYS const PEER_RELAYS = process.env.PEER_RELAYS
? process.env.PEER_RELAYS.split(',').map((p) => p.trim()).filter(Boolean) ? process.env.PEER_RELAYS.split(',').map((p) => p.trim()).filter(Boolean)
: []; : [];
const SAVE_INTERVAL_SECONDS = process.env.SAVE_INTERVAL_SECONDS const REQUIRE_API_KEY = process.env.REQUIRE_API_KEY !== 'false'; // Default: true
? parseInt(process.env.SAVE_INTERVAL_SECONDS, 10)
: 300;
async function main(): Promise<void> { async function main(): Promise<void> {
const app = express(); const app = express();

View File

@ -21,7 +21,7 @@ export function createAuthMiddleware(
let apiKey: string | undefined; let apiKey: string | undefined;
if (authHeader !== undefined && authHeader.startsWith('Bearer ')) { if (authHeader?.startsWith('Bearer ') === true) {
apiKey = authHeader.slice(7); apiKey = authHeader.slice(7);
} else if (apiKeyHeader !== undefined) { } else if (apiKeyHeader !== undefined) {
apiKey = apiKeyHeader; apiKey = apiKeyHeader;

View File

@ -1,11 +1,11 @@
import { Router, type Request, type Response } from 'express'; import { Router, type Request, type Response } from 'express';
import bloomFilters from 'bloom-filters'; import bloomFilters from 'bloom-filters';
import type { StorageService } from '../services/storage.js'; import type { StorageServiceInterface } from '../services/storageInterface.js';
import { logger } from '../lib/logger.js'; import { logger } from '../lib/logger.js';
const BLOOM_ERROR_RATE = 0.01; const BLOOM_ERROR_RATE = 0.01;
export function createBloomRouter(storage: StorageService): Router { export function createBloomRouter(storage: StorageServiceInterface): Router {
const router = Router(); const router = Router();
/** /**

View File

@ -1,12 +1,12 @@
import { Router, type Request, type Response } from 'express'; import { Router, type Request, type Response } from 'express';
import type { StorageService } from '../services/storage.js'; import type { StorageServiceInterface } from '../services/storageInterface.js';
import type { RelayService } from '../services/relay.js'; import type { RelayService } from '../services/relay.js';
import type { MsgCle, StoredKey } from '../types/message.js'; import type { MsgCle, StoredKey } from '../types/message.js';
import { validateMsgCle } from '../lib/validate.js'; import { validateMsgCle } from '../lib/validate.js';
import { logger } from '../lib/logger.js'; import { logger } from '../lib/logger.js';
export function createKeysRouter( export function createKeysRouter(
storage: StorageService, storage: StorageServiceInterface,
relay: RelayService, relay: RelayService,
): Router { ): Router {
const router = Router(); const router = Router();
@ -55,29 +55,31 @@ export function createKeysRouter(
/** /**
* POST /keys - Store and relay a decryption key. * POST /keys - Store and relay a decryption key.
*/ */
router.post('/', async (req: Request, res: Response): Promise<void> => { router.post('/', (req: Request, res: Response): void => {
try { void (async (): Promise<void> => {
if (!validateMsgCle(req.body)) { try {
res.status(400).json({ error: 'Invalid key format' }); if (!validateMsgCle(req.body)) {
return; res.status(400).json({ error: 'Invalid key format' });
return;
}
const key = req.body as MsgCle;
const stored: StoredKey = {
msg: key,
received_at: Date.now(),
relayed: false,
};
storage.storeKey(stored);
await relay.relayKey(key);
stored.relayed = true;
res.status(201).json({ stored: true });
} catch (error) {
logger.error({ err: error }, 'Error storing key');
res.status(500).json({ error: 'Internal server error' });
} }
const key = req.body as MsgCle; })();
const stored: StoredKey = {
msg: key,
received_at: Date.now(),
relayed: false,
};
storage.storeKey(stored);
await relay.relayKey(key);
stored.relayed = true;
res.status(201).json({ stored: true });
} catch (error) {
logger.error({ err: error }, 'Error storing key');
res.status(500).json({ error: 'Internal server error' });
}
}); });
return router; return router;

View File

@ -1,12 +1,12 @@
import { Router, type Request, type Response } from 'express'; import { Router, type Request, type Response } from 'express';
import type { StorageService } from '../services/storage.js'; import type { StorageServiceInterface } from '../services/storageInterface.js';
import type { RelayService } from '../services/relay.js'; import type { RelayService } from '../services/relay.js';
import type { MsgChiffre, StoredMessage } from '../types/message.js'; import type { MsgChiffre, StoredMessage } from '../types/message.js';
import { validateMsgChiffre } from '../lib/validate.js'; import { validateMsgChiffre } from '../lib/validate.js';
import { logger } from '../lib/logger.js'; import { logger } from '../lib/logger.js';
export function createMessagesRouter( export function createMessagesRouter(
storage: StorageService, storage: StorageServiceInterface,
relay: RelayService, relay: RelayService,
): Router { ): Router {
const router = Router(); const router = Router();
@ -38,33 +38,35 @@ export function createMessagesRouter(
/** /**
* POST /messages - Store and relay an encrypted message. * POST /messages - Store and relay an encrypted message.
*/ */
router.post('/', async (req: Request, res: Response): Promise<void> => { router.post('/', (req: Request, res: Response): void => {
try { void (async (): Promise<void> => {
if (!validateMsgChiffre(req.body)) { try {
res.status(400).json({ error: 'Invalid message format' }); if (!validateMsgChiffre(req.body)) {
return; res.status(400).json({ error: 'Invalid message format' });
return;
}
const msg = req.body as MsgChiffre;
const alreadySeen = storage.hasSeenHash(msg.hash);
const stored: StoredMessage = {
msg,
received_at: Date.now(),
relayed: false,
};
storage.storeMessage(stored);
if (!alreadySeen) {
await relay.relayMessage(msg);
stored.relayed = true;
}
res.status(201).json({ hash: msg.hash, stored: true });
} catch (error) {
logger.error({ err: error }, 'Error storing message');
res.status(500).json({ error: 'Internal server error' });
} }
const msg = req.body as MsgChiffre; })();
const alreadySeen = storage.hasSeenHash(msg.hash);
const stored: StoredMessage = {
msg,
received_at: Date.now(),
relayed: false,
};
storage.storeMessage(stored);
if (!alreadySeen) {
await relay.relayMessage(msg);
stored.relayed = true;
}
res.status(201).json({ hash: msg.hash, stored: true });
} catch (error) {
logger.error({ err: error }, 'Error storing message');
res.status(500).json({ error: 'Internal server error' });
}
}); });
/** /**

View File

@ -1,9 +1,9 @@
import { Router, type Request, type Response } from 'express'; import { Router, type Request, type Response } from 'express';
import { Gauge, register } from 'prom-client'; import { Gauge, register } from 'prom-client';
import type { StorageService } from '../services/storage.js'; import type { StorageServiceInterface } from '../services/storageInterface.js';
import { logger } from '../lib/logger.js'; import { logger } from '../lib/logger.js';
export function createMetricsRouter(storage: StorageService): Router { export function createMetricsRouter(storage: StorageServiceInterface): Router {
const router = Router(); const router = Router();
const gauge = new Gauge({ const gauge = new Gauge({
@ -15,20 +15,22 @@ export function createMetricsRouter(storage: StorageService): Router {
/** /**
* GET /metrics - Prometheus metrics. * GET /metrics - Prometheus metrics.
*/ */
router.get('/', async (_req: Request, res: Response): Promise<void> => { router.get('/', (_req: Request, res: Response): void => {
try { void (async (): Promise<void> => {
const msgCount = storage.getMessages(0, Number.MAX_SAFE_INTEGER).length; try {
gauge.set({ kind: 'messages' }, msgCount); const msgCount = storage.getMessages(0, Number.MAX_SAFE_INTEGER).length;
gauge.set({ kind: 'signatures' }, storage.getSignatureCount()); gauge.set({ kind: 'messages' }, msgCount);
gauge.set({ kind: 'keys' }, storage.getKeyCount()); gauge.set({ kind: 'signatures' }, storage.getSignatureCount());
gauge.set({ kind: 'seen_hashes' }, storage.getSeenHashCount()); gauge.set({ kind: 'keys' }, storage.getKeyCount());
gauge.set({ kind: 'seen_hashes' }, storage.getSeenHashCount());
res.set('Content-Type', register.contentType); res.set('Content-Type', register.contentType);
res.send(await register.metrics()); res.send(await register.metrics());
} catch (err) { } catch (err) {
logger.error({ err }, 'Error generating metrics'); logger.error({ err }, 'Error generating metrics');
res.status(500).json({ error: 'Internal server error' }); res.status(500).json({ error: 'Internal server error' });
} }
})();
}); });
return router; return router;

View File

@ -1,12 +1,12 @@
import { Router, type Request, type Response } from 'express'; import { Router, type Request, type Response } from 'express';
import type { StorageService } from '../services/storage.js'; import type { StorageServiceInterface } from '../services/storageInterface.js';
import type { RelayService } from '../services/relay.js'; import type { RelayService } from '../services/relay.js';
import type { MsgSignature, StoredSignature } from '../types/message.js'; import type { MsgSignature, StoredSignature } from '../types/message.js';
import { validateMsgSignature } from '../lib/validate.js'; import { validateMsgSignature } from '../lib/validate.js';
import { logger } from '../lib/logger.js'; import { logger } from '../lib/logger.js';
export function createSignaturesRouter( export function createSignaturesRouter(
storage: StorageService, storage: StorageServiceInterface,
relay: RelayService, relay: RelayService,
): Router { ): Router {
const router = Router(); const router = Router();
@ -33,29 +33,31 @@ export function createSignaturesRouter(
/** /**
* POST /signatures - Store and relay a signature. * POST /signatures - Store and relay a signature.
*/ */
router.post('/', async (req: Request, res: Response): Promise<void> => { router.post('/', (req: Request, res: Response): void => {
try { void (async (): Promise<void> => {
if (!validateMsgSignature(req.body)) { try {
res.status(400).json({ error: 'Invalid signature format' }); if (!validateMsgSignature(req.body)) {
return; res.status(400).json({ error: 'Invalid signature format' });
return;
}
const sig = req.body as MsgSignature;
const stored: StoredSignature = {
msg: sig,
received_at: Date.now(),
relayed: false,
};
storage.storeSignature(stored);
await relay.relaySignature(sig);
stored.relayed = true;
res.status(201).json({ stored: true });
} catch (error) {
logger.error({ err: error }, 'Error storing signature');
res.status(500).json({ error: 'Internal server error' });
} }
const sig = req.body as MsgSignature; })();
const stored: StoredSignature = {
msg: sig,
received_at: Date.now(),
relayed: false,
};
storage.storeSignature(stored);
await relay.relaySignature(sig);
stored.relayed = true;
res.status(201).json({ stored: true });
} catch (error) {
logger.error({ err: error }, 'Error storing signature');
res.status(500).json({ error: 'Internal server error' });
}
}); });
return router; return router;

View File

@ -41,7 +41,12 @@ export class ApiKeyService {
last_used_at: number | null; last_used_at: number | null;
description: string | null; description: string | null;
}> { }> {
return this.db.listApiKeys(); return this.db.listApiKeys().map((k) => ({
prefix: k.key_prefix,
created_at: k.created_at,
last_used_at: k.last_used_at,
description: k.description,
}));
} }
/** /**

View File

@ -9,7 +9,7 @@ import { logger } from '../lib/logger.js';
* Replaces in-memory storage for production use. * Replaces in-memory storage for production use.
*/ */
export class DatabaseStorageService { export class DatabaseStorageService {
private db: Database.Database; private db!: Database.Database;
private dbPath: string; private dbPath: string;
constructor(storagePath: string) { constructor(storagePath: string) {
@ -326,19 +326,24 @@ export class DatabaseStorageService {
relayed: number; relayed: number;
}>; }>;
return rows.map((row) => ({ return rows.map((row) => {
msg: { const msg: StoredKey['msg'] = {
hash_message: row.hash_message, hash_message: row.hash_message,
cle_de_chiffrement_message: { cle_de_chiffrement_message: {
algo: row.algo, algo: row.algo,
params: JSON.parse(row.params), params: JSON.parse(row.params),
cle_chiffree: row.cle_chiffree ?? undefined,
}, },
df_ecdh_scannable: row.df_ecdh_scannable, df_ecdh_scannable: row.df_ecdh_scannable,
}, };
received_at: row.received_at, if (row.cle_chiffree !== null) {
relayed: row.relayed === 1, msg.cle_de_chiffrement_message.cle_chiffree = row.cle_chiffree;
})); }
return {
msg,
received_at: row.received_at,
relayed: row.relayed === 1,
};
});
} }
/** /**
@ -359,19 +364,24 @@ export class DatabaseStorageService {
relayed: number; relayed: number;
}>; }>;
return rows.map((row) => ({ return rows.map((row) => {
msg: { const msg: StoredKey['msg'] = {
hash_message: row.hash_message, hash_message: row.hash_message,
cle_de_chiffrement_message: { cle_de_chiffrement_message: {
algo: row.algo, algo: row.algo,
params: JSON.parse(row.params), params: JSON.parse(row.params),
cle_chiffree: row.cle_chiffree ?? undefined,
}, },
df_ecdh_scannable: row.df_ecdh_scannable, df_ecdh_scannable: row.df_ecdh_scannable,
}, };
received_at: row.received_at, if (row.cle_chiffree !== null) {
relayed: row.relayed === 1, msg.cle_de_chiffrement_message.cle_chiffree = row.cle_chiffree;
})); }
return {
msg,
received_at: row.received_at,
relayed: row.relayed === 1,
};
});
} }
/** /**

View File

@ -1,4 +1,4 @@
import type { StorageService } from './storage.js'; import type { StorageServiceInterface } from './storageInterface.js';
import type { MsgChiffre, MsgSignature, MsgCle } from '../types/message.js'; import type { MsgChiffre, MsgSignature, MsgCle } from '../types/message.js';
import { logger } from '../lib/logger.js'; import { logger } from '../lib/logger.js';
@ -9,7 +9,7 @@ import { logger } from '../lib/logger.js';
export class RelayService { export class RelayService {
private peerRelays: string[]; private peerRelays: string[];
constructor(_storage: StorageService, peerRelays: string[]) { constructor(_storage: StorageServiceInterface, peerRelays: string[]) {
this.peerRelays = peerRelays; this.peerRelays = peerRelays;
} }

View File

@ -4,12 +4,13 @@ import type {
StoredKey, StoredKey,
} from '../types/message.js'; } from '../types/message.js';
import type { DatabaseStorageService } from './database.js'; import type { DatabaseStorageService } from './database.js';
import type { StorageServiceInterface } from './storageInterface.js';
/** /**
* Adapter to make DatabaseStorageService compatible with StorageService interface. * Adapter to make DatabaseStorageService compatible with StorageServiceInterface.
* This allows existing code to work with the new database implementation. * This allows existing code to work with the new database implementation.
*/ */
export class StorageAdapter { export class StorageAdapter implements StorageServiceInterface {
constructor(private db: DatabaseStorageService) {} constructor(private db: DatabaseStorageService) {}
hasSeenHash(hash: string): boolean { hasSeenHash(hash: string): boolean {

View File

@ -0,0 +1,23 @@
import type { StoredMessage, StoredSignature, StoredKey } from '../types/message.js';
/**
* Common interface for storage services (in-memory or database).
*/
export interface StorageServiceInterface {
hasSeenHash(hash: string): boolean;
markHashSeen(hash: string): void;
storeMessage(msg: StoredMessage): void;
getMessage(hash: string): StoredMessage | undefined;
getMessages(start: number, end: number, serviceUuid?: string): StoredMessage[];
storeSignature(sig: StoredSignature): void;
getSignatures(hash: string): StoredSignature[];
storeKey(key: StoredKey): void;
getKeys(hash: string): StoredKey[];
getKeysInWindow(start: number, end: number): StoredKey[];
getSeenHashCount(): number;
getSeenHashes(): string[];
getSignatureCount(): number;
getKeyCount(): number;
initialize?(): Promise<void>;
saveToDisk?(): Promise<void>;
}

View File

@ -0,0 +1,110 @@
# Correction : Combinaison de plusieurs petits UTXOs pour les ancrages avec provisionnement
**Auteur** : Équipe 4NK
**Date** : 2026-01-28
## Problème Identifié
L'API d'ancrage retournait une erreur "No UTXO large enough for anchor with provisioning. Required: 0.00023055 BTC, Largest available: 0.000025 BTC" même lorsque le wallet contenait de nombreux petits UTXOs dont la somme était suffisante.
### Symptômes
- Erreur : "No UTXO large enough for anchor with provisioning. Required: 0.00023055 BTC, Largest available: 0.000025 BTC"
- Le wallet contient plusieurs petits UTXOs de 2500 sats (0.000025 BTC) créés par le provisionnement précédent
- Aucun UTXO assez grand individuellement pour créer une transaction avec provisionnement
- L'API ne peut pas créer de transaction d'ancrage malgré un solde total suffisant
## Cause Racine
La logique de sélection d'UTXOs cherchait uniquement un seul UTXO assez grand pour couvrir le montant total nécessaire (ancrage + 7 UTXOs de provisionnement + frais ≈ 0.00023055 BTC).
**Problème technique** : Le système ne combinait pas plusieurs petits UTXOs pour atteindre le montant requis. Tous les UTXOs disponibles étaient des petits UTXOs de 2500 sats créés par le provisionnement précédent, mais aucun UTXO individuel n'était assez grand.
## Correctifs Appliqués
### Combinaison automatique de plusieurs UTXOs
**Fichier** : `api-anchorage/src/bitcoin-rpc.js`
**Modification** : Ajout d'une logique de combinaison automatique de plusieurs petits UTXOs si aucun UTXO assez grand n'est trouvé.
**Stratégie** :
1. Si aucun UTXO assez grand n'est trouvé, récupérer tous les petits UTXOs disponibles
2. Sélectionner les UTXOs par ordre décroissant jusqu'à atteindre le montant nécessaire
3. Ajuster l'estimation des frais pour tenir compte des inputs supplémentaires (~148 bytes par input)
4. Créer une transaction avec plusieurs inputs combinant tous les UTXOs sélectionnés
5. Verrouiller/déverrouiller tous les UTXOs de manière cohérente
**Détails techniques** :
- Limite de 20 UTXOs maximum pour éviter des transactions trop grandes
- Estimation des frais ajustée dynamiquement : +0.0000001 BTC par input supplémentaire
- Vérification de disponibilité de tous les UTXOs avant création de la transaction
- Marquage de tous les UTXOs comme dépensés après création réussie
- Déverrouillage de tous les UTXOs en cas d'erreur ou de retry
## Modifications
### Fichiers Modifiés
- `api-anchorage/src/bitcoin-rpc.js` :
- Logique de sélection d'UTXOs : combine plusieurs petits UTXOs si nécessaire
- Création de transaction : utilise plusieurs inputs au lieu d'un seul
- Calcul du change : utilise `totalSelectedAmount` au lieu de `selectedUtxo.amount`
- Vérification de disponibilité : vérifie tous les UTXOs sélectionnés
- Gestion des erreurs : déverrouille et marque tous les UTXOs comme dépensés
- Logs : inclut le nombre d'UTXOs combinés
## Modalités de Déploiement
### Redémarrage de l'API
1. **Arrêter l'API** :
```bash
ps aux | grep "node.*api-anchorage" | grep -v grep | awk '{print $2}' | xargs kill
```
2. **Redémarrer l'API** :
```bash
cd /srv/4NK/<domaine>/api-anchorage
npm start
```
### Vérification
1. **Tester avec plusieurs petits UTXOs** :
- Vérifier que l'API peut créer un ancrage même si tous les UTXOs sont petits
- Vérifier que les logs indiquent "Combining multiple UTXOs for anchor transaction"
2. **Vérifier les transactions** :
- Les transactions doivent avoir plusieurs inputs si plusieurs UTXOs sont combinés
- Le change doit être calculé correctement avec le montant total des inputs
## Résultat
✅ **Problème résolu**
- L'API peut maintenant combiner plusieurs petits UTXOs pour créer un ancrage avec provisionnement
- Plus d'erreur "No UTXO large enough" si le solde total est suffisant
- Les transactions avec plusieurs inputs sont correctement créées et gérées
- Tous les UTXOs sont correctement verrouillés/déverrouillés et marqués comme dépensés
**Exemple de transaction avec UTXOs combinés** :
- Transaction : N inputs (petits UTXOs) → 1 OP_RETURN + 1 ancrage (2500 sats) + 7 provisioning (2500 sats chacun) + change
- Les frais sont ajustés pour tenir compte des inputs supplémentaires
## Prévention
Pour éviter ce problème à l'avenir :
1. **Combinaison automatique** : Le système combine maintenant automatiquement plusieurs petits UTXOs si nécessaire
2. **Limite de taille** : Limite de 20 UTXOs maximum pour éviter des transactions trop grandes
3. **Estimation des frais** : Les frais sont ajustés dynamiquement pour les transactions avec plusieurs inputs
4. **Gestion cohérente** : Tous les UTXOs sont gérés de manière cohérente (verrouillage, déverrouillage, marquage comme dépensé)
## Pages Affectées
- `api-anchorage/src/bitcoin-rpc.js` :
- Fonction `createAnchorTransaction()` : Logique de combinaison de plusieurs UTXOs
- Gestion des inputs multiples dans la création de transaction
- Vérification et gestion de tous les UTXOs sélectionnés
- `fixKnowledge/api-anchorage-combine-utxos.md` : Documentation (nouveau)

31
restart-bitcoind.sh Executable file
View File

@ -0,0 +1,31 @@
#!/bin/bash
# Restart bitcoind service on production server via SSH.
# Usage: ./restart-bitcoind.sh
# Requires: SSH access to prod server (ncantu@192.168.1.103).
set -e
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
PROD_HOST="192.168.1.103"
PROD_USER="ncantu"
SIGNET_DIR="/srv/4NK/signet.4nkweb.com"
echo "=== Redémarrage de bitcoind sur le serveur prod (${PROD_USER}@${PROD_HOST}) ==="
# Vérifier l'état du service avant redémarrage
echo "État du service avant redémarrage:"
ssh "${PROD_USER}@${PROD_HOST}" "cd ${SIGNET_DIR} && ./manage.sh status"
echo ""
echo "Redémarrage du service via manage.sh..."
ssh "${PROD_USER}@${PROD_HOST}" "cd ${SIGNET_DIR} && ./manage.sh restart"
# Attendre quelques secondes pour que le service démarre
sleep 3
echo ""
echo "État du service après redémarrage:"
ssh "${PROD_USER}@${PROD_HOST}" "cd ${SIGNET_DIR} && ./manage.sh status"
echo ""
echo "✅ Redémarrage de bitcoind terminé"