docs: reconcile development validation guidance

This commit is contained in:
fy59 2026-09-02 22:10:32 +02:00
parent 4278d124c2
commit 7574362ba9
3 changed files with 853 additions and 434 deletions

View file

@ -1,193 +1,293 @@
# Instructions de build
# Build
## Prérequis
## Status
- **OS** : Linux (testé sur distributions récentes)
- **Compilateur** : Clang (recommandé) ou GCC
- **Système de build** : Meson + Ninja
- **Dépendances principales** : ncursesw, SQLite, OpenSSL, GIO/GLib, OpenCV,
LibRaw, libexif, libpng, libdeflate, Ceres ; Vulkan reste optionnel
- **Langages** : API publiques C17 et implémentation mixte C17/C++17
```text
BUILD_SYSTEM=MESON_NINJA
PUBLIC_API_LANGUAGE=C17
IMPLEMENTATION_LANGUAGES=C17_CXX17
BUILD_PARALLELISM=HOST_AWARE
RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
SERIALISM_REQUIRES_PROOF=CANONICAL
```
## Bootstrap des outils
Meson is the build-system authority. Do not duplicate dependency-version truth
in this document when `meson.build` already enforces it.
Les commandes ci-dessous installent seulement le compilateur, Meson/Ninja,
`pkg-config` et ncurses. Les bibliothèques listées plus haut doivent aussi être
disponibles dans les versions acceptées par `meson.build`; Meson reste la source
de vérité et refuse explicitement une dépendance absente ou incompatible.
## Requirements
Lardon3D targets Linux.
Primary toolchain:
```text
Clang or GCC
Meson
Ninja
pkg-config
```
Major dependencies currently include ncursesw, SQLite, OpenSSL, GIO/GLib,
OpenCV, LibRaw, libexif, libpng, libdeflate and Ceres. Vulkan remains optional
at configuration level.
Public APIs are C17. Implementation is mixed C17/C++17.
## Bootstrap examples
These commands install only the basic compiler/build front end; Meson remains
authoritative for the complete dependency set.
Debian/Ubuntu:
```sh
# Debian / Ubuntu
sudo apt install clang meson ninja-build libncursesw5-dev pkg-config
```
# Fedora
Fedora:
```sh
sudo dnf install clang meson ninja-build ncurses-devel pkg-config
```
# Arch
Arch Linux:
```sh
sudo pacman -S clang meson ninja ncurses pkgconf
```
## Build standard
## Standard build
First configuration:
```sh
# Première configuration
CC=clang meson setup build
CC=clang CXX=clang++ meson setup build
```
# Arbre existant
Existing tree:
```sh
meson setup --reconfigure build
meson compile -C build -j8
meson compile -C build
```
### Options utiles
Do not hard-code `-j8` as project policy.
Meson/Ninja should use host-appropriate parallelism unless a specific
validation has a reason to constrain it.
## Build parallelism policy
The build is not governed by a portable fixed job count.
Canonical policy:
```text
preserve the interactive host reserve
then use maximum safe useful throughput
```
A reference host measurement such as 8 or 12 useful jobs is evidence for that
host at that time, not a repository constant.
If memory-heavy compilation or another active workload creates pressure,
reduce build width for that run. Do not convert the temporary reduction into a
global documentation rule.
## Reconfigure versus wipe
Prefer incremental reuse:
```sh
# Build de debug (défaut)
meson setup build --wipe
# Build de release
meson setup build --wipe --buildtype=release
# Build avec optimisations aggressive
meson setup build --wipe --buildtype=release -Db_lto=true
meson setup --reconfigure build
meson compile -C build
```
Use `--wipe` only when a fresh configuration is actually required, such as:
- switching sanitizer configuration in the same directory;
- changing compiler family;
- changing a configuration whose cached state cannot be reused safely;
- reproducing a clean release/global-maintenance proof;
- recovering from a stale or corrupt build directory.
A normal edit/test loop should not wipe the build tree repeatedly.
## Release build
Use an explicit release directory or deliberate reconfiguration.
Example:
```sh
CC=clang CXX=clang++ meson setup build-release --buildtype=release
meson compile -C build-release
```
For LTO:
```sh
CC=clang CXX=clang++ meson setup build-release-lto --buildtype=release -Db_lto=true
meson compile -C build-release-lto
```
Separate directories avoid destroying a useful incremental debug tree.
## Vulkan configuration
Portable CPU-only proof:
```sh
CC=clang CXX=clang++ meson setup build-portable -Dvulkan_orb=disabled
meson compile -C build-portable
```
Vulkan-enabled proof:
```sh
CC=clang CXX=clang++ meson setup build-vulkan -Dvulkan_orb=enabled
meson compile -C build-vulkan
```
A Vulkan-on build is not automatically a proof that every scientific path uses
or should use the GPU.
Current production GPU promotion remains limited by each subsystem's validated
backend contract.
## Validation
Normal configured tests:
```sh
# Tests unitaires
meson test -C build --print-errorlogs
# Vérification du style (whitespace)
git diff --check
# Vérification autonome d'un header C public modifié
cc -x c -std=c17 -fsyntax-only -Iinclude \
-include lardon3d/<header>.h /dev/null
```
### Preuve fraîche de maintenance globale — 1er septembre 2026
Le [registre canonique](../architecture/global_maintenance_audit.md) conserve
le détail et les qualifications. Les résultats reproductibles acquis sont :
| Configuration fraîche | Compilateurs/options | Build | Suite |
| --- | --- | ---: | ---: |
| portable | Clang/Clang++ 22.1.8, C17/C++17, `-Dvulkan_orb=disabled` | 931/931 | 64/64 sériel |
| Vulkan | Clang/Clang++ 22.1.8, C17/C++17, `-Dvulkan_orb=enabled` | 939/939 | 65/65 sériel |
| ASan/UBSan portable | Clang/Clang++ 22.1.8, `address,undefined` | graphe complet | 64/64 avec LSan désactivé après attribution externe |
| TSan portable | GCC/G++ 16.2.1, Vulkan désactivé | cibles concurrentes | 14/14 + 220 répétitions |
La suite Vulkan comprend `orb-vulkan-backend` sur la Radeon 780M RADV PHOENIX
réelle. La cible de feasibility SIFT/RootSIFT, non enregistrée dans la suite,
a été compilée/exécutée séparément : zéro divergence de décision Lowe mais des
divergences d'index et de bits de distance, donc aucune promotion en backend
production. Les probes stricts GCC/Clang C17+C++17 passent 76/76 sur les
19 headers publics modifiés ou nouveaux, ainsi que le fixture ABI, le lien
application et `git diff --check`; `scan3d/` reste intact.
L'unique revue finale indépendante GPT-5.6 SOL/ULTRA a conclu PASS sans finding
bloquant. Elle a indépendamment rejoué le build portable, la suite complète
64/64, une matrice focalisée 15/15, les 76/76 probes de headers, l'ABI, les
négatifs de seams production, le SHA-256 du manifest GV retenu et le diff-check.
Le statut canonique est donc `GLOBAL_MAINTENANCE_AUDIT=PASS/FROZEN` ; les
qualifications sanitizer ci-dessous restent néanmoins partie de la preuve.
Une validation post-freeze a ensuite attribué le délai intermittent de
`test-feature-task` à la capture de la télémétrie hôte réelle par ses Governors
synthétiques. Le fixture utilise maintenant un `ResourceSnapshot` complet,
privé, par Governor et compilé pour cette seule cible ; production continue de
lire la télémétrie réelle et n'exporte aucun seam. Après correction du second
Governor relevé en revue, Feature passe 100/100, la matrice ordonnée 4/4 et les
suites finales portable/Vulkan 64/64 et 65/65 ; ASan/UBSan ciblé avec
`detect_leaks=0` et TSan passent. Le registre canonique conserve la régression
charge 5 `WAIT`/charge 0 `START` et la qualification exacte. Un timeout `task`
isolé dans une suite normale mixte après reconstruction large reste
non reproductible : le ciblé immédiat et sa matrice de revue 100/100 passent,
sans modification de Task ni de son timeout.
La première suite LSan complète est volontairement conservée comme non-PASS :
57 OK, 6 FAIL et 1 timeout. Cinq échecs partagent exactement la fuite externe
OpenCL de 3 808 octets/68 allocations ; les deux anomalies de 30 s n'ont aucun
diagnostic sanitizer. Le délai Feature, absent du suivi initial, a ensuite été
reproduit et corrigé comme décrit ci-dessus ; le délai Task reste non
reproductible. La suite entière passe 64/64 avec ASan/UBSan actifs et
`detect_leaks=0`, tandis qu'un sous-ensemble prouvé sans loader OpenCV/OpenCL
passe 20/20 avec LSan actif. Il est donc incorrect de résumer cette preuve par
« LSan 64/64 ».
Le log Clang complet a aussi été audité. La conversion publique Sparse SfM
`uint32_t → int` était matérielle et a été corrigée avec validation ciblée ; les
autres émissions sont soit des conversions baseline déjà bornées, soit des
tests/benchmarks, soit des headers OpenCV/Ceres externes. Les emplacements et
justifications exacts restent centralisés dans le registre afin de ne pas
dupliquer une seconde liste normative ici.
## Build ASan/UBSan (debug mémoire)
À exécuter pour tout ticket touchant la mémoire, les durées de vie ou les
allocations :
Whitespace/style boundary:
```sh
CC=clang meson setup build-asan --wipe \
-Db_sanitize=address,undefined
meson compile -C build-asan -j8
git diff --check
```
Public C header probe:
```sh
cc -x c -std=c17 -fsyntax-only -Iinclude -include lardon3d/<header>.h /dev/null
```
Use `docs/development/testing.md` for sanitizer and validation policy.
## ASan / UBSan build
Example dedicated directory:
```sh
CC=clang CXX=clang++ meson setup build-asan -Db_sanitize=address,undefined
meson compile -C build-asan
meson test -C build-asan --print-errorlogs
```
## Build TSan (concurrence)
Do not claim an unqualified full LeakSanitizer pass from the retained global
maintenance checkpoint. The external OpenCL loader qualification documented in
the canonical audit remains part of that evidence.
À exécuter pour tout ticket touchant la concurrence (pthread, mutex,
variables de condition, états partagés) :
## TSan build
Use TSan only with the configuration that matches the intended proof.
The retained global maintenance concurrency proof used GCC/G++ with Vulkan
disabled, because the project TSan matrix and the Vulkan runtime validation are
separate evidence boundaries.
Example:
```sh
CC=clang meson setup build-tsan --wipe \
-Db_sanitize=thread \
-Db_lundef=false
meson compile -C build-tsan -j8
CC=gcc CXX=g++ meson setup build-tsan -Db_sanitize=thread -Db_lundef=false -Dvulkan_orb=disabled
meson compile -C build-tsan
meson test -C build-tsan --print-errorlogs
```
## Variables d'environnement
The exact target subset, suppression qualification and repetition evidence are
documented in `docs/development/concurrency.md` and the global maintenance
audit.
| Variable | Description |
|---|---|
| `CC` | Compilateur C (défaut : gcc) |
| `CFLAGS` | Drapeaux de compilation supplémentaires |
| `LDFLAGS` | Drapeaux de liaison supplémentaires |
## Current retained maintenance checkpoint
## Structure du build
The canonical detailed evidence is:
```text
docs/architecture/global_maintenance_audit.md
GLOBAL_MAINTENANCE_AUDIT=PASS/FROZEN
```
The 2026-09-01 checkpoint retained:
```text
portable Clang/Clang++ build and suite
Vulkan-on Clang/Clang++ build and suite
ASan/UBSan qualified run
portable GCC/G++ TSan matrix
public-header C17/C++17 probes
ABI and application-link checks
independent review
```
Those exact historical counts belong to the audit and should not be duplicated
as a new current build contract.
## Build directory layout
A configured Meson tree typically contains:
```text
build/
├── src/ # objets et binaires
├── tests/ # binaires de tests
└── compile_commands.json # pour LSP / clangd
src/
tests/
compile_commands.json
```
## Dépannage
Exact generated layout is Meson/Ninja output and may evolve.
### Erreur : ncursesw introuvable
## Environment
Common variables include:
| Variable | Purpose |
| --- | --- |
| `CC` | C compiler |
| `CXX` | C++ compiler |
| `CFLAGS` | additional C flags |
| `CXXFLAGS` | additional C++ flags |
| `LDFLAGS` | additional linker flags |
Prefer Meson options for project features rather than ad-hoc environment flags
that make builds difficult to reproduce.
## Troubleshooting
Check ncursesw discovery:
```sh
# Vérifier l'installation
pkg-config --libs ncursesw
# Si absent, installer le paquet de développement ncursesw
```
### Erreur : clang introuvable
If Clang is unavailable, GCC is supported where the current Meson checks allow
it.
```sh
# Utiliser gcc en alternative
meson setup build --wipe
# ou installer clang
sudo apt install clang
For a slow build, first preserve the existing build tree and let Ninja use
normal host-aware scheduling. Reduce concurrency only when actual host pressure
or another active workload justifies it.
`ccache` may be used when available, but it is optional operational tooling and
not part of scientific identity.
## Rules
```text
NO_FIXED_GLOBAL_J8=YES
NO_REPEATED_UNCHANGED_WIPE=YES
HOST_AWARE_BUILD_PARALLELISM=YES
```
### Build lent
```sh
# Réduire la parallélisation
meson compile -C build -j4
# ou utiliser ccache
CC="ccache clang" meson setup build --wipe
```
Build configuration is operational state. It must not silently redefine
scientific formats, fingerprints or persistence contracts.

View file

@ -1,255 +1,412 @@
# Règles de concurrence
# Concurrency
## Vue d'ensemble
Lardon3D utilise un modèle de concurrence à thread unique pour ncurses
et un modèle multi-thread pour le traitement. La séparation est stricte :
le thread ncurses ne fait jamais de travail métier, et les workers ne
touchent jamais ncurses.
## Modèle de concurrence
## Status
```text
Thread principal (ncurses)
├── Gestion des entrées
├── Affichage TUI
└── Orchestration
NCURSES_OWNER=MAIN_THREAD_ONLY
ACTIVE_HEAVY_QUEUE_CALLBACKS=1
TASK_CANCELLATION=COOPERATIVE
INTERNAL_PARALLELISM=BOUNDED
OWNER_ONLY_PUBLICATION=CANONICAL_WHERE_REQUIRED
Worker thread
├── Exécution des tâches
├── Calculs métier
└── Écritures de résultats
TSAN_PORTABLE_PROJECT_MATRIX=QUALIFIED_PASS
TSAN_EXTERNAL_OPENCV_TBB=QUALIFIED
VULKAN_CONCURRENCY_VALIDATION=SEPARATE
SSD operation thread (0 ou 1, joinable)
└── Un poll ou contrôle UDisks synchrone borné, sans ncurses ni Task
RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
SERIALISM_REQUIRES_PROOF=CANONICAL
```
## Règles fondamentales
Lardon3D separates UI ownership from heavy processing.
### 1. ncurses appartient au thread principal
The main thread owns ncurses. The Task Queue owns one active heavy callback.
Individual validated Task Kinds may create bounded internal participants inside
that callback.
```c
// ✅ Correct : appel depuis le thread principal
mvprintw(0, 0, "Progression: %d%%", progress);
An SSD controller operation may also use at most one bounded joinable operation
thread under its own ownership contract.
// ❌ Interdit : appel depuis un worker
// mvprintw() dans un thread secondaire
## Execution model
```text
main thread
input
ncurses
TUI orchestration
Task Queue worker
one active heavy callback
admitted Task sequence
optional bounded internal participants
deterministic owner publication
SSD operation thread
zero or one bounded joinable controller operation
no ncurses
no Task callback
```
### 2. Variables partagées protégées par mutex
Internal participants are not a second global scheduler or Queue.
```c
// ✅ Correct
pthread_mutex_lock(&queue->mutex);
queue->count++;
pthread_mutex_unlock(&queue->mutex);
## Fundamental rules
// ❌ Interdit
// queue->count++; sans protection
### ncurses ownership
Only the main thread calls ncurses.
Workers publish observable state through protected data. They never call
`mvprintw`, `wrefresh`, or other ncurses APIs.
### Shared mutable state
Shared mutable state must have an explicit synchronization owner:
```text
mutex
condition variable
atomic primitive where the contract explicitly permits it
single-thread ownership
immutable-after-publication
```
### 3. Variables de condition pour la synchronisation
Do not rely on timing or "normally only one caller".
```c
// Producteur (caller de la Task Queue)
pthread_mutex_lock(&queue->mutex);
queue->ready = true;
pthread_cond_signal(&queue->cond);
pthread_mutex_unlock(&queue->mutex);
### Condition variables
// Consommateur (worker)
pthread_mutex_lock(&queue->mutex);
while (!queue->ready) {
pthread_cond_wait(&queue->cond, &queue->mutex);
}
// traitement
pthread_mutex_unlock(&queue->mutex);
Always test the predicate in a loop around `pthread_cond_wait()`.
A signal is not durable state; the protected predicate is.
### Cooperative cancellation
Production Task cancellation is cooperative.
Do not use `pthread_cancel()` to stop a Task.
Task-specific non-preemptible operations finish their current atomic boundary
before pause/cancel is observed.
### Reservation before callback
No Task callback runs without the Resource Governor admission/reservation
required by its installed sequence contract.
Fixed-resource Tasks do not bypass the Governor.
### Terminal lifetime
Terminal callback completion precedes destruction of Task userdata.
Queue/Task ownership must ensure no observer dereferences freed userdata.
## Lock ordering
When multiple locks are required, the owning subsystem must define and preserve
one order.
Never add a reverse-order path to solve a local problem.
Avoid holding one subsystem mutex while calling into another subsystem that may
call back.
Where practical:
```text
copy bounded state under lock
release lock
perform I/O / expensive work
reacquire only for publication
```
### 4. Pas de callback ncurses depuis un worker
## Queue ingress lifetime
```c
// ✅ Correct : le worker signale au thread principal
void worker_callback(task_t *task, void *userdata) {
shared_state_t *state = userdata;
pthread_mutex_lock(&state->mutex);
state->result_ready = true;
pthread_cond_signal(&state->cond);
pthread_mutex_unlock(&state->mutex);
}
The Queue owner closes ingress before destruction.
// ❌ Interdit : appel ncurses depuis le worker
// void worker_callback(...) {
// mvprintw(...);
// }
Shutdown waits for:
- active worker completion;
- registered in-flight API calls covered by the ownership contract;
- terminal callbacks.
This cannot make a raw C pointer safe if a caller begins a new call after the
object has already been freed. Callers must obey lifetime ownership.
## Bounded internal parallelism
A validated Task Kind may use internal participants while the Queue callback
remains the sole Task owner.
Required shape:
```text
one admitted Task owner
-> bounded participant count
-> bounded private work
-> join all participants
-> owner-only deterministic publication when required
-> Task-specific durable cursor
-> generic checkpoint
-> sequence_break
```
## Primitives utilisées
Participant count and memory must fit the admitted Resource Governor contract.
| Primitive | Usage |
|---|---|
| `pthread_mutex_t` | Protection des données partagées |
| `pthread_cond_t` | Synchronisation producteur/consommateur |
| `pthread_create()` | Création des workers |
| `pthread_join()` | Attente de fin des workers |
| `pthread_cancel()` | Non utilisé pour interrompre une Task ; annulation coopérative |
No participant may silently exceed the installed sequence contract.
## Invariants de concurrence
## Atomicity does not imply serialism
1. **Un seul thread ncurses** : ncurses n'est jamais appelé depuis un
worker. Toute mise à jour de l'UI passe par des variables partagées
protégées.
Per-item scientific atomicity and cross-item execution width are separate.
2. **Mutex hiérarchique** : si plusieurs mutex sont acquis, toujours dans
le même ordre pour éviter les deadlocks.
3. **Annulation coopérative** : les workers vérifient périodiquement un
drapeau d'annulation. Une Task n'est pas interrompue brutalement.
4. **Réservation atomique** : la réservation du gouverneur est atomique.
Deux threads ne peuvent pas obtenir la même réservation.
5. **Pas de callback sans réservation** : aucun callback de tâche n'est
invoqué sans réservation active. Cet invariant est maintenu même en
présence d'erreurs.
6. **Retraite après callback** : la notification terminale finit avant la
destruction du userdata. Queue détruit la Task hors de son mutex et ne
conserve ensuite qu'un snapshot borné.
7. **Fermeture d'ingress** : le propriétaire empêche les nouveaux appels Queue
avant `destroy()`. La fermeture interne attend le worker et chaque appel
enregistré avant le close ; elle ne peut rendre sûr un appel démarré après
la libération d'un pointeur C brut.
8. **Parallélisme scientifique propriétaire** : lorsqu'un kind emploie des
participants internes, le callback Queue demeure l'unique propriétaire. Le
nombre de participants et leur mémoire sont admis par le Governor ; seul le
propriétaire publie le préfixe durable ordonné et joint tous les enfants.
9. **Lease SSD par objet** : un lease scratch appartient à l'adresse exacte de
l'objet fourni par le caller. Tous ses champs sont lus/écrits sous le mutex
du contrôleur. Le caller lui garantit un accès exclusif et ne le copie, ne le
déplace ni ne le présente simultanément à deux contrôleurs. En production,
acquire/release passent par les wrappers Governor ; le Governor relâche son
mutex avant l'appel contrôleur, et le contrôleur ne rappelle jamais le
Governor. À la saturation légale `generation == UINT64_MAX`, seule la fin
du wrapper exact déjà sérialisé peut réconcilier sa propre opération et le
compte fondé sur les adresses ; une update publique au même watermark ne
peut pas rendre une autorité stale.
10. **Owner SSD unique** : la TUI/main demande et poll l'opération ; au plus un
thread joinable exécute une opération bornée et ne touche jamais ncurses.
Le destroy le joint avant unregister. Une observation malformée enregistre
`ERROR` et ne confère aucune autorité de contrôle ou de lease.
11. **Frontière projet** : les vues libèrent leurs borrows, puis la Queue est
annulée/jointe/détruite avant Project DB. Une Queue vide est créée ensuite.
Aucun callback terminal ne peut donc déréférencer une DB déjà fermée et
l'histoire d'un projet ne fuit pas dans le suivant.
12. **Ordre d'arrêt global** : Queue et leases Task, puis fermeture projet,
join/unregister du binding SSD, contrôleur SSD, et enfin Governor. Un
unregister encore bloqué par un lease est un échec observable, jamais un
pointeur abandonné.
## Anti-patterns
### Deadlock
```c
// ❌ Risque de deadlock
pthread_mutex_lock(&mutex_a);
pthread_mutex_lock(&mutex_b); // attend mutex_b
// Dans un autre thread :
pthread_mutex_lock(&mutex_b);
pthread_mutex_lock(&mutex_a); // attend mutex_a → DEADLOCK
```text
PER_ITEM_ATOMICITY_REQUIRES_CROSS_ITEM_SERIALISM=NO
OWNER_ONLY_PUBLICATION_REQUIRES_SERIAL_PREPARATION=NO
```
**Solution** : toujours acquérir les mutex dans le même ordre.
Current examples include selected RAW, selected Feature extraction, Candidate
Pair source work and outer Geometric Verification preparation.
### Race condition
Serialization is valid only where the subsystem's scientific, persistence,
library or measured-throughput contract proves it necessary.
```c
// ❌ Race condition
if (task->state == TASK_STATE_QUEUED) {
task->state = TASK_STATE_RUNNING;
}
## CPU/batch coupling
// ✅ Correct
pthread_mutex_lock(&task->mutex);
if (task->state == TASK_STATE_QUEUED) {
task->state = TASK_STATE_RUNNING;
}
pthread_mutex_unlock(&task->mutex);
CPU and batch/window are not globally independent dimensions.
For a Task whose additional participants cannot do useful work while the
admitted item window remains one, a Task-specific capability may couple those
dimensions.
Current validated examples include:
```text
candidate_pair.generate/1
features.extract.batch/1
```
### Use-after-free
This is not a universal rule for all Task Kinds.
```c
// ❌ Use-after-free
task_destroy(task);
task_callback(task); // task est libéré
## Project lifetime boundary
// ✅ Correct : le callback est entièrement revenu avant la destruction
task_callback(task);
task_destroy(task);
Before closing a project:
```text
views release Project DB borrows
-> Queue is cancelled/joined/destroyed
-> Project DB closes
-> fresh empty Queue may be created for the next project
```
## Validation
A terminal callback must never observe a Project DB already destroyed.
Les readers Visual Index sont sans état partagé mutable. Une query copie la
liste bornée des segments sous le mutex DB, puis effectue hash, lectures et
accumulation après déverrouillage. Un update ne rend le nouveau segment visible
qu'au commit memberships+segment ; une query en cours garde son snapshot.
Project-specific runtime history must not leak into the next project.
Pour tout ticket touchant la concurrence, exécuter :
## Global shutdown boundary
The current shutdown order preserves ownership across:
```text
Task Queue and Task leases
-> project close
-> join/unregister SSD binding
-> SSD controller
-> Resource Governor
```
A scratch unregister blocked by a real outstanding lease is an observable
failure, not permission to abandon a live pointer.
## SSD lease ownership
A scratch lease belongs to the exact caller-owned lease object used for the
operation.
Its mutable fields are controlled under the SSD controller mutex.
The caller must not:
- copy a live lease;
- move a live lease;
- present the same lease object to two controllers;
- release through a different ownership path.
Production acquire/release uses the Resource Governor wrappers.
The Governor releases its own mutex before entering the controller, and the
controller does not callback into the Governor while holding its mutex.
Scratch remains storage capacity, never RAM admission.
## Visual Index readers
Visual Index query readers do not share mutable query state.
A query obtains a bounded segment snapshot under the Project DB boundary, then
performs hash/read/accumulation after release.
A concurrent update makes a new segment visible only at its canonical commit
boundary. An already running query continues with its retained snapshot.
## OpenCV process-wide state
OpenCV thread configuration is process-wide.
The active heavy Queue callback owns temporary mutation of that setting where a
Task contract requires it and restores the previous/baseline value on all exit
paths.
Internal participants must not independently race `cv::setNumThreads()`.
For Feature batch, cross-image participants are used while internal OpenCV
threading is controlled explicitly.
## Vulkan boundary
ORB Vulkan concurrency is validated separately from portable TSan.
The production AUTO contract currently uses:
```text
normal inflight depth = 1
private validated safety depth = 2
helpers = 0
```
Depth 2 is a private safety/benchmark capability and was rejected as the normal
useful setting by measured throughput.
A Vulkan backend failure produces complete CPU fallback before publication.
Partial GPU scientific output is never published.
## TSan policy
For project concurrency changes, use a dedicated TSan build that matches the
supported proof boundary.
The retained global maintenance matrix used:
```text
GCC/G++
Vulkan disabled
selected concurrent targets
deterministic repetitions
```
and completed the retained 14/14 target matrix plus 220 repetitions.
The narrow suppression file is:
```text
tests/tsan-opencv.supp
```
It covers external non-instrumented OpenCV/TBB objects only.
It must not suppress Lardon3D frames.
Therefore the correct retained claim is not "TSan proves all concurrency".
It is:
```text
portable project concurrency matrix passed under the documented qualification
external OpenCV/TBB reports are narrowly qualified
Vulkan concurrency has a separate validation boundary
```
## What TSan does not prove
TSan is useful for instrumented conflicting memory access and some
synchronization misuse.
It does not prove absence of:
- deadlock;
- lost wakeup caused by incorrect predicate design;
- lifetime bugs outside the exercised paths;
- races hidden inside non-instrumented external libraries;
- Vulkan driver/runtime correctness;
- scientific determinism.
Lock-order review, ownership reasoning and deterministic tests remain required.
## Sanitizer command policy
Do not encode fixed `-j8` as canonical validation.
Example configuration:
```sh
# Build TSan
CC=clang meson setup build-tsan --wipe -Db_sanitize=thread -Db_lundef=false
meson compile -C build-tsan -j8
CC=gcc CXX=g++ meson setup build-tsan -Db_sanitize=thread -Db_lundef=false -Dvulkan_orb=disabled
meson compile -C build-tsan
meson test -C build-tsan --print-errorlogs
```
TSan détecte automatiquement :
Use host-aware compile/test parallelism unless the proof itself requires
serialization.
- les accès concurrents conflictuels instrumentés ;
- certaines utilisations incohérentes des primitives de synchronisation.
Do not repeatedly wipe an unchanged TSan tree.
Il ne prouve pas l'absence de deadlock, de signal perdu ou de bug dans une
bibliothèque non instrumentée. Les invariants de lifetime et d'ordre de locks
restent donc soumis aux tests déterministes et à la revue.
## Concurrency review checklist
### Preuve TSan globale courante
Before closing a concurrency-sensitive change, verify:
La matrice fraîche emploie GCC/G++ 16.2.1 et désactive explicitement Vulkan.
Elle passe 14/14 cibles couvrant Task, Project, Queue, Governor, registre/leases
SSD, contrôleur SSD, observateur/TUI async, Candidate, Visual Index, Feature,
Matcher et GV, puis 220/220 répétitions déterministes : **234/234** au total.
- ncurses remains main-thread-only;
- every shared mutable field has an explicit synchronization owner;
- condition predicates are checked in loops;
- lock order remains consistent;
- no Task uses forced asynchronous cancellation;
- Queue callbacks have an active reservation;
- internal participants stay within the admitted contract;
- all children join on every exit path;
- owner-only publication remains ordered where required;
- project-close ordering prevents DB use-after-close;
- SSD lease ownership remains exact;
- Task userdata outlives terminal notification;
- appropriate deterministic concurrency tests pass;
- portable TSan qualification is preserved;
- Vulkan validation is reported separately;
- ASan/UBSan is run when the change also affects lifetime/memory.
La seule liste de suppressions est `tests/tsan-opencv.supp`, limitée aux objets
partagés externes non instrumentés `libopencv_features.so`,
`libopencv_core.so` et `libtbb.so`. Elle ne masque aucune frame Lardon3D. Les
warnings GCC `-Wmaybe-uninitialized` des contrôles OpenCV Feature/SIFT sont
classés non matériels : le callback fournit une Task non nulle et le helper
initialise la structure avant toute autre sortie d'échec. Les warnings OpenCV
du build GV appartiennent aux headers externes.
## Current retained evidence
Cette preuve TSan ne vaut pas validation de concurrence Vulkan. Le backend
ORB Vulkan réel est couvert séparément par le build Clang Vulkan-on 939/939,
la suite 65/65 et ses tests de backend/handle/publication ; cette séparation
doit rester explicite dans tout rapport.
The canonical global-maintenance record is:
## Checklist de concurrence
```text
docs/architecture/global_maintenance_audit.md
GLOBAL_MAINTENANCE_AUDIT=PASS/FROZEN
```
Avant de livrer un ticket touchant la concurrence :
The current A6000 checkpoint is later:
- [ ] Toutes les variables partagées sont protégées par un mutex
- [ ] Les mutex sont toujours libérés (même en cas d'erreur)
- [ ] Les variables de condition sont vérifiées dans une boucle `while`
- [ ] Aucun appel ncurses depuis un worker
- [ ] L'annulation des Tasks est coopérative (pas de `pthread_cancel`)
- [ ] TSan ne signale aucune erreur
- [ ] Le build ASan ne signale aucune fuite mémoire liée aux threads
```text
real-a6000-pre-sfm-2026-09-02
REAL_A6000_PRE_SFM=PASS/FROZEN
```
The later A6000 proof exercised current bounded parallel paths through selected
Feature batch, Candidate, Matcher, Geometric Verifier v3 and Tracks without
changing the historical TSan qualification.
Historical evidence remains historical; new changes require validation scoped
to their actual concurrency surface.
## Summary
```text
NCURSES_OWNER=MAIN_THREAD_ONLY
ACTIVE_HEAVY_QUEUE_CALLBACKS=1
TASK_CANCELLATION=COOPERATIVE
INTERNAL_PARALLELISM=BOUNDED
PER_ITEM_ATOMICITY_REQUIRES_CROSS_ITEM_SERIALISM=NO
OWNER_ONLY_PUBLICATION_REQUIRES_SERIAL_PREPARATION=NO
TSAN_PORTABLE_PROJECT_MATRIX=QUALIFIED_PASS
TSAN_EXTERNAL_OPENCV_TBB=QUALIFIED
VULKAN_CONCURRENCY_VALIDATION=SEPARATE
NO_FIXED_GLOBAL_J8=YES
NO_REPEATED_UNCHANGED_WIPE=YES
RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
SERIALISM_REQUIRES_PROOF=CANONICAL
```

View file

@ -1,136 +1,298 @@
# Procédures de test
# Testing
## Vue d'ensemble
## Status
Lardon3D utilise le framework de test intégré à Meson. Chaque module possède
un fichier de test dans `tests/` correspondant au module testé.
```text
DOCUMENTATION_LANGUAGE=ENGLISH
TEST_POLICY=HOST_AWARE
REPEATED_UNCHANGED_EXPENSIVE_VALIDATION=AVOID
TSAN_OPEN_CV_TBB_QUALIFICATION=REQUIRED
VULKAN_CONCURRENCY_VALIDATION=SEPARATE
```
## Lancer les tests
Lardon3D uses Meson's test runner. Tests live under `tests/` and combine unit,
integration, persistence, restart, resource and real-path validation.
## Normal commands
Run the configured suite:
```sh
# Tous les tests
meson test -C build --print-errorlogs
```
# Un test spécifique
meson test -C build test_task_queue --print-errorlogs
Run one named test:
# Tests avec verbose
```sh
meson test -C build <test-name> --print-errorlogs
```
Verbose execution:
```sh
meson test -C build -v --print-errorlogs
```
# Réexécuter uniquement les tests échoués
Re-run failures only:
```sh
meson test -C build --reprint=failed
```
## Structure des tests
Use the names registered by the current `meson.build`; this document does not
maintain a second authoritative list of every test target.
## Validation policy
Validation must match the change.
A documentation-only change normally requires:
```text
tests/
├── test_task_queue.c # tests de la file de tâches
├── test_task.c # tests du module task
├── test_resource_governor.c # tests du gouverneur
├── test_hardware_profile.c # tests du profil matériel
├── test_import.c # tests de l'import
├── test_project.c # tests des projets
└── test_*.c # autres modules
git diff --check
targeted content checks
targeted link/authority review
```
## Écrire un test
It does not justify wiping and rebuilding unchanged code.
```c
#include <glib.h>
#include "lardon3d/task.h"
A code change normally requires, in increasing scope:
void test_task_create(void) {
task_estimate_t est = {
.ram_bytes = 1024 * 1024,
.gpu_bytes = 0,
.cpu_weight = 1,
.io_weight = 0,
.batch_size = 10,
.batch_max = 100
};
task_t *t = task_create("test", &est, NULL, NULL);
g_assert_nonnull(t);
g_assert_cmpint(task_get_state(t), ==, TASK_STATE_IDLE);
task_destroy(t);
}
int main(int argc, char **argv) {
g_test_init(&argc, &argv, NULL);
g_test_add_func("/task/create", test_task_create);
return g_test_run();
}
```text
targeted build
targeted tests
broader affected suite
sanitizer or concurrency validation when relevant
full suite when the change or release boundary justifies it
```
## Conventions
Do not repeatedly rerun an unchanged expensive suite between documentation
edits merely to create activity.
1. **Préfixe `test_`** : chaque fonction de test porte le préfixe `test_`.
2. **Chemin hiérarchique** : le nom du test suit le pattern `/module/action`.
3. **Asserts GLib** : utiliser `g_assert_*` pour les vérifications.
4. **Nettoyage** : chaque test libère toutes ses ressources.
5. **Isolation** : un test ne dépend pas de l'état d'un autre test.
6. **Déterminisme** : les tests ne dépendent pas de l'heure, du filesystem
ou de l'état réseau (sauf test d'import).
## Host-aware parallelism
## Commentaires source
Build and test parallelism are host-aware.
Les commentaires documentent le pourquoi et les contrats non évidents :
invariants, propriété et durée de vie, persistance, ainsi que limites et
frontières de ressources. Les API publiques documentent leurs contrats non
évidents. Ils ne paraphrasent pas le code ligne par ligne et sont mis à jour
avec tout changement de comportement.
Do not encode a project-wide fixed `-j8`, `--num-processes 1`, or equivalent
constant as canonical policy.
## Tests unitaires vs tests d'intégration
The correct width depends on the current machine, interactive reserve, memory,
toolchain and workload. Use all safe useful host capacity while preserving the
defined interactive reserve.
`test-visual-index` couvre les descriptors synthétiques, le retrieval ORB réel,
les filtres inter-ScanSets, quatre queries concurrentes, la corruption/absence/
troncature d'un segment et 4 000 Feature Sets synthétiques. Le scénario de
reprise `visual_index.update` est exercé dans `test-feature-task`.
```text
RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
SERIALISM_REQUIRES_PROOF=CANONICAL
```
| Type | Portée | Fichier |
|---|---|---|
| Unitaire | Un module isolé | `tests/test_<module>.c` |
| Intégration | Interaction entre modules | `tests/test_<module>.c` avec dépendances réelles |
If a temporary validation must be serialized for determinism, diagnosis or a
known tool limitation, label that serialization as test-specific evidence
rather than a global default.
## Validation par ticket
## Fresh build policy
Avant de livrer un ticket, exécuter la séquence complète :
Do not use `meson setup --wipe` by default.
Prefer:
```sh
# 1. Build clean
CC=clang meson setup build --wipe
meson compile -C build -j8
# 2. Tests
meson test -C build --print-errorlogs
# 3. Style
git diff --check
# 4. Si mémoire/concurrence touchés
CC=clang meson setup build-asan --wipe -Db_sanitize=address,undefined
meson compile -C build-asan -j8
meson test -C build-asan --print-errorlogs
# 5. Si concurrence touchée
CC=clang meson setup build-tsan --wipe -Db_sanitize=thread -Db_lundef=false
meson compile -C build-tsan -j8
meson test -C build-tsan --print-errorlogs
meson setup --reconfigure build
meson compile -C build
```
## Dépannage
Create or wipe a build directory when the configuration genuinely needs a
fresh environment, for example:
### Test qui échoue en ASan
```text
different sanitizer set
portable Vulkan-off proof
Vulkan-on proof
compiler-family change
known stale/corrupt build directory
release-grade clean proof
```
Vérifier les durées de vie des allocations. Ne jamais libérer un objet puis
y accéder. Vérifier que chaque `task_destroy()` est appelée.
Repeated wipes of the same unchanged configuration waste time and invalidate
incremental-build advantages.
### Test qui échoue en TSan
## Sanitizers
Vérifier que toutes les variables partagées sont protégées par un mutex.
Vérifier que ncurses est utilisé uniquement depuis le thread principal.
### ASan / UBSan
### Test qui échoue uniquement en release
For memory, lifetime, ownership or undefined-behavior changes, use a dedicated
sanitizer build.
Vérifier les assertions et les overflow arithmétiques. Compiler avec
`-fsanitize=undefined` pour détecter les comportements indéfinis.
Example configuration:
```sh
CC=clang CXX=clang++ meson setup build-asan -Db_sanitize=address,undefined
meson compile -C build-asan
meson test -C build-asan --print-errorlogs
```
Reconfigure or wipe only when the existing sanitizer directory does not match
the requested configuration.
### LeakSanitizer qualification
The retained global maintenance evidence must not be summarized as
`LSan 64/64`.
The full first leak-enabled run exposed an externally attributed OpenCL loader
leak and two timeout anomalies. The retained qualified result is:
```text
ASan/UBSan full suite: PASS with detect_leaks=0
proved subset without the external loader: LSan PASS
full leak-enabled suite: not an unqualified PASS
```
Preserve that distinction in future reports unless new evidence supersedes it.
## ThreadSanitizer
Concurrency changes require TSan where the instrumented boundary is meaningful.
The retained global maintenance TSan proof used GCC/G++ with Vulkan disabled
and covered the selected concurrent targets plus deterministic repetitions.
The only retained suppression file is:
```text
tests/tsan-opencv.supp
```
Its purpose is limited to external OpenCV/TBB objects. It must not suppress
Lardon3D frames.
Therefore never report a blanket statement such as:
```text
TSan proves the entire Vulkan build race-free
```
The valid qualification is:
```text
project concurrent paths covered by the retained portable TSan matrix
external OpenCV/TBB reports qualified by the narrow suppression boundary
Vulkan concurrency validated separately
```
## Vulkan validation
ORB Vulkan uses a separate validation boundary.
The retained global maintenance evidence includes a Vulkan-on build and suite
on the real Radeon 780M/RADV host, plus dedicated backend/handle/publication
tests.
That evidence is not interchangeable with TSan.
SIFT/RootSIFT feasibility results did not establish a production GPU backend;
do not turn feasibility checks into production validation claims.
## Determinism and repetition
Repeat tests when repetition proves something specific:
```text
deterministic restart
race sensitivity
resource adaptation
ordering stability
flaky regression reproduction
```
Do not repeat unchanged tests without a stated purpose.
When repetition is the evidence, record:
```text
exact test/corpus
run count
relevant configuration
success/failure count
digest or invariant when applicable
```
## Test isolation
Tests should:
- own and clean up their temporary resources;
- avoid depending on another test's execution order;
- avoid network state unless the test explicitly owns that dependency;
- use synthetic/private Resource snapshots where the test is about deterministic
policy rather than live host telemetry;
- avoid changing global process state without restoring it.
OpenCV thread configuration is process-wide and must be restored on every exit
path in tests that change it.
## Public-header validation
When a public C header changes, run a standalone C17 syntax probe in addition to
normal build coverage.
Conceptually:
```sh
cc -x c -std=c17 -fsyntax-only -Iinclude -include lardon3d/<header>.h /dev/null
```
Use the current supported compiler matrix when the change affects ABI or
C/C++ interoperability.
## Source comments
Source comments explain non-obvious contracts:
```text
invariants
ownership and lifetime
persistence ordering
resource boundaries
recovery behavior
scientific constraints
```
They should not paraphrase obvious code line by line.
Repository source comments are English.
## Current retained global maintenance evidence
The canonical detailed record is:
```text
docs/architecture/global_maintenance_audit.md
GLOBAL_MAINTENANCE_AUDIT=PASS/FROZEN
```
That historical checkpoint includes fresh portable/Vulkan builds, full suites,
sanitizer work, TSan work, public-header probes, ABI/application-link checks and
independent review.
It remains historical evidence. New changes require only the validation
appropriate to the changed surface unless a new global checkpoint is being
created.
## Ticket closure checklist
Before closing a code ticket:
- confirm the requested scope only was changed;
- run `git diff --check`;
- run targeted tests for changed behavior;
- run the affected broader suite when justified;
- run ASan/UBSan for memory/lifetime-sensitive changes;
- run TSan for concurrency-sensitive project code when applicable;
- keep Vulkan validation separate from portable TSan claims;
- preserve exact external-library qualifications;
- avoid fixed host-parallelism constants;
- avoid repeated unchanged clean builds or suites;
- report what actually ran, not a stronger claim.
For documentation-only remediation, use documentation checks rather than
rebuilding unchanged production code.