docs: reconcile development validation guidance

This commit is contained in:
fy59 2026-09-02 22:10:32 +02:00
parent 4278d124c2
commit 7574362ba9
3 changed files with 853 additions and 434 deletions

View file

@ -1,193 +1,293 @@
# Instructions de build # Build
## Prérequis ## Status
- **OS** : Linux (testé sur distributions récentes) ```text
- **Compilateur** : Clang (recommandé) ou GCC BUILD_SYSTEM=MESON_NINJA
- **Système de build** : Meson + Ninja PUBLIC_API_LANGUAGE=C17
- **Dépendances principales** : ncursesw, SQLite, OpenSSL, GIO/GLib, OpenCV, IMPLEMENTATION_LANGUAGES=C17_CXX17
LibRaw, libexif, libpng, libdeflate, Ceres ; Vulkan reste optionnel BUILD_PARALLELISM=HOST_AWARE
- **Langages** : API publiques C17 et implémentation mixte C17/C++17 RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
SERIALISM_REQUIRES_PROOF=CANONICAL
```
## Bootstrap des outils Meson is the build-system authority. Do not duplicate dependency-version truth
in this document when `meson.build` already enforces it.
Les commandes ci-dessous installent seulement le compilateur, Meson/Ninja, ## Requirements
`pkg-config` et ncurses. Les bibliothèques listées plus haut doivent aussi être
disponibles dans les versions acceptées par `meson.build`; Meson reste la source Lardon3D targets Linux.
de vérité et refuse explicitement une dépendance absente ou incompatible.
Primary toolchain:
```text
Clang or GCC
Meson
Ninja
pkg-config
```
Major dependencies currently include ncursesw, SQLite, OpenSSL, GIO/GLib,
OpenCV, LibRaw, libexif, libpng, libdeflate and Ceres. Vulkan remains optional
at configuration level.
Public APIs are C17. Implementation is mixed C17/C++17.
## Bootstrap examples
These commands install only the basic compiler/build front end; Meson remains
authoritative for the complete dependency set.
Debian/Ubuntu:
```sh ```sh
# Debian / Ubuntu
sudo apt install clang meson ninja-build libncursesw5-dev pkg-config sudo apt install clang meson ninja-build libncursesw5-dev pkg-config
```
# Fedora Fedora:
```sh
sudo dnf install clang meson ninja-build ncurses-devel pkg-config sudo dnf install clang meson ninja-build ncurses-devel pkg-config
```
# Arch Arch Linux:
```sh
sudo pacman -S clang meson ninja ncurses pkgconf sudo pacman -S clang meson ninja ncurses pkgconf
``` ```
## Build standard ## Standard build
First configuration:
```sh ```sh
# Première configuration CC=clang CXX=clang++ meson setup build
CC=clang meson setup build ```
# Arbre existant Existing tree:
```sh
meson setup --reconfigure build meson setup --reconfigure build
meson compile -C build -j8 meson compile -C build
``` ```
### Options utiles Do not hard-code `-j8` as project policy.
Meson/Ninja should use host-appropriate parallelism unless a specific
validation has a reason to constrain it.
## Build parallelism policy
The build is not governed by a portable fixed job count.
Canonical policy:
```text
preserve the interactive host reserve
then use maximum safe useful throughput
```
A reference host measurement such as 8 or 12 useful jobs is evidence for that
host at that time, not a repository constant.
If memory-heavy compilation or another active workload creates pressure,
reduce build width for that run. Do not convert the temporary reduction into a
global documentation rule.
## Reconfigure versus wipe
Prefer incremental reuse:
```sh ```sh
# Build de debug (défaut) meson setup --reconfigure build
meson setup build --wipe meson compile -C build
# Build de release
meson setup build --wipe --buildtype=release
# Build avec optimisations aggressive
meson setup build --wipe --buildtype=release -Db_lto=true
``` ```
Use `--wipe` only when a fresh configuration is actually required, such as:
- switching sanitizer configuration in the same directory;
- changing compiler family;
- changing a configuration whose cached state cannot be reused safely;
- reproducing a clean release/global-maintenance proof;
- recovering from a stale or corrupt build directory.
A normal edit/test loop should not wipe the build tree repeatedly.
## Release build
Use an explicit release directory or deliberate reconfiguration.
Example:
```sh
CC=clang CXX=clang++ meson setup build-release --buildtype=release
meson compile -C build-release
```
For LTO:
```sh
CC=clang CXX=clang++ meson setup build-release-lto --buildtype=release -Db_lto=true
meson compile -C build-release-lto
```
Separate directories avoid destroying a useful incremental debug tree.
## Vulkan configuration
Portable CPU-only proof:
```sh
CC=clang CXX=clang++ meson setup build-portable -Dvulkan_orb=disabled
meson compile -C build-portable
```
Vulkan-enabled proof:
```sh
CC=clang CXX=clang++ meson setup build-vulkan -Dvulkan_orb=enabled
meson compile -C build-vulkan
```
A Vulkan-on build is not automatically a proof that every scientific path uses
or should use the GPU.
Current production GPU promotion remains limited by each subsystem's validated
backend contract.
## Validation ## Validation
Normal configured tests:
```sh ```sh
# Tests unitaires
meson test -C build --print-errorlogs meson test -C build --print-errorlogs
# Vérification du style (whitespace)
git diff --check
# Vérification autonome d'un header C public modifié
cc -x c -std=c17 -fsyntax-only -Iinclude \
-include lardon3d/<header>.h /dev/null
``` ```
### Preuve fraîche de maintenance globale — 1er septembre 2026 Whitespace/style boundary:
Le [registre canonique](../architecture/global_maintenance_audit.md) conserve
le détail et les qualifications. Les résultats reproductibles acquis sont :
| Configuration fraîche | Compilateurs/options | Build | Suite |
| --- | --- | ---: | ---: |
| portable | Clang/Clang++ 22.1.8, C17/C++17, `-Dvulkan_orb=disabled` | 931/931 | 64/64 sériel |
| Vulkan | Clang/Clang++ 22.1.8, C17/C++17, `-Dvulkan_orb=enabled` | 939/939 | 65/65 sériel |
| ASan/UBSan portable | Clang/Clang++ 22.1.8, `address,undefined` | graphe complet | 64/64 avec LSan désactivé après attribution externe |
| TSan portable | GCC/G++ 16.2.1, Vulkan désactivé | cibles concurrentes | 14/14 + 220 répétitions |
La suite Vulkan comprend `orb-vulkan-backend` sur la Radeon 780M RADV PHOENIX
réelle. La cible de feasibility SIFT/RootSIFT, non enregistrée dans la suite,
a été compilée/exécutée séparément : zéro divergence de décision Lowe mais des
divergences d'index et de bits de distance, donc aucune promotion en backend
production. Les probes stricts GCC/Clang C17+C++17 passent 76/76 sur les
19 headers publics modifiés ou nouveaux, ainsi que le fixture ABI, le lien
application et `git diff --check`; `scan3d/` reste intact.
L'unique revue finale indépendante GPT-5.6 SOL/ULTRA a conclu PASS sans finding
bloquant. Elle a indépendamment rejoué le build portable, la suite complète
64/64, une matrice focalisée 15/15, les 76/76 probes de headers, l'ABI, les
négatifs de seams production, le SHA-256 du manifest GV retenu et le diff-check.
Le statut canonique est donc `GLOBAL_MAINTENANCE_AUDIT=PASS/FROZEN` ; les
qualifications sanitizer ci-dessous restent néanmoins partie de la preuve.
Une validation post-freeze a ensuite attribué le délai intermittent de
`test-feature-task` à la capture de la télémétrie hôte réelle par ses Governors
synthétiques. Le fixture utilise maintenant un `ResourceSnapshot` complet,
privé, par Governor et compilé pour cette seule cible ; production continue de
lire la télémétrie réelle et n'exporte aucun seam. Après correction du second
Governor relevé en revue, Feature passe 100/100, la matrice ordonnée 4/4 et les
suites finales portable/Vulkan 64/64 et 65/65 ; ASan/UBSan ciblé avec
`detect_leaks=0` et TSan passent. Le registre canonique conserve la régression
charge 5 `WAIT`/charge 0 `START` et la qualification exacte. Un timeout `task`
isolé dans une suite normale mixte après reconstruction large reste
non reproductible : le ciblé immédiat et sa matrice de revue 100/100 passent,
sans modification de Task ni de son timeout.
La première suite LSan complète est volontairement conservée comme non-PASS :
57 OK, 6 FAIL et 1 timeout. Cinq échecs partagent exactement la fuite externe
OpenCL de 3 808 octets/68 allocations ; les deux anomalies de 30 s n'ont aucun
diagnostic sanitizer. Le délai Feature, absent du suivi initial, a ensuite été
reproduit et corrigé comme décrit ci-dessus ; le délai Task reste non
reproductible. La suite entière passe 64/64 avec ASan/UBSan actifs et
`detect_leaks=0`, tandis qu'un sous-ensemble prouvé sans loader OpenCV/OpenCL
passe 20/20 avec LSan actif. Il est donc incorrect de résumer cette preuve par
« LSan 64/64 ».
Le log Clang complet a aussi été audité. La conversion publique Sparse SfM
`uint32_t → int` était matérielle et a été corrigée avec validation ciblée ; les
autres émissions sont soit des conversions baseline déjà bornées, soit des
tests/benchmarks, soit des headers OpenCV/Ceres externes. Les emplacements et
justifications exacts restent centralisés dans le registre afin de ne pas
dupliquer une seconde liste normative ici.
## Build ASan/UBSan (debug mémoire)
À exécuter pour tout ticket touchant la mémoire, les durées de vie ou les
allocations :
```sh ```sh
CC=clang meson setup build-asan --wipe \ git diff --check
-Db_sanitize=address,undefined ```
meson compile -C build-asan -j8
Public C header probe:
```sh
cc -x c -std=c17 -fsyntax-only -Iinclude -include lardon3d/<header>.h /dev/null
```
Use `docs/development/testing.md` for sanitizer and validation policy.
## ASan / UBSan build
Example dedicated directory:
```sh
CC=clang CXX=clang++ meson setup build-asan -Db_sanitize=address,undefined
meson compile -C build-asan
meson test -C build-asan --print-errorlogs meson test -C build-asan --print-errorlogs
``` ```
## Build TSan (concurrence) Do not claim an unqualified full LeakSanitizer pass from the retained global
maintenance checkpoint. The external OpenCL loader qualification documented in
the canonical audit remains part of that evidence.
À exécuter pour tout ticket touchant la concurrence (pthread, mutex, ## TSan build
variables de condition, états partagés) :
Use TSan only with the configuration that matches the intended proof.
The retained global maintenance concurrency proof used GCC/G++ with Vulkan
disabled, because the project TSan matrix and the Vulkan runtime validation are
separate evidence boundaries.
Example:
```sh ```sh
CC=clang meson setup build-tsan --wipe \ CC=gcc CXX=g++ meson setup build-tsan -Db_sanitize=thread -Db_lundef=false -Dvulkan_orb=disabled
-Db_sanitize=thread \ meson compile -C build-tsan
-Db_lundef=false
meson compile -C build-tsan -j8
meson test -C build-tsan --print-errorlogs meson test -C build-tsan --print-errorlogs
``` ```
## Variables d'environnement The exact target subset, suppression qualification and repetition evidence are
documented in `docs/development/concurrency.md` and the global maintenance
audit.
| Variable | Description | ## Current retained maintenance checkpoint
|---|---|
| `CC` | Compilateur C (défaut : gcc) |
| `CFLAGS` | Drapeaux de compilation supplémentaires |
| `LDFLAGS` | Drapeaux de liaison supplémentaires |
## Structure du build The canonical detailed evidence is:
```text
docs/architecture/global_maintenance_audit.md
GLOBAL_MAINTENANCE_AUDIT=PASS/FROZEN
```
The 2026-09-01 checkpoint retained:
```text
portable Clang/Clang++ build and suite
Vulkan-on Clang/Clang++ build and suite
ASan/UBSan qualified run
portable GCC/G++ TSan matrix
public-header C17/C++17 probes
ABI and application-link checks
independent review
```
Those exact historical counts belong to the audit and should not be duplicated
as a new current build contract.
## Build directory layout
A configured Meson tree typically contains:
```text ```text
build/ build/
├── src/ # objets et binaires src/
├── tests/ # binaires de tests tests/
└── compile_commands.json # pour LSP / clangd compile_commands.json
``` ```
## Dépannage Exact generated layout is Meson/Ninja output and may evolve.
### Erreur : ncursesw introuvable ## Environment
Common variables include:
| Variable | Purpose |
| --- | --- |
| `CC` | C compiler |
| `CXX` | C++ compiler |
| `CFLAGS` | additional C flags |
| `CXXFLAGS` | additional C++ flags |
| `LDFLAGS` | additional linker flags |
Prefer Meson options for project features rather than ad-hoc environment flags
that make builds difficult to reproduce.
## Troubleshooting
Check ncursesw discovery:
```sh ```sh
# Vérifier l'installation
pkg-config --libs ncursesw pkg-config --libs ncursesw
# Si absent, installer le paquet de développement ncursesw
``` ```
### Erreur : clang introuvable If Clang is unavailable, GCC is supported where the current Meson checks allow
it.
```sh For a slow build, first preserve the existing build tree and let Ninja use
# Utiliser gcc en alternative normal host-aware scheduling. Reduce concurrency only when actual host pressure
meson setup build --wipe or another active workload justifies it.
# ou installer clang
sudo apt install clang `ccache` may be used when available, but it is optional operational tooling and
not part of scientific identity.
## Rules
```text
NO_FIXED_GLOBAL_J8=YES
NO_REPEATED_UNCHANGED_WIPE=YES
HOST_AWARE_BUILD_PARALLELISM=YES
``` ```
### Build lent Build configuration is operational state. It must not silently redefine
scientific formats, fingerprints or persistence contracts.
```sh
# Réduire la parallélisation
meson compile -C build -j4
# ou utiliser ccache
CC="ccache clang" meson setup build --wipe
```

View file

@ -1,255 +1,412 @@
# Règles de concurrence # Concurrency
## Vue d'ensemble ## Status
Lardon3D utilise un modèle de concurrence à thread unique pour ncurses
et un modèle multi-thread pour le traitement. La séparation est stricte :
le thread ncurses ne fait jamais de travail métier, et les workers ne
touchent jamais ncurses.
## Modèle de concurrence
```text ```text
Thread principal (ncurses) NCURSES_OWNER=MAIN_THREAD_ONLY
├── Gestion des entrées ACTIVE_HEAVY_QUEUE_CALLBACKS=1
├── Affichage TUI TASK_CANCELLATION=COOPERATIVE
└── Orchestration INTERNAL_PARALLELISM=BOUNDED
OWNER_ONLY_PUBLICATION=CANONICAL_WHERE_REQUIRED
Worker thread TSAN_PORTABLE_PROJECT_MATRIX=QUALIFIED_PASS
├── Exécution des tâches TSAN_EXTERNAL_OPENCV_TBB=QUALIFIED
├── Calculs métier VULKAN_CONCURRENCY_VALIDATION=SEPARATE
└── Écritures de résultats
SSD operation thread (0 ou 1, joinable) RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
└── Un poll ou contrôle UDisks synchrone borné, sans ncurses ni Task SERIALISM_REQUIRES_PROOF=CANONICAL
``` ```
## Règles fondamentales Lardon3D separates UI ownership from heavy processing.
### 1. ncurses appartient au thread principal The main thread owns ncurses. The Task Queue owns one active heavy callback.
Individual validated Task Kinds may create bounded internal participants inside
that callback.
```c An SSD controller operation may also use at most one bounded joinable operation
// ✅ Correct : appel depuis le thread principal thread under its own ownership contract.
mvprintw(0, 0, "Progression: %d%%", progress);
// ❌ Interdit : appel depuis un worker ## Execution model
// mvprintw() dans un thread secondaire
```text
main thread
input
ncurses
TUI orchestration
Task Queue worker
one active heavy callback
admitted Task sequence
optional bounded internal participants
deterministic owner publication
SSD operation thread
zero or one bounded joinable controller operation
no ncurses
no Task callback
``` ```
### 2. Variables partagées protégées par mutex Internal participants are not a second global scheduler or Queue.
```c ## Fundamental rules
// ✅ Correct
pthread_mutex_lock(&queue->mutex);
queue->count++;
pthread_mutex_unlock(&queue->mutex);
// ❌ Interdit ### ncurses ownership
// queue->count++; sans protection
Only the main thread calls ncurses.
Workers publish observable state through protected data. They never call
`mvprintw`, `wrefresh`, or other ncurses APIs.
### Shared mutable state
Shared mutable state must have an explicit synchronization owner:
```text
mutex
condition variable
atomic primitive where the contract explicitly permits it
single-thread ownership
immutable-after-publication
``` ```
### 3. Variables de condition pour la synchronisation Do not rely on timing or "normally only one caller".
```c ### Condition variables
// Producteur (caller de la Task Queue)
pthread_mutex_lock(&queue->mutex);
queue->ready = true;
pthread_cond_signal(&queue->cond);
pthread_mutex_unlock(&queue->mutex);
// Consommateur (worker) Always test the predicate in a loop around `pthread_cond_wait()`.
pthread_mutex_lock(&queue->mutex);
while (!queue->ready) { A signal is not durable state; the protected predicate is.
pthread_cond_wait(&queue->cond, &queue->mutex);
} ### Cooperative cancellation
// traitement
pthread_mutex_unlock(&queue->mutex); Production Task cancellation is cooperative.
Do not use `pthread_cancel()` to stop a Task.
Task-specific non-preemptible operations finish their current atomic boundary
before pause/cancel is observed.
### Reservation before callback
No Task callback runs without the Resource Governor admission/reservation
required by its installed sequence contract.
Fixed-resource Tasks do not bypass the Governor.
### Terminal lifetime
Terminal callback completion precedes destruction of Task userdata.
Queue/Task ownership must ensure no observer dereferences freed userdata.
## Lock ordering
When multiple locks are required, the owning subsystem must define and preserve
one order.
Never add a reverse-order path to solve a local problem.
Avoid holding one subsystem mutex while calling into another subsystem that may
call back.
Where practical:
```text
copy bounded state under lock
release lock
perform I/O / expensive work
reacquire only for publication
``` ```
### 4. Pas de callback ncurses depuis un worker ## Queue ingress lifetime
```c The Queue owner closes ingress before destruction.
// ✅ Correct : le worker signale au thread principal
void worker_callback(task_t *task, void *userdata) {
shared_state_t *state = userdata;
pthread_mutex_lock(&state->mutex);
state->result_ready = true;
pthread_cond_signal(&state->cond);
pthread_mutex_unlock(&state->mutex);
}
// ❌ Interdit : appel ncurses depuis le worker Shutdown waits for:
// void worker_callback(...) {
// mvprintw(...); - active worker completion;
// } - registered in-flight API calls covered by the ownership contract;
- terminal callbacks.
This cannot make a raw C pointer safe if a caller begins a new call after the
object has already been freed. Callers must obey lifetime ownership.
## Bounded internal parallelism
A validated Task Kind may use internal participants while the Queue callback
remains the sole Task owner.
Required shape:
```text
one admitted Task owner
-> bounded participant count
-> bounded private work
-> join all participants
-> owner-only deterministic publication when required
-> Task-specific durable cursor
-> generic checkpoint
-> sequence_break
``` ```
## Primitives utilisées Participant count and memory must fit the admitted Resource Governor contract.
| Primitive | Usage | No participant may silently exceed the installed sequence contract.
|---|---|
| `pthread_mutex_t` | Protection des données partagées |
| `pthread_cond_t` | Synchronisation producteur/consommateur |
| `pthread_create()` | Création des workers |
| `pthread_join()` | Attente de fin des workers |
| `pthread_cancel()` | Non utilisé pour interrompre une Task ; annulation coopérative |
## Invariants de concurrence ## Atomicity does not imply serialism
1. **Un seul thread ncurses** : ncurses n'est jamais appelé depuis un Per-item scientific atomicity and cross-item execution width are separate.
worker. Toute mise à jour de l'UI passe par des variables partagées
protégées.
2. **Mutex hiérarchique** : si plusieurs mutex sont acquis, toujours dans ```text
le même ordre pour éviter les deadlocks. PER_ITEM_ATOMICITY_REQUIRES_CROSS_ITEM_SERIALISM=NO
OWNER_ONLY_PUBLICATION_REQUIRES_SERIAL_PREPARATION=NO
3. **Annulation coopérative** : les workers vérifient périodiquement un
drapeau d'annulation. Une Task n'est pas interrompue brutalement.
4. **Réservation atomique** : la réservation du gouverneur est atomique.
Deux threads ne peuvent pas obtenir la même réservation.
5. **Pas de callback sans réservation** : aucun callback de tâche n'est
invoqué sans réservation active. Cet invariant est maintenu même en
présence d'erreurs.
6. **Retraite après callback** : la notification terminale finit avant la
destruction du userdata. Queue détruit la Task hors de son mutex et ne
conserve ensuite qu'un snapshot borné.
7. **Fermeture d'ingress** : le propriétaire empêche les nouveaux appels Queue
avant `destroy()`. La fermeture interne attend le worker et chaque appel
enregistré avant le close ; elle ne peut rendre sûr un appel démarré après
la libération d'un pointeur C brut.
8. **Parallélisme scientifique propriétaire** : lorsqu'un kind emploie des
participants internes, le callback Queue demeure l'unique propriétaire. Le
nombre de participants et leur mémoire sont admis par le Governor ; seul le
propriétaire publie le préfixe durable ordonné et joint tous les enfants.
9. **Lease SSD par objet** : un lease scratch appartient à l'adresse exacte de
l'objet fourni par le caller. Tous ses champs sont lus/écrits sous le mutex
du contrôleur. Le caller lui garantit un accès exclusif et ne le copie, ne le
déplace ni ne le présente simultanément à deux contrôleurs. En production,
acquire/release passent par les wrappers Governor ; le Governor relâche son
mutex avant l'appel contrôleur, et le contrôleur ne rappelle jamais le
Governor. À la saturation légale `generation == UINT64_MAX`, seule la fin
du wrapper exact déjà sérialisé peut réconcilier sa propre opération et le
compte fondé sur les adresses ; une update publique au même watermark ne
peut pas rendre une autorité stale.
10. **Owner SSD unique** : la TUI/main demande et poll l'opération ; au plus un
thread joinable exécute une opération bornée et ne touche jamais ncurses.
Le destroy le joint avant unregister. Une observation malformée enregistre
`ERROR` et ne confère aucune autorité de contrôle ou de lease.
11. **Frontière projet** : les vues libèrent leurs borrows, puis la Queue est
annulée/jointe/détruite avant Project DB. Une Queue vide est créée ensuite.
Aucun callback terminal ne peut donc déréférencer une DB déjà fermée et
l'histoire d'un projet ne fuit pas dans le suivant.
12. **Ordre d'arrêt global** : Queue et leases Task, puis fermeture projet,
join/unregister du binding SSD, contrôleur SSD, et enfin Governor. Un
unregister encore bloqué par un lease est un échec observable, jamais un
pointeur abandonné.
## Anti-patterns
### Deadlock
```c
// ❌ Risque de deadlock
pthread_mutex_lock(&mutex_a);
pthread_mutex_lock(&mutex_b); // attend mutex_b
// Dans un autre thread :
pthread_mutex_lock(&mutex_b);
pthread_mutex_lock(&mutex_a); // attend mutex_a → DEADLOCK
``` ```
**Solution** : toujours acquérir les mutex dans le même ordre. Current examples include selected RAW, selected Feature extraction, Candidate
Pair source work and outer Geometric Verification preparation.
### Race condition Serialization is valid only where the subsystem's scientific, persistence,
library or measured-throughput contract proves it necessary.
```c ## CPU/batch coupling
// ❌ Race condition
if (task->state == TASK_STATE_QUEUED) {
task->state = TASK_STATE_RUNNING;
}
// ✅ Correct CPU and batch/window are not globally independent dimensions.
pthread_mutex_lock(&task->mutex);
if (task->state == TASK_STATE_QUEUED) { For a Task whose additional participants cannot do useful work while the
task->state = TASK_STATE_RUNNING; admitted item window remains one, a Task-specific capability may couple those
} dimensions.
pthread_mutex_unlock(&task->mutex);
Current validated examples include:
```text
candidate_pair.generate/1
features.extract.batch/1
``` ```
### Use-after-free This is not a universal rule for all Task Kinds.
```c ## Project lifetime boundary
// ❌ Use-after-free
task_destroy(task);
task_callback(task); // task est libéré
// ✅ Correct : le callback est entièrement revenu avant la destruction Before closing a project:
task_callback(task);
task_destroy(task); ```text
views release Project DB borrows
-> Queue is cancelled/joined/destroyed
-> Project DB closes
-> fresh empty Queue may be created for the next project
``` ```
## Validation A terminal callback must never observe a Project DB already destroyed.
Les readers Visual Index sont sans état partagé mutable. Une query copie la Project-specific runtime history must not leak into the next project.
liste bornée des segments sous le mutex DB, puis effectue hash, lectures et
accumulation après déverrouillage. Un update ne rend le nouveau segment visible
qu'au commit memberships+segment ; une query en cours garde son snapshot.
Pour tout ticket touchant la concurrence, exécuter : ## Global shutdown boundary
The current shutdown order preserves ownership across:
```text
Task Queue and Task leases
-> project close
-> join/unregister SSD binding
-> SSD controller
-> Resource Governor
```
A scratch unregister blocked by a real outstanding lease is an observable
failure, not permission to abandon a live pointer.
## SSD lease ownership
A scratch lease belongs to the exact caller-owned lease object used for the
operation.
Its mutable fields are controlled under the SSD controller mutex.
The caller must not:
- copy a live lease;
- move a live lease;
- present the same lease object to two controllers;
- release through a different ownership path.
Production acquire/release uses the Resource Governor wrappers.
The Governor releases its own mutex before entering the controller, and the
controller does not callback into the Governor while holding its mutex.
Scratch remains storage capacity, never RAM admission.
## Visual Index readers
Visual Index query readers do not share mutable query state.
A query obtains a bounded segment snapshot under the Project DB boundary, then
performs hash/read/accumulation after release.
A concurrent update makes a new segment visible only at its canonical commit
boundary. An already running query continues with its retained snapshot.
## OpenCV process-wide state
OpenCV thread configuration is process-wide.
The active heavy Queue callback owns temporary mutation of that setting where a
Task contract requires it and restores the previous/baseline value on all exit
paths.
Internal participants must not independently race `cv::setNumThreads()`.
For Feature batch, cross-image participants are used while internal OpenCV
threading is controlled explicitly.
## Vulkan boundary
ORB Vulkan concurrency is validated separately from portable TSan.
The production AUTO contract currently uses:
```text
normal inflight depth = 1
private validated safety depth = 2
helpers = 0
```
Depth 2 is a private safety/benchmark capability and was rejected as the normal
useful setting by measured throughput.
A Vulkan backend failure produces complete CPU fallback before publication.
Partial GPU scientific output is never published.
## TSan policy
For project concurrency changes, use a dedicated TSan build that matches the
supported proof boundary.
The retained global maintenance matrix used:
```text
GCC/G++
Vulkan disabled
selected concurrent targets
deterministic repetitions
```
and completed the retained 14/14 target matrix plus 220 repetitions.
The narrow suppression file is:
```text
tests/tsan-opencv.supp
```
It covers external non-instrumented OpenCV/TBB objects only.
It must not suppress Lardon3D frames.
Therefore the correct retained claim is not "TSan proves all concurrency".
It is:
```text
portable project concurrency matrix passed under the documented qualification
external OpenCV/TBB reports are narrowly qualified
Vulkan concurrency has a separate validation boundary
```
## What TSan does not prove
TSan is useful for instrumented conflicting memory access and some
synchronization misuse.
It does not prove absence of:
- deadlock;
- lost wakeup caused by incorrect predicate design;
- lifetime bugs outside the exercised paths;
- races hidden inside non-instrumented external libraries;
- Vulkan driver/runtime correctness;
- scientific determinism.
Lock-order review, ownership reasoning and deterministic tests remain required.
## Sanitizer command policy
Do not encode fixed `-j8` as canonical validation.
Example configuration:
```sh ```sh
# Build TSan CC=gcc CXX=g++ meson setup build-tsan -Db_sanitize=thread -Db_lundef=false -Dvulkan_orb=disabled
CC=clang meson setup build-tsan --wipe -Db_sanitize=thread -Db_lundef=false meson compile -C build-tsan
meson compile -C build-tsan -j8
meson test -C build-tsan --print-errorlogs meson test -C build-tsan --print-errorlogs
``` ```
TSan détecte automatiquement : Use host-aware compile/test parallelism unless the proof itself requires
serialization.
- les accès concurrents conflictuels instrumentés ; Do not repeatedly wipe an unchanged TSan tree.
- certaines utilisations incohérentes des primitives de synchronisation.
Il ne prouve pas l'absence de deadlock, de signal perdu ou de bug dans une ## Concurrency review checklist
bibliothèque non instrumentée. Les invariants de lifetime et d'ordre de locks
restent donc soumis aux tests déterministes et à la revue.
### Preuve TSan globale courante Before closing a concurrency-sensitive change, verify:
La matrice fraîche emploie GCC/G++ 16.2.1 et désactive explicitement Vulkan. - ncurses remains main-thread-only;
Elle passe 14/14 cibles couvrant Task, Project, Queue, Governor, registre/leases - every shared mutable field has an explicit synchronization owner;
SSD, contrôleur SSD, observateur/TUI async, Candidate, Visual Index, Feature, - condition predicates are checked in loops;
Matcher et GV, puis 220/220 répétitions déterministes : **234/234** au total. - lock order remains consistent;
- no Task uses forced asynchronous cancellation;
- Queue callbacks have an active reservation;
- internal participants stay within the admitted contract;
- all children join on every exit path;
- owner-only publication remains ordered where required;
- project-close ordering prevents DB use-after-close;
- SSD lease ownership remains exact;
- Task userdata outlives terminal notification;
- appropriate deterministic concurrency tests pass;
- portable TSan qualification is preserved;
- Vulkan validation is reported separately;
- ASan/UBSan is run when the change also affects lifetime/memory.
La seule liste de suppressions est `tests/tsan-opencv.supp`, limitée aux objets ## Current retained evidence
partagés externes non instrumentés `libopencv_features.so`,
`libopencv_core.so` et `libtbb.so`. Elle ne masque aucune frame Lardon3D. Les
warnings GCC `-Wmaybe-uninitialized` des contrôles OpenCV Feature/SIFT sont
classés non matériels : le callback fournit une Task non nulle et le helper
initialise la structure avant toute autre sortie d'échec. Les warnings OpenCV
du build GV appartiennent aux headers externes.
Cette preuve TSan ne vaut pas validation de concurrence Vulkan. Le backend The canonical global-maintenance record is:
ORB Vulkan réel est couvert séparément par le build Clang Vulkan-on 939/939,
la suite 65/65 et ses tests de backend/handle/publication ; cette séparation
doit rester explicite dans tout rapport.
## Checklist de concurrence ```text
docs/architecture/global_maintenance_audit.md
GLOBAL_MAINTENANCE_AUDIT=PASS/FROZEN
```
Avant de livrer un ticket touchant la concurrence : The current A6000 checkpoint is later:
- [ ] Toutes les variables partagées sont protégées par un mutex ```text
- [ ] Les mutex sont toujours libérés (même en cas d'erreur) real-a6000-pre-sfm-2026-09-02
- [ ] Les variables de condition sont vérifiées dans une boucle `while` REAL_A6000_PRE_SFM=PASS/FROZEN
- [ ] Aucun appel ncurses depuis un worker ```
- [ ] L'annulation des Tasks est coopérative (pas de `pthread_cancel`)
- [ ] TSan ne signale aucune erreur The later A6000 proof exercised current bounded parallel paths through selected
- [ ] Le build ASan ne signale aucune fuite mémoire liée aux threads Feature batch, Candidate, Matcher, Geometric Verifier v3 and Tracks without
changing the historical TSan qualification.
Historical evidence remains historical; new changes require validation scoped
to their actual concurrency surface.
## Summary
```text
NCURSES_OWNER=MAIN_THREAD_ONLY
ACTIVE_HEAVY_QUEUE_CALLBACKS=1
TASK_CANCELLATION=COOPERATIVE
INTERNAL_PARALLELISM=BOUNDED
PER_ITEM_ATOMICITY_REQUIRES_CROSS_ITEM_SERIALISM=NO
OWNER_ONLY_PUBLICATION_REQUIRES_SERIAL_PREPARATION=NO
TSAN_PORTABLE_PROJECT_MATRIX=QUALIFIED_PASS
TSAN_EXTERNAL_OPENCV_TBB=QUALIFIED
VULKAN_CONCURRENCY_VALIDATION=SEPARATE
NO_FIXED_GLOBAL_J8=YES
NO_REPEATED_UNCHANGED_WIPE=YES
RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
SERIALISM_REQUIRES_PROOF=CANONICAL
```

View file

@ -1,136 +1,298 @@
# Procédures de test # Testing
## Vue d'ensemble ## Status
Lardon3D utilise le framework de test intégré à Meson. Chaque module possède ```text
un fichier de test dans `tests/` correspondant au module testé. DOCUMENTATION_LANGUAGE=ENGLISH
TEST_POLICY=HOST_AWARE
REPEATED_UNCHANGED_EXPENSIVE_VALIDATION=AVOID
TSAN_OPEN_CV_TBB_QUALIFICATION=REQUIRED
VULKAN_CONCURRENCY_VALIDATION=SEPARATE
```
## Lancer les tests Lardon3D uses Meson's test runner. Tests live under `tests/` and combine unit,
integration, persistence, restart, resource and real-path validation.
## Normal commands
Run the configured suite:
```sh ```sh
# Tous les tests
meson test -C build --print-errorlogs meson test -C build --print-errorlogs
```
# Un test spécifique Run one named test:
meson test -C build test_task_queue --print-errorlogs
# Tests avec verbose ```sh
meson test -C build <test-name> --print-errorlogs
```
Verbose execution:
```sh
meson test -C build -v --print-errorlogs meson test -C build -v --print-errorlogs
```
# Réexécuter uniquement les tests échoués Re-run failures only:
```sh
meson test -C build --reprint=failed meson test -C build --reprint=failed
``` ```
## Structure des tests Use the names registered by the current `meson.build`; this document does not
maintain a second authoritative list of every test target.
## Validation policy
Validation must match the change.
A documentation-only change normally requires:
```text ```text
tests/ git diff --check
├── test_task_queue.c # tests de la file de tâches targeted content checks
├── test_task.c # tests du module task targeted link/authority review
├── test_resource_governor.c # tests du gouverneur
├── test_hardware_profile.c # tests du profil matériel
├── test_import.c # tests de l'import
├── test_project.c # tests des projets
└── test_*.c # autres modules
``` ```
## Écrire un test It does not justify wiping and rebuilding unchanged code.
```c A code change normally requires, in increasing scope:
#include <glib.h>
#include "lardon3d/task.h"
void test_task_create(void) { ```text
task_estimate_t est = { targeted build
.ram_bytes = 1024 * 1024, targeted tests
.gpu_bytes = 0, broader affected suite
.cpu_weight = 1, sanitizer or concurrency validation when relevant
.io_weight = 0, full suite when the change or release boundary justifies it
.batch_size = 10,
.batch_max = 100
};
task_t *t = task_create("test", &est, NULL, NULL);
g_assert_nonnull(t);
g_assert_cmpint(task_get_state(t), ==, TASK_STATE_IDLE);
task_destroy(t);
}
int main(int argc, char **argv) {
g_test_init(&argc, &argv, NULL);
g_test_add_func("/task/create", test_task_create);
return g_test_run();
}
``` ```
## Conventions Do not repeatedly rerun an unchanged expensive suite between documentation
edits merely to create activity.
1. **Préfixe `test_`** : chaque fonction de test porte le préfixe `test_`. ## Host-aware parallelism
2. **Chemin hiérarchique** : le nom du test suit le pattern `/module/action`.
3. **Asserts GLib** : utiliser `g_assert_*` pour les vérifications.
4. **Nettoyage** : chaque test libère toutes ses ressources.
5. **Isolation** : un test ne dépend pas de l'état d'un autre test.
6. **Déterminisme** : les tests ne dépendent pas de l'heure, du filesystem
ou de l'état réseau (sauf test d'import).
## Commentaires source Build and test parallelism are host-aware.
Les commentaires documentent le pourquoi et les contrats non évidents : Do not encode a project-wide fixed `-j8`, `--num-processes 1`, or equivalent
invariants, propriété et durée de vie, persistance, ainsi que limites et constant as canonical policy.
frontières de ressources. Les API publiques documentent leurs contrats non
évidents. Ils ne paraphrasent pas le code ligne par ligne et sont mis à jour
avec tout changement de comportement.
## Tests unitaires vs tests d'intégration The correct width depends on the current machine, interactive reserve, memory,
toolchain and workload. Use all safe useful host capacity while preserving the
defined interactive reserve.
`test-visual-index` couvre les descriptors synthétiques, le retrieval ORB réel, ```text
les filtres inter-ScanSets, quatre queries concurrentes, la corruption/absence/ RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
troncature d'un segment et 4 000 Feature Sets synthétiques. Le scénario de SERIALISM_REQUIRES_PROOF=CANONICAL
reprise `visual_index.update` est exercé dans `test-feature-task`. ```
| Type | Portée | Fichier | If a temporary validation must be serialized for determinism, diagnosis or a
|---|---|---| known tool limitation, label that serialization as test-specific evidence
| Unitaire | Un module isolé | `tests/test_<module>.c` | rather than a global default.
| Intégration | Interaction entre modules | `tests/test_<module>.c` avec dépendances réelles |
## Validation par ticket ## Fresh build policy
Avant de livrer un ticket, exécuter la séquence complète : Do not use `meson setup --wipe` by default.
Prefer:
```sh ```sh
# 1. Build clean meson setup --reconfigure build
CC=clang meson setup build --wipe meson compile -C build
meson compile -C build -j8
# 2. Tests
meson test -C build --print-errorlogs
# 3. Style
git diff --check
# 4. Si mémoire/concurrence touchés
CC=clang meson setup build-asan --wipe -Db_sanitize=address,undefined
meson compile -C build-asan -j8
meson test -C build-asan --print-errorlogs
# 5. Si concurrence touchée
CC=clang meson setup build-tsan --wipe -Db_sanitize=thread -Db_lundef=false
meson compile -C build-tsan -j8
meson test -C build-tsan --print-errorlogs
``` ```
## Dépannage Create or wipe a build directory when the configuration genuinely needs a
fresh environment, for example:
### Test qui échoue en ASan ```text
different sanitizer set
portable Vulkan-off proof
Vulkan-on proof
compiler-family change
known stale/corrupt build directory
release-grade clean proof
```
Vérifier les durées de vie des allocations. Ne jamais libérer un objet puis Repeated wipes of the same unchanged configuration waste time and invalidate
y accéder. Vérifier que chaque `task_destroy()` est appelée. incremental-build advantages.
### Test qui échoue en TSan ## Sanitizers
Vérifier que toutes les variables partagées sont protégées par un mutex. ### ASan / UBSan
Vérifier que ncurses est utilisé uniquement depuis le thread principal.
### Test qui échoue uniquement en release For memory, lifetime, ownership or undefined-behavior changes, use a dedicated
sanitizer build.
Vérifier les assertions et les overflow arithmétiques. Compiler avec Example configuration:
`-fsanitize=undefined` pour détecter les comportements indéfinis.
```sh
CC=clang CXX=clang++ meson setup build-asan -Db_sanitize=address,undefined
meson compile -C build-asan
meson test -C build-asan --print-errorlogs
```
Reconfigure or wipe only when the existing sanitizer directory does not match
the requested configuration.
### LeakSanitizer qualification
The retained global maintenance evidence must not be summarized as
`LSan 64/64`.
The full first leak-enabled run exposed an externally attributed OpenCL loader
leak and two timeout anomalies. The retained qualified result is:
```text
ASan/UBSan full suite: PASS with detect_leaks=0
proved subset without the external loader: LSan PASS
full leak-enabled suite: not an unqualified PASS
```
Preserve that distinction in future reports unless new evidence supersedes it.
## ThreadSanitizer
Concurrency changes require TSan where the instrumented boundary is meaningful.
The retained global maintenance TSan proof used GCC/G++ with Vulkan disabled
and covered the selected concurrent targets plus deterministic repetitions.
The only retained suppression file is:
```text
tests/tsan-opencv.supp
```
Its purpose is limited to external OpenCV/TBB objects. It must not suppress
Lardon3D frames.
Therefore never report a blanket statement such as:
```text
TSan proves the entire Vulkan build race-free
```
The valid qualification is:
```text
project concurrent paths covered by the retained portable TSan matrix
external OpenCV/TBB reports qualified by the narrow suppression boundary
Vulkan concurrency validated separately
```
## Vulkan validation
ORB Vulkan uses a separate validation boundary.
The retained global maintenance evidence includes a Vulkan-on build and suite
on the real Radeon 780M/RADV host, plus dedicated backend/handle/publication
tests.
That evidence is not interchangeable with TSan.
SIFT/RootSIFT feasibility results did not establish a production GPU backend;
do not turn feasibility checks into production validation claims.
## Determinism and repetition
Repeat tests when repetition proves something specific:
```text
deterministic restart
race sensitivity
resource adaptation
ordering stability
flaky regression reproduction
```
Do not repeat unchanged tests without a stated purpose.
When repetition is the evidence, record:
```text
exact test/corpus
run count
relevant configuration
success/failure count
digest or invariant when applicable
```
## Test isolation
Tests should:
- own and clean up their temporary resources;
- avoid depending on another test's execution order;
- avoid network state unless the test explicitly owns that dependency;
- use synthetic/private Resource snapshots where the test is about deterministic
policy rather than live host telemetry;
- avoid changing global process state without restoring it.
OpenCV thread configuration is process-wide and must be restored on every exit
path in tests that change it.
## Public-header validation
When a public C header changes, run a standalone C17 syntax probe in addition to
normal build coverage.
Conceptually:
```sh
cc -x c -std=c17 -fsyntax-only -Iinclude -include lardon3d/<header>.h /dev/null
```
Use the current supported compiler matrix when the change affects ABI or
C/C++ interoperability.
## Source comments
Source comments explain non-obvious contracts:
```text
invariants
ownership and lifetime
persistence ordering
resource boundaries
recovery behavior
scientific constraints
```
They should not paraphrase obvious code line by line.
Repository source comments are English.
## Current retained global maintenance evidence
The canonical detailed record is:
```text
docs/architecture/global_maintenance_audit.md
GLOBAL_MAINTENANCE_AUDIT=PASS/FROZEN
```
That historical checkpoint includes fresh portable/Vulkan builds, full suites,
sanitizer work, TSan work, public-header probes, ABI/application-link checks and
independent review.
It remains historical evidence. New changes require only the validation
appropriate to the changed surface unless a new global checkpoint is being
created.
## Ticket closure checklist
Before closing a code ticket:
- confirm the requested scope only was changed;
- run `git diff --check`;
- run targeted tests for changed behavior;
- run the affected broader suite when justified;
- run ASan/UBSan for memory/lifetime-sensitive changes;
- run TSan for concurrency-sensitive project code when applicable;
- keep Vulkan validation separate from portable TSan claims;
- preserve exact external-library qualifications;
- avoid fixed host-parallelism constants;
- avoid repeated unchanged clean builds or suites;
- report what actually ran, not a stronger claim.
For documentation-only remediation, use documentation checks rather than
rebuilding unchanged production code.