docs: reconcile development validation guidance
This commit is contained in:
parent
4278d124c2
commit
7574362ba9
3 changed files with 853 additions and 434 deletions
|
|
@ -1,193 +1,293 @@
|
||||||
# Instructions de build
|
# Build
|
||||||
|
|
||||||
## Prérequis
|
## Status
|
||||||
|
|
||||||
- **OS** : Linux (testé sur distributions récentes)
|
```text
|
||||||
- **Compilateur** : Clang (recommandé) ou GCC
|
BUILD_SYSTEM=MESON_NINJA
|
||||||
- **Système de build** : Meson + Ninja
|
PUBLIC_API_LANGUAGE=C17
|
||||||
- **Dépendances principales** : ncursesw, SQLite, OpenSSL, GIO/GLib, OpenCV,
|
IMPLEMENTATION_LANGUAGES=C17_CXX17
|
||||||
LibRaw, libexif, libpng, libdeflate, Ceres ; Vulkan reste optionnel
|
BUILD_PARALLELISM=HOST_AWARE
|
||||||
- **Langages** : API publiques C17 et implémentation mixte C17/C++17
|
RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
|
||||||
|
SERIALISM_REQUIRES_PROOF=CANONICAL
|
||||||
|
```
|
||||||
|
|
||||||
## Bootstrap des outils
|
Meson is the build-system authority. Do not duplicate dependency-version truth
|
||||||
|
in this document when `meson.build` already enforces it.
|
||||||
|
|
||||||
Les commandes ci-dessous installent seulement le compilateur, Meson/Ninja,
|
## Requirements
|
||||||
`pkg-config` et ncurses. Les bibliothèques listées plus haut doivent aussi être
|
|
||||||
disponibles dans les versions acceptées par `meson.build`; Meson reste la source
|
Lardon3D targets Linux.
|
||||||
de vérité et refuse explicitement une dépendance absente ou incompatible.
|
|
||||||
|
Primary toolchain:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Clang or GCC
|
||||||
|
Meson
|
||||||
|
Ninja
|
||||||
|
pkg-config
|
||||||
|
```
|
||||||
|
|
||||||
|
Major dependencies currently include ncursesw, SQLite, OpenSSL, GIO/GLib,
|
||||||
|
OpenCV, LibRaw, libexif, libpng, libdeflate and Ceres. Vulkan remains optional
|
||||||
|
at configuration level.
|
||||||
|
|
||||||
|
Public APIs are C17. Implementation is mixed C17/C++17.
|
||||||
|
|
||||||
|
## Bootstrap examples
|
||||||
|
|
||||||
|
These commands install only the basic compiler/build front end; Meson remains
|
||||||
|
authoritative for the complete dependency set.
|
||||||
|
|
||||||
|
Debian/Ubuntu:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
# Debian / Ubuntu
|
|
||||||
sudo apt install clang meson ninja-build libncursesw5-dev pkg-config
|
sudo apt install clang meson ninja-build libncursesw5-dev pkg-config
|
||||||
|
```
|
||||||
|
|
||||||
# Fedora
|
Fedora:
|
||||||
|
|
||||||
|
```sh
|
||||||
sudo dnf install clang meson ninja-build ncurses-devel pkg-config
|
sudo dnf install clang meson ninja-build ncurses-devel pkg-config
|
||||||
|
```
|
||||||
|
|
||||||
# Arch
|
Arch Linux:
|
||||||
|
|
||||||
|
```sh
|
||||||
sudo pacman -S clang meson ninja ncurses pkgconf
|
sudo pacman -S clang meson ninja ncurses pkgconf
|
||||||
```
|
```
|
||||||
|
|
||||||
## Build standard
|
## Standard build
|
||||||
|
|
||||||
|
First configuration:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
# Première configuration
|
CC=clang CXX=clang++ meson setup build
|
||||||
CC=clang meson setup build
|
```
|
||||||
|
|
||||||
# Arbre existant
|
Existing tree:
|
||||||
|
|
||||||
|
```sh
|
||||||
meson setup --reconfigure build
|
meson setup --reconfigure build
|
||||||
meson compile -C build -j8
|
meson compile -C build
|
||||||
```
|
```
|
||||||
|
|
||||||
### Options utiles
|
Do not hard-code `-j8` as project policy.
|
||||||
|
|
||||||
|
Meson/Ninja should use host-appropriate parallelism unless a specific
|
||||||
|
validation has a reason to constrain it.
|
||||||
|
|
||||||
|
## Build parallelism policy
|
||||||
|
|
||||||
|
The build is not governed by a portable fixed job count.
|
||||||
|
|
||||||
|
Canonical policy:
|
||||||
|
|
||||||
|
```text
|
||||||
|
preserve the interactive host reserve
|
||||||
|
then use maximum safe useful throughput
|
||||||
|
```
|
||||||
|
|
||||||
|
A reference host measurement such as 8 or 12 useful jobs is evidence for that
|
||||||
|
host at that time, not a repository constant.
|
||||||
|
|
||||||
|
If memory-heavy compilation or another active workload creates pressure,
|
||||||
|
reduce build width for that run. Do not convert the temporary reduction into a
|
||||||
|
global documentation rule.
|
||||||
|
|
||||||
|
## Reconfigure versus wipe
|
||||||
|
|
||||||
|
Prefer incremental reuse:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
# Build de debug (défaut)
|
meson setup --reconfigure build
|
||||||
meson setup build --wipe
|
meson compile -C build
|
||||||
|
|
||||||
# Build de release
|
|
||||||
meson setup build --wipe --buildtype=release
|
|
||||||
|
|
||||||
# Build avec optimisations aggressive
|
|
||||||
meson setup build --wipe --buildtype=release -Db_lto=true
|
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Use `--wipe` only when a fresh configuration is actually required, such as:
|
||||||
|
|
||||||
|
- switching sanitizer configuration in the same directory;
|
||||||
|
- changing compiler family;
|
||||||
|
- changing a configuration whose cached state cannot be reused safely;
|
||||||
|
- reproducing a clean release/global-maintenance proof;
|
||||||
|
- recovering from a stale or corrupt build directory.
|
||||||
|
|
||||||
|
A normal edit/test loop should not wipe the build tree repeatedly.
|
||||||
|
|
||||||
|
## Release build
|
||||||
|
|
||||||
|
Use an explicit release directory or deliberate reconfiguration.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
CC=clang CXX=clang++ meson setup build-release --buildtype=release
|
||||||
|
meson compile -C build-release
|
||||||
|
```
|
||||||
|
|
||||||
|
For LTO:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
CC=clang CXX=clang++ meson setup build-release-lto --buildtype=release -Db_lto=true
|
||||||
|
meson compile -C build-release-lto
|
||||||
|
```
|
||||||
|
|
||||||
|
Separate directories avoid destroying a useful incremental debug tree.
|
||||||
|
|
||||||
|
## Vulkan configuration
|
||||||
|
|
||||||
|
Portable CPU-only proof:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
CC=clang CXX=clang++ meson setup build-portable -Dvulkan_orb=disabled
|
||||||
|
meson compile -C build-portable
|
||||||
|
```
|
||||||
|
|
||||||
|
Vulkan-enabled proof:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
CC=clang CXX=clang++ meson setup build-vulkan -Dvulkan_orb=enabled
|
||||||
|
meson compile -C build-vulkan
|
||||||
|
```
|
||||||
|
|
||||||
|
A Vulkan-on build is not automatically a proof that every scientific path uses
|
||||||
|
or should use the GPU.
|
||||||
|
|
||||||
|
Current production GPU promotion remains limited by each subsystem's validated
|
||||||
|
backend contract.
|
||||||
|
|
||||||
## Validation
|
## Validation
|
||||||
|
|
||||||
|
Normal configured tests:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
# Tests unitaires
|
|
||||||
meson test -C build --print-errorlogs
|
meson test -C build --print-errorlogs
|
||||||
|
|
||||||
# Vérification du style (whitespace)
|
|
||||||
git diff --check
|
|
||||||
|
|
||||||
# Vérification autonome d'un header C public modifié
|
|
||||||
cc -x c -std=c17 -fsyntax-only -Iinclude \
|
|
||||||
-include lardon3d/<header>.h /dev/null
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Preuve fraîche de maintenance globale — 1er septembre 2026
|
Whitespace/style boundary:
|
||||||
|
|
||||||
Le [registre canonique](../architecture/global_maintenance_audit.md) conserve
|
|
||||||
le détail et les qualifications. Les résultats reproductibles acquis sont :
|
|
||||||
|
|
||||||
| Configuration fraîche | Compilateurs/options | Build | Suite |
|
|
||||||
| --- | --- | ---: | ---: |
|
|
||||||
| portable | Clang/Clang++ 22.1.8, C17/C++17, `-Dvulkan_orb=disabled` | 931/931 | 64/64 sériel |
|
|
||||||
| Vulkan | Clang/Clang++ 22.1.8, C17/C++17, `-Dvulkan_orb=enabled` | 939/939 | 65/65 sériel |
|
|
||||||
| ASan/UBSan portable | Clang/Clang++ 22.1.8, `address,undefined` | graphe complet | 64/64 avec LSan désactivé après attribution externe |
|
|
||||||
| TSan portable | GCC/G++ 16.2.1, Vulkan désactivé | cibles concurrentes | 14/14 + 220 répétitions |
|
|
||||||
|
|
||||||
La suite Vulkan comprend `orb-vulkan-backend` sur la Radeon 780M RADV PHOENIX
|
|
||||||
réelle. La cible de feasibility SIFT/RootSIFT, non enregistrée dans la suite,
|
|
||||||
a été compilée/exécutée séparément : zéro divergence de décision Lowe mais des
|
|
||||||
divergences d'index et de bits de distance, donc aucune promotion en backend
|
|
||||||
production. Les probes stricts GCC/Clang C17+C++17 passent 76/76 sur les
|
|
||||||
19 headers publics modifiés ou nouveaux, ainsi que le fixture ABI, le lien
|
|
||||||
application et `git diff --check`; `scan3d/` reste intact.
|
|
||||||
|
|
||||||
L'unique revue finale indépendante GPT-5.6 SOL/ULTRA a conclu PASS sans finding
|
|
||||||
bloquant. Elle a indépendamment rejoué le build portable, la suite complète
|
|
||||||
64/64, une matrice focalisée 15/15, les 76/76 probes de headers, l'ABI, les
|
|
||||||
négatifs de seams production, le SHA-256 du manifest GV retenu et le diff-check.
|
|
||||||
Le statut canonique est donc `GLOBAL_MAINTENANCE_AUDIT=PASS/FROZEN` ; les
|
|
||||||
qualifications sanitizer ci-dessous restent néanmoins partie de la preuve.
|
|
||||||
|
|
||||||
Une validation post-freeze a ensuite attribué le délai intermittent de
|
|
||||||
`test-feature-task` à la capture de la télémétrie hôte réelle par ses Governors
|
|
||||||
synthétiques. Le fixture utilise maintenant un `ResourceSnapshot` complet,
|
|
||||||
privé, par Governor et compilé pour cette seule cible ; production continue de
|
|
||||||
lire la télémétrie réelle et n'exporte aucun seam. Après correction du second
|
|
||||||
Governor relevé en revue, Feature passe 100/100, la matrice ordonnée 4/4 et les
|
|
||||||
suites finales portable/Vulkan 64/64 et 65/65 ; ASan/UBSan ciblé avec
|
|
||||||
`detect_leaks=0` et TSan passent. Le registre canonique conserve la régression
|
|
||||||
charge 5 `WAIT`/charge 0 `START` et la qualification exacte. Un timeout `task`
|
|
||||||
isolé dans une suite normale mixte après reconstruction large reste
|
|
||||||
non reproductible : le ciblé immédiat et sa matrice de revue 100/100 passent,
|
|
||||||
sans modification de Task ni de son timeout.
|
|
||||||
|
|
||||||
La première suite LSan complète est volontairement conservée comme non-PASS :
|
|
||||||
57 OK, 6 FAIL et 1 timeout. Cinq échecs partagent exactement la fuite externe
|
|
||||||
OpenCL de 3 808 octets/68 allocations ; les deux anomalies de 30 s n'ont aucun
|
|
||||||
diagnostic sanitizer. Le délai Feature, absent du suivi initial, a ensuite été
|
|
||||||
reproduit et corrigé comme décrit ci-dessus ; le délai Task reste non
|
|
||||||
reproductible. La suite entière passe 64/64 avec ASan/UBSan actifs et
|
|
||||||
`detect_leaks=0`, tandis qu'un sous-ensemble prouvé sans loader OpenCV/OpenCL
|
|
||||||
passe 20/20 avec LSan actif. Il est donc incorrect de résumer cette preuve par
|
|
||||||
« LSan 64/64 ».
|
|
||||||
|
|
||||||
Le log Clang complet a aussi été audité. La conversion publique Sparse SfM
|
|
||||||
`uint32_t → int` était matérielle et a été corrigée avec validation ciblée ; les
|
|
||||||
autres émissions sont soit des conversions baseline déjà bornées, soit des
|
|
||||||
tests/benchmarks, soit des headers OpenCV/Ceres externes. Les emplacements et
|
|
||||||
justifications exacts restent centralisés dans le registre afin de ne pas
|
|
||||||
dupliquer une seconde liste normative ici.
|
|
||||||
|
|
||||||
## Build ASan/UBSan (debug mémoire)
|
|
||||||
|
|
||||||
À exécuter pour tout ticket touchant la mémoire, les durées de vie ou les
|
|
||||||
allocations :
|
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
CC=clang meson setup build-asan --wipe \
|
git diff --check
|
||||||
-Db_sanitize=address,undefined
|
```
|
||||||
meson compile -C build-asan -j8
|
|
||||||
|
Public C header probe:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cc -x c -std=c17 -fsyntax-only -Iinclude -include lardon3d/<header>.h /dev/null
|
||||||
|
```
|
||||||
|
|
||||||
|
Use `docs/development/testing.md` for sanitizer and validation policy.
|
||||||
|
|
||||||
|
## ASan / UBSan build
|
||||||
|
|
||||||
|
Example dedicated directory:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
CC=clang CXX=clang++ meson setup build-asan -Db_sanitize=address,undefined
|
||||||
|
meson compile -C build-asan
|
||||||
meson test -C build-asan --print-errorlogs
|
meson test -C build-asan --print-errorlogs
|
||||||
```
|
```
|
||||||
|
|
||||||
## Build TSan (concurrence)
|
Do not claim an unqualified full LeakSanitizer pass from the retained global
|
||||||
|
maintenance checkpoint. The external OpenCL loader qualification documented in
|
||||||
|
the canonical audit remains part of that evidence.
|
||||||
|
|
||||||
À exécuter pour tout ticket touchant la concurrence (pthread, mutex,
|
## TSan build
|
||||||
variables de condition, états partagés) :
|
|
||||||
|
Use TSan only with the configuration that matches the intended proof.
|
||||||
|
|
||||||
|
The retained global maintenance concurrency proof used GCC/G++ with Vulkan
|
||||||
|
disabled, because the project TSan matrix and the Vulkan runtime validation are
|
||||||
|
separate evidence boundaries.
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
CC=clang meson setup build-tsan --wipe \
|
CC=gcc CXX=g++ meson setup build-tsan -Db_sanitize=thread -Db_lundef=false -Dvulkan_orb=disabled
|
||||||
-Db_sanitize=thread \
|
meson compile -C build-tsan
|
||||||
-Db_lundef=false
|
|
||||||
meson compile -C build-tsan -j8
|
|
||||||
meson test -C build-tsan --print-errorlogs
|
meson test -C build-tsan --print-errorlogs
|
||||||
```
|
```
|
||||||
|
|
||||||
## Variables d'environnement
|
The exact target subset, suppression qualification and repetition evidence are
|
||||||
|
documented in `docs/development/concurrency.md` and the global maintenance
|
||||||
|
audit.
|
||||||
|
|
||||||
| Variable | Description |
|
## Current retained maintenance checkpoint
|
||||||
|---|---|
|
|
||||||
| `CC` | Compilateur C (défaut : gcc) |
|
|
||||||
| `CFLAGS` | Drapeaux de compilation supplémentaires |
|
|
||||||
| `LDFLAGS` | Drapeaux de liaison supplémentaires |
|
|
||||||
|
|
||||||
## Structure du build
|
The canonical detailed evidence is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
docs/architecture/global_maintenance_audit.md
|
||||||
|
GLOBAL_MAINTENANCE_AUDIT=PASS/FROZEN
|
||||||
|
```
|
||||||
|
|
||||||
|
The 2026-09-01 checkpoint retained:
|
||||||
|
|
||||||
|
```text
|
||||||
|
portable Clang/Clang++ build and suite
|
||||||
|
Vulkan-on Clang/Clang++ build and suite
|
||||||
|
ASan/UBSan qualified run
|
||||||
|
portable GCC/G++ TSan matrix
|
||||||
|
public-header C17/C++17 probes
|
||||||
|
ABI and application-link checks
|
||||||
|
independent review
|
||||||
|
```
|
||||||
|
|
||||||
|
Those exact historical counts belong to the audit and should not be duplicated
|
||||||
|
as a new current build contract.
|
||||||
|
|
||||||
|
## Build directory layout
|
||||||
|
|
||||||
|
A configured Meson tree typically contains:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
build/
|
build/
|
||||||
├── src/ # objets et binaires
|
src/
|
||||||
├── tests/ # binaires de tests
|
tests/
|
||||||
└── compile_commands.json # pour LSP / clangd
|
compile_commands.json
|
||||||
```
|
```
|
||||||
|
|
||||||
## Dépannage
|
Exact generated layout is Meson/Ninja output and may evolve.
|
||||||
|
|
||||||
### Erreur : ncursesw introuvable
|
## Environment
|
||||||
|
|
||||||
|
Common variables include:
|
||||||
|
|
||||||
|
| Variable | Purpose |
|
||||||
|
| --- | --- |
|
||||||
|
| `CC` | C compiler |
|
||||||
|
| `CXX` | C++ compiler |
|
||||||
|
| `CFLAGS` | additional C flags |
|
||||||
|
| `CXXFLAGS` | additional C++ flags |
|
||||||
|
| `LDFLAGS` | additional linker flags |
|
||||||
|
|
||||||
|
Prefer Meson options for project features rather than ad-hoc environment flags
|
||||||
|
that make builds difficult to reproduce.
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
Check ncursesw discovery:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
# Vérifier l'installation
|
|
||||||
pkg-config --libs ncursesw
|
pkg-config --libs ncursesw
|
||||||
# Si absent, installer le paquet de développement ncursesw
|
|
||||||
```
|
```
|
||||||
|
|
||||||
### Erreur : clang introuvable
|
If Clang is unavailable, GCC is supported where the current Meson checks allow
|
||||||
|
it.
|
||||||
|
|
||||||
```sh
|
For a slow build, first preserve the existing build tree and let Ninja use
|
||||||
# Utiliser gcc en alternative
|
normal host-aware scheduling. Reduce concurrency only when actual host pressure
|
||||||
meson setup build --wipe
|
or another active workload justifies it.
|
||||||
# ou installer clang
|
|
||||||
sudo apt install clang
|
`ccache` may be used when available, but it is optional operational tooling and
|
||||||
|
not part of scientific identity.
|
||||||
|
|
||||||
|
## Rules
|
||||||
|
|
||||||
|
```text
|
||||||
|
NO_FIXED_GLOBAL_J8=YES
|
||||||
|
NO_REPEATED_UNCHANGED_WIPE=YES
|
||||||
|
HOST_AWARE_BUILD_PARALLELISM=YES
|
||||||
```
|
```
|
||||||
|
|
||||||
### Build lent
|
Build configuration is operational state. It must not silently redefine
|
||||||
|
scientific formats, fingerprints or persistence contracts.
|
||||||
```sh
|
|
||||||
# Réduire la parallélisation
|
|
||||||
meson compile -C build -j4
|
|
||||||
# ou utiliser ccache
|
|
||||||
CC="ccache clang" meson setup build --wipe
|
|
||||||
```
|
|
||||||
|
|
|
||||||
|
|
@ -1,255 +1,412 @@
|
||||||
# Règles de concurrence
|
# Concurrency
|
||||||
|
|
||||||
## Vue d'ensemble
|
## Status
|
||||||
|
|
||||||
Lardon3D utilise un modèle de concurrence à thread unique pour ncurses
|
|
||||||
et un modèle multi-thread pour le traitement. La séparation est stricte :
|
|
||||||
le thread ncurses ne fait jamais de travail métier, et les workers ne
|
|
||||||
touchent jamais ncurses.
|
|
||||||
|
|
||||||
## Modèle de concurrence
|
|
||||||
|
|
||||||
```text
|
```text
|
||||||
Thread principal (ncurses)
|
NCURSES_OWNER=MAIN_THREAD_ONLY
|
||||||
├── Gestion des entrées
|
ACTIVE_HEAVY_QUEUE_CALLBACKS=1
|
||||||
├── Affichage TUI
|
TASK_CANCELLATION=COOPERATIVE
|
||||||
└── Orchestration
|
INTERNAL_PARALLELISM=BOUNDED
|
||||||
|
OWNER_ONLY_PUBLICATION=CANONICAL_WHERE_REQUIRED
|
||||||
|
|
||||||
Worker thread
|
TSAN_PORTABLE_PROJECT_MATRIX=QUALIFIED_PASS
|
||||||
├── Exécution des tâches
|
TSAN_EXTERNAL_OPENCV_TBB=QUALIFIED
|
||||||
├── Calculs métier
|
VULKAN_CONCURRENCY_VALIDATION=SEPARATE
|
||||||
└── Écritures de résultats
|
|
||||||
|
|
||||||
SSD operation thread (0 ou 1, joinable)
|
RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
|
||||||
└── Un poll ou contrôle UDisks synchrone borné, sans ncurses ni Task
|
SERIALISM_REQUIRES_PROOF=CANONICAL
|
||||||
```
|
```
|
||||||
|
|
||||||
## Règles fondamentales
|
Lardon3D separates UI ownership from heavy processing.
|
||||||
|
|
||||||
### 1. ncurses appartient au thread principal
|
The main thread owns ncurses. The Task Queue owns one active heavy callback.
|
||||||
|
Individual validated Task Kinds may create bounded internal participants inside
|
||||||
|
that callback.
|
||||||
|
|
||||||
```c
|
An SSD controller operation may also use at most one bounded joinable operation
|
||||||
// ✅ Correct : appel depuis le thread principal
|
thread under its own ownership contract.
|
||||||
mvprintw(0, 0, "Progression: %d%%", progress);
|
|
||||||
|
|
||||||
// ❌ Interdit : appel depuis un worker
|
## Execution model
|
||||||
// mvprintw() dans un thread secondaire
|
|
||||||
|
```text
|
||||||
|
main thread
|
||||||
|
input
|
||||||
|
ncurses
|
||||||
|
TUI orchestration
|
||||||
|
|
||||||
|
Task Queue worker
|
||||||
|
one active heavy callback
|
||||||
|
admitted Task sequence
|
||||||
|
optional bounded internal participants
|
||||||
|
deterministic owner publication
|
||||||
|
|
||||||
|
SSD operation thread
|
||||||
|
zero or one bounded joinable controller operation
|
||||||
|
no ncurses
|
||||||
|
no Task callback
|
||||||
```
|
```
|
||||||
|
|
||||||
### 2. Variables partagées protégées par mutex
|
Internal participants are not a second global scheduler or Queue.
|
||||||
|
|
||||||
```c
|
## Fundamental rules
|
||||||
// ✅ Correct
|
|
||||||
pthread_mutex_lock(&queue->mutex);
|
|
||||||
queue->count++;
|
|
||||||
pthread_mutex_unlock(&queue->mutex);
|
|
||||||
|
|
||||||
// ❌ Interdit
|
### ncurses ownership
|
||||||
// queue->count++; sans protection
|
|
||||||
|
Only the main thread calls ncurses.
|
||||||
|
|
||||||
|
Workers publish observable state through protected data. They never call
|
||||||
|
`mvprintw`, `wrefresh`, or other ncurses APIs.
|
||||||
|
|
||||||
|
### Shared mutable state
|
||||||
|
|
||||||
|
Shared mutable state must have an explicit synchronization owner:
|
||||||
|
|
||||||
|
```text
|
||||||
|
mutex
|
||||||
|
condition variable
|
||||||
|
atomic primitive where the contract explicitly permits it
|
||||||
|
single-thread ownership
|
||||||
|
immutable-after-publication
|
||||||
```
|
```
|
||||||
|
|
||||||
### 3. Variables de condition pour la synchronisation
|
Do not rely on timing or "normally only one caller".
|
||||||
|
|
||||||
```c
|
### Condition variables
|
||||||
// Producteur (caller de la Task Queue)
|
|
||||||
pthread_mutex_lock(&queue->mutex);
|
|
||||||
queue->ready = true;
|
|
||||||
pthread_cond_signal(&queue->cond);
|
|
||||||
pthread_mutex_unlock(&queue->mutex);
|
|
||||||
|
|
||||||
// Consommateur (worker)
|
Always test the predicate in a loop around `pthread_cond_wait()`.
|
||||||
pthread_mutex_lock(&queue->mutex);
|
|
||||||
while (!queue->ready) {
|
A signal is not durable state; the protected predicate is.
|
||||||
pthread_cond_wait(&queue->cond, &queue->mutex);
|
|
||||||
}
|
### Cooperative cancellation
|
||||||
// traitement
|
|
||||||
pthread_mutex_unlock(&queue->mutex);
|
Production Task cancellation is cooperative.
|
||||||
|
|
||||||
|
Do not use `pthread_cancel()` to stop a Task.
|
||||||
|
|
||||||
|
Task-specific non-preemptible operations finish their current atomic boundary
|
||||||
|
before pause/cancel is observed.
|
||||||
|
|
||||||
|
### Reservation before callback
|
||||||
|
|
||||||
|
No Task callback runs without the Resource Governor admission/reservation
|
||||||
|
required by its installed sequence contract.
|
||||||
|
|
||||||
|
Fixed-resource Tasks do not bypass the Governor.
|
||||||
|
|
||||||
|
### Terminal lifetime
|
||||||
|
|
||||||
|
Terminal callback completion precedes destruction of Task userdata.
|
||||||
|
|
||||||
|
Queue/Task ownership must ensure no observer dereferences freed userdata.
|
||||||
|
|
||||||
|
## Lock ordering
|
||||||
|
|
||||||
|
When multiple locks are required, the owning subsystem must define and preserve
|
||||||
|
one order.
|
||||||
|
|
||||||
|
Never add a reverse-order path to solve a local problem.
|
||||||
|
|
||||||
|
Avoid holding one subsystem mutex while calling into another subsystem that may
|
||||||
|
call back.
|
||||||
|
|
||||||
|
Where practical:
|
||||||
|
|
||||||
|
```text
|
||||||
|
copy bounded state under lock
|
||||||
|
release lock
|
||||||
|
perform I/O / expensive work
|
||||||
|
reacquire only for publication
|
||||||
```
|
```
|
||||||
|
|
||||||
### 4. Pas de callback ncurses depuis un worker
|
## Queue ingress lifetime
|
||||||
|
|
||||||
```c
|
The Queue owner closes ingress before destruction.
|
||||||
// ✅ Correct : le worker signale au thread principal
|
|
||||||
void worker_callback(task_t *task, void *userdata) {
|
|
||||||
shared_state_t *state = userdata;
|
|
||||||
pthread_mutex_lock(&state->mutex);
|
|
||||||
state->result_ready = true;
|
|
||||||
pthread_cond_signal(&state->cond);
|
|
||||||
pthread_mutex_unlock(&state->mutex);
|
|
||||||
}
|
|
||||||
|
|
||||||
// ❌ Interdit : appel ncurses depuis le worker
|
Shutdown waits for:
|
||||||
// void worker_callback(...) {
|
|
||||||
// mvprintw(...);
|
- active worker completion;
|
||||||
// }
|
- registered in-flight API calls covered by the ownership contract;
|
||||||
|
- terminal callbacks.
|
||||||
|
|
||||||
|
This cannot make a raw C pointer safe if a caller begins a new call after the
|
||||||
|
object has already been freed. Callers must obey lifetime ownership.
|
||||||
|
|
||||||
|
## Bounded internal parallelism
|
||||||
|
|
||||||
|
A validated Task Kind may use internal participants while the Queue callback
|
||||||
|
remains the sole Task owner.
|
||||||
|
|
||||||
|
Required shape:
|
||||||
|
|
||||||
|
```text
|
||||||
|
one admitted Task owner
|
||||||
|
-> bounded participant count
|
||||||
|
-> bounded private work
|
||||||
|
-> join all participants
|
||||||
|
-> owner-only deterministic publication when required
|
||||||
|
-> Task-specific durable cursor
|
||||||
|
-> generic checkpoint
|
||||||
|
-> sequence_break
|
||||||
```
|
```
|
||||||
|
|
||||||
## Primitives utilisées
|
Participant count and memory must fit the admitted Resource Governor contract.
|
||||||
|
|
||||||
| Primitive | Usage |
|
No participant may silently exceed the installed sequence contract.
|
||||||
|---|---|
|
|
||||||
| `pthread_mutex_t` | Protection des données partagées |
|
|
||||||
| `pthread_cond_t` | Synchronisation producteur/consommateur |
|
|
||||||
| `pthread_create()` | Création des workers |
|
|
||||||
| `pthread_join()` | Attente de fin des workers |
|
|
||||||
| `pthread_cancel()` | Non utilisé pour interrompre une Task ; annulation coopérative |
|
|
||||||
|
|
||||||
## Invariants de concurrence
|
## Atomicity does not imply serialism
|
||||||
|
|
||||||
1. **Un seul thread ncurses** : ncurses n'est jamais appelé depuis un
|
Per-item scientific atomicity and cross-item execution width are separate.
|
||||||
worker. Toute mise à jour de l'UI passe par des variables partagées
|
|
||||||
protégées.
|
|
||||||
|
|
||||||
2. **Mutex hiérarchique** : si plusieurs mutex sont acquis, toujours dans
|
```text
|
||||||
le même ordre pour éviter les deadlocks.
|
PER_ITEM_ATOMICITY_REQUIRES_CROSS_ITEM_SERIALISM=NO
|
||||||
|
OWNER_ONLY_PUBLICATION_REQUIRES_SERIAL_PREPARATION=NO
|
||||||
3. **Annulation coopérative** : les workers vérifient périodiquement un
|
|
||||||
drapeau d'annulation. Une Task n'est pas interrompue brutalement.
|
|
||||||
|
|
||||||
4. **Réservation atomique** : la réservation du gouverneur est atomique.
|
|
||||||
Deux threads ne peuvent pas obtenir la même réservation.
|
|
||||||
|
|
||||||
5. **Pas de callback sans réservation** : aucun callback de tâche n'est
|
|
||||||
invoqué sans réservation active. Cet invariant est maintenu même en
|
|
||||||
présence d'erreurs.
|
|
||||||
|
|
||||||
6. **Retraite après callback** : la notification terminale finit avant la
|
|
||||||
destruction du userdata. Queue détruit la Task hors de son mutex et ne
|
|
||||||
conserve ensuite qu'un snapshot borné.
|
|
||||||
|
|
||||||
7. **Fermeture d'ingress** : le propriétaire empêche les nouveaux appels Queue
|
|
||||||
avant `destroy()`. La fermeture interne attend le worker et chaque appel
|
|
||||||
enregistré avant le close ; elle ne peut rendre sûr un appel démarré après
|
|
||||||
la libération d'un pointeur C brut.
|
|
||||||
|
|
||||||
8. **Parallélisme scientifique propriétaire** : lorsqu'un kind emploie des
|
|
||||||
participants internes, le callback Queue demeure l'unique propriétaire. Le
|
|
||||||
nombre de participants et leur mémoire sont admis par le Governor ; seul le
|
|
||||||
propriétaire publie le préfixe durable ordonné et joint tous les enfants.
|
|
||||||
|
|
||||||
9. **Lease SSD par objet** : un lease scratch appartient à l'adresse exacte de
|
|
||||||
l'objet fourni par le caller. Tous ses champs sont lus/écrits sous le mutex
|
|
||||||
du contrôleur. Le caller lui garantit un accès exclusif et ne le copie, ne le
|
|
||||||
déplace ni ne le présente simultanément à deux contrôleurs. En production,
|
|
||||||
acquire/release passent par les wrappers Governor ; le Governor relâche son
|
|
||||||
mutex avant l'appel contrôleur, et le contrôleur ne rappelle jamais le
|
|
||||||
Governor. À la saturation légale `generation == UINT64_MAX`, seule la fin
|
|
||||||
du wrapper exact déjà sérialisé peut réconcilier sa propre opération et le
|
|
||||||
compte fondé sur les adresses ; une update publique au même watermark ne
|
|
||||||
peut pas rendre une autorité stale.
|
|
||||||
|
|
||||||
10. **Owner SSD unique** : la TUI/main demande et poll l'opération ; au plus un
|
|
||||||
thread joinable exécute une opération bornée et ne touche jamais ncurses.
|
|
||||||
Le destroy le joint avant unregister. Une observation malformée enregistre
|
|
||||||
`ERROR` et ne confère aucune autorité de contrôle ou de lease.
|
|
||||||
|
|
||||||
11. **Frontière projet** : les vues libèrent leurs borrows, puis la Queue est
|
|
||||||
annulée/jointe/détruite avant Project DB. Une Queue vide est créée ensuite.
|
|
||||||
Aucun callback terminal ne peut donc déréférencer une DB déjà fermée et
|
|
||||||
l'histoire d'un projet ne fuit pas dans le suivant.
|
|
||||||
|
|
||||||
12. **Ordre d'arrêt global** : Queue et leases Task, puis fermeture projet,
|
|
||||||
join/unregister du binding SSD, contrôleur SSD, et enfin Governor. Un
|
|
||||||
unregister encore bloqué par un lease est un échec observable, jamais un
|
|
||||||
pointeur abandonné.
|
|
||||||
|
|
||||||
## Anti-patterns
|
|
||||||
|
|
||||||
### Deadlock
|
|
||||||
|
|
||||||
```c
|
|
||||||
// ❌ Risque de deadlock
|
|
||||||
pthread_mutex_lock(&mutex_a);
|
|
||||||
pthread_mutex_lock(&mutex_b); // attend mutex_b
|
|
||||||
|
|
||||||
// Dans un autre thread :
|
|
||||||
pthread_mutex_lock(&mutex_b);
|
|
||||||
pthread_mutex_lock(&mutex_a); // attend mutex_a → DEADLOCK
|
|
||||||
```
|
```
|
||||||
|
|
||||||
**Solution** : toujours acquérir les mutex dans le même ordre.
|
Current examples include selected RAW, selected Feature extraction, Candidate
|
||||||
|
Pair source work and outer Geometric Verification preparation.
|
||||||
|
|
||||||
### Race condition
|
Serialization is valid only where the subsystem's scientific, persistence,
|
||||||
|
library or measured-throughput contract proves it necessary.
|
||||||
|
|
||||||
```c
|
## CPU/batch coupling
|
||||||
// ❌ Race condition
|
|
||||||
if (task->state == TASK_STATE_QUEUED) {
|
|
||||||
task->state = TASK_STATE_RUNNING;
|
|
||||||
}
|
|
||||||
|
|
||||||
// ✅ Correct
|
CPU and batch/window are not globally independent dimensions.
|
||||||
pthread_mutex_lock(&task->mutex);
|
|
||||||
if (task->state == TASK_STATE_QUEUED) {
|
For a Task whose additional participants cannot do useful work while the
|
||||||
task->state = TASK_STATE_RUNNING;
|
admitted item window remains one, a Task-specific capability may couple those
|
||||||
}
|
dimensions.
|
||||||
pthread_mutex_unlock(&task->mutex);
|
|
||||||
|
Current validated examples include:
|
||||||
|
|
||||||
|
```text
|
||||||
|
candidate_pair.generate/1
|
||||||
|
features.extract.batch/1
|
||||||
```
|
```
|
||||||
|
|
||||||
### Use-after-free
|
This is not a universal rule for all Task Kinds.
|
||||||
|
|
||||||
```c
|
## Project lifetime boundary
|
||||||
// ❌ Use-after-free
|
|
||||||
task_destroy(task);
|
|
||||||
task_callback(task); // task est libéré
|
|
||||||
|
|
||||||
// ✅ Correct : le callback est entièrement revenu avant la destruction
|
Before closing a project:
|
||||||
task_callback(task);
|
|
||||||
task_destroy(task);
|
```text
|
||||||
|
views release Project DB borrows
|
||||||
|
-> Queue is cancelled/joined/destroyed
|
||||||
|
-> Project DB closes
|
||||||
|
-> fresh empty Queue may be created for the next project
|
||||||
```
|
```
|
||||||
|
|
||||||
## Validation
|
A terminal callback must never observe a Project DB already destroyed.
|
||||||
|
|
||||||
Les readers Visual Index sont sans état partagé mutable. Une query copie la
|
Project-specific runtime history must not leak into the next project.
|
||||||
liste bornée des segments sous le mutex DB, puis effectue hash, lectures et
|
|
||||||
accumulation après déverrouillage. Un update ne rend le nouveau segment visible
|
|
||||||
qu'au commit memberships+segment ; une query en cours garde son snapshot.
|
|
||||||
|
|
||||||
Pour tout ticket touchant la concurrence, exécuter :
|
## Global shutdown boundary
|
||||||
|
|
||||||
|
The current shutdown order preserves ownership across:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Task Queue and Task leases
|
||||||
|
-> project close
|
||||||
|
-> join/unregister SSD binding
|
||||||
|
-> SSD controller
|
||||||
|
-> Resource Governor
|
||||||
|
```
|
||||||
|
|
||||||
|
A scratch unregister blocked by a real outstanding lease is an observable
|
||||||
|
failure, not permission to abandon a live pointer.
|
||||||
|
|
||||||
|
## SSD lease ownership
|
||||||
|
|
||||||
|
A scratch lease belongs to the exact caller-owned lease object used for the
|
||||||
|
operation.
|
||||||
|
|
||||||
|
Its mutable fields are controlled under the SSD controller mutex.
|
||||||
|
|
||||||
|
The caller must not:
|
||||||
|
|
||||||
|
- copy a live lease;
|
||||||
|
- move a live lease;
|
||||||
|
- present the same lease object to two controllers;
|
||||||
|
- release through a different ownership path.
|
||||||
|
|
||||||
|
Production acquire/release uses the Resource Governor wrappers.
|
||||||
|
|
||||||
|
The Governor releases its own mutex before entering the controller, and the
|
||||||
|
controller does not callback into the Governor while holding its mutex.
|
||||||
|
|
||||||
|
Scratch remains storage capacity, never RAM admission.
|
||||||
|
|
||||||
|
## Visual Index readers
|
||||||
|
|
||||||
|
Visual Index query readers do not share mutable query state.
|
||||||
|
|
||||||
|
A query obtains a bounded segment snapshot under the Project DB boundary, then
|
||||||
|
performs hash/read/accumulation after release.
|
||||||
|
|
||||||
|
A concurrent update makes a new segment visible only at its canonical commit
|
||||||
|
boundary. An already running query continues with its retained snapshot.
|
||||||
|
|
||||||
|
## OpenCV process-wide state
|
||||||
|
|
||||||
|
OpenCV thread configuration is process-wide.
|
||||||
|
|
||||||
|
The active heavy Queue callback owns temporary mutation of that setting where a
|
||||||
|
Task contract requires it and restores the previous/baseline value on all exit
|
||||||
|
paths.
|
||||||
|
|
||||||
|
Internal participants must not independently race `cv::setNumThreads()`.
|
||||||
|
|
||||||
|
For Feature batch, cross-image participants are used while internal OpenCV
|
||||||
|
threading is controlled explicitly.
|
||||||
|
|
||||||
|
## Vulkan boundary
|
||||||
|
|
||||||
|
ORB Vulkan concurrency is validated separately from portable TSan.
|
||||||
|
|
||||||
|
The production AUTO contract currently uses:
|
||||||
|
|
||||||
|
```text
|
||||||
|
normal inflight depth = 1
|
||||||
|
private validated safety depth = 2
|
||||||
|
helpers = 0
|
||||||
|
```
|
||||||
|
|
||||||
|
Depth 2 is a private safety/benchmark capability and was rejected as the normal
|
||||||
|
useful setting by measured throughput.
|
||||||
|
|
||||||
|
A Vulkan backend failure produces complete CPU fallback before publication.
|
||||||
|
Partial GPU scientific output is never published.
|
||||||
|
|
||||||
|
## TSan policy
|
||||||
|
|
||||||
|
For project concurrency changes, use a dedicated TSan build that matches the
|
||||||
|
supported proof boundary.
|
||||||
|
|
||||||
|
The retained global maintenance matrix used:
|
||||||
|
|
||||||
|
```text
|
||||||
|
GCC/G++
|
||||||
|
Vulkan disabled
|
||||||
|
selected concurrent targets
|
||||||
|
deterministic repetitions
|
||||||
|
```
|
||||||
|
|
||||||
|
and completed the retained 14/14 target matrix plus 220 repetitions.
|
||||||
|
|
||||||
|
The narrow suppression file is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
tests/tsan-opencv.supp
|
||||||
|
```
|
||||||
|
|
||||||
|
It covers external non-instrumented OpenCV/TBB objects only.
|
||||||
|
|
||||||
|
It must not suppress Lardon3D frames.
|
||||||
|
|
||||||
|
Therefore the correct retained claim is not "TSan proves all concurrency".
|
||||||
|
It is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
portable project concurrency matrix passed under the documented qualification
|
||||||
|
external OpenCV/TBB reports are narrowly qualified
|
||||||
|
Vulkan concurrency has a separate validation boundary
|
||||||
|
```
|
||||||
|
|
||||||
|
## What TSan does not prove
|
||||||
|
|
||||||
|
TSan is useful for instrumented conflicting memory access and some
|
||||||
|
synchronization misuse.
|
||||||
|
|
||||||
|
It does not prove absence of:
|
||||||
|
|
||||||
|
- deadlock;
|
||||||
|
- lost wakeup caused by incorrect predicate design;
|
||||||
|
- lifetime bugs outside the exercised paths;
|
||||||
|
- races hidden inside non-instrumented external libraries;
|
||||||
|
- Vulkan driver/runtime correctness;
|
||||||
|
- scientific determinism.
|
||||||
|
|
||||||
|
Lock-order review, ownership reasoning and deterministic tests remain required.
|
||||||
|
|
||||||
|
## Sanitizer command policy
|
||||||
|
|
||||||
|
Do not encode fixed `-j8` as canonical validation.
|
||||||
|
|
||||||
|
Example configuration:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
# Build TSan
|
CC=gcc CXX=g++ meson setup build-tsan -Db_sanitize=thread -Db_lundef=false -Dvulkan_orb=disabled
|
||||||
CC=clang meson setup build-tsan --wipe -Db_sanitize=thread -Db_lundef=false
|
meson compile -C build-tsan
|
||||||
meson compile -C build-tsan -j8
|
|
||||||
meson test -C build-tsan --print-errorlogs
|
meson test -C build-tsan --print-errorlogs
|
||||||
```
|
```
|
||||||
|
|
||||||
TSan détecte automatiquement :
|
Use host-aware compile/test parallelism unless the proof itself requires
|
||||||
|
serialization.
|
||||||
|
|
||||||
- les accès concurrents conflictuels instrumentés ;
|
Do not repeatedly wipe an unchanged TSan tree.
|
||||||
- certaines utilisations incohérentes des primitives de synchronisation.
|
|
||||||
|
|
||||||
Il ne prouve pas l'absence de deadlock, de signal perdu ou de bug dans une
|
## Concurrency review checklist
|
||||||
bibliothèque non instrumentée. Les invariants de lifetime et d'ordre de locks
|
|
||||||
restent donc soumis aux tests déterministes et à la revue.
|
|
||||||
|
|
||||||
### Preuve TSan globale courante
|
Before closing a concurrency-sensitive change, verify:
|
||||||
|
|
||||||
La matrice fraîche emploie GCC/G++ 16.2.1 et désactive explicitement Vulkan.
|
- ncurses remains main-thread-only;
|
||||||
Elle passe 14/14 cibles couvrant Task, Project, Queue, Governor, registre/leases
|
- every shared mutable field has an explicit synchronization owner;
|
||||||
SSD, contrôleur SSD, observateur/TUI async, Candidate, Visual Index, Feature,
|
- condition predicates are checked in loops;
|
||||||
Matcher et GV, puis 220/220 répétitions déterministes : **234/234** au total.
|
- lock order remains consistent;
|
||||||
|
- no Task uses forced asynchronous cancellation;
|
||||||
|
- Queue callbacks have an active reservation;
|
||||||
|
- internal participants stay within the admitted contract;
|
||||||
|
- all children join on every exit path;
|
||||||
|
- owner-only publication remains ordered where required;
|
||||||
|
- project-close ordering prevents DB use-after-close;
|
||||||
|
- SSD lease ownership remains exact;
|
||||||
|
- Task userdata outlives terminal notification;
|
||||||
|
- appropriate deterministic concurrency tests pass;
|
||||||
|
- portable TSan qualification is preserved;
|
||||||
|
- Vulkan validation is reported separately;
|
||||||
|
- ASan/UBSan is run when the change also affects lifetime/memory.
|
||||||
|
|
||||||
La seule liste de suppressions est `tests/tsan-opencv.supp`, limitée aux objets
|
## Current retained evidence
|
||||||
partagés externes non instrumentés `libopencv_features.so`,
|
|
||||||
`libopencv_core.so` et `libtbb.so`. Elle ne masque aucune frame Lardon3D. Les
|
|
||||||
warnings GCC `-Wmaybe-uninitialized` des contrôles OpenCV Feature/SIFT sont
|
|
||||||
classés non matériels : le callback fournit une Task non nulle et le helper
|
|
||||||
initialise la structure avant toute autre sortie d'échec. Les warnings OpenCV
|
|
||||||
du build GV appartiennent aux headers externes.
|
|
||||||
|
|
||||||
Cette preuve TSan ne vaut pas validation de concurrence Vulkan. Le backend
|
The canonical global-maintenance record is:
|
||||||
ORB Vulkan réel est couvert séparément par le build Clang Vulkan-on 939/939,
|
|
||||||
la suite 65/65 et ses tests de backend/handle/publication ; cette séparation
|
|
||||||
doit rester explicite dans tout rapport.
|
|
||||||
|
|
||||||
## Checklist de concurrence
|
```text
|
||||||
|
docs/architecture/global_maintenance_audit.md
|
||||||
|
GLOBAL_MAINTENANCE_AUDIT=PASS/FROZEN
|
||||||
|
```
|
||||||
|
|
||||||
Avant de livrer un ticket touchant la concurrence :
|
The current A6000 checkpoint is later:
|
||||||
|
|
||||||
- [ ] Toutes les variables partagées sont protégées par un mutex
|
```text
|
||||||
- [ ] Les mutex sont toujours libérés (même en cas d'erreur)
|
real-a6000-pre-sfm-2026-09-02
|
||||||
- [ ] Les variables de condition sont vérifiées dans une boucle `while`
|
REAL_A6000_PRE_SFM=PASS/FROZEN
|
||||||
- [ ] Aucun appel ncurses depuis un worker
|
```
|
||||||
- [ ] L'annulation des Tasks est coopérative (pas de `pthread_cancel`)
|
|
||||||
- [ ] TSan ne signale aucune erreur
|
The later A6000 proof exercised current bounded parallel paths through selected
|
||||||
- [ ] Le build ASan ne signale aucune fuite mémoire liée aux threads
|
Feature batch, Candidate, Matcher, Geometric Verifier v3 and Tracks without
|
||||||
|
changing the historical TSan qualification.
|
||||||
|
|
||||||
|
Historical evidence remains historical; new changes require validation scoped
|
||||||
|
to their actual concurrency surface.
|
||||||
|
|
||||||
|
## Summary
|
||||||
|
|
||||||
|
```text
|
||||||
|
NCURSES_OWNER=MAIN_THREAD_ONLY
|
||||||
|
ACTIVE_HEAVY_QUEUE_CALLBACKS=1
|
||||||
|
TASK_CANCELLATION=COOPERATIVE
|
||||||
|
INTERNAL_PARALLELISM=BOUNDED
|
||||||
|
|
||||||
|
PER_ITEM_ATOMICITY_REQUIRES_CROSS_ITEM_SERIALISM=NO
|
||||||
|
OWNER_ONLY_PUBLICATION_REQUIRES_SERIAL_PREPARATION=NO
|
||||||
|
|
||||||
|
TSAN_PORTABLE_PROJECT_MATRIX=QUALIFIED_PASS
|
||||||
|
TSAN_EXTERNAL_OPENCV_TBB=QUALIFIED
|
||||||
|
VULKAN_CONCURRENCY_VALIDATION=SEPARATE
|
||||||
|
|
||||||
|
NO_FIXED_GLOBAL_J8=YES
|
||||||
|
NO_REPEATED_UNCHANGED_WIPE=YES
|
||||||
|
|
||||||
|
RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
|
||||||
|
SERIALISM_REQUIRES_PROOF=CANONICAL
|
||||||
|
```
|
||||||
|
|
|
||||||
|
|
@ -1,136 +1,298 @@
|
||||||
# Procédures de test
|
# Testing
|
||||||
|
|
||||||
## Vue d'ensemble
|
## Status
|
||||||
|
|
||||||
Lardon3D utilise le framework de test intégré à Meson. Chaque module possède
|
```text
|
||||||
un fichier de test dans `tests/` correspondant au module testé.
|
DOCUMENTATION_LANGUAGE=ENGLISH
|
||||||
|
TEST_POLICY=HOST_AWARE
|
||||||
|
REPEATED_UNCHANGED_EXPENSIVE_VALIDATION=AVOID
|
||||||
|
TSAN_OPEN_CV_TBB_QUALIFICATION=REQUIRED
|
||||||
|
VULKAN_CONCURRENCY_VALIDATION=SEPARATE
|
||||||
|
```
|
||||||
|
|
||||||
## Lancer les tests
|
Lardon3D uses Meson's test runner. Tests live under `tests/` and combine unit,
|
||||||
|
integration, persistence, restart, resource and real-path validation.
|
||||||
|
|
||||||
|
## Normal commands
|
||||||
|
|
||||||
|
Run the configured suite:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
# Tous les tests
|
|
||||||
meson test -C build --print-errorlogs
|
meson test -C build --print-errorlogs
|
||||||
|
```
|
||||||
|
|
||||||
# Un test spécifique
|
Run one named test:
|
||||||
meson test -C build test_task_queue --print-errorlogs
|
|
||||||
|
|
||||||
# Tests avec verbose
|
```sh
|
||||||
|
meson test -C build <test-name> --print-errorlogs
|
||||||
|
```
|
||||||
|
|
||||||
|
Verbose execution:
|
||||||
|
|
||||||
|
```sh
|
||||||
meson test -C build -v --print-errorlogs
|
meson test -C build -v --print-errorlogs
|
||||||
|
```
|
||||||
|
|
||||||
# Réexécuter uniquement les tests échoués
|
Re-run failures only:
|
||||||
|
|
||||||
|
```sh
|
||||||
meson test -C build --reprint=failed
|
meson test -C build --reprint=failed
|
||||||
```
|
```
|
||||||
|
|
||||||
## Structure des tests
|
Use the names registered by the current `meson.build`; this document does not
|
||||||
|
maintain a second authoritative list of every test target.
|
||||||
|
|
||||||
|
## Validation policy
|
||||||
|
|
||||||
|
Validation must match the change.
|
||||||
|
|
||||||
|
A documentation-only change normally requires:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
tests/
|
git diff --check
|
||||||
├── test_task_queue.c # tests de la file de tâches
|
targeted content checks
|
||||||
├── test_task.c # tests du module task
|
targeted link/authority review
|
||||||
├── test_resource_governor.c # tests du gouverneur
|
|
||||||
├── test_hardware_profile.c # tests du profil matériel
|
|
||||||
├── test_import.c # tests de l'import
|
|
||||||
├── test_project.c # tests des projets
|
|
||||||
└── test_*.c # autres modules
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Écrire un test
|
It does not justify wiping and rebuilding unchanged code.
|
||||||
|
|
||||||
```c
|
A code change normally requires, in increasing scope:
|
||||||
#include <glib.h>
|
|
||||||
#include "lardon3d/task.h"
|
|
||||||
|
|
||||||
void test_task_create(void) {
|
```text
|
||||||
task_estimate_t est = {
|
targeted build
|
||||||
.ram_bytes = 1024 * 1024,
|
targeted tests
|
||||||
.gpu_bytes = 0,
|
broader affected suite
|
||||||
.cpu_weight = 1,
|
sanitizer or concurrency validation when relevant
|
||||||
.io_weight = 0,
|
full suite when the change or release boundary justifies it
|
||||||
.batch_size = 10,
|
|
||||||
.batch_max = 100
|
|
||||||
};
|
|
||||||
task_t *t = task_create("test", &est, NULL, NULL);
|
|
||||||
g_assert_nonnull(t);
|
|
||||||
g_assert_cmpint(task_get_state(t), ==, TASK_STATE_IDLE);
|
|
||||||
task_destroy(t);
|
|
||||||
}
|
|
||||||
|
|
||||||
int main(int argc, char **argv) {
|
|
||||||
g_test_init(&argc, &argv, NULL);
|
|
||||||
g_test_add_func("/task/create", test_task_create);
|
|
||||||
return g_test_run();
|
|
||||||
}
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Conventions
|
Do not repeatedly rerun an unchanged expensive suite between documentation
|
||||||
|
edits merely to create activity.
|
||||||
|
|
||||||
1. **Préfixe `test_`** : chaque fonction de test porte le préfixe `test_`.
|
## Host-aware parallelism
|
||||||
2. **Chemin hiérarchique** : le nom du test suit le pattern `/module/action`.
|
|
||||||
3. **Asserts GLib** : utiliser `g_assert_*` pour les vérifications.
|
|
||||||
4. **Nettoyage** : chaque test libère toutes ses ressources.
|
|
||||||
5. **Isolation** : un test ne dépend pas de l'état d'un autre test.
|
|
||||||
6. **Déterminisme** : les tests ne dépendent pas de l'heure, du filesystem
|
|
||||||
ou de l'état réseau (sauf test d'import).
|
|
||||||
|
|
||||||
## Commentaires source
|
Build and test parallelism are host-aware.
|
||||||
|
|
||||||
Les commentaires documentent le pourquoi et les contrats non évidents :
|
Do not encode a project-wide fixed `-j8`, `--num-processes 1`, or equivalent
|
||||||
invariants, propriété et durée de vie, persistance, ainsi que limites et
|
constant as canonical policy.
|
||||||
frontières de ressources. Les API publiques documentent leurs contrats non
|
|
||||||
évidents. Ils ne paraphrasent pas le code ligne par ligne et sont mis à jour
|
|
||||||
avec tout changement de comportement.
|
|
||||||
|
|
||||||
## Tests unitaires vs tests d'intégration
|
The correct width depends on the current machine, interactive reserve, memory,
|
||||||
|
toolchain and workload. Use all safe useful host capacity while preserving the
|
||||||
|
defined interactive reserve.
|
||||||
|
|
||||||
`test-visual-index` couvre les descriptors synthétiques, le retrieval ORB réel,
|
```text
|
||||||
les filtres inter-ScanSets, quatre queries concurrentes, la corruption/absence/
|
RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
|
||||||
troncature d'un segment et 4 000 Feature Sets synthétiques. Le scénario de
|
SERIALISM_REQUIRES_PROOF=CANONICAL
|
||||||
reprise `visual_index.update` est exercé dans `test-feature-task`.
|
```
|
||||||
|
|
||||||
| Type | Portée | Fichier |
|
If a temporary validation must be serialized for determinism, diagnosis or a
|
||||||
|---|---|---|
|
known tool limitation, label that serialization as test-specific evidence
|
||||||
| Unitaire | Un module isolé | `tests/test_<module>.c` |
|
rather than a global default.
|
||||||
| Intégration | Interaction entre modules | `tests/test_<module>.c` avec dépendances réelles |
|
|
||||||
|
|
||||||
## Validation par ticket
|
## Fresh build policy
|
||||||
|
|
||||||
Avant de livrer un ticket, exécuter la séquence complète :
|
Do not use `meson setup --wipe` by default.
|
||||||
|
|
||||||
|
Prefer:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
# 1. Build clean
|
meson setup --reconfigure build
|
||||||
CC=clang meson setup build --wipe
|
meson compile -C build
|
||||||
meson compile -C build -j8
|
|
||||||
|
|
||||||
# 2. Tests
|
|
||||||
meson test -C build --print-errorlogs
|
|
||||||
|
|
||||||
# 3. Style
|
|
||||||
git diff --check
|
|
||||||
|
|
||||||
# 4. Si mémoire/concurrence touchés
|
|
||||||
CC=clang meson setup build-asan --wipe -Db_sanitize=address,undefined
|
|
||||||
meson compile -C build-asan -j8
|
|
||||||
meson test -C build-asan --print-errorlogs
|
|
||||||
|
|
||||||
# 5. Si concurrence touchée
|
|
||||||
CC=clang meson setup build-tsan --wipe -Db_sanitize=thread -Db_lundef=false
|
|
||||||
meson compile -C build-tsan -j8
|
|
||||||
meson test -C build-tsan --print-errorlogs
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## Dépannage
|
Create or wipe a build directory when the configuration genuinely needs a
|
||||||
|
fresh environment, for example:
|
||||||
|
|
||||||
### Test qui échoue en ASan
|
```text
|
||||||
|
different sanitizer set
|
||||||
|
portable Vulkan-off proof
|
||||||
|
Vulkan-on proof
|
||||||
|
compiler-family change
|
||||||
|
known stale/corrupt build directory
|
||||||
|
release-grade clean proof
|
||||||
|
```
|
||||||
|
|
||||||
Vérifier les durées de vie des allocations. Ne jamais libérer un objet puis
|
Repeated wipes of the same unchanged configuration waste time and invalidate
|
||||||
y accéder. Vérifier que chaque `task_destroy()` est appelée.
|
incremental-build advantages.
|
||||||
|
|
||||||
### Test qui échoue en TSan
|
## Sanitizers
|
||||||
|
|
||||||
Vérifier que toutes les variables partagées sont protégées par un mutex.
|
### ASan / UBSan
|
||||||
Vérifier que ncurses est utilisé uniquement depuis le thread principal.
|
|
||||||
|
|
||||||
### Test qui échoue uniquement en release
|
For memory, lifetime, ownership or undefined-behavior changes, use a dedicated
|
||||||
|
sanitizer build.
|
||||||
|
|
||||||
Vérifier les assertions et les overflow arithmétiques. Compiler avec
|
Example configuration:
|
||||||
`-fsanitize=undefined` pour détecter les comportements indéfinis.
|
|
||||||
|
```sh
|
||||||
|
CC=clang CXX=clang++ meson setup build-asan -Db_sanitize=address,undefined
|
||||||
|
meson compile -C build-asan
|
||||||
|
meson test -C build-asan --print-errorlogs
|
||||||
|
```
|
||||||
|
|
||||||
|
Reconfigure or wipe only when the existing sanitizer directory does not match
|
||||||
|
the requested configuration.
|
||||||
|
|
||||||
|
### LeakSanitizer qualification
|
||||||
|
|
||||||
|
The retained global maintenance evidence must not be summarized as
|
||||||
|
`LSan 64/64`.
|
||||||
|
|
||||||
|
The full first leak-enabled run exposed an externally attributed OpenCL loader
|
||||||
|
leak and two timeout anomalies. The retained qualified result is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
ASan/UBSan full suite: PASS with detect_leaks=0
|
||||||
|
proved subset without the external loader: LSan PASS
|
||||||
|
full leak-enabled suite: not an unqualified PASS
|
||||||
|
```
|
||||||
|
|
||||||
|
Preserve that distinction in future reports unless new evidence supersedes it.
|
||||||
|
|
||||||
|
## ThreadSanitizer
|
||||||
|
|
||||||
|
Concurrency changes require TSan where the instrumented boundary is meaningful.
|
||||||
|
|
||||||
|
The retained global maintenance TSan proof used GCC/G++ with Vulkan disabled
|
||||||
|
and covered the selected concurrent targets plus deterministic repetitions.
|
||||||
|
|
||||||
|
The only retained suppression file is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
tests/tsan-opencv.supp
|
||||||
|
```
|
||||||
|
|
||||||
|
Its purpose is limited to external OpenCV/TBB objects. It must not suppress
|
||||||
|
Lardon3D frames.
|
||||||
|
|
||||||
|
Therefore never report a blanket statement such as:
|
||||||
|
|
||||||
|
```text
|
||||||
|
TSan proves the entire Vulkan build race-free
|
||||||
|
```
|
||||||
|
|
||||||
|
The valid qualification is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
project concurrent paths covered by the retained portable TSan matrix
|
||||||
|
external OpenCV/TBB reports qualified by the narrow suppression boundary
|
||||||
|
Vulkan concurrency validated separately
|
||||||
|
```
|
||||||
|
|
||||||
|
## Vulkan validation
|
||||||
|
|
||||||
|
ORB Vulkan uses a separate validation boundary.
|
||||||
|
|
||||||
|
The retained global maintenance evidence includes a Vulkan-on build and suite
|
||||||
|
on the real Radeon 780M/RADV host, plus dedicated backend/handle/publication
|
||||||
|
tests.
|
||||||
|
|
||||||
|
That evidence is not interchangeable with TSan.
|
||||||
|
|
||||||
|
SIFT/RootSIFT feasibility results did not establish a production GPU backend;
|
||||||
|
do not turn feasibility checks into production validation claims.
|
||||||
|
|
||||||
|
## Determinism and repetition
|
||||||
|
|
||||||
|
Repeat tests when repetition proves something specific:
|
||||||
|
|
||||||
|
```text
|
||||||
|
deterministic restart
|
||||||
|
race sensitivity
|
||||||
|
resource adaptation
|
||||||
|
ordering stability
|
||||||
|
flaky regression reproduction
|
||||||
|
```
|
||||||
|
|
||||||
|
Do not repeat unchanged tests without a stated purpose.
|
||||||
|
|
||||||
|
When repetition is the evidence, record:
|
||||||
|
|
||||||
|
```text
|
||||||
|
exact test/corpus
|
||||||
|
run count
|
||||||
|
relevant configuration
|
||||||
|
success/failure count
|
||||||
|
digest or invariant when applicable
|
||||||
|
```
|
||||||
|
|
||||||
|
## Test isolation
|
||||||
|
|
||||||
|
Tests should:
|
||||||
|
|
||||||
|
- own and clean up their temporary resources;
|
||||||
|
- avoid depending on another test's execution order;
|
||||||
|
- avoid network state unless the test explicitly owns that dependency;
|
||||||
|
- use synthetic/private Resource snapshots where the test is about deterministic
|
||||||
|
policy rather than live host telemetry;
|
||||||
|
- avoid changing global process state without restoring it.
|
||||||
|
|
||||||
|
OpenCV thread configuration is process-wide and must be restored on every exit
|
||||||
|
path in tests that change it.
|
||||||
|
|
||||||
|
## Public-header validation
|
||||||
|
|
||||||
|
When a public C header changes, run a standalone C17 syntax probe in addition to
|
||||||
|
normal build coverage.
|
||||||
|
|
||||||
|
Conceptually:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cc -x c -std=c17 -fsyntax-only -Iinclude -include lardon3d/<header>.h /dev/null
|
||||||
|
```
|
||||||
|
|
||||||
|
Use the current supported compiler matrix when the change affects ABI or
|
||||||
|
C/C++ interoperability.
|
||||||
|
|
||||||
|
## Source comments
|
||||||
|
|
||||||
|
Source comments explain non-obvious contracts:
|
||||||
|
|
||||||
|
```text
|
||||||
|
invariants
|
||||||
|
ownership and lifetime
|
||||||
|
persistence ordering
|
||||||
|
resource boundaries
|
||||||
|
recovery behavior
|
||||||
|
scientific constraints
|
||||||
|
```
|
||||||
|
|
||||||
|
They should not paraphrase obvious code line by line.
|
||||||
|
|
||||||
|
Repository source comments are English.
|
||||||
|
|
||||||
|
## Current retained global maintenance evidence
|
||||||
|
|
||||||
|
The canonical detailed record is:
|
||||||
|
|
||||||
|
```text
|
||||||
|
docs/architecture/global_maintenance_audit.md
|
||||||
|
GLOBAL_MAINTENANCE_AUDIT=PASS/FROZEN
|
||||||
|
```
|
||||||
|
|
||||||
|
That historical checkpoint includes fresh portable/Vulkan builds, full suites,
|
||||||
|
sanitizer work, TSan work, public-header probes, ABI/application-link checks and
|
||||||
|
independent review.
|
||||||
|
|
||||||
|
It remains historical evidence. New changes require only the validation
|
||||||
|
appropriate to the changed surface unless a new global checkpoint is being
|
||||||
|
created.
|
||||||
|
|
||||||
|
## Ticket closure checklist
|
||||||
|
|
||||||
|
Before closing a code ticket:
|
||||||
|
|
||||||
|
- confirm the requested scope only was changed;
|
||||||
|
- run `git diff --check`;
|
||||||
|
- run targeted tests for changed behavior;
|
||||||
|
- run the affected broader suite when justified;
|
||||||
|
- run ASan/UBSan for memory/lifetime-sensitive changes;
|
||||||
|
- run TSan for concurrency-sensitive project code when applicable;
|
||||||
|
- keep Vulkan validation separate from portable TSan claims;
|
||||||
|
- preserve exact external-library qualifications;
|
||||||
|
- avoid fixed host-parallelism constants;
|
||||||
|
- avoid repeated unchanged clean builds or suites;
|
||||||
|
- report what actually ran, not a stronger claim.
|
||||||
|
|
||||||
|
For documentation-only remediation, use documentation checks rather than
|
||||||
|
rebuilding unchanged production code.
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue