docs: reconcile production Task Kind Registry
This commit is contained in:
fy59
parent
475f3f1eb0
commit
1d073fab46
1 changed files with 588 additions and 228 deletions
|
|
@ -1,268 +1,628 @@
|
|||
# Registry des types métier de tâches
|
||||
# Task Kind Registry
|
||||
|
||||
## Feature kinds v1A
|
||||
## Status
|
||||
|
||||
La registry statique conserve `features.extract` v1 pour ORB et ajoute
|
||||
`features.extract.sift` v1 et `features.extract.rootsift` v1. Les reconstructeurs
|
||||
chargent la table dédiée, revalident le fingerprint et ne capturent aucun
|
||||
`AppState`.
|
||||
```text
|
||||
CURRENT_PROJECT_DB_SCHEMA=v25
|
||||
CURRENT_PRODUCTION_TASK_KINDS=16
|
||||
|
||||
## Contrat
|
||||
TASK_KIND_REGISTRY_STATUS=IMPLEMENTED
|
||||
COMPUTE_GOVERNOR_V2=PASS/FROZEN
|
||||
ORB_VULKAN_ASYNC_EXECUTION=PASS/FROZEN
|
||||
|
||||
Une instance possède un **task ID** stable. Son **task kind** décrit son
|
||||
comportement métier, son **task state** décrit son état d'avancement, la
|
||||
**checkpoint version** décrit le codec générique et la **task kind version**
|
||||
versionne les paramètres nécessaires au reconstructeur. Ces identités ne sont
|
||||
pas interchangeables.
|
||||
RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
|
||||
SERIALISM_REQUIRES_PROOF=CANONICAL
|
||||
```
|
||||
|
||||
Le kind v1 est une chaîne ASCII de 1 à 64 caractères au format
|
||||
`[a-z0-9][a-z0-9._-]*`. La version est un entier non nul. Aucun kind n'est
|
||||
déduit d'un nom, callback ou pointeur et aucune normalisation n'est effectuée.
|
||||
The production registry contains exactly sixteen version-1 Task Kinds.
|
||||
|
||||
## Registry et ownership
|
||||
The registry is a bounded static dispatch table. It maps durable Task Kind identity to a reconstruction
|
||||
function. It is not a scheduler, dependency graph, plugin loader, resource-policy database or
|
||||
scientific identity resolver.
|
||||
|
||||
La registry est une vue bornée à 64 descriptors sur un tableau statique
|
||||
immutable. Le lookup est linéaire, déterministe, sans allocation et sûr en
|
||||
lecture concurrente. Elle ne charge aucun code dynamiquement.
|
||||
Project DB v25 is the current schema head. Earlier schema versions remain authoritative for the Task
|
||||
payloads they introduced.
|
||||
|
||||
Un descriptor contient exactement le kind, sa version et un reconstructeur.
|
||||
Le reconstructeur produit callback, userdata et destructeur optionnel. Avant le
|
||||
transfert, la registry nettoie le userdata sur toute erreur ; après restauration
|
||||
réussie, `Lardon3DTask` en devient propriétaire et le détruit une fois après la
|
||||
fin de l'exécution. Le constructeur métier n'est jamais appelé sous mutex DB.
|
||||
## Identity contract
|
||||
|
||||
La registry peut normaliser une ancienne estimation opérationnelle connue. Le
|
||||
reconstructeur reçoit toujours le snapshot durable original afin de valider le
|
||||
mode exact ; la registry applique ensuite l'estimation effective uniquement à
|
||||
la copie privée transmise à la restauration de `Task`. Cette normalisation est
|
||||
éphémère et déterministe : elle ne stage, ne promeut et ne publie aucun
|
||||
checkpoint contenant seulement une estimation différente sous le même résumé.
|
||||
Une panne pré-terminale peut donc répéter la normalisation exacte. Cette couture
|
||||
ne peut modifier ni identité, paramètres scientifiques, progression ou curseur
|
||||
métier, et toute forme voisine est rejetée.
|
||||
A durable Task has several distinct identities and versions:
|
||||
|
||||
## Inventaire production et entrées runtime
|
||||
- stable Task ID;
|
||||
- Task Kind string;
|
||||
- Task Kind version;
|
||||
- generic checkpoint codec version;
|
||||
- Task state;
|
||||
- scientific input identity owned by the Task-specific payload.
|
||||
|
||||
`src/task_kinds.c::lardon3d_task_kind_registry_production()` enregistre les
|
||||
16 kinds v1 du profil de production courant. La colonne « reprise » nomme le
|
||||
reconstructeur du binding ; « callback » nomme l'entrée runtime privée dans le
|
||||
même fichier. Le détail chiffré des capacités est centralisé dans l'[audit des
|
||||
16 kinds](resource_governor.md#audit-des-15-kinds-de-production).
|
||||
These fields are not interchangeable.
|
||||
|
||||
| Kind v1 | Source, reprise et callback | Réconciliation pré-admission courante |
|
||||
| --- | --- | --- |
|
||||
| `raw.develop` | `raw_development_task.cpp`; `lardon3d_raw_development_task_reconstruct`; `run` | Aucune |
|
||||
| `raw.develop.batch` | `raw_development_task.cpp`; `lardon3d_raw_development_batch_task_reconstruct`; `run_batch` | Association v24 Task → exécution sélectionnée ; le curseur reste dans l'exécution |
|
||||
| `photo_quality.triage` | `photo_quality_task.cpp`; `lardon3d_photo_quality_task_reconstruct`; `run` | Aucune |
|
||||
| `acquisition_campaign.run` | `acquisition_campaign_task.cpp`; `lardon3d_acquisition_campaign_task_reconstruct`; `run` | Forme courante ou forme v22 exacte vérifiée contre la requête immuable → capacité courante en mémoire |
|
||||
| `import.images` | `import_task.c`; `lardon3d_image_import_reconstruct`; `run_image_import` | Aucune |
|
||||
| `features.extract` | `feature_task.c`; `lardon3d_feature_extract_reconstruct`; `run` | Formes CPU12/CPU1 historiques exactes → demande OpenCV portable ; runtime borné au compute-pool |
|
||||
| `features.extract.batch` | `feature_task.c`; `lardon3d_feature_extract_batch_reconstruct`; `run_batch` | Association v25 à l'exécution sélectionnée, domaine ORB exact et curseur typé ; CPU/lot bornés 1..12 |
|
||||
| `features.extract.sift` | `sift_task.c`; `lardon3d_sift_extract_reconstruct`; `run` | CPU12/CPU1 historiques exacts → demande OpenCV portable ; runtime borné au compute-pool |
|
||||
| `features.extract.rootsift` | `sift_task.c`; `lardon3d_sift_extract_reconstruct`; `run` | Même réconciliation SIFT ; aucune voie GPU validée |
|
||||
| `visual_index.update` | `visual_index_task.c`; `lardon3d_visual_index_update_reconstruct`; `run` | Formes CPU12/CPU1 historiques exactes → CPU/lot 1..16 |
|
||||
| `candidate_pair.generate` | `candidate_pair_task.c`; `lardon3d_candidate_pair_generate_reconstruct`; `run` | Formes CPU12 et CPU1 historiques exactes → CPU/lot 1..64 |
|
||||
| `matcher.run` | `matcher_task.c`; `lardon3d_matcher_task_reconstruct`; `run` | Signatures CPU/Vulkan historiques exactes → formes courantes en mémoire |
|
||||
| `geometric_verifier.run` | `geometric_verifier_task.c`; `lardon3d_geometric_verifier_task_reconstruct`; `run` | Forme sérielle CPU1/batch8 exacte → CPU utile 1..8, lot 1..16 |
|
||||
| `track_builder.run` | `track_builder_task.cpp`; `lardon3d_track_builder_task_reconstruct`; `run` | Aucune |
|
||||
| `sparse_sfm.run` | `sparse_sfm_task.cpp`; `lardon3d_sparse_sfm_task_reconstruct`; `run` | Aucune |
|
||||
| `incremental_reconstruction.run` | `incremental_reconstruction_task.cpp`; `lardon3d_incremental_reconstruction_task_reconstruct`; `run` | Aucune |
|
||||
A Task Kind v1 string is ASCII, 1 to 64 characters, with the form:
|
||||
|
||||
## Couture privée Compute Governor v2
|
||||
```text
|
||||
[a-z0-9][a-z0-9._-]*
|
||||
```
|
||||
|
||||
**COMPUTE_GOVERNOR_V2 — PASS / FROZEN.** Le descriptor C public
|
||||
reste limité à kind, version et reconstructeur. L'enveloppe de capacités est
|
||||
intégrée sans changement d'ABI dans le `struct Lardon3DTask` opaque et les
|
||||
coutures privées `src/task_internal.h` / `src/resource_governor_internal.h`.
|
||||
Les coutures d'admission sont `src/task_queue.c::select_admissible()`,
|
||||
`src/task.c::lardon3d_task_sequence_break()` et, côté Governor,
|
||||
la sélection multi-capacité sur un snapshot unique. La normalisation historique
|
||||
exacte reste dans `src/task_kind_registry.c::normalize_known_legacy_estimate()`.
|
||||
The Task Kind version is a non-zero integer.
|
||||
|
||||
Cette enveloppe n'est ni une identité scientifique, ni un nouveau payload
|
||||
Project DB, ni un nouveau scheduler. Le Governor possède l'admission de tous
|
||||
les kinds, y compris les formes entièrement fixes. Le contrat choisi est
|
||||
immutable pendant une séquence et seule la suivante peut être adaptée. Une Task
|
||||
sans alternative reçoit automatiquement une capacité égale à son estimation
|
||||
durable. Le Governor conserve un état borné par kind/backend et un dernier
|
||||
diagnostic ; ni l'enveloppe ni ce choix ne sont persistés.
|
||||
No Task Kind is inferred from:
|
||||
|
||||
La politique CPU hôte reste privée au Governor : masque permis, groupes
|
||||
package/core/SMT, compute-pool et résultat d'application du worker Queue. Le
|
||||
compute-pool borne l'admission de chaque kind. Feature/SIFT/RootSIFT utilisent
|
||||
le maximum `int` positif comme borne de l'API OpenCV, puis consomment le compte
|
||||
immutable réellement admis ; les CPU12 durables ne sont plus que des signatures
|
||||
historiques exactes. Les kinds CPU1 justifiés restent fixes. Aucun ID CPU ou
|
||||
choix d'affinité n'entre dans le descriptor, le checkpoint ou le Project DB.
|
||||
- Task display name;
|
||||
- callback address;
|
||||
- userdata address;
|
||||
- source file name;
|
||||
- persisted function pointer;
|
||||
- Task ID;
|
||||
- scientific fingerprint.
|
||||
|
||||
Le feedback ne requalifie pas un succès de reprise en travail durable : les
|
||||
kinds Feature/SIFT/RootSIFT comptent un item seulement après extraction et
|
||||
publication durable propre. READY, collision `ALREADY_PRESENT` ou publication
|
||||
incertaine compte zéro ; Visual Index compte pareillement zéro pour un segment
|
||||
`PUBLISHED_NOT_DURABLE`.
|
||||
No normalization is performed on the persisted kind string.
|
||||
|
||||
L'état privé par kind/backend coordonne désormais une seule dimension d'essai.
|
||||
Les CPU réductibles slow-startent par doubles successifs depuis 1, puis le
|
||||
maximum exact de leur capacité, toujours bornés par le compute-pool. Après deux
|
||||
observations de baseline, deux observations à au moins +5 % sont nécessaires
|
||||
pour accepter le palier.
|
||||
Une fois CPU stabilisé, seuls les kinds dont le callback consomme réellement
|
||||
son lot peuvent ouvrir un essai de lot. `features.extract`, SIFT et RootSIFT
|
||||
enregistrent une observation atomique réussie partagée entre Tasks ; Visual
|
||||
Index, Candidate et Matcher enregistrent chaque séquence. Les autres formes ou
|
||||
dimensions non adaptables restent égales à leur capacité fixe honnête.
|
||||
## Registry ownership
|
||||
|
||||
## Persistance et legacy
|
||||
The registry contains at most 64 descriptors in a static immutable array.
|
||||
|
||||
Le checkpoint générique reste en version 1. Project Database v7 conserve le
|
||||
kind/version ; les lignes migrées depuis v1 restent `NULL/NULL` et sont classées
|
||||
`LEGACY_UNTYPED`. Un kind inconnu ou une version non supportée reste inspectable
|
||||
mais inexécutable. Aucun type n'est inventé et aucun code n'est sélectionné par
|
||||
adresse persistée.
|
||||
Lookup is:
|
||||
|
||||
## Statut
|
||||
- linear;
|
||||
- deterministic;
|
||||
- allocation-free;
|
||||
- safe for concurrent readers.
|
||||
|
||||
**IMPLEMENTED** — identité typée immutable, registry statique, lookup,
|
||||
migration DB v1→v2, classification recovery et restauration explicite testée.
|
||||
The registry never dynamically loads code.
|
||||
|
||||
**IMPLEMENTED** — le descriptor production `import.images`, version 1, charge
|
||||
le chemin source borné depuis la table dédiée et reconstruit callback et
|
||||
userdata sans `AppState *` ancien.
|
||||
A descriptor contains only:
|
||||
|
||||
**IMPLEMENTED** — `project_open()` utilise la registry production immutable
|
||||
pour restaurer hors mutex DB et transférer chaque tâche acceptée à la queue.
|
||||
- Task Kind string;
|
||||
- Task Kind version;
|
||||
- reconstruction function.
|
||||
|
||||
**NOT_YET_WIRED** — réconciliation orpheline et dépendances/DAG. Les kinds
|
||||
reconstructibles checkpointent déjà à leurs frontières métier ; la Registry ne
|
||||
possède pas un timer autosave et ne doit pas devancer leurs curseurs durables.
|
||||
The public descriptor does not contain resource-policy state, CPU topology, GPU identity, scratch
|
||||
state or persisted scheduler configuration.
|
||||
|
||||
**IMPLEMENTED** — `features.extract` version 1 reconstruit une extraction ORB
|
||||
depuis `image_id` et ses paramètres bornés.
|
||||
## Reconstruction ownership
|
||||
|
||||
**IMPLEMENTED** — `visual_index.update`, version 1, recharge
|
||||
`visual_index_id + after_feature_set_id` et reconstruit un contexte neuf.
|
||||
A reconstruction function returns:
|
||||
|
||||
**IMPLEMENTED** — `candidate_pair.generate`, version 1, recharge
|
||||
`visual_index_id + after_feature_set_id + top_k + minimum_evidence_count
|
||||
+ scanset_filter + exclude_same_asset` depuis `candidate_pair_generate_tasks`
|
||||
et reconstruit un contexte boundé. La restauration reconnaît le snapshot
|
||||
opérationnel sériel historique v1 exact et la forme CPU12 immédiatement
|
||||
antérieure, puis les remplace éphémèrement par la demande courante CPU64 avec
|
||||
8 Mio par item avant admission. Le checkpoint historique et le curseur typé
|
||||
restent inchangés ; une forme voisine est rejetée.
|
||||
- callback;
|
||||
- callback userdata;
|
||||
- optional userdata destructor;
|
||||
- Task-specific private binding state where required.
|
||||
|
||||
**PASS / FROZEN — Compute Governor v2.** `matcher.run`, version 1,
|
||||
recharge la configuration Matcher, l'identité Feature Set et le curseur
|
||||
`after_candidate_pair_id`. Il traite une Candidate Pair atomique à la fois dans
|
||||
des lots bornés à douze, checkpoint le curseur et repasse par le Governor entre
|
||||
les lots. La table durable `matcher_tasks` est introduite par Project DB v11,
|
||||
après le Match Result v10. Son reconstructeur accepte les formes courantes
|
||||
CPU12/GPU0 et CPU1/GPU1/640 Kio à lot `1..12`, les signatures historiques
|
||||
CPU8/GPU0 et Vulkan à lot maximal 8, puis les formes CPU12
|
||||
pré-estimation-par-paire. La normalisation reste en mémoire. Une forme voisine
|
||||
échoue au lieu de servir d'indice de backend ; le payload Project DB ne change
|
||||
pas et ne persiste aucune identité matérielle. Les nouvelles Tasks ORB normales
|
||||
ont une signature de classe `MIXED`, dont les autres champs restent une demande
|
||||
de ressources réelle ; elle seule reconstruit la politique Governor `AUTO`.
|
||||
Toutes les formes ORB de classe `CPU`, anciennes ou courantes, reconstruisent
|
||||
un CPU fixe pour préserver les overrides explicites et une compatibilité sûre ;
|
||||
une forme Vulkan restaurée reste fixe Vulkan. Un build portable reconstruit la
|
||||
même politique `MIXED` mais n'expose que sa capacité CPU. Les snapshots tout à
|
||||
zéro Candidate/SIFT/RootSIFT sont explicitement corrompus ; seules leurs
|
||||
signatures historiques complètes exactes sont acceptées. Seule la forme AUTO
|
||||
restaurée établit la disponibilité Vulkan partagée. Restaurer ensuite CPU,
|
||||
Vulkan ou une signature historique fixe n'écrit rien dans cet état : la
|
||||
co-restauration est indépendante de l'ordre. Aucun nouvel état de
|
||||
backend n'est persisté.
|
||||
Before ownership transfer, the registry destroys any newly allocated userdata on failure.
|
||||
|
||||
Pour une Task AUTO, la Registry reconstruit aussi l'enveloppe privée Vulkan
|
||||
CPU1/GPU1, lot opérationnel `1..8`, helpers 0 et inflight 1. La signature
|
||||
durable historique reste à lot `1..12`; la signature 640 Kio
|
||||
reste la forme depth-1 minimale et n'est pas mutée; l'admission normale facture
|
||||
exactement 640 Kio une seule fois sur UMA. Le choix inflight est immutable dans
|
||||
la séquence et ne devient ni payload, ni fingerprint, ni indice de reprise.
|
||||
Vulkan explicite reste depth 1. La capacité privée de sûreté/benchmark peut
|
||||
forcer deux slots et 1,25 Mio sans changer la reconstruction normale.
|
||||
Le backend ne mappe pas le maximum de l'enveloppe à sa création : il retient
|
||||
exactement un slot à depth 1 et deux seulement sous une séquence depth 2 admise,
|
||||
puis libère le second avant l'admission suivante. La signature durable 640 Kio
|
||||
reste donc inchangée sans sous-facturer une allocation depth 2 forcée.
|
||||
After a successful `Lardon3DTask` restoration, the Task owns the userdata and destroys it exactly once
|
||||
after execution lifetime ends.
|
||||
|
||||
L'A/B forcé ABBA a mesuré 54,661652238 paires/s à depth 1 et 55,797311953 à
|
||||
depth 2, soit +2,077617 %, sous le deadband 5 %, avec digest identique, quatre
|
||||
séquences de fallback local par exécution et zéro panne/discard. La Registry conserve donc
|
||||
`DEPTH_MAX_VALIDATED_SAFETY=2` pour les seules coutures privées, mais la
|
||||
capacité AUTO normale suit `DEPTH_MAX_USEFUL=1` : depth 2 est
|
||||
**REJECTED_WITH_MEASURED_REASON**, sans nouvelle signature durable.
|
||||
A business-specific constructor is never called while the Project DB mutex is held.
|
||||
|
||||
La télémétrie privée de `matcher.run` conserve les classes de fallback par
|
||||
séquence et compte aussi les items exacts local-ineligible/backend-failure/other
|
||||
après leur publication durable. Ce détail opérationnel n'ajoute aucun kind,
|
||||
champ durable ou identité et empêche le regroupement batch de devenir un
|
||||
comparateur scientifique. Le commit immédiat par item reste acquis si une
|
||||
paire suivante avorte, tandis que le feedback de séquence n'est pas enregistré;
|
||||
la déduplication actuelle vit seulement avec la Task reconstruite en mémoire.
|
||||
Les logs batch 2/4 antérieurs à ce compteur restent préliminaires et prouvent
|
||||
seulement l'invalidité du comparateur par séquences. Les huit runs item-valides
|
||||
`forced-batch{2,4,8,12}-items{,-b}.stdout.jsonl` conservent chacun 4113 paires,
|
||||
six items locaux, zéro panne/autre et le même digest. Les débits combinés sont
|
||||
54,180767704, 66,094373197, 74,784998723 et 76,755814095 paires/s. Les gains
|
||||
jusqu'à batch 8 dépassent 5 %, celui de 8 à 12 vaut seulement +2,635308425 % :
|
||||
la Registry expose `BATCH_MAX_USEFUL=8` en AUTO normal et réserve batch 12 aux
|
||||
preuves privées (`REJECTED_WITH_MEASURED_REASON`).
|
||||
Le S21 final confirme l'enveloppe Registry en production : `matcher.run` v1
|
||||
reste le même kind durable, AUTO choisit Vulkan pour 21 630 admissions et
|
||||
termine 172 741/172 741 résultats à batch 8/inflight 1/helpers 0. Le passage
|
||||
transitoire 8 → 1 → 2 → 4 → 8 ne modifie ni signature durable, ni
|
||||
fingerprint, ni digest scientifique. Aucun backend ou champ persistant n'est
|
||||
ajouté par cette adaptation.
|
||||
Project-open recovery copies the durable record out of the DB boundary before registry lookup and
|
||||
Task-specific reconstruction.
|
||||
|
||||
La reconstruction AUTO ne sonde ni n'initialise Vulkan sur le thread
|
||||
`project_open()`. Elle expose la capacité depuis les seules métadonnées runtime
|
||||
build/backend/GPU ; le Governor possède le dimensionnement exact et
|
||||
l'admission UMA sur son snapshot. Le premier begin appartient au worker Queue après son
|
||||
affinité. La politique Mesa sûre est déjà établie avant les pthreads et
|
||||
l'initialisation du driver ; aucun sweep/latch auxiliaire n'appartient donc au
|
||||
contexte Task. Une paire localement inéligible n'initialise pas le backend. Une
|
||||
panne réelle produit des paires CPU complètes et rend le backend indisponible
|
||||
aux admissions AUTO suivantes sans réécrire le snapshot durable.
|
||||
## Exact legacy estimate normalization
|
||||
|
||||
Le contrôle de benchmark `synchronous` du runner réel n'étend pas le descriptor
|
||||
ni le reconstructeur. Il est compilé hors du binaire production, attaché
|
||||
seulement au contexte éphémère d'une nouvelle Task et refusé par le runner si
|
||||
une Task Matcher doit être reprise. La Registry continue donc à reconstruire
|
||||
uniquement la politique AUTO/fixe déduite de la signature durable, jamais un
|
||||
pipeline de benchmark.
|
||||
The registry may recognize exact historical operational resource descriptors for restart compatibility.
|
||||
|
||||
**IMPLEMENTED** — ORB, SIFT et RootSIFT acceptent leurs formes CPU12/CPU1
|
||||
historiques complètes et les normalisent vers la demande OpenCV portable
|
||||
`INT_MAX`. Le Governor borne l'exécution au compute-pool ; les sorties testées
|
||||
à 1/2/4/8/12 restent égales. Cette compatibilité opérationnelle n'altère ni
|
||||
fingerprint, Feature Set, checkpoint durable, ni politique scientifique.
|
||||
The reconstruction function always receives the original durable snapshot.
|
||||
|
||||
**IMPLEMENTED** — `geometric_verifier.run`, version 1, recharge la configuration
|
||||
Fundamental immuable, en revalide le fingerprint et reprend `after_match_result_id`.
|
||||
Project DB v13 ajoute uniquement `geometric_verifier_tasks`, car le checkpoint
|
||||
générique v1 ne possède aucun payload propre au kind.
|
||||
If an exact historical descriptor is recognized, the registry may replace only the private effective
|
||||
resource estimate passed to the restored runtime Task.
|
||||
|
||||
La forme historique série exacte (4 Mio fixes, CPU1, batch 1..8) est normalisée
|
||||
en mémoire vers 8 Mio par item, CPU utile 8 et batch maximal 16. Cette évolution
|
||||
ne touche ni fingerprint, GVR, ordre, curseur ni checkpoint historique ; une
|
||||
forme voisine est refusée.
|
||||
This normalization is:
|
||||
|
||||
**IMPLEMENTED** — `track_builder.run`, version 1, reconstruit un scope explicite
|
||||
depuis son payload Project DB v15 et son asset little-endian validé. Le callback
|
||||
réutilise l'orchestration Gate C et le reconstructeur refuse toute corruption,
|
||||
version, fingerprint, checksum, tri, unicité ou L3DTSIS1 incohérents.
|
||||
- ephemeral;
|
||||
- deterministic;
|
||||
- exact-shape only;
|
||||
- non-persistent.
|
||||
|
||||
**PASS / FROZEN** — `sparse_sfm.run`, version 1, recharge le
|
||||
payload scientifique explicite Project DB v17, restaure l'estimation générique
|
||||
persistée et rejoue D puis E depuis les références Track Set/calibration. Le
|
||||
fingerprint F0 est recalculé ; le checkpoint générique v1 reste inchangé.
|
||||
It never rewrites:
|
||||
|
||||
**PASS / FROZEN** — `incremental_reconstruction.run`,
|
||||
version 1, recharge le payload Project DB v18 composé du prédécesseur, du Track
|
||||
Set d'extension, du scope de calibration et du fingerprint H. La tâche atomique
|
||||
recalcule depuis ces entrées après redémarrage, passe par la Queue et le
|
||||
Governor avec son estimation H immuable, et ne persiste aucun état de solveur.
|
||||
Elle n'ajoute ni DAG ni dépendance implicite.
|
||||
- the generic checkpoint;
|
||||
- Task ID;
|
||||
- Task Kind/version;
|
||||
- Task progress;
|
||||
- Task-specific cursor;
|
||||
- scientific parameters;
|
||||
- scientific fingerprint;
|
||||
- Project DB payload.
|
||||
|
||||
A neighboring or partially matching historical resource descriptor is rejected rather than guessed.
|
||||
|
||||
A crash before a later terminal checkpoint may therefore repeat the same exact normalization on the
|
||||
next restart.
|
||||
|
||||
## Production inventory
|
||||
|
||||
The production registry is created by:
|
||||
|
||||
```text
|
||||
src/task_kinds.c::lardon3d_task_kind_registry_production()
|
||||
```
|
||||
|
||||
It contains exactly these sixteen version-1 kinds:
|
||||
|
||||
| # | Task Kind | Durable payload / introduction | Reconstruction |
|
||||
| ---: | --- | --- | --- |
|
||||
| 1 | `raw.develop` | Project DB v22 `raw_development_tasks` | `lardon3d_raw_development_task_reconstruct` |
|
||||
| 2 | `raw.develop.batch` | Project DB v24 `raw_development_batch_tasks` | `lardon3d_raw_development_batch_task_reconstruct` |
|
||||
| 3 | `photo_quality.triage` | Project DB v21 | `lardon3d_photo_quality_task_reconstruct` |
|
||||
| 4 | `acquisition_campaign.run` | Project DB v20 | `lardon3d_acquisition_campaign_task_reconstruct` |
|
||||
| 5 | `import.images` | historical typed import payload | `lardon3d_image_import_reconstruct` |
|
||||
| 6 | `features.extract` | historical Feature Task payload | `lardon3d_feature_extract_reconstruct` |
|
||||
| 7 | `features.extract.batch` | Project DB v25 `feature_extract_batch_tasks` | `lardon3d_feature_extract_batch_reconstruct` |
|
||||
| 8 | `features.extract.sift` | typed SIFT Feature payload | `lardon3d_sift_extract_reconstruct` |
|
||||
| 9 | `features.extract.rootsift` | typed RootSIFT Feature payload | `lardon3d_sift_extract_reconstruct` |
|
||||
| 10 | `visual_index.update` | Visual Index Task payload | `lardon3d_visual_index_update_reconstruct` |
|
||||
| 11 | `candidate_pair.generate` | Project DB v9 `candidate_pair_generate_tasks` | `lardon3d_candidate_pair_generate_reconstruct` |
|
||||
| 12 | `matcher.run` | Project DB v11 `matcher_tasks` | `lardon3d_matcher_task_reconstruct` |
|
||||
| 13 | `geometric_verifier.run` | Project DB v13 `geometric_verifier_tasks` | `lardon3d_geometric_verifier_task_reconstruct` |
|
||||
| 14 | `track_builder.run` | Project DB v15 `track_builder_tasks` | `lardon3d_track_builder_task_reconstruct` |
|
||||
| 15 | `sparse_sfm.run` | Project DB v17 `sparse_sfm_tasks` | `lardon3d_sparse_sfm_task_reconstruct` |
|
||||
| 16 | `incremental_reconstruction.run` | Project DB v18 `incremental_reconstruction_tasks` | `lardon3d_incremental_reconstruction_task_reconstruct` |
|
||||
|
||||
The current detailed resource-capability inventory is owned by
|
||||
[`resource_governor.md`](resource_governor.md), in its production sixteen-kind audit.
|
||||
|
||||
Historical documents that correctly recorded fewer kinds at their checkpoint remain historical
|
||||
evidence. They must not be rewritten merely to make their old count equal the current count.
|
||||
|
||||
## Current additive Task Kinds
|
||||
|
||||
The two Task Kinds added after the historical fourteen-kind maintenance inventory are:
|
||||
|
||||
```text
|
||||
raw.develop.batch/1
|
||||
features.extract.batch/1
|
||||
```
|
||||
|
||||
### `raw.develop.batch/1`
|
||||
|
||||
Project DB v24 binds one Task to one immutable selected execution.
|
||||
|
||||
The selected-execution cursor remains authoritative. Independent RAW items may be prepared within a
|
||||
bounded admitted window, participants join, and the owner publishes the selected prefix in canonical
|
||||
order.
|
||||
|
||||
The Task-specific restart path reuses exact already published RAW-derived representations and does not
|
||||
infer identity from path, basename or processing position.
|
||||
|
||||
### `features.extract.batch/1`
|
||||
|
||||
Project DB v25 binds one Task to:
|
||||
|
||||
- one immutable selected execution;
|
||||
- one monotonic `next_item_index`;
|
||||
- exact ORB kind/version/parameters/fingerprint.
|
||||
|
||||
Independent selected images may be prepared concurrently without participant SQLite publication.
|
||||
|
||||
After join, the owner validates/reuses the exact READY Feature Set, publishes in selected-item order,
|
||||
advances the typed cursor and then advances generic Task progress/checkpoint.
|
||||
|
||||
The historical `features.extract/1` kind remains valid and reconstructible.
|
||||
|
||||
```text
|
||||
PER_IMAGE_FEATURE_RESULT=ATOMIC
|
||||
PER_IMAGE_ATOMICITY_REQUIRES_CROSS_IMAGE_SERIALISM=NO
|
||||
OWNER_ONLY_PUBLICATION_REQUIRES_SERIAL_PREPARATION=NO
|
||||
```
|
||||
|
||||
## Compute Governor v2 private seam
|
||||
|
||||
```text
|
||||
COMPUTE_GOVERNOR_V2=PASS/FROZEN
|
||||
```
|
||||
|
||||
The public registry descriptor remains limited to kind, version and reconstruct function.
|
||||
|
||||
Resource capability alternatives live in private runtime state, including the opaque `Lardon3DTask`
|
||||
and private Task/Resource Governor seams.
|
||||
|
||||
The key production admission boundaries remain:
|
||||
|
||||
- Queue admissibility selection;
|
||||
- `lardon3d_task_sequence_break()`;
|
||||
- Resource Governor capability selection from one fresh snapshot.
|
||||
|
||||
This private capability envelope is not:
|
||||
|
||||
- scientific identity;
|
||||
- a new Project DB payload;
|
||||
- a Task Kind version;
|
||||
- a scheduler;
|
||||
- a dependency graph;
|
||||
- durable hardware identity.
|
||||
|
||||
The selected resource contract is immutable for one admitted sequence. A later sequence may receive a
|
||||
different admitted contract.
|
||||
|
||||
A Task with no valid alternatives receives the honest fixed capability represented by its contract.
|
||||
|
||||
## Host CPU policy
|
||||
|
||||
The Resource Governor owns host CPU policy.
|
||||
|
||||
It determines:
|
||||
|
||||
- allowed affinity mask;
|
||||
- package/core/SMT topology where available;
|
||||
- interactive reserve;
|
||||
- compute pool;
|
||||
- actual worker affinity;
|
||||
- per-kind admission bounds.
|
||||
|
||||
CPU IDs never enter:
|
||||
|
||||
- Task Kind descriptor;
|
||||
- generic checkpoint;
|
||||
- Project DB Task payload;
|
||||
- scientific fingerprint.
|
||||
|
||||
Reference-host observations such as a 12-logical-CPU compute pool are evidence for that host, not a
|
||||
portable registry ceiling.
|
||||
|
||||
The canonical repository policy is:
|
||||
|
||||
```text
|
||||
RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
|
||||
SERIALISM_REQUIRES_PROOF=CANONICAL
|
||||
```
|
||||
|
||||
After the interactive reserve and all safety constraints are preserved, useful validated compute
|
||||
capacity should not remain idle merely because an older durable resource descriptor was smaller.
|
||||
|
||||
## OpenCV kinds
|
||||
|
||||
The OpenCV-controlled kinds include:
|
||||
|
||||
```text
|
||||
features.extract
|
||||
features.extract.sift
|
||||
features.extract.rootsift
|
||||
```
|
||||
|
||||
Exact historical CPU1/CPU12 descriptors may be recognized for restart compatibility.
|
||||
|
||||
The effective current Task request can use the portable positive OpenCV upper bound, while the Resource
|
||||
Governor limits actual admission to the host compute pool.
|
||||
|
||||
The callback applies the admitted OpenCV thread count and restores the process-wide baseline on every
|
||||
exit path.
|
||||
|
||||
Controlled validation at 1/2/4/8/12 threads preserved the required scientific outputs for the
|
||||
validated ORB/SIFT/RootSIFT contracts.
|
||||
|
||||
Those thread counts are operational evidence and do not change:
|
||||
|
||||
- extractor fingerprint;
|
||||
- Feature Set identity;
|
||||
- Feature File format;
|
||||
- Task Kind version.
|
||||
|
||||
For `features.extract.batch/1`, cross-image participants are the primary concurrency mechanism.
|
||||
OpenCV CPU teams are not multiplied blindly inside every participant.
|
||||
|
||||
## Candidate Pair normalization
|
||||
|
||||
`candidate_pair.generate/1` reconstructs:
|
||||
|
||||
- `visual_index_id`;
|
||||
- `after_feature_set_id`;
|
||||
- `top_k`;
|
||||
- `minimum_evidence_count`;
|
||||
- `scanset_filter`;
|
||||
- `exclude_same_asset`.
|
||||
|
||||
Exact earlier CPU1 and CPU12 resource shapes may be normalized in memory.
|
||||
|
||||
The current validated Candidate capability is represented by the current source/runtime contract,
|
||||
including:
|
||||
|
||||
```text
|
||||
fixed RAM 256 KiB
|
||||
per-item RAM 8 MiB
|
||||
batch range 1..64
|
||||
CPU/batch coupled during useful scaling
|
||||
GPU none
|
||||
```
|
||||
|
||||
The registry does not reinterpret the old durable checkpoint as if it had been created with those
|
||||
current values.
|
||||
|
||||
## Matcher reconstruction
|
||||
|
||||
`matcher.run/1` reconstructs the immutable Matcher configuration and durable
|
||||
`after_candidate_pair_id` cursor.
|
||||
|
||||
Project DB v11 introduced the typed Matcher payload after Match Result persistence in v10.
|
||||
|
||||
### Durable backend-class compatibility
|
||||
|
||||
The reconstruct path accepts only explicitly supported exact historical/current resource signatures.
|
||||
|
||||
These include the validated classes required for:
|
||||
|
||||
- current CPU operation;
|
||||
- current ORB AUTO/MIXED policy;
|
||||
- explicit Vulkan operation;
|
||||
- exact historical CPU/Vulkan restart shapes.
|
||||
|
||||
A neighboring signature fails rather than being treated as a backend hint.
|
||||
|
||||
No hardware identity is persisted in `matcher_tasks`.
|
||||
|
||||
### ORB AUTO
|
||||
|
||||
Only the AUTO/MIXED durable class reconstructs the shared GPU-first policy.
|
||||
|
||||
On a portable build with no usable Vulkan backend, the same durable policy can reconstruct while
|
||||
exposing only the CPU capability.
|
||||
|
||||
Restoring a fixed CPU or fixed Vulkan historical Task does not mutate shared AUTO availability state.
|
||||
|
||||
Backend probing/initialization does not occur on the `project_open()` thread. Actual backend begin
|
||||
belongs to admitted Queue execution.
|
||||
|
||||
### Current useful ORB bounds
|
||||
|
||||
The current production AUTO policy retains:
|
||||
|
||||
```text
|
||||
GPU-first when eligible
|
||||
CPU fallback complete
|
||||
normal Vulkan inflight depth = 1
|
||||
validated private safety depth = 2
|
||||
normal useful Vulkan batch <= 8
|
||||
```
|
||||
|
||||
Depth 2 and Matcher batch 12 remain validated private/safety or benchmark capabilities where
|
||||
applicable, but did not meet the useful-throughput deadband for normal AUTO policy.
|
||||
|
||||
Those performance decisions do not change:
|
||||
|
||||
- `matcher.run/1`;
|
||||
- Matcher fingerprint;
|
||||
- Match Result identity;
|
||||
- Match File bytes;
|
||||
- durable cursor.
|
||||
|
||||
## Geometric Verifier reconstruction
|
||||
|
||||
`geometric_verifier.run/1` reloads:
|
||||
|
||||
- immutable Fundamental verifier configuration;
|
||||
- validated verifier fingerprint;
|
||||
- `after_match_result_id`.
|
||||
|
||||
Project DB v13 adds only its typed Task payload.
|
||||
|
||||
The exact historical serial resource descriptor is recognized for compatibility and may be normalized
|
||||
ephemerally to the current outer-parallel capability.
|
||||
|
||||
The current validated execution shape allows:
|
||||
|
||||
- up to 8 useful participants;
|
||||
- a safe admitted window up to 16 Match Results;
|
||||
- 8 MiB per admitted parent;
|
||||
- owner-only ordered publication.
|
||||
|
||||
The scientific USAC/MAGSAC inner solver remains in its validated serial configuration. Outer
|
||||
cross-item concurrency does not change verifier science.
|
||||
|
||||
## Track Builder reconstruction
|
||||
|
||||
`track_builder.run/1` reconstructs the explicit immutable scope from:
|
||||
|
||||
- Project DB v15 typed Task payload;
|
||||
- validated little-endian scope asset;
|
||||
- exact builder selector/fingerprint;
|
||||
- exact GVR scope identity.
|
||||
|
||||
Corruption, unsupported version, fingerprint mismatch, checksum mismatch, non-canonical ordering,
|
||||
duplicates or invalid scope identity make the Task non-reconstructible.
|
||||
|
||||
Track Builder restart does not invent a scope from current Project DB contents.
|
||||
|
||||
## Sparse SfM reconstruction
|
||||
|
||||
```text
|
||||
sparse_sfm.run/1
|
||||
```
|
||||
|
||||
is **PASS/FROZEN**.
|
||||
|
||||
It reloads the explicit Project DB v17 payload, restores the persisted generic Task estimate and
|
||||
replays the frozen Sparse SfM D/E execution from explicit Track Set and calibration-scope references.
|
||||
|
||||
F0 is recomputed from the scientific payload.
|
||||
|
||||
The generic checkpoint remains version 1.
|
||||
|
||||
The historical fixed CPU1/batch1 Sparse SfM estimate is part of that frozen Task contract. It is not a
|
||||
global argument for serializing unrelated modern Task Kinds.
|
||||
|
||||
## Incremental reconstruction
|
||||
|
||||
```text
|
||||
incremental_reconstruction.run/1
|
||||
```
|
||||
|
||||
is **PASS/FROZEN**.
|
||||
|
||||
It reloads the Project DB v18 payload containing:
|
||||
|
||||
- predecessor reconstruction;
|
||||
- extension Track Set;
|
||||
- calibration scope;
|
||||
- Phase H fingerprint.
|
||||
|
||||
Restart recomputes from those immutable inputs.
|
||||
|
||||
No solver state, generic DAG or hidden dependency edge is persisted.
|
||||
|
||||
## Generic checkpoint and Project DB boundary
|
||||
|
||||
The generic checkpoint codec remains version 1.
|
||||
|
||||
Project DB stores Task Kind and Task Kind version in the generic Task summary.
|
||||
|
||||
Rows migrated from the oldest untyped schema remain:
|
||||
|
||||
```text
|
||||
task_kind = NULL
|
||||
task_kind_version = NULL
|
||||
```
|
||||
|
||||
and are classified as:
|
||||
|
||||
```text
|
||||
LEGACY_UNTYPED
|
||||
```
|
||||
|
||||
They remain inspectable but cannot be reconstructed as a typed production Task.
|
||||
|
||||
Unknown kinds are classified separately from unsupported versions.
|
||||
|
||||
A future unknown Task Kind or unsupported Task Kind version must never trigger guessed code execution.
|
||||
|
||||
No function address is persisted.
|
||||
|
||||
## Project-open recovery
|
||||
|
||||
`project_open()` uses the immutable production registry to restore eligible Tasks.
|
||||
|
||||
The recovery flow is conceptually:
|
||||
|
||||
```text
|
||||
load bounded Task page
|
||||
-> copy durable record outside DB mutex
|
||||
-> validate checkpoint and summary
|
||||
-> classify Task Kind/version
|
||||
-> reconstruct Task-specific binding
|
||||
-> apply exact legacy resource normalization when allowed
|
||||
-> restore Lardon3DTask
|
||||
-> enqueue through normal Queue
|
||||
```
|
||||
|
||||
A full Queue window stops further recovery without mutating unprocessed Tasks.
|
||||
|
||||
Task-local failures remain task-local unless the project schema/identity itself is invalid.
|
||||
|
||||
## Resource feedback is not scientific state
|
||||
|
||||
Private feedback may record bounded throughput observations per Task Kind/backend.
|
||||
|
||||
It may influence a later resource contract only where the capability contract permits adaptation.
|
||||
|
||||
It must not modify:
|
||||
|
||||
- scientific inputs;
|
||||
- Task-specific durable cursor;
|
||||
- Project DB identity;
|
||||
- Task Kind/version;
|
||||
- generic checkpoint codec;
|
||||
- scientific fingerprint;
|
||||
- canonical result ordering.
|
||||
|
||||
Only successfully completed durable work counts as useful work for adaptive feedback where that Task
|
||||
contract requires it.
|
||||
|
||||
Uncertain publication does not become a successful scientific sample.
|
||||
|
||||
## Scratch boundary
|
||||
|
||||
The current production registry contains sixteen Task Kinds, but none has an authoritative
|
||||
scratch-consuming scientific Task contract.
|
||||
|
||||
```text
|
||||
CURRENT_PRODUCTION_TASK_KINDS=16
|
||||
CURRENT_SCRATCH_CONSUMING_TASK_KINDS=0
|
||||
```
|
||||
|
||||
The external SSD controller and Resource Governor scratch wrappers remain operational infrastructure.
|
||||
|
||||
Scratch, swap and zram do not enlarge RAM admission.
|
||||
|
||||
A future scratch consumer requires an explicit Task-specific eligibility, lifetime and cleanup
|
||||
contract. Merely registering the SSD does not change a Task Kind.
|
||||
|
||||
## What the registry does not own
|
||||
|
||||
The registry does not own:
|
||||
|
||||
- Queue scheduling;
|
||||
- resource admission;
|
||||
- CPU topology;
|
||||
- GPU selection;
|
||||
- scratch allocation policy;
|
||||
- Project DB migration;
|
||||
- scientific fingerprint definitions;
|
||||
- artifact path resolution;
|
||||
- dependency/DAG planning;
|
||||
- autosave timers;
|
||||
- orphan reconciliation.
|
||||
|
||||
It only provides typed durable dispatch and reconstruction ownership.
|
||||
|
||||
## Current intentionally unfinished work
|
||||
|
||||
These remain outside the registry contract:
|
||||
|
||||
- global orphan-file reconciliation;
|
||||
- generic dependency/DAG scheduling;
|
||||
- any future new Task Kind not explicitly added to the production array;
|
||||
- future resource-capability changes requiring new proof.
|
||||
|
||||
Existing reconstructible Task Kinds already checkpoint at their Task-specific durable boundaries.
|
||||
The registry must not move those cursors forward with an independent autosave timer.
|
||||
|
||||
## Current checkpoint evidence
|
||||
|
||||
The current registry inventory includes both additive selected-execution kinds:
|
||||
|
||||
```text
|
||||
raw.develop.batch/1
|
||||
features.extract.batch/1
|
||||
```
|
||||
|
||||
The retained A6000 checkpoint is:
|
||||
|
||||
```text
|
||||
real-a6000-pre-sfm-2026-09-02
|
||||
REAL_A6000_PRE_SFM=PASS/FROZEN
|
||||
```
|
||||
|
||||
That proof exercised the current upstream/downstream registry through:
|
||||
|
||||
- selected RAW batch;
|
||||
- selected Feature batch;
|
||||
- Visual Index;
|
||||
- Candidate Pair;
|
||||
- Matcher;
|
||||
- Geometric Verifier v3;
|
||||
- Track Builder;
|
||||
|
||||
and stopped before real Sparse SfM.
|
||||
|
||||
The current production inventory remains exactly sixteen Task Kinds.
|
||||
|
||||
## Summary
|
||||
|
||||
```text
|
||||
CURRENT_PROJECT_DB_SCHEMA=v25
|
||||
CURRENT_PRODUCTION_TASK_KINDS=16
|
||||
CURRENT_SCRATCH_CONSUMING_TASK_KINDS=0
|
||||
|
||||
TASK_KIND_REGISTRY_STATUS=IMPLEMENTED
|
||||
TASK_KIND_REGISTRY_CAPACITY=64_DESCRIPTORS
|
||||
GENERIC_CHECKPOINT_VERSION=1
|
||||
|
||||
RAW_BATCH_TASK=raw.develop.batch/1
|
||||
FEATURE_BATCH_TASK=features.extract.batch/1
|
||||
|
||||
COMPUTE_GOVERNOR_V2=PASS/FROZEN
|
||||
ORB_VULKAN_ASYNC_EXECUTION=PASS/FROZEN
|
||||
|
||||
RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
|
||||
SERIALISM_REQUIRES_PROOF=CANONICAL
|
||||
|
||||
REAL_A6000_PRE_SFM=PASS/FROZEN
|
||||
```
|
||||
|
|
|
|||
Loading…
Reference in a new issue