diff --git a/AGENTS.md b/AGENTS.md index a8a0fff..f3ea997 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -49,7 +49,7 @@ or persistence format. Historical references to older Project DB versions remain valid when they describe the actual historical contract or migration path. Do not rewrite -legitimate v16/v17/v18/v19 history merely because v20 is current. +legitimate v16/v17/v18/v19 history merely because v22 is current. When a ticket declares `NO_NEW_SUBSYSTEM`, do not introduce an unrelated: @@ -552,7 +552,7 @@ Persistence changes require explicit attention to: For Project DB: -- preserve current v20 semantics unless a ticket explicitly authorizes a schema +- preserve current v22 semantics unless a ticket explicitly authorizes a schema change; - schema-version changes require explicit human authorization; - migrations must be additive unless a different migration is explicitly diff --git a/docs/architecture/internal_parallelism.md b/docs/architecture/internal_parallelism.md index 6be9b2b..714e87f 100644 --- a/docs/architecture/internal_parallelism.md +++ b/docs/architecture/internal_parallelism.md @@ -2,26 +2,61 @@ ## Statut -**PASS / FROZEN — INTERNAL_PARALLELISM_COMPUTE_RESOURCES_V1.** Ce document -décrit le contrat validé pour `features.extract`, `features.extract.sift`, -`visual_index.update`, `candidate_pair.generate` et `matcher.run`. Ce gel porte -sur le parallélisme interne borné, les réservations Governor et les sorties -canoniques ; il ne transforme pas les résultats de microbenchmarks GPU en une -preuve de débit durable sur corpus sous pression hôte. +**INTERNAL_PARALLELISM_COMPUTE_RESOURCES_V1 — PASS / FROZEN.** +**COMPUTE_GOVERNOR_V2 — PASS / FROZEN.** +**ORB_VULKAN_ASYNC_EXECUTION — PASS / FROZEN.** + +Ce document décrit le contrat validé pour `features.extract`, +`features.extract.sift`, `visual_index.update`, `candidate_pair.generate` et +`matcher.run`. Le gel v1 porte sur le parallélisme interne borné, les +réservations Governor et les sorties canoniques. Il ne préjuge pas du choix +opérationnel CPU/Vulkan ajouté par la tranche v2. ## Frontière runtime La Task Queue conserve un seul callback actif. Une Task peut exploiter à l'intérieur de ce callback les `cpu_threads` effectivement admis par le Resource Governor. Ce parallélisme interne ne crée ni pool global, ni deuxième -scheduler, ni file de travail persistante. L'affinité CPU éventuelle appartient -au lanceur ou à l'hôte : la bibliothèque ne choisit et ne persiste aucun ID de -CPU. +scheduler, ni file de travail persistante. Le Governor dérive désormais un +masque privé depuis l'affinité permise et la topologie package/core Linux. Il +réserve des coeurs physiques complets, frères SMT inclus, en ordre décroissant +package/core. Sur l'hôte unrestricted validé, cela donne `0-5,8-13` pour le +calcul lourd et `6,7,14,15` pour le desktop, l'audio et l'interaction. Un masque +caller déjà borné à la capacité de calcul est repris sans seconde réserve ; si +topologie ou affinité manque, seul le budget portable est conservé et aucun ID +n'est inventé. + +L'unique worker Queue applique et vérifie son propre masque (`pid=0`) avant les +callbacks ; le creator/main/TUI reste inchangé et les enfants créés depuis le +worker héritent normalement de son affinité. Aucun TID auxiliaire énuméré n'est +passé à `sched_getaffinity()` ou `sched_setaffinity()` : retenir un +`PIDFD_THREAD` ne réserve pas le numéro TID consommé par ces interfaces. Avant +toute création de pthread applicatif et toute initialisation Vulkan, le +démarrage établit donc `MESA_SHADER_CACHE_DISABLE=true`. L'absence de variable +prend ce défaut sûr et une valeur explicite exacte `true` ou `1` est respectée ; +une valeur explicite fausse ou malformée est préservée mais le démarrage est +refusé. Sur la 780M validée, cette politique supprime les helpers de cache +`*:disk$0` qui élargissaient leur masque, et les threads runtime restants +conservent le compute-pool hérité. Elle est inoffensive hors Mesa, n'est ni +persistée ni scientifique, et son état/sa raison sont diagnostiqués sans faux +indicateur de recontrainte auxiliaire. +Le backend Vulkan possède en plus une barrière tardive non mutante : avant son +premier appel Mesa, une requête non vide exige la valeur exacte `true` ou `1`. +Absent, faux ou malformé produit un backend `UNAVAILABLE` mémorisé et aucune +sortie partielle. `backend_info` et les requêtes vides restent non initialisants. +Cette défense couvre les consumers publics ou de feasibility qui ne traversent +pas la Queue ; seuls leurs `main` autonomes peuvent prendre le défaut sûr avant +tout pthread. +Le compute-pool borne `cpu_threads` à l'admission. Un échec d'application est +diagnostiqué sans altérer Task, Queue, durabilité ou science. Cette couture v2 +est **PASS / FROZEN**, sans persistance ni ABI publique. Elle complète le gel +v1 sans le redéfinir. ## Audit GPU — 29 août 2026 -Cet audit est une constatation de capacité et une recommandation de périmètre ; -il n'ajoute aucun backend ni aucune politique du Governor. La machine contrôlée +Cet audit est une constatation de capacité et une recommandation de périmètre. +La tranche v2 en tire une politique GPU-first seulement pour un backend validé, +déterministe et mesurément supérieur. La machine contrôlée expose `AMD Radeon 780M Graphics (RADV PHOENIX)`, iGPU Vulkan API 1.4.354 sous Mesa 26.2.1. OpenCV 5.0.0 y indique le chargement dynamique de Vulkan et d'OpenCL, avec TBB comme framework parallèle ; ses modules CUDA ne sont pas @@ -29,7 +64,11 @@ disponibles. L'outil `clinfo` n'était pas installé : cette absence ne prouve n ne valide une exécution OpenCL. Elle ne constitue donc pas une couture GPU utilisable par Lardon3D. -La 780M est UMA. Toute mémoire de travail GPU, toute double résidence +La 780M est UMA. Les fichiers amdgpu de l'hôte publient un petit aperture VRAM +de 512 Mio mais un GTT système de 7 986 020 352 octets ; Hardware Profile garde +le premier comme capacité de payload observable et le classe shared d'après ces +preuves bornées, sans liste de device IDs. Toute mémoire de travail GPU, toute +double résidence CPU/GPU et tout staging host-visible consomment la RAM hôte et doivent être comptés une seule fois par le Resource Governor, conformément à la règle UMA du [Resource Boundary](resource_boundary.md#selected-gpu) ; ils ne créent pas @@ -43,7 +82,7 @@ transfert/synchronisation à mesurer : l'UMA ne les rend pas gratuites. | `features.extract` (ORB/SIFT) | Extraction OpenCV CPU ; aucune couture GPU de production. | Aucune API d'extraction GPU n'est validée. Les indicateurs OpenCV Vulkan/OpenCL dynamiques ne fournissent pas à eux seuls un backend d'extraction ; CUDA est indisponible dans ce build. | Il faudrait prouver l'algorithme, les keypoints, descripteurs, ordre et Feature File produits. Les descripteurs et leurs buffers devraient être résidents ou stagés en RAM UMA ; le transfert et la synchronisation risquent de dominer les images bornées. Bénéfice non mesuré, complexité élevée. | Rester CPU. Une étude ne peut commencer qu'avec une API/backend concret et une preuve d'équivalence. | | `visual_index.update` | Construction et publication CPU ; aucune couture GPU. | Aucun backend/API GPU validé. | Les postings, leur compaction, tri total et publication déterministe sont aujourd'hui CPU. Un backend devrait préserver exactement `table_id,key24,feature_set_id,feature_index`, le segment, SHA-256 et les memberships ; il ajouterait double résidence UMA et synchronisation pour un bénéfice non mesuré. Complexité élevée. | Rester CPU ; aucun chantier GPU n'est justifié par cet audit. | | `candidate_pair.generate` | Requête Visual Index, top-K et publication CPU ; aucune couture GPU. | Aucun backend/API GPU validé. | Les scores, top-K, tie-breaks, normalisation et ordre de publication sont canoniques. Une accélération devrait rendre ces résultats identiques malgré les accès DB et les petits résultats bornés ; copies/synchronisations UMA et l'accès persistant réduisent le bénéfice attendu. Bénéfice non mesuré, complexité élevée. | Rester CPU ; ne pas introduire un backend GPU ou une seconde politique de sélection. | -| `matcher.run` | BFMatcher CPU ; couture existante limitée au backend Vulkan ORB sériel. Le mode parallèle CPU ne réserve ni n'utilise le GPU. | Vulkan compute existant pour ORB/Hamming top-2 uniquement. Aucun chemin Vulkan/OpenCL/CUDA validé pour SIFT/RootSIFT. | ORB Vulkan déjà prouve une parité top-2 et Match File complète avec le CPU, ce qui permet l'identité persistante commune. Les dispatchs restent soumis aux buffers descriptors et à la synchronisation UMA. SIFT/RootSIFT Vulkan n'est pas équivalent sur égalités adversariales et reste CPU. | Conserver le chemin ORB Vulkan sériel explicitement admis et le CPU parallèle séparé. Ne pas étendre à SIFT/RootSIFT ni fusionner les modes sans nouvelle validation. | +| `matcher.run` | BFMatcher CPU jusqu'à 12 participants ; backend Vulkan ORB exact en conversion begin/finish backend-owned. | Vulkan compute existant pour ORB/Hamming top-2 uniquement. Aucun chemin Vulkan/OpenCL/CUDA validé pour SIFT/RootSIFT. | ORB Vulkan prouve la parité top-2 et Match File complète avec le CPU. La 780M UMA impose de compter buffers et staging une fois en RAM hôte ; SIFT/RootSIFT restent CPU. | Workload GPU primaire validé et supérieur : AUTO GPU-first, CPU fallback. Ne pas étendre à SIFT/RootSIFT ni changer l'identité persistante. | L'attente « sortie identique » est une exigence de preuve, non une présomption attachée au GPU. Un futur backend ne peut partager une identité, un fingerprint @@ -57,14 +96,19 @@ prouvée à cette règle d'identité commune. ### Décision GPU finale — Radeon 780M La validation de production de `matcher.run` distingue l'identité scientifique -de son mode opérationnel. Les API historiques créent toujours le Matcher CPU -parallèle (`CPU=8`, `GPU=0`). L'API additive de mode explicite peut créer ORB -Vulkan seulement avant admission, avec `CPU=1`, un slot GPU et 640 Kio UMA. -Cette réservation couvre les buffers persistants A (256 Kio), B (256 Kio) et -top-2 (128 Kio); les 10 Mio de stage Matcher restent la mémoire CPU par paire. -Une panne de dispatch reprend entièrement le top-2 CPU avant toute publication; -elle ne publie jamais une sortie GPU partielle ni ne change fingerprint, -identité Match Result ou SHA du Match File. +de son mode opérationnel. La production normale ORB crée une enveloppe AUTO +CPU12/GPU0 et ORB Vulkan CPU1/GPU1; les API explicites conservent une seule +forme depth 1. Sa signature durable `MIXED` reste CPU12/GPU0 et décrit cette +politique, tandis que l'override CPU persiste la classe `CPU`. Chaque slot vaut +640 Kio: buffers A (256 Kio), B (256 Kio) et top-2 (128 Kio). AUTO normal +réserve 640 Kio à inflight 1. La capacité privée de sûreté/benchmark peut +réserver 1,25 Mio à depth 2. Le backend ne mappe aucun slot avant initialisation, +retient exactement la capacité admise pendant la séquence et libère le second +avant l'admission depth 1 suivante; `backend_info` rapporte cette rétention +réelle. Les 10 Mio de stage restent la mémoire CPU par paire. +Une panne de dispatch reprend entièrement le top-2 CPU avant +toute publication ; elle ne publie jamais une sortie GPU partielle et ne +change ni fingerprint, ni identité Match Result, ni SHA du Match File. Sur l'hôte contrôlé (`AMD Radeon 780M Graphics (RADV PHOENIX)`, Vulkan 1.4.354, Mesa 26.2.1), le backend a créé le device et exécuté les dispatchs réels. Les @@ -78,20 +122,22 @@ mesures de kernel chaud, distinctes du coût Task/SQLite/checkpoint, sont : La parité top-2, Match File et publication durable est couverte par une Task Queue réelle à 769 × 769 descriptors, par une répétition déterministe et par -le fallback forcé. La reprise accepte seulement les signatures entières CPU8, -Vulkan CPU1 et les deux signatures CPU12 historiques; celles-ci sont -normalisées éphémèrement vers leur forme courante avant admission. Une signature voisine -est rejetée. La configuration portable `-Dvulkan_orb=disabled` refuse le mode -Vulkan avant allocation de Task ID et conserve le Matcher CPU. +le fallback forcé. La forme explicite CPU12/GPU0, la signature AUTO `MIXED` +CPU12/GPU0 et la forme Vulkan CPU1/GPU1 utilisent des lots `1..12` et 10 Mio +par paire. La reprise accepte +en plus seulement les anciennes signatures entières CPU8/GPU0 et Vulkan +CPU1/GPU1 à lot maximal 8, puis les formes CPU12 antérieures au coût par paire. +Elles sont des signatures historiques de récupération, normalisées +éphémèrement avant admission ; une forme voisine est rejetée. La configuration +portable `-Dvulkan_orb=disabled` refuse Vulkan et conserve le fallback CPU. -`HOST_PRESSURE_CONTAMINATED=TRUE` pour une comparaison de débit de corpus : au -moment de l'audit, `MemAvailable` était ~7,6 Gio, 7,6 Gio de swap étaient en -usage, même si la PSI mémoire courante était nulle. Il n'existe donc pas de -mesure défendable CPU t1/t6/t12 contre corpus Vulkan dans cette tranche. Le -Matcher CPU durable actuel plafonne en outre à huit participants utiles; un -CPU t12 ne serait pas une comparaison de Task valide. Les chiffres ci-dessus -sont volontairement limités au kernel commun et ne sélectionnent pas une -politique AUTO ou le backend par défaut. +Le gel v1 ne revendiquait pas de comparaison de débit corpus saine sous la +pression hôte alors observée. L'évidence directe apportée à la tranche v2 +établit désormais ORB Vulkan comme backend déterministe, exact et mesurément +supérieur pour ce hot path. La politique canonique v2 est donc GPU-first pour +ORB Matcher lorsque son contrat GPU/UMA est admissible, avec fallback CPU. +Cette décision opérationnelle ne rouvre pas le gel scientifique v1 et ne +s'étend pas aux trois domaines GPU rejetés ci-dessous. Les classifications finales sont : @@ -100,29 +146,42 @@ Les classifications finales sont : | `candidate_pair.generate` | `CANDIDATE_GPU=REJECTED_WITH_MEASURED_REASON` | Le coût est Visual Index, filtrage, branchement et publication SQLite ordonnée; aucune primitive GPU existante ne préserve ces identités et le CPU parallèle Candidate a déjà démontré 7,114× à t12. | | `features.extract` | `FEATURE_GPU=REJECTED_WITH_MEASURED_OR_IMPLEMENTATION_EVIDENCE` | OpenCV 5.0.0 installé n'expose aucun ORB/SIFT Vulkan/OpenCL utilisable, CUDA est absent, et aucun seam existant ne peut prouver les Feature Files byte-identiques. | | `visual_index.update` | `VISUAL_INDEX_GPU=REJECTED_WITH_MEASURED_REASON` | Postings, tri total, SHA et publication déterministe sont CPU; aucun kernel GPU borné existant ne couvre cette frontière. | -| `matcher.run` ORB | `MATCHER_GPU=EXISTING_BACKEND_VALIDATED_BUT_CPU_PREFERRED` | Le backend est réellement admis, dispatché et exact, mais la pression hôte et l'absence d'une comparaison de débit Task/corpus saine interdisent de préférer Vulkan au CPU parallèle comme défaut. | +| `matcher.run` ORB | `MATCHER_GPU=EXISTING_BACKEND_VALIDATED_AND_PREFERRED` | Le backend est exact, déterministe et mesurément supérieur pour le hot path validé. AUTO le préfère quand GPU/UMA sont admis et conserve le CPU en fallback. | -Cette dernière classification ne rejette pas le backend : ORB Vulkan reste un -mode explicite validé. Elle interdit seulement de présenter les microbenchmarks -comme une preuve que le mode sériel GPU bat le Matcher CPU parallèle dans un -corpus durable complet. +Le comportement de production normal est `AUTO`, choisi par le Governor à +chaque admission de séquence. Les modes CPU/Vulkan explicites restent des +overrides de debug, benchmark et reproductibilité. L'enveloppe privée +CPU/Vulkan, le fallback CPU complet et les diagnostics bornés sont +**PASS / FROZEN** dans les limites validées. ## Extraction OpenCV -Le processus configure une seule fois la limite de threads interne OpenCV, -avant la création de la Queue, au budget interactif audité -`min(12, logical_cpu_count - system_cpu_reserve)`. Les Tasks ORB et SIFT demandent -le plafond douze au Governor, qui réduit l'admission à ce même budget ; la -réservation vit pendant l'exécution bornée d'une -image et est libérée avec la Task. OpenCV possède son fan-out interne, tandis -que la Queue reste propriétaire de l'unique callback actif. +Le processus conserve une limite OpenCV globale. Sous l'unique callback Queue, +RAW et Photo Quality appliquent/restaurent CPU1. ORB, SIFT et RootSIFT +Extraction appliquent/restaurent exactement le `cpu_threads` immutable admis +dans `1..min(12, compute_pool)`. Matcher applique OpenCV1 à l'intérieur de sa +propre fenêtre et utilise les participants Task admis autour de cette +primitive. La réservation vit pendant l'exécution bornée et aucun second pool +runtime n'est créé. Le nombre admis est une politique de ressources, pas une identité scientifique. -L'audit OpenCV 5.0.0 contrôlé à 1/2/4/8/12 threads obtient des Feature Files -ORB v1 et SIFT v2 byte-identiques. Aucun fingerprint, format ou schéma n'est -modifié. Les utilisateurs imbriqués d'OpenCV qui réduisent temporairement cette -limite doivent restaurer sa valeur avant de libérer leur réservation ; ils ne -peuvent pas faire varier la configuration pendant une extraction concurrente. +Les tests OpenCV 5.0.0 à 1/2/4/8/12 threads obtiennent les mêmes keypoints, +descripteurs et métriques ORB, SIFT et RootSIFT. Aucun fingerprint, format ou +schéma n'est modifié. Les utilisateurs imbriqués d'OpenCV ne peuvent pas faire +varier cette configuration process-wide pendant une extraction. Le callback +assure donc qu'une admission CPU est réellement consommée, sans confondre cette +dimension avec le lot admis. + +Le Governor slow-start les dimensions CPU validées selon `1/2/4/8/12`. Chaque +palier utilise deux observations et exige au moins 5 % de débit durable en plus. +Un seul essai CPU ou lot est actif à la fois ; après plafonnement ou refus CPU, +un kind dont le lot est adaptable peut seulement alors explorer le lot. Les +callbacks atomiques ORB/SIFT/RootSIFT publient un item seulement après extraction +et publication durable propre ; READY, `ALREADY_PRESENT` ou +`PUBLISHED_NOT_DURABLE` publie une observation zéro qui ne fait progresser aucun +essai. Visual Index suit la même règle par +segment, tandis que Candidate et Matcher observent chaque séquence. Les formes +fixes restent fixes mais conservent un diagnostic d'admission Governor-owned. Pour Candidate Generation, l'estimation demande jusqu'à douze threads CPU et un slot I/O. Le callback de Queue compte comme un de ces threads ; il crée donc @@ -246,29 +305,126 @@ de la Task. Les sources Matcher sont les Candidate Pairs durables, pagées en ordre croissant de `candidate_pair_id`. Une fenêtre contient au plus -`2 * cpu_threads` paires et jamais plus de huit. Le callback Queue compte comme +`2 * cpu_threads` paires et jamais plus de douze. Le callback Queue compte comme un participant et crée donc au plus `cpu_threads - 1` enfants ; tous sont joints avant publication, libération de réservation ou `sequence_break`. -Chaque participant calcule un Match File temporaire privé. OpenCV est configuré -à un thread interne pendant la séquence afin que `BFMatcher` ne crée pas un -second fan-out sous les participants admis. Le backend Vulkan ORB partagé reste -utilisé uniquement par un mode déclaré sériel avant admission, dont l'estimation -réserve un thread CPU et le GPU. Le mode parallèle reste CPU-only même si le -Governor ne lui accorde finalement qu'un participant ; il ne sélectionne donc -jamais tardivement une ressource non réservée. Ce choix runtime ne modifie ni le -fingerprint, ni Lowe ratio, ni les correspondances brutes, ni l'identité Match -Result. +Chaque participant CPU calcule un Match File temporaire privé. OpenCV reste à +un thread interne pendant cette séquence afin que `BFMatcher` ne crée pas un +second fan-out. Le Governor sélectionne avant admission soit la forme +CPU1..12/GPU0, soit ORB Vulkan CPU1/GPU1, 640 Kio UMA et inflight 1. +Les participants +CPU sont décrits par `cpu_threads`; `helpers` reste zéro. Le fallback CPU +complet est choisi avant la séquence si GPU/UMA n'est pas admis ; une panne de +backend ou une paire sous le seuil Vulkan pendant une séquence reprend aussi le +top-2 CPU exact dans la réservation immutable. Le diagnostic distingue backend +sélectionné et backend réel, y compris plusieurs paires complètes CPU/Vulkan. +Ce choix ne +modifie ni fingerprint, ni Lowe ratio, ni correspondances brutes, ni identité +Match Result. -Les APIs historiques sélectionnent toujours le mode CPU parallèle ; le mode -ORB Vulkan est demandé par une API additive explicite et n'est jamais activé -par variable d'environnement ou réduction Governor. Sa création exige ORB, un -GPU sélectionné et un backend présent. Après reprise, la forme immutable GPU -peut conserver le fallback CPU exact si le backend runtime n'est plus -disponible. Les signatures courantes complètes sont CPU8/GPU0 et -CPU1/GPU1/640 Kio ; les signatures historiques CPU12 correspondantes sont -normalisées éphémèrement comme un tout avant admission. Toute forme -voisine est rejetée afin de ne pas inférer un mode depuis un champ isolé. +La couture v2 emploie une enveloppe privée et le mode `AUTO` Governor-owned. +CPU/Vulkan explicites restent des overrides de +debug, benchmark et reproductibilité. Une fois une séquence admise, son +contrat est immutable jusqu'à libération ; seul le contrat de la séquence +suivante peut changer. Une Task ORB normale nouvelle persiste la classe +`MIXED` avec ses champs CPU12/GPU0 réels ; elle seule reconstruit AUTO. Les +signatures CPU ORB courantes ou historiques reconstruisent CPU fixe, et les +signatures Vulkan reconstruisent Vulkan fixe. Le build portable conserve +`MIXED` mais expose seulement CPU. Les signatures historiques exactes CPU8, +Vulkan à lot maximal 8 et CPU12 antérieures au coût par paire restent reconnues; +toute forme voisine est rejetée. + +Le pipeline Vulkan rolling repose uniquement sur la couture privée interne. +Device/pipeline/layout/cache restent partagés; deux slots maximum dupliquent +command/fence/descriptors/buffers/query et portent chacun un handle +`slot+generation`. Toute fin, sortie invalide, annulation ou erreur nettoie le +handle exact ; un échec d'attente condamne la session. Une instrumentation +bornée prouve sans sommeil `SUBMIT(i) < SUBMIT(i+1) < FINISH(i) < +PUBLICATION_START(i) < PUBLICATION_FINISH(i)` à depth 2, ainsi que les chemins +begin/publish/cancel en échec et la réutilisation. Aucun helper GPU n'est créé. +Le payload mappé suit la capacité de séquence admise : un slot à depth 1, deux +seulement pendant depth 2, sans redimensionnement pending et avec libération du +second avant la prochaine admission. Une génération arrivée à `UINT64_MAX` ne +boucle pas; son slot est retiré définitivement avant toute nouvelle soumission. + +Le runner `pre-sfm-real-execution` compile en plus un contrôle de benchmark +synchrone privé. Il force le contrat GPU depth 1 et exécute +`top2 synchrone → postprocess canonique → publication` avant la paire suivante. +Le défaut du runner reste rolling et le défaut Matcher normal reste AUTO ; CPU +ou Vulkan explicites demeurent des overrides. Le contrôle synchrone est absent +du binaire et du comportement de production, n'est pas persisté et ne peut pas +être repris depuis une Task pendante. Il sert uniquement à comparer l'overlap à +sortie identique. Le corpus court retenu démontre +10,27 % pour rolling depth 1 +face à ce contrôle, avec digest identique. L'A/B forcé ABBA mesure ensuite +54,661652238 paires/s à depth 1 contre 55,797311953 à depth 2 (+2,077617 %), +sous le deadband 5 %, avec digest identique, quatre séquences de fallback local et zéro +panne/discard dans chaque exécution. Depth 2 est donc +**REJECTED_WITH_MEASURED_REASON** pour AUTO normal; il reste une capacité privée +de sûreté/benchmark; le lifecycle normal est **PASS / FROZEN**. + +Le target runner/test compile aussi un contrôle A/B `--matcher-inflight 1|2`, +absent de la production. Pour AUTO rolling, il fixe min=max inflight et batch=2 +par défaut. `--matcher-batch 2|4|8|12`, valable seulement avec inflight, fixe +aussi min=max batch dans la même enveloppe privée à capacité Vulkan unique; +l'unique +Governor conserve sélection, réservation UMA et contrat immutable. Un budget +GPU nul est refusé avant ouverture du Project, et une indisponibilité +GPU/backend/mémoire ne peut jamais sélectionner CPU à la place. Synchronous +accepte seulement 1. +Cette dimension opérationnelle n'est ni persistée ni scientifique, ne s'applique +pas à une Task pendante et est rapportée explicitement dans le JSON. Les +fixtures comparent rolling 1, rolling 2 et synchrone 1 sur des Projects neufs +avec sorties exactes. Une paire localement inéligible peut toujours produire sa +preuve CPU complète. Matcher l'attribue exactement une fois après publication +durable à un compteur d'items local-ineligible, backend-failure ou other; le +travail CPU sélectionné n'est pas un fallback. Les compteurs de séquences +et l'apprentissage de débit restent séparés : si la paire suivante échoue en +calcul/publication ou si la Task est annulée, le préfixe durable reste compté, +sans transformer la séquence avortée en observation valide. La cause locale +d'une paire n'est jamais écrasée par une panne begin/finish voisine. +Ils restent diagnostiques, mais seul le nombre d'items localement inéligibles doit +être identique entre cohortes de lots différents. Un item backend-failure/other, +un discard, un slot pending, un contrat différent ou toute admission CPU rend +`experiment_valid=false` et fait échouer la Task de benchmark après checkpoint +de l'éventuel fallback déjà durable. Les logs préliminaires batch 2/4 retenus +montrent quatre contre trois séquences locales pour les mêmes 4113 IDs/digest : +ils prouvent seulement que l'ancien comparateur dépendait du regroupement et ne +participent pas à la décision. La matrice item-valide +`forced-batch{2,4,8,12}-items{,-b}.stdout.jsonl` donne respectivement +54,180767704, 66,094373197, 74,784998723 et 76,755814095 paires/s, avec six +items locaux, zéro panne/autre et le même digest dans chaque run. Les gains de +palier sont +21,988624373 %, +13,148812987 % et +2,635308425 %. Le dernier est +sous le deadband 5 % : AUTO normal conserve `BATCH_MAX_USEFUL=8`, tandis que +batch 12 reste une capacité privée sûre +`REJECTED_WITH_MEASURED_REASON`. +Le contrôle de production sans override est conservé dans +`short-auto-batch8-governor-v2.stdout.jsonl` : la cadence durable complète fait +monter les contrats `1 → 2 → 4 → 8`, puis garde 8 jusqu'au résultat 4113/4113, +à 76,072 paires/s avec digest identique, inflight 1, helpers 0 et zéro panne ou +discard. +La preuve S21 finale sans override conserve exactement ces limites sur +172 741 paires : dernier contrat batch 8/inflight 1/helpers 0, 73,649 résultats +durables/s et zéro panne Vulkan. Un seul épisode YELLOW réduit batch 8 à 1; +après retour GREEN, la progression contrôlée 1 → 2 → 4 → 8 est rejouée. +Depth 2, helpers et batch 12 ne sont jamais admis en production. Le thread +Queue lourd reste sur `0-5,8-13`; `6,7,14,15` restent réservés au desktop. + +La création et la reprise AUTO ne sondent plus le backend sur le caller/main. +Elles exposent la capacité depuis les seules métadonnées build/backend/GPU ; le +Governor possède le dimensionnement exact et l'admission UMA sur son snapshot. Le premier +`begin` initialise Vulkan seulement sur le worker Queue déjà contraint, après +établissement pré-pthread de la politique de cache Mesa. Aucun sweep auxiliaire +post-init, latch de Task ou retry de recontrainte par TID n'existe. Une paire +localement inéligible n'initialise pas le backend. Une panne d'initialisation +désactive les admissions GPU AUTO +suivantes, tandis qu'une paire sous le seuil reste une inéligibilité locale et +non une panne. +L'état request-bound distingue un handle soumis de ces deux causes : aucune +paire sans handle ne passe par `finish`. Une panne réelle invalide la +disponibilité Governor avant tout fallback ou sortie précoce ; une inéligibilité +locale conserve cette disponibilité. Les restaurations CPU/Vulkan/historiques +fixes ne peuvent pas écraser l'état établi par une restauration AUTO. Après jointure, seul le callback propriétaire publie les stages, dans l'ordre croissant de `candidate_pair_id`, par le chemin atomique Match Store existant. diff --git a/docs/architecture/matcher.md b/docs/architecture/matcher.md index b2970c2..5a6bf01 100644 --- a/docs/architecture/matcher.md +++ b/docs/architecture/matcher.md @@ -1,5 +1,11 @@ # Matcher v1 et Match Store v1 +## Statut opérationnel + +**INTERNAL_PARALLELISM_COMPUTE_RESOURCES_V1 — PASS / FROZEN.** +**COMPUTE_GOVERNOR_V2 — PASS / FROZEN.** +**ORB_VULKAN_ASYNC_EXECUTION — PASS / FROZEN.** + ## Contrat Le pipeline v1 est `ORB → BFMatcher Hamming` ou `SIFT/RootSIFT → BFMatcher L2`, @@ -83,15 +89,16 @@ flottante ne garantit pas le top-2 OpenCV sur les égalités adversariales et le gain n'est présent que sur les grandes paires carrées. Ils restent intégralement sur BFMatcher CPU. -**IMPLEMENTATION IN PROGRESS — parallélisme interne P4.** `matcher.run` v1 +**INTERNAL_PARALLELISM_COMPUTE_RESOURCES_V1 — PASS / FROZEN.** `matcher.run` v1 orchestre le Matcher sans connaître son backend interne. La tâche persiste uniquement la configuration, l'identité des Feature Sets à sélectionner et un curseur Candidate Pair. Une paire est atomique et publiée -immédiatement. Les lots 1/2/4/8 sont séparés par checkpoint et +immédiatement. Les lots bornés sont séparés par checkpoint et `task_sequence_break()`. La tâche utilise l'unique Resource Governor avec une estimation couvrant ce working set ; aucune seconde logique de budget n'est introduite. Dans un lot, des fenêtres bornées calculent au plus deux paires par -thread CPU effectivement admis, avec un plafond de huit. Le callback Queue +thread CPU effectivement admis, avec un plafond opérationnel validé de douze. +Le callback Queue compte comme un participant, joint au plus `cpu_threads - 1` enfants, puis publie seul les stages en ordre `candidate_pair_id`. OpenCV reste à un thread interne pendant cette séquence, ce qui interdit un fan-out BFMatcher imbriqué. @@ -99,6 +106,15 @@ Le curseur ne suit que le préfixe contigu durable ; le premier échec scelle to le suffixe calculé. Sa ligne durable `matcher_tasks` appartient au schéma Project DB v11 ; le Match Result reste le contrat publié en v10. +Compute Governor v2 fait évoluer seulement le choix opérationnel et la +télémétrie de cette exécution gelée. Un contrat de séquence admis est immutable +jusqu'à libération ; seuls la séquence suivante et son lot peuvent être +adaptés. Admission CPU et admission de lot sont indépendantes : CPU12 avec un +lot de 1 est valide et ne permet pas d'emprunter une paire à la séquence +suivante. Sous un signal de pression qui autorise encore une admission, la +capacité CPU réductible abandonne l'essai et réserve CPU1/lot minimum dans cette +même décision ; un seuil Gate G qui produit `WAIT` reste sans réservation. + Project DB v12 ajoute un enfant `Geometric Verification Result` référencé par `match_result_id`. Le Matcher ne calcule, ne stocke et ne valide aucun inlier géométrique ; seul son Match File canonique définit l'ordre indexé par le masque. @@ -111,21 +127,237 @@ le tie-break du plus petit index. Elle ne matérialise jamais A×B. Lowe, canonicalisation et persistance restent communs. Sur Radeon 780M, la parité top-2 avec OpenCV est exacte et le gain warm est supérieur à 90 % à 4096/8192. Le backend de production conserve exactement cette frontière. Sa parité entière -permet au CPU et à Vulkan de partager l'identité persistante. La sélection est -une politique runtime explicite ; le CPU reste le fallback portable. Les APIs -historiques de création/soumission choisissent toujours le mode CPU parallèle. -Les variantes additives `*_with_mode` permettent de demander ORB Vulkan avant -l'admission ; elles refusent SIFT/RootSIFT, un profil sans GPU sélectionné ou -un backend absent. Ce mode immutable demande exactement CPU 1, GPU 1 et -640 Kio GPU/UMA. Un échec Vulkan pendant une exécution déjà admise reprend le -top-2 CPU exact sans changer fingerprint ni résultat. Le contrat détaillé est -décrit dans [vulkan_matcher.md](vulkan_matcher.md). +permet au CPU et à Vulkan de partager l'identité persistante. L'évidence +directe de la tranche v2 établit ORB Vulkan comme déterministe, exact et +mesurément supérieur pour ce hot path : il est le workload GPU primaire de la +politique canonique GPU-first. Le CPU reste le fallback portable et de panne. -La reprise reconnaît uniquement quatre estimations complètes : CPU8/GPU0 et -CPU1/GPU1/640 Kio courantes, plus CPU12/GPU0 et CPU12/GPU1/640 Kio -historiques. Les deux formes CPU12 sont normalisées éphémèrement vers leur -forme courante avant admission, sans checkpoint d'estimation seule. -Une forme voisine est rejetée ; aucun champ isolé ne sert à deviner le backend. +Les API normales créent AUTO. Pour ORB, elles exposent Vulkan depuis les faits +build, objet backend, GPU et enveloppe UMA sûre ; création et reprise ne sondent +ni n'initialisent le driver sur le caller/main. Le Governor essaie cette +capacité avant le CPU complet, puis le premier `begin` initialise Vulkan sur le +worker Queue déjà contraint. Avant ce worker, le démarrage a établi la politique +sûre `MESA_SHADER_CACHE_DISABLE=true`; elle supprime sur la 780M validée les +helpers de cache Mesa qui élargissaient leur masque. Une paire sous le seuil +n'initialise toujours pas Vulkan. Aucun sweep/latch Matcher ni appel +`sched_*affinity(tid)` auxiliaire n'existe ; le creator/main/TUI reste +inchangé. La frontière d'initialisation du backend vérifie elle-même, sans +`setenv`, que `MESA_SHADER_CACHE_DISABLE` vaut exactement `true` ou `1` avant +tout appel Mesa. Un consumer public tardif avec une valeur absente, fausse ou +malformée reçoit `UNAVAILABLE`, contexte mémorisé indisponible et aucune sortie +partielle ; la metadata non initialisante reste lisible. Les variantes `*_with_mode` +demandent encore explicitement CPU ou ORB Vulkan et conservent leurs erreurs +historiques. Le mode Vulkan demande CPU1, GPU1 et 640 Kio GPU/UMA, débités +exactement une fois de la RAM hôte sur la 780M. Un échec Vulkan pendant cette +séquence reprend la paire top-2 CPU complète sans changer fingerprint ni +résultat. Une panne d'initialisation retire Vulkan des admissions AUTO +suivantes ; une paire sous le seuil validé reste seulement inéligible. Le +rolling conserve explicitement l'état de soumission de chaque paire : une paire +sans handle exécute directement son stage CPU complet et n'appelle jamais +`finish(NULL)`. Toute panne backend est publiée au Governor dès son observation, +avant le fallback, l'annulation ou la publication qui peuvent encore échouer ; +l'inéligibilité locale ne modifie jamais cette santé partagée et n'est jamais +réétiquetée par l'échec d'une soumission ou d'une finition voisine. Le +contrat détaillé est décrit dans +[vulkan_matcher.md](vulkan_matcher.md). +Les coutures privées renvoient séparément le résultat Matcher et la faute +backend. Lecture/allocation locale avant submit, puis filtrage/allocation ou +staging Match File après un finish réussi, produisent un fallback CPU complet +classé `other`. Elles ne désactivent pas Vulkan et ne jettent pas un successeur +soumis sain. Seul l'échec de la transaction backend exacte produit +`backend-failure` et invalide les slots partagés. + +La production normale est `AUTO`, choisi par l'unique Governor avant chaque +séquence depuis une enveloppe privée CPU/Vulkan. Les modes explicites +CPU/Vulkan sont des overrides de debug, benchmark et reproductibilité. Le +contrat installé est immutable jusqu'au prochain `sequence_break`. Cette +couture et ses dimensions retenues sont **PASS / FROZEN**. + +Le CPU Matcher compte ses participants uniquement dans `cpu_threads=1..12`; +`helpers=0` tant qu'aucun helper GPU distinct n'est réellement admis. Le +diagnostic borné conserve le backend de capacité sélectionné et le backend réel +de la séquence (`CPU`, `ORB_VULKAN` ou plusieurs paires complètes des deux), +avec la raison d'inéligibilité ou de fallback. Ce retour opérationnel ne change +jamais l'identité ou le contenu scientifique. + +La couture backend privée expose un snapshot cumulatif thread-safe : +soumissions, complétions, temps CPU de submit/dispatch, attente de fence, +readback, temps GPU issu des timestamps lorsqu'ils existent, intervalle entre +une complétion observée et la soumission suivante, pannes, discards et nombre +de slots actifs. Les compteurs saturent à `UINT64_MAX` au lieu de boucler. Matcher en +dérive par séquence les soumissions/complétions, temps GPU/fence/starvation, +travail CPU de fallback ou de staging et durée de publication durable. Ces +chronométrages n'ajoutent ni worker, ni attente artificielle, ni changement +d'ordre. Une séquence réellement fallback ne forme pas un échantillon de débit +Vulkan pur. + +Une nouvelle Task ORB normale persiste une signature `MIXED` avec les champs +CPU12/GPU0 réels ; elle exprime honnêtement que la politique AUTO peut choisir +CPU ou Vulkan par séquence, sans réserver les deux et sans coder un backend +factice. L'override CPU conserve la classe `CPU`. La forme Vulkan explicite est +CPU1/GPU1/640 Kio. Toutes sont à lot `1..12` et 10 Mio par paire. Les signatures historiques +CPU8/GPU0 et CPU1/GPU1 à lot maximal 8, ainsi que les formes CPU12 +pré-estimation-par-paire, restent reconnues uniquement pour la reprise et sont +normalisées éphémèrement sans checkpoint d'estimation seule. Une forme voisine +est rejetée ; aucun champ isolé ne sert à deviner le backend. Seule la nouvelle +classe `MIXED` reconstruit AUTO. Toute forme CPU ORB historique ou courante +reconstruit CPU fixe par compatibilité/sûreté des overrides ; Vulkan reste fixe. +Le build portable conserve la signature AUTO mais son enveloppe n'expose que CPU. +Seule une reconstruction `MIXED`/AUTO établit la disponibilité runtime partagée ; +la co-restauration ultérieure de formes CPU, Vulkan ou historiques fixes ne la +modifie pas. L'ordre de restauration ne peut donc pas déclasser une AUTO avant +son admission. + +L'évolution Compute Governor v2 n'ajoute aucune dimension de ressources au +fingerprint. `begin/finish/discard` sont privés à +`src/orb_vulkan_backend_internal.h` et absents du header/ABI public. `begin` +transfère au backend les comptes exacts de la requête et un handle +`slot+generation`; `finish` utilise ces comptes, borne `FEATURE_MAX` et ne +consomme que ce slot sur tout résultat, même une capacité de sortie invalide. +`discard` attend sa fence ; une attente en échec +condamne/détruit la session avant une soumission suivante. Le wrapper public +synchrone top-2 reste stable et s'implémente par cette couture. Les objets +pending, fds et temporaires utilisent un nettoyage structuré, les chemins +tronqués sont refusés et aucune exception C++ ne franchit C. + +Le propriétaire Queue conserve seul Lowe, canonicalisation, publication et +avancement du curseur. Device/pipeline/layout/cache sont partagés; deux slots +maximum dupliquent seulement command/fence/descriptors/buffers/query. La trace +déterministe prouve à depth 2, pour deux paires successives 769×769, +`GPU_SUBMIT(i) < GPU_SUBMIT(i+1) < GPU_FINISH(i) < PUBLICATION_START(i) < +PUBLICATION_FINISH(i)`, avec cardinalité/sortie intactes. + +Le payload mappé suit le contrat de séquence plutôt que le maximum de +l'enveloppe. Un backend frais retient zéro, depth 1 retient exactement 640 Kio +et depth 2 retient 1,25 Mio jusqu'au nettoyage de la séquence; le second slot est +libéré avant la prochaine admission depth 1. Le redimensionnement est interdit +avec une requête pending et une croissance échouée conserve la capacité +antérieure. `backend_info` rapporte la rétention réelle. Une génération arrivée +à `UINT64_MAX` retire définitivement son slot avant tout nouveau submit; elle ne +boucle jamais vers un ancien handle. +Elle couvre aussi échec du begin successeur, faute locale avant/après backend, +échec de publication, annulation et réutilisation du slot. Le lifecycle est +**ORB_VULKAN_ASYNC_EXECUTION — PASS / FROZEN** et helpers reste 0. + +Le runner réel opt-in utilise désormais `AUTO` par défaut ; `cpu` et `vulkan` +restent des overrides explicites de debug/benchmark/reproductibilité. Son +pipeline normal est le rolling Governor-owned inflight 1. Un +contrôle `synchronous` force depth 1 et est +compilé uniquement dans ce runner et le test Matcher dédié : pour une capacité +GPU sélectionnée, chaque paire passe par le wrapper public top-2 synchrone, +puis par la même canonicalisation et la même publication avant la paire +suivante. Ce contrôle historique de mesure n'existe pas dans le binaire +`lardon3d`, ne répond à aucun environnement en production et ne devient ni +identité, ni checkpoint, ni capacité Governor. Il est refusé pour CPU explicite +et pour la reprise d'une Task Matcher pendante, car son choix n'est pas +persisté. Les fixtures Vulkan comparent rolling depth 1 forcé, rolling depth 2 +forcé et synchrone depth 1 sur des Projects neufs et obtiennent les mêmes champs +scientifiques, SHA et tailles d'assets. + +Pour isoler la profondeur du reste de la boucle adaptative, ce même runner +accepte en contexte `--resume-pre-gv-existing` neuf le contrôle privé +`--matcher-inflight 1|2`. Il est limité à `AUTO`; rolling force une capacité +Vulkan unique et fixe min=max à la valeur demandée, avec un lot fixe de deux +par défaut. Le contrôle additionnel `--matcher-batch 2|4|8|12` exige inflight, +AUTO et rolling, puis fixe aussi min=max batch pour une matrice reproductible. +Le Governor choisit et réserve toujours cette capacité depuis son snapshot +courant, y compris 640 Kio par slot et la charge UMA exacte; le contrôle ne +court-circuite aucune admission et n'expose aucune alternative CPU. GPU budget +zéro est refusé avant accès au Project; une capacité GPU/backend/mémoire non +admissible invalide l'expérience au lieu d'exécuter AUTO CPU. Synchronous reste +depth 1 et refuse la valeur 2. La valeur est absente du binaire `lardon3d`, de l'ABI, du payload, du +checkpoint et de l'identité scientifique; le runner la restaure dans +l'environnement sur toute sortie, la refuse sur une Task pendante et l'émet +comme `1`, `2` ou `null` dans le JSON d'évidence; batch est pareillement émis +comme `2`, `4`, `8`, `12` ou `null`. L'agrégat exige uniquement des contrats +sélectionnés Vulkan batch/depth demandés, réservation hôte et payload exacts, +zéro panne, +discard ou slot pending. Il accepte seulement les fallbacks CPU complets dus à +l'inéligibilité locale. Matcher compte chaque item une seule fois, après sa +publication CPU durable, dans les classes local-ineligible/backend-failure/other; +une séquence peut en contenir plusieurs et le CPU sélectionné normalement +n'entre dans aucune classe. Le commit par item est séparé du feedback de débit +de fin de séquence : une annulation ou panne ultérieure conserve le préfixe +durable, mais n'entraîne jamais la séquence avortée. La déduplication bornée est +éphémère à la Task et n'ajoute aucun état de reprise. L'agrégat conserve aussi +les comptes de séquences +pour diagnostic, mais l'égalité inter-cohortes porte exclusivement sur les +items localement inéligibles; tout item backend-failure/other donne +`experiment_valid=false`. Une panne backend tardive peut conserver la preuve +CPU déjà publiée, mais la Task de benchmark échoue après son checkpoint et la +cohorte ne peut pas être déclarée réussie. La mesure forcée ABBA donne +54,661652238 paires/s à depth 1 et 55,797311953 à depth 2 (+2,077617 %), sous +le deadband 5 %. Pour 4113 paires durables par run, le débit de cohorte est +`(2 * 4113 * 1e9) / somme(wall_ns)`, pas la moyenne des débits par run. Les +walls bruts 75326831673/75162582080 et 73662096698/73764360098 ns donnent les +moyennes 75,244706877/73,713228398 s. Fence vaut 6,0684/3,6776 s, starvation +54,4534/50,1465 s, publication 29,2582/30,0548 s, submit CPU +0,2655/0,3818 s, readback 0,0460/0,0873 s et GPU busy max 23/24 %. +Chaque bras A/B garde le digest +`7a9dbc38a23a600379167d55e24836b7acbb22eea25573e7440bdc9e4602b3b3`, quatre +séquences de fallback local par exécution et zéro panne/discard. Depth 2 reste donc validé pour la +sûreté privée (`DEPTH_MAX_VALIDATED_SAFETY=2`) mais est +**REJECTED_WITH_MEASURED_REASON** pour la politique normale +(`DEPTH_MAX_USEFUL=1`). Le corpus établit par ailleurs +10,27 % pour rolling +depth 1 face au contrôle synchrone, avec le même digest. La matrice de batch +runner-only item-valide donne, pour batch 2/4/8/12, les walls A/B bruts +76150272845/75674818393, 61319835797/63138565155, +55148130595/54847191200 et 53446248173/53724786321 ns. La même formule de débit +combiné donne 54,180767704, 66,094373197, 74,784998723 et 76,755814095 paires/s, +soit +21,988624373 %, +13,148812987 % puis +2,635308425 %. Chaque run conserve +4113 paires durables, six items locaux, zéro item backend-failure/other et le +même digest ci-dessus. Batch 12 est donc +**REJECTED_WITH_MEASURED_REASON** sous le deadband 5 % : la sûreté privée reste +`BATCH_MAX_VALIDATED_SAFETY=12`, mais AUTO normal suit +`BATCH_MAX_USEFUL=8`. Les fichiers sont +`forced-batch{2,4,8,12}-items{,-b}.stdout.jsonl`. Les anciens +`forced-batch2-current.stdout.jsonl` et `forced-batch4.stdout.jsonl` restent +historiques : leur comparateur par séquences était invalide et ne fonde pas la +décision. +Le run normal sans option backend/lot/inflight +`short-auto-batch8-governor-v2.stdout.jsonl` suit effectivement +`1 → 2 → 4 → 8`, conserve batch 8 et publie 4113/4113 résultats en +54,066973393 s (76,072 paires/s). Il garde le même digest `L3DMRD1`, six items +locaux, zéro panne/discard, inflight 1 et helpers 0. Ce run est la preuve du +contrôle AUTO; les cohortes forcées ci-dessus restent la preuve de sélection de +l'enveloppe utile. +Le run S21 final normal +`final-s21-auto.stdout.jsonl` publie 172 741 Match Results pour 172 741 +Candidate Pairs en 2 345,444485079 s, avec zéro mapping dupliqué, curseur +contigu/complet et digest `L3DMRD1` +`e5128a2e599ff593c4f79850e067254b1f249d19e8480a44973306b1af250f70`. +AUTO reste Vulkan/rolling; 172 507 soumissions ont 172 507 complétions, zéro +panne/discard/pending, tandis que 234 items localement inéligibles sont +recalculés comme paires CPU complètes. L'agrégat compte 21 550 séquences +Vulkan pures, 77 mixtes et 3 CPU, sans jamais publier d'évidence partielle. +La Task 2831 finit `COMPLETE`, progression 100 et `sequence_count=21629`; +le checkpoint final SHA-256 vaut +`636f4f4a20f27308d90142c495c9f6ffc04b4c0dfcca0fdc75cfeb5366ab50b1`. +Les quatre sources retenues sous +`/home/fy59/Documents/Lardon/.real-pre-sfm-2026-08-30/governor-v2-evidence/` +sont `forced-depth1-a.stdout.jsonl`, `forced-depth1-b.stdout.jsonl`, +`forced-depth2-a.stdout.jsonl` et `forced-depth2-b.stdout.jsonl`. + +AUTO Vulkan normal adapte le lot seulement dans `1..8`. Il exige huit +observations pures consécutives pour la référence et huit au palier d'essai, +puis accepte seulement un gain moyen d'au moins 5 %. Pression, fallback ou +travail durable nul abandonnent la fenêtre; un ou deux échantillons ne décident +rien. La durée entraînant AUTO couvre la réadmission réussie, le calcul, les +publications ordonnées et le checkpoint générique durable; la seule durée du +noyau Matcher reste diagnostique et ne décide pas le lot. Une séquence dont le +checkpoint échoue ne peut donc pas entraîner le contrat suivant. CPU Matcher +garde sa rampe et son maximum 12. Helpers reste 0 : la +publication owner-only vaut environ 29,5 s dans ces cohortes, et ni le gain +depth 2 sous 5 % ni une autre mesure ne prouve qu'un helper supplémentaire +surmonterait cette frontière de durabilité ordonnée. + +Le même runner audite la bijection ordonnée Candidate Pair/Match Result, le +curseur final contigu et chaque Match File par SHA, taille, header et entrées. +Son digest `L3DMRD1` sérialise explicitement en largeurs fixes les IDs +scientifiques, kind/version/fingerprint, status/cardinalité et SHA/taille +d'asset ; Task IDs, timestamps, chemins et sélection opérationnelle sont +exclus. Ce digest est une preuve de comparaison du harness, pas une nouvelle +identité Project DB ou scientifique. ## Déterminisme et fingerprint diff --git a/docs/architecture/precision_feature_pipeline.md b/docs/architecture/precision_feature_pipeline.md index 53048b3..1c3ac69 100644 --- a/docs/architecture/precision_feature_pipeline.md +++ b/docs/architecture/precision_feature_pipeline.md @@ -56,11 +56,14 @@ validé. Après crash, la registry statique reconstruit le même `task_id`; l'im est recommencée, sans micro-checkpoint trompeur. Aucun Feature Set partiel ne devient READY. Le SHA-256 de l'Image Asset géré est vérifié avant décodage. -L'estimation SIFT demande jusqu'à douze threads CPU ; le Governor réduit ce -plafond à la limite interne OpenCV configurée au démarrage. Elle réserve aussi +L'estimation SIFT demande jusqu'à douze threads CPU. Le réglage OpenCV du +démarrage est une baseline/plafond sûre ; pour chaque séquence, l'unique +callback Queue applique temporairement le compte immuable admis dans `1..12`, +le vérifie puis restaure la baseline sur toute sortie. Une mutation process-wide +concurrente par plusieurs workers n'est pas supportée. La tâche réserve aussi un slot IO, aucun GPU et environ 1,06 Gio structurels : image décodée, pyramides OpenCV, candidats et descriptors. Le pic de lot est -zéro. Le Resource Governor admet donc le fan-out OpenCV complet ; la Queue +zéro. Le Resource Governor admet donc le fan-out OpenCV exact ; la Queue conserve un seul callback actif et ne superpose pas un second pool SIFT. Le nombre de threads reste opérationnel et absent du fingerprint SIFT/RootSIFT. diff --git a/docs/architecture/resource_aware_pipeline.md b/docs/architecture/resource_aware_pipeline.md index 2773e49..899aee2 100644 --- a/docs/architecture/resource_aware_pipeline.md +++ b/docs/architecture/resource_aware_pipeline.md @@ -38,9 +38,13 @@ chemins de production existants et leur validation finale est terminée. Les snapshots emploient `CLOCK_MONOTONIC` et leur âge maximal est 1000 ms. Une capture complète impossible est une erreur opérationnelle, tandis qu'une PSI -ou télémétrie swap optionnelle absente reste inconnue. Le modèle cible un hôte -Linux natif non contraint ; cgroups, limites systemd/RLIMIT, multi-GPU, -monitoring RSS, scratch et stockage externe restent différés. +ou télémétrie swap optionnelle absente reste inconnue. Compute Governor v2 +observe maintenant le RSS/HWM courant dans un buffer borné, uniquement comme +diagnostic du processus : il ne le confond ni avec la réservation Task ni avec +un coût attribuable. Le modèle cible un hôte Linux natif non contraint ; cgroups, +limites systemd/RLIMIT, multi-GPU, historique/monitoring RSS long terme, +redimensionnement d'admission depuis le RSS, scratch et stockage externe restent +différés. ## Feature Extraction @@ -49,9 +53,14 @@ publication Feature Store, métadonnées DB, checkpoint terminal et libération buffer. Le batch vaut donc une image et la granularité de reprise est une image. Le worker unique et la file bornée fournissent la backpressure actuelle. -OpenCV est configuré une seule fois avant le démarrage des workers. La tâche -réserve le nombre réel de threads OpenCV au lieu d'annoncer artificiellement un -thread pendant qu'une primitive interne en utilise davantage. +Le démarrage configure une baseline et un plafond OpenCV sûrs avant la création +de Queue. L'unique callback lourd applique ensuite temporairement le compte CPU +immuable admis pour sa séquence, dans `1..12`, et restaure la baseline sur toute +sortie, y compris après une mutation suivie d'un échec de vérification. La tâche +réserve donc le nombre réellement appliqué au lieu d'annoncer artificiellement +un thread pendant qu'une primitive interne en utilise davantage. Une mutation +process-wide concurrente par plusieurs workers n'est pas supportée ; Queue +conserve un seul callback actif. ## Matcher @@ -82,12 +91,17 @@ slow-start restent exclusivement décidés par Runtime et Governor. ## GPU et files -La Radeon 780M est UMA : toute mémoire GPU compte aussi comme pression RAM. Un -backend GPU emploie un unique job actif, des dispatchs courts, puis publie avant -de continuer. Vulkan 1.4.354 énumère la 780M RADV et une file compute dédiée. Le -backend ORB top-2 de production possède un contexte lazy réutilisable, 640 Kio -de buffers bornés et un fallback CPU exact. Le CPU reste le fallback portable -si Vulkan est absent, incompatible ou désactivé pour la session. +La Radeon 780M est UMA : toute mémoire GPU compte aussi comme pression RAM. +Vulkan 1.4.354 énumère la 780M RADV et une file compute dédiée. Le backend ORB +top-2 de production possède un contexte lazy réutilisable et jusqu'à deux jobs +privés en vol sur cette file, sans helper hôte. Chaque slot mappé vaut 640 Kio. +Le backend part de zéro et retient exactement un slot après une initialisation +ou séquence AUTO normale depth 1. Il n'alloue le second que sous un contrat +privé de sûreté/benchmark depth 2 déjà admis, puis le libère avant de franchir +la prochaine admission depth 1. Les +publications restent strictement ordonnées et le fallback CPU reste exact. Le +CPU reste le fallback portable si Vulkan est absent, incompatible ou désactivé +pour la session. La feasibility SIFT/RootSIFT a borné son prototype Vulkan à 8,125 Mio de payload lazy, mais n'a pas franchi la Gate de production. Le Governor ne réserve diff --git a/docs/architecture/resource_governor.md b/docs/architecture/resource_governor.md index 4013295..0ca62e9 100644 --- a/docs/architecture/resource_governor.md +++ b/docs/architecture/resource_governor.md @@ -1,13 +1,303 @@ # Resource Governor Lardon3D +## COMPUTE_GOVERNOR_V2 — PASS / FROZEN + +**COMPUTE_GOVERNOR_V2 — PASS / FROZEN.** +**ORB_VULKAN_ASYNC_EXECUTION — PASS / FROZEN.** Gate G core reste +**PASS / FROZEN**. Cette évolution opérationnelle ne change ni le contrat +scientifique Matcher gelé, ni les octets Match File, ni l'ordre du curseur, ni +la durabilité. Sa cible est une boucle fermée à `sequence_break()` : le +Governor observe une télémétrie hôte/Task bornée, choisit un contrat pour la +séquence suivante, puis le maintient immutable pendant toute son exécution. +Cette couture privée est gelée ; seule une séquence ultérieure peut recevoir +un autre contrat. Les dimensions retenues, le corpus S21 complet, les suites +portable/Vulkan, les sanitizers et l'audit XHIGH ferment le statut v2. + +Sur le profil validé Ryzen 7 8845HS, le Governor lit le masque d'affinité permis +et la topologie package/core Linux. Il réserve des groupes de coeurs physiques +complets, frères SMT inclus, en préférant déterministiquement les plus grands +IDs package/core. Avec le masque unrestricted 0–15 et une réserve logique de +quatre, il dérive le pool lourd `0-5,8-13` et réserve `6,7,14,15` au desktop +Arch/Sway, à l'audio et à l'interaction ordinaire. Si le caller est déjà limité +à au plus `logical_total - reserve`, son masque permis devient directement le +pool de calcul sans seconde soustraction. Sans affinité/topologie exploitable, +le budget portable `logical_total - reserve` subsiste sans exclusion arbitraire +de frères SMT et l'affinité est diagnostiquée inactive. Cette politique privée +est **PASS / FROZEN** sur le profil validé ; aucun ID CPU n'est persisté ni ne +devient une identité scientifique. + +Seul le worker lourd de l'unique Queue applique et relit son propre masque +(`pid=0`) avant les callbacks. Le caller/main/TUI reste unrestricted et aucun +autre processus n'est touché. Un pidfd de thread ne stabilise pas l'identifiant +numérique consommé par `sched_setaffinity(tid)` après la sortie du thread : le +Governor n'énumère ni ne mute donc jamais un TID auxiliaire. À la place, +Lardon3D établit `MESA_SHADER_CACHE_DISABLE=true` avant toute création de pthread +applicatif et avant toute initialisation Vulkan. L'absence de variable prend ce +défaut sûr ; les valeurs explicites exactes `true` et `1` sont conservées. Une +valeur explicite fausse ou malformée n'est pas écrasée et fait échouer le +démarrage, car elle permettrait à Mesa de créer ses helpers de cache disque +observés capables d'élargir leur affinité. La variable est inoffensive pour les +drivers non-Mesa, opérationnelle seulement, non persistée et absente de toute +identité scientifique. Sur le profil 780M contrôlé, les helpers `*:disk$0` +disparaissent ; tous les threads runtime restants observés héritent et gardent +`0-5,8-13`. Les diagnostics privés exposent l'activité de cette politique, la +valeur sûre et sa raison, sans prétendre recontraindre des auxiliaires. +Cette garantie est aussi défensive dans le backend public : toute requête +non vide qui devrait initialiser Vulkan vérifie sans mutation la valeur exacte +`true`/`1` avant le premier appel Mesa. Une valeur absente ou différente rend ce +contexte backend indisponible et retourne `UNAVAILABLE` sans sortie partielle. +La lecture metadata demeure non initialisante. Ainsi un consumer direct qui ne +passe ni par l'application ni par le runner ne contourne pas la réserve CPU. +Le nombre de CPUs du pool borne directement l'admission. +CPU Matcher reste validé dans `1..12`, indépendamment du lot : +`cpu_threads=12, batch_size=1` demeure un contrat valide. + +La cible Compute Governor v2 conserve 3 GiB de `MemAvailable` et ne franchit +pas intentionnellement le plancher dur de 2 GiB sur cette classe d'hôte 16 GiB. +Les entrées de pression actives sont `MemAvailable`, les PSI CPU/mémoire/I/O et +les deltas swap-in/swap-out entre observations. L'occupation totale du swap est +un état historique, pas à elle seule une activité récente. Ces objectifs v2 ne +réécrivent pas rétroactivement les constantes Gate G core gelées ci-dessous. +Les observations saines autorisent une croissance lente. Pour une dimension +CPU réellement réductible, la rampe est exactement `1 → 2 → 4 → 8 → 12`, +bornée par l'enveloppe et le compute-pool. Deux séquences établissent d'abord +une référence de débit durable puis ouvrent un essai borné au palier supérieur. +Les dimensions CPU/génériques conservent deux observations d'essai et un gain +d'au moins 5 % avant une nouvelle croissance. Le lot ORB Vulkan normal exige +désormais huit observations pures consécutives pour sa référence puis huit pour +chaque essai; sa décision compare les moyennes bornées au même deadband de 5 %. +Pour Matcher, cette observation est la cadence durable complète de la séquence : +à partir de la deuxième séquence elle commence juste avant la rupture/réadmission +et se termine seulement après calcul, publication owner-only et checkpoint +générique durable. Le temps interne du calcul reste une métrique séparée, mais +ne peut pas masquer le coût de réadmission que le choix du lot doit amortir. +CPU et lot partagent un unique essai : deux dimensions ne changent jamais dans +la même mesure. L'infrastructure privée sait aussi +borner un essai inflight, mais l'enveloppe ORB AUTO normale le fixe maintenant +à 1 après la mesure A/B décrite ci-dessous. Un échantillon rapide isolé ne +décide rien ; sans gain, le plafond revient au dernier palier accepté et s'y +arrête. PSI ou delta swap actif abandonne immédiatement l'essai ; toute +admission encore permise prend lot minimum, inflight minimum et CPU1 +avant réservation. Un `WAIT`/`REJECT` Gate G reste inchangé. Après +l'hystérésis `RED → YELLOW → GREEN`, une nouvelle référence +permet de reprendre des essais contrôlés. Les diagnostics distinguent les +essais/gains/refus CPU, inflight et lot, `backend-fallback-hold` et +`pressure-decrease`. En production normale, ORB est inflight 1 et helpers 0. + +La télémétrie hôte privée lit, avec capacités fixes et parse strict, les deltas +`/proc/stat` limités au masque du compute-pool, `MemAvailable`, PSI mémoire et +I/O `some`/`full`, les deltas actifs `pswpin`/`pswpout`, RSS/HWM du processus et +le `gpu_busy_percent` de l'unique index DRM retenu par Hardware Profile, sans +scan ni fallback vers une autre carte. Utilisation CPU et GPU +sont exprimées en basis points avec un bit `known`; régression, overflow, +troncature, token malformé ou signal absent donnent `unknown`. La charge globale +n'est pas rebaptisée utilisation du pool et un Task qui utilise son pool admis +n'est pas réduit pour ce seul fait. RSS/HWM reste une observation du processus, +jamais une réservation mémoire Task. Aucun échec de cette capture optionnelle +ne fait échouer l'exécution scientifique. + +Pour ORB Vulkan normal, seul le lot `1..8` reste essayable ; inflight est fixé à +1 et helpers à 0. Huit séquences pures saines construisent la référence, puis +huit séquences au palier d'essai sont nécessaires avant acceptation ou refus. +Fallback, travail durable nul et pression sont exclus et réinitialisent la +fenêtre pertinente. Un backend affamé ou un GPU peu occupé sous hôte sain peut +ouvrir cet essai ; `gpu_busy` inconnu ne l'interdit pas lorsque la starvation +backend est observable. Seul le gain moyen de débit durable accepte le palier. +Une occupation GPU plus haute sans gain le fait revenir au dernier contrat +accepté et arrête cette croissance. Un ou deux échantillons, hauts ou bas, ne +décident donc jamais le lot GPU. + +La mesure forcée ABBA du même corpus et du même binaire donne 54,661652238 +paires/s à depth 1 et 55,797311953 paires/s à depth 2, soit +2,077617 %, sous +le deadband matériel de 5 %. Ces débits combinés valent +`(2 * 4113 * 1e9) / (wall_ns_a + wall_ns_b)`, et non la moyenne des deux débits +par run. Les `wall_ns` bruts sont 75326831673/75162582080 à depth 1 et +73662096698/73764360098 à depth 2; les walls moyens sont donc +75,244706877/73,713228398 s. Les autres moyennes depth 1/depth 2 sont : fence +6,0684/3,6776 s, starvation 54,4534/50,1465 s, +publication 29,2582/30,0548 s, submit CPU 0,2655/0,3818 s, readback +0,0460/0,0873 s et GPU busy max 23/24 %. Les quatre exécutions ont le digest +`7a9dbc38a23a600379167d55e24836b7acbb22eea25573e7440bdc9e4602b3b3`, quatre +séquences de fallback local par exécution et zéro panne/discard. Conclusion opérationnelle : +`DEPTH_MAX_VALIDATED_SAFETY=2`, mais `DEPTH_MAX_USEFUL=1`; depth 2 est +**REJECTED_WITH_MEASURED_REASON** pour AUTO normal. +Les agrégats retenus sont +`/home/fy59/Documents/Lardon/.real-pre-sfm-2026-08-30/governor-v2-evidence/` +`forced-depth1-a.stdout.jsonl`, `forced-depth1-b.stdout.jsonl`, +`forced-depth2-a.stdout.jsonl` et `forced-depth2-b.stdout.jsonl`. + +La Radeon 780M possède un slot GPU et utilise une mémoire UMA : buffers Vulkan, +staging, descripteurs/commandes et readback en vol sont débités exactement une +fois de la RAM hôte. Hardware Profile ne conclut plus « VRAM séparée » au seul +motif qu'amdgpu publie `mem_info_vram_total`. Il conserve la capacité de payload +rapportée, mais classe conservativement shared/UMA lorsqu'un petit aperture +VRAM volé/dédié est accompagné d'un GTT à l'échelle de la RAM système, ou +lorsque cette petite capacité reste incertaine. Sur l'hôte validé, les preuves +exactes sont 512 Mio de VRAM visible et 7 986 020 352 octets de GTT pour environ +16 Gio de RAM. Une classification UMA conservatrice d'un GPU à faible VRAM peut +refuser inutilement une admission ; classer à tort cet iGPU comme mémoire libre +séparée pourrait contourner les cibles 3 Gio/2 Gio et est interdit. La capacité +rapportée ne forme donc aucun second budget VRAM indépendant sur UMA. + +La politique canonique v2 est GPU-first lorsqu'un backend est validé, +déterministe et mesurément plus rapide, sous réserve que son contrat GPU/UMA +soit admis. ORB Matcher est aujourd'hui le workload primaire qui satisfait ces +conditions : la frontière top-2 et la sortie complète sont exactes, et +l'évidence contrôlée le classe supérieure au CPU pour ce hot path. CPU demeure +le fallback portable et de panne. Les choix CPU/Vulkan explicites restent des +overrides de debug, benchmark et reproductibilité. La production normale cible +`AUTO`, choisi par le Governor et observable par séquence. Le code courant +implémente ce comportement pour le Matcher ORB normal : GPU validé d'abord, +puis capacité CPU complète si le build, le backend, le GPU ou l'admission UMA +ne sont pas sûrs. La création/reprise AUTO n'initialise ni ne sonde Vulkan sur +le caller : le premier `begin` initialise le driver sur le worker Queue déjà +contraint. Un échec produit une paire CPU complète sans preuve partielle et +désactive l'admission GPU ultérieure lorsqu'il s'agit réellement du backend ; +une paire sous le seuil Vulkan reste une simple inéligibilité, pas une panne. +Le Matcher représente séparément `SUBMITTED`, inéligibilité locale et panne : +il n'appelle `finish` qu'avec le handle soumis correspondant. Une panne réelle +ne reclasse que les requêtes effectivement soumises ou non encore tentées; la +cause locale déjà établie pour une paire voisine reste locale. Elle met la +disponibilité partagée à faux immédiatement, même si le fallback CPU, +l'annulation ou la publication échoue ensuite. Seule une reprise AUTO peut +réétablir l'éligibilité depuis les faits runtime ; les reprises fixes et +historiques ne l'écrasent pas. +Une faute locale avant soumission (lecture Feature, allocation, chemin) ou +après `finish` Vulkan réussi (filtrage, allocation, staging/fsync Match File) +appartient au bucket `other`, jamais à `backend-failure`. La paire est +recalculée entièrement sur CPU; le backend partagé et les successeurs déjà +soumis restent valides. Le booléen privé `backend_fault` n'est vrai que si la +transaction Vulkan `begin` ou `finish` elle-même échoue. +Les API CPU/Vulkan explicites restent fixes. La télémétrie bornée conserve un +dernier diagnostic sérialisé et un état de rampe par kind/backend ; elle +distingue backend sélectionné/réel, contrat complet, pression, mesures hôte, +débit durable et compteurs Matcher/Vulkan. Une couture privée permet un pull +`since(serial)` et un format texte borné ; rien n'est imprimé directement dans +ncurses, persisté ou accumulé en grande histoire. Les callbacks atomiques +Feature/SIFT/RootSIFT enregistrent un item seulement après extraction et +publication durable propre. READY/`ALREADY_PRESENT` réutilisé ou publication non +durable enregistre zéro et n'avance aucun essai. Visual Index applique la même règle par segment ; +Candidate enregistre chaque séquence durable. Un fallback CPU complet d'une +séquence sélectionnée Vulkan annule l'essai et reconstruit ultérieurement une +référence pure ; il n'entraîne jamais la baseline Vulkan. + +Le runner opt-in `pre-sfm-real-execution` consomme maintenant cette couture +d'évidence. Ses échantillons JSON sont explicitement qualifiés +`latest-change-coalescing` : le polling peut fusionner des séquences rapides et +ne prétend donc pas les énumérer toutes. En parallèle, le Governor maintient par +kind/backend des compteurs cumulatifs saturants et des extrema de télémétrie de +taille fixe. Le résumé final rapporte ainsi les admissions et séquences +enregistrées, items durables, changements de contrat et sommes +CPU/Vulkan/publication sans histoire non bornée ni double comptage. Ces agrégats +vivent seulement avec l'instance Governor ; ils ne sont ni un budget RSS, ni un +nouveau payload, ni une persistance. +Ils conservent les classifications par séquence et ajoutent les comptes exacts +par item pour les fallbacks Matcher : inéligibilité locale, panne backend ou +raison autre/inconnue. La Task incrémente l'unique compteur de l'item seulement +après publication durable de son fallback CPU complet; une admission CPU +normale n'est pas un fallback. Cet incrément immédiat est séparé du feedback de +fin de séquence : un préfixe déjà durable reste compté si une paire suivante +échoue ou est annulée, sans créer une observation de débit pour la séquence +avortée. Un high-water mark Task borné évite le double comptage in-process; il +n'est ni persisté ni reconstruit au redémarrage. Ces compteurs fixes saturent +avec le drapeau +commun de l'agrégat et, contrairement au nombre de séquences, restent invariants +quand le lot change. Le contrôle matriciel forcé du runner n'expose qu'une +capacité Vulkan aux batch/depth demandés : zéro admission CPU, payload exact, +aucun changement de contrat, panne, discard ou slot pending sont nécessaires à +`experiment_valid=true`. Les items localement inéligibles ne rendent une +comparaison valide que si leurs comptes sont égaux entre cohortes; tout item de +panne ou autre cause échoue fermement. + +La matrice forcée item-valide retenue est +`forced-batch{2,4,8,12}-items{,-b}.stdout.jsonl` sous le répertoire d'évidence +ci-dessus. Chaque run publie 4113 paires, six items localement inéligibles, zéro +item backend-failure/other et le digest +`7a9dbc38a23a600379167d55e24836b7acbb22eea25573e7440bdc9e4602b3b3`. Selon +`(2 * 4113 * 1e9) / (wall_ns_a + wall_ns_b)`, les walls bruts +76150272845/75674818393, 61319835797/63138565155, +55148130595/54847191200 et 53446248173/53724786321 ns donnent respectivement +54,180767704, 66,094373197, 74,784998723 et 76,755814095 paires/s. Les gains de +palier sont +21,988624373 %, +13,148812987 % et +2,635308425 %. Le dernier est +sous le deadband 5 % : `BATCH_MAX_VALIDATED_SAFETY=12` reste disponible au +benchmark privé, `BATCH_MAX_USEFUL=8` borne AUTO normal et batch 12 est +**REJECTED_WITH_MEASURED_REASON**. Les anciens +`forced-batch2-current.stdout.jsonl` et `forced-batch4.stdout.jsonl`, comparés +par nombre de séquences au lieu d'items, restent une preuve historique du +comparateur obsolète et ne participent pas à cette décision. +Le run de production sans override +`short-auto-batch8-governor-v2.stdout.jsonl` confirme ensuite la boucle réelle : +contrats `1 → 2 → 4 → 8`, dernier lot 8, inflight 1, helpers 0, 4113 résultats +durables en 54,066973393 s soit 76,072 paires/s, six fallbacks locaux, zéro +panne/discard et le même digest `L3DMRD1`. Le compute-pool contient 12 CPU, +les quatre CPU `6,7,14,15` restent réservés, l'UMA est comptée, le minimum +`MemAvailable` vaut 12 421 971 968 octets et aucun swap-in/out n'est observé. +La preuve S21 finale sans override, +`final-s21-auto.stdout.jsonl`, ferme la boucle sur 172 741 paires : AUTO +sélectionne Vulkan pour les 21 630 admissions, publie 172 741 résultats en +2 345,444485079 s (73,649 paires/s), termine avec batch 8, inflight 1 et +helpers 0, et ne compte aucune panne, aucun discard ni slot pending. La seule +admission classée YELLOW ramène batch 8 à 1; plusieurs séquences GREEN +reconstruisent ensuite la référence et remontent 1 → 2 → 4 → 8, sans +rebond. Le minimum `MemAvailable` est 10 927 390 720 octets, PSI mémoire +maximal 0, swap-in/out maximal 0, GPU busy moyen/médian/maximal 26/27/36 % et +HWM processus maximal 250 658 816 octets. L'UMA admise reste 655 360 octets; +les masques exacts sont compute `0-5,8-13` et reserve `6,7,14,15`. Cette preuve +est opérationnelle : le digest scientifique et la reprise sont documentés par +le contrat Matcher, pas redéfinis ici. +L'ensemble v2 est **PASS / FROZEN**. Le gel porte sur cette architecture et +ses bornes validées, sans rouvrir Gate G ni le Matcher scientifique. + +Les décisions GPU négatives existantes restent inchangées : Candidate, Feature +et Visual Index restent CPU; SIFT/RootSIFT Matcher restent BFMatcher L2 CPU. Il +n'est introduit ni second scheduler, ni Queue, ni daemon, ni sous-système de +ressources. + +### Audit des 14 kinds de production + +Tous les kinds passent par l'unique Queue et l'unique Governor, y compris ceux +dont toutes les dimensions sont fixes. Dans le tableau, `CPU 1..N` décrit la +réduction possible par l'admission; `lot 1..N` décrit la dimension de lot +adaptable. La mémoire réservée vaut `fixe + par_item * batch_size`. Tous les +coûts GPU par item valent zéro; la forme ORB Vulkan normale réserve exactement +un slot inflight de 640 Kio, débité une fois de la RAM sur UMA. La capacité +privée de sûreté/benchmark peut retenir deux slots, soit 1,25 Mio, seulement +sous un contrat forcé depth 2. Device, pipeline, layouts et cache restent partagés; leurs +allocations driver opaques ne reçoivent pas un coût inventé. + +| Kind v1 | Estimation courante | Dimensions réellement consommées / constat Phase 1 | +| --- | --- | --- | +| `raw.develop` | MIXED; CPU 1; lot 1; hôte `2 Gio + contexte`, 0/item; I/O 1; GPU 0 | Fixe et atomique. Un garde Queue applique CPU1 au pool OpenCV process-wide puis restaure la valeur précédente. | +| `photo_quality.triage` | IMPORT; CPU 1; lot 1; hôte `contexte retenu + 20 Mio`, 0/item; I/O 1; GPU 0 | Un groupe par séquence. Un garde Queue applique/restaure CPU1 ; lot et contexte sont honnêtes. | +| `acquisition_campaign.run` | IMPORT; CPU 1; lot 1; hôte `contexte retenu + 256 Kio` + 64 Kio/item; I/O 1; GPU 0 | Sources, confirmations, requête et plan sont chargés avant admission. La reprise remplace seulement l'enveloppe opérationnelle historique sous-estimée par ce coût exact ; le snapshot durable reste inchangé. | +| `import.images` | IMPORT; CPU 1; lot 1..32; hôte 128 Kio + `NAME_MAX+64`/item; I/O 1; GPU 0 | Le callback consomme exactement le lot admis et réadmet entre lots. | +| `features.extract` | CPU; CPU 1..12; lot 1; hôte 64 Mio + 512 Mio/item; I/O 1; GPU 0 | Une image. Le callback applique/restaure exactement le CPU admis dans OpenCV, y compris si la vérification échoue après mutation. Sortie égale à 1/2/4/8/12. | +| `features.extract.sift` | CPU; CPU 1..12; lot 1; hôte 64 Mio + 1 Gio/item; I/O 1; GPU 0 | Même enforcement/rollback adaptatif. La forme durable courante reste CPU12 et la forme historique CPU1 exacte est normalisée en mémoire. | +| `features.extract.rootsift` | CPU; CPU 1..12; lot 1; hôte 64 Mio + 1 Gio/item; I/O 1; GPU 0 | Même exécution adaptative que SIFT ; aucune couture GPU validée. | +| `visual_index.update` | CPU; CPU 1..12; lot 1..16; hôte 8 Mio + 2 Mio/item; I/O 1; GPU 0 | CPU et lot sont lus du contrat; au plus `cpu_threads-1` enfants joints avant publication. | +| `candidate_pair.generate` | CPU; CPU 1..12; lot 1..64; hôte 256 Kio + 64 Kio/item; I/O 1; GPU 0 | CPU et lot sont consommés; fenêtre `min(2*CPU, 24)`. La forme sérielle historique exacte est normalisée en mémoire. | +| `matcher.run` | CPU: CPU 1..12, lot 1..12, hôte 0 + 10 Mio/item, I/O 1, GPU 0. ORB Vulkan normal: CPU 1, lot 1..8, même hôte/item, I/O 1, GPU 1 + 640 Kio, inflight 1. | ORB AUTO adapte seulement le lot sur huit observations pures par palier; CPU complet en fallback. Le benchmark privé conserve batch 12 et depth 2 (1,25 Mio), pas la politique normale. Vulkan explicite et contrôle synchrone restent depth 1; SIFT/RootSIFT et CPU explicite restent fixes. Helpers=0. | +| `geometric_verifier.run` | CPU; CPU 1; lot 1..8; hôte 4 Mio, 0/item; I/O 1; GPU 0 | Le lot est consommé séquentiellement; USAC conserve `isParallel=false`. | +| `track_builder.run` | CPU; CPU 1; lot 1; fixe `(4 Mio + arêtes * (48 + 2*160)) * facteur`, facteur 2 jusqu'à 400k arêtes puis 8; 0/item; I/O 1; GPU 0 | Rebuild atomique. La reprise valide le scope mais ne recalcule pas l'estimation avant admission. | +| `sparse_sfm.run` | CPU; CPU 1; lot 1; fixe `ceil_Mio(128 Mio + 64 Kio/image + 2 Kio/track + 512 octets/observation)`; 0/item; I/O 1; GPU 0 | Exécution atomique; BA à un thread. La forme est redérivée à la reprise sans réconciliation de l'estimation persistée. | +| `incremental_reconstruction.run` | CPU; CPU 1; lot 1; fixe `ceil_Mio(256 Mio + 128 Kio*(caméras base + images extension) + 4 Kio*(landmarks base + tracks extension) + 1 Kio*(observations base + extension))`; 0/item; I/O 1; GPU 0 | Exécution atomique; la forme est redérivée à la reprise sans réconciliation de l'estimation persistée. | + +Ces écarts sont des constats d'implémentation Compute Governor v2. Ils ne +créent ni nouvelle limite scientifique, ni nouvelle identité, ni modification +du schéma Project DB. Une enveloppe privée de capacités peut corriger la +sélection/réconciliation sans modifier l'ABI C public du Task Kind Registry. + ## SIFT v1A Une extraction SIFT demande jusqu'à douze threads CPU, un slot IO, aucun GPU, -pour une image. Le Governor réduit ce plafond au budget hôte, identique à la -limite OpenCV process-wide configurée avant les workers. L'estimation -structurelle conservatrice est environ 1,06 Gio (décodage, -pyramides, candidats et F32×128), lot 1, pic de record batch zéro. La -réservation couvre ainsi le fan-out interne sans créer un second pool runtime. +pour une image. L'estimation structurelle conservatrice est environ 1,06 Gio +(décodage, pyramides, candidats et F32×128), lot 1, pic de record batch zéro. +Le callback applique exactement le contrat admis dans `1..12` au pool OpenCV +process-wide sous l'unique propriétaire Queue, puis restaure la valeur +précédente sur toutes les sorties. Les tests déterministes ORB, SIFT et +RootSIFT couvrent 1/2/4/8/12 et obtiennent les mêmes keypoints, descripteurs et +métriques. Cette dimension opérationnelle ne change ni fingerprint ni Feature +Set. ## Responsabilité @@ -19,7 +309,9 @@ Le profil interactif par défaut conserve un quart de la RAM détectée et un quart des threads logiques pour le système hôte. Sur 16 Gio/16 threads, cela donne environ 3,8 Gio et 4 threads de headroom. Une nouvelle admission attend également lorsque PSI CPU `some avg10` atteint 20 %, ou PSI mémoire 1 %. Ces -signaux n'interrompent jamais le petit job déjà réservé. +signaux n'interrompent jamais le petit job déjà réservé. Ces valeurs décrivent +le profil Gate G core gelé; la cible opérationnelle v2 active est le couple +3 GiB/2 GiB et les signaux différentiels définis en tête de document. La soft floor vaut un quart et la hard floor un huitième de la RAM détectée. La soft floor place le Governor au minimum en YELLOW ; la hard floor le place @@ -55,8 +347,9 @@ une fois du budget RAM. Le multi-GPU est différé. Le Governor ne garantit aucune allocation et ne transforme ni swap, ni zram, ni stockage externe en RAM. Il ne modifie aucun paramètre scientifique. Aucun -scratch, cache de télémétrie, suivi RSS, redimensionnement de réservation ou -monitoring live n'appartient à Gate G core. +scratch, historique RSS long terme, redimensionnement de réservation depuis le +RSS ou monitoring live n'appartient à Gate G core. L'observation courante +bornée RSS/HWM de Compute Governor v2 reste strictement diagnostique. ## API principale @@ -91,6 +384,8 @@ monitoring live n'appartient à Gate G core. 4. Les réservations sont libérées exactement une fois 5. Les estimations de ressources sont immuables 6. L'historique des métriques est strictement borné (8 entrées par classe) +7. Un contrat de séquence est immutable jusqu'à sa libération; seule la + séquence suivante peut être adaptée ## Cycle de vie @@ -118,7 +413,8 @@ monitoring live n'appartient à Gate G core. ## Réserves - Sous-estimation temporaire possible avec des estimations statiques -- Pas d'adaptation basée sur le débit (duration_ns non encore utilisé) +- L'adaptation de débit v2 reste bornée au dernier état par kind/backend ; elle + ne persiste ni historique volumineux ni décision matérielle. - L'import `import.images` est admis avec 128 Kio fixes, un coût borné par item, un thread CPU, un slot I/O et des lots de 1 à 32. Il enregistre le nombre d'images logiques nouvellement enregistrées dans le ScanSet et la durée @@ -132,7 +428,9 @@ monitoring live n'appartient à Gate G core. un slot I/O, avec 64 Mio fixes et 512 Mio par image. Cette estimation conservatrice couvre le chemin actuel sans prétendre mesurer les allocations internes d'OpenCV. - `record_batch` couvre la validation source, le décodage, ORB, la publication + l'admission choisit 1..`min(12, compute_pool)` et le callback applique ce + nombre immutable au pool OpenCV. `record_batch` couvre la validation source, + le décodage, ORB, la publication et la finalisation DB ; `peak_memory_bytes == 0` signifie « mesure inconnue ». - `visual_index.update` demande jusqu'à douze threads CPU, un slot I/O, 8 Mio fixes et 2 Mio par Feature Set, par lots de 1 à 16. Le GPU vaut zéro. Le @@ -155,35 +453,82 @@ monitoring live n'appartient à Gate G core. et 24 sources au total ; le propriétaire de Task persiste ensuite seul et en ordre canonique. Cette estimation opérationnelle ne limite pas la taille scientifique du dataset. -- `matcher.run` demande jusqu'à huit threads CPU, un slot IO et 10 Mio par +- `matcher.run` demande jusqu'à douze threads CPU, un slot IO et 10 Mio par Candidate Pair admise, correspondant au working set contrôlé inférieur à environ 10 Mio par paire au maximum SIFT/RootSIFT (8 Mio de descripteurs contigus, KNN `k=2`, sorties et - fichier bornés), hors scratch interne OpenCV. Ses lots sont bornés à 1, 2, 4 - ou 8 Candidate Pairs. **IMPLEMENTATION IN PROGRESS — P4 :** une fenêtre - contient au plus deux paires par thread CPU effectivement admis et huit + fichier bornés), hors scratch interne OpenCV. Ses lots CPU restent bornés à + 1..12 Candidate Pairs; ORB Vulkan AUTO normal est borné à 1..8. + **PASS / FROZEN — P4 / Governor v2 :** une fenêtre + contient au plus deux paires par thread CPU effectivement admis et douze paires au total. Le callback Queue est un participant, crée au plus `cpu_threads - 1` enfants et les joint avant publication et libération de la réservation. Chaque paire conserve ses buffers dans un stage privé jusqu'à sa publication ordonnée ou son nettoyage ; OpenCV reste à un thread interne pour éviter une sursouscription imbriquée. Le Governor réserve donc au plus - 80 Mio contrôlés pour un lot de huit ; cette borne opérationnelle ne limite + 120 Mio contrôlés pour un lot de douze ; cette borne opérationnelle ne limite pas la cardinalité scientifique du dataset. - Le mode d'exécution est fixé par l'estimation immutable avant admission. Le - mode parallèle par défaut est CPU-only et ne réserve aucun GPU, même si le - Governor réduit ensuite son admission à un thread. Un mode explicitement - sériel ORB, avec profil GPU et backend runtime disponibles, demande exactement - un thread CPU, un slot GPU et 640 Kio ; sur UMA ces 640 Kio sont aussi débités - du budget RAM. Sa reconstruction conserve ce mode à partir de l'estimation - durable, sans sélection tardive d'une ressource non réservée. - La reprise accepte les formes courantes exactes CPU8/GPU0 et - CPU1/GPU1/640 Kio ainsi que leurs prédécesseurs exacts CPU12. Elle normalise - éphémèrement CPU12 vers la forme courante du même mode avant admission. Toute - forme voisine est rejetée ; cette compatibilité opérationnelle ne persiste ni - ne déduit une identité backend dans Project DB. + Une Task ORB normale nouvelle persiste la classe honnête `MIXED` avec les + champs opérationnels CPU12/GPU0, lot 1..12 et 10 Mio par paire : cette classe + signifie que la politique AUTO peut exécuter une séquence CPU ou Vulkan, pas + qu'elle réserve les deux simultanément et pas un tag de backend. L'override + CPU explicite persiste la même forme de ressources en classe `CPU`. + La signature durable ORB Vulkan courante demande CPU1/GPU1, lot 1..12 et + 640 Kio; son maximum 12 reste une sûreté durable/benchmark, pas le plafond + utile AUTO. L'enveloppe AUTO normale borne le lot à 8 et fige inflight 1; sur + UMA ce payload est + débité une seule fois du budget RAM. Le backend est créé sans payload mappé, + initialise et conserve exactement 640 Kio pour depth 1. La couture privée de + sûreté/benchmark peut porter 1,25 Mio pendant un contrat forcé depth 2. Il ne + redimensionne jamais avec une + requête pending; la fin de séquence libère le second slot avant la prochaine + admission depth 1. Une croissance échouée restaure la capacité antérieure, + et `backend_info` rapporte le payload réellement retenu, non le maximum de + l'enveloppe. Les anciennes + formes CPU8/GPU0 et CPU1/GPU1 à lot maximal 8, puis les formes plus anciennes + CPU12 à 10 Mio fixes, sont seulement des signatures exactes de reprise. Elles + sont normalisées éphémèrement vers la forme courante correspondante; toute + forme voisine est rejetée. + La production normale ORB reconstruit désormais une enveloppe privée + CPU/Vulkan et demande au Governor un choix `AUTO` avant chaque séquence. Une + fois admis, ce choix ne varie jamais dans la séquence. La nouvelle signature + `MIXED` reconstruit AUTO, y compris dans un build portable où son enveloppe + n'expose que CPU. Pour la compatibilité et la sûreté des overrides, toutes les + signatures ORB historiques/courantes de classe `CPU` reconstruisent un CPU + fixe ; une signature Vulkan reste fixe Vulkan. Aucun champ de ressource ne + sert de faux tag et aucun backend ou matériel n'entre dans l'identité + scientifique ou le payload Project DB. + La création et la reconstruction AUTO exposent une capacité Vulkan depuis + les seules métadonnées build/backend/GPU, sans appeler le driver ni + pré-dimensionner la RAM. Le Governor évalue ensuite l'enveloppe exacte contre + son snapshot MemAvailable/PSI/swap et sa charge UMA. Le premier begin et + donc toute initialisation se déroulent sur le worker Queue après application + de son affinité. La politique de cache Mesa est déjà établie au démarrage, + avant ce worker : aucun balayage post-init, latch de Task ou appel d'affinité + par TID auxiliaire n'existe. Une paire localement inéligible n'initialise + toujours pas le backend. Le diagnostic de séquence conserve séparément le + backend + sélectionné et le backend réel (`CPU`, `ORB_VULKAN` ou mix de paires + complètes), avec une raison de fallback. Les participants Matcher CPU sont + comptés uniquement dans `cpu_threads`; `helpers` reste zéro. + La soumission asynchrone est une couture privée de + `src/orb_vulkan_backend_internal.h`, absente de l'ABI public. Le backend + conserve deux slots maximum et un handle exact `slot+generation` par requête; + `finish` ne peut consommer que ce handle et les comptes soumis. Une génération + arrivée à `UINT64_MAX` ne boucle jamais : le slot est retiré avant toute + nouvelle soumission, de sorte qu'un ancien handle ne peut redevenir courant. + Toute + sortie/capacité invalide libère son slot, et + un échec d'attente de fence détruit/met en échec la session avant toute + nouvelle soumission. Les temporaires et objets pending sont nettoyés sur + erreur, exception C++, annulation et échec de publication. Une trace de test + bornée prouve sans temporisation, à depth 2, `SUBMIT(i) < SUBMIT(i+1) < + FINISH(i) < PUBLICATION_START(i) < PUBLICATION_FINISH(i)` pour deux paires + 769×769, sans modifier leur sortie. - `track_builder.run` réserve un worker CPU, aucun GPU et aucun fan-out GVR. L'estimation est `4 MiB + raw_inlier_edges * (48 + 2*160)` avec facteur 2 - sous 400000 arêtes et facteur 8 au-delà, après vérification d'overflow. Le + jusqu'à 400000 arêtes inclusivement et facteur 8 au-delà, après vérification + d'overflow. Le facteur élevé protège la transition mémoire observée à grande échelle ; le Governor reste l'unique propriétaire de l'admission et de la pression. @@ -192,6 +537,12 @@ monitoring live n'appartient à Gate G core. - Worker unique (pas de pools multiples) - Pas de priorités entre tâches - Pas de persistance des métriques +- Inflight Vulkan normal est fixé à 1, helpers GPU reste 0 et le lot AUTO + maximal utile est 8. Batch 12 et depth 2 demeurent des capacités privées de + sûreté/benchmark, chacune rejetée comme politique normale faute du gain de + débit durable de 5 %. La publication canonique reste owner-only et représente + environ 29,5 s dans les cohortes contrôlées; avec depth 2 sous 5 %, aucune + preuve ne justifie un helper supplémentaire face à cette frontière durable. - Pas de communication avec d'autres gouverneurs Les corrections dérivables G-D01 (`UINT64_MAX` est le dernier ID valide et la diff --git a/docs/architecture/task_kind_registry.md b/docs/architecture/task_kind_registry.md index 34e57e3..66fc5e0 100644 --- a/docs/architecture/task_kind_registry.md +++ b/docs/architecture/task_kind_registry.md @@ -41,6 +41,72 @@ Une panne pré-terminale peut donc répéter la normalisation exacte. Cette cout ne peut modifier ni identité, paramètres scientifiques, progression ou curseur métier, et toute forme voisine est rejetée. +## Inventaire production et entrées runtime + +`src/task_kinds.c::lardon3d_task_kind_registry_production()` enregistre les +14 kinds v1 du profil de production courant. La colonne « reprise » nomme le +reconstructeur du binding ; « callback » nomme l'entrée runtime privée dans le +même fichier. Le détail chiffré des capacités est centralisé dans l'[audit des +14 kinds](resource_governor.md#audit-des-14-kinds-de-production). + +| Kind v1 | Source, reprise et callback | Réconciliation pré-admission courante | +| --- | --- | --- | +| `raw.develop` | `raw_development_task.cpp`; `lardon3d_raw_development_task_reconstruct`; `run` | Aucune | +| `photo_quality.triage` | `photo_quality_task.cpp`; `lardon3d_photo_quality_task_reconstruct`; `run` | Aucune | +| `acquisition_campaign.run` | `acquisition_campaign_task.cpp`; `lardon3d_acquisition_campaign_task_reconstruct`; `run` | Aucune | +| `import.images` | `import_task.c`; `lardon3d_image_import_reconstruct`; `run_image_import` | Aucune | +| `features.extract` | `feature_task.c`; `lardon3d_feature_extract_reconstruct`; `run` | Durable CPU12; admission/runtime OpenCV 1..12 | +| `features.extract.sift` | `sift_task.c`; `lardon3d_sift_extract_reconstruct`; `run` | CPU1 historique exact → durable CPU12; runtime 1..12 | +| `features.extract.rootsift` | `sift_task.c`; `lardon3d_sift_extract_reconstruct`; `run` | CPU1 historique exact → durable CPU12; runtime 1..12 | +| `visual_index.update` | `visual_index_task.c`; `lardon3d_visual_index_update_reconstruct`; `run` | CPU runtime 1..12 ; feedback par segment durable | +| `candidate_pair.generate` | `candidate_pair_task.c`; `lardon3d_candidate_pair_generate_reconstruct`; `run` | Forme sérielle historique exacte → CPU12 ; runtime CPU 1..12 et lot 1..64 | +| `matcher.run` | `matcher_task.c`; `lardon3d_matcher_task_reconstruct`; `run` | Signatures CPU/Vulkan historiques exactes → formes courantes en mémoire | +| `geometric_verifier.run` | `geometric_verifier_task.c`; `lardon3d_geometric_verifier_task_reconstruct`; `run` | Aucune | +| `track_builder.run` | `track_builder_task.cpp`; `lardon3d_track_builder_task_reconstruct`; `run` | Aucune | +| `sparse_sfm.run` | `sparse_sfm_task.cpp`; `lardon3d_sparse_sfm_task_reconstruct`; `run` | Aucune | +| `incremental_reconstruction.run` | `incremental_reconstruction_task.cpp`; `lardon3d_incremental_reconstruction_task_reconstruct`; `run` | Aucune | + +## Couture privée Compute Governor v2 + +**COMPUTE_GOVERNOR_V2 — PASS / FROZEN.** Le descriptor C public +reste limité à kind, version et reconstructeur. L'enveloppe de capacités est +intégrée sans changement d'ABI dans le `struct Lardon3DTask` opaque et les +coutures privées `src/task_internal.h` / `src/resource_governor_internal.h`. +Les coutures d'admission sont `src/task_queue.c::select_admissible()`, +`src/task.c::lardon3d_task_sequence_break()` et, côté Governor, +la sélection multi-capacité sur un snapshot unique. La normalisation historique +exacte reste dans `src/task_kind_registry.c::normalize_known_legacy_estimate()`. + +Cette enveloppe n'est ni une identité scientifique, ni un nouveau payload +Project DB, ni un nouveau scheduler. Le Governor possède l'admission de tous +les kinds, y compris les formes entièrement fixes. Le contrat choisi est +immutable pendant une séquence et seule la suivante peut être adaptée. Une Task +sans alternative reçoit automatiquement une capacité égale à son estimation +durable. Le Governor conserve un état borné par kind/backend et un dernier +diagnostic ; ni l'enveloppe ni ce choix ne sont persistés. + +La politique CPU hôte reste privée au Governor : masque permis, groupes +package/core/SMT, compute-pool et résultat d'application du worker Queue. Le +compute-pool borne l'admission de chaque kind, y compris une capacité durable +CPU12. Feature/SIFT/RootSIFT consomment le compte immutable 1..12 dans OpenCV ; +les kinds CPU1 restent fixes. Aucun ID CPU ou choix d'affinité n'entre dans le +descriptor, le checkpoint ou le Project DB. + +Le feedback ne requalifie pas un succès de reprise en travail durable : les +kinds Feature/SIFT/RootSIFT comptent un item seulement après extraction et +publication durable propre. READY, collision `ALREADY_PRESENT` ou publication +incertaine compte zéro ; Visual Index compte pareillement zéro pour un segment +`PUBLISHED_NOT_DURABLE`. + +L'état privé par kind/backend coordonne désormais une seule dimension d'essai. +Les CPU validés slow-startent `1/2/4/8/12`; après deux observations de baseline, +deux observations à au moins +5 % sont nécessaires pour accepter le palier. +Une fois CPU stabilisé, seuls les kinds dont le callback consomme réellement +son lot peuvent ouvrir un essai de lot. `features.extract`, SIFT et RootSIFT +enregistrent une observation atomique réussie partagée entre Tasks ; Visual +Index, Candidate et Matcher enregistrent chaque séquence. Les autres formes ou +dimensions non adaptables restent égales à leur capacité fixe honnête. + ## Persistance et legacy Le checkpoint générique reste en version 1. Project Database v7 conserve le @@ -78,21 +144,95 @@ demande d'un thread CPU par douze avant admission. Le checkpoint historique et le curseur typé restent inchangés. Les snapshots Candidate courants et tous les autres kinds restent inchangés. -**IMPLEMENTED** — `matcher.run`, version 1, recharge la configuration Matcher, -l'identité Feature Set et le curseur `after_candidate_pair_id`. Il traite une -Candidate Pair atomique à la fois dans des lots bornés à huit, checkpoint le -curseur et repasse par le Governor entre les lots. La table durable -`matcher_tasks` est introduite par Project DB v11, après le Match Result v10. -Son reconstructeur accepte les formes opérationnelles exactes CPU8/GPU0 et -CPU1/GPU1/640 Kio, puis normalise éphémèrement leurs deux prédécesseurs CPU12 -vers la forme courante correspondante avant l'admission. Une forme -voisine échoue au lieu de servir d'indice de backend ; le payload Project DB ne -change pas et ne persiste aucune identité matérielle. +**PASS / FROZEN — Compute Governor v2.** `matcher.run`, version 1, +recharge la configuration Matcher, l'identité Feature Set et le curseur +`after_candidate_pair_id`. Il traite une Candidate Pair atomique à la fois dans +des lots bornés à douze, checkpoint le curseur et repasse par le Governor entre +les lots. La table durable `matcher_tasks` est introduite par Project DB v11, +après le Match Result v10. Son reconstructeur accepte les formes courantes +CPU12/GPU0 et CPU1/GPU1/640 Kio à lot `1..12`, les signatures historiques +CPU8/GPU0 et Vulkan à lot maximal 8, puis les formes CPU12 +pré-estimation-par-paire. La normalisation reste en mémoire. Une forme voisine +échoue au lieu de servir d'indice de backend ; le payload Project DB ne change +pas et ne persiste aucune identité matérielle. Les nouvelles Tasks ORB normales +ont une signature de classe `MIXED`, dont les autres champs restent une demande +de ressources réelle ; elle seule reconstruit la politique Governor `AUTO`. +Toutes les formes ORB de classe `CPU`, anciennes ou courantes, reconstruisent +un CPU fixe pour préserver les overrides explicites et une compatibilité sûre ; +une forme Vulkan restaurée reste fixe Vulkan. Un build portable reconstruit la +même politique `MIXED` mais n'expose que sa capacité CPU. Les snapshots tout à +zéro Candidate/SIFT/RootSIFT sont explicitement corrompus ; seules leurs +signatures historiques complètes exactes sont acceptées. Seule la forme AUTO +restaurée établit la disponibilité Vulkan partagée. Restaurer ensuite CPU, +Vulkan ou une signature historique fixe n'écrit rien dans cet état : la +co-restauration est indépendante de l'ordre. Aucun nouvel état de +backend n'est persisté. + +Pour une Task AUTO, la Registry reconstruit aussi l'enveloppe privée Vulkan +CPU1/GPU1, lot opérationnel `1..8`, helpers 0 et inflight 1. La signature +durable historique reste à lot `1..12`; la signature 640 Kio +reste la forme depth-1 minimale et n'est pas mutée; l'admission normale facture +exactement 640 Kio une seule fois sur UMA. Le choix inflight est immutable dans +la séquence et ne devient ni payload, ni fingerprint, ni indice de reprise. +Vulkan explicite reste depth 1. La capacité privée de sûreté/benchmark peut +forcer deux slots et 1,25 Mio sans changer la reconstruction normale. +Le backend ne mappe pas le maximum de l'enveloppe à sa création : il retient +exactement un slot à depth 1 et deux seulement sous une séquence depth 2 admise, +puis libère le second avant l'admission suivante. La signature durable 640 Kio +reste donc inchangée sans sous-facturer une allocation depth 2 forcée. + +L'A/B forcé ABBA a mesuré 54,661652238 paires/s à depth 1 et 55,797311953 à +depth 2, soit +2,077617 %, sous le deadband 5 %, avec digest identique, quatre +séquences de fallback local par exécution et zéro panne/discard. La Registry conserve donc +`DEPTH_MAX_VALIDATED_SAFETY=2` pour les seules coutures privées, mais la +capacité AUTO normale suit `DEPTH_MAX_USEFUL=1` : depth 2 est +**REJECTED_WITH_MEASURED_REASON**, sans nouvelle signature durable. + +La télémétrie privée de `matcher.run` conserve les classes de fallback par +séquence et compte aussi les items exacts local-ineligible/backend-failure/other +après leur publication durable. Ce détail opérationnel n'ajoute aucun kind, +champ durable ou identité et empêche le regroupement batch de devenir un +comparateur scientifique. Le commit immédiat par item reste acquis si une +paire suivante avorte, tandis que le feedback de séquence n'est pas enregistré; +la déduplication actuelle vit seulement avec la Task reconstruite en mémoire. +Les logs batch 2/4 antérieurs à ce compteur restent préliminaires et prouvent +seulement l'invalidité du comparateur par séquences. Les huit runs item-valides +`forced-batch{2,4,8,12}-items{,-b}.stdout.jsonl` conservent chacun 4113 paires, +six items locaux, zéro panne/autre et le même digest. Les débits combinés sont +54,180767704, 66,094373197, 74,784998723 et 76,755814095 paires/s. Les gains +jusqu'à batch 8 dépassent 5 %, celui de 8 à 12 vaut seulement +2,635308425 % : +la Registry expose `BATCH_MAX_USEFUL=8` en AUTO normal et réserve batch 12 aux +preuves privées (`REJECTED_WITH_MEASURED_REASON`). +Le S21 final confirme l'enveloppe Registry en production : `matcher.run` v1 +reste le même kind durable, AUTO choisit Vulkan pour 21 630 admissions et +termine 172 741/172 741 résultats à batch 8/inflight 1/helpers 0. Le passage +transitoire 8 → 1 → 2 → 4 → 8 ne modifie ni signature durable, ni +fingerprint, ni digest scientifique. Aucun backend ou champ persistant n'est +ajouté par cette adaptation. + +La reconstruction AUTO ne sonde ni n'initialise Vulkan sur le thread +`project_open()`. Elle expose la capacité depuis les seules métadonnées runtime +build/backend/GPU ; le Governor possède le dimensionnement exact et +l'admission UMA sur son snapshot. Le premier begin appartient au worker Queue après son +affinité. La politique Mesa sûre est déjà établie avant les pthreads et +l'initialisation du driver ; aucun sweep/latch auxiliaire n'appartient donc au +contexte Task. Une paire localement inéligible n'initialise pas le backend. Une +panne réelle produit des paires CPU complètes et rend le backend indisponible +aux admissions AUTO suivantes sans réécrire le snapshot durable. + +Le contrôle de benchmark `synchronous` du runner réel n'étend pas le descriptor +ni le reconstructeur. Il est compilé hors du binaire production, attaché +seulement au contexte éphémère d'une nouvelle Task et refusé par le runner si +une Task Matcher doit être reprise. La Registry continue donc à reconstruire +uniquement la politique AUTO/fixe déduite de la signature durable, jamais un +pipeline de benchmark. **IMPLEMENTED** — `features.extract.sift` et `features.extract.rootsift` acceptent leur forme CPU12 courante et normalisent uniquement leur forme CPU1 -historique exacte. Cette compatibilité opérationnelle n'altère ni fingerprint, -Feature Set, checkpoint durable, ni politique scientifique. +historique exacte. ORB/SIFT/RootSIFT appliquent ensuite le compte CPU admis dans +`1..12`; les sorties testées à 1/2/4/8/12 restent égales. Cette compatibilité +opérationnelle n'altère ni fingerprint, Feature Set, checkpoint durable, ni +politique scientifique. **IMPLEMENTED** — `geometric_verifier.run`, version 1, recharge la configuration Fundamental immuable, en revalide le fingerprint et reprend `after_match_result_id`. diff --git a/docs/architecture/task_system.md b/docs/architecture/task_system.md index ffddb8a..13e8014 100644 --- a/docs/architecture/task_system.md +++ b/docs/architecture/task_system.md @@ -87,8 +87,11 @@ affecter les autres entrées de l'inventaire. ## Invariants -1. **Estimation immuable** : une fois créée, l'estimation d'une tâche ne change - jamais. Elle est copiée en lecture seule lors de la réservation. +1. **Estimation durable immuable** : l'estimation stockée dans la Task courante + ne change jamais. Compute Governor v2 utilise une enveloppe privée de + capacités, distincte de ce snapshot et non persistée comme identité. Le + contrat choisi pour une séquence est lui aussi immutable jusqu'à sa + libération ; seule une séquence suivante peut être adaptée. 2. **Transitions d'état validées** : le cycle nominal est `PENDING → RUNNING → COMPLETED/FAILED/CANCELLED`, avec pause coopérative. 3. **Pause et annulation coopératives** : le callback appelle périodiquement @@ -105,7 +108,147 @@ affecter les autres entrées de l'inventaire. - **task_queue** : la file gère l'ordre d'exécution et invoque les callbacks. - **resource_governor** : l'estimation est utilisée pour la réservation avant exécution. -- **scheduler** : le scheduler transmet l'estimation lors de la soumission. +- **scheduler** : ses responsabilités restent représentées par le runtime et + l'unique Queue existants ; aucun second scheduler n'est introduit. + +### Frontière Compute Governor v2 + +**COMPUTE_GOVERNOR_V2 — PASS / FROZEN.** Chaque kind de production +reste propriétaire d'une Task et passe par l'unique Queue à un worker, puis par +l'unique Resource Governor, même lorsque CPU, lot, mémoire, I/O et GPU sont +tous fixes. Une dimension fixe n'autorise jamais à contourner l'admission. + +Le code conserve une `Lardon3DResourceEstimate` canonique immutable pour la +durabilité. Derrière les types opaques, chaque Task possède désormais une +enveloppe privée bornée : par défaut une capacité fixe exactement égale à cette +estimation ; seuls les kinds possédant des alternatives réelles en ajoutent. +Queue et `sequence_break()` demandent au Governor de choisir et réserver une +capacité depuis un seul snapshot courant. L'exécution reçoit ce contrat +immutable et ne le renégocie pas en cours de séquence. Cette couture n'étend ni +le descriptor public du Task Kind Registry, ni le checkpoint, ni le payload +Project DB et n'ajoute aucune dimension d'identité scientifique. + +Le Governor possède aussi la politique CPU hôte privée. Depuis le masque +permis et la topologie package/core, il dérive un compute-pool par coeurs +physiques complets ; un caller déjà précontraint fournit directement son pool. +Sans topologie exploitable, le budget portable subsiste avec affinité inactive. +Seul le worker lourd de Queue applique et vérifie son propre masque (`pid=0`) +avant les callbacks, donc le creator/main/TUI reste libre. Le Governor ne mute +jamais un TID auxiliaire énuméré : un `PIDFD_THREAD` ne stabilise pas le numéro +TID consommé par `sched_setaffinity(tid)`. Le démarrage établit plutôt +`MESA_SHADER_CACHE_DISABLE=true` avant tout pthread applicatif et toute +initialisation Vulkan. Une absence prend ce défaut sûr ; `true`/`1` explicites +sont conservés, tandis qu'une valeur explicite fausse ou malformée est respectée +mais entraîne un refus de démarrage. Cette politique opérationnelle, sans état +Task durable ni identité scientifique, supprime les helpers de cache Mesa +observés qui élargissaient leur masque. Il n'existe plus de sweep post-init, +latch de Task ou retry de recontrainte auxiliaire ; les diagnostics indiquent +l'état et la raison de la politique sans prétendre une activité auxiliaire. +Le nombre du pool borne toute admission CPU. Cette couture +n'ajoute ni scheduler, ni worker, ni champ durable ou ABI publique. + +Le comportement de production normal ORB est `AUTO` Governor-owned. Les modes +CPU/Vulkan explicites sont réservés au debug, benchmark et à la +reproductibilité. Une Task normale nouvelle persiste une classe `MIXED` +sémantiquement honnête ; cette signature reconstruit AUTO, tandis que toute +signature CPU ORB ancienne ou courante reconstruit un CPU fixe et que Vulkan +reste fixe. Création et reprise n'initialisent pas Vulkan sur le caller ; le +premier begin appartient au worker Queue contraint. Le choix matériel ne +devient ni payload ni identité. Seule une reprise AUTO établit la disponibilité +Vulkan partagée ; les reprises fixes et historiques sont sans effet global, +donc leur ordre ne dégrade pas AUTO. Une panne backend est publiée avant tout +fallback ou sortie précoce ; une inéligibilité de paire reste locale et ne +possède aucun handle à terminer. Le Governor conserve une télémétrie fixe et +bornée par kind/backend (dernière durée, travail durable, débit, backend +sélectionné/réel, contrat, pression et raison) et aucune grande histoire +persistante. L'adaptation générique utilise deux observations de référence et +deux du palier d'essai. Le lot ORB Vulkan, plus bruité, exige huit séquences +pures consécutives pour chacune de ces fenêtres avant toute décision ; une +observation Matcher mesure la cadence de bout en bout depuis la réadmission +réussie jusqu'au checkpoint générique durable, et non le seul callback de +calcul. Le temps de calcul seul reste diagnostique. Ainsi le Governor apprend +le coût de séquence amorti par le lot et un checkpoint échoué ne l'entraîne pas. +Une +admission adaptative encore permise sous pression installe +CPU1, lot minimum et inflight minimum avant sa réservation. Inflight ORB normal +reste fixé à 1; helpers reste 0. Les callbacks atomiques n'annoncent +un item que si extraction et publication propre sont durables ; READY, +`ALREADY_PRESENT` et `PUBLISHED_NOT_DURABLE` sont des observations zéro qui +n'avancent pas la rampe. +L'A/B forcé ABBA a mesuré seulement +2,077617 % à depth 2 +(54,661652238 contre 55,797311953 paires/s), sous le deadband 5 %, avec digest identique, +quatre séquences de fallback local par exécution et zéro panne/discard. Depth 2 est donc +**REJECTED_WITH_MEASURED_REASON** pour AUTO normal. Compute Governor v2 reste +en cours jusqu'aux réconciliations restantes. + +Pour Matcher Vulkan, le lease privé de capacité matérialise le contrat de la +séquence seulement après son admission. Il alloue un ou deux payloads de +640 Kio sans requête pending, les conserve jusqu'au nettoyage complet de la +séquence, puis rend depth 2 à depth 1 avant `sequence_break()`. Une allocation +depth 2 échouée ne modifie pas le contrat scientifique et conduit à la paire CPU +complète; elle ne laisse ni réservation suivante sous-facturée, ni résultat +Vulkan partiel. + +Cette boucle est maintenant `observe → choose → execute → measure → adapt next`. +Les CPU réductibles progressent uniquement par `1/2/4/8/12`, bornés par le +compute-pool ; CPU et lot ne sont jamais essayés ensemble. Les observations +hôte privées comprennent utilisation du pool, mémoire/PSI/swap actif, GPU busy +et RSS observé, sans confondre RSS et réservation. Le dernier diagnostic peut +être tiré par numéro de série ou formaté dans un buffer borné ; le runtime ne +l'imprime pas directement dans le TUI. Le contrat déjà installé demeure +immutable même si une nouvelle observation arrive pendant son exécution. +Un agrégat privé de taille fixe complète ce dernier diagnostic : il compte les +admissions et séquences effectivement enregistrées et somme leurs métriques en +saturant, afin qu'un poller lent ne transforme pas des changements coalescés en +fausse histoire exhaustive. Il vit uniquement avec le Governor courant. +Pour la matrice forcée du runner, cet agrégat sépare les fallbacks par +inéligibilité locale, panne backend et raison autre/inconnue, à la fois par +séquence diagnostique et par item exact. Le compte d'items avance une seule fois +après la publication durable du fallback CPU complet; le travail d'une +séquence sélectionnée CPU reste à zéro. Ce commit opérationnel immédiat survit à +l'échec ou l'annulation d'une paire suivante, mais ne crée ni séquence réussie, +ni débit durable, ni adaptation. Un high-water mark non persisté le rend +idempotent pendant la vie de la Task. La Task n'expose alors qu'une capacité +Vulkan aux batch/depth demandés : absence de GPU/backend/mémoire ne peut donc +pas devenir une admission CPU. Une panne backend tardive laisse la publication +CPU complète déjà durable mais fait échouer la Task de benchmark après +checkpoint; seul le compte d'items localement inéligibles, égal entre cohortes +comparées même lorsque leur batch diffère, reste admissible pour l'évidence. +Les anciens logs batch 2/4, à quatre contre trois séquences locales pour les +mêmes items/digest, sont préliminaires et ne déterminent aucun débit utile. La +matrice item-valide suivante mesure batch 2/4/8/12 à +54,180767704/66,094373197/74,784998723/76,755814095 paires/s. Les deux premiers +gains de palier dépassent le deadband, celui de 8 à 12 vaut seulement ++2,635308425 % : AUTO normal s'arrête à batch 8 et batch 12 reste un contrôle +privé sûr rejeté pour la politique normale. +Le run sans override `short-auto-batch8-governor-v2.stdout.jsonl` valide ensuite +la Task réelle : `1 → 2 → 4 → 8`, 4113/4113 résultats, 76,072 paires/s, digest +identique, six items locaux et aucune panne/discard. La cadence de séquence +inclut réadmission et checkpoint durable; inflight reste 1 et helpers 0. +Le run S21 final crée ensuite la Task normale 2831 en mode AUTO, sans option +backend/lot/inflight. Ses 21 630 admissions sont toutes Vulkan; la Task publie +172 741/172 741 résultats, termine `COMPLETE` à 100 %, avec zéro doublon et +curseur complet. La seule admission YELLOW produit un contrat batch 1, puis +les admissions GREEN suivantes remontent de façon bornée jusqu'à batch 8. Le +contrat actif n'est jamais muté sous le callback : chaque changement appartient +à la séquence suivante. + +Le runner d'évidence réel crée les nouvelles Tasks Matcher normales par l'API +AUTO. Son contrôle `synchronous` est un flag de contexte compilé seulement dans +le target benchmark/test : il ne modifie ni Task durable, ni envelope, ni +contrat installé, ni callback de production. Puisque ce flag n'est pas +checkpointé, le runner refuse expressément de l'appliquer à une reprise Matcher +pendante. Le chemin rolling/recovery normal reste inchangé. + +Dans ce même target seulement, `--matcher-inflight 1|2` reconstruit avant la +création d'une Task neuve une enveloppe AUTO Vulkan à profondeur fixe et batch +2 par défaut. `--matcher-batch 2|4|8|12`, valable seulement avec inflight et +rolling AUTO, fixe aussi le lot. Ces contrôles ne remplacent pas le Governor : l'admission, la mémoire +par slot, l'UMA et la réservation de séquence restent identiques. Le contrôle +est refusé pour les modes explicites, pour synchronous depth 2 et pour une Task +pendante. Une garde de processus restaure les tokens privés sur toute sortie; +aucun token, champ de contexte, payload ou comportement correspondant n'est +compilé dans `lardon3d`. ## Statut @@ -134,11 +277,25 @@ candidates avec idempotence, checkpoint après chaque lot et repasse par le Governor via `sequence_break`. La reprise est idempotente avec le curseur `after_feature_set_id` rechargé depuis la DB. -**IMPLEMENTED** — `matcher.run` traite une Candidate Pair atomique à la fois, -par lots adaptatifs de 1, 2, 4 ou 8. Il persiste le curseur +**PASS / FROZEN — Compute Governor v2.** `matcher.run` traite une +Candidate Pair atomique à la fois, par lots opérationnels bornés jusqu'à 12. +Le code courant consomme honnêtement CPU, lot et GPU du contrat choisi. Feature, +SIFT et RootSIFT appliquent l'admission OpenCV `1..12`; RAW et Photo Quality +restent CPU1. Pour ORB normal, le Governor choisit GPU-first ou CPU complet pour la prochaine +séquence, sans mutation pendant son exécution. Ces dimensions ne changent ni +identité scientifique ni publication. Le callback persiste le curseur `after_candidate_pair_id`, checkpoint après publication de chaque lot et effectue une rupture de séquence avant le suivant. Une paire repassée après un -crash est réutilisée par son Match Result. +crash est réutilisée par son Match Result. La soumission Vulkan rolling est +privée et request-bound; le contrat normal de séquence fige inflight 1. Deux slots +maximum restent disponibles à la couture privée de sûreté/benchmark et portent +chacun command/fence/buffers/query et un handle exact tandis +que device/pipeline/layout/cache restent partagés. Le propriétaire soumet +jusqu'à la profondeur admise, finit le plus ancien et publie toujours le +préfixe canonique contigu; toute sortie, annulation ou exception nettoie les +handles encore privés. Les buffers du second slot ne sont mappés que pendant +une séquence depth 2 admise et sont libérés avant la rupture suivante. Helpers +reste 0 et le contrôle synchrone force depth 1. **IMPLEMENTED** — `geometric_verifier.run` v1 traite un Match Result atomique à la fois, par lots adaptatifs de 1, 2, 4 ou 8. Project DB v13 conserve sa diff --git a/docs/architecture/vulkan_matcher.md b/docs/architecture/vulkan_matcher.md index fac7abc..0db89a9 100644 --- a/docs/architecture/vulkan_matcher.md +++ b/docs/architecture/vulkan_matcher.md @@ -33,6 +33,18 @@ device, la file, le pipeline, le command buffer et trois buffers bornés. Un mutex impose un dispatch à la fois. Une famille compute sans graphics est préférée, avec fallback vers toute famille compute compatible. +Le propriétaire du processus doit établir `MESA_SHADER_CACHE_DISABLE` à la +valeur exacte `true` ou `1` avant de créer ses threads. Les exécutables +Lardon3D normaux prennent le défaut sûr si la variable est absente et refusent +une valeur explicite différente. Le backend public peut cependant être appelé +par un autre consumer après son propre démarrage : sa frontière d'initialisation +ne mute donc jamais l'environnement. Avant tout appel Vulkan/Mesa, elle refuse +une valeur absente, fausse ou malformée, mémorise le backend comme indisponible +et retourne `LARDON3D_ORB_VULKAN_UNAVAILABLE` sans sortie partielle. Les appels +vides et la lecture metadata restent non initialisants. Les exécutables autonomes +de benchmark/feasibility établissent le même défaut sûr comme première action de +`main` ; cette politique reste opérationnelle, non persistée et non scientifique. + Le sélecteur utilise le travail `feature_count_a × feature_count_b`. Sous le seuil mesuré de `768 × 768` comparaisons, OpenCV reste utilisé afin d'éviter le coût fixe du dispatch. Le seuil est une politique d'exécution et ne modifie ni @@ -41,8 +53,15 @@ matche aucun grand couple ORB ne paie aucun cold start. ## Mémoire et pannes -Les buffers maximaux contiennent 256 Kio pour A, 256 Kio pour B et 128 Kio pour -8192 sorties top-2, soit 640 Kio de payload, hors petits objets du driver. Une +Chaque slot contient 256 Kio pour A, 256 Kio pour B et 128 Kio pour 8192 +sorties top-2, soit 640 Kio. Le contexte fraîchement créé ne mappe aucun +payload. L'initialisation depth 1, rolling AUTO normal et le wrapper public +synchrone retiennent exactement un slot. Seul un contrat privé de +sûreté/benchmark depth 2 mappe le second, soit 1,25 Mio pendant cette séquence. +Device, pipeline, +layouts et cache restent partagés, hors objets opaques du driver. Commandes, +fences, descriptor sets et queries sont des métadonnées bornées à deux slots, +mais ne rendent pas le second payload mappé tant qu'il n'est pas admis. Une mémoire host-visible, cohérente et cached est préférée sur UMA, car elle réduit nettement le coût CPU de copie/readback observé sur RADV. Le backend sait appliquer flush/invalidate lorsque le type retenu n'est pas cohérent. @@ -52,12 +71,44 @@ utilise le CPU sans nouvelle tentative par paire. Une panne de soumission ou un device lost désactive Vulkan pour la session ; la paire courante est reprise sur CPU avant toute publication. Un Match Result n'est jamais créé depuis une sortie GPU partielle. +La couche Matcher distingue cette faute backend d'une faute locale. Une lecture +Feature/allocation échouée avant `begin` ne consomme aucun slot; une faute de +filtrage/allocation/staging après un `finish` backend réussi a déjà consommé +seulement son handle exact. Ces deux cas recalculent la paire complète sur CPU, +comptent `other` après publication durable et conservent la disponibilité du +backend ainsi que tout successeur soumis sain. -Le job est synchrone au niveau du Matcher. Il soumet sur l'unique queue détenue -par le contexte puis attend cette queue, sans `vkDeviceWaitIdle` sur le chemin -normal. Le Resource Governor admet la tâche avant le callback ; en RED aucun -nouveau batch ne démarre, tandis qu'une paire déjà soumise finit et se publie. -Le worker unique et le mutex interdisent plusieurs dispatchs concurrents en v1. +Le Resource Governor fige inflight 1 avant le callback AUTO normal. La couture +benchmark/test peut forcer 1 ou 2. Au début de la séquence, sans requête pending, +le backend alloue la capacité exacte avant le +premier submit ; une croissance échouée conserve la capacité antérieure et +provoque le fallback CPU complet. Le worker unique soumet jusqu'à cette +profondeur sur l'unique queue, attend chaque fence exacte et publie le préfixe +en ordre, sans `vkDeviceWaitIdle` sur le chemin normal. La fin de séquence, +succès, annulation ou erreur nettoyée, libère le second payload avant la +prochaine admission depth 1. En RED aucun nouveau batch ne démarre; la pression +active ramène l'admission courante permise au depth minimum. Le wrapper public +et le contrôle de benchmark synchrone forcent depth 1. Aucun helper hôte n'est +créé. + +L'ABBA corpus forcé mesure 54,661652238 paires/s à depth 1 et 55,797311953 à +depth 2, soit +2,077617 %, sous le deadband 5 %, avec digest identique, quatre fallbacks +locaux et zéro panne/discard. Depth 2 est donc +**REJECTED_WITH_MEASURED_REASON** pour AUTO normal; sa capacité maximale 2 +reste disponible uniquement pour les preuves privées reproductibles. +Le S21 AUTO final exerce le backend normal sur 172 741 paires : 172 507 +soumissions et 172 507 complétions, zéro panne, zéro discard et aucun slot +pending à la sortie. Les 234 items sous le seuil restent des fallbacks locaux +CPU complets; ils ne sont jamais passés à `finish`. Le digest canonique final +est `e5128a2e599ff593c4f79850e067254b1f249d19e8480a44973306b1af250f70` +pour 172 741 mappings contigus et sans doublon. La réservation AUTO reste un +slot de 655 360 octets UMA pendant tout ce run. + +Chaque requête privée porte le couple exact `slot+generation`. La génération +ne boucle jamais : après l'usage de `UINT64_MAX`, ce slot est retiré avant toute +nouvelle soumission et ne peut plus faire correspondre un handle ancien. Un +second slot sain peut terminer sa requête indépendamment. `finish` et `discard` +consomment uniquement le handle exact et ne mélangent jamais leurs sorties. ## Shader et déterminisme diff --git a/docs/performance/target_hardware.md b/docs/performance/target_hardware.md index abafd15..0536ce8 100644 --- a/docs/performance/target_hardware.md +++ b/docs/performance/target_hardware.md @@ -16,17 +16,48 @@ chemins SIMD jusqu'à AVX512-SKX. Il a été compilé avec OpenCL, mais `AMD Radeon 780M Graphics (RADV PHOENIX)`, API 1.4.354, avec une file compute dédiée et de la mémoire UMA host-visible/cohérente. -Le runtime actuel de Lardon3D possède un worker. Le profil interactif conserve -12 threads OpenCV process-wide, quatre threads logiques pour le desktop et un -seul Matcher actif. Lorsque -les pools multi-workers seront introduits, la cible de départ recommandée est -deux Matchers avec huit threads OpenCV chacun pour une charge mixte. Les mesures -montrent toutefois que quatre Matchers à quatre threads favorisent SIFT et les -cas 4096, tandis que deux à huit favorisent ORB 8192. Le Governor devra donc -choisir à partir de la classe de charge, pas d'une constante universelle. Le -nombre de threads OpenCV devra être réglé une fois au démarrage : -`cv::setNumThreads()` est une configuration globale et ne doit jamais être -modifiée concurremment par des workers. +Le sysfs amdgpu de cet hôte expose 512 Mio dans `mem_info_vram_total` et +7 986 020 352 octets dans `mem_info_gtt_total`. Le profil matériel conserve les +512 Mio comme capacité de payload observable, mais la combinaison petit +aperture/GTT à l'échelle de la RAM suffit à classer ce GPU shared/UMA sans +hardcoder son device ID. Le Governor débite alors les ressources GPU exactement +une fois de `MemAvailable` et n'utilise jamais cet aperture comme mémoire libre +séparée pour contourner les objectifs 3 Gio/2 Gio. + +Le runtime actuel de Lardon3D possède un worker lourd. Le profil interactif +établit au démarrage une baseline/plafond OpenCV de 12 threads et réserve quatre +threads logiques au desktop. Pour chaque séquence, l'unique callback Queue +applique temporairement le compte CPU immuable admis dans `1..12`, le vérifie et +restaure la baseline sur toute sortie. `cv::setNumThreads()` reste une +configuration process-wide : sa mutation concurrente par plusieurs workers +n'est pas supportée. Un futur pool multi-worker devrait donc changer ce modèle +explicitement, pas multiplier silencieusement ces mutations globales. + +Le worker Queue applique à lui-même le compute-pool `0-5,8-13`, tandis que le +creator/main reste unrestricted `0-15`. Certains helpers de cache disque Mesa +observés avaient réélargi leur affinité après l'initialisation lazy. Comme un +pidfd ne stabilise pas le TID numérique pour `sched_setaffinity(tid)`, Lardon3D +ne tente aucune mutation auxiliaire. Il établit plutôt +`MESA_SHADER_CACHE_DISABLE=true` avant toute création de pthread applicatif et +toute initialisation Vulkan. Une absence prend ce défaut sûr ; les valeurs +explicites exactes `true`/`1` sont respectées, tandis qu'une valeur explicite +fausse ou malformée reste inchangée mais interdit le démarrage. Sur la 780M +validée, les helpers `*:disk$0` sont absents et tous les threads runtime vivants +observés gardent `0-5,8-13`. Cette politique est opérationnelle et inoffensive +hors Mesa ; elle n'ajoute ni identité scientifique, ni état durable, ni sweep +post-init. +Le backend public ne suppose pas que tout consumer traverse ce démarrage : sa +première requête Vulkan non vide vérifie `true`/`1` sans modifier +l'environnement, avant tout appel Mesa. Une valeur absente ou différente +retourne `UNAVAILABLE`, mémorise l'indisponibilité et ne produit aucune sortie. +Les benchmarks et la feasibility autonomes établissent le défaut sûr comme +première action de `main` ; les lectures metadata restent non initialisantes. + +Les anciennes mesures exploratoires montrent que quatre Matchers à quatre +threads favorisent SIFT et les cas 4096, tandis que deux à huit favorisent ORB +8192. Elles n'établissent pas un pool de production actuel. Tout futur modèle +devrait être choisi par le Governor à partir de la classe de charge et prouver +un contrôle de concurrence compatible avec la configuration globale OpenCV. Quatre Matchers avec un ou deux threads chacun dégradent fortement les grands cas ORB. Quatre fois quatre threads augmente le working set et la variance, mais @@ -65,10 +96,24 @@ environ 0,10, 0,96, 14,7 et 59,6 ms pour BFMatcher CPU lors de la campagne production. L'initialisation lazy mesurée vaut environ 129–136 ms. Le seuil `feature_count_a × feature_count_b >= 768²` évite le GPU pour les petits travaux. -La mémoire permanente directement contrôlée vaut 640 Kio de payload. Les tests -de parité couvrent exactement le top-2 jusqu'à 8192, le Match File complet et le -fallback CPU. Ces nombres décrivent la machine mesurée et ne sont pas un contrat -portable de latence. +La mémoire directement contrôlée vaut 640 Kio par slot; rolling AUTO peut +réserver un ou deux slots, soit au plus 1,25 Mio, débités une fois de la RAM sur +la 780M UMA. Le backend mappe exactement cette capacité pendant la séquence et +rend le second slot avant une admission depth 1 suivante. Les tests de parité +couvrent exactement le top-2 jusqu'à 8192, le Match File complet et le fallback +CPU. Ces nombres décrivent la machine mesurée et ne sont pas un contrat portable +de latence. + +Le run S21 AUTO final sur cette cible mesure 172 741 résultats durables en +2 345,444485079 s, soit 73,649/s. Sur les échantillons connus, GPU busy vaut +26 % en moyenne, 27 % en médiane et 36 % au maximum; l'utilisation moyenne +connue du compute-pool vaut 9,68 % de ses 12 CPU logiques. Le RSS observé +culmine à 168 980 480 octets et le HWM à 250 658 816 octets. Le minimum +`MemAvailable` reste 10 927 390 720 octets, PSI mémoire maximal et deltas swap +restent nuls. Après le run, Sway répond, Firefox et son flux PipeWire vers le +casque actif restent présents; PSI mémoire avg10/60/300 est nul. Ces signaux +objectifs attestent la conservation du desktop, sans prétendre mesurer une +perception subjective. ## Geometric Verifier Fundamental — Gate A diff --git a/docs/roadmap/roadmap.md b/docs/roadmap/roadmap.md index 7da3b85..2a2d6f9 100644 --- a/docs/roadmap/roadmap.md +++ b/docs/roadmap/roadmap.md @@ -232,11 +232,224 @@ GPU soit approprié ni qu'une sortie GPU partage automatiquement l'identité scientifique CPU. Le gel ne revendique pas de comparaison de débit durable CPU-versus-GPU sur corpus : la pression hôte a contaminé cette mesure. -Un éventuel **GPU COMPUTE v1** est postérieur et optionnel. Il ne peut être -ouvert que si un audit futur et une preuve d'équivalence à la frontière concernée -le justifient ; sinon le chemin CPU demeure le chemin retenu. Il ne modifie pas -les contrats scientifiques, les résultats canoniques ni l'unique gouvernance des -ressources. +Cette limite de preuve v1 n'annule pas l'évidence directe fournie à la tranche +suivante. Pour le hot path ORB Matcher, Vulkan est désormais validé, +déterministe et mesurément supérieur ; il devient donc le workload primaire de +la politique v2 GPU-first, sous admission GPU/UMA, avec fallback CPU. Candidate, +Feature et Visual Index restent explicitement rejetés pour le GPU, et +SIFT/RootSIFT Matcher restent BFMatcher L2 CPU. + +**COMPUTE_GOVERNOR_V2 — PASS / FROZEN.** +**ORB_VULKAN_ASYNC_EXECUTION — PASS / FROZEN.** Cette tranche +autorisée fait évoluer l'unique Governor, sans créer de scheduler, Queue, +daemon ou persistance parallèle : télémétrie bornée, admission adaptative par +séquence et `AUTO` GPU-first pour les backends exacts et mesurément supérieurs. +La couture ORB Matcher normale est gelée avec CPU complet en fallback ; les +choix CPU/Vulkan explicites restent des overrides de debug, benchmark et +reproductibilité. Les dimensions retenues et toutes les validations v2 sont +closes. + +CPU12 est validé. Le Governor dérive désormais le pool lourd depuis le masque +permis et les groupes package/core/SMT. Sur l'hôte unrestricted courant, il +obtient `0-5,8-13` et réserve `6,7,14,15`; un caller déjà précontraint ne subit +pas une seconde réserve. Le worker Queue seul applique/vérifie son propre +masque ; aucun TID auxiliaire énuméré n'est muté, car un pidfd ne stabilise pas +le numéro consommé par `sched_setaffinity(tid)`. Avant tout pthread applicatif +ou driver, le démarrage établit `MESA_SHADER_CACHE_DISABLE=true`; une valeur +absente prend ce défaut, `true`/`1` explicites sont conservées et toute autre +valeur explicite est préservée mais refusée. Cette politique non scientifique +supprime les helpers de cache Mesa observés qui élargissaient leur masque. Les +threads runtime restants héritent le compute-pool ; il n'existe plus de sweep, +latch ou retry auxiliaire et le diagnostic expose la politique réelle. +Creator/main/TUI reste unrestricted. Le fallback au budget portable ne crée +aucune exclusion +inventée. Cette couture est **PASS / FROZEN** sur le profil validé; ces IDs ne +sont pas une politique portable. La cible RAM conserve +3 GiB de `MemAvailable` et ne franchit pas intentionnellement le plancher dur de +2 GiB. Les PSI CPU/mémoire/I/O et les deltas swap-in/swap-out sont des signaux +actifs ; l'occupation totale du swap reste historique. Admission CPU et lot +sont indépendantes. Sur la 780M, Hardware Profile classe conservativement comme +UMA le petit aperture VRAM amdgpu de 512 Mio accompagné d'environ 7,99 Go de +GTT système ; la capacité rapportée reste observable mais ne devient pas un +budget séparé. Les coûts GPU sont débités exactement une fois de la RAM hôte. + +L'audit Phase 1 couvre les 14 kinds de production et sépare leurs dimensions +fixes des dimensions réellement adaptables. Il confirme que tous passent par +l'unique Governor, même les formes fixes, et que le contrat reste immutable +pendant une séquence. Cette tranche ferme l'enveloppe privée, la sélection AUTO, +les diagnostics bornés, l'enforcement OpenCV adaptatif 1..12, la politique +d'affinité privée et la réconciliation du contexte retenu des campagnes +nouvelles ou restaurées. La création/reprise AUTO ne touche plus Vulkan sur le +main ; le premier begin appartient au worker contraint. Inflight ORB normal est +maintenant fixé à 1, helpers reste 0; depth 2 reste une capacité privée de +sûreté/benchmark. La clôture v2 est acquise. Ces choix ne créent aucune limite +de dataset, identité scientifique ou version Project DB. + +La signature durable des nouvelles Tasks ORB normales est maintenant la classe +`MIXED`, sémantiquement réelle pour une politique susceptible d'exécuter CPU ou +Vulkan. Elle reconstruit AUTO ; toutes les signatures CPU anciennes/courantes +restent CPU fixes et Vulkan reste fixe, sans migration DB/codec. La couture +asynchrone est privée, request-bound et nettoie son slot sur toute sortie. Une +preuve événementielle bornée établit la soumission du successeur avant la +publication du prédécesseur pour deux paires 769×769. La rampe ne croît plus sur +la seule santé : l'adaptation générique exige deux observations par fenêtre, +tandis que le lot ORB Vulkan en exige huit, avec retour au palier accepté sans +gain et reset immédiat sous pression. Ces +éléments sont **PASS / FROZEN** dans les limites validées. + +Les diagnostics de séquence distinguent maintenant backend sélectionné et +backend réel ; les participants CPU Matcher restent `cpu_threads` et +`helpers=0`. Une paire Vulkan inéligible ou en panne est recalculée entièrement +sur CPU sans preuve partielle. Les extractions ORB/SIFT/RootSIFT testées à +1/2/4/8/12 consomment leur contrat OpenCV immutable avec sorties égales. +Le rolling distingue désormais un handle soumis d'une inéligibilité locale et +n'appelle jamais `finish` sans requête. La panne backend invalide immédiatement +l'admission partagée sur toute sortie précoce ; seules les reprises AUTO peuvent +établir cette disponibilité, indépendamment de l'ordre des reprises fixes ou +historiques. Le statut est **PASS / FROZEN**. + +La boucle privée mesure désormais l'utilisation `/proc/stat` du compute-pool, +`MemAvailable`, PSI mémoire/I/O `some/full`, deltas swap actifs, RSS/HWM observé +et GPU busy DRM, avec `unknown` sur absence ou parse non strict. Le backend +Vulkan fournit des compteurs cumulatifs bornés de submit/complétion/fence/ +readback/GPU/starvation/panne/discard ; Matcher agrège en plus CPU et publication +par séquence. Le diagnostic est tirable par numéro de série, sans log ncurses ni +histoire persistée. Les CPU réductibles progressent `1/2/4/8/12` après deux +observations de baseline et deux gains d'au moins 5 % ; CPU et lot ne changent +jamais dans le même essai. Sous pression, une admission adaptative encore +permise réserve immédiatement CPU1 et lot minimum. Feature/SIFT/RootSIFT ne +comptent un item qu'après extraction et publication durable propre ; READY, +`ALREADY_PRESENT` et publication incertaine comptent zéro, comme un segment +Visual Index non durable. +Candidate mesure chaque séquence. Un +fallback réel annule l'essai Vulkan au lieu d'empoisonner sa baseline. Cette +implémentation est **PASS / FROZEN** sans ajouter de helper GPU. + +L'évidence retenue avant cette implémentation compare le contrôle synchrone à +49,989 paires/s et rolling depth 1 à 55,124 paires/s (+10,27 %), avec le même +digest `7a9dbc38a23a600379167d55e24836b7acbb22eea25573e7440bdc9e4602b3b3`. +La starvation depth 1 (53,847 s sur 74,613 s, GPU busy max 25 %) motive deux +slots bornés mais ne constitue pas une mesure depth 2. Le backend partage +device/pipeline/layout/cache et duplique seulement 640 Kio de payload, +command/fence/descriptors/query par slot. Le payload mappé suit désormais +exactement la capacité de séquence admise : zéro avant initialisation, 640 Kio +à depth 1 et 1,25 Mio uniquement pendant un contrôle privé depth 2, avec retour +à 640 Kio avant la prochaine admission depth 1. L'enveloppe normale n'essaie +plus inflight 2. Les générations de requête ne bouclent pas; un slot épuisé est +retiré définitivement. + +Le harness réel a exécuté le corpus 4113 paires pour le contrôle synchrone, +rolling depth 1 et l'A/B forcé depth 1/depth 2. L'ABBA forcé donne +54,661652238 et 55,797311953 paires/s (+2,077617 %, sous le deadband 5 %). +Chaque run porte 4113 paires durables; le débit combiné est +`(2 * 4113 * 1e9) / somme(wall_ns)`, pas la moyenne des débits par run. Les +walls bruts 75326831673/75162582080 et 73662096698/73764360098 ns donnent les +moyennes 75,244706877/73,713228398 s. Fence vaut 6,0684/3,6776 s, starvation 54,4534/50,1465 s, +publication 29,2582/30,0548 s, submit CPU 0,2655/0,3818 s, readback +0,0460/0,0873 s et GPU busy max 23/24 %. Les quatre exécutions conservent le +digest `7a9dbc38a23a600379167d55e24836b7acbb22eea25573e7440bdc9e4602b3b3`, quatre +séquences de fallback local par exécution et zéro panne/discard. Depth 2 est donc +**REJECTED_WITH_MEASURED_REASON** pour la politique normale : +`DEPTH_MAX_VALIDATED_SAFETY=2`, `DEPTH_MAX_USEFUL=1`. Une comparaison +whole-corpus adaptative antérieure (+1,12 %) ne séparait pas les changements de +contrat; l'ABBA ci-dessus la remplace comme décision de profondeur. Le +répertoire retenu est +`/home/fy59/Documents/Lardon/.real-pre-sfm-2026-08-30/governor-v2-evidence/`, +avec `forced-depth1-a.stdout.jsonl`, `forced-depth1-b.stdout.jsonl`, +`forced-depth2-a.stdout.jsonl` et `forced-depth2-b.stdout.jsonl`. Le +mode Matcher par défaut est AUTO/rolling ; CPU et Vulkan explicites restent des +overrides. Un contrôle synchrone fence par fence est compilé seulement dans le +runner/test, absent de `lardon3d`, non persisté et non applicable à une reprise +Matcher pendante. Les fixtures et ces deux exécutions établissent l'égalité +exacte des sorties; les chiffres depth 1 sont rapportés ci-dessus. Le runner émet +des diagnostics JSON échantillonnés sans prétendre voir chaque séquence, plus +un agrégat Governor fixe exact pour les compteurs enregistrés. Il valide la +bijection Candidate Pair/Match Result, les assets et le curseur, puis produit le +digest canonique de comparaison `L3DMRD1`, qui exclut IDs Task, timestamps et +choix opérationnels. Ces preuves ferment **PASS / FROZEN** sans promouvoir +depth 2 dans AUTO normal. + +Le runner possède maintenant, dans ce seul target, `--matcher-inflight 1|2` : +AUTO rolling fixe batch 2 et min=max inflight pour permettre la mesure du même +binaire à profondeur 1 puis 2, en n'exposant que la capacité Vulkan forcée. +`--matcher-batch 2|4|8|12`, valable seulement avec inflight et AUTO rolling, +fixe aussi min=max batch pour la prochaine matrice contrôlée. +Synchronous reste depth 1. Le Governor conserve son admission et sa charge UMA; +GPU budget zéro ou capacité GPU/backend/mémoire indisponible échoue au lieu de +sélectionner CPU. La valeur n'est ni persistée ni scientifique, +est refusée sur une Task pendante, restaurée dans l'environnement à toute sortie +et émise comme `1`, `2` ou `null` dans les résumés/agrégats. Le runner valide le +contrat exact et zéro panne/discard/pending; seule l'inéligibilité locale peut +produire un fallback CPU complet. La production compte désormais chaque item +exactement une fois après publication durable dans local-ineligible, +backend-failure ou other; une admission CPU normale reste à zéro. Le compteur +est incrémenté immédiatement et reste visible si une paire suivante échoue ou est +annulée, sans valider ni entraîner la séquence incomplète; son high-water mark +de déduplication reste privé et non persisté. L'égalité +inter-cohortes porte sur les items locaux, pas sur les séquences dont le nombre +dépend du batch. Une panne/raison autre, ou une panne tardive, invalide la +cohorte; la Task de benchmark échoue après checkpoint de toute preuve CPU déjà +durable. Les fixtures +établissent l'égalité exacte rolling1/rolling2/synchrone1 et des batches +2/4/8/12 sur fixture, avec le même compte de 29 items locaux malgré des comptes +de séquences différents. Les logs corpus préliminaires +`forced-batch2-current.stdout.jsonl` et `forced-batch4.stdout.jsonl` couvrent +les mêmes 4113 IDs/digest mais portent quatre contre trois séquences locales : +ils sont conservés comme preuve que l'ancien comparateur était invalide et ne +participent pas à la décision. Les huit runs item-valides +`forced-batch{2,4,8,12}-items{,-b}.stdout.jsonl` publient chacun 4113 paires, +six items locaux, zéro panne/autre et le même digest. Leurs débits combinés +sont 54,180767704, 66,094373197, 74,784998723 et 76,755814095 paires/s. Batch +4 puis 8 gagnent +21,988624373 % et +13,148812987 %; batch 12 ne gagne que ++2,635308425 %, sous le deadband 5 %. AUTO normal suit donc +`BATCH_MAX_USEFUL=8`; batch 12 reste une capacité privée sûre +`REJECTED_WITH_MEASURED_REASON`. +Le contrôle de production sans override +`short-auto-batch8-governor-v2.stdout.jsonl` atteint réellement +`1 → 2 → 4 → 8`, puis publie 4113/4113 résultats à 76,072 paires/s avec le +même digest, six fallbacks locaux, zéro panne/discard, inflight 1 et helpers 0. +La preuve de fermeture S21 +`final-s21-auto.stdout.jsonl` exécute ensuite le chemin production normal sur +172 741 Candidate Pairs : 172 741 Match Results, zéro doublon, curseur complet, +digest `e5128a2e599ff593c4f79850e067254b1f249d19e8480a44973306b1af250f70` +et 73,649 résultats durables/s. AUTO choisit Vulkan sur toutes les admissions, +termine batch 8/inflight 1/helpers 0 et ne compte aucune panne/discard/pending. +Une admission YELLOW réduit batch 8 à 1, puis les séquences GREEN rétablissent +1 → 2 → 4 → 8; le gate possède donc aussi une preuve réelle de recovery. + +Les gates de fermeture sont acquis : + +```text +GOVERNOR_CONTROLS_ALL_TASKS=PASS +HOST_CPU_RESERVE=PASS +HOST_RAM_RESERVE=PASS +REAL_TIME_TELEMETRY=PASS +SLOW_START=PASS +HYSTERESIS=PASS +PRESSURE_THROTTLE=PASS +PRESSURE_RECOVERY=PASS +GPU_FIRST_AUTO=PASS +ORB_VULKAN_TRUE_ASYNC=PASS +TRUE_GPU_CPU_OVERLAP=PASS +VULKAN_RESOURCE_BOUNDS=PASS +UMA_ACCOUNTING=PASS +SCIENTIFIC_EQUIVALENCE=PASS +RESTART_IDEMPOTENCE=PASS +FINAL_FULL_S21_AUTO_RUN=PASS +FULL_NORMAL_SUITE=PASS +C17=PASS +ASAN_UBSAN=PASS +GIT_DIFF_CHECK=PASS +FINAL_XHIGH_REVIEW=PASS +DOC_CONSISTENCY=PASS +MATCHER_GPU=EXISTING_BACKEND_VALIDATED_AND_PREFERRED +COMPUTE_GOVERNOR_V2=PASS/FROZEN +ORB_VULKAN_ASYNC_EXECUTION=PASS/FROZEN +``` + +L'expérience normale est donc : l'utilisateur lance une Task; l'unique +Governor choisit et explique le contrat borné de sa prochaine séquence. Aucun +réglage CPU/GPU/lot/inflight/helper n'est requis en production ordinaire. L'ordre de travail autorisé à court terme est : @@ -245,7 +458,7 @@ fondation scientifique pré-SfM courante → INTERNAL PARALLELISM + COMPUTE RESOURCES v1 (Candidate, Matcher, Visual Index, audit feature threading, correction progression/Resource Governor, audit GPU) -→ GPU COMPUTE v1 optionnel, seulement si audit et équivalence le justifient +→ COMPUTE GOVERNOR v2 / ORB VULKAN ASYNC EXECUTION (PASS / FROZEN) → poursuite pré-SfM réelle complète de S21 → acquisition dédiée de calibration → Sparse SfM réel @@ -561,7 +774,8 @@ L'ordre demeure sans ambiguïté : CURRENT NEXT fondation scientifique pré-SfM courante → INTERNAL PARALLELISM + COMPUTE RESOURCES v1 - → GPU COMPUTE v1 optionnel si audit/équivalence le justifient + → COMPUTE GOVERNOR v2 / ORB VULKAN ASYNC EXECUTION + (PASS / FROZEN; AUTO GPU-first ORB et preuve S21 Matcher complète) → poursuite pré-SfM réelle complète de S21 → acquisition physique dédiée de calibration → calibration connue validée → Sparse SfM réel multi-campagne diff --git a/include/lardon3d/matcher_task.h b/include/lardon3d/matcher_task.h index 9ba0065..3f3ec43 100644 --- a/include/lardon3d/matcher_task.h +++ b/include/lardon3d/matcher_task.h @@ -13,7 +13,9 @@ enum { LARDON3D_MATCHER_TASK_KIND_VERSION = 1, LARDON3D_MATCHER_TASK_MINIMUM_BATCH = 1, - LARDON3D_MATCHER_TASK_MAXIMUM_BATCH = 8, + /* This is an operational window and participant ceiling, not a scientific + * dataset limit. Every staged pair remains private until ordered publish. */ + LARDON3D_MATCHER_TASK_MAXIMUM_BATCH = 12, }; typedef struct { @@ -23,11 +25,12 @@ typedef struct { Lardon3DMatcherParams matcher; } Lardon3DMatcherTaskConfiguration; -/* Selects only the admitted execution resources and runtime backend. Matcher - * parameters, fingerprints, Match Results, and Match Files are identical - * across modes. CPU_PARALLEL is portable and is the default used by the - * existing create/enqueue APIs. ORB_VULKAN requires an ORB configuration, a - * selected GPU, and an available Vulkan backend when the task is created. */ +/* Selects only an explicit debug/benchmark/reproduction override. Normal + * create/enqueue is Governor-owned AUTO: validated ORB Vulkan is preferred + * when runtime support and admission are safe, otherwise the complete CPU + * implementation is selected. Matcher parameters, fingerprints, Match + * Results, and Match Files are identical across operational selections. + * SIFT/RootSIFT remain CPU-only. */ typedef enum { LARDON3D_MATCHER_TASK_MODE_CPU_PARALLEL = 0, LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN = 1, diff --git a/include/lardon3d/orb_vulkan_backend.h b/include/lardon3d/orb_vulkan_backend.h index 3fbf705..7a4a695 100644 --- a/include/lardon3d/orb_vulkan_backend.h +++ b/include/lardon3d/orb_vulkan_backend.h @@ -12,6 +12,11 @@ extern "C" { typedef struct Lardon3DOrbVulkanBackend Lardon3DOrbVulkanBackend; enum { + /* Stable minimum/depth-1 payload cost used by durable resource signatures: + * two 8192x32 descriptor buffers plus one 8192x4x32-bit top-2 buffer. The + * private rolling backend may own two such slots; backend_info reports the + * ORB request-slot payload actually retained at the observation instant + * without changing this public ABI. */ LARDON3D_ORB_VULKAN_PERMANENT_BUFFER_BYTES = 640 * 1024, }; @@ -52,24 +57,45 @@ typedef struct { uint64_t gpu_ns; } Lardon3DOrbVulkanInfo; +/* Create an uninitialized backend without probing Vulkan or changing process + * environment. The caller owns the returned object and must exclude every + * concurrent or future use before destroy; destroy accepts NULL. */ Lardon3DOrbVulkanBackend *lardon3d_orb_vulkan_backend_create(void); void lardon3d_orb_vulkan_backend_destroy(Lardon3DOrbVulkanBackend *backend); bool lardon3d_orb_vulkan_should_use(uint32_t feature_count_a, uint32_t feature_count_b); +/* Compute deterministic ORB/Hamming top-2 results. Nonzero counts are bounded + * to 8192; output_capacity covers feature_count_a. + * Empty A succeeds without output, and empty B writes zero-neighbor rows. + * + * In a Vulkan-enabled build, the first nonempty request may initialize Mesa. + * Before any process thread is created, the process owner must establish + * MESA_SHADER_CACHE_DISABLE to exact "true" or "1". The backend never mutates + * that environment. An absent, false, or malformed value makes the first + * initializing request return UNAVAILABLE, caches that state for this backend, + * and leaves output untouched. Invalid arguments return INVALID_ARGUMENT; + * initialization absence/failure returns UNAVAILABLE, while an initialized + * request/session failure returns FAILED. */ Lardon3DOrbVulkanResult lardon3d_orb_vulkan_top2( Lardon3DOrbVulkanBackend *backend, const unsigned char *descriptors_a, uint32_t feature_count_a, const unsigned char *descriptors_b, uint32_t feature_count_b, Lardon3DOrbTop2 *output, size_t output_capacity); #ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY +/* Feasibility-only SIFT top-2 obeys the same bounds, ownership, process-policy, + * UNAVAILABLE caching, and no-partial-output rules as ORB top-2. */ Lardon3DOrbVulkanResult lardon3d_sift_vulkan_top2( Lardon3DOrbVulkanBackend *backend, const float *descriptors_a, uint32_t feature_count_a, const float *descriptors_b, uint32_t feature_count_b, Lardon3DSiftTop2 *output, size_t output_capacity); #endif +/* Read current backend metadata without probing or initializing Vulkan and + * without requiring or changing MESA_SHADER_CACHE_DISABLE. Both pointers are + * required; the caller owns the output snapshot. A policy-rejected backend is + * initialized=true and available=false. */ bool lardon3d_orb_vulkan_backend_info(Lardon3DOrbVulkanBackend *backend, Lardon3DOrbVulkanInfo *info); diff --git a/meson.build b/meson.build index c18d974..9c135ea 100644 --- a/meson.build +++ b/meson.build @@ -129,6 +129,7 @@ if matcher_vulkan_enabled endif opencv_test_environment = environment() +opencv_test_environment.set('MESA_SHADER_CACHE_DISABLE', 'true') if get_option('b_sanitize').contains('thread') opencv_test_environment.set( 'TSAN_OPTIONS', @@ -136,7 +137,7 @@ if get_option('b_sanitize').contains('thread') ) endif -executable( +lardon3d_app = executable( 'lardon3d', sources: [ 'src/main.c', @@ -221,6 +222,15 @@ executable( + matcher_backend_dependencies, ) +# The helper directly execs the production binary and checks its exact early +# rejection; no shell or ncurses session participates in this boundary test. +test( + 'app-driver-policy-unsafe', + python, + args: [files('tests/test_driver_policy_startup.py'), lardon3d_app.full_path()], + timeout: 5, +) + import_test = executable( 'test-import', sources: [ @@ -489,10 +499,7 @@ photo_quality_restart_scan = executable( # Opt-in real-data evidence runner. It is intentionally not registered with # meson test: invocation requires explicit source roots and a fresh project # directory, and may perform bounded but long RAW/ORB/matching work. -pre_sfm_real_execution = executable( - 'pre-sfm-real-execution', - sources: [ - 'tests/pre_sfm_real_execution.cpp', +pre_sfm_real_execution_support_sources = [ 'src/app_state.c', 'src/acquisition_pairing.cpp', 'src/acquisition_ingest.cpp', 'src/acquisition_campaign.cpp', 'src/acquisition_campaign_task.cpp', @@ -512,19 +519,45 @@ pre_sfm_real_execution = executable( 'src/task.c', 'src/task_checkpoint.c', 'src/task_kind_registry.c', 'src/task_kinds.c', 'src/task_queue.c', 'src/resource_governor.c', 'src/resource_snapshot.c', 'src/hardware_profile.c', - ] + matcher_backend_sources, +] + matcher_backend_sources + +pre_sfm_real_execution = executable( + 'pre-sfm-real-execution', + sources: ['tests/pre_sfm_real_execution.cpp'] + + pre_sfm_real_execution_support_sources, include_directories: include_directories('include'), c_args: [ '-DLARDON3D_TRACK_BUILDER_TASK_AVAILABLE', '-DLARDON3D_ACQUISITION_CAMPAIGN_TASK_AVAILABLE', '-DLARDON3D_PHOTO_QUALITY_TASK_AVAILABLE', '-DLARDON3D_RAW_DEVELOPMENT_TASK_AVAILABLE', + '-DLARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE', ], + cpp_args: ['-DLARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE'], dependencies: [ threads, sqlite3, openssl, opencv, libraw, libexif, libpng, libdeflate, ] + matcher_backend_dependencies, ) +pre_sfm_runner_options_test = executable( + 'test-pre-sfm-runner-options', + sources: ['tests/test_pre_sfm_runner_options.cpp'] + + pre_sfm_real_execution_support_sources, + include_directories: include_directories('include'), + c_args: [ + '-DLARDON3D_TRACK_BUILDER_TASK_AVAILABLE', + '-DLARDON3D_ACQUISITION_CAMPAIGN_TASK_AVAILABLE', + '-DLARDON3D_PHOTO_QUALITY_TASK_AVAILABLE', + '-DLARDON3D_RAW_DEVELOPMENT_TASK_AVAILABLE', + '-DLARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE', + ], + cpp_args: ['-DLARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE'], + dependencies: [ + threads, sqlite3, openssl, opencv, libraw, libexif, libpng, libdeflate, + ] + matcher_backend_dependencies, +) +test('pre-sfm-runner-options', pre_sfm_runner_options_test, timeout: 30) + feature_store_test = executable( 'test-feature-store', sources: [ @@ -627,9 +660,7 @@ candidate_pair_task_test = executable( test('candidate-pair-task', candidate_pair_task_test, timeout: 60, env: opencv_test_environment) -matcher_task_test = executable( - 'test-matcher-task', - sources: [ +matcher_task_test_sources = [ 'tests/test_matcher_task.c', 'src/app_state.c', 'src/project.c', 'src/project_db.c', 'src/project_db_sparse_sfm.c', 'src/task.c', 'src/task_checkpoint.c', 'src/task_kind_registry.c', 'src/task_kinds.c', 'src/task_queue.c', @@ -643,7 +674,11 @@ matcher_task_test = executable( 'src/feature_extractor_opencv.cpp', 'src/visual_index.c', 'src/visual_index_task.c', 'src/candidate_pair_gen.c', 'src/candidate_pair_task.c', - ] + matcher_backend_sources, +] + matcher_backend_sources + +matcher_task_test = executable( + 'test-matcher-task', + sources: matcher_task_test_sources, c_args: [ '-DLARDON3D_PROJECT_DB_TESTING', '-DLARDON3D_MATCHER_TASK_TESTING', @@ -653,6 +688,27 @@ matcher_task_test = executable( ) test('matcher-task', matcher_task_test, timeout: 60, env: opencv_test_environment) +# This separately named evidence test is the only test-side consumer of the +# synchronous-fence control. Ordinary Matcher tests compile the same production +# sources without the macro, matching the production executable boundary. +matcher_task_benchmark_pipeline_test = executable( + 'test-matcher-task-benchmark-pipeline', + sources: matcher_task_test_sources, + c_args: [ + '-DLARDON3D_PROJECT_DB_TESTING', + '-DLARDON3D_MATCHER_TASK_TESTING', + '-DLARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE', + ] + (matcher_vulkan_enabled ? ['-DLARDON3D_MATCHER_TASK_VULKAN'] : []), + include_directories: include_directories('include'), + dependencies: [threads, sqlite3, openssl, opencv] + matcher_backend_dependencies, +) +test( + 'matcher-task-benchmark-pipeline', + matcher_task_benchmark_pipeline_test, + timeout: 60, + env: opencv_test_environment, +) + feature_task_test = executable( 'test-feature-task', sources: [ @@ -672,8 +728,11 @@ feature_task_test = executable( ] + matcher_backend_sources, c_args: [ '-DLARDON3D_PROJECT_DB_TESTING', '-DLARDON3D_FEATURE_TASK_TESTING', + '-DLARDON3D_FEATURE_STORE_TESTING', '-DLARDON3D_VISUAL_INDEX_TASK_TESTING', + '-DLARDON3D_VISUAL_INDEX_TESTING', ], + cpp_args: ['-DLARDON3D_FEATURE_TASK_TESTING'], include_directories: include_directories('include'), dependencies: [threads, sqlite3, openssl, opencv] + matcher_backend_dependencies, ) @@ -737,6 +796,7 @@ task_test = executable( 'src/resource_governor.c', 'src/resource_snapshot.c', ], + c_args: ['-DLARDON3D_TASK_TESTING'], include_directories: include_directories('include'), dependencies: [threads], ) @@ -1060,7 +1120,7 @@ matcher_test = executable( dependencies: [threads, sqlite3, openssl, opencv] + matcher_backend_dependencies, ) -test('matcher', matcher_test, timeout: 60) +test('matcher', matcher_test, timeout: 60, env: opencv_test_environment) if matcher_vulkan_enabled orb_vulkan_backend_test = executable( @@ -1130,7 +1190,8 @@ matcher_e2e_test = executable( dependencies: [threads, sqlite3, openssl, opencv] + matcher_backend_dependencies, ) -test('matcher-e2e', matcher_e2e_test, timeout: 60) +test('matcher-e2e', matcher_e2e_test, timeout: 60, + env: opencv_test_environment) executable( 'benchmark-matcher', diff --git a/src/acquisition_campaign_task.cpp b/src/acquisition_campaign_task.cpp index 5017ad8..d2361f1 100644 --- a/src/acquisition_campaign_task.cpp +++ b/src/acquisition_campaign_task.cpp @@ -3,6 +3,7 @@ extern "C" { #include #include #include +#include "task_internal.h" } #include @@ -184,6 +185,30 @@ struct Context { Lardon3DAcquisitionIngestOptions options{}; }; +bool context_owned_bytes(const Context &context, uint64_t &bytes) { + uint64_t total = sizeof(Context); + const size_t source_count = context.sources.capacity(); + const size_t confirmation_count = context.confirmations.capacity(); + if (source_count > UINT64_MAX / sizeof(Lardon3DAcquisitionCampaignSource) || + confirmation_count > + UINT64_MAX / sizeof(Lardon3DAcquisitionCampaignConfirmation)) + return false; + const uint64_t retained[] = { + static_cast(source_count) * + sizeof(Lardon3DAcquisitionCampaignSource), + static_cast(confirmation_count) * + sizeof(Lardon3DAcquisitionCampaignConfirmation), + static_cast(context.encoded.capacity()), + }; + for (uint64_t amount : retained) { + if (amount > UINT64_MAX - total) + return false; + total += amount; + } + bytes = total; + return true; +} + void destroy(void *p) { delete static_cast(p); } void runtime(Context *c, Lardon3DAppState &s) { lardon3d_app_state_init(&s); @@ -335,6 +360,32 @@ Context *make_context(const char *path, Lardon3DProjectDb *db, } } // namespace +extern "C" bool +lardon3d_acquisition_campaign_task_internal_configure_restored( + Lardon3DTask *task, void *userdata) { + try { + auto *context = static_cast(userdata); + uint64_t retained = 0; + if (!task || !context || !context_owned_bytes(*context, retained) || + retained > UINT64_MAX - 256 * 1024) + return false; + Lardon3DTaskCapability capability{}; + capability.estimate = Lardon3DResourceEstimate{ + retained + 256 * 1024, 0, 64 * 1024, 0, 1, 1, 1, 0, 1, + LARDON3D_RESOURCE_TASK_IMPORT}; + capability.backend = LARDON3D_RESOURCE_BACKEND_FIXED; + capability.inflight_limit = 1; + Lardon3DTaskCapabilityEnvelope envelope{}; + envelope.count = 1; + envelope.capabilities[0] = capability; + /* Recovery may carry the historical under-estimate, but admission uses the + * exact newly reconstructed retained context. Durable bytes stay untouched. */ + return lardon3d_task_internal_set_capability_envelope(task, &envelope); + } catch (...) { + return false; + } +} + bool request_encode_impl( const Lardon3DAcquisitionCampaignTaskRequest *r, unsigned char *out, size_t cap, size_t *size) { @@ -498,8 +549,16 @@ Lardon3DTask *create_task_impl( delete c; return nullptr; } + uint64_t retained = 0; + if (!context_owned_bytes(*c, retained) || retained > UINT64_MAX - 256 * 1024) { + delete c; + return nullptr; + } + /* The request vectors and inline campaign plan remain live for the Task + * lifetime. Charge them before admission; the per-group working estimate is + * separate and this operational bound does not limit scientific cardinality. */ Lardon3DResourceEstimate e{ - 256 * 1024, 0, 64 * 1024, 0, 1, + retained + 256 * 1024, 0, 64 * 1024, 0, 1, 1, 1, 0, 1, LARDON3D_RESOURCE_TASK_IMPORT}; auto *t = lardon3d_task_create_typed("Campagne d'acquisition", &e, LARDON3D_ACQUISITION_CAMPAIGN_TASK_KIND, diff --git a/src/app.c b/src/app.c index 5d417df..2afd6e1 100644 --- a/src/app.c +++ b/src/app.c @@ -1,4 +1,5 @@ #include +#include #include #include @@ -12,9 +13,22 @@ #include #include +#include "resource_governor_internal.h" + int lardon3d_app_run(void) { + /* CONTRACT: establish the process-wide driver policy before Queue, OpenCV, + * ncurses, or any other application pthread can exist. An explicit unsafe + * Mesa cache value is rejected instead of silently overwritten. */ + Lardon3DResourceDriverPolicyResult driver_policy = + lardon3d_resource_governor_internal_configure_driver_policy(); + if (driver_policy == LARDON3D_RESOURCE_DRIVER_POLICY_FAILED + || driver_policy == LARDON3D_RESOURCE_DRIVER_POLICY_REJECTED_UNSAFE) { + (void)fprintf(stderr, + "MESA_SHADER_CACHE_DISABLE must be true for safe CPU affinity\n"); + return EXIT_FAILURE; + } Lardon3DAppState state; lardon3d_app_state_init(&state); @@ -39,10 +53,11 @@ lardon3d_app_run(void) if (feature_threads > 12) { feature_threads = 12; } - /* OpenCV owns an internal process-wide pool. Configure it once, before - * Queue workers exist, to the empirically audited part of the host-reserved - * CPU budget. Feature Tasks request the same count from the Governor. This - * operational ceiling is not part of a fingerprint or FeatureSet identity. */ + /* OpenCV owns one process-wide CPU setting. Startup establishes the safe + * audited baseline/ceiling before Queue exists; the sole Queue callback may + * temporarily apply its immutable admitted count and must restore this + * baseline on every path. Concurrent multi-worker mutation is unsupported. + * This operational count is never FeatureSet identity or fingerprint. */ if (!lardon3d_feature_opencv_configure_threads(feature_threads)) { return EXIT_FAILURE; } diff --git a/src/candidate_pair_task.c b/src/candidate_pair_task.c index b9043e7..ad871ff 100644 --- a/src/candidate_pair_task.c +++ b/src/candidate_pair_task.c @@ -14,6 +14,7 @@ #include #include "candidate_pair_gen_internal.h" +#include "task_internal.h" enum { CANDIDATE_PAIR_MINIMUM_BATCH = 1, @@ -339,6 +340,8 @@ static bool run(Lardon3DTask *task, void *userdata) { (void)lardon3d_resource_governor_record_batch( context->governor, LARDON3D_RESOURCE_TASK_CPU, (size_t)seq_generated, elapsed_ns(begin, end), 0); + (void)lardon3d_task_internal_record_sequence( + task, elapsed_ns(begin, end), processed_in_sequence); Lardon3DAppState state; runtime_state(context, &state); diff --git a/src/feature_extractor_opencv.cpp b/src/feature_extractor_opencv.cpp index fa32427..463ac6e 100644 --- a/src/feature_extractor_opencv.cpp +++ b/src/feature_extractor_opencv.cpp @@ -20,13 +20,36 @@ extern "C" bool lardon3d_feature_opencv_configure_threads(unsigned int threads) if (threads == 0 || threads > static_cast(std::numeric_limits::max())) { return false; } - cv::setNumThreads(static_cast(threads)); - return cv::getNumThreads() == static_cast(threads); + try { + cv::setNumThreads(static_cast(threads)); +#ifdef LARDON3D_FEATURE_TASK_TESTING + const char *forced = std::getenv( + "LARDON3D_TEST_OPENCV_CONFIGURE_FAILURE_THREADS"); + if (forced) { + char *end = nullptr; + unsigned long parsed = std::strtoul(forced, &end, 10); + if (end && *end == '\0' && parsed == threads) { + /* Inject after mutation so Task cleanup must restore the captured + * process-wide value rather than treating failure as side-effect-free. */ + return false; + } + } +#endif + return cv::getNumThreads() == static_cast(threads); + } catch (...) { + /* This process-wide operational control is a C ABI seam. OpenCV failures + * are reported to the Task owner and no C++ exception may escape. */ + return false; + } } extern "C" unsigned int lardon3d_feature_opencv_thread_count(void) { - int threads = cv::getNumThreads(); - return threads > 0 ? static_cast(threads) : 1U; + try { + int threads = cv::getNumThreads(); + return threads > 0 ? static_cast(threads) : 1U; + } catch (...) { + return 1U; + } } extern "C" bool diff --git a/src/feature_store.c b/src/feature_store.c index d99931e..65d7c5d 100644 --- a/src/feature_store.c +++ b/src/feature_store.c @@ -602,6 +602,12 @@ Lardon3DFeatureStoreResult lardon3d_feature_store_publish_v2( } unlink(temporary); bool durable = sync_directory(prefix); +#ifdef LARDON3D_FEATURE_STORE_TESTING + const char *fail_sync = getenv("LARDON3D_TEST_FEATURE_FAIL_DIRECTORY_SYNC"); + if (fail_sync && strcmp(fail_sync, "1") == 0) { + durable = false; + } +#endif return register_published_feature_set( state, image_id, producer_task_id, &image_asset, kind, version, fingerprint, descriptor_type, descriptor_dimension, features, file_hash, relative, file_size, durable, feature_set); diff --git a/src/feature_task.c b/src/feature_task.c index 8747ec4..7adc76a 100644 --- a/src/feature_task.c +++ b/src/feature_task.c @@ -15,6 +15,9 @@ #include #include +#include "opencv_task_thread_control.h" +#include "task_internal.h" + typedef struct { char project_path[PATH_MAX]; Lardon3DProjectDb *database; @@ -125,13 +128,14 @@ static bool load_validated_source(const Lardon3DFeatureTaskContext *context, cha return file_hash(path, actual) && memcmp(actual, asset.sha256, 32) == 0; } -static bool run(Lardon3DTask *task, void *userdata) { +static bool run_body(Lardon3DTask *task, void *userdata, size_t *durable_items) { Lardon3DFeatureTaskContext *context = userdata; + *durable_items = 0; if (!lardon3d_task_checkpoint(task)) { return false; } - struct timespec begin; - (void)clock_gettime(CLOCK_MONOTONIC, &begin); + struct timespec begin = {0}; + bool timing_known = clock_gettime(CLOCK_MONOTONIC, &begin) == 0; unsigned char fingerprint[32]; lardon3d_feature_extractor_parameter_fingerprint( &(Lardon3DFeatureExtractorParameters){context->parameters.max_features, @@ -186,13 +190,47 @@ static bool run(Lardon3DTask *task, void *userdata) { published != LARDON3D_FEATURE_STORE_PUBLISHED_NOT_DURABLE) { return lardon3d_task_fail(task, "Publication Feature Store impossible."); } - struct timespec end; - (void)clock_gettime(CLOCK_MONOTONIC, &end); - (void)lardon3d_resource_governor_record_batch(context->governor, LARDON3D_RESOURCE_TASK_CPU, 1, - elapsed_ns(begin, end), 0); + *durable_items = published == LARDON3D_FEATURE_STORE_OK ? 1 : 0; + struct timespec end = {0}; + timing_known = timing_known && clock_gettime(CLOCK_MONOTONIC, &end) == 0; + if (*durable_items > 0 && timing_known) { + (void)lardon3d_resource_governor_record_batch( + context->governor, LARDON3D_RESOURCE_TASK_CPU, *durable_items, + elapsed_ns(begin, end), 0); + } return lardon3d_task_set_progress(task, 100, "Feature Set publié."); } +static bool run(Lardon3DTask *task, void *userdata) { + struct timespec begin = {0}; + struct timespec end = {0}; + bool timing_known = clock_gettime(CLOCK_MONOTONIC, &begin) == 0; + Lardon3DOpenCvTaskThreadControl threads; + if (!lardon3d_opencv_task_threads_begin(task, 12, &threads)) { + return lardon3d_task_fail(task, "Contrat CPU OpenCV Features invalide."); + } + /* Queue has one callback owner, so this process-wide OpenCV setting cannot + * race another Task. The immutable admission selects 1..12, and this guard + * applies exactly that count before extraction and restores it on every + * return without changing Feature identity. */ + size_t durable_items = 0; + bool result = run_body(task, userdata, &durable_items); + if (!lardon3d_opencv_task_threads_end(&threads)) { + (void)lardon3d_task_fail(task, "Restauration OpenCV Features impossible."); + return false; + } + timing_known = timing_known && clock_gettime(CLOCK_MONOTONIC, &end) == 0; + if (result && timing_known) { + /* Only extraction followed by this Task's durable publication contributes + * work. READY reuse, an ALREADY_PRESENT collision, and uncertain directory + * sync remain successful outcomes, but record zero so feedback can neither + * train nor advance the next immutable CPU trial. */ + (void)lardon3d_task_internal_record_sequence( + task, elapsed_ns(begin, end), durable_items); + } + return result; +} + static Lardon3DFeatureTaskContext * make_context(const Lardon3DTaskReconstructionContext *runtime, const Lardon3DProjectDbFeatureExtractTask *parameters) { @@ -293,11 +331,8 @@ lardon3d_project_create_feature_extract_task(Lardon3DAppState *state, uint64_t i .memory_bytes_per_item = 512ULL * 1024 * 1024, .minimum_batch_size = 1, .maximum_batch_size = 1, - /* Request the audited operational ceiling, not OpenCV's - * mutable current value: Matcher temporarily sets that - * process-wide value to one inside its Queue callback. The - * Governor reduces twelve to the host-reserved CPU budget, - * matching the stable startup configuration. */ + /* Canonical durable maximum. Governor admission may select + * any validated OpenCV count in 1..12 for this execution. */ .desired_cpu_threads = 12, .desired_io_slots = 1, .task_class = LARDON3D_RESOURCE_TASK_CPU}; diff --git a/src/hardware_profile.c b/src/hardware_profile.c index ad8765d..86ea67a 100644 --- a/src/hardware_profile.c +++ b/src/hardware_profile.c @@ -26,22 +26,38 @@ read_text(const char *path, char *text, size_t capacity) if (!text || capacity < 2) { return false; } - int descriptor = open(path, O_RDONLY | O_CLOEXEC); + int descriptor = open(path, O_RDONLY | O_CLOEXEC | O_NOFOLLOW); if (descriptor < 0) { return false; } - ssize_t count; - do { - count = read(descriptor, text, capacity - 1); - } while (count < 0 && errno == EINTR); - bool success = count >= 0 && close(descriptor) == 0; + size_t total = 0; + bool success = true; + while (total + 1 < capacity) { + ssize_t count = read(descriptor, text + total, capacity - total - 1); + if (count < 0 && errno == EINTR) continue; + if (count < 0) { + success = false; + break; + } + if (count == 0) break; + total += (size_t)count; + } + if (success && total + 1 == capacity) { + char extra; + ssize_t count; + do { + count = read(descriptor, &extra, 1); + } while (count < 0 && errno == EINTR); + success = count == 0; + } + if (close(descriptor) != 0) success = false; if (!success) { return false; } - text[(size_t)count] = '\0'; - while (count > 0 && (text[(size_t)count - 1] == '\n' - || text[(size_t)count - 1] == '\r')) { - text[--count] = '\0'; + text[total] = '\0'; + while (total > 0 && (text[total - 1] == '\n' + || text[total - 1] == '\r')) { + text[--total] = '\0'; } return true; } @@ -49,25 +65,51 @@ read_text(const char *path, char *text, size_t capacity) static bool parse_uint64(const char *text, uint64_t *value) { - if (!text || !text[0] || !value || text[0] == '-') { + if (!text || !text[0] || !value) { return false; } - errno = 0; - char *end; - unsigned long long parsed = strtoull(text, &end, 0); - if (errno != 0 || end == text) { - return false; + const unsigned char *cursor = (const unsigned char *)text; + uint64_t parsed = 0; + size_t digits = 0; + while (*cursor >= '0' && *cursor <= '9') { + uint64_t digit = (uint64_t)(*cursor - '0'); + if (parsed > (UINT64_MAX - digit) / 10U) return false; + parsed = parsed * 10U + digit; + ++cursor; + ++digits; } - while (*end == ' ' || *end == '\t' || *end == '\n' || *end == '\r') { - ++end; + if (digits == 0) return false; + while (*cursor == ' ' || *cursor == '\t' || *cursor == '\n' + || *cursor == '\r' || *cursor == '\f' || *cursor == '\v') { + ++cursor; } - if (*end) { - return false; - } - *value = (uint64_t)parsed; + if (*cursor) return false; + *value = parsed; return true; } +static bool +shared_memory_evidence( + const Lardon3DHardwareProfile *profile, + uint64_t vram_total, + bool gtt_known, + uint64_t gtt_total +) +{ + if (profile->memory_total_bytes == 0) return false; + const uint64_t two_gibibytes = UINT64_C(2) * 1024 * 1024 * 1024; + bool conservatively_small = vram_total <= two_gibibytes + && vram_total <= profile->memory_total_bytes / 8; + bool system_scale_gtt = gtt_known + && gtt_total >= profile->memory_total_bytes / 4 + && vram_total <= gtt_total / 2; + /* WHY: amdgpu exposes a small stolen/dedicated VRAM aperture even for an + * integrated GPU. Treating that positive number as separate free memory + * undercharges host RAM. A low-VRAM uncertain device may conservatively + * become UMA; the reverse error could violate the 3 GiB/2 GiB host floors. */ + return conservatively_small || system_scale_gtt; +} + static const char * vendor_name(const char *vendor) { @@ -92,9 +134,16 @@ lardon3d_hardware_profile_detect_gpu_at_root( if (!profile || !drm_root) { return; } + profile->gpu_available = false; + profile->gpu_drm_card_index = 0; + profile->gpu_memory_known = false; + profile->gpu_uses_shared_memory = false; + profile->gpu_memory_total_bytes = 0; + profile->gpu_name[0] = '\0'; for (unsigned int index = 0; index < 64; ++index) { char vendor_path[PATH_MAX]; char memory_path[PATH_MAX]; + char gtt_path[PATH_MAX]; int vendor_written = snprintf( vendor_path, sizeof(vendor_path), @@ -109,9 +158,17 @@ lardon3d_hardware_profile_detect_gpu_at_root( drm_root, index ); + int gtt_written = snprintf( + gtt_path, + sizeof(gtt_path), + "%s/card%u/device/mem_info_gtt_total", + drm_root, + index + ); if (vendor_written < 0 || (size_t)vendor_written >= sizeof(vendor_path) || memory_written < 0 - || (size_t)memory_written >= sizeof(memory_path)) { + || (size_t)memory_written >= sizeof(memory_path) + || gtt_written < 0 || (size_t)gtt_written >= sizeof(gtt_path)) { continue; } char vendor[32]; @@ -132,7 +189,16 @@ lardon3d_hardware_profile_detect_gpu_at_root( && parse_uint64(memory, &bytes) && bytes > 0) { profile->gpu_memory_known = true; profile->gpu_memory_total_bytes = bytes; + char gtt[64]; + uint64_t gtt_bytes = 0; + bool gtt_known = read_text(gtt_path, gtt, sizeof(gtt)) + && parse_uint64(gtt, >t_bytes) && gtt_bytes > 0; + profile->gpu_uses_shared_memory = shared_memory_evidence( + profile, bytes, gtt_known, gtt_bytes); } else { + /* Unknown payload capacity must never be treated as an independent + * VRAM budget. Governor can still use the GPU conservatively while + * charging every admitted byte to MemAvailable. */ profile->gpu_uses_shared_memory = true; } return; diff --git a/src/matcher.cpp b/src/matcher.cpp index fba5e1f..f7a116d 100644 --- a/src/matcher.cpp +++ b/src/matcher.cpp @@ -2,6 +2,7 @@ #include #include #include +#include #include #include @@ -20,11 +21,49 @@ extern "C" { #include #include "matcher_internal.h" +#include "orb_vulkan_backend_internal.h" } static const char matcher_orb_str[] = "orb_bf"; static const char matcher_sift_str[] = "sift_bf"; static const char matcher_rootsift_str[] = "rootsift_bf"; +/* The pending object owns only unpublished operational state. GPU submission + * has occurred before it escapes begin; the Queue owner later consumes it into + * the canonical staged Match File or discards it without DB mutation. */ +struct Lardon3DMatcherPendingVulkanStage { + Lardon3DOrbVulkanBackend *backend = nullptr; + Lardon3DProjectDbFeatureSet a{}; + Lardon3DProjectDbFeatureSet b{}; + Lardon3DMatcherParams params{}; + char project_path[4096]{}; + std::vector top2; + Lardon3DMatcherStats stats{}; + Lardon3DOrbVulkanRequest request{}; + bool backend_slot_owned = false; + + ~Lardon3DMatcherPendingVulkanStage() { + if (backend_slot_owned && backend) { + (void)lardon3d_orb_vulkan_internal_top2_discard(backend, + &request); + } + } +}; + +struct MatcherStagedFileGuard { + Lardon3DMatcherStagedResult *staged = nullptr; + int fd = -1; + bool retained = false; + + ~MatcherStagedFileGuard() { + if (fd >= 0) { + (void)close(fd); + } + if (!retained && staged) { + lardon3d_matcher_discard_staged(staged); + } + } +}; + static uint64_t elapsed_ns(std::chrono::steady_clock::time_point start) { auto elapsed = std::chrono::steady_clock::now() - start; @@ -525,6 +564,10 @@ extern "C" Lardon3DMatcherResult lardon3d_matcher_run_with_backend( return LARDON3D_MATCHER_FAILED; } catch (const std::bad_alloc &) { return LARDON3D_MATCHER_FAILED; + } catch (...) { + /* Public C boundary: mutex/system/runtime exceptions are operational + * failures and must never escape into the C Task callback. */ + return LARDON3D_MATCHER_FAILED; } } @@ -556,7 +599,7 @@ extern "C" void lardon3d_matcher_discard_staged(Lardon3DMatcherStagedResult *sta std::memset(staged, 0, sizeof(*staged)); } -extern "C" Lardon3DMatcherResult lardon3d_matcher_stage( +static Lardon3DMatcherResult matcher_stage_impl( const char *project_path, const Lardon3DProjectDbFeatureSet *feature_set_a, const Lardon3DProjectDbFeatureSet *feature_set_b, @@ -591,7 +634,217 @@ extern "C" Lardon3DMatcherResult lardon3d_matcher_stage( return computed; } -extern "C" Lardon3DMatcherResult lardon3d_matcher_match_and_publish_with_backend( +extern "C" Lardon3DMatcherResult lardon3d_matcher_stage( + const char *project_path, + const Lardon3DProjectDbFeatureSet *feature_set_a, + const Lardon3DProjectDbFeatureSet *feature_set_b, + const Lardon3DMatcherParams *params, Lardon3DOrbVulkanBackend *backend, + Lardon3DMatcherStagedResult *staged) { + if (staged) std::memset(staged, 0, sizeof(*staged)); + try { + return matcher_stage_impl(project_path, feature_set_a, feature_set_b, + params, backend, staged); + } catch (...) { + if (staged) lardon3d_matcher_discard_staged(staged); + return LARDON3D_MATCHER_FAILED; + } +} + +static Lardon3DMatcherResult matcher_begin_vulkan_stage_impl( + const char *project_path, const Lardon3DProjectDbFeatureSet *a, + const Lardon3DProjectDbFeatureSet *b, const Lardon3DMatcherParams *params, + Lardon3DOrbVulkanBackend *backend, Lardon3DMatcherPendingVulkanStage **out, + bool *backend_fault) { + if (out) *out = nullptr; + if (backend_fault) *backend_fault = false; + if (!project_path || !a || !b || !params || !backend || !out || + !backend_fault || + params->kind != LARDON3D_MATCHER_ORB_BF || + !std::isfinite(params->ratio_threshold) || + params->ratio_threshold <= 0.0F || params->ratio_threshold >= 1.0F || + a->descriptor_type != LARDON3D_FEATURE_DESCRIPTOR_U8 || + b->descriptor_type != LARDON3D_FEATURE_DESCRIPTOR_U8 || + a->descriptor_dimension != 32 || b->descriptor_dimension != 32 || + !lardon3d_orb_vulkan_should_use(a->feature_count, b->feature_count)) { + return LARDON3D_MATCHER_INVALID_ARGUMENT; + } + std::unique_ptr pending( + new (std::nothrow) Lardon3DMatcherPendingVulkanStage()); + if (!pending) return LARDON3D_MATCHER_FAILED; + pending->backend = backend; + pending->a = *a; + pending->b = *b; + pending->params = *params; + pending->top2.resize(a->feature_count); + int written = std::snprintf(pending->project_path, + sizeof(pending->project_path), "%s", + project_path); + if (written <= 0 || + static_cast(written) >= sizeof(pending->project_path)) { + return LARDON3D_MATCHER_IO_ERROR; + } + FeatureReaderPair readers; + Lardon3DFeatureFileMetadata ma, mb; + auto start = std::chrono::steady_clock::now(); + if (lardon3d_feature_reader_open(project_path, a, &readers.a, &ma) != LARDON3D_FEATURE_STORE_OK || + lardon3d_feature_reader_open(project_path, b, &readers.b, &mb) != + LARDON3D_FEATURE_STORE_OK) { + return LARDON3D_MATCHER_IO_ERROR; + } + std::vector da, db; + if (!fill_descriptors_u8(da, readers.a, a->feature_count, 32) || + !fill_descriptors_u8(db, readers.b, b->feature_count, 32)) { + return LARDON3D_MATCHER_IO_ERROR; + } + pending->stats.descriptor_read_ns = elapsed_ns(start); + pending->stats.feature_count_a = a->feature_count; + pending->stats.feature_count_b = b->feature_count; + if (lardon3d_orb_vulkan_internal_top2_begin( + backend, da.data(), a->feature_count, db.data(), b->feature_count, + &pending->request + ) != LARDON3D_ORB_VULKAN_OK) { + *backend_fault = true; + return LARDON3D_MATCHER_FAILED; + } + pending->backend_slot_owned = true; + *out = pending.release(); + return LARDON3D_MATCHER_OK; +} + +extern "C" Lardon3DMatcherResult lardon3d_matcher_begin_vulkan_stage( + const char *project_path, const Lardon3DProjectDbFeatureSet *a, + const Lardon3DProjectDbFeatureSet *b, const Lardon3DMatcherParams *params, + Lardon3DOrbVulkanBackend *backend, + Lardon3DMatcherPendingVulkanStage **out, bool *backend_fault) { + if (out) *out = nullptr; + if (backend_fault) *backend_fault = false; + try { + return matcher_begin_vulkan_stage_impl( + project_path, a, b, params, backend, out, backend_fault); + } catch (...) { + /* All potentially throwing C++ work precedes the no-throw C backend + * begin call or follows a successful return. An exception here is a + * local allocation/reader fault, never backend-health evidence. */ + return LARDON3D_MATCHER_FAILED; + } +} + +static Lardon3DMatcherResult matcher_finish_vulkan_stage_impl( + Lardon3DMatcherPendingVulkanStage *pending, + Lardon3DMatcherStagedResult *staged, bool *backend_fault) { + if (backend_fault) *backend_fault = false; + if (!pending || !staged || !backend_fault) { + return LARDON3D_MATCHER_INVALID_ARGUMENT; + } + std::unique_ptr owned(pending); + std::memset(staged, 0, sizeof(*staged)); + auto start = std::chrono::steady_clock::now(); + Lardon3DOrbVulkanResult finished = + lardon3d_orb_vulkan_internal_top2_finish( + pending->backend, &pending->request, pending->top2.data(), + pending->top2.size()); + /* Internal finish consumes this exact request on every active-handle + * result; a backend session failure separately invalidates all slots. */ + pending->backend_slot_owned = false; + if (finished != LARDON3D_ORB_VULKAN_OK) { + *backend_fault = true; + return LARDON3D_MATCHER_FAILED; + } + pending->stats.knn_ns = elapsed_ns(start); + pending->stats.used_vulkan = true; + pending->stats.knn_query_count = pending->a.feature_count; + std::vector filtered; + filtered.reserve(pending->a.feature_count); + start = std::chrono::steady_clock::now(); + for (uint32_t i = 0; i < pending->a.feature_count; ++i) { + MatchEntry entry; + if (accept_orb_top2(pending->top2[i], i, pending->params.ratio_threshold, &entry)) filtered.push_back(entry); + } + pending->stats.filter_ns = elapsed_ns(start); + start = std::chrono::steady_clock::now(); + std::sort(filtered.begin(), filtered.end()); + for (size_t i = 1; i < filtered.size(); ++i) { + if (filtered[i].query_idx == filtered[i - 1].query_idx) { + return LARDON3D_MATCHER_FAILED; + } + } + if (filtered.size() > LARDON3D_MATCH_FILE_MAX_MATCHES) { + return LARDON3D_MATCHER_FAILED; + } + pending->stats.match_count = (uint32_t)filtered.size(); + pending->stats.canonicalize_ns = elapsed_ns(start); + char assets[4096], matches[4096]; + if (!join_path(assets, pending->project_path, "assets") || + !ensure_directory(assets) || !join_path(matches, assets, "matches") || + !ensure_directory(matches)) { + return LARDON3D_MATCHER_IO_ERROR; + } + int written = std::snprintf(staged->temporary_path, + sizeof(staged->temporary_path), + "%s/.match-XXXXXX", matches); + if (written <= 0 || + static_cast(written) >= sizeof(staged->temporary_path)) { + staged->temporary_path[0] = '\0'; + return LARDON3D_MATCHER_IO_ERROR; + } + int fd = mkstemp(staged->temporary_path); + if (fd < 0) { + staged->temporary_path[0] = '\0'; + return LARDON3D_MATCHER_IO_ERROR; + } + MatcherStagedFileGuard file_guard{staged, fd, false}; + std::vector entries(filtered.size()); + for (size_t i = 0; i < filtered.size(); ++i) { + entries[i] = {(uint32_t)filtered[i].query_idx, + (uint32_t)filtered[i].train_idx, filtered[i].distance}; + } + Lardon3DMatchFileResult wr = lardon3d_match_file_write( + fd, 1, pending->a.descriptor_dimension, pending->a.feature_set_id, + pending->b.feature_set_id, entries.data(), + static_cast(entries.size())); + if (wr == LARDON3D_MATCH_FILE_OK && fsync(fd) != 0) wr = LARDON3D_MATCH_FILE_IO_ERROR; + if (close(fd) != 0) wr = LARDON3D_MATCH_FILE_IO_ERROR; + file_guard.fd = -1; + if (wr != LARDON3D_MATCH_FILE_OK) { + return LARDON3D_MATCHER_IO_ERROR; + } + staged->stats = pending->stats; + file_guard.retained = true; + return LARDON3D_MATCHER_OK; +} + +extern "C" Lardon3DMatcherResult lardon3d_matcher_finish_vulkan_stage( + Lardon3DMatcherPendingVulkanStage *pending, + Lardon3DMatcherStagedResult *staged, bool *backend_fault) { + if (backend_fault) *backend_fault = false; + try { + return matcher_finish_vulkan_stage_impl( + pending, staged, backend_fault); + } catch (...) { + if (staged) lardon3d_matcher_discard_staged(staged); + /* The backend C transaction cannot throw. Exceptions after its + * successful return belong to local filtering/staging and must not + * disable a healthy shared device. */ + return LARDON3D_MATCHER_FAILED; + } +} + +extern "C" void lardon3d_matcher_discard_vulkan_stage(Lardon3DMatcherPendingVulkanStage *pending) { + if (!pending) return; + try { + std::unique_ptr owned(pending); + if (pending->backend_slot_owned) { + (void)lardon3d_orb_vulkan_internal_top2_discard( + pending->backend, &pending->request); + pending->backend_slot_owned = false; + } + } catch (...) { + /* C cancellation seam: the pending destructor performs the same + * bounded discard attempt and no C++ exception may escape. */ + } +} + +static Lardon3DMatcherResult matcher_match_and_publish_with_backend_impl( const char *project_path, Lardon3DProjectDb *database, const Lardon3DProjectDbCandidatePair *pair, @@ -822,6 +1075,38 @@ extern "C" Lardon3DMatcherResult lardon3d_matcher_match_and_publish_with_backend return LARDON3D_MATCHER_OK; } +extern "C" Lardon3DMatcherResult lardon3d_matcher_match_and_publish_with_backend( + const char *project_path, + Lardon3DProjectDb *database, + const Lardon3DProjectDbCandidatePair *pair, + const Lardon3DProjectDbFeatureSet *feature_set_a, + const Lardon3DProjectDbFeatureSet *feature_set_b, + const Lardon3DMatcherParams *params, + Lardon3DOrbVulkanBackend *backend, + Lardon3DProjectDbMatchResult *result, + Lardon3DMatcherStats *profile) { + try { + return matcher_match_and_publish_with_backend_impl( + project_path, database, pair, feature_set_a, feature_set_b, params, + backend, result, profile); + } catch (...) { + if (profile) std::memset(profile, 0, sizeof(*profile)); + if (result) std::memset(result, 0, sizeof(*result)); + return LARDON3D_MATCHER_FAILED; + } +} + +struct MatcherPublicationStageGuard { + Lardon3DMatcherStagedResult *staged; + + ~MatcherPublicationStageGuard() { + matcher_publication_stage = nullptr; + /* Publication either consumed the path or left it operation-owned. + * Both failure and exception paths end that ownership here. */ + lardon3d_matcher_discard_staged(staged); + } +}; + extern "C" Lardon3DMatcherResult lardon3d_matcher_publish_staged( const char *project_path, Lardon3DProjectDb *database, const Lardon3DProjectDbCandidatePair *pair, @@ -832,15 +1117,19 @@ extern "C" Lardon3DMatcherResult lardon3d_matcher_publish_staged( if (!staged || staged->temporary_path[0] == '\0' || matcher_publication_stage) { return LARDON3D_MATCHER_INVALID_ARGUMENT; } - matcher_publication_stage = staged; - Lardon3DMatcherResult published = lardon3d_matcher_match_and_publish_with_backend( - project_path, database, pair, feature_set_a, feature_set_b, params, nullptr, - result, nullptr); - matcher_publication_stage = nullptr; - /* Reuse and validation failures can return before the normal publication - * path takes ownership. In all cases the operation-owned temp ends here. */ - lardon3d_matcher_discard_staged(staged); - return published; + try { + matcher_publication_stage = staged; + MatcherPublicationStageGuard guard{staged}; + return lardon3d_matcher_match_and_publish_with_backend( + project_path, database, pair, feature_set_a, feature_set_b, params, + nullptr, result, nullptr); + } catch (...) { + /* The guard normally handles cleanup. This catch also protects + * construction-time failures before it exists. */ + matcher_publication_stage = nullptr; + lardon3d_matcher_discard_staged(staged); + return LARDON3D_MATCHER_FAILED; + } } extern "C" Lardon3DMatcherResult lardon3d_matcher_match_and_publish( diff --git a/src/matcher_internal.h b/src/matcher_internal.h index e10ac0a..dfabda4 100644 --- a/src/matcher_internal.h +++ b/src/matcher_internal.h @@ -1,6 +1,8 @@ #ifndef LARDON3D_MATCHER_INTERNAL_H #define LARDON3D_MATCHER_INTERNAL_H +#include + #include enum { LARDON3D_MATCHER_STAGED_PATH_CAPACITY = 4096 }; @@ -10,25 +12,53 @@ typedef struct { Lardon3DMatcherStats stats; } Lardon3DMatcherStagedResult; +typedef struct Lardon3DMatcherPendingVulkanStage Lardon3DMatcherPendingVulkanStage; + #ifdef __cplusplus extern "C" { #endif +#if defined(__GNUC__) || defined(__clang__) +#define LARDON3D_MATCHER_INTERNAL_VISIBILITY __attribute__((visibility("hidden"))) +#else +#define LARDON3D_MATCHER_INTERNAL_VISIBILITY +#endif + /* Computes one pair into an operation-owned temporary Match File without * publishing an asset or mutating Project DB. The owner must either publish * the stage or discard it; this split is what permits deterministic parallel * compute followed by ordered, single-owner durable publication. */ -Lardon3DMatcherResult lardon3d_matcher_stage( +LARDON3D_MATCHER_INTERNAL_VISIBILITY Lardon3DMatcherResult lardon3d_matcher_stage( const char *project_path, const Lardon3DProjectDbFeatureSet *feature_set_a, const Lardon3DProjectDbFeatureSet *feature_set_b, const Lardon3DMatcherParams *params, Lardon3DOrbVulkanBackend *backend, Lardon3DMatcherStagedResult *staged); +/* Begin transfers one exact request into the backend and returns one + * operation-owned pending handle. Finish consumes every non-null handle on + * every result; discard is the cancellation path. backend_fault is mandatory, + * starts false, and becomes true only when the corresponding Vulkan begin or + * finish transaction fails. Local feature I/O, allocation, post-processing or + * Match File staging errors remain ordinary Matcher failures so the Task does + * not poison shared backend health. Neither private stage nor any partial GPU + * evidence may be published by these seams. */ +LARDON3D_MATCHER_INTERNAL_VISIBILITY Lardon3DMatcherResult +lardon3d_matcher_begin_vulkan_stage( + const char *, const Lardon3DProjectDbFeatureSet *, + const Lardon3DProjectDbFeatureSet *, const Lardon3DMatcherParams *, + Lardon3DOrbVulkanBackend *, Lardon3DMatcherPendingVulkanStage **, bool *); +LARDON3D_MATCHER_INTERNAL_VISIBILITY Lardon3DMatcherResult +lardon3d_matcher_finish_vulkan_stage( + Lardon3DMatcherPendingVulkanStage *, Lardon3DMatcherStagedResult *, bool *); +LARDON3D_MATCHER_INTERNAL_VISIBILITY void +lardon3d_matcher_discard_vulkan_stage(Lardon3DMatcherPendingVulkanStage *); + /* Publishes a successful stage using the existing Match Result identity, * reuse, repair, and atomic asset rules. This function consumes the stage on * every return path and must be called only by the Task callback owner. */ -Lardon3DMatcherResult lardon3d_matcher_publish_staged( +LARDON3D_MATCHER_INTERNAL_VISIBILITY Lardon3DMatcherResult +lardon3d_matcher_publish_staged( const char *project_path, Lardon3DProjectDb *database, const Lardon3DProjectDbCandidatePair *pair, const Lardon3DProjectDbFeatureSet *feature_set_a, @@ -36,7 +66,10 @@ Lardon3DMatcherResult lardon3d_matcher_publish_staged( const Lardon3DMatcherParams *params, Lardon3DMatcherStagedResult *staged, Lardon3DProjectDbMatchResult *result); -void lardon3d_matcher_discard_staged(Lardon3DMatcherStagedResult *staged); +LARDON3D_MATCHER_INTERNAL_VISIBILITY void +lardon3d_matcher_discard_staged(Lardon3DMatcherStagedResult *staged); + +#undef LARDON3D_MATCHER_INTERNAL_VISIBILITY #ifdef __cplusplus } diff --git a/src/matcher_task.c b/src/matcher_task.c index 39aa122..45f8239 100644 --- a/src/matcher_task.c +++ b/src/matcher_task.c @@ -17,6 +17,10 @@ #include "matcher_vulkan_config.h" #include "matcher_internal.h" +#include "matcher_task_benchmark_internal.h" +#include "orb_vulkan_backend_internal.h" +#include "resource_governor_internal.h" +#include "task_internal.h" enum { MATCHER_TASK_PAGE_CAPACITY = LARDON3D_MATCHER_TASK_MAXIMUM_BATCH + 1, @@ -24,7 +28,28 @@ enum { MATCHER_TASK_CPU_THREADS = LARDON3D_MATCHER_TASK_MAXIMUM_BATCH, MATCHER_TASK_WINDOW_PER_THREAD = 2, MATCHER_TASK_WINDOW_MAX = LARDON3D_MATCHER_TASK_MAXIMUM_BATCH, - MATCHER_TASK_LEGACY_CPU_THREADS = 12, + MATCHER_TASK_FIXED_LEGACY_CPU_THREADS = 12, + MATCHER_TASK_PREVIOUS_CPU_THREADS = 8, + /* The forced item-valid batch matrix is retained under + * governor-v2-evidence/forced-batch{2,4,8,12}-items{,-b}.stdout.jsonl. + * Combined cohort rate is (2*4113*1e9)/sum(wall_ns): 54.180767704, + * 66.094373197, 74.784998723 and 76.755814095 pairs/s. Gains are + * +21.988624373%, +13.148812987% and +2.635308425%; batch twelve is below + * the 5% deadband. Safety remains proved through twelve for private forced + * evidence, while normal AUTO exposes the useful maximum eight only. */ + MATCHER_TASK_BATCH_MAX_USEFUL = 8, + MATCHER_TASK_BATCH_MAX_VALIDATED_SAFETY = + LARDON3D_MATCHER_TASK_MAXIMUM_BATCH, + /* Depth two is structurally safe and remains reproducible in private tests. + * Controlled ABBA evidence is retained under + * /home/fy59/Documents/Lardon/.real-pre-sfm-2026-08-30/ + * governor-v2-evidence/forced-depth{1,2}-{a,b}.stdout.jsonl. Each run records + * 4113 durable pairs. Cohort rate is (2*4113*1e9)/sum(wall_ns), not + * mean(per-run rate): 54.661652238 vs 55.797311953 pairs/s, +2.077617%. + * That is below the established 5% throughput deadband, so normal AUTO + * exposes useful depth one only. */ + MATCHER_TASK_DEPTH_MAX_USEFUL = 1, + MATCHER_TASK_DEPTH_MAX_VALIDATED_SAFETY = LARDON3D_ORB_VULKAN_MAX_INFLIGHT, }; typedef struct { @@ -32,6 +57,18 @@ typedef struct { Lardon3DProjectDb *database; Lardon3DResourceGovernor *governor; Lardon3DOrbVulkanBackend *orb_vulkan_backend; + bool normal_auto; + bool auto_vulkan_available; + bool explicit_vulkan; +#ifdef LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE + /* Benchmark-only and Task-private. This operational control is reconstructed + * from the opt-in runner process, never from Project DB or checkpoint state; + * normal production builds contain neither the field nor its environment + * token. The installed execution contract remains immutable either way. */ + bool benchmark_synchronous_pipeline; + size_t benchmark_inflight_override; + size_t benchmark_batch_override; +#endif Lardon3DProjectDbMatcherTask parameters; } Lardon3DMatcherTaskContext; @@ -56,10 +93,41 @@ typedef struct { #ifdef LARDON3D_MATCHER_TASK_TESTING static atomic_size_t test_vulkan_uses; static atomic_size_t test_forced_fallbacks; +static atomic_size_t test_overlap_publications; +static atomic_uint_fast64_t test_max_retained_vulkan_payload; +enum { + TEST_EVENT_GPU_SUBMIT = 1, + TEST_EVENT_GPU_FINISH = 2, + TEST_EVENT_PUBLICATION_START = 3, + TEST_EVENT_PUBLICATION_FINISH = 4, + TEST_EVENT_CAPACITY = 256, +}; +typedef struct { + int kind; + uint64_t candidate_pair_id; + size_t order; +} Lardon3DMatcherTaskTestEvent; +static Lardon3DMatcherTaskTestEvent test_events[TEST_EVENT_CAPACITY]; +static atomic_size_t test_event_count; + +static void test_record_event(int kind, uint64_t candidate_pair_id) { + size_t order = atomic_fetch_add(&test_event_count, 1); + if (order < TEST_EVENT_CAPACITY) { + test_events[order] = (Lardon3DMatcherTaskTestEvent){ + .kind = kind, + .candidate_pair_id = candidate_pair_id, + .order = order, + }; + } +} void lardon3d_matcher_task_test_reset_backend_counters(void) { atomic_store(&test_vulkan_uses, 0); atomic_store(&test_forced_fallbacks, 0); + atomic_store(&test_overlap_publications, 0); + atomic_store(&test_max_retained_vulkan_payload, 0); + atomic_store(&test_event_count, 0); + memset(test_events, 0, sizeof(test_events)); } size_t lardon3d_matcher_task_test_vulkan_uses(void) { @@ -69,6 +137,32 @@ size_t lardon3d_matcher_task_test_vulkan_uses(void) { size_t lardon3d_matcher_task_test_forced_fallbacks(void) { return atomic_load(&test_forced_fallbacks); } + +uint64_t lardon3d_matcher_task_test_max_retained_vulkan_payload(void) { + return atomic_load(&test_max_retained_vulkan_payload); +} + +size_t lardon3d_matcher_task_test_overlap_publications(void) { + return atomic_load(&test_overlap_publications); +} + +size_t lardon3d_matcher_task_test_event_count(void) { + size_t count = atomic_load(&test_event_count); + return count < TEST_EVENT_CAPACITY ? count : TEST_EVENT_CAPACITY; +} + +bool lardon3d_matcher_task_test_event( + size_t index, int *kind, uint64_t *candidate_pair_id, size_t *order) { + if (!kind || !candidate_pair_id || !order || + index >= lardon3d_matcher_task_test_event_count()) { + return false; + } + *kind = test_events[index].kind; + *candidate_pair_id = test_events[index].candidate_pair_id; + *order = test_events[index].order; + return true; +} + #endif static void destroy_context(void *userdata) { free(userdata); } @@ -115,6 +209,149 @@ static Lardon3DResourceEstimate matcher_estimate(Lardon3DMatcherTaskMode mode) { }; } +static bool auto_vulkan_backend_candidate(Lardon3DOrbVulkanBackend *backend) { + Lardon3DOrbVulkanInfo info; + return backend && lardon3d_orb_vulkan_backend_info(backend, &info) && + (!info.initialized || info.available); +} + +static bool auto_vulkan_runtime_candidate(const Lardon3DAppState *state) { + /* AUTO creation is caller-thread metadata work only. Memory sizing does not + * belong here: the Governor owns the exact reconstructed batch/depth, UMA + * charge, current MemAvailable/PSI/swap snapshot, and 3 GiB/2 GiB policy. + * A caller-side maximum-window guess could suppress a safe depth-1 contract + * before CPU fallback was even considered. Driver initialization remains + * deferred to begin() on Queue's affinity-constrained worker. */ + return LARDON3D_HAVE_VULKAN && state->hardware_profile.gpu_available && + auto_vulkan_backend_candidate(state->orb_vulkan_backend); +} + +static Lardon3DTaskCapabilityEnvelope matcher_auto_envelope( + const Lardon3DResourceEstimate *cpu, bool expose_vulkan, + bool allow_depth_two, size_t benchmark_inflight_override, + size_t benchmark_batch_override) { + Lardon3DResourceEstimate vulkan = matcher_estimate( + LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN); + /* The AUTO operation is semantically MIXED even when this admission selects + * one CPU or Vulkan capability. Preserve that truthful class in each + * operational alternative; backend choice remains private and ephemeral. */ + vulkan.task_class = cpu->task_class; + vulkan.maximum_batch_size = MATCHER_TASK_BATCH_MAX_USEFUL; + /* The durable AUTO estimate remains the historical minimum depth-1 + * signature. Operational slot payload is reconstructed here: 2*8192*32 + * descriptor bytes plus 8192*4*uint32 readback = 655360 bytes per slot, + * with no invented fixed charge for opaque shared driver objects. */ + vulkan.gpu_memory_fixed_bytes = LARDON3D_ORB_VULKAN_FIXED_BYTES; + size_t maximum_inflight = MATCHER_TASK_DEPTH_MAX_USEFUL; + size_t minimum_inflight = MATCHER_TASK_DEPTH_MAX_USEFUL; + bool inflight_adaptive = false; + bool benchmark_forced_vulkan_only = false; +#ifdef LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE + if (expose_vulkan && benchmark_inflight_override != 0) { + /* Benchmark matrices still travel through the normal Governor. Fixed + * batch/depth remove adaptive-history confounding, while the one chosen + * capability remains fully charged and immutable for every sequence. */ + minimum_inflight = benchmark_inflight_override; + maximum_inflight = benchmark_inflight_override; + inflight_adaptive = false; + size_t batch = benchmark_batch_override != 0 + ? benchmark_batch_override : 2; + vulkan.minimum_batch_size = batch; + vulkan.maximum_batch_size = batch; + benchmark_forced_vulkan_only = true; + } +#else + (void)benchmark_inflight_override; + (void)benchmark_batch_override; +#endif + (void)allow_depth_two; +#ifdef LARDON3D_MATCHER_TASK_TESTING + /* Test-only deterministic admission seam: exercise production depth-two + * ordering independently of rolling feedback history. Normal binaries do + * not compile or respond to this token. */ + const char *forced_inflight = getenv("LARDON3D_TEST_MATCHER_INFLIGHT_LIMIT"); + if (expose_vulkan && benchmark_inflight_override == 0 && allow_depth_two + && forced_inflight + && strcmp(forced_inflight, "2") == 0) { + minimum_inflight = MATCHER_TASK_DEPTH_MAX_VALIDATED_SAFETY; + maximum_inflight = MATCHER_TASK_DEPTH_MAX_VALIDATED_SAFETY; + inflight_adaptive = false; + vulkan.minimum_batch_size = 2; + vulkan.maximum_batch_size = 2; + } +#endif + Lardon3DTaskCapabilityEnvelope envelope = { + /* A forced A/B cohort is evidence about one admitted Vulkan contract, + * not AUTO fallback policy. Exposing only that capability makes GPU, + * backend, UMA, and memory non-admission reject the experiment instead + * of silently selecting adaptive CPU. Normal AUTO still exposes both. */ + .count = expose_vulkan ? benchmark_forced_vulkan_only ? 1 : 2 : 1, + .capabilities = { + { + .estimate = expose_vulkan ? vulkan : *cpu, + .backend = expose_vulkan + ? LARDON3D_RESOURCE_BACKEND_ORB_VULKAN + : LARDON3D_RESOURCE_BACKEND_CPU, + .inflight_limit = expose_vulkan ? maximum_inflight : 1, + .minimum_inflight_limit = expose_vulkan ? minimum_inflight : 0, + .gpu_memory_bytes_per_inflight = + expose_vulkan ? LARDON3D_ORB_VULKAN_PER_SLOT_BYTES : 0, + .preferred = expose_vulkan, + .cpu_reducible = !expose_vulkan, + .batch_adaptive = expose_vulkan + ? vulkan.minimum_batch_size != vulkan.maximum_batch_size + : true, + .sustained_gpu_batch_feedback = expose_vulkan + && !benchmark_forced_vulkan_only + && vulkan.minimum_batch_size != vulkan.maximum_batch_size, + .inflight_adaptive = expose_vulkan && inflight_adaptive, + .requires_runtime_backend = expose_vulkan, + }, + { + .estimate = *cpu, + .backend = LARDON3D_RESOURCE_BACKEND_CPU, + .inflight_limit = 1, + /* CPU participants are the admitted cpu_threads dimension. + * helpers remain zero until a distinct GPU helper is admitted. */ + .helper_limit = 0, + .cpu_reducible = true, + .batch_adaptive = true, + }, + }, + }; + return envelope; +} + +#ifdef LARDON3D_MATCHER_TASK_TESTING +bool lardon3d_matcher_task_test_auto_capability_envelope( + size_t benchmark_inflight_override, size_t benchmark_batch_override, + Lardon3DTaskCapabilityEnvelope *envelope) { + if (!envelope) return false; + Lardon3DResourceEstimate cpu = matcher_estimate( + LARDON3D_MATCHER_TASK_MODE_CPU_PARALLEL); + cpu.task_class = LARDON3D_RESOURCE_TASK_MIXED; + *envelope = matcher_auto_envelope( + &cpu, true, true, benchmark_inflight_override, + benchmark_batch_override); + return true; +} +#endif + +static Lardon3DTaskCapabilityEnvelope matcher_fixed_envelope( + const Lardon3DResourceEstimate *estimate, bool vulkan) { + return (Lardon3DTaskCapabilityEnvelope){ + .count = 1, + .capabilities = {{ + .estimate = *estimate, + .backend = vulkan ? LARDON3D_RESOURCE_BACKEND_ORB_VULKAN + : LARDON3D_RESOURCE_BACKEND_CPU, + .inflight_limit = 1, + .helper_limit = 0, + .cpu_reducible = !vulkan, + }}, + }; +} + static bool estimate_equals(const Lardon3DResourceEstimate *left, const Lardon3DResourceEstimate *right) { return left && right && @@ -139,7 +376,17 @@ static Lardon3DResourceEstimate legacy_matcher_estimate(bool vulkan) { * shape before converting it to the current per-pair reservation. */ estimate.memory_fixed_bytes = MATCHER_TASK_MEMORY_BYTES; estimate.memory_bytes_per_item = 0; - estimate.desired_cpu_threads = MATCHER_TASK_LEGACY_CPU_THREADS; + estimate.maximum_batch_size = 8; + estimate.desired_cpu_threads = MATCHER_TASK_FIXED_LEGACY_CPU_THREADS; + return estimate; +} + +static Lardon3DResourceEstimate previous_matcher_estimate(bool vulkan) { + Lardon3DResourceEstimate estimate = matcher_estimate( + vulkan ? LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN + : LARDON3D_MATCHER_TASK_MODE_CPU_PARALLEL); + estimate.maximum_batch_size = 8; + estimate.desired_cpu_threads = vulkan ? 1U : MATCHER_TASK_PREVIOUS_CPU_THREADS; return estimate; } @@ -157,6 +404,35 @@ static uint64_t elapsed_ns(struct timespec begin, struct timespec end) { return seconds * 1000000000ULL + (uint64_t)nanoseconds; } +static void saturating_add_ns(uint64_t *total, uint64_t value) { + *total = *total > UINT64_MAX - value ? UINT64_MAX : *total + value; +} + +static uint64_t cumulative_delta(uint64_t before, uint64_t after) { + return after >= before ? after - before : 0; +} + +static void add_vulkan_telemetry_delta( + const Lardon3DOrbVulkanTelemetry *before, + const Lardon3DOrbVulkanTelemetry *after, + Lardon3DResourceExecutionMetrics *metrics) { + metrics->vulkan_submits = cumulative_delta(before->submits, after->submits); + metrics->vulkan_completions = + cumulative_delta(before->completions, after->completions); + metrics->vulkan_submit_cpu_ns = + cumulative_delta(before->submit_cpu_ns, after->submit_cpu_ns); + metrics->vulkan_fence_wait_ns = + cumulative_delta(before->fence_wait_ns, after->fence_wait_ns); + metrics->vulkan_readback_ns = + cumulative_delta(before->readback_ns, after->readback_ns); + metrics->vulkan_gpu_time_known = after->gpu_timestamps_available + && metrics->vulkan_completions > 0; + metrics->vulkan_gpu_ns = + cumulative_delta(before->gpu_execution_ns, after->gpu_execution_ns); + metrics->vulkan_starvation_ns = + cumulative_delta(before->starvation_ns, after->starvation_ns); +} + static bool load_feature_sets(Lardon3DMatcherTaskContext *context, const Lardon3DProjectDbCandidatePair *pair, Lardon3DProjectDbFeatureSet *feature_set_a, @@ -244,7 +520,7 @@ static bool compute_window(const Lardon3DMatcherTaskContext *context, workers[participant] = (Lardon3DMatcherWorker){ .context = context, .matcher = matcher, - .backend = cpu_threads == 1 ? context->orb_vulkan_backend : NULL, + .backend = NULL, .stages = stages, .count = count, .participant = participant, @@ -265,7 +541,7 @@ static bool compute_window(const Lardon3DMatcherTaskContext *context, workers[0] = (Lardon3DMatcherWorker){ .context = context, .matcher = matcher, - .backend = cpu_threads == 1 ? context->orb_vulkan_backend : NULL, + .backend = NULL, .stages = stages, .count = count, .participant = 0, @@ -284,28 +560,551 @@ static bool compute_window(const Lardon3DMatcherTaskContext *context, static bool publish_pair(Lardon3DTask *task, Lardon3DMatcherTaskContext *context, const Lardon3DMatcherParams *matcher, - Lardon3DMatcherPairStage *stage) { + Lardon3DMatcherPairStage *stage, + uint64_t *publication_ns) { if (test_fail_pair("LARDON3D_TEST_MATCHER_FAIL_PUBLISH_PAIR_ID", stage->pair.candidate_pair_id)) { return fail_task(task, "Publication Matcher injectée impossible."); } Lardon3DProjectDbMatchResult result; - if (lardon3d_matcher_publish_staged( + struct timespec begin; + struct timespec end; + (void)clock_gettime(CLOCK_MONOTONIC, &begin); + Lardon3DMatcherResult published = lardon3d_matcher_publish_staged( context->project_path, context->database, &stage->pair, &stage->feature_set_a, &stage->feature_set_b, matcher, &stage->staged, - &result) != LARDON3D_MATCHER_OK) { + &result); + (void)clock_gettime(CLOCK_MONOTONIC, &end); + if (publication_ns) { + saturating_add_ns(publication_ns, elapsed_ns(begin, end)); + } + if (published != LARDON3D_MATCHER_OK) { return fail_task(task, "Matching de la Candidate Pair impossible."); } + return true; +} + +static bool checkpoint_after_publication(Lardon3DTask *task) { #ifdef LARDON3D_MATCHER_TASK_TESTING const char *pause = getenv("LARDON3D_TEST_MATCHER_PAUSE_AFTER_PUBLICATION"); if (pause && strcmp(pause, "1") == 0) { (void)lardon3d_task_pause(task); return lardon3d_task_checkpoint(task); } +#else + (void)task; #endif return true; } +typedef enum { + VULKAN_SUBMISSION_EMPTY = 0, + VULKAN_SUBMISSION_SUBMITTED, + VULKAN_SUBMISSION_LOCAL_INELIGIBLE, + VULKAN_SUBMISSION_OTHER_FAILED, + VULKAN_SUBMISSION_BACKEND_FAILED, +} Lardon3DMatcherVulkanSubmissionState; + +typedef enum { + MATCHER_FALLBACK_NONE = 0, + MATCHER_FALLBACK_LOCAL_INELIGIBLE, + MATCHER_FALLBACK_BACKEND_FAILURE, + MATCHER_FALLBACK_OTHER, +} Lardon3DMatcherFallbackCause; + +typedef struct { + Lardon3DMatcherVulkanSubmissionState state; + Lardon3DMatcherPendingVulkanStage *pending; +} Lardon3DMatcherVulkanSubmission; + +static void note_completed_fallback_item( + Lardon3DResourceExecutionMetrics *metrics, + Lardon3DMatcherFallbackCause cause) { + uint64_t *counter = NULL; + switch (cause) { + case MATCHER_FALLBACK_NONE: + return; + case MATCHER_FALLBACK_LOCAL_INELIGIBLE: + counter = &metrics->local_ineligible_fallback_items; + break; + case MATCHER_FALLBACK_BACKEND_FAILURE: + counter = &metrics->backend_failure_fallback_items; + break; + case MATCHER_FALLBACK_OTHER: + counter = &metrics->backend_other_fallback_items; + break; + } + if (*counter == UINT64_MAX) { + metrics->fallback_items_saturated = true; + } else { + ++*counter; + } +} + +static void commit_completed_fallback_item( + Lardon3DTask *task, uint64_t candidate_pair_id, + Lardon3DResourceExecutionMetrics *metrics, + Lardon3DMatcherFallbackCause cause) { + if (cause == MATCHER_FALLBACK_NONE) { + return; + } + note_completed_fallback_item(metrics, cause); + Lardon3DResourceFallbackItemCause governor_cause = + cause == MATCHER_FALLBACK_LOCAL_INELIGIBLE + ? LARDON3D_RESOURCE_FALLBACK_ITEM_LOCAL_INELIGIBLE + : cause == MATCHER_FALLBACK_BACKEND_FAILURE + ? LARDON3D_RESOURCE_FALLBACK_ITEM_BACKEND_FAILURE + : LARDON3D_RESOURCE_FALLBACK_ITEM_OTHER; + /* CONTRACT: publication above is already durable. Commit the operational + * item class now so a later pair's cancellation/computation/publication + * failure cannot erase this prefix. Task's current-run watermark makes an + * in-process retry idempotent; this does not create throughput feedback. */ + (void)lardon3d_task_internal_record_fallback_item( + task, candidate_pair_id, governor_cause); +} + +static void note_vulkan_backend_failure( + Lardon3DMatcherTaskContext *context, bool *backend_failed) { + *backend_failed = true; + /* Backend failure is shared execution evidence, regardless of whether the + * failing Task was AUTO or an explicit diagnostic override. Publish it to + * the Governor immediately: later CPU fallback, cancellation, or durable + * publication may fail and must not leave a broken backend advertised. */ + (void)lardon3d_resource_governor_internal_set_backend_available( + context->governor, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, false); +} + +static Lardon3DMatcherVulkanSubmission begin_vulkan_submission( + Lardon3DMatcherTaskContext *context, const Lardon3DMatcherParams *matcher, + const Lardon3DMatcherPairStage *stage, bool *backend_failed, + bool *vulkan_ineligible, bool *vulkan_other_failure) { + Lardon3DMatcherPendingVulkanStage *pending = NULL; + Lardon3DMatcherResult result; + bool backend_fault = false; + if (test_fail_pair("LARDON3D_TEST_MATCHER_INELIGIBLE_VULKAN_BEGIN_PAIR_ID", + stage->pair.candidate_pair_id)) { + result = LARDON3D_MATCHER_INVALID_ARGUMENT; + } else if (test_fail_pair("LARDON3D_TEST_MATCHER_FAIL_VULKAN_BEGIN_PAIR_ID", + stage->pair.candidate_pair_id)) { + result = LARDON3D_MATCHER_FAILED; + backend_fault = true; + } else if (test_fail_pair( + "LARDON3D_TEST_MATCHER_FAIL_LOCAL_VULKAN_BEGIN_PAIR_ID", + stage->pair.candidate_pair_id)) { + result = LARDON3D_MATCHER_IO_ERROR; + } else { + result = lardon3d_matcher_begin_vulkan_stage( + context->project_path, &stage->feature_set_a, &stage->feature_set_b, + matcher, context->orb_vulkan_backend, &pending, &backend_fault); + } + if (result == LARDON3D_MATCHER_OK && pending) { + return (Lardon3DMatcherVulkanSubmission) { + .state = VULKAN_SUBMISSION_SUBMITTED, + .pending = pending, + }; + } + if (pending) { + /* The private begin contract should return a handle only with OK. Keep + * cleanup deterministic even if a backend violates that contract. */ + lardon3d_matcher_discard_vulkan_stage(pending); + pending = NULL; + } + if (result == LARDON3D_MATCHER_INVALID_ARGUMENT) { + /* Eligibility belongs to this complete pair only. It neither consumes a + * pending slot nor changes shared backend health. */ + *vulkan_ineligible = true; + return (Lardon3DMatcherVulkanSubmission) { + .state = VULKAN_SUBMISSION_LOCAL_INELIGIBLE, + }; + } + if (!backend_fault) { + /* Feature I/O, allocation and other pre-submit faults consume no request + * and cannot say anything about the shared backend. This pair alone falls + * back to CPU while already-submitted or later successors remain valid. */ + *vulkan_other_failure = true; + return (Lardon3DMatcherVulkanSubmission) { + .state = VULKAN_SUBMISSION_OTHER_FAILED, + }; + } + note_vulkan_backend_failure(context, backend_failed); + return (Lardon3DMatcherVulkanSubmission) { + .state = VULKAN_SUBMISSION_BACKEND_FAILED, + }; +} + +static void discard_vulkan_submission( + Lardon3DMatcherVulkanSubmission *submission) { + if (submission->state == VULKAN_SUBMISSION_SUBMITTED + && submission->pending) { + lardon3d_matcher_discard_vulkan_stage(submission->pending); + } + submission->pending = NULL; + submission->state = VULKAN_SUBMISSION_EMPTY; +} + +static void discard_vulkan_submissions( + Lardon3DMatcherVulkanSubmission *submissions, size_t count) { + for (size_t index = 0; index < count; ++index) { + discard_vulkan_submission(&submissions[index]); + } +} + +static void fail_active_vulkan_submissions( + Lardon3DMatcherVulkanSubmission *submissions, size_t count) { + for (size_t index = 0; index < count; ++index) { + Lardon3DMatcherVulkanSubmission *submission = &submissions[index]; + if (submission->state != VULKAN_SUBMISSION_SUBMITTED) { + /* Pair-local INVALID_ARGUMENT evidence is immutable with respect to a + * neighboring request failure. Published/empty entries likewise remain + * consumed. Only exact live requests lose backend health here. */ + continue; + } + if (submission->pending) { + lardon3d_matcher_discard_vulkan_stage(submission->pending); + } + submission->pending = NULL; + submission->state = VULKAN_SUBMISSION_BACKEND_FAILED; + } +} + +static void submit_vulkan_until_limit( + Lardon3DMatcherTaskContext *context, + const Lardon3DMatcherParams *matcher, + Lardon3DMatcherPairStage *stages, + size_t count, + size_t inflight_limit, + size_t *next_to_submit, + size_t *pending_count, + Lardon3DMatcherVulkanSubmission *submissions, + bool *backend_failed, + bool *vulkan_ineligible, + bool *vulkan_other_failure) { + while (!*backend_failed && *next_to_submit < count + && *pending_count < inflight_limit) { + size_t index = *next_to_submit; + submissions[index] = begin_vulkan_submission( + context, matcher, &stages[index], backend_failed, + vulkan_ineligible, vulkan_other_failure); + ++*next_to_submit; + if (submissions[index].state == VULKAN_SUBMISSION_SUBMITTED) { + ++*pending_count; +#ifdef LARDON3D_MATCHER_TASK_TESTING + test_record_event(TEST_EVENT_GPU_SUBMIT, + stages[index].pair.candidate_pair_id); + Lardon3DOrbVulkanTelemetry capacity_telemetry = {0}; + if (lardon3d_orb_vulkan_internal_telemetry( + context->orb_vulkan_backend, &capacity_telemetry)) { + uint_fast64_t observed = + atomic_load(&test_max_retained_vulkan_payload); + while (observed < capacity_telemetry.retained_payload_bytes + && !atomic_compare_exchange_weak( + &test_max_retained_vulkan_payload, &observed, + capacity_telemetry.retained_payload_bytes)) { + } + } +#endif + } else if (submissions[index].state + == VULKAN_SUBMISSION_BACKEND_FAILED) { + /* An actual backend failure invalidates every request slot in the shared + * session. Consume the Task-private handles now and classify every later + * pair for whole-pair CPU fallback; no stale finish may be redirected. */ + fail_active_vulkan_submissions(submissions, *next_to_submit); + *pending_count = 0; + } + } +} + +/* Queue worker count is one, so this owner alone advances the durable cursor. + * Keep at most the immutable admitted depth (one or two) submitted while the + * owner publishes the completed prefix. Publication (hash/fsync/SQLite) can + * overlap private Vulkan work without allowing completion order to affect + * ascending candidate identity. A pair without a submitted request always + * executes wholly on CPU; finish() is called only for its exact request. */ +static bool compute_publish_vulkan_window( + Lardon3DTask *task, Lardon3DMatcherTaskContext *context, + const Lardon3DMatcherParams *matcher, Lardon3DMatcherPairStage *stages, + size_t count, size_t *published, bool *used_cpu, bool *used_vulkan, + bool *backend_failed, bool *vulkan_ineligible, + bool *vulkan_other_failure, size_t inflight_limit, + Lardon3DResourceExecutionMetrics *metrics) { + *published = 0; + if (count == 0) { + return true; + } + if (inflight_limit == 0 + || inflight_limit > LARDON3D_ORB_VULKAN_MAX_INFLIGHT) return false; + Lardon3DMatcherVulkanSubmission submissions[MATCHER_TASK_WINDOW_MAX] = {0}; + size_t next_to_submit = 0; + size_t pending_count = 0; + submit_vulkan_until_limit( + context, matcher, stages, count, inflight_limit, &next_to_submit, + &pending_count, submissions, backend_failed, vulkan_ineligible, + vulkan_other_failure); + for (size_t index = 0; index < count; ++index) { + if (!lardon3d_task_checkpoint(task)) { + discard_vulkan_submissions(submissions, count); + discard_window(stages, count); + return false; + } + if (index >= next_to_submit) { + submissions[index].state = VULKAN_SUBMISSION_BACKEND_FAILED; + next_to_submit = index + 1; + } + Lardon3DMatcherVulkanSubmission *current = &submissions[index]; + Lardon3DMatcherResult compute_result = LARDON3D_MATCHER_OK; + Lardon3DMatcherFallbackCause fallback_cause = + current->state == VULKAN_SUBMISSION_LOCAL_INELIGIBLE + ? MATCHER_FALLBACK_LOCAL_INELIGIBLE + : current->state == VULKAN_SUBMISSION_BACKEND_FAILED + ? MATCHER_FALLBACK_BACKEND_FAILURE + : current->state == VULKAN_SUBMISSION_OTHER_FAILED + ? MATCHER_FALLBACK_OTHER + : current->state == VULKAN_SUBMISSION_SUBMITTED + ? MATCHER_FALLBACK_NONE + : MATCHER_FALLBACK_OTHER; + if (current->state == VULKAN_SUBMISSION_SUBMITTED) { + bool finish_backend_fault = false; +#ifdef LARDON3D_MATCHER_TASK_TESTING + bool injected_finish_failure = test_fail_pair( + "LARDON3D_TEST_MATCHER_FAIL_VULKAN_FINISH_PAIR_ID", + stages[index].pair.candidate_pair_id); + if (injected_finish_failure) { + lardon3d_matcher_discard_vulkan_stage(current->pending); + compute_result = LARDON3D_MATCHER_FAILED; + finish_backend_fault = true; + } else +#endif + { + compute_result = lardon3d_matcher_finish_vulkan_stage( + current->pending, &stages[index].staged, + &finish_backend_fault); +#ifdef LARDON3D_MATCHER_TASK_TESTING + if (!finish_backend_fault && test_fail_pair( + "LARDON3D_TEST_MATCHER_FAIL_LOCAL_VULKAN_FINISH_PAIR_ID", + stages[index].pair.candidate_pair_id)) { + /* Deterministically model a local staging failure after the exact + * backend finish succeeded and consumed its request. */ + lardon3d_matcher_discard_staged(&stages[index].staged); + compute_result = LARDON3D_MATCHER_IO_ERROR; + } +#endif + } + current->pending = NULL; /* finish consumes this exact request. */ + if (pending_count > 0) --pending_count; +#ifdef LARDON3D_MATCHER_TASK_TESTING + if (!finish_backend_fault) { + test_record_event(TEST_EVENT_GPU_FINISH, + stages[index].pair.candidate_pair_id); + } +#endif + if (compute_result != LARDON3D_MATCHER_OK) { + if (finish_backend_fault) { + fallback_cause = MATCHER_FALLBACK_BACKEND_FAILURE; + note_vulkan_backend_failure(context, backend_failed); + fail_active_vulkan_submissions(submissions, next_to_submit); + pending_count = 0; + } else { + /* Successful top2 completion followed by local filtering, memory or + * Match File staging failure invalidates only this pair's stage. + * Preserve shared health and unrelated submitted successors. */ + fallback_cause = MATCHER_FALLBACK_OTHER; + current->state = VULKAN_SUBMISSION_OTHER_FAILED; + *vulkan_other_failure = true; + } + } + } + if (current->state != VULKAN_SUBMISSION_SUBMITTED + || compute_result != LARDON3D_MATCHER_OK) { + /* GPU failure is an operational fallback, not a partial scientific + * result. Local ineligibility and local begin/finish faults likewise + * publish no partial evidence. + * Execute this complete pair on CPU. Locally eligible successors may + * already be submitted at depth two but remain private until their own + * ordered turn; a backend failure has discarded them above. No + * successor can publish ahead of this CPU fallback. */ + lardon3d_matcher_discard_staged(&stages[index].staged); + *used_cpu = true; + struct timespec cpu_begin; + struct timespec cpu_end; + (void)clock_gettime(CLOCK_MONOTONIC, &cpu_begin); + bool cpu_fallback_failed = + test_fail_pair("LARDON3D_TEST_MATCHER_FAIL_COMPUTE_PAIR_ID", + stages[index].pair.candidate_pair_id); + if (cpu_fallback_failed || + lardon3d_matcher_stage(context->project_path, + &stages[index].feature_set_a, + &stages[index].feature_set_b, matcher, NULL, + &stages[index].staged) != LARDON3D_MATCHER_OK) { + /* CONTRACT: a locally ineligible oldest pair consumes no GPU slot, so + * depth two may already own a submitted successor here. Consume every + * request-bound handle before staged-output cleanup and return; only + * then can sequence end shrink its mapped payload under the still-live + * Governor reservation. No successor evidence is publishable. */ + discard_vulkan_submissions(submissions, count); + discard_window(stages, count); + return false; + } + (void)clock_gettime(CLOCK_MONOTONIC, &cpu_end); + saturating_add_ns(&metrics->matcher_cpu_ns, + elapsed_ns(cpu_begin, cpu_end)); + } + stages[index].computed = LARDON3D_MATCHER_OK; + bool pair_used_vulkan = stages[index].staged.stats.used_vulkan; + submit_vulkan_until_limit( + context, matcher, stages, count, inflight_limit, &next_to_submit, + &pending_count, submissions, backend_failed, vulkan_ineligible, + vulkan_other_failure); +#ifdef LARDON3D_MATCHER_TASK_TESTING + bool successor_submitted = false; + for (size_t successor = index + 1; successor < next_to_submit; + ++successor) { + if (submissions[successor].state == VULKAN_SUBMISSION_SUBMITTED) { + successor_submitted = true; + break; + } + } +#endif + if (pair_used_vulkan) { + *used_vulkan = true; + } else { + *used_cpu = true; + } +#ifdef LARDON3D_MATCHER_TASK_TESTING + /* This event is deliberately ordered after begin returned and immediately + * before owner publication. It proves the rolling invariant without + * timing/sleep assumptions: the successor is in the backend before this + * candidate can enter hashing, fsync, or DB publication. */ + if (successor_submitted) { + atomic_fetch_add(&test_overlap_publications, 1); + } + test_record_event(TEST_EVENT_PUBLICATION_START, + stages[index].pair.candidate_pair_id); +#endif + bool publication_succeeded = + publish_pair(task, context, matcher, &stages[index], + &metrics->publication_ns); +#ifdef LARDON3D_MATCHER_TASK_TESTING + test_record_event(TEST_EVENT_PUBLICATION_FINISH, + stages[index].pair.candidate_pair_id); +#endif + if (!publication_succeeded) { + discard_vulkan_submissions(submissions, count); + discard_window(stages, count); + return false; + } + /* CONTRACT: item telemetry follows the same durable boundary as cursor + * movement. Classify this Vulkan-selected pair exactly once only after its + * complete CPU result is published; pending or partial GPU evidence never + * contributes, and successful Vulkan work contributes no fallback item. */ + if (!pair_used_vulkan) { + commit_completed_fallback_item( + task, stages[index].pair.candidate_pair_id, metrics, + fallback_cause == MATCHER_FALLBACK_NONE + ? MATCHER_FALLBACK_OTHER : fallback_cause); + } + /* A stage is consumed by publication. Move the cursor only after its + * atomic Match Result is durable; later in-flight work remains private. */ + context->parameters.after_candidate_pair_id = stages[index].pair.candidate_pair_id; + ++*published; + if (!checkpoint_after_publication(task)) { + discard_vulkan_submissions(submissions, count); + discard_window(stages, count); + return false; + } +#ifdef LARDON3D_MATCHER_TASK_TESTING + if (pair_used_vulkan) { + atomic_fetch_add(&test_vulkan_uses, 1); + } +#endif + current->state = VULKAN_SUBMISSION_EMPTY; + } + return true; +} + +#ifdef LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE +/* Historical synchronous-fence control for benchmark evidence only. Each + * selected GPU pair completes the existing public top2 transaction before its + * canonical stage is published. A Vulkan error produces the public primitive's + * complete CPU fallback; no partial GPU evidence or backend choice is durable. */ +static bool compute_publish_synchronous_vulkan_window( + Lardon3DTask *task, Lardon3DMatcherTaskContext *context, + const Lardon3DMatcherParams *matcher, Lardon3DMatcherPairStage *stages, + size_t count, size_t *published, bool *used_cpu, bool *used_vulkan, + bool *backend_failed, bool *vulkan_ineligible, + Lardon3DResourceExecutionMetrics *metrics) { + *published = 0; + for (size_t index = 0; index < count; ++index) { + if (!lardon3d_task_checkpoint(task)) { + discard_window(stages, count); + return false; + } + struct timespec compute_begin; + struct timespec compute_end; + (void)clock_gettime(CLOCK_MONOTONIC, &compute_begin); + bool injected_local = test_fail_pair( + "LARDON3D_TEST_MATCHER_INELIGIBLE_VULKAN_BEGIN_PAIR_ID", + stages[index].pair.candidate_pair_id); + bool injected_backend_failure = test_fail_pair( + "LARDON3D_TEST_MATCHER_FAIL_VULKAN_BEGIN_PAIR_ID", + stages[index].pair.candidate_pair_id) + || test_fail_pair( + "LARDON3D_TEST_MATCHER_FAIL_VULKAN_FINISH_PAIR_ID", + stages[index].pair.candidate_pair_id); + bool injected_compute_failure = test_fail_pair( + "LARDON3D_TEST_MATCHER_FAIL_COMPUTE_PAIR_ID", + stages[index].pair.candidate_pair_id); + Lardon3DOrbVulkanBackend *pair_backend = + injected_local || injected_backend_failure + ? NULL : context->orb_vulkan_backend; + stages[index].computed = injected_compute_failure + ? LARDON3D_MATCHER_FAILED + : lardon3d_matcher_stage( + context->project_path, &stages[index].feature_set_a, + &stages[index].feature_set_b, matcher, pair_backend, + &stages[index].staged); + (void)clock_gettime(CLOCK_MONOTONIC, &compute_end); + if (stages[index].computed != LARDON3D_MATCHER_OK) { + discard_window(stages, count); + return false; + } + Lardon3DMatcherFallbackCause fallback_cause = MATCHER_FALLBACK_NONE; + if (stages[index].staged.stats.used_vulkan) { + *used_vulkan = true; +#ifdef LARDON3D_MATCHER_TASK_TESTING + atomic_fetch_add(&test_vulkan_uses, 1); +#endif + } else { + *used_cpu = true; + saturating_add_ns(&metrics->matcher_cpu_ns, + elapsed_ns(compute_begin, compute_end)); + if (injected_backend_failure + || stages[index].staged.stats.vulkan_fallback) { + fallback_cause = MATCHER_FALLBACK_BACKEND_FAILURE; + note_vulkan_backend_failure(context, backend_failed); + } else { + fallback_cause = MATCHER_FALLBACK_LOCAL_INELIGIBLE; + *vulkan_ineligible = true; + } + } + if (!publish_pair(task, context, matcher, &stages[index], + &metrics->publication_ns)) { + discard_window(stages, count); + return false; + } + commit_completed_fallback_item( + task, stages[index].pair.candidate_pair_id, metrics, fallback_cause); + context->parameters.after_candidate_pair_id = + stages[index].pair.candidate_pair_id; + ++*published; + if (!checkpoint_after_publication(task)) { + discard_window(stages, count); + return false; + } + } + return true; +} +#endif + static bool checkpoint_batch(Lardon3DTask *task, Lardon3DMatcherTaskContext *context, unsigned int progress, uint64_t processed) { @@ -325,15 +1124,24 @@ static bool checkpoint_batch(Lardon3DTask *task, static bool run(Lardon3DTask *task, void *userdata) { Lardon3DMatcherTaskContext *context = userdata; uint64_t total_processed = 0; + struct timespec durable_cycle_begin; + bool durable_cycle_timing_known = + clock_gettime(CLOCK_MONOTONIC, &durable_cycle_begin) == 0; for (;;) { if (!lardon3d_task_checkpoint(task)) { return false; } Lardon3DTaskExecutionContract contract; + Lardon3DResourceCapabilitySelection execution_selection; if (!lardon3d_task_execution_contract(task, &contract) || + !lardon3d_task_internal_execution_selection( + task, &execution_selection) || contract.batch_size < LARDON3D_MATCHER_TASK_MINIMUM_BATCH || - contract.batch_size > LARDON3D_MATCHER_TASK_MAXIMUM_BATCH) { + contract.batch_size > LARDON3D_MATCHER_TASK_MAXIMUM_BATCH || + execution_selection.inflight_limit == 0 || + execution_selection.inflight_limit > + LARDON3D_ORB_VULKAN_MAX_INFLIGHT) { return fail_task(task, "Contrat de lot Matcher invalide."); } @@ -359,14 +1167,47 @@ static bool run(Lardon3DTask *task, void *userdata) { } unsigned int previous_opencv_threads = lardon3d_feature_opencv_thread_count(); if (!lardon3d_feature_opencv_configure_threads(1)) { + (void)lardon3d_feature_opencv_configure_threads(previous_opencv_threads); return fail_task(task, "Configuration OpenCV Matcher impossible."); } size_t processed_in_batch = 0; bool batch_ok = true; + bool sequence_used_cpu = false; + bool sequence_used_vulkan = false; + bool sequence_backend_failed = false; + bool sequence_vulkan_ineligible = false; + bool sequence_vulkan_other_failure = false; Lardon3DMatcherParams matcher = { .kind = (Lardon3DMatcherKind)context->parameters.matcher_kind, .ratio_threshold = context->parameters.ratio_threshold, }; +#ifdef LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE + bool sequence_vulkan_synchronous = + context->benchmark_synchronous_pipeline; +#else + bool sequence_vulkan_synchronous = false; +#endif + bool sequence_vulkan_requested = context->orb_vulkan_backend != NULL + && contract.gpu_slots == 1 + && matcher.kind == LARDON3D_MATCHER_ORB_BF; + bool sequence_capacity_lease = false; + bool sequence_capacity_ready = true; + if (sequence_vulkan_requested && !sequence_vulkan_synchronous) { + sequence_capacity_lease = + lardon3d_orb_vulkan_internal_begin_sequence( + context->orb_vulkan_backend, + (uint32_t)execution_selection.inflight_limit); + if (!sequence_capacity_lease) { + sequence_capacity_ready = false; + note_vulkan_backend_failure(context, &sequence_backend_failed); + } + } + Lardon3DResourceExecutionMetrics execution_metrics = {0}; + Lardon3DOrbVulkanTelemetry vulkan_before = {0}; + Lardon3DOrbVulkanTelemetry vulkan_after = {0}; + bool vulkan_telemetry_known = context->orb_vulkan_backend + && lardon3d_orb_vulkan_internal_telemetry( + context->orb_vulkan_backend, &vulkan_before); while (processed_in_batch < batch_count && batch_ok) { size_t remaining = batch_count - processed_in_batch; size_t window_count = (size_t)contract.cpu_threads * MATCHER_TASK_WINDOW_PER_THREAD; @@ -382,29 +1223,111 @@ static bool run(Lardon3DTask *task, void *userdata) { break; } } - /* The Queue callback is one admitted compute participant. At most - * cpu_threads-1 children compute private stages, all are joined before - * ordered publication and before reservation release/sequence_break. */ - if (batch_ok && !compute_window(context, &matcher, stages, window_count, - contract.cpu_threads)) { - batch_ok = false; - } - for (size_t index = 0; index < window_count && batch_ok; ++index) { - if (!lardon3d_task_checkpoint(task) || - stages[index].computed != LARDON3D_MATCHER_OK || - !publish_pair(task, context, &matcher, &stages[index])) { - batch_ok = false; - break; + /* Only the GPU-admitted one-owner path may use a pending Vulkan slot. + * CPU mode remains the established parallel staging path. */ + bool vulkan_selected = context->orb_vulkan_backend != NULL && + contract.gpu_slots == 1 && + matcher.kind == LARDON3D_MATCHER_ORB_BF; + bool vulkan_synchronous = + vulkan_selected && sequence_vulkan_synchronous; + bool vulkan_rolling = vulkan_selected && !vulkan_synchronous + && sequence_capacity_ready; +#ifdef LARDON3D_MATCHER_TASK_TESTING + const char *force_fallback = getenv("LARDON3D_TEST_MATCHER_FORCE_FALLBACK"); + if (force_fallback && strcmp(force_fallback, "1") == 0) { + if (vulkan_rolling) { + atomic_fetch_add(&test_forced_fallbacks, 1); + } + vulkan_rolling = false; + if (contract.gpu_slots == 1) { + note_vulkan_backend_failure(context, &sequence_backend_failed); + } + } +#endif + if (batch_ok && vulkan_rolling) { + size_t published = 0; + batch_ok = compute_publish_vulkan_window(task, context, &matcher, stages, + window_count, &published, + &sequence_used_cpu, + &sequence_used_vulkan, + &sequence_backend_failed, + &sequence_vulkan_ineligible, + &sequence_vulkan_other_failure, + execution_selection.inflight_limit, + &execution_metrics); + processed_in_batch += published; + total_processed += published; + } +#ifdef LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE + else if (batch_ok && vulkan_synchronous) { + size_t published = 0; + batch_ok = compute_publish_synchronous_vulkan_window( + task, context, &matcher, stages, window_count, &published, + &sequence_used_cpu, &sequence_used_vulkan, + &sequence_backend_failed, &sequence_vulkan_ineligible, + &execution_metrics); + processed_in_batch += published; + total_processed += published; + } +#endif + else { + /* CPU stages remain parallel private computation followed by ordered + * owner publication. Vulkan uses the rolling path above because its + * successor must be submitted before this prefix is made durable. */ + if (batch_ok) { + struct timespec cpu_begin; + struct timespec cpu_end; + (void)clock_gettime(CLOCK_MONOTONIC, &cpu_begin); + if (!compute_window(context, &matcher, stages, window_count, + contract.cpu_threads)) { + batch_ok = false; + } + (void)clock_gettime(CLOCK_MONOTONIC, &cpu_end); + saturating_add_ns(&execution_metrics.matcher_cpu_ns, + elapsed_ns(cpu_begin, cpu_end)); + } + if (batch_ok) { + sequence_used_cpu = true; + } + Lardon3DMatcherFallbackCause fallback_cause = + contract.gpu_slots != 1 + ? MATCHER_FALLBACK_NONE + : sequence_backend_failed + ? MATCHER_FALLBACK_BACKEND_FAILURE + : MATCHER_FALLBACK_OTHER; + for (size_t index = 0; index < window_count && batch_ok; ++index) { + if (!lardon3d_task_checkpoint(task) || + stages[index].computed != LARDON3D_MATCHER_OK || + !publish_pair(task, context, &matcher, &stages[index], + &execution_metrics.publication_ns)) { + batch_ok = false; + break; + } + /* Cursor movement follows only the durable, ascending publication + * prefix. A failed stage and every later stage remain unpublished. */ + context->parameters.after_candidate_pair_id = stages[index].pair.candidate_pair_id; + commit_completed_fallback_item( + task, stages[index].pair.candidate_pair_id, + &execution_metrics, fallback_cause); + ++processed_in_batch; + ++total_processed; + if (!checkpoint_after_publication(task)) { + batch_ok = false; + break; + } } - /* Cursor movement follows only the durable, ascending publication - * prefix. A failed stage and every later stage remain unpublished. */ - context->parameters.after_candidate_pair_id = stages[index].pair.candidate_pair_id; - ++processed_in_batch; - ++total_processed; } discard_window(stages, window_count); } - (void)lardon3d_feature_opencv_configure_threads(previous_opencv_threads); + bool sequence_capacity_released = !sequence_capacity_lease + || lardon3d_orb_vulkan_internal_end_sequence( + context->orb_vulkan_backend); + if (!lardon3d_feature_opencv_configure_threads(previous_opencv_threads)) { + return fail_task(task, "Restauration OpenCV Matcher impossible."); + } + if (!sequence_capacity_released) { + return fail_task(task, "Libération capacité Vulkan Matcher impossible."); + } if (!batch_ok) { Lardon3DTaskSnapshot snapshot; if (lardon3d_task_snapshot(task, &snapshot) && snapshot.state != TASK_FAILED) { @@ -413,15 +1336,72 @@ static bool run(Lardon3DTask *task, void *userdata) { return false; } (void)clock_gettime(CLOCK_MONOTONIC, &end); + uint64_t execution_wall_ns = elapsed_ns(begin, end); + if (vulkan_telemetry_known + && lardon3d_orb_vulkan_internal_telemetry( + context->orb_vulkan_backend, &vulkan_after)) { + add_vulkan_telemetry_delta( + &vulkan_before, &vulkan_after, &execution_metrics); + } (void)lardon3d_resource_governor_record_batch( context->governor, LARDON3D_RESOURCE_TASK_CPU, batch_count, - elapsed_ns(begin, end), 0); - + execution_wall_ns, 0); + Lardon3DResourceBackend actual_backend = sequence_used_cpu + && sequence_used_vulkan + ? LARDON3D_RESOURCE_BACKEND_MIXED + : sequence_used_vulkan + ? LARDON3D_RESOURCE_BACKEND_ORB_VULKAN + : LARDON3D_RESOURCE_BACKEND_CPU; + const char *backend_reason = actual_backend == + LARDON3D_RESOURCE_BACKEND_MIXED + ? sequence_backend_failed + ? "vulkan-and-whole-pair-cpu-fallback" + : sequence_vulkan_other_failure + ? "vulkan-and-local-failure-cpu-fallback" + : "vulkan-and-ineligible-pair-cpu-fallback" + : actual_backend == LARDON3D_RESOURCE_BACKEND_CPU + && contract.gpu_slots == 1 + ? sequence_backend_failed + ? "vulkan-failed-whole-pair-cpu-fallback" + : sequence_vulkan_other_failure + ? "vulkan-local-failure-whole-pair-cpu-fallback" + : sequence_vulkan_ineligible + ? "vulkan-ineligible-whole-pair-cpu-fallback" + : "gpu-selected-cpu-completed" + : actual_backend == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN + ? "vulkan-completed" + : "cpu-completed"; bool exhausted = count <= contract.batch_size; unsigned int progress = exhausted ? 100U : 99U; if (!checkpoint_batch(task, context, progress, total_processed)) { return fail_task(task, "Checkpoint Matcher impossible."); } + struct timespec durable_cycle_end; + uint64_t durable_cycle_wall_ns = execution_wall_ns; + if (durable_cycle_timing_known + && clock_gettime(CLOCK_MONOTONIC, &durable_cycle_end) == 0) { + uint64_t measured = elapsed_ns(durable_cycle_begin, durable_cycle_end); + if (measured != 0) durable_cycle_wall_ns = measured; + } + /* CONTRACT: GPU batch adaptation optimizes the durable Task cadence, not + * only shader/CPU execution. From the second sequence onward this interval + * begins immediately before sequence_break, so it includes the successful + * Governor observation/admission plus computation, owner publication and + * the durable generic Matcher checkpoint. Failed checkpoints never train + * the next immutable contract. Scientific output and cursor identity stay + * independent of this operational clock. */ + (void)lardon3d_task_internal_record_sequence_execution_metrics( + task, durable_cycle_wall_ns, processed_in_batch, actual_backend, + backend_reason, &execution_metrics); +#ifdef LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE + if (context->benchmark_inflight_override != 0 && sequence_backend_failed) { + /* Whole-pair CPU fallback remains canonical and any publication above is + * already durable/checkpointed. It cannot, however, turn a failed Vulkan + * A/B cohort into a successful CPU measurement. Fail only the private + * benchmark Task; normal AUTO retains its established fallback result. */ + return fail_task(task, "Échec backend Vulkan pendant le benchmark A/B."); + } +#endif if (exhausted) { return lardon3d_task_set_progress(task, 100, "Matching terminé."); } @@ -438,6 +1418,8 @@ static bool run(Lardon3DTask *task, void *userdata) { #endif Lardon3DResourceReservation *reservation = NULL; + durable_cycle_timing_known = + clock_gettime(CLOCK_MONOTONIC, &durable_cycle_begin) == 0; if (!lardon3d_task_sequence_break(task, context->governor, &reservation, &contract)) { return false; @@ -484,6 +1466,37 @@ make_context(const Lardon3DTaskReconstructionContext *runtime, context->database = runtime->project_db; context->governor = runtime->resource_governor; context->orb_vulkan_backend = runtime->orb_vulkan_backend; +#ifdef LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE + const char *benchmark_pipeline = + getenv(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV); + const char *benchmark_inflight = + getenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV); + const char *benchmark_batch = + getenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV); + bool valid_batch = !benchmark_batch || strcmp(benchmark_batch, "2") == 0 || + strcmp(benchmark_batch, "4") == 0 || + strcmp(benchmark_batch, "8") == 0 || + strcmp(benchmark_batch, "12") == 0; + if ((benchmark_pipeline && strcmp(benchmark_pipeline, "1") != 0) || + (benchmark_inflight && strcmp(benchmark_inflight, "1") != 0 && + strcmp(benchmark_inflight, "2") != 0) || + !valid_batch || (benchmark_batch && !benchmark_inflight) || + (benchmark_pipeline && benchmark_batch) || + (benchmark_pipeline && benchmark_inflight && + strcmp(benchmark_inflight, "2") == 0)) { + /* Invalid inherited benchmark controls fail before any Task callback can + * run. Production builds do not contain these strings or this branch. */ + free(context); + return NULL; + } + context->benchmark_synchronous_pipeline = benchmark_pipeline != NULL; + context->benchmark_inflight_override = !benchmark_inflight + ? 0 : strcmp(benchmark_inflight, "1") == 0 ? 1 : 2; + context->benchmark_batch_override = !benchmark_batch + ? 0 : strcmp(benchmark_batch, "2") == 0 ? 2 + : strcmp(benchmark_batch, "4") == 0 ? 4 + : strcmp(benchmark_batch, "8") == 0 ? 8 : 12; +#endif context->parameters = *parameters; return context; } @@ -517,29 +1530,67 @@ bool lardon3d_matcher_task_reconstruct( } const Lardon3DResourceEstimate cpu = matcher_estimate( LARDON3D_MATCHER_TASK_MODE_CPU_PARALLEL); + Lardon3DResourceEstimate automatic = cpu; + automatic.task_class = LARDON3D_RESOURCE_TASK_MIXED; const Lardon3DResourceEstimate vulkan = matcher_estimate(LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN); const Lardon3DResourceEstimate legacy_cpu = legacy_matcher_estimate(false); const Lardon3DResourceEstimate legacy_vulkan = legacy_matcher_estimate(true); + const Lardon3DResourceEstimate previous_cpu = previous_matcher_estimate(false); + const Lardon3DResourceEstimate previous_vulkan = previous_matcher_estimate(true); bool current_cpu = estimate_equals(&snapshot->estimate, &cpu); + bool current_auto = estimate_equals(&snapshot->estimate, &automatic); bool current_vulkan = estimate_equals(&snapshot->estimate, &vulkan); bool historical_cpu = estimate_equals(&snapshot->estimate, &legacy_cpu); bool historical_vulkan = estimate_equals(&snapshot->estimate, &legacy_vulkan); - bool vulkan_mode = current_vulkan || historical_vulkan; - if ((!current_cpu && !current_vulkan && !historical_cpu && - !historical_vulkan) || - (vulkan_mode && configuration.matcher.kind != LARDON3D_MATCHER_ORB_BF)) { + bool previous_cpu_mode = estimate_equals(&snapshot->estimate, &previous_cpu); + bool previous_vulkan_mode = estimate_equals(&snapshot->estimate, &previous_vulkan); + bool vulkan_mode = current_vulkan || historical_vulkan || previous_vulkan_mode; + if ((!current_auto && !current_cpu && !current_vulkan && !historical_cpu && + !historical_vulkan && + !previous_cpu_mode && !previous_vulkan_mode) || + ((vulkan_mode || current_auto) && + configuration.matcher.kind != LARDON3D_MATCHER_ORB_BF)) { return false; } Lardon3DMatcherTaskContext *context = make_context(runtime, ¶meters); if (!context) { return false; } - /* Exact whole-estimate signatures select the operational mode. This avoids - * guessing backend identity from one field and rejects neighboring malformed - * snapshots. A missing backend after restart remains the already validated - * exact CPU fallback, while the immutable GPU reservation is retained. */ - if (!vulkan_mode) { + bool auto_cpu_mode = current_auto; + bool backend_available = false; + if (auto_cpu_mode) { + Lardon3DTaskCapabilityEnvelope runtime_probe_envelope = + matcher_auto_envelope(&cpu, true, true, 0, 0); + bool hardware_safe = + lardon3d_resource_governor_internal_capability_hardware_safe( + runtime->resource_governor, + &runtime_probe_envelope.capabilities[0]); + /* Only normal AUTO recovery owns runtime eligibility reconstruction. + * Fixed CPU/Vulkan and historical overrides must be order-independent and + * cannot clear shared AUTO state. This metadata check performs no Vulkan + * call; first initialization still belongs to Queue's constrained worker. */ + backend_available = LARDON3D_HAVE_VULKAN && hardware_safe && + auto_vulkan_backend_candidate( + runtime->orb_vulkan_backend); + (void)lardon3d_resource_governor_internal_set_backend_available( + runtime->resource_governor, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, + backend_available); + } + context->normal_auto = auto_cpu_mode; + context->auto_vulkan_available = backend_available; + context->explicit_vulkan = vulkan_mode; +#ifdef LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE + if (context->benchmark_inflight_override != 0 && + (!auto_cpu_mode || !backend_available)) { + free(context); + return false; + } +#endif + /* Exact whole-estimate signatures reject neighboring malformed snapshots. + * New MIXED ORB is normal AUTO. Every CPU-class ORB signature is restored as + * fixed CPU for explicit/recovery safety; Vulkan and non-ORB forms stay fixed. */ + if (!vulkan_mode && !auto_cpu_mode) { context->orb_vulkan_backend = NULL; } *binding = (Lardon3DTaskKindBinding){ @@ -552,10 +1603,36 @@ bool lardon3d_matcher_task_reconstruct( return true; } -Lardon3DTask *lardon3d_project_create_matcher_task_with_mode( +bool lardon3d_matcher_task_internal_configure_restored( + Lardon3DTask *task, void *userdata) { + Lardon3DMatcherTaskContext *context = userdata; + if (!task || !context) { + return task && context; + } + Lardon3DResourceEstimate cpu; + if (!lardon3d_task_resource_estimate(task, &cpu)) { + return false; + } + bool allow_depth_two = true; + size_t benchmark_inflight_override = 0; + size_t benchmark_batch_override = 0; +#ifdef LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE + allow_depth_two = !context->benchmark_synchronous_pipeline; + benchmark_inflight_override = context->benchmark_inflight_override; + benchmark_batch_override = context->benchmark_batch_override; +#endif + Lardon3DTaskCapabilityEnvelope envelope = context->normal_auto + ? matcher_auto_envelope(&cpu, context->auto_vulkan_available, + allow_depth_two, benchmark_inflight_override, + benchmark_batch_override) + : matcher_fixed_envelope(&cpu, context->explicit_vulkan); + return lardon3d_task_internal_set_capability_envelope(task, &envelope); +} + +static Lardon3DTask *create_matcher_task( Lardon3DAppState *state, const Lardon3DMatcherTaskConfiguration *configuration, - Lardon3DMatcherTaskMode mode, uint64_t *task_id) { + Lardon3DMatcherTaskMode mode, bool automatic, uint64_t *task_id) { if (task_id) { *task_id = 0; } @@ -595,20 +1672,43 @@ Lardon3DTask *lardon3d_project_create_matcher_task_with_mode( .orb_vulkan_backend = state->orb_vulkan_backend, }; bool vulkan_mode = mode == LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN; + bool orb_auto = automatic && + configuration->matcher.kind == LARDON3D_MATCHER_ORB_BF; + bool runtime_vulkan = orb_auto && auto_vulkan_runtime_candidate(state); + bool auto_vulkan = runtime_vulkan; + if (orb_auto) { + /* Availability is Governor-owned runtime state. Portable builds and + * unsafe/unavailable hardware publish false without touching a GPU. */ + (void)lardon3d_resource_governor_internal_set_backend_available( + state->resource_governor, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, + runtime_vulkan); + } Lardon3DMatcherTaskContext *context = make_context(&runtime, ¶meters); if (!context) { return NULL; } +#ifdef LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE + if (context->benchmark_inflight_override != 0 && + (!orb_auto || !auto_vulkan)) { + free(context); + return NULL; + } +#endif /* Execution mode is fixed before admission. The Governor may reduce a * parallel task to one CPU thread, but that CPU-only task still must not use * Vulkan without the GPU resources declared by its immutable estimate. */ - if (!vulkan_mode) { + if (!vulkan_mode && !auto_vulkan) { context->orb_vulkan_backend = NULL; } /* Each staged pair can retain the full bounded Matcher working set until * ordered publication. The selected immutable estimate covers the entire * window and never limits scientific dataset cardinality. */ Lardon3DResourceEstimate estimate = matcher_estimate(mode); + if (orb_auto) { + /* Normal AUTO may execute a complete CPU or Vulkan sequence. MIXED is an + * honest durable resource class, not a backend tag or scientific identity. */ + estimate.task_class = LARDON3D_RESOURCE_TASK_MIXED; + } #ifdef LARDON3D_MATCHER_TASK_TESTING /* Tests may reduce CPU fan-out without selecting a backend. Vulkan remains * reachable only through the explicit public mode selector above. */ @@ -627,7 +1727,26 @@ Lardon3DTask *lardon3d_project_create_matcher_task_with_mode( Lardon3DTask *task = lardon3d_task_create_typed( "Matching Candidate Pairs", &estimate, LARDON3D_MATCHER_TASK_KIND, LARDON3D_MATCHER_TASK_KIND_VERSION, run, context, destroy_context); - if (!task || !lardon3d_task_assign_id(task, id) || + Lardon3DTaskCapabilityEnvelope automatic_envelope; + bool envelope_ready = true; + if (task) { + bool allow_depth_two = true; + size_t benchmark_inflight_override = 0; + size_t benchmark_batch_override = 0; +#ifdef LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE + allow_depth_two = !context->benchmark_synchronous_pipeline; + benchmark_inflight_override = context->benchmark_inflight_override; + benchmark_batch_override = context->benchmark_batch_override; +#endif + automatic_envelope = orb_auto + ? matcher_auto_envelope(&estimate, auto_vulkan, allow_depth_two, + benchmark_inflight_override, + benchmark_batch_override) + : matcher_fixed_envelope(&estimate, vulkan_mode); + envelope_ready = lardon3d_task_internal_set_capability_envelope( + task, &automatic_envelope); + } + if (!task || !envelope_ready || !lardon3d_task_assign_id(task, id) || !lardon3d_task_set_finished_callback(task, finished_callback, context) || lardon3d_project_checkpoint_matcher_task(state, task, ¶meters) != LARDON3D_PROJECT_TASK_CHECKPOINT_OK) { @@ -638,11 +1757,19 @@ Lardon3DTask *lardon3d_project_create_matcher_task_with_mode( return task; } +Lardon3DTask *lardon3d_project_create_matcher_task_with_mode( + Lardon3DAppState *state, + const Lardon3DMatcherTaskConfiguration *configuration, + Lardon3DMatcherTaskMode mode, uint64_t *task_id) { + return create_matcher_task(state, configuration, mode, false, task_id); +} + Lardon3DTask *lardon3d_project_create_matcher_task( Lardon3DAppState *state, const Lardon3DMatcherTaskConfiguration *configuration, uint64_t *task_id) { - return lardon3d_project_create_matcher_task_with_mode( - state, configuration, LARDON3D_MATCHER_TASK_MODE_CPU_PARALLEL, task_id); + return create_matcher_task( + state, configuration, LARDON3D_MATCHER_TASK_MODE_CPU_PARALLEL, true, + task_id); } bool lardon3d_project_enqueue_matcher_task_with_mode( @@ -668,6 +1795,17 @@ bool lardon3d_project_enqueue_matcher_task_with_mode( bool lardon3d_project_enqueue_matcher_task( Lardon3DAppState *state, const Lardon3DMatcherTaskConfiguration *configuration, uint64_t *task_id) { - return lardon3d_project_enqueue_matcher_task_with_mode( - state, configuration, LARDON3D_MATCHER_TASK_MODE_CPU_PARALLEL, task_id); + if (!state || !state->task_queue) { + return false; + } + Lardon3DTask *task = lardon3d_project_create_matcher_task( + state, configuration, task_id); + if (!task) { + return false; + } + if (!lardon3d_task_queue_add(state->task_queue, task, NULL)) { + lardon3d_task_destroy(task); + return false; + } + return true; } diff --git a/src/matcher_task_benchmark_internal.h b/src/matcher_task_benchmark_internal.h new file mode 100644 index 0000000..b9075aa --- /dev/null +++ b/src/matcher_task_benchmark_internal.h @@ -0,0 +1,16 @@ +#ifndef LARDON3D_MATCHER_TASK_BENCHMARK_INTERNAL_H +#define LARDON3D_MATCHER_TASK_BENCHMARK_INTERNAL_H + +/* This token exists only in the opt-in real-corpus runner and dedicated + * Matcher tests. Production matcher_task.c is compiled without the macro and + * therefore cannot observe or retain the benchmark pipeline selection. */ +#ifdef LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE +#define LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV \ + "LARDON3D_BENCHMARK_MATCHER_SYNCHRONOUS_FENCE_V1" +#define LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV \ + "LARDON3D_BENCHMARK_MATCHER_INFLIGHT_V1" +#define LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV \ + "LARDON3D_BENCHMARK_MATCHER_BATCH_V1" +#endif + +#endif diff --git a/src/opencv_task_thread_control.h b/src/opencv_task_thread_control.h new file mode 100644 index 0000000..c1310ef --- /dev/null +++ b/src/opencv_task_thread_control.h @@ -0,0 +1,46 @@ +#ifndef LARDON3D_OPENCV_TASK_THREAD_CONTROL_H +#define LARDON3D_OPENCV_TASK_THREAD_CONTROL_H + +#include + +#include +#include + +typedef struct { + unsigned int previous; + bool restore_required; +} Lardon3DOpenCvTaskThreadControl; + +/* OpenCV owns one process-wide CPU pool. Queue's single callback owner makes + * the change race-free, but configure may mutate before its verification + * fails. Therefore begin attempts rollback on every post-capture failure and + * end restores on every callback result. */ +static inline bool lardon3d_opencv_task_threads_begin( + Lardon3DTask *task, unsigned int validated_maximum, + Lardon3DOpenCvTaskThreadControl *control) { + if (!task || !control || validated_maximum == 0) return false; + *control = (Lardon3DOpenCvTaskThreadControl){0}; + Lardon3DTaskExecutionContract contract; + if (!lardon3d_task_execution_contract(task, &contract) || + contract.cpu_threads == 0 || contract.cpu_threads > validated_maximum) { + return false; + } + control->previous = lardon3d_feature_opencv_thread_count(); + control->restore_required = true; + if (!lardon3d_feature_opencv_configure_threads(contract.cpu_threads)) { + /* Verification failure is after a possibly successful setNumThreads(). */ + (void)lardon3d_feature_opencv_configure_threads(control->previous); + control->restore_required = false; + return false; + } + return true; +} + +static inline bool lardon3d_opencv_task_threads_end( + Lardon3DOpenCvTaskThreadControl *control) { + if (!control || !control->restore_required) return false; + control->restore_required = false; + return lardon3d_feature_opencv_configure_threads(control->previous); +} + +#endif diff --git a/src/opencv_task_thread_guard.h b/src/opencv_task_thread_guard.h new file mode 100644 index 0000000..d25a7f1 --- /dev/null +++ b/src/opencv_task_thread_guard.h @@ -0,0 +1,69 @@ +#ifndef LARDON3D_OPENCV_TASK_THREAD_GUARD_H +#define LARDON3D_OPENCV_TASK_THREAD_GUARD_H + +#include + +extern "C" { +#include +} + +#include + +/* OpenCV's thread count is process-wide. Queue's single execution owner makes + * this scoped set/restore safe for Task callbacks: the admitted count is + * applied for the whole callback and the previous runtime setting is restored + * on success, failure, cancellation, and C++ exception unwinding. */ +class Lardon3DOpenCvTaskThreadGuard { + public: + explicit Lardon3DOpenCvTaskThreadGuard(Lardon3DTask *task) noexcept { + try { + Lardon3DTaskExecutionContract contract{}; + previous_ = cv::getNumThreads(); + previous_known_ = true; + valid_ = lardon3d_task_execution_contract(task, &contract) && + contract.cpu_threads > 0 && contract.cpu_threads <= INT_MAX; + if (valid_) { + cv::setNumThreads(static_cast(contract.cpu_threads)); + valid_ = cv::getNumThreads() == static_cast(contract.cpu_threads); + } + } catch (...) { + valid_ = false; + } + } + + ~Lardon3DOpenCvTaskThreadGuard() noexcept { + (void)restore(); + } + + bool restore() noexcept { + if (restored_) + return restore_succeeded_; + if (!previous_known_) + return false; + try { + cv::setNumThreads(previous_ > 0 ? previous_ : 1); + restored_ = true; + restore_succeeded_ = + cv::getNumThreads() == (previous_ > 0 ? previous_ : 1); + return restore_succeeded_; + } catch (...) { + /* C callback boundary: restoration failure cannot escape as C++. */ + return false; + } + } + + Lardon3DOpenCvTaskThreadGuard(const Lardon3DOpenCvTaskThreadGuard &) = delete; + Lardon3DOpenCvTaskThreadGuard &operator=( + const Lardon3DOpenCvTaskThreadGuard &) = delete; + + bool valid() const noexcept { return valid_; } + + private: + int previous_{}; + bool previous_known_{}; + bool valid_{}; + bool restored_{}; + bool restore_succeeded_{}; +}; + +#endif diff --git a/src/orb_vulkan_backend.cpp b/src/orb_vulkan_backend.cpp index b7893b3..328badb 100644 --- a/src/orb_vulkan_backend.cpp +++ b/src/orb_vulkan_backend.cpp @@ -9,6 +9,7 @@ #include #include "matcher_vulkan_config.h" +#include "orb_vulkan_backend_internal.h" #if LARDON3D_HAVE_VULKAN @@ -61,6 +62,22 @@ struct RawTop2 { uint32_t second_distance; }; +struct OrbRequestSlot { + VkCommandBuffer command_buffer = VK_NULL_HANDLE; + VkFence completion_fence = VK_NULL_HANDLE; + VkDescriptorSet descriptor_set = VK_NULL_HANDLE; + VkQueryPool query_pool = VK_NULL_HANDLE; + Buffer descriptors_a; + Buffer descriptors_b; + Buffer output; + bool payload_allocated = false; + bool completion_pending = false; + uint32_t pending_feature_count_a = 0; + uint32_t pending_feature_count_b = 0; + uint64_t generation = 0; + bool generation_retired = false; +}; + #ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY struct RawSiftTop2 { uint32_t best_index; @@ -113,6 +130,13 @@ static bool has_validation_layer() { } // namespace struct Lardon3DOrbVulkanBackend { + /* Public top2 is one synchronous transaction even though its private begin + * and finish deliberately release request-state ownership between calls. + * Lock order is always synchronous_transaction_mutex -> mutex. Private + * async begin/finish/discard and info take only mutex, so Matcher may keep a + * request in flight across publication without holding this transaction + * lock or deadlocking observation. */ + std::mutex synchronous_transaction_mutex; std::mutex mutex; BackendState state = BackendState::kUninitialized; VkInstance instance = VK_NULL_HANDLE; @@ -127,7 +151,19 @@ struct Lardon3DOrbVulkanBackend { VkPhysicalDeviceFeatures features{}; #endif VkCommandPool command_pool = VK_NULL_HANDLE; - VkCommandBuffer command_buffer = VK_NULL_HANDLE; + /* CONTRACT: the device, queue, command pool, pipeline and layouts are shared + * immutable backend state. Only the bounded request slots duplicate command, + * fence, descriptor, mapped input/readback and timestamp resources. A slot + * generation makes private completion ownership request-bound. */ + OrbRequestSlot slots[LARDON3D_ORB_VULKAN_MAX_INFLIGHT]; + /* Mapped request payload follows the frozen sequence admission. Retained + * count, rather than slot index, is authoritative because an exhausted + * generation retires that slot permanently and depth one must retain the + * other usable slot. Command/fence/descriptor/query objects remain bounded + * session metadata and never imply retained mapped payload. */ + uint32_t configured_capacity = 1; + uint32_t retained_capacity = 0; + bool sequence_capacity_active = false; VkDescriptorSetLayout descriptor_set_layout = VK_NULL_HANDLE; VkPipelineLayout pipeline_layout = VK_NULL_HANDLE; VkPipeline pipeline = VK_NULL_HANDLE; @@ -135,15 +171,13 @@ struct Lardon3DOrbVulkanBackend { VkPipeline sift_pipeline = VK_NULL_HANDLE; #endif VkDescriptorPool descriptor_pool = VK_NULL_HANDLE; - VkDescriptorSet descriptor_set = VK_NULL_HANDLE; #ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY + VkCommandBuffer sift_command_buffer = VK_NULL_HANDLE; + VkFence sift_completion_fence = VK_NULL_HANDLE; VkDescriptorSet sift_descriptor_set = VK_NULL_HANDLE; + VkQueryPool sift_query_pool = VK_NULL_HANDLE; #endif - VkQueryPool query_pool = VK_NULL_HANDLE; bool timestamps_available = false; - Buffer descriptors_a; - Buffer descriptors_b; - Buffer output; #ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY Buffer sift_descriptors_a; Buffer sift_descriptors_b; @@ -153,10 +187,24 @@ struct Lardon3DOrbVulkanBackend { uint64_t initialization_ns = 0; uint64_t last_dispatch_ns = 0; uint64_t last_gpu_ns = 0; + /* Telemetry is monotonic, bounded, and request-state owned. Saturation is + * explicit: measurement can stop gaining precision after UINT64_MAX, but + * must never wrap into a false low-utilization control signal. */ + Lardon3DOrbVulkanTelemetry telemetry{}; + bool completion_observed = false; + std::chrono::steady_clock::time_point completion_observed_at{}; }; namespace { +static void saturating_add(uint64_t *value, uint64_t increment) { + *value = *value > UINT64_MAX - increment ? UINT64_MAX : *value + increment; +} + +static void telemetry_event(Lardon3DOrbVulkanBackend *backend) { + saturating_add(&backend->telemetry.serial, 1); +} + static void destroy_buffer(Lardon3DOrbVulkanBackend *backend, Buffer *buffer) { if (!backend || !buffer || backend->device == VK_NULL_HANDLE) { return; @@ -180,17 +228,24 @@ static void destroy_vulkan(Lardon3DOrbVulkanBackend *backend) { if (backend->device != VK_NULL_HANDLE) { (void)vkDeviceWaitIdle(backend->device); } - destroy_buffer(backend, &backend->descriptors_a); - destroy_buffer(backend, &backend->descriptors_b); - destroy_buffer(backend, &backend->output); + for (OrbRequestSlot &slot : backend->slots) { + destroy_buffer(backend, &slot.descriptors_a); + destroy_buffer(backend, &slot.descriptors_b); + destroy_buffer(backend, &slot.output); + if (slot.query_pool != VK_NULL_HANDLE) { + vkDestroyQueryPool(backend->device, slot.query_pool, nullptr); + } + } #ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY destroy_buffer(backend, &backend->sift_descriptors_a); destroy_buffer(backend, &backend->sift_descriptors_b); destroy_buffer(backend, &backend->sift_output); #endif - if (backend->query_pool != VK_NULL_HANDLE) { - vkDestroyQueryPool(backend->device, backend->query_pool, nullptr); +#ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY + if (backend->sift_query_pool != VK_NULL_HANDLE) { + vkDestroyQueryPool(backend->device, backend->sift_query_pool, nullptr); } +#endif if (backend->pipeline != VK_NULL_HANDLE) { vkDestroyPipeline(backend->device, backend->pipeline, nullptr); } @@ -211,6 +266,16 @@ static void destroy_vulkan(Lardon3DOrbVulkanBackend *backend) { if (backend->command_pool != VK_NULL_HANDLE) { vkDestroyCommandPool(backend->device, backend->command_pool, nullptr); } + for (OrbRequestSlot &slot : backend->slots) { + if (slot.completion_fence != VK_NULL_HANDLE) { + vkDestroyFence(backend->device, slot.completion_fence, nullptr); + } + } +#ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY + if (backend->sift_completion_fence != VK_NULL_HANDLE) { + vkDestroyFence(backend->device, backend->sift_completion_fence, nullptr); + } +#endif if (backend->device != VK_NULL_HANDLE) { vkDestroyDevice(backend->device, nullptr); } @@ -222,7 +287,12 @@ static void destroy_vulkan(Lardon3DOrbVulkanBackend *backend) { backend->device = VK_NULL_HANDLE; backend->queue = VK_NULL_HANDLE; backend->command_pool = VK_NULL_HANDLE; - backend->command_buffer = VK_NULL_HANDLE; + for (OrbRequestSlot &slot : backend->slots) { + slot = OrbRequestSlot{}; + } + backend->configured_capacity = 1; + backend->retained_capacity = 0; + backend->sequence_capacity_active = false; backend->descriptor_set_layout = VK_NULL_HANDLE; backend->pipeline_layout = VK_NULL_HANDLE; backend->pipeline = VK_NULL_HANDLE; @@ -230,11 +300,12 @@ static void destroy_vulkan(Lardon3DOrbVulkanBackend *backend) { backend->sift_pipeline = VK_NULL_HANDLE; #endif backend->descriptor_pool = VK_NULL_HANDLE; - backend->descriptor_set = VK_NULL_HANDLE; #ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY + backend->sift_command_buffer = VK_NULL_HANDLE; + backend->sift_completion_fence = VK_NULL_HANDLE; backend->sift_descriptor_set = VK_NULL_HANDLE; + backend->sift_query_pool = VK_NULL_HANDLE; #endif - backend->query_pool = VK_NULL_HANDLE; backend->timestamps_available = false; } @@ -387,11 +458,30 @@ static bool create_device_and_commands(Lardon3DOrbVulkanBackend *backend) { command_info.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO; command_info.commandPool = backend->command_pool; command_info.level = VK_COMMAND_BUFFER_LEVEL_PRIMARY; - command_info.commandBufferCount = 1; + VkCommandBuffer command_buffers[LARDON3D_ORB_VULKAN_MAX_INFLIGHT]{}; + command_info.commandBufferCount = LARDON3D_ORB_VULKAN_MAX_INFLIGHT; if (vkAllocateCommandBuffers(backend->device, &command_info, - &backend->command_buffer) != VK_SUCCESS) { + command_buffers) != VK_SUCCESS) { return false; } + VkFenceCreateInfo fence_info{}; + fence_info.sType = VK_STRUCTURE_TYPE_FENCE_CREATE_INFO; + for (uint32_t index = 0; index < LARDON3D_ORB_VULKAN_MAX_INFLIGHT; ++index) { + backend->slots[index].command_buffer = command_buffers[index]; + if (vkCreateFence(backend->device, &fence_info, nullptr, + &backend->slots[index].completion_fence) != VK_SUCCESS) { + return false; + } + } +#ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY + command_info.commandBufferCount = 1; + if (vkAllocateCommandBuffers(backend->device, &command_info, + &backend->sift_command_buffer) != VK_SUCCESS || + vkCreateFence(backend->device, &fence_info, nullptr, + &backend->sift_completion_fence) != VK_SUCCESS) { + return false; + } +#endif return true; } @@ -555,20 +645,21 @@ static bool create_buffer(Lardon3DOrbVulkanBackend *backend, VkDeviceSize size, return true; } -static bool create_buffers_and_descriptors(Lardon3DOrbVulkanBackend *backend) { +static bool create_descriptors_and_queries(Lardon3DOrbVulkanBackend *backend) { VkDescriptorPoolSize pool_size{}; pool_size.type = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER; #ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY - pool_size.descriptorCount = 6; + pool_size.descriptorCount = + 3 * (LARDON3D_ORB_VULKAN_MAX_INFLIGHT + 1); #else - pool_size.descriptorCount = 3; + pool_size.descriptorCount = 3 * LARDON3D_ORB_VULKAN_MAX_INFLIGHT; #endif VkDescriptorPoolCreateInfo pool_info{}; pool_info.sType = VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO; #ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY - pool_info.maxSets = 2; + pool_info.maxSets = LARDON3D_ORB_VULKAN_MAX_INFLIGHT + 1; #else - pool_info.maxSets = 1; + pool_info.maxSets = LARDON3D_ORB_VULKAN_MAX_INFLIGHT; #endif pool_info.poolSizeCount = 1; pool_info.pPoolSizes = &pool_size; @@ -581,30 +672,13 @@ static bool create_buffers_and_descriptors(Lardon3DOrbVulkanBackend *backend) { set_info.descriptorPool = backend->descriptor_pool; set_info.descriptorSetCount = 1; set_info.pSetLayouts = &backend->descriptor_set_layout; - if (vkAllocateDescriptorSets(backend->device, &set_info, - &backend->descriptor_set) != VK_SUCCESS || - !create_buffer(backend, kDescriptorBufferBytes, &backend->descriptors_a) || - !create_buffer(backend, kDescriptorBufferBytes, &backend->descriptors_b) || - !create_buffer(backend, kOutputBufferBytes, &backend->output)) { - return false; + for (OrbRequestSlot &slot : backend->slots) { + if (vkAllocateDescriptorSets(backend->device, &set_info, + &slot.descriptor_set) != VK_SUCCESS) { + return false; + } } - VkDescriptorBufferInfo buffer_info[3] = { - {backend->descriptors_a.buffer, 0, backend->descriptors_a.size}, - {backend->descriptors_b.buffer, 0, backend->descriptors_b.size}, - {backend->output.buffer, 0, backend->output.size}, - }; - VkWriteDescriptorSet writes[3]{}; - for (uint32_t index = 0; index < 3; ++index) { - writes[index].sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET; - writes[index].dstSet = backend->descriptor_set; - writes[index].dstBinding = index; - writes[index].descriptorCount = 1; - writes[index].descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER; - writes[index].pBufferInfo = &buffer_info[index]; - } - vkUpdateDescriptorSets(backend->device, 3, writes, 0, nullptr); - uint32_t family_count = 0; vkGetPhysicalDeviceQueueFamilyProperties(backend->physical_device, &family_count, nullptr); std::vector families(family_count); @@ -618,14 +692,135 @@ static bool create_buffers_and_descriptors(Lardon3DOrbVulkanBackend *backend) { query_info.sType = VK_STRUCTURE_TYPE_QUERY_POOL_CREATE_INFO; query_info.queryType = VK_QUERY_TYPE_TIMESTAMP; query_info.queryCount = 2; - if (vkCreateQueryPool(backend->device, &query_info, nullptr, - &backend->query_pool) != VK_SUCCESS) { + for (OrbRequestSlot &slot : backend->slots) { + if (vkCreateQueryPool(backend->device, &query_info, nullptr, + &slot.query_pool) != VK_SUCCESS) { + backend->timestamps_available = false; + break; + } + } +#ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY + if (backend->timestamps_available && + vkCreateQueryPool(backend->device, &query_info, nullptr, + &backend->sift_query_pool) != VK_SUCCESS) { backend->timestamps_available = false; } +#endif } return true; } +static void release_slot_payload(Lardon3DOrbVulkanBackend *backend, + OrbRequestSlot *slot) { + if (!backend || !slot) return; + destroy_buffer(backend, &slot->descriptors_a); + destroy_buffer(backend, &slot->descriptors_b); + destroy_buffer(backend, &slot->output); + slot->payload_allocated = false; +} + +static bool allocate_slot_payload(Lardon3DOrbVulkanBackend *backend, + uint32_t slot_index) { + if (!backend || slot_index >= LARDON3D_ORB_VULKAN_MAX_INFLIGHT) return false; + OrbRequestSlot *slot = &backend->slots[slot_index]; + if (slot->payload_allocated) return true; + if (slot->generation_retired) return false; +#ifdef LARDON3D_ORB_VULKAN_TESTING + const char *forced_slot = std::getenv( + "LARDON3D_TEST_VULKAN_SLOT_ALLOCATION_FAILURE"); + if (forced_slot && forced_slot[0] == static_cast('0' + slot_index) + && forced_slot[1] == '\0') { + return false; + } +#endif + if (!create_buffer(backend, kDescriptorBufferBytes, &slot->descriptors_a) + || !create_buffer(backend, kDescriptorBufferBytes, &slot->descriptors_b) + || !create_buffer(backend, kOutputBufferBytes, &slot->output)) { + release_slot_payload(backend, slot); + return false; + } + VkDescriptorBufferInfo buffer_info[3] = { + {slot->descriptors_a.buffer, 0, slot->descriptors_a.size}, + {slot->descriptors_b.buffer, 0, slot->descriptors_b.size}, + {slot->output.buffer, 0, slot->output.size}, + }; + VkWriteDescriptorSet writes[3]{}; + for (uint32_t index = 0; index < 3; ++index) { + writes[index].sType = VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET; + writes[index].dstSet = slot->descriptor_set; + writes[index].dstBinding = index; + writes[index].descriptorCount = 1; + writes[index].descriptorType = VK_DESCRIPTOR_TYPE_STORAGE_BUFFER; + writes[index].pBufferInfo = &buffer_info[index]; + } + vkUpdateDescriptorSets(backend->device, 3, writes, 0, nullptr); + slot->payload_allocated = true; + return true; +} + +static bool request_pending_locked(const Lardon3DOrbVulkanBackend *backend) { + for (const OrbRequestSlot &slot : backend->slots) { + if (slot.completion_pending) return true; + } + return false; +} + +static bool resize_payload_locked(Lardon3DOrbVulkanBackend *backend, + uint32_t capacity) { + if (!backend || capacity == 0 + || capacity > LARDON3D_ORB_VULKAN_MAX_INFLIGHT + || request_pending_locked(backend)) { + return false; + } + /* An exhausted generation can never be reset by payload recreation. Retire + * its allocation first; a smaller/default capacity may then retain another + * usable slot without ever reviving an ancient handle. */ + for (OrbRequestSlot &slot : backend->slots) { + if (slot.payload_allocated && slot.generation_retired) { + if (backend->retained_capacity == 0) return false; + release_slot_payload(backend, &slot); + --backend->retained_capacity; + } + } + while (backend->retained_capacity < capacity) { + bool allocated = false; + for (uint32_t index = 0; index < LARDON3D_ORB_VULKAN_MAX_INFLIGHT; + ++index) { + OrbRequestSlot *slot = &backend->slots[index]; + if (!slot->payload_allocated && !slot->generation_retired) { + if (!allocate_slot_payload(backend, index)) return false; + ++backend->retained_capacity; + allocated = true; + break; + } + } + if (!allocated) return false; + } + while (backend->retained_capacity > capacity) { + OrbRequestSlot *release = nullptr; + for (OrbRequestSlot &slot : backend->slots) { + if (slot.payload_allocated && slot.generation_retired) { + release = &slot; + break; + } + } + if (!release) { + for (uint32_t index = LARDON3D_ORB_VULKAN_MAX_INFLIGHT; index > 0; + --index) { + if (backend->slots[index - 1].payload_allocated) { + release = &backend->slots[index - 1]; + break; + } + } + } + if (!release) return false; + release_slot_payload(backend, release); + --backend->retained_capacity; + } + backend->configured_capacity = capacity; + return true; +} + #ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY static bool create_sift_resources_locked(Lardon3DOrbVulkanBackend *backend) { if (backend->sift_pipeline != VK_NULL_HANDLE) { @@ -681,6 +876,18 @@ static bool initialize_locked(Lardon3DOrbVulkanBackend *backend) { if (backend->state != BackendState::kUninitialized) { return false; } + const char *mesa_cache_disabled = std::getenv("MESA_SHADER_CACHE_DISABLE"); + /* CONTRACT: only a process-start boundary may establish this environment. + * A backend call can occur after arbitrary library threads exist, so it must + * never call setenv here. Reject and cache UNAVAILABLE before the first Mesa + * or Vulkan symbol can create affinity-widening disk-cache helpers. Exact + * true/1 are the only process policy values validated on the target host. */ + if (!mesa_cache_disabled + || (std::strcmp(mesa_cache_disabled, "true") != 0 + && std::strcmp(mesa_cache_disabled, "1") != 0)) { + backend->state = BackendState::kUnavailable; + return false; + } const char *disabled = std::getenv("LARDON3D_VULKAN_DISABLE"); if (disabled && std::strcmp(disabled, "1") == 0) { backend->state = BackendState::kUnavailable; @@ -689,7 +896,8 @@ static bool initialize_locked(Lardon3DOrbVulkanBackend *backend) { auto start = std::chrono::steady_clock::now(); bool success = create_instance(backend) && select_device(backend) && create_device_and_commands(backend) && create_pipeline(backend) && - create_buffers_and_descriptors(backend); + create_descriptors_and_queries(backend) && + resize_payload_locked(backend, backend->configured_capacity); backend->initialization_ns = elapsed_ns(start); if (!success) { destroy_vulkan(backend); @@ -702,6 +910,8 @@ static bool initialize_locked(Lardon3DOrbVulkanBackend *backend) { static Lardon3DOrbVulkanResult fail_session_locked( Lardon3DOrbVulkanBackend *backend) { + saturating_add(&backend->telemetry.failures, 1); + telemetry_event(backend); destroy_vulkan(backend); backend->state = BackendState::kFailed; return LARDON3D_ORB_VULKAN_FAILED; @@ -738,121 +948,132 @@ static VkResult record_and_submit_sift(Lardon3DOrbVulkanBackend *backend, VkPipeline pipeline, VkDescriptorSet descriptor_set, uint32_t count_a, uint32_t count_b) { - VkResult result = vkResetCommandBuffer(backend->command_buffer, 0); + VkCommandBuffer command_buffer = backend->sift_command_buffer; + VkResult result = vkResetCommandBuffer(command_buffer, 0); if (result != VK_SUCCESS) { return result; } VkCommandBufferBeginInfo begin_info{}; begin_info.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO; begin_info.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT; - result = vkBeginCommandBuffer(backend->command_buffer, &begin_info); + result = vkBeginCommandBuffer(command_buffer, &begin_info); if (result != VK_SUCCESS) { return result; } if (backend->timestamps_available) { - vkCmdResetQueryPool(backend->command_buffer, backend->query_pool, 0, 2); - vkCmdWriteTimestamp(backend->command_buffer, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, - backend->query_pool, 0); + vkCmdResetQueryPool(command_buffer, backend->sift_query_pool, 0, 2); + vkCmdWriteTimestamp(command_buffer, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, + backend->sift_query_pool, 0); } - vkCmdBindPipeline(backend->command_buffer, VK_PIPELINE_BIND_POINT_COMPUTE, + vkCmdBindPipeline(command_buffer, VK_PIPELINE_BIND_POINT_COMPUTE, pipeline); - vkCmdBindDescriptorSets(backend->command_buffer, VK_PIPELINE_BIND_POINT_COMPUTE, + vkCmdBindDescriptorSets(command_buffer, VK_PIPELINE_BIND_POINT_COMPUTE, backend->pipeline_layout, 0, 1, &descriptor_set, 0, nullptr); uint32_t counts[2] = {count_a, count_b}; - vkCmdPushConstants(backend->command_buffer, backend->pipeline_layout, + vkCmdPushConstants(command_buffer, backend->pipeline_layout, VK_SHADER_STAGE_COMPUTE_BIT, 0, sizeof(counts), counts); - vkCmdDispatch(backend->command_buffer, count_a, 1, 1); + vkCmdDispatch(command_buffer, count_a, 1, 1); VkMemoryBarrier barrier{}; barrier.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER; barrier.srcAccessMask = VK_ACCESS_SHADER_WRITE_BIT; barrier.dstAccessMask = VK_ACCESS_HOST_READ_BIT; - vkCmdPipelineBarrier(backend->command_buffer, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, + vkCmdPipelineBarrier(command_buffer, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, VK_PIPELINE_STAGE_HOST_BIT, 0, 1, &barrier, 0, nullptr, 0, nullptr); if (backend->timestamps_available) { - vkCmdWriteTimestamp(backend->command_buffer, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, - backend->query_pool, 1); + vkCmdWriteTimestamp(command_buffer, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, + backend->sift_query_pool, 1); } - result = vkEndCommandBuffer(backend->command_buffer); + result = vkEndCommandBuffer(command_buffer); if (result != VK_SUCCESS) { return result; } VkSubmitInfo submit_info{}; submit_info.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; submit_info.commandBufferCount = 1; - submit_info.pCommandBuffers = &backend->command_buffer; - result = vkQueueSubmit(backend->queue, 1, &submit_info, VK_NULL_HANDLE); + submit_info.pCommandBuffers = &command_buffer; + result = vkResetFences(backend->device, 1, &backend->sift_completion_fence); + if (result != VK_SUCCESS) return result; + result = vkQueueSubmit(backend->queue, 1, &submit_info, + backend->sift_completion_fence); if (result != VK_SUCCESS) { return result; } - return vkQueueWaitIdle(backend->queue); + return VK_SUCCESS; } #endif static VkResult record_and_submit(Lardon3DOrbVulkanBackend *backend, + OrbRequestSlot *slot, uint32_t count_a, uint32_t count_b) { - VkResult result = vkResetCommandBuffer(backend->command_buffer, 0); + VkResult result = vkResetCommandBuffer(slot->command_buffer, 0); if (result != VK_SUCCESS) { return result; } VkCommandBufferBeginInfo begin_info{}; begin_info.sType = VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO; begin_info.flags = VK_COMMAND_BUFFER_USAGE_ONE_TIME_SUBMIT_BIT; - result = vkBeginCommandBuffer(backend->command_buffer, &begin_info); + result = vkBeginCommandBuffer(slot->command_buffer, &begin_info); if (result != VK_SUCCESS) { return result; } if (backend->timestamps_available) { - vkCmdResetQueryPool(backend->command_buffer, backend->query_pool, 0, 2); - vkCmdWriteTimestamp(backend->command_buffer, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, - backend->query_pool, 0); + vkCmdResetQueryPool(slot->command_buffer, slot->query_pool, 0, 2); + vkCmdWriteTimestamp(slot->command_buffer, VK_PIPELINE_STAGE_TOP_OF_PIPE_BIT, + slot->query_pool, 0); } - vkCmdBindPipeline(backend->command_buffer, VK_PIPELINE_BIND_POINT_COMPUTE, + vkCmdBindPipeline(slot->command_buffer, VK_PIPELINE_BIND_POINT_COMPUTE, backend->pipeline); - vkCmdBindDescriptorSets(backend->command_buffer, VK_PIPELINE_BIND_POINT_COMPUTE, + vkCmdBindDescriptorSets(slot->command_buffer, VK_PIPELINE_BIND_POINT_COMPUTE, backend->pipeline_layout, 0, 1, - &backend->descriptor_set, 0, nullptr); + &slot->descriptor_set, 0, nullptr); uint32_t counts[2] = {count_a, count_b}; - vkCmdPushConstants(backend->command_buffer, backend->pipeline_layout, + vkCmdPushConstants(slot->command_buffer, backend->pipeline_layout, VK_SHADER_STAGE_COMPUTE_BIT, 0, sizeof(counts), counts); uint32_t groups = (count_a + backend->workgroup_size - 1) / backend->workgroup_size; - vkCmdDispatch(backend->command_buffer, groups, 1, 1); + vkCmdDispatch(slot->command_buffer, groups, 1, 1); VkMemoryBarrier barrier{}; barrier.sType = VK_STRUCTURE_TYPE_MEMORY_BARRIER; barrier.srcAccessMask = VK_ACCESS_SHADER_WRITE_BIT; barrier.dstAccessMask = VK_ACCESS_HOST_READ_BIT; - vkCmdPipelineBarrier(backend->command_buffer, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, + vkCmdPipelineBarrier(slot->command_buffer, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, VK_PIPELINE_STAGE_HOST_BIT, 0, 1, &barrier, 0, nullptr, 0, nullptr); if (backend->timestamps_available) { - vkCmdWriteTimestamp(backend->command_buffer, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, - backend->query_pool, 1); + vkCmdWriteTimestamp(slot->command_buffer, VK_PIPELINE_STAGE_BOTTOM_OF_PIPE_BIT, + slot->query_pool, 1); } - result = vkEndCommandBuffer(backend->command_buffer); + result = vkEndCommandBuffer(slot->command_buffer); if (result != VK_SUCCESS) { return result; } VkSubmitInfo submit_info{}; submit_info.sType = VK_STRUCTURE_TYPE_SUBMIT_INFO; submit_info.commandBufferCount = 1; - submit_info.pCommandBuffers = &backend->command_buffer; - result = vkQueueSubmit(backend->queue, 1, &submit_info, VK_NULL_HANDLE); + submit_info.pCommandBuffers = &slot->command_buffer; + /* A slot fence establishes completion ownership without draining unrelated + * queue work. The synchronous wrapper still waits before exposing output. */ + result = vkResetFences(backend->device, 1, &slot->completion_fence); + if (result != VK_SUCCESS) return result; + result = vkQueueSubmit(backend->queue, 1, &submit_info, + slot->completion_fence); if (result != VK_SUCCESS) { return result; } - return vkQueueWaitIdle(backend->queue); + return VK_SUCCESS; } -static void read_gpu_time(Lardon3DOrbVulkanBackend *backend) { +static void read_gpu_time(Lardon3DOrbVulkanBackend *backend, + VkQueryPool query_pool) { backend->last_gpu_ns = 0; if (!backend->timestamps_available) { return; } uint64_t timestamps[2]{}; VkResult result = vkGetQueryPoolResults( - backend->device, backend->query_pool, 0, 2, sizeof(timestamps), timestamps, + backend->device, query_pool, 0, 2, sizeof(timestamps), timestamps, sizeof(uint64_t), VK_QUERY_RESULT_64_BIT | VK_QUERY_RESULT_WAIT_BIT); if (result == VK_SUCCESS && timestamps[1] >= timestamps[0]) { double nanoseconds = static_cast(timestamps[1] - timestamps[0]) * @@ -864,7 +1085,11 @@ static void read_gpu_time(Lardon3DOrbVulkanBackend *backend) { } // namespace extern "C" Lardon3DOrbVulkanBackend *lardon3d_orb_vulkan_backend_create(void) { - return new (std::nothrow) Lardon3DOrbVulkanBackend(); + try { + return new (std::nothrow) Lardon3DOrbVulkanBackend(); + } catch (...) { + return nullptr; + } } extern "C" void lardon3d_orb_vulkan_backend_destroy( @@ -872,11 +1097,21 @@ extern "C" void lardon3d_orb_vulkan_backend_destroy( if (!backend) { return; } - { - std::lock_guard lock(backend->mutex); - destroy_vulkan(backend); + try { + { + /* Destruction follows the public wrapper's lock order and cannot tear + * down a session in the middle of one synchronous begin->finish call. + * As for every destroy API, callers still own exclusion from future use. */ + std::lock_guard transaction_lock( + backend->synchronous_transaction_mutex); + std::lock_guard state_lock(backend->mutex); + destroy_vulkan(backend); + } + delete backend; + } catch (...) { + /* Destruction is a C ABI boundary. A synchronization exception must not + * escape; retaining an unusable backend is safer than an unlocked delete. */ } - delete backend; } extern "C" bool lardon3d_orb_vulkan_should_use(uint32_t feature_count_a, @@ -890,39 +1125,189 @@ extern "C" bool lardon3d_orb_vulkan_should_use(uint32_t feature_count_a, kDefaultVulkanWorkThreshold; } +static Lardon3DOrbVulkanResult orb_vulkan_top2_begin_impl( + Lardon3DOrbVulkanBackend *backend, const unsigned char *descriptors_a, + uint32_t feature_count_a, const unsigned char *descriptors_b, + uint32_t feature_count_b, Lardon3DOrbVulkanRequest *request, + bool private_sequence_request); + +extern "C" bool lardon3d_orb_vulkan_internal_begin_sequence( + Lardon3DOrbVulkanBackend *backend, uint32_t inflight_capacity) { + if (!backend || inflight_capacity == 0 + || inflight_capacity > LARDON3D_ORB_VULKAN_MAX_INFLIGHT) { + return false; + } + try { + std::lock_guard lock(backend->mutex); + if (backend->sequence_capacity_active || request_pending_locked(backend)) { + return false; + } + if (backend->state == BackendState::kUninitialized) { + backend->configured_capacity = inflight_capacity; + } else if (backend->state == BackendState::kAvailable) { + if (!resize_payload_locked(backend, inflight_capacity)) return false; + } else { + return false; + } + backend->sequence_capacity_active = true; + return true; + } catch (...) { + return false; + } +} + +extern "C" bool lardon3d_orb_vulkan_internal_end_sequence( + Lardon3DOrbVulkanBackend *backend) { + if (!backend) return false; + try { + std::lock_guard lock(backend->mutex); + if (request_pending_locked(backend)) { + return false; + } + if (!backend->sequence_capacity_active) { + /* Session failure destroys payload and clears the lease before Matcher + * reaches its cleanup boundary. Treat that already-complete cleanup as + * success, while rejecting a duplicate end on a healthy backend. */ + return backend->state != BackendState::kAvailable; + } + if (backend->state == BackendState::kAvailable) { + if (!resize_payload_locked(backend, 1)) { + /* Failure to restore the depth-one allocation cannot leave a stale + * sequence lease or ambiguous retained payload. No work is pending at + * this boundary, so failing the session is deterministic cleanup. */ + (void)fail_session_locked(backend); + return false; + } + } else { + backend->configured_capacity = 1; + } + backend->sequence_capacity_active = false; + return true; + } catch (...) { + return false; + } +} + +#ifdef LARDON3D_ORB_VULKAN_TESTING +extern "C" bool lardon3d_orb_vulkan_internal_test_set_slot_generation( + Lardon3DOrbVulkanBackend *backend, uint32_t slot, uint64_t generation) { + if (!backend || slot >= LARDON3D_ORB_VULKAN_MAX_INFLIGHT + || generation == 0) { + return false; + } + try { + std::lock_guard lock(backend->mutex); + OrbRequestSlot *request_slot = &backend->slots[slot]; + if (backend->state != BackendState::kAvailable + || request_slot->completion_pending + || !request_slot->payload_allocated + || request_slot->generation_retired) { + return false; + } + request_slot->generation = generation; + return true; + } catch (...) { + return false; + } +} +#endif + extern "C" Lardon3DOrbVulkanResult lardon3d_orb_vulkan_top2( Lardon3DOrbVulkanBackend *backend, const unsigned char *descriptors_a, uint32_t feature_count_a, const unsigned char *descriptors_b, uint32_t feature_count_b, Lardon3DOrbTop2 *output, size_t output_capacity) { - if (!backend || feature_count_a > LARDON3D_FEATURE_MAX_FEATURES || - feature_count_b > LARDON3D_FEATURE_MAX_FEATURES || - (feature_count_a > 0 && (!descriptors_a || !output || - output_capacity < feature_count_a)) || - (feature_count_b > 0 && !descriptors_b)) { - return LARDON3D_ORB_VULKAN_INVALID_ARGUMENT; - } - if (feature_count_a == 0) { - return LARDON3D_ORB_VULKAN_OK; - } - if (feature_count_b == 0) { - for (uint32_t index = 0; index < feature_count_a; ++index) { - output[index] = Lardon3DOrbTop2{}; + try { + if (!backend || feature_count_a > LARDON3D_FEATURE_MAX_FEATURES || + feature_count_b > LARDON3D_FEATURE_MAX_FEATURES || + (feature_count_a > 0 && (!descriptors_a || !output || + output_capacity < feature_count_a)) || + (feature_count_b > 0 && !descriptors_b)) { + return LARDON3D_ORB_VULKAN_INVALID_ARGUMENT; + } + /* Two public callers must each own an indivisible synchronous request. + * The dedicated transaction lock spans private begin->finish; the private + * request-state mutex remains short-lived so async Matcher overlap is not + * serialized across its publication boundary. */ + std::lock_guard transaction_lock( + backend->synchronous_transaction_mutex); + if (feature_count_a == 0) { + return LARDON3D_ORB_VULKAN_OK; + } + if (feature_count_b == 0) { + for (uint32_t index = 0; index < feature_count_a; ++index) { + output[index] = Lardon3DOrbTop2{}; + } + return LARDON3D_ORB_VULKAN_OK; } - return LARDON3D_ORB_VULKAN_OK; - } + Lardon3DOrbVulkanRequest request{}; + Lardon3DOrbVulkanResult started = orb_vulkan_top2_begin_impl( + backend, descriptors_a, feature_count_a, descriptors_b, + feature_count_b, &request, false); + if (started != LARDON3D_ORB_VULKAN_OK) return started; + return lardon3d_orb_vulkan_internal_top2_finish( + backend, &request, output, output_capacity); + } catch (...) { + return LARDON3D_ORB_VULKAN_FAILED; + } +} + +static Lardon3DOrbVulkanResult orb_vulkan_top2_begin_impl( + Lardon3DOrbVulkanBackend *backend, const unsigned char *descriptors_a, + uint32_t feature_count_a, const unsigned char *descriptors_b, + uint32_t feature_count_b, Lardon3DOrbVulkanRequest *request, + bool private_sequence_request) { + if (!backend || feature_count_a == 0 || feature_count_b == 0 || + feature_count_a > LARDON3D_FEATURE_MAX_FEATURES || + feature_count_b > LARDON3D_FEATURE_MAX_FEATURES || !descriptors_a || + !descriptors_b || !request) return LARDON3D_ORB_VULKAN_INVALID_ARGUMENT; + *request = Lardon3DOrbVulkanRequest{}; std::lock_guard lock(backend->mutex); + if (!private_sequence_request) { + if (backend->sequence_capacity_active) { + return LARDON3D_ORB_VULKAN_FAILED; + } + backend->configured_capacity = 1; + if (backend->state == BackendState::kAvailable + && !resize_payload_locked(backend, 1)) { + return LARDON3D_ORB_VULKAN_FAILED; + } + } if (!initialize_locked(backend)) { return LARDON3D_ORB_VULKAN_UNAVAILABLE; } + uint32_t slot_index = LARDON3D_ORB_VULKAN_MAX_INFLIGHT; + for (uint32_t index = 0; index < LARDON3D_ORB_VULKAN_MAX_INFLIGHT; ++index) { + OrbRequestSlot *candidate = &backend->slots[index]; + if (!candidate->payload_allocated || candidate->completion_pending + || candidate->generation_retired) { + continue; + } + if (candidate->generation == UINT64_MAX) { + /* Generation is request identity, not a wrapping counter. Once the last + * value has been issued this slot is retired before any new submission; + * an ancient generation-one handle can therefore never become current. */ + candidate->generation_retired = true; + continue; + } + if (!candidate->completion_pending) { + slot_index = index; + break; + } + } + if (slot_index == LARDON3D_ORB_VULKAN_MAX_INFLIGHT) { + return LARDON3D_ORB_VULKAN_FAILED; + } + OrbRequestSlot *slot = &backend->slots[slot_index]; + auto submit_cpu_start = std::chrono::steady_clock::now(); VkDeviceSize bytes_a = static_cast(feature_count_a) * 32; VkDeviceSize bytes_b = static_cast(feature_count_b) * 32; - std::memcpy(backend->descriptors_a.mapping, descriptors_a, + std::memcpy(slot->descriptors_a.mapping, descriptors_a, static_cast(bytes_a)); - std::memcpy(backend->descriptors_b.mapping, descriptors_b, + std::memcpy(slot->descriptors_b.mapping, descriptors_b, static_cast(bytes_b)); - if (!synchronize_host_write(backend, backend->descriptors_a, bytes_a) || - !synchronize_host_write(backend, backend->descriptors_b, bytes_b)) { + if (!synchronize_host_write(backend, slot->descriptors_a, bytes_a) || + !synchronize_host_write(backend, slot->descriptors_b, bytes_b)) { return fail_session_locked(backend); } @@ -933,16 +1318,98 @@ extern "C" Lardon3DOrbVulkanResult lardon3d_orb_vulkan_top2( return fail_session_locked(backend); } #endif - VkResult dispatch_result = record_and_submit(backend, feature_count_a, + VkResult dispatch_result = record_and_submit(backend, slot, feature_count_a, feature_count_b); backend->last_dispatch_ns = elapsed_ns(start); - if (dispatch_result != VK_SUCCESS || - !synchronize_host_read(backend, backend->output)) { + if (dispatch_result != VK_SUCCESS) { return fail_session_locked(backend); } - read_gpu_time(backend); + const uint64_t next_generation = slot->generation + 1; + slot->completion_pending = true; + slot->pending_feature_count_a = feature_count_a; + slot->pending_feature_count_b = feature_count_b; + slot->generation = next_generation; + request->slot = slot_index; + request->generation = slot->generation; + uint64_t submit_cpu_ns = elapsed_ns(submit_cpu_start); + saturating_add(&backend->telemetry.submits, 1); + saturating_add(&backend->telemetry.submit_cpu_ns, submit_cpu_ns); + if (backend->completion_observed) { + saturating_add(&backend->telemetry.starvation_ns, + elapsed_ns(backend->completion_observed_at)); + } + backend->completion_observed = false; + telemetry_event(backend); + return LARDON3D_ORB_VULKAN_OK; +} - const RawTop2 *raw = static_cast(backend->output.mapping); +static Lardon3DOrbVulkanResult orb_vulkan_top2_finish_impl( + Lardon3DOrbVulkanBackend *backend, + const Lardon3DOrbVulkanRequest *request, Lardon3DOrbTop2 *output, + size_t output_capacity) { + if (!backend || !request || request->generation == 0 || + request->slot >= LARDON3D_ORB_VULKAN_MAX_INFLIGHT) { + return LARDON3D_ORB_VULKAN_INVALID_ARGUMENT; + } + std::lock_guard lock(backend->mutex); + OrbRequestSlot *slot = &backend->slots[request->slot]; + if (!slot->completion_pending || slot->generation != request->generation) { + return LARDON3D_ORB_VULKAN_FAILED; + } + const uint32_t feature_count_a = slot->pending_feature_count_a; + const uint32_t feature_count_b = slot->pending_feature_count_b; + auto wait_start = std::chrono::steady_clock::now(); + VkResult wait = VK_SUCCESS; +#ifdef LARDON3D_ORB_VULKAN_TESTING + const char *force_wait_failure = std::getenv( + "LARDON3D_TEST_VULKAN_FINISH_WAIT_FAILURE"); + if (force_wait_failure && std::strcmp(force_wait_failure, "1") == 0) { + wait = VK_ERROR_DEVICE_LOST; + } else +#endif + { + wait = vkWaitForFences(backend->device, 1, &slot->completion_fence, + VK_TRUE, UINT64_MAX); + } + saturating_add(&backend->telemetry.fence_wait_ns, elapsed_ns(wait_start)); + slot->completion_pending = false; + slot->pending_feature_count_a = 0; + slot->pending_feature_count_b = 0; + if (wait != VK_SUCCESS) { + return fail_session_locked(backend); + } + if (feature_count_a == 0 || feature_count_b == 0 || + feature_count_a > LARDON3D_FEATURE_MAX_FEATURES || + feature_count_b > LARDON3D_FEATURE_MAX_FEATURES) { + return fail_session_locked(backend); + } + saturating_add(&backend->telemetry.completions, 1); + backend->completion_observed = true; + backend->completion_observed_at = std::chrono::steady_clock::now(); + telemetry_event(backend); + if (!output || output_capacity < feature_count_a) { + /* Even invalid consumer storage consumes the unique completed request; + * the next begin can never inherit or overwrite an abandoned slot. */ + return LARDON3D_ORB_VULKAN_INVALID_ARGUMENT; + } + auto readback_start = std::chrono::steady_clock::now(); +#ifdef LARDON3D_ORB_VULKAN_TESTING + const char *force_readback_failure = std::getenv( + "LARDON3D_TEST_VULKAN_READBACK_FAILURE"); + /* Test builds inject the failure after exact fence ownership was consumed. + * Production takes the same fail-session branch only on a real mapped-memory + * synchronization failure, so no request or partial top-2 evidence survives. */ + if (force_readback_failure + && std::strcmp(force_readback_failure, "1") == 0) { + return fail_session_locked(backend); + } +#endif + if (!synchronize_host_read(backend, slot->output)) { + return fail_session_locked(backend); + } + read_gpu_time(backend, slot->query_pool); + + const RawTop2 *raw = static_cast(slot->output.mapping); uint32_t neighbors = std::min(feature_count_b, 2U); for (uint32_t index = 0; index < feature_count_a; ++index) { output[index].neighbor_count = neighbors; @@ -951,11 +1418,132 @@ extern "C" Lardon3DOrbVulkanResult lardon3d_orb_vulkan_top2( output[index].second_index = neighbors == 2 ? raw[index].second_index : 0; output[index].second_distance = neighbors == 2 ? raw[index].second_distance : 0; } + saturating_add(&backend->telemetry.readback_ns, + elapsed_ns(readback_start)); + saturating_add(&backend->telemetry.gpu_execution_ns, backend->last_gpu_ns); + telemetry_event(backend); return LARDON3D_ORB_VULKAN_OK; } +static Lardon3DOrbVulkanResult orb_vulkan_top2_discard_impl( + Lardon3DOrbVulkanBackend *backend, + const Lardon3DOrbVulkanRequest *request) { + if (!backend || !request || request->generation == 0 || + request->slot >= LARDON3D_ORB_VULKAN_MAX_INFLIGHT) { + return LARDON3D_ORB_VULKAN_INVALID_ARGUMENT; + } + std::lock_guard lock(backend->mutex); + OrbRequestSlot *slot = &backend->slots[request->slot]; + if (!slot->completion_pending || slot->generation != request->generation) { + return LARDON3D_ORB_VULKAN_FAILED; + } + if (slot->completion_pending) { + saturating_add(&backend->telemetry.discards, 1); + telemetry_event(backend); + VkResult wait = VK_SUCCESS; + auto wait_start = std::chrono::steady_clock::now(); +#ifdef LARDON3D_ORB_VULKAN_TESTING + const char *force_failure = std::getenv( + "LARDON3D_TEST_VULKAN_WAIT_FAILURE"); + if (force_failure && std::strcmp(force_failure, "1") == 0) { + wait = VK_ERROR_DEVICE_LOST; + } else +#endif + { + wait = vkWaitForFences(backend->device, 1, &slot->completion_fence, + VK_TRUE, UINT64_MAX); + } + saturating_add(&backend->telemetry.fence_wait_ns, + elapsed_ns(wait_start)); + slot->completion_pending = false; + slot->pending_feature_count_a = 0; + slot->pending_feature_count_b = 0; + if (wait != VK_SUCCESS) { + /* A failed wait invalidates all reusable command/buffer state. Destroy + * and permanently fail this session before another submit can race it. */ + return fail_session_locked(backend); + } + } + return LARDON3D_ORB_VULKAN_OK; +} + +extern "C" Lardon3DOrbVulkanResult +lardon3d_orb_vulkan_internal_top2_begin( + Lardon3DOrbVulkanBackend *backend, const unsigned char *descriptors_a, + uint32_t feature_count_a, const unsigned char *descriptors_b, + uint32_t feature_count_b, Lardon3DOrbVulkanRequest *request) { + try { + return orb_vulkan_top2_begin_impl( + backend, descriptors_a, feature_count_a, descriptors_b, + feature_count_b, request, true); + } catch (...) { + return LARDON3D_ORB_VULKAN_FAILED; + } +} + +extern "C" Lardon3DOrbVulkanResult +lardon3d_orb_vulkan_internal_top2_finish( + Lardon3DOrbVulkanBackend *backend, + const Lardon3DOrbVulkanRequest *request, Lardon3DOrbTop2 *output, + size_t output_capacity) { + try { + return orb_vulkan_top2_finish_impl(backend, request, output, + output_capacity); + } catch (...) { + /* If locking/host access raised after a request became active, make one + * bounded discard attempt before reporting failure. This preserves the + * consume-on-every-result contract even for C++ runtime failures. */ + try { + (void)orb_vulkan_top2_discard_impl(backend, request); + } catch (...) { + /* A second synchronization exception is contained at the C boundary. */ + } + return LARDON3D_ORB_VULKAN_FAILED; + } +} + +extern "C" Lardon3DOrbVulkanResult +lardon3d_orb_vulkan_internal_top2_discard( + Lardon3DOrbVulkanBackend *backend, + const Lardon3DOrbVulkanRequest *request) { + try { + return orb_vulkan_top2_discard_impl(backend, request); + } catch (...) { + return LARDON3D_ORB_VULKAN_FAILED; + } +} + +extern "C" bool lardon3d_orb_vulkan_internal_telemetry( + Lardon3DOrbVulkanBackend *backend, + Lardon3DOrbVulkanTelemetry *telemetry) { + if (!backend || !telemetry) return false; + /* Private C consumers receive an all-or-nothing snapshot. Mutex/runtime + * exceptions are contained here and can only make telemetry unknown; they + * never escape C or alter the active scientific request. */ + try { + std::lock_guard lock(backend->mutex); + *telemetry = backend->telemetry; + telemetry->gpu_timestamps_available = backend->timestamps_available; + telemetry->pending_slots = 0; + for (const OrbRequestSlot &slot : backend->slots) { + if (slot.completion_pending) ++telemetry->pending_slots; + } + telemetry->slot_pending = telemetry->pending_slots != 0; + telemetry->retained_capacity = backend->retained_capacity; + telemetry->retained_payload_bytes = + static_cast(backend->retained_capacity) + * LARDON3D_ORB_VULKAN_PER_SLOT_BYTES; + telemetry->sequence_capacity_active = + backend->sequence_capacity_active; + return true; + } catch (...) { + *telemetry = Lardon3DOrbVulkanTelemetry{}; + return false; + } +} + #ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY -extern "C" Lardon3DOrbVulkanResult lardon3d_sift_vulkan_top2( +static Lardon3DOrbVulkanResult sift_vulkan_top2_impl( Lardon3DOrbVulkanBackend *backend, const float *descriptors_a, uint32_t feature_count_a, const float *descriptors_b, uint32_t feature_count_b, Lardon3DSiftTop2 *output, size_t output_capacity) { @@ -1002,10 +1590,12 @@ extern "C" Lardon3DOrbVulkanResult lardon3d_sift_vulkan_top2( feature_count_a, feature_count_b); backend->last_dispatch_ns = elapsed_ns(start); if (dispatch_result != VK_SUCCESS || + vkWaitForFences(backend->device, 1, &backend->sift_completion_fence, + VK_TRUE, UINT64_MAX) != VK_SUCCESS || !synchronize_host_read(backend, backend->sift_output)) { return fail_session_locked(backend); } - read_gpu_time(backend); + read_gpu_time(backend, backend->sift_query_pool); const RawSiftTop2 *raw = static_cast(backend->sift_output.mapping); @@ -1020,6 +1610,22 @@ extern "C" Lardon3DOrbVulkanResult lardon3d_sift_vulkan_top2( } return LARDON3D_ORB_VULKAN_OK; } + +extern "C" Lardon3DOrbVulkanResult lardon3d_sift_vulkan_top2( + Lardon3DOrbVulkanBackend *backend, const float *descriptors_a, + uint32_t feature_count_a, const float *descriptors_b, + uint32_t feature_count_b, Lardon3DSiftTop2 *output, + size_t output_capacity) { + /* Feasibility remains a C ABI. Mutex/allocation/runtime exceptions must not + * cross it, including failures while the shared initialization gate runs. */ + try { + return sift_vulkan_top2_impl( + backend, descriptors_a, feature_count_a, descriptors_b, + feature_count_b, output, output_capacity); + } catch (...) { + return LARDON3D_ORB_VULKAN_FAILED; + } +} #endif extern "C" bool lardon3d_orb_vulkan_backend_info( @@ -1027,41 +1633,71 @@ extern "C" bool lardon3d_orb_vulkan_backend_info( if (!backend || !info) { return false; } - std::lock_guard lock(backend->mutex); - std::memset(info, 0, sizeof(*info)); - info->available = backend->state == BackendState::kAvailable; - info->initialized = backend->state != BackendState::kUninitialized; - info->dedicated_compute_queue = backend->dedicated_compute_queue; - info->workgroup_size = backend->workgroup_size; - info->permanent_payload_bytes = static_cast( - kDescriptorBufferBytes * 2 + kOutputBufferBytes); - info->initialization_ns = backend->initialization_ns; - info->dispatch_ns = backend->last_dispatch_ns; - info->gpu_ns = backend->last_gpu_ns; - if (backend->physical_device != VK_NULL_HANDLE) { - std::snprintf(info->device_name, sizeof(info->device_name), "%s", - backend->properties.deviceName); + try { + std::lock_guard lock(backend->mutex); + std::memset(info, 0, sizeof(*info)); + info->available = backend->state == BackendState::kAvailable; + info->initialized = backend->state != BackendState::kUninitialized; + info->dedicated_compute_queue = backend->dedicated_compute_queue; + info->workgroup_size = backend->workgroup_size; + info->permanent_payload_bytes = static_cast( + LARDON3D_ORB_VULKAN_FIXED_BYTES + + backend->retained_capacity * + LARDON3D_ORB_VULKAN_PER_SLOT_BYTES); + info->initialization_ns = backend->initialization_ns; + info->dispatch_ns = backend->last_dispatch_ns; + info->gpu_ns = backend->last_gpu_ns; + if (backend->physical_device != VK_NULL_HANDLE) { + std::snprintf(info->device_name, sizeof(info->device_name), "%s", + backend->properties.deviceName); + } + return true; + } catch (...) { + std::memset(info, 0, sizeof(*info)); + return false; } - return true; } #else -struct Lardon3DOrbVulkanBackend {}; +struct Lardon3DOrbVulkanBackend { + std::mutex mutex; + Lardon3DOrbVulkanTelemetry telemetry{}; +}; extern "C" Lardon3DOrbVulkanBackend *lardon3d_orb_vulkan_backend_create(void) { - return new (std::nothrow) Lardon3DOrbVulkanBackend(); + try { + return new (std::nothrow) Lardon3DOrbVulkanBackend(); + } catch (...) { + return nullptr; + } } extern "C" void lardon3d_orb_vulkan_backend_destroy( Lardon3DOrbVulkanBackend *backend) { - delete backend; + try { + delete backend; + } catch (...) { + /* Portable C boundary retains the same exception-containment contract. */ + } } extern "C" bool lardon3d_orb_vulkan_should_use(uint32_t, uint32_t) { return false; } +extern "C" bool lardon3d_orb_vulkan_internal_begin_sequence( + Lardon3DOrbVulkanBackend *backend, uint32_t inflight_capacity) { + (void)backend; + (void)inflight_capacity; + return false; +} + +extern "C" bool lardon3d_orb_vulkan_internal_end_sequence( + Lardon3DOrbVulkanBackend *backend) { + return backend != nullptr; +} + extern "C" Lardon3DOrbVulkanResult lardon3d_orb_vulkan_top2( Lardon3DOrbVulkanBackend *backend, const unsigned char *, uint32_t feature_count_a, const unsigned char *, uint32_t feature_count_b, Lardon3DOrbTop2 *output, @@ -1078,6 +1714,61 @@ extern "C" Lardon3DOrbVulkanResult lardon3d_orb_vulkan_top2( return LARDON3D_ORB_VULKAN_UNAVAILABLE; } +extern "C" Lardon3DOrbVulkanResult +lardon3d_orb_vulkan_internal_top2_begin( + Lardon3DOrbVulkanBackend *backend, const unsigned char *, + uint32_t feature_count_a, const unsigned char *, uint32_t feature_count_b, + Lardon3DOrbVulkanRequest *request) { + if (!backend || feature_count_a == 0 || feature_count_b == 0 || !request) { + return LARDON3D_ORB_VULKAN_INVALID_ARGUMENT; + } + *request = Lardon3DOrbVulkanRequest{}; + return LARDON3D_ORB_VULKAN_UNAVAILABLE; +} + +extern "C" Lardon3DOrbVulkanResult +lardon3d_orb_vulkan_internal_top2_finish( + Lardon3DOrbVulkanBackend *backend, + const Lardon3DOrbVulkanRequest *request, Lardon3DOrbTop2 *output, + size_t output_capacity) { + if (!backend || !request || request->generation == 0 || !output || + output_capacity == 0) { + return LARDON3D_ORB_VULKAN_INVALID_ARGUMENT; + } + return LARDON3D_ORB_VULKAN_UNAVAILABLE; +} + +extern "C" Lardon3DOrbVulkanResult +lardon3d_orb_vulkan_internal_top2_discard( + Lardon3DOrbVulkanBackend *backend, + const Lardon3DOrbVulkanRequest *request) { + return backend && request && request->generation != 0 + ? LARDON3D_ORB_VULKAN_OK + : LARDON3D_ORB_VULKAN_INVALID_ARGUMENT; +} + +extern "C" bool lardon3d_orb_vulkan_internal_telemetry( + Lardon3DOrbVulkanBackend *backend, + Lardon3DOrbVulkanTelemetry *telemetry) { + if (!backend || !telemetry) return false; + /* Preserve the same C exception boundary in the portable build. */ + try { + std::lock_guard lock(backend->mutex); + *telemetry = backend->telemetry; + return true; + } catch (...) { + *telemetry = Lardon3DOrbVulkanTelemetry{}; + return false; + } +} + +#ifdef LARDON3D_ORB_VULKAN_TESTING +extern "C" bool lardon3d_orb_vulkan_internal_test_set_slot_generation( + Lardon3DOrbVulkanBackend *, uint32_t, uint64_t) { + return false; +} +#endif + #ifdef LARDON3D_SIFT_VULKAN_FEASIBILITY extern "C" Lardon3DOrbVulkanResult lardon3d_sift_vulkan_top2( Lardon3DOrbVulkanBackend *backend, const float *, uint32_t feature_count_a, diff --git a/src/orb_vulkan_backend_internal.h b/src/orb_vulkan_backend_internal.h new file mode 100644 index 0000000..e39fb66 --- /dev/null +++ b/src/orb_vulkan_backend_internal.h @@ -0,0 +1,121 @@ +#ifndef LARDON3D_ORB_VULKAN_BACKEND_INTERNAL_H +#define LARDON3D_ORB_VULKAN_BACKEND_INTERNAL_H + +#include +#include + +#include + +#ifdef __cplusplus +extern "C" { +#endif + +#if defined(__GNUC__) || defined(__clang__) +#define LARDON3D_INTERNAL_VISIBILITY __attribute__((visibility("hidden"))) +#else +#define LARDON3D_INTERNAL_VISIBILITY +#endif + +typedef struct { + uint64_t serial; + uint64_t submits; + uint64_t completions; + uint64_t submit_cpu_ns; + uint64_t fence_wait_ns; + uint64_t readback_ns; + bool gpu_timestamps_available; + uint64_t gpu_execution_ns; + uint64_t starvation_ns; + uint64_t failures; + uint64_t discards; + bool slot_pending; + uint32_t pending_slots; + uint32_t retained_capacity; + uint64_t retained_payload_bytes; + bool sequence_capacity_active; +} Lardon3DOrbVulkanTelemetry; + +enum { + LARDON3D_ORB_VULKAN_MAX_INFLIGHT = 2, + /* Explicit host-visible Vulkan buffer payload. Each slot owns two 8192x32 + * descriptor inputs and one 8192x4x32-bit readback buffer. Device, + * pipeline, layouts and cache are immutable shared objects with opaque + * driver allocations; no invented byte charge is assigned to them. */ + LARDON3D_ORB_VULKAN_FIXED_BYTES = 0, + LARDON3D_ORB_VULKAN_PER_SLOT_BYTES = + LARDON3D_ORB_VULKAN_PERMANENT_BUFFER_BYTES +}; + +typedef struct { + uint32_t slot; + uint64_t generation; +} Lardon3DOrbVulkanRequest; + +/* Matcher owns one capacity lease for the immutable executing sequence. The + * backend may grow/shrink mapped request payload only while no request is + * pending. End releases any depth-two payload and restores the public/default + * depth-one capacity before the Task crosses its next admission boundary. */ +LARDON3D_INTERNAL_VISIBILITY bool +lardon3d_orb_vulkan_internal_begin_sequence( + Lardon3DOrbVulkanBackend *backend, + uint32_t inflight_capacity +); +LARDON3D_INTERNAL_VISIBILITY bool +lardon3d_orb_vulkan_internal_end_sequence( + Lardon3DOrbVulkanBackend *backend +); + +/* Up to two private backend-owned requests may be in flight. The handle is the + * exact slot identity plus a nonzero generation: finish/discard can consume + * only that request and stale or mismatched handles never redirect evidence. + * Begin stores the submitted feature counts in the slot; finish never trusts + * fresh caller counts. Every finish result for a valid active handle consumes + * or fails that slot, including invalid output capacity. */ +LARDON3D_INTERNAL_VISIBILITY Lardon3DOrbVulkanResult +lardon3d_orb_vulkan_internal_top2_begin( + Lardon3DOrbVulkanBackend *backend, + const unsigned char *descriptors_a, + uint32_t feature_count_a, + const unsigned char *descriptors_b, + uint32_t feature_count_b, + Lardon3DOrbVulkanRequest *request +); +LARDON3D_INTERNAL_VISIBILITY Lardon3DOrbVulkanResult +lardon3d_orb_vulkan_internal_top2_finish( + Lardon3DOrbVulkanBackend *backend, + const Lardon3DOrbVulkanRequest *request, + Lardon3DOrbTop2 *output, + size_t output_capacity +); +LARDON3D_INTERNAL_VISIBILITY Lardon3DOrbVulkanResult +lardon3d_orb_vulkan_internal_top2_discard( + Lardon3DOrbVulkanBackend *backend, + const Lardon3DOrbVulkanRequest *request +); +/* Cumulative counters saturate at UINT64_MAX. The snapshot is protected by + * backend request-state ownership and therefore observes slot state and every + * counter from one instant without widening the public backend-info ABI. */ +LARDON3D_INTERNAL_VISIBILITY bool +lardon3d_orb_vulkan_internal_telemetry( + Lardon3DOrbVulkanBackend *backend, + Lardon3DOrbVulkanTelemetry *telemetry +); + +#ifdef LARDON3D_ORB_VULKAN_TESTING +/* Inactive-slot saturation seam. It exercises production generation-retire + * branching without billions of submissions and never exists in lardon3d. */ +LARDON3D_INTERNAL_VISIBILITY bool +lardon3d_orb_vulkan_internal_test_set_slot_generation( + Lardon3DOrbVulkanBackend *backend, + uint32_t slot, + uint64_t generation +); +#endif + +#undef LARDON3D_INTERNAL_VISIBILITY + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/src/photo_quality_task.cpp b/src/photo_quality_task.cpp index 89d6263..57054ef 100644 --- a/src/photo_quality_task.cpp +++ b/src/photo_quality_task.cpp @@ -5,6 +5,8 @@ extern "C" { #include } +#include "opencv_task_thread_guard.h" + #include #include #include @@ -119,7 +121,15 @@ bool run_impl(Lardon3DTask *task, void *value) { } bool run(Lardon3DTask *task, void *value) noexcept { - try { return run_impl(task, value); } + try { + Lardon3DOpenCvTaskThreadGuard threads(task); + if (!threads.valid()) + return lardon3d_task_fail(task, "Contrat CPU OpenCV triage invalide."); + bool result = run_impl(task, value); + if (!threads.restore()) + return lardon3d_task_fail(task, "Restauration OpenCV triage impossible."); + return result; + } catch (const std::bad_alloc &) { return lardon3d_task_fail(task, "Mémoire insuffisante."); } catch (...) { return lardon3d_task_fail(task, "Erreur interne du triage photo."); } } diff --git a/src/raw_development_task.cpp b/src/raw_development_task.cpp index f9dd1e0..43d511f 100644 --- a/src/raw_development_task.cpp +++ b/src/raw_development_task.cpp @@ -5,6 +5,8 @@ extern "C" { #include } +#include "opencv_task_thread_guard.h" + #include #include @@ -84,7 +86,13 @@ bool run_impl(Lardon3DTask *task, Context *context) { bool run(Lardon3DTask *task, void *value) noexcept { try { - return run_impl(task, static_cast(value)); + Lardon3DOpenCvTaskThreadGuard threads(task); + if (!threads.valid()) + return lardon3d_task_fail(task, "Contrat CPU OpenCV RAW invalide."); + bool result = run_impl(task, static_cast(value)); + if (!threads.restore()) + return lardon3d_task_fail(task, "Restauration OpenCV RAW impossible."); + return result; } catch (const std::bad_alloc &) { return lardon3d_task_fail(task, "Mémoire insuffisante pour le développement RAW."); } catch (...) { diff --git a/src/resource_governor.c b/src/resource_governor.c index 3aad74e..2c83236 100644 --- a/src/resource_governor.c +++ b/src/resource_governor.c @@ -1,8 +1,21 @@ +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif + #include +#include +#include #include +#ifdef __linux__ +#include +#include +#include +#endif #include #include #include +#include +#include #include @@ -16,6 +29,11 @@ struct Lardon3DResourceReservation { enum { LARDON3D_BATCH_METRICS_CAPACITY = 8, + LARDON3D_CAPABILITY_FEEDBACK_CAPACITY = 32, + LARDON3D_CAPABILITY_THROUGHPUT_OBSERVATIONS = 2, + LARDON3D_GPU_BATCH_THROUGHPUT_OBSERVATIONS = 8, + LARDON3D_TELEMETRY_TEXT_CAPACITY = 16384, + LARDON3D_PROC_STAT_CAPACITY = 196608, }; typedef struct { @@ -24,6 +42,48 @@ typedef struct { size_t peak_memory_bytes; } Lardon3DBatchMetrics; +typedef enum { + LARDON3D_CAPABILITY_TRIAL_NONE = 0, + LARDON3D_CAPABILITY_TRIAL_CPU = 1, + LARDON3D_CAPABILITY_TRIAL_INFLIGHT = 2, + LARDON3D_CAPABILITY_TRIAL_BATCH = 3, +} Lardon3DCapabilityTrialDimension; + +typedef struct { + bool used; + char task_kind[LARDON3D_TASK_KIND_CAPACITY]; + uint32_t task_kind_version; + Lardon3DResourceBackend backend; + unsigned int adaptive_cpu_limit; + unsigned int accepted_cpu_limit; + size_t adaptive_batch_limit; + size_t accepted_batch_limit; + size_t adaptive_inflight_limit; + size_t accepted_inflight_limit; + Lardon3DCapabilityTrialDimension trial_dimension; + unsigned int baseline_observations; + unsigned int trial_observations; + uint64_t baseline_rate_milli; + uint64_t baseline_rates_milli[ + LARDON3D_GPU_BATCH_THROUGHPUT_OBSERVATIONS]; + uint64_t trial_rates_milli[ + LARDON3D_GPU_BATCH_THROUGHPUT_OBSERVATIONS]; + bool cpu_growth_stopped; + bool inflight_growth_stopped; + bool batch_growth_stopped; + uint64_t diagnostic_serial; + uint64_t diagnostic_update_order; + Lardon3DResourceSequenceDiagnostic diagnostic; + Lardon3DResourceSequenceAggregate aggregate; + bool aggregate_selection_known; + unsigned int aggregate_cpu_threads; + unsigned int aggregate_gpu_slots; + unsigned int aggregate_io_slots; + size_t aggregate_batch_size; + size_t aggregate_inflight_limit; + unsigned int aggregate_helper_limit; +} Lardon3DCapabilityFeedback; + struct Lardon3DResourceGovernor { pthread_mutex_t mutex; pthread_cond_t cond; @@ -39,6 +99,16 @@ struct Lardon3DResourceGovernor { Lardon3DBatchMetrics batch_metrics[LARDON3D_RESOURCE_TASK_MIXED + 1][LARDON3D_BATCH_METRICS_CAPACITY]; size_t batch_metrics_count[LARDON3D_RESOURCE_TASK_MIXED + 1]; size_t batch_metrics_head[LARDON3D_RESOURCE_TASK_MIXED + 1]; + Lardon3DCapabilityFeedback capability_feedback[ + LARDON3D_CAPABILITY_FEEDBACK_CAPACITY + ]; + size_t capability_feedback_replace; + uint64_t capability_diagnostic_serial; + uint64_t capability_diagnostic_update_order; + bool orb_vulkan_backend_available; + Lardon3DResourceCpuTopologyInput cpu_topology; + Lardon3DResourceCpuPolicyDiagnostic cpu_policy; + bool internal_force_worker_affinity_failure; size_t active_count; bool swap_baseline_known; uint64_t last_swap_pages_in; @@ -52,10 +122,886 @@ struct Lardon3DResourceGovernor { bool internal_now_known; struct timespec internal_now; bool internal_force_capture_failure; + /* Host observations are private operational evidence. In particular RSS + * is never charged as Task-owned memory and pool utilization never + * substitutes for Gate G load/pressure admission. Cumulative inputs are + * retained only long enough to form the next bounded delta. */ + Lardon3DResourceHostTelemetry host_telemetry; + bool telemetry_override; + bool telemetry_cpu_baseline_known; + uint64_t telemetry_cpu_total[LARDON3D_RESOURCE_CPU_MAX]; + uint64_t telemetry_cpu_idle[LARDON3D_RESOURCE_CPU_MAX]; + bool telemetry_swap_baseline_known; + uint64_t telemetry_swap_pages_in; + uint64_t telemetry_swap_pages_out; Lardon3DResourceReservation *active; Lardon3DResourceReservation *released; }; +static bool cpu_mask_test( + const uint64_t mask[LARDON3D_RESOURCE_CPU_MASK_WORDS], + unsigned int cpu +); + +static bool +ascii_space(char value) +{ + return value == ' ' || value == '\t' || value == '\n' + || value == '\r' || value == '\f' || value == '\v'; +} + +static bool +parse_u64_token(const char **cursor, const char *end, uint64_t *value) +{ + const char *position = *cursor; + while (position < end && (*position == ' ' || *position == '\t')) { + ++position; + } + if (position == end || *position < '0' || *position > '9') return false; + uint64_t parsed = 0; + do { + unsigned int digit = (unsigned int)(*position - '0'); + if (parsed > (UINT64_MAX - digit) / 10) return false; + parsed = parsed * 10 + digit; + ++position; + } while (position < end && *position >= '0' && *position <= '9'); + *cursor = position; + *value = parsed; + return true; +} + +static bool +parse_percent_basis_points( + const char *begin, + const char *end, + uint32_t *basis_points +) +{ + const char *position = begin; + uint64_t whole = 0; + if (!parse_u64_token(&position, end, &whole) || whole > 100) return false; + uint32_t fraction = 0; + unsigned int digits = 0; + if (position < end && *position == '.') { + ++position; + if (position == end || *position < '0' || *position > '9') return false; + while (position < end && *position >= '0' && *position <= '9') { + if (digits == 2) return false; + fraction = fraction * 10 + (uint32_t)(*position - '0'); + ++digits; + ++position; + } + } + while (digits < 2) { + fraction *= 10; + ++digits; + } + if (position != end || (whole == 100 && fraction != 0)) return false; + *basis_points = (uint32_t)whole * 100 + fraction; + return true; +} + +static bool +line_value_u64( + const char *text, + const char *name, + uint64_t *value, + bool kilobytes +) +{ + size_t name_length = strlen(name); + const char *line = text; + while (*line) { + const char *end = strchr(line, '\n'); + if (!end) end = line + strlen(line); + if ((size_t)(end - line) > name_length + && memcmp(line, name, name_length) == 0 + && (line[name_length] == ' ' || line[name_length] == '\t' + || line[name_length] == ':')) { + const char *cursor = line + name_length; + if (*cursor == ':') ++cursor; + uint64_t parsed = 0; + if (!parse_u64_token(&cursor, end, &parsed)) return false; + while (cursor < end && (*cursor == ' ' || *cursor == '\t')) ++cursor; + if (kilobytes) { + if ((size_t)(end - cursor) != 2 || cursor[0] != 'k' + || cursor[1] != 'B' || parsed > UINT64_MAX / 1024) { + return false; + } + parsed *= 1024; + } else if (cursor != end) { + return false; + } + *value = parsed; + return true; + } + line = *end ? end + 1 : end; + } + return false; +} + +static bool +psi_value(const char *text, const char *category, uint32_t *basis_points) +{ + size_t category_length = strlen(category); + const char *line = text; + while (*line) { + const char *end = strchr(line, '\n'); + if (!end) end = line + strlen(line); + const char *cursor = line; + const char *token_end = cursor; + while (token_end < end && !ascii_space(*token_end)) ++token_end; + if ((size_t)(token_end - cursor) == category_length + && memcmp(cursor, category, category_length) == 0) { + bool found = false; + uint32_t parsed = 0; + cursor = token_end; + while (cursor < end) { + while (cursor < end && ascii_space(*cursor)) ++cursor; + if (cursor == end) break; + token_end = cursor; + while (token_end < end && !ascii_space(*token_end)) ++token_end; + if ((size_t)(token_end - cursor) >= 6 + && memcmp(cursor, "avg10=", 6) == 0) { + /* PSI input is operational safety evidence. Accept one + * exact standalone avg10 field only: substring matches, + * duplicates, signs, and silently rounded precision must + * degrade the selected line to unknown. */ + if (found || !parse_percent_basis_points( + cursor + 6, token_end, &parsed)) { + return false; + } + found = true; + } + cursor = token_end; + } + if (!found) return false; + *basis_points = parsed; + return true; + } + line = *end ? end + 1 : end; + } + return false; +} + +typedef struct { + bool seen[LARDON3D_RESOURCE_CPU_MAX]; + uint64_t total[LARDON3D_RESOURCE_CPU_MAX]; + uint64_t idle[LARDON3D_RESOURCE_CPU_MAX]; +} Lardon3DComputeCpuTicks; + +static bool +compute_cpu_ticks_locked( + const Lardon3DResourceGovernor *governor, + const char *text, + Lardon3DComputeCpuTicks *ticks +) +{ + if (!governor->cpu_policy.affinity_configured) { + /* A portable count is an admission budget, not an invented CPU mask. + * Without an active compute mask, whole-host ticks cannot honestly be + * relabelled as compute-pool utilization. */ + return false; + } + *ticks = (Lardon3DComputeCpuTicks){0}; + unsigned int found_count = 0; + const char *line = text; + while (*line) { + const char *end = strchr(line, '\n'); + if (!end) end = line + strlen(line); + if ((size_t)(end - line) > 4 && memcmp(line, "cpu", 3) == 0 + && line[3] >= '0' && line[3] <= '9') { + const char *cursor = line + 3; + uint64_t cpu = 0; + if (!parse_u64_token(&cursor, end, &cpu) + || cpu >= LARDON3D_RESOURCE_CPU_MAX + || (cursor < end && *cursor != ' ' && *cursor != '\t')) { + return false; + } + bool selected = cpu_mask_test(governor->cpu_policy.compute_mask, + (unsigned int)cpu); + if (selected) { + if (ticks->seen[cpu]) return false; + uint64_t fields[10] = {0}; + unsigned int count = 0; + while (cursor < end) { + while (cursor < end + && (*cursor == ' ' || *cursor == '\t')) ++cursor; + if (cursor == end) break; + if (count == 10) return false; + if (!parse_u64_token(&cursor, end, &fields[count])) return false; + ++count; + } + if (count < 4) return false; + uint64_t line_total = 0; + unsigned int accounting_fields = count < 8 ? count : 8; + for (unsigned int index = 0; index < accounting_fields; ++index) { + if (line_total > UINT64_MAX - fields[index]) return false; + line_total += fields[index]; + } + uint64_t line_idle = fields[3]; + if (count > 4) { + if (line_idle > UINT64_MAX - fields[4]) return false; + line_idle += fields[4]; + } + if (line_idle > line_total) return false; + ticks->seen[cpu] = true; + ticks->total[cpu] = line_total; + ticks->idle[cpu] = line_idle; + ++found_count; + } + } + line = *end ? end + 1 : end; + } + if (found_count != governor->cpu_policy.compute_cpu_count) return false; + for (unsigned int cpu = 0; cpu < LARDON3D_RESOURCE_CPU_MAX; ++cpu) { + if (cpu_mask_test(governor->cpu_policy.compute_mask, cpu) + && !ticks->seen[cpu]) { + return false; + } + } + return true; +} + +static bool +parse_gpu_busy(const char *text, uint32_t *basis_points) +{ + const char *end = text + strlen(text); + const char *cursor = text; + uint64_t percent = 0; + if (!parse_u64_token(&cursor, end, &percent) || percent > 100) return false; + while (cursor < end && ascii_space(*cursor)) ++cursor; + if (cursor != end) return false; + *basis_points = (uint32_t)percent * 100; + return true; +} + +static bool +read_bounded_text(const char *path, char *text, size_t capacity) +{ + if (!path || !text || capacity < 2) return false; + int fd = open(path, O_RDONLY | O_CLOEXEC | O_NOFOLLOW | O_NONBLOCK); + if (fd < 0) return false; + size_t used = 0; + bool ok = true; + while (used < capacity - 1) { + ssize_t count = read(fd, text + used, capacity - 1 - used); + if (count < 0 && errno == EINTR) continue; + if (count < 0) { + ok = false; + break; + } + if (count == 0) break; + used += (size_t)count; + } + if (ok && used == capacity - 1) { + char extra; + ssize_t count; + do { + count = read(fd, &extra, 1); + } while (count < 0 && errno == EINTR); + ok = count == 0; + } + if (close(fd) != 0) ok = false; + if (!ok) return false; + text[used] = '\0'; + return true; +} + +static bool +read_gpu_busy_at_root( + const char *root, + unsigned int card_index, + char *text, + size_t capacity, + uint32_t *basis_points +) +{ + if (!root || !root[0] || !text || !basis_points || card_index > 63) { + return false; + } + char path[PATH_MAX]; + int written = snprintf(path, sizeof(path), + "%s/card%u/device/gpu_busy_percent", root, card_index); + return written > 0 && (size_t)written < sizeof(path) + && read_bounded_text(path, text, capacity) + && parse_gpu_busy(text, basis_points); +} + +bool +lardon3d_resource_governor_internal_read_gpu_busy_at_root( + const char *root, + unsigned int card_index, + uint32_t *basis_points +) +{ + char text[32]; + return read_gpu_busy_at_root( + root, card_index, text, sizeof(text), basis_points); +} + +static void +sample_telemetry_raw_locked( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceTelemetryRaw *raw, + Lardon3DResourceHostTelemetry *telemetry +) +{ + *telemetry = (Lardon3DResourceHostTelemetry){0}; + if (raw->meminfo) { + telemetry->memory_available_known = line_value_u64( + raw->meminfo, "MemAvailable", &telemetry->memory_available_bytes, + true); + } + if (raw->memory_psi) { + telemetry->memory_psi_some_known = psi_value( + raw->memory_psi, "some", + &telemetry->memory_psi_some_basis_points); + telemetry->memory_psi_full_known = psi_value( + raw->memory_psi, "full", + &telemetry->memory_psi_full_basis_points); + } + if (raw->io_psi) { + telemetry->io_psi_some_known = psi_value( + raw->io_psi, "some", &telemetry->io_psi_some_basis_points); + telemetry->io_psi_full_known = psi_value( + raw->io_psi, "full", &telemetry->io_psi_full_basis_points); + } + if (raw->process_status) { + telemetry->process_rss_known = line_value_u64( + raw->process_status, "VmRSS", &telemetry->process_rss_bytes, + true); + telemetry->process_peak_rss_known = line_value_u64( + raw->process_status, "VmHWM", &telemetry->process_peak_rss_bytes, + true); + } + if (raw->gpu_busy_percent) { + telemetry->gpu_busy_known = parse_gpu_busy( + raw->gpu_busy_percent, &telemetry->gpu_busy_basis_points); + } + if (raw->proc_stat) { + Lardon3DComputeCpuTicks ticks; + if (compute_cpu_ticks_locked(governor, raw->proc_stat, &ticks)) { + if (governor->telemetry_cpu_baseline_known) { + uint64_t total_delta = 0; + uint64_t idle_delta = 0; + bool delta_valid = true; + for (unsigned int cpu = 0; + cpu < LARDON3D_RESOURCE_CPU_MAX; ++cpu) { + if (!ticks.seen[cpu]) continue; + if (ticks.total[cpu] < governor->telemetry_cpu_total[cpu] + || ticks.idle[cpu] + < governor->telemetry_cpu_idle[cpu]) { + delta_valid = false; + break; + } + uint64_t cpu_total_delta = ticks.total[cpu] + - governor->telemetry_cpu_total[cpu]; + uint64_t cpu_idle_delta = ticks.idle[cpu] + - governor->telemetry_cpu_idle[cpu]; + if (cpu_idle_delta > cpu_total_delta + || total_delta > UINT64_MAX - cpu_total_delta + || idle_delta > UINT64_MAX - cpu_idle_delta) { + delta_valid = false; + break; + } + total_delta += cpu_total_delta; + idle_delta += cpu_idle_delta; + } + if (delta_valid && total_delta > 0) { + uint64_t busy_delta = total_delta - idle_delta; + __uint128_t scaled = (__uint128_t)busy_delta * 10000; + telemetry->compute_pool_utilization_basis_points = + (uint32_t)(scaled / total_delta); + telemetry->compute_pool_utilization_known = true; + } + } + memcpy(governor->telemetry_cpu_total, ticks.total, + sizeof(ticks.total)); + memcpy(governor->telemetry_cpu_idle, ticks.idle, + sizeof(ticks.idle)); + governor->telemetry_cpu_baseline_known = true; + } else { + governor->telemetry_cpu_baseline_known = false; + } + } + if (raw->vmstat) { + uint64_t pages_in = 0; + uint64_t pages_out = 0; + if (line_value_u64(raw->vmstat, "pswpin", &pages_in, false) + && line_value_u64(raw->vmstat, "pswpout", &pages_out, false)) { + if (governor->telemetry_swap_baseline_known + && pages_in >= governor->telemetry_swap_pages_in + && pages_out >= governor->telemetry_swap_pages_out) { + telemetry->swap_delta_known = true; + telemetry->swap_pages_in_delta = + pages_in - governor->telemetry_swap_pages_in; + telemetry->swap_pages_out_delta = + pages_out - governor->telemetry_swap_pages_out; + } + governor->telemetry_swap_pages_in = pages_in; + governor->telemetry_swap_pages_out = pages_out; + governor->telemetry_swap_baseline_known = true; + } else { + governor->telemetry_swap_baseline_known = false; + } + } + governor->host_telemetry = *telemetry; +} + +bool +lardon3d_resource_governor_internal_sample_telemetry_raw( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceTelemetryRaw *raw, + Lardon3DResourceHostTelemetry *telemetry +) +{ + if (!governor || !raw || !telemetry) return false; + (void)pthread_mutex_lock(&governor->mutex); + sample_telemetry_raw_locked(governor, raw, telemetry); + /* Tests may install deterministic telemetry while exercising admissions; + * production capture resumes only on a fresh Governor. This is a private + * seam and cannot affect persisted Task identity or restart state. */ + governor->telemetry_override = true; + (void)pthread_mutex_unlock(&governor->mutex); + return true; +} + +static void +capture_host_telemetry(Lardon3DResourceGovernor *governor) +{ + (void)pthread_mutex_lock(&governor->mutex); + bool overridden = governor->telemetry_override; + (void)pthread_mutex_unlock(&governor->mutex); + if (overridden) return; + + char *proc_stat = malloc(LARDON3D_PROC_STAT_CAPACITY); + if (!proc_stat) return; + char meminfo[LARDON3D_TELEMETRY_TEXT_CAPACITY]; + char memory_psi[LARDON3D_TELEMETRY_TEXT_CAPACITY]; + char io_psi[LARDON3D_TELEMETRY_TEXT_CAPACITY]; + char vmstat[LARDON3D_TELEMETRY_TEXT_CAPACITY]; + char status[LARDON3D_TELEMETRY_TEXT_CAPACITY]; + char gpu_busy[LARDON3D_TELEMETRY_TEXT_CAPACITY]; + bool stat_ok = read_bounded_text( + "/proc/stat", proc_stat, LARDON3D_PROC_STAT_CAPACITY); + bool meminfo_ok = read_bounded_text( + "/proc/meminfo", meminfo, sizeof(meminfo)); + bool memory_psi_ok = read_bounded_text( + "/proc/pressure/memory", memory_psi, sizeof(memory_psi)); + bool io_psi_ok = read_bounded_text( + "/proc/pressure/io", io_psi, sizeof(io_psi)); + bool vmstat_ok = read_bounded_text( + "/proc/vmstat", vmstat, sizeof(vmstat)); + bool status_ok = read_bounded_text( + "/proc/self/status", status, sizeof(status)); + uint32_t ignored_gpu_busy = 0; + /* Hardware discovery owns DRM identity. Falling through to another card + * could train this Task from an unrelated GPU, so a missing/malformed + * retained card is unknown rather than an invitation to scan. */ + bool gpu_busy_ok = governor->profile.gpu_available + && read_gpu_busy_at_root( + "/sys/class/drm", governor->profile.gpu_drm_card_index, + gpu_busy, sizeof(gpu_busy), &ignored_gpu_busy); + Lardon3DResourceTelemetryRaw raw = { + .proc_stat = stat_ok ? proc_stat : NULL, + .meminfo = meminfo_ok ? meminfo : NULL, + .memory_psi = memory_psi_ok ? memory_psi : NULL, + .io_psi = io_psi_ok ? io_psi : NULL, + .vmstat = vmstat_ok ? vmstat : NULL, + .process_status = status_ok ? status : NULL, + .gpu_busy_percent = gpu_busy_ok ? gpu_busy : NULL, + }; + (void)pthread_mutex_lock(&governor->mutex); + Lardon3DResourceHostTelemetry telemetry; + sample_telemetry_raw_locked(governor, &raw, &telemetry); + (void)pthread_mutex_unlock(&governor->mutex); + free(proc_stat); +} + +static void +cpu_mask_set(uint64_t mask[LARDON3D_RESOURCE_CPU_MASK_WORDS], unsigned int cpu) +{ + mask[cpu / 64] |= UINT64_C(1) << (cpu % 64); +} + +static bool +cpu_mask_test( + const uint64_t mask[LARDON3D_RESOURCE_CPU_MASK_WORDS], + unsigned int cpu +) +{ + return (mask[cpu / 64] & (UINT64_C(1) << (cpu % 64))) != 0; +} + +static bool +mesa_shader_cache_disabled_value(const char *value) +{ + return value + && (strcmp(value, "true") == 0 || strcmp(value, "1") == 0); +} + +Lardon3DResourceDriverPolicyResult +lardon3d_resource_governor_internal_configure_driver_policy(void) +{ + const char *value = getenv("MESA_SHADER_CACHE_DISABLE"); + if (value) { + return mesa_shader_cache_disabled_value(value) + ? LARDON3D_RESOURCE_DRIVER_POLICY_INHERITED_SAFE + : LARDON3D_RESOURCE_DRIVER_POLICY_REJECTED_UNSAFE; + } + /* CONTRACT: this executes on the startup thread before application pthread + * creation. Mesa's disk-cache workers are known to widen affinity after + * inheritance, while Linux exposes no stable-identity affinity operation + * for a foreign thread. Defaulting the cache off removes those workers; + * explicit user values are never overwritten. */ + if (setenv("MESA_SHADER_CACHE_DISABLE", "true", 0) != 0) { + return LARDON3D_RESOURCE_DRIVER_POLICY_FAILED; + } + value = getenv("MESA_SHADER_CACHE_DISABLE"); + return mesa_shader_cache_disabled_value(value) + ? LARDON3D_RESOURCE_DRIVER_POLICY_DEFAULTED + : LARDON3D_RESOURCE_DRIVER_POLICY_FAILED; +} + +static void +cpu_policy_fallback( + const Lardon3DHardwareProfile *profile, + const Lardon3DResourcePolicy *policy, + Lardon3DResourceCpuPolicyDiagnostic *diagnostic, + const char *reason +) +{ + bool mesa_cache_disabled = mesa_shader_cache_disabled_value( + getenv("MESA_SHADER_CACHE_DISABLE")); + *diagnostic = (Lardon3DResourceCpuPolicyDiagnostic) { + .runtime_thread_policy_active = mesa_cache_disabled, + .mesa_shader_cache_disabled = mesa_cache_disabled, + .compute_cpu_count = profile->logical_cpu_count + - policy->system_cpu_reserve, + .reserved_cpu_count = policy->system_cpu_reserve, + }; + (void)snprintf(diagnostic->reason, sizeof(diagnostic->reason), "%s", + reason); + (void)snprintf(diagnostic->runtime_thread_policy_reason, + sizeof(diagnostic->runtime_thread_policy_reason), "%s", + mesa_cache_disabled + ? "worker-self-affinity-plus-mesa-disk-cache-disabled" + : "mesa-disk-cache-safety-policy-not-established"); +} + +static bool +cpu_topology_input_masks( + const Lardon3DResourceCpuTopologyInput *input, + uint64_t allowed[LARDON3D_RESOURCE_CPU_MASK_WORDS], + unsigned int package_ids[LARDON3D_RESOURCE_CPU_MAX], + unsigned int core_ids[LARDON3D_RESOURCE_CPU_MAX] +) +{ + if (!input || !input->affinity_available + || input->allowed_cpu_count == 0 + || input->allowed_cpu_count > LARDON3D_RESOURCE_CPU_MAX) { + return false; + } + memset(allowed, 0, + sizeof(uint64_t) * LARDON3D_RESOURCE_CPU_MASK_WORDS); + for (size_t index = 0; index < input->allowed_cpu_count; ++index) { + unsigned int cpu = input->allowed_cpu_ids[index]; + if (cpu >= LARDON3D_RESOURCE_CPU_MAX + || cpu_mask_test(allowed, cpu)) { + return false; + } + cpu_mask_set(allowed, cpu); + } + if (!input->topology_available) { + return true; + } + if (input->topology_entry_count != input->allowed_cpu_count) { + return false; + } + bool seen[LARDON3D_RESOURCE_CPU_MAX] = {false}; + for (size_t index = 0; index < input->topology_entry_count; ++index) { + const Lardon3DResourceCpuTopologyEntry *entry = + &input->topology_entries[index]; + if (entry->cpu_id >= LARDON3D_RESOURCE_CPU_MAX + || !cpu_mask_test(allowed, entry->cpu_id) + || seen[entry->cpu_id]) { + return false; + } + seen[entry->cpu_id] = true; + package_ids[entry->cpu_id] = entry->package_id; + core_ids[entry->cpu_id] = entry->core_id; + } + return true; +} + +static void +build_cpu_policy( + const Lardon3DHardwareProfile *profile, + const Lardon3DResourcePolicy *policy, + const Lardon3DResourceCpuTopologyInput *input, + Lardon3DResourceCpuPolicyDiagnostic *diagnostic +) +{ + cpu_policy_fallback(profile, policy, diagnostic, + "fallback-portable-affinity-unavailable"); + if (!input || !input->affinity_available) { + return; + } + uint64_t allowed[LARDON3D_RESOURCE_CPU_MASK_WORDS]; + unsigned int package_ids[LARDON3D_RESOURCE_CPU_MAX] = {0}; + unsigned int core_ids[LARDON3D_RESOURCE_CPU_MAX] = {0}; + if (!cpu_topology_input_masks(input, allowed, package_ids, core_ids) + || input->allowed_cpu_count > profile->logical_cpu_count) { + cpu_policy_fallback(profile, policy, diagnostic, + "fallback-portable-affinity-invalid"); + return; + } + memcpy(diagnostic->allowed_mask, allowed, sizeof(allowed)); + unsigned int portable_budget = profile->logical_cpu_count + - policy->system_cpu_reserve; + if (input->allowed_cpu_count < profile->logical_cpu_count + && input->allowed_cpu_count <= portable_budget) { + diagnostic->affinity_configured = true; + diagnostic->externally_constrained = true; + diagnostic->compute_cpu_count = (unsigned int)input->allowed_cpu_count; + diagnostic->reserved_cpu_count = 0; + memcpy(diagnostic->compute_mask, allowed, sizeof(allowed)); + (void)snprintf(diagnostic->reason, sizeof(diagnostic->reason), + "external-allowed-mask-is-compute-pool"); + return; + } + if (!input->topology_available) { + cpu_policy_fallback(profile, policy, diagnostic, + "fallback-portable-topology-unavailable"); + memcpy(diagnostic->allowed_mask, allowed, sizeof(allowed)); + return; + } + + uint64_t compute[LARDON3D_RESOURCE_CPU_MASK_WORDS]; + uint64_t reserved[LARDON3D_RESOURCE_CPU_MASK_WORDS] = {0}; + memcpy(compute, allowed, sizeof(compute)); + unsigned int compute_count = (unsigned int)input->allowed_cpu_count; + unsigned int reserved_count = 0; + while (reserved_count < policy->system_cpu_reserve) { + bool group_found = false; + unsigned int selected_package = 0; + unsigned int selected_core = 0; + for (size_t index = 0; index < input->allowed_cpu_count; ++index) { + unsigned int cpu = input->allowed_cpu_ids[index]; + if (!cpu_mask_test(compute, cpu)) { + continue; + } + if (!group_found || package_ids[cpu] > selected_package + || (package_ids[cpu] == selected_package + && core_ids[cpu] > selected_core)) { + selected_package = package_ids[cpu]; + selected_core = core_ids[cpu]; + group_found = true; + } + } + unsigned int group_count = 0; + if (group_found) { + for (size_t index = 0; index < input->allowed_cpu_count; ++index) { + unsigned int cpu = input->allowed_cpu_ids[index]; + if (cpu_mask_test(compute, cpu) + && package_ids[cpu] == selected_package + && core_ids[cpu] == selected_core) { + ++group_count; + } + } + } + if (!group_found || group_count == 0 || group_count >= compute_count) { + cpu_policy_fallback(profile, policy, diagnostic, + "fallback-portable-topology-incomplete"); + memcpy(diagnostic->allowed_mask, allowed, sizeof(allowed)); + return; + } + for (size_t index = 0; index < input->allowed_cpu_count; ++index) { + unsigned int cpu = input->allowed_cpu_ids[index]; + if (cpu_mask_test(compute, cpu) + && package_ids[cpu] == selected_package + && core_ids[cpu] == selected_core) { + compute[cpu / 64] &= ~(UINT64_C(1) << (cpu % 64)); + cpu_mask_set(reserved, cpu); + } + } + compute_count -= group_count; + reserved_count += group_count; + } + diagnostic->affinity_configured = true; + diagnostic->compute_cpu_count = compute_count; + diagnostic->reserved_cpu_count = reserved_count; + memcpy(diagnostic->compute_mask, compute, sizeof(compute)); + memcpy(diagnostic->reserved_mask, reserved, sizeof(reserved)); + (void)snprintf(diagnostic->reason, sizeof(diagnostic->reason), + "topology-complete-core-reserve"); +} + +static bool +parse_topology_value( + const char *text, + size_t length, + unsigned int *value +) +{ + if (!text || length == 0 || !value) { + return false; + } + size_t index = 0; + unsigned int parsed = 0; + while (index < length && text[index] >= '0' && text[index] <= '9') { + unsigned int digit = (unsigned int)(text[index] - '0'); + if (parsed > (UINT_MAX - digit) / 10U) { + return false; + } + parsed = parsed * 10U + digit; + ++index; + } + if (index == 0) { + return false; + } + for (; index < length; ++index) { + if (text[index] != ' ' && text[index] != '\t' + && text[index] != '\r' && text[index] != '\n') { + return false; + } + } + *value = parsed; + return true; +} + +static bool +read_topology_value_file(const char *path, unsigned int *value) +{ +#ifdef __linux__ + if (!path || !value) { + return false; + } + int descriptor = open(path, O_RDONLY | O_CLOEXEC); + if (descriptor < 0) { + return false; + } + char text[32]; + size_t used = 0; + bool eof = false; + bool read_ok = true; + while (used < sizeof(text) && !eof) { + ssize_t count; + do { + count = read(descriptor, text + used, sizeof(text) - used); + } while (count < 0 && errno == EINTR); + if (count < 0) { + read_ok = false; + } else if (count == 0) { + eof = true; + } else { + used += (size_t)count; + } + if (!read_ok) { + break; + } + } + if (read_ok && !eof) { + char extra; + ssize_t count; + do { + count = read(descriptor, &extra, 1); + } while (count < 0 && errno == EINTR); + read_ok = count == 0; + eof = count == 0; + } + bool closed = close(descriptor) == 0; + /* sysfs values are tiny. EOF is part of the contract: accepting a full + * buffer without the sentinel read would silently parse a truncated CPU + * identity and could reserve the wrong physical core group. */ + return read_ok && eof && closed + && parse_topology_value(text, used, value); +#else + (void)path; + (void)value; + return false; +#endif +} + +static bool +read_topology_value(unsigned int cpu, const char *field, unsigned int *value) +{ +#ifdef __linux__ + char path[160]; + int written = snprintf(path, sizeof(path), + "/sys/devices/system/cpu/cpu%u/topology/%s", cpu, field); + return written >= 0 && (size_t)written < sizeof(path) + && read_topology_value_file(path, value); +#else + (void)cpu; + (void)field; + (void)value; + return false; +#endif +} + +bool +lardon3d_resource_governor_internal_read_topology_value_file( + const char *path, + unsigned int *value +) +{ + /* Private deterministic seam: tests use ordinary temporary files to + * exercise the exact bounded sysfs reader without mutating host sysfs. */ + return read_topology_value_file(path, value); +} + +static void +capture_live_cpu_topology( + const Lardon3DHardwareProfile *profile, + Lardon3DResourceCpuTopologyInput *input +) +{ + *input = (Lardon3DResourceCpuTopologyInput) {0}; +#ifdef __linux__ + long system_cpu_count = sysconf(_SC_NPROCESSORS_ONLN); + if (system_cpu_count < 1 + || (unsigned long)system_cpu_count != profile->logical_cpu_count + || profile->logical_cpu_count > LARDON3D_RESOURCE_CPU_MAX) { + return; + } + cpu_set_t allowed; + CPU_ZERO(&allowed); + if (sched_getaffinity(0, sizeof(allowed), &allowed) != 0) { + return; + } + for (unsigned int cpu = 0; cpu < LARDON3D_RESOURCE_CPU_MAX + && cpu < CPU_SETSIZE; ++cpu) { + if (CPU_ISSET((size_t)cpu, &allowed)) { + input->allowed_cpu_ids[input->allowed_cpu_count++] = cpu; + } + } + if (input->allowed_cpu_count == 0) { + *input = (Lardon3DResourceCpuTopologyInput) {0}; + return; + } + input->affinity_available = true; + input->topology_available = true; + for (size_t index = 0; index < input->allowed_cpu_count; ++index) { + unsigned int cpu = input->allowed_cpu_ids[index]; + Lardon3DResourceCpuTopologyEntry *entry = + &input->topology_entries[input->topology_entry_count]; + entry->cpu_id = cpu; + if (!read_topology_value(cpu, "physical_package_id", + &entry->package_id) + || !read_topology_value(cpu, "core_id", &entry->core_id)) { + input->topology_available = false; + input->topology_entry_count = 0; + return; + } + ++input->topology_entry_count; + } +#else + (void)profile; +#endif +} + static bool valid_profile(const Lardon3DHardwareProfile *profile) { @@ -222,6 +1168,468 @@ resource_size( return true; } +static bool +add_uint64(uint64_t left, uint64_t right, uint64_t *sum) +{ + if (!sum || left > UINT64_MAX - right) { + return false; + } + *sum = left + right; + return true; +} + +static void +aggregate_add(uint64_t *value, uint64_t amount, bool *saturated) +{ + if (*value > UINT64_MAX - amount) { + *value = UINT64_MAX; + *saturated = true; + } else { + *value += amount; + } +} + +static void +aggregate_admission( + Lardon3DResourceGovernor *governor, + Lardon3DCapabilityFeedback *feedback, + const Lardon3DResourceCapabilitySelection *selection +) +{ + Lardon3DResourceSequenceAggregate *aggregate = &feedback->aggregate; + aggregate_add(&aggregate->admission_count, 1, &aggregate->saturated); + aggregate_add( + &aggregate->selected_backend_admissions[selection->capability.backend], + 1, &aggregate->saturated); + /* CONTRACT: a backend switch is a contract change too. Feedback remains + * keyed by backend for adaptation, so compare this admission with the most + * recently updated selection for the whole kind/version while the Governor + * lock is held. The private update-order tie-break is bounded and already + * renormalized by diagnostic ownership; no history or persistence grows. */ + const Lardon3DCapabilityFeedback *previous = NULL; + for (size_t index = 0; index < LARDON3D_CAPABILITY_FEEDBACK_CAPACITY; + ++index) { + const Lardon3DCapabilityFeedback *candidate = + &governor->capability_feedback[index]; + if (candidate->used && candidate->aggregate_selection_known + && candidate->task_kind_version == feedback->task_kind_version + && strcmp(candidate->task_kind, feedback->task_kind) == 0 + && (!previous || candidate->diagnostic_update_order + > previous->diagnostic_update_order)) { + previous = candidate; + } + } + bool changed = previous + && (previous->backend != selection->capability.backend + || previous->aggregate_cpu_threads + != selection->decision.cpu_threads + || previous->aggregate_gpu_slots != selection->decision.gpu_slots + || previous->aggregate_io_slots != selection->decision.io_slots + || previous->aggregate_batch_size != selection->decision.batch_size + || previous->aggregate_inflight_limit + != selection->inflight_limit + || previous->aggregate_helper_limit + != selection->capability.helper_limit); + if (changed) { + aggregate_add(&aggregate->contract_change_count, 1, + &aggregate->saturated); + } + feedback->aggregate_selection_known = true; + feedback->aggregate_cpu_threads = selection->decision.cpu_threads; + feedback->aggregate_gpu_slots = selection->decision.gpu_slots; + feedback->aggregate_io_slots = selection->decision.io_slots; + feedback->aggregate_batch_size = selection->decision.batch_size; + feedback->aggregate_inflight_limit = selection->inflight_limit; + feedback->aggregate_helper_limit = selection->capability.helper_limit; +} + +static void +aggregate_sequence( + Lardon3DCapabilityFeedback *feedback, + uint64_t wall_time_ns, + size_t items_completed, + Lardon3DResourceBackend actual_backend, + const char *backend_reason, + const Lardon3DResourceExecutionMetrics *metrics +) +{ + Lardon3DResourceSequenceAggregate *aggregate = &feedback->aggregate; + uint64_t items = (uint64_t)items_completed; + if ((size_t)items != items_completed) { + items = UINT64_MAX; + aggregate->saturated = true; + } + aggregate_add(&aggregate->sequence_count, 1, &aggregate->saturated); + aggregate_add(&aggregate->durable_items, items, &aggregate->saturated); + aggregate_add(&aggregate->total_wall_time_ns, wall_time_ns, + &aggregate->saturated); + aggregate_add(&aggregate->actual_backend_sequences[actual_backend], 1, + &aggregate->saturated); + if (actual_backend != feedback->backend) { + aggregate_add(&aggregate->backend_fallback_sequences, 1, + &aggregate->saturated); + if (strcmp(backend_reason, + "vulkan-and-ineligible-pair-cpu-fallback") == 0 + || strcmp(backend_reason, + "vulkan-ineligible-whole-pair-cpu-fallback") == 0) { + aggregate_add( + &aggregate->backend_ineligible_fallback_sequences, 1, + &aggregate->saturated); + } else if (strcmp(backend_reason, + "vulkan-and-whole-pair-cpu-fallback") == 0 + || strcmp(backend_reason, + "vulkan-failed-whole-pair-cpu-fallback") == 0) { + aggregate_add(&aggregate->backend_failure_fallback_sequences, 1, + &aggregate->saturated); + } else { + /* Unknown fallback reasons fail closed in benchmark evidence. + * Keeping a separate bounded bucket prevents a new operational + * reason from being silently treated as local ineligibility. */ + aggregate_add(&aggregate->backend_other_fallback_sequences, 1, + &aggregate->saturated); + } + } + const Lardon3DResourceHostTelemetry *host = &feedback->diagnostic.host; + if (host->memory_available_known + && (!aggregate->memory_available_known + || host->memory_available_bytes + < aggregate->minimum_memory_available_bytes)) { + aggregate->memory_available_known = true; + aggregate->minimum_memory_available_bytes = + host->memory_available_bytes; + } + if (host->gpu_busy_known + && (!aggregate->gpu_busy_known + || host->gpu_busy_basis_points + > aggregate->maximum_gpu_busy_basis_points)) { + aggregate->gpu_busy_known = true; + aggregate->maximum_gpu_busy_basis_points = + host->gpu_busy_basis_points; + } + if (host->process_rss_known + && (!aggregate->process_rss_known + || host->process_rss_bytes > aggregate->maximum_process_rss_bytes)) { + aggregate->process_rss_known = true; + aggregate->maximum_process_rss_bytes = host->process_rss_bytes; + } + if (host->process_peak_rss_known + && (!aggregate->process_peak_rss_known + || host->process_peak_rss_bytes + > aggregate->maximum_process_peak_rss_bytes)) { + aggregate->process_peak_rss_known = true; + aggregate->maximum_process_peak_rss_bytes = + host->process_peak_rss_bytes; + } + if (!metrics) return; +#define AGGREGATE_METRIC(field) \ + aggregate_add(&aggregate->field, metrics->field, &aggregate->saturated) + AGGREGATE_METRIC(vulkan_submits); + AGGREGATE_METRIC(vulkan_completions); + AGGREGATE_METRIC(vulkan_submit_cpu_ns); + AGGREGATE_METRIC(vulkan_fence_wait_ns); + AGGREGATE_METRIC(vulkan_readback_ns); + if (metrics->vulkan_gpu_time_known) { + aggregate_add(&aggregate->vulkan_gpu_known_sequences, 1, + &aggregate->saturated); + AGGREGATE_METRIC(vulkan_gpu_ns); + } + AGGREGATE_METRIC(vulkan_starvation_ns); + AGGREGATE_METRIC(matcher_cpu_ns); + AGGREGATE_METRIC(publication_ns); +#undef AGGREGATE_METRIC +} + +static Lardon3DCapabilityFeedback * +capability_feedback_locked( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + Lardon3DResourceBackend backend, + bool create +) +{ + if (!task_kind || !task_kind[0]) { + return NULL; + } + for (size_t index = 0; index < LARDON3D_CAPABILITY_FEEDBACK_CAPACITY; + ++index) { + Lardon3DCapabilityFeedback *feedback = + &governor->capability_feedback[index]; + if (feedback->used && feedback->task_kind_version == task_kind_version + && feedback->backend == backend + && strcmp(feedback->task_kind, task_kind) == 0) { + return feedback; + } + } + if (!create) { + return NULL; + } + size_t selected = LARDON3D_CAPABILITY_FEEDBACK_CAPACITY; + for (size_t index = 0; index < LARDON3D_CAPABILITY_FEEDBACK_CAPACITY; + ++index) { + if (!governor->capability_feedback[index].used) { + selected = index; + break; + } + } + if (selected == LARDON3D_CAPABILITY_FEEDBACK_CAPACITY) { + /* The history is deliberately bounded. Replacement is deterministic + * and does not affect durable Task state or scientific output. */ + selected = governor->capability_feedback_replace; + governor->capability_feedback_replace = + (selected + 1) % LARDON3D_CAPABILITY_FEEDBACK_CAPACITY; + } + Lardon3DCapabilityFeedback *feedback = + &governor->capability_feedback[selected]; + *feedback = (Lardon3DCapabilityFeedback) { + .used = true, + .task_kind_version = task_kind_version, + .backend = backend, + }; + (void)snprintf(feedback->task_kind, sizeof(feedback->task_kind), "%s", + task_kind); + return feedback; +} + +static void +clear_baseline_rate_observations(Lardon3DCapabilityFeedback *feedback) +{ + feedback->baseline_observations = 0; + memset(feedback->baseline_rates_milli, 0, + sizeof(feedback->baseline_rates_milli)); +} + +static void +clear_trial_rate_observations(Lardon3DCapabilityFeedback *feedback) +{ + feedback->trial_observations = 0; + memset(feedback->trial_rates_milli, 0, + sizeof(feedback->trial_rates_milli)); +} + +static unsigned int +required_throughput_observations( + const Lardon3DTaskCapability *capability, + Lardon3DCapabilityTrialDimension dimension +) +{ + return capability->sustained_gpu_batch_feedback + && (dimension == LARDON3D_CAPABILITY_TRIAL_NONE + || dimension == LARDON3D_CAPABILITY_TRIAL_BATCH) + ? LARDON3D_GPU_BATCH_THROUGHPUT_OBSERVATIONS + : LARDON3D_CAPABILITY_THROUGHPUT_OBSERVATIONS; +} + +static uint64_t +average_rate_observations(const uint64_t *rates, unsigned int count) +{ + if (!rates || count == 0 + || count > LARDON3D_GPU_BATCH_THROUGHPUT_OBSERVATIONS) { + return 0; + } + uint64_t quotient_sum = 0; + unsigned int remainder_sum = 0; + for (unsigned int index = 0; index < count; ++index) { + /* Sum floor(rate/count), then the bounded remainders. With at most + * `count` UINT64 inputs the quotient sum cannot exceed UINT64_MAX; + * this computes floor(sum/count) without ever forming `sum`. */ + quotient_sum += rates[index] / count; + remainder_sum += (unsigned int)(rates[index] % count); + } + return quotient_sum + remainder_sum / count; +} + +static const char * +trial_observation_reason( + const Lardon3DTaskCapability *capability, + Lardon3DCapabilityTrialDimension dimension +) +{ + if (dimension == LARDON3D_CAPABILITY_TRIAL_CPU) { + return "cpu-throughput-trial"; + } + if (dimension == LARDON3D_CAPABILITY_TRIAL_INFLIGHT) { + return "inflight-throughput-trial"; + } + return capability->sustained_gpu_batch_feedback + ? "gpu-batch-throughput-trial" : "throughput-trial"; +} + +static const char * +trial_result_reason( + const Lardon3DTaskCapability *capability, + Lardon3DCapabilityTrialDimension dimension, + bool improved +) +{ + if (dimension == LARDON3D_CAPABILITY_TRIAL_CPU) { + return improved + ? "cpu-throughput-improved" : "cpu-throughput-no-gain"; + } + if (dimension == LARDON3D_CAPABILITY_TRIAL_INFLIGHT) { + return improved + ? "inflight-throughput-improved" + : "inflight-throughput-no-gain"; + } + if (capability->sustained_gpu_batch_feedback) { + return improved + ? "gpu-batch-throughput-improved" + : "gpu-batch-throughput-no-gain"; + } + return improved ? "throughput-improved" : "throughput-no-gain"; +} + +static void +reset_capability_throughput_feedback( + Lardon3DCapabilityFeedback *feedback, + const Lardon3DTaskCapability *capability +) +{ + /* Pressure abandons any trial immediately. Recovery starts from the full + * capability-owned baseline window (eight for ORB Vulkan batch, two for + * generic adaptation); no stale fast sample can make the next immutable + * sequence jump back to a memory-heavy limit. */ + unsigned int minimum_cpu = capability->cpu_reducible + ? 1 : capability->estimate.desired_cpu_threads; + feedback->adaptive_cpu_limit = minimum_cpu; + feedback->accepted_cpu_limit = minimum_cpu; + feedback->adaptive_batch_limit = capability->batch_adaptive + ? capability->estimate.minimum_batch_size + : capability->estimate.maximum_batch_size; + feedback->accepted_batch_limit = feedback->adaptive_batch_limit; + size_t minimum_inflight = capability->minimum_inflight_limit != 0 + ? capability->minimum_inflight_limit : capability->inflight_limit; + feedback->adaptive_inflight_limit = minimum_inflight; + feedback->accepted_inflight_limit = minimum_inflight; + feedback->trial_dimension = LARDON3D_CAPABILITY_TRIAL_NONE; + clear_baseline_rate_observations(feedback); + clear_trial_rate_observations(feedback); + feedback->baseline_rate_milli = 0; + feedback->cpu_growth_stopped = false; + feedback->inflight_growth_stopped = false; + feedback->batch_growth_stopped = false; +} + +static bool +throughput_materially_improved(uint64_t trial, uint64_t baseline) +{ + if (baseline == 0) return trial > 0; + /* Five percent is a deadband, not a scientific threshold. Expressing it + * as baseline + ceil(baseline/20) avoids overflow-prone multiplication. */ + uint64_t increase = baseline / 20 + (baseline % 20 != 0); + return baseline <= UINT64_MAX - increase + && trial >= baseline + increase; +} + +static size_t +next_trial_batch(size_t current, size_t maximum) +{ + if (current >= maximum) return maximum; + return current > maximum / 2 ? maximum : current * 2; +} + +static unsigned int +next_trial_cpu(unsigned int current, unsigned int maximum) +{ + if (current >= maximum) return maximum; + unsigned int next = current < 2 ? 2 + : current < 4 ? 4 + : current < 8 ? 8 + : 12; + return next < maximum ? next : maximum; +} + +static void +open_next_capability_trial_locked( + const Lardon3DResourceGovernor *governor, + Lardon3DCapabilityFeedback *feedback, + const Lardon3DTaskCapability *capability +) +{ + unsigned int cpu_max = capability->estimate.desired_cpu_threads + < governor->cpu_policy.compute_cpu_count + ? capability->estimate.desired_cpu_threads + : governor->cpu_policy.compute_cpu_count; + if (capability->cpu_reducible && !feedback->cpu_growth_stopped + && feedback->accepted_cpu_limit < cpu_max) { + feedback->adaptive_cpu_limit = next_trial_cpu( + feedback->accepted_cpu_limit, cpu_max); + feedback->adaptive_batch_limit = feedback->accepted_batch_limit; + feedback->adaptive_inflight_limit = + feedback->accepted_inflight_limit; + feedback->trial_dimension = LARDON3D_CAPABILITY_TRIAL_CPU; + clear_trial_rate_observations(feedback); + return; + } + if (capability->inflight_adaptive + && !feedback->inflight_growth_stopped + && feedback->accepted_inflight_limit < capability->inflight_limit) { + /* Depth two cannot be exercised by a one-item sequence. Establish an + * accepted batch of at least two first, changing only that dimension; + * if batch two has already shown no gain, do not run a fake inflight + * trial whose second slot can never be used. */ + if (capability->batch_adaptive + && feedback->accepted_batch_limit < 2) { + if (!feedback->batch_growth_stopped + && capability->estimate.maximum_batch_size >= 2) { + feedback->adaptive_cpu_limit = feedback->accepted_cpu_limit; + feedback->adaptive_inflight_limit = + feedback->accepted_inflight_limit; + feedback->adaptive_batch_limit = 2; + feedback->trial_dimension = LARDON3D_CAPABILITY_TRIAL_BATCH; + clear_trial_rate_observations(feedback); + } + return; + } + feedback->adaptive_cpu_limit = feedback->accepted_cpu_limit; + feedback->adaptive_inflight_limit = + feedback->accepted_inflight_limit; + feedback->adaptive_batch_limit = feedback->accepted_batch_limit; + feedback->adaptive_inflight_limit = next_trial_batch( + feedback->accepted_inflight_limit, capability->inflight_limit); + feedback->trial_dimension = LARDON3D_CAPABILITY_TRIAL_INFLIGHT; + clear_trial_rate_observations(feedback); + return; + } + if (capability->batch_adaptive && !feedback->batch_growth_stopped + && feedback->accepted_batch_limit + < capability->estimate.maximum_batch_size) { + feedback->adaptive_cpu_limit = feedback->accepted_cpu_limit; + feedback->adaptive_batch_limit = next_trial_batch( + feedback->accepted_batch_limit, + capability->estimate.maximum_batch_size); + feedback->trial_dimension = LARDON3D_CAPABILITY_TRIAL_BATCH; + clear_trial_rate_observations(feedback); + return; + } + feedback->adaptive_cpu_limit = feedback->accepted_cpu_limit; + feedback->adaptive_batch_limit = feedback->accepted_batch_limit; + feedback->adaptive_inflight_limit = feedback->accepted_inflight_limit; + feedback->trial_dimension = LARDON3D_CAPABILITY_TRIAL_NONE; +} + +static void +internal_pressure_floors_locked( + const Lardon3DResourceGovernor *governor, + uint64_t *soft_floor, + uint64_t *hard_floor +) +{ + *soft_floor = governor->policy.system_memory_reserve_bytes; + *hard_floor = governor->policy.emergency_memory_floor_bytes; + const uint64_t gib = 1024ULL * 1024ULL * 1024ULL; + const uint64_t total = governor->profile.memory_total_bytes; + /* Compute Governor v2's validated current-host operating point is scoped + * to the default 16 GiB policy. Generic Gate G defaults remain unchanged + * for every other profile or an explicitly customized policy. */ + if (total >= 15 * gib && total <= 17 * gib + && governor->policy.system_memory_reserve_bytes == total / 4 + && governor->policy.emergency_memory_floor_bytes == total / 8) { + *soft_floor = 3 * gib; + *hard_floor = 2 * gib; + } +} + static size_t batch_capacity(uint64_t available, uint64_t fixed, uint64_t per_item) { @@ -415,6 +1823,9 @@ lardon3d_resource_governor_create( governor->generation = 0; governor->profile = *profile; governor->policy = *policy; + capture_live_cpu_topology(profile, &governor->cpu_topology); + build_cpu_policy(profile, policy, &governor->cpu_topology, + &governor->cpu_policy); governor->next_reservation_id = 1; governor->slow_start_limit = SIZE_MAX; governor->pressure = LARDON3D_RESOURCE_PRESSURE_GREEN; @@ -464,15 +1875,22 @@ lardon3d_resource_governor_set_policy( ? governor->profile.gpu_memory_total_bytes - policy->gpu_memory_reserve_bytes : UINT64_MAX; + Lardon3DResourceCpuPolicyDiagnostic cpu_policy; + if (policy_valid) { + build_cpu_policy(&governor->profile, policy, &governor->cpu_topology, + &cpu_policy); + } else { + memset(&cpu_policy, 0, sizeof(cpu_policy)); + } bool accepted = policy_valid && memory_budget >= governor->memory_reserved_bytes && gpu_budget >= governor->gpu_memory_reserved_bytes - && governor->profile.logical_cpu_count - policy->system_cpu_reserve - >= governor->cpu_reserved + && cpu_policy.compute_cpu_count >= governor->cpu_reserved && policy->gpu_slot_capacity >= governor->gpu_slots_reserved && policy->io_slot_capacity >= governor->io_slots_reserved; if (accepted) { governor->policy = *policy; + governor->cpu_policy = cpu_policy; } (void)pthread_mutex_unlock(&governor->mutex); return accepted; @@ -482,6 +1900,7 @@ static bool availability_locked( const Lardon3DResourceGovernor *governor, const Lardon3DResourceSnapshot *snapshot, + uint64_t memory_reserve_bytes, Lardon3DResourceAvailability *availability ) { @@ -494,7 +1913,7 @@ availability_locked( ); uint64_t memory_budget = subtract_floor( detected_memory, - governor->policy.system_memory_reserve_bytes + memory_reserve_bytes ); uint64_t gpu_budget = 0; bool gpu_known = false; @@ -509,8 +1928,7 @@ availability_locked( governor->policy.gpu_memory_reserve_bytes ); } - unsigned int cpu_budget = governor->profile.logical_cpu_count - - governor->policy.system_cpu_reserve; + unsigned int cpu_budget = governor->cpu_policy.compute_cpu_count; *availability = (Lardon3DResourceAvailability) { .memory_budget_bytes = memory_budget, .memory_reserved_bytes = governor->memory_reserved_bytes, @@ -555,7 +1973,12 @@ lardon3d_resource_governor_availability( return false; } (void)pthread_mutex_lock(&governor->mutex); - bool success = availability_locked(governor, snapshot, availability); + bool success = availability_locked( + governor, + snapshot, + governor->policy.system_memory_reserve_bytes, + availability + ); (void)pthread_mutex_unlock(&governor->mutex); return success; } @@ -621,7 +2044,10 @@ evaluate_locked( Lardon3DResourceGovernor *governor, const Lardon3DResourceSnapshot *snapshot, const Lardon3DResourceEstimate *estimate, - Lardon3DResourceDecision *decision + Lardon3DResourceDecision *decision, + bool update_pressure, + uint64_t soft_memory_floor, + uint64_t hard_memory_floor ) { if (!valid_estimate(estimate)) { @@ -637,22 +2063,6 @@ evaluate_locked( "Instantané de ressources périmé."); return true; } - bool swap_changed = false; - if (snapshot->swap_activity_known) { - if (governor->swap_baseline_known) { - swap_changed = snapshot->swap_pages_in > governor->last_swap_pages_in - || snapshot->swap_pages_out > governor->last_swap_pages_out; - } - governor->last_swap_pages_in = snapshot->swap_pages_in; - governor->last_swap_pages_out = snapshot->swap_pages_out; - governor->swap_baseline_known = true; - } - bool hard_memory_pressure = governor->policy.emergency_memory_floor_bytes > 0 - && snapshot->memory_available_bytes - <= governor->policy.emergency_memory_floor_bytes; - bool soft_memory_pressure = governor->policy.system_memory_reserve_bytes > 0 - && snapshot->memory_available_bytes - <= governor->policy.system_memory_reserve_bytes; bool psi_pressure = (governor->policy.maximum_cpu_pressure_avg10 > 0.0 && snapshot->cpu_pressure_known && snapshot->cpu_pressure_avg10 @@ -665,68 +2075,88 @@ evaluate_locked( && snapshot->io_pressure_known && snapshot->io_pressure_avg10 >= governor->policy.maximum_io_pressure_avg10); - bool pressure_signal = soft_memory_pressure || psi_pressure || swap_changed; + if (update_pressure) { + bool swap_changed = false; + if (snapshot->swap_activity_known) { + if (governor->swap_baseline_known) { + swap_changed = snapshot->swap_pages_in + > governor->last_swap_pages_in + || snapshot->swap_pages_out + > governor->last_swap_pages_out; + } + governor->last_swap_pages_in = snapshot->swap_pages_in; + governor->last_swap_pages_out = snapshot->swap_pages_out; + governor->swap_baseline_known = true; + } + bool hard_memory_pressure = hard_memory_floor > 0 + && snapshot->memory_available_bytes <= hard_memory_floor; + bool soft_memory_pressure = soft_memory_floor > 0 + && snapshot->memory_available_bytes <= soft_memory_floor; + bool pressure_signal = soft_memory_pressure || psi_pressure + || swap_changed; - if (hard_memory_pressure) { - governor->pressure = LARDON3D_RESOURCE_PRESSURE_RED; - governor->pressure_streak = 0; - governor->recovery_streak = 0; - governor->slow_start_streak = 0; - governor->slow_start_limit = 1; - governor->slow_start_active = true; - } else if (pressure_signal) { - governor->recovery_streak = 0; - governor->slow_start_streak = 0; - governor->pressure_streak = increment_to_limit( - governor->pressure_streak, - 2 - ); - if (governor->pressure == LARDON3D_RESOURCE_PRESSURE_RED - || governor->pressure_streak >= 2) { + if (hard_memory_pressure) { governor->pressure = LARDON3D_RESOURCE_PRESSURE_RED; + governor->pressure_streak = 0; + governor->recovery_streak = 0; + governor->slow_start_streak = 0; governor->slow_start_limit = 1; governor->slow_start_active = true; - } else { - governor->pressure = LARDON3D_RESOURCE_PRESSURE_YELLOW; - } - } else { - governor->pressure_streak = 0; - if (governor->pressure == LARDON3D_RESOURCE_PRESSURE_RED) { - governor->recovery_streak = increment_to_limit( - governor->recovery_streak, - 3 + } else if (pressure_signal) { + governor->recovery_streak = 0; + governor->slow_start_streak = 0; + governor->pressure_streak = increment_to_limit( + governor->pressure_streak, + 2 ); - if (governor->recovery_streak >= 3) { - governor->pressure = LARDON3D_RESOURCE_PRESSURE_YELLOW; - governor->recovery_streak = 0; - } - } else if (governor->pressure == LARDON3D_RESOURCE_PRESSURE_YELLOW) { - if (!governor->slow_start_active) { - governor->slow_start_active = true; + if (governor->pressure == LARDON3D_RESOURCE_PRESSURE_RED + || governor->pressure_streak >= 2) { + governor->pressure = LARDON3D_RESOURCE_PRESSURE_RED; governor->slow_start_limit = 1; - governor->slow_start_streak = 0; + governor->slow_start_active = true; + } else { + governor->pressure = LARDON3D_RESOURCE_PRESSURE_YELLOW; } - governor->recovery_streak = increment_to_limit( - governor->recovery_streak, - 3 - ); - if (governor->recovery_streak >= 3) { - governor->pressure = LARDON3D_RESOURCE_PRESSURE_GREEN; - governor->recovery_streak = 0; - governor->slow_start_streak = 0; - } - } else if (governor->slow_start_active) { - governor->slow_start_streak = increment_to_limit( - governor->slow_start_streak, - 3 - ); - if (governor->slow_start_streak >= 3) { - governor->slow_start_streak = 0; - if (governor->slow_start_limit > SIZE_MAX / 2) { - governor->slow_start_limit = SIZE_MAX; - governor->slow_start_active = false; - } else { - governor->slow_start_limit *= 2; + } else { + governor->pressure_streak = 0; + if (governor->pressure == LARDON3D_RESOURCE_PRESSURE_RED) { + governor->recovery_streak = increment_to_limit( + governor->recovery_streak, + 3 + ); + if (governor->recovery_streak >= 3) { + governor->pressure = LARDON3D_RESOURCE_PRESSURE_YELLOW; + governor->recovery_streak = 0; + } + } else if (governor->pressure + == LARDON3D_RESOURCE_PRESSURE_YELLOW) { + if (!governor->slow_start_active) { + governor->slow_start_active = true; + governor->slow_start_limit = 1; + governor->slow_start_streak = 0; + } + governor->recovery_streak = increment_to_limit( + governor->recovery_streak, + 3 + ); + if (governor->recovery_streak >= 3) { + governor->pressure = LARDON3D_RESOURCE_PRESSURE_GREEN; + governor->recovery_streak = 0; + governor->slow_start_streak = 0; + } + } else if (governor->slow_start_active) { + governor->slow_start_streak = increment_to_limit( + governor->slow_start_streak, + 3 + ); + if (governor->slow_start_streak >= 3) { + governor->slow_start_streak = 0; + if (governor->slow_start_limit > SIZE_MAX / 2) { + governor->slow_start_limit = SIZE_MAX; + governor->slow_start_active = false; + } else { + governor->slow_start_limit *= 2; + } } } } @@ -759,7 +2189,7 @@ evaluate_locked( size_t theoretical_batch = batch_capacity( governor->profile.memory_total_bytes - - governor->policy.system_memory_reserve_bytes, + - soft_memory_floor, memory_fixed, memory_per_item ); @@ -812,7 +2242,12 @@ evaluate_locked( } Lardon3DResourceAvailability available; - if (!availability_locked(governor, snapshot, &available)) { + if (!availability_locked( + governor, + snapshot, + soft_memory_floor, + &available + )) { set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Instantané de ressources invalide."); return true; } @@ -908,6 +2343,71 @@ evaluate_locked( return true; } +static bool +activate_reservation_locked( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceEstimate *estimate, + const Lardon3DResourceDecision *decision, + Lardon3DResourceReservation *created +) +{ + if (governor->next_reservation_id == 0) { + return false; + } + uint64_t memory_bytes; + uint64_t gpu_memory_bytes; + uint64_t charged_memory_bytes = 0; + struct timespec created_at; + if (!resource_size( + estimate->memory_fixed_bytes, + estimate->memory_bytes_per_item, + decision->batch_size, + &memory_bytes + ) + || !resource_size( + estimate->gpu_memory_fixed_bytes, + estimate->gpu_memory_bytes_per_item, + decision->batch_size, + &gpu_memory_bytes + ) + || (governor->profile.gpu_uses_shared_memory + && !add_uint64(memory_bytes, gpu_memory_bytes, + &charged_memory_bytes)) + || clock_gettime(CLOCK_REALTIME, &created_at) != 0) { + return false; + } + if (!governor->profile.gpu_uses_shared_memory) { + charged_memory_bytes = memory_bytes; + } + /* UMA memory is charged to the host reservation exactly once. The public + * diagnostic retains the GPU component, but it is not a second physical + * budget alongside charged_memory_bytes. */ + created->information = (Lardon3DResourceReservationInfo) { + .id = governor->next_reservation_id++, + .memory_bytes = memory_bytes, + .gpu_memory_bytes = gpu_memory_bytes, + .cpu_threads = decision->cpu_threads, + .gpu_slots = decision->gpu_slots, + .io_slots = decision->io_slots, + .batch_size = decision->batch_size, + .task_class = estimate->task_class, + .state = LARDON3D_RESERVATION_ACTIVE, + .created_at = created_at, + }; + created->charged_memory_bytes = charged_memory_bytes; + created->next = governor->active; + governor->active = created; + governor->memory_reserved_bytes += charged_memory_bytes; + governor->gpu_memory_reserved_bytes += gpu_memory_bytes; + governor->cpu_reserved += decision->cpu_threads; + governor->gpu_slots_reserved += decision->gpu_slots; + governor->io_slots_reserved += decision->io_slots; + ++governor->active_count; + ++governor->generation; + (void)pthread_cond_broadcast(&governor->cond); + return true; +} + bool lardon3d_resource_governor_reserve( Lardon3DResourceGovernor *governor, @@ -931,7 +2431,15 @@ lardon3d_resource_governor_reserve( free(created); return false; } - if (!evaluate_locked(governor, snapshot, estimate, decision)) { + if (!evaluate_locked( + governor, + snapshot, + estimate, + decision, + true, + governor->policy.system_memory_reserve_bytes, + governor->policy.emergency_memory_floor_bytes + )) { (void)pthread_mutex_unlock(&governor->mutex); free(created); return false; @@ -942,55 +2450,14 @@ lardon3d_resource_governor_reserve( free(created); return true; } - if (governor->next_reservation_id == 0) { + if (!activate_reservation_locked(governor, estimate, decision, created)) { set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, - "Identités de réservation épuisées."); + "Impossible d'activer la réservation."); (void)pthread_mutex_unlock(&governor->mutex); free(created); return false; } - uint64_t memory_bytes; - uint64_t gpu_memory_bytes; - if (!resource_size( - estimate->memory_fixed_bytes, - estimate->memory_bytes_per_item, - decision->batch_size, - &memory_bytes - ) - || !resource_size( - estimate->gpu_memory_fixed_bytes, - estimate->gpu_memory_bytes_per_item, - decision->batch_size, - &gpu_memory_bytes - ) - || clock_gettime(CLOCK_REALTIME, &created->information.created_at) != 0) { - (void)pthread_mutex_unlock(&governor->mutex); - free(created); - return false; - } - created->information.id = governor->next_reservation_id++; - created->information.memory_bytes = memory_bytes; - created->information.gpu_memory_bytes = gpu_memory_bytes; - created->information.cpu_threads = decision->cpu_threads; - created->information.gpu_slots = decision->gpu_slots; - created->information.io_slots = decision->io_slots; - created->information.batch_size = decision->batch_size; - created->information.task_class = estimate->task_class; - created->information.state = LARDON3D_RESERVATION_ACTIVE; - created->charged_memory_bytes = governor->profile.gpu_uses_shared_memory - ? memory_bytes + gpu_memory_bytes - : memory_bytes; - created->next = governor->active; - governor->active = created; - governor->memory_reserved_bytes += created->charged_memory_bytes; - governor->gpu_memory_reserved_bytes += gpu_memory_bytes; - governor->cpu_reserved += decision->cpu_threads; - governor->gpu_slots_reserved += decision->gpu_slots; - governor->io_slots_reserved += decision->io_slots; - ++governor->active_count; *reservation = created; - ++governor->generation; - (void)pthread_cond_broadcast(&governor->cond); (void)pthread_mutex_unlock(&governor->mutex); return true; } @@ -1239,11 +2706,1240 @@ lardon3d_resource_governor_decide( (void)pthread_mutex_unlock(&governor->mutex); return false; } - bool success = evaluate_locked(governor, snapshot, &estimate, decision); + bool success = evaluate_locked( + governor, + snapshot, + &estimate, + decision, + true, + governor->policy.system_memory_reserve_bytes, + governor->policy.emergency_memory_floor_bytes + ); (void)pthread_mutex_unlock(&governor->mutex); return success; } +static bool +valid_capability_envelope(const Lardon3DTaskCapabilityEnvelope *envelope) +{ + if (!envelope || envelope->count == 0 + || envelope->count > LARDON3D_RESOURCE_CAPABILITY_MAX) { + return false; + } + for (size_t index = 0; index < envelope->count; ++index) { + const Lardon3DTaskCapability *capability = + &envelope->capabilities[index]; + if (!valid_estimate(&capability->estimate) + || capability->backend < LARDON3D_RESOURCE_BACKEND_FIXED + || capability->backend > LARDON3D_RESOURCE_BACKEND_ORB_VULKAN + || capability->inflight_limit == 0) { + return false; + } + size_t minimum_inflight = capability->minimum_inflight_limit != 0 + ? capability->minimum_inflight_limit : capability->inflight_limit; + if (minimum_inflight > capability->inflight_limit + || (capability->inflight_adaptive + && (capability->minimum_inflight_limit == 0 + || capability->gpu_memory_bytes_per_inflight == 0)) + || (!capability->inflight_adaptive + && capability->minimum_inflight_limit != 0 + && capability->minimum_inflight_limit + != capability->inflight_limit) + || (capability->gpu_memory_bytes_per_inflight != 0 + && capability->inflight_limit + > UINT64_MAX / capability->gpu_memory_bytes_per_inflight) + || (capability->gpu_memory_bytes_per_inflight != 0 + && capability->estimate.gpu_memory_fixed_bytes + > UINT64_MAX + - (uint64_t)capability->inflight_limit + * capability->gpu_memory_bytes_per_inflight)) { + return false; + } + } + return true; +} + +static void +selection_reason( + Lardon3DResourceCapabilitySelection *selection, + const char *reason +) +{ + (void)snprintf(selection->reason, sizeof(selection->reason), "%s", reason); +} + +static bool +try_capability_locked( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceSnapshot *snapshot, + const char *task_kind, + uint32_t task_kind_version, + const Lardon3DTaskCapability *capability, + size_t capability_index, + bool update_pressure, + uint64_t soft_floor, + uint64_t hard_floor, + Lardon3DResourceCapabilitySelection *selection +) +{ + Lardon3DResourceEstimate operational = capability->estimate; + Lardon3DCapabilityFeedback *feedback = capability_feedback_locked( + governor, + task_kind, + task_kind_version, + capability->backend, + true + ); + if (!feedback) { + return false; + } + if (capability->cpu_reducible || capability->batch_adaptive + || capability->inflight_adaptive) { + if (feedback->adaptive_cpu_limit == 0 + || feedback->adaptive_batch_limit == 0 + || feedback->adaptive_inflight_limit == 0) { + reset_capability_throughput_feedback(feedback, capability); + } + } + size_t selected_inflight = capability->inflight_adaptive + ? feedback->adaptive_inflight_limit : capability->inflight_limit; + if (capability->cpu_reducible) { + operational.desired_cpu_threads = feedback->adaptive_cpu_limit; + } + if (capability->batch_adaptive) { + operational.maximum_batch_size = minimum_size( + operational.maximum_batch_size, + feedback->adaptive_batch_limit + ); + } + if (capability->gpu_memory_bytes_per_inflight != 0) { + uint64_t inflight_bytes = + (uint64_t)selected_inflight + * capability->gpu_memory_bytes_per_inflight; + if (!add_uint64(capability->estimate.gpu_memory_fixed_bytes, + inflight_bytes, &operational.gpu_memory_fixed_bytes)) { + return false; + } + } + Lardon3DResourceDecision decision; + if (!evaluate_locked( + governor, + snapshot, + &operational, + &decision, + update_pressure, + soft_floor, + hard_floor + )) { + return false; + } + if ((capability->batch_adaptive || capability->cpu_reducible + || capability->inflight_adaptive) + && governor->pressure != LARDON3D_RESOURCE_PRESSURE_GREEN) { + /* The generic Gate G evaluator deliberately reduces YELLOW batches + * gradually. A private adaptive capability has a stronger contract: + * the first active pressure signal abandons its memory-heavy trial in + * this admission, before the immutable sequence selection and its + * reservation are installed. WAIT/REJECT decisions remain untouched. */ + bool admitting = decision.kind == LARDON3D_RESOURCE_START + || decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH; + if (admitting) { + if (capability->batch_adaptive) { + decision.batch_size = operational.minimum_batch_size; + } + if (capability->cpu_reducible) { + decision.cpu_threads = 1; + } + if (capability->inflight_adaptive) { + selected_inflight = capability->minimum_inflight_limit; + } + if ((capability->batch_adaptive + && decision.batch_size + < capability->estimate.maximum_batch_size) + || (capability->cpu_reducible + && decision.cpu_threads + < capability->estimate.desired_cpu_threads) + || (capability->inflight_adaptive + && selected_inflight < capability->inflight_limit)) { + decision.kind = LARDON3D_RESOURCE_REDUCE_BATCH; + } + (void)snprintf(decision.reason, sizeof(decision.reason), + "Pression active : capacité adaptative réduite au minimum."); + } + reset_capability_throughput_feedback(feedback, capability); + } + if (capability->gpu_memory_bytes_per_inflight != 0) { + uint64_t inflight_bytes = + (uint64_t)selected_inflight + * capability->gpu_memory_bytes_per_inflight; + if (!add_uint64(capability->estimate.gpu_memory_fixed_bytes, + inflight_bytes, &operational.gpu_memory_fixed_bytes)) { + return false; + } + } + if ((decision.kind == LARDON3D_RESOURCE_START + || decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH) + && !capability->cpu_reducible + && decision.cpu_threads != capability->estimate.desired_cpu_threads) { + set_decision( + &decision, + LARDON3D_RESOURCE_WAIT, + 0, + 0, + 0, + 0, + "Le compte CPU fixe n'est pas disponible." + ); + } + *selection = (Lardon3DResourceCapabilitySelection) { + .capability_index = capability_index, + .capability = *capability, + .reservation_estimate = operational, + .decision = decision, + .inflight_limit = selected_inflight, + .pressure = governor->pressure, + }; + if (decision.kind == LARDON3D_RESOURCE_START + || decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH) { + if (governor->pressure != LARDON3D_RESOURCE_PRESSURE_GREEN) { + selection_reason(selection, "pressure-decrease"); + } else if (feedback->trial_dimension + == LARDON3D_CAPABILITY_TRIAL_CPU) { + selection_reason(selection, "cpu-throughput-trial"); + } else if (feedback->trial_dimension + == LARDON3D_CAPABILITY_TRIAL_INFLIGHT) { + selection_reason(selection, "inflight-throughput-trial"); + } else if (feedback->trial_dimension + == LARDON3D_CAPABILITY_TRIAL_BATCH) { + bool starvation_signal = capability->backend + == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN + && feedback->diagnostic.execution.vulkan_starvation_ns > 0; + bool low_or_unknown_busy = + !feedback->diagnostic.host.gpu_busy_known + || feedback->diagnostic.host.gpu_busy_basis_points < 8000; + selection_reason(selection, + capability->sustained_gpu_batch_feedback + ? "gpu-batch-throughput-trial" + : starvation_signal && low_or_unknown_busy + ? "gpu-starvation-throughput-trial" + : "throughput-trial"); + } else if (capability->batch_adaptive + && feedback->batch_growth_stopped) { + selection_reason(selection, + capability->sustained_gpu_batch_feedback + ? "gpu-batch-throughput-no-gain" + : "throughput-no-gain"); + } else if (capability->cpu_reducible + && feedback->cpu_growth_stopped) { + selection_reason(selection, "cpu-throughput-no-gain"); + } else if (capability->inflight_adaptive + && feedback->inflight_growth_stopped) { + selection_reason(selection, "inflight-throughput-no-gain"); + } else if ((capability->batch_adaptive + && operational.maximum_batch_size + < capability->estimate.maximum_batch_size) + || (capability->cpu_reducible + && operational.desired_cpu_threads + < capability->estimate.desired_cpu_threads) + || (capability->inflight_adaptive + && selected_inflight < capability->inflight_limit)) { + selection_reason(selection, "healthy-slow-start"); + } else if (capability->backend + == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN) { + selection_reason(selection, "gpu-first"); + } else { + selection_reason(selection, + capability->backend == LARDON3D_RESOURCE_BACKEND_FIXED + ? "fixed-envelope" : "cpu-selected"); + } + } + return true; +} + +static bool +capability_hardware_safe_locked( + const Lardon3DResourceGovernor *governor, + const Lardon3DTaskCapability *capability +) +{ + /* A capability maximum is an upper operational bound, not a mandatory + * allocation. Memory and UMA safety belong to evaluate_locked(), which + * sizes the exact immutable sequence contract against total RAM, current + * availability, pressure and both desktop floors. Rejecting here because + * the maximum batch does not fit would suppress a safe smaller contract + * before the Governor can reduce it. */ + return capability->backend == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN + && governor->profile.gpu_available; +} + +static bool +runtime_capability_available_locked( + const Lardon3DResourceGovernor *governor, + const Lardon3DTaskCapability *capability +) +{ + return !capability->requires_runtime_backend + || (governor->orb_vulkan_backend_available + && capability_hardware_safe_locked(governor, capability)); +} + +static void +advance_diagnostic_serial_locked( + Lardon3DResourceGovernor *governor, + Lardon3DCapabilityFeedback *feedback +) +{ + /* The public-like private pull cursor saturates instead of wrapping, so an + * ancient after_serial never observes a false new decision. A separate + * bounded update order preserves deterministic most-recent selection once + * multiple backend records share UINT64_MAX. It may be renormalized + * because no caller observes it and the Governor owns all 32 entries. */ + if (governor->capability_diagnostic_serial < UINT64_MAX) { + ++governor->capability_diagnostic_serial; + } + if (governor->capability_diagnostic_update_order == UINT64_MAX) { + for (size_t index = 0; + index < LARDON3D_CAPABILITY_FEEDBACK_CAPACITY; ++index) { + governor->capability_feedback[index].diagnostic_update_order = 0; + } + governor->capability_diagnostic_update_order = 0; + } + feedback->diagnostic_update_order = + ++governor->capability_diagnostic_update_order; + feedback->diagnostic_serial = governor->capability_diagnostic_serial; + feedback->diagnostic.serial = feedback->diagnostic_serial; +} + +bool +lardon3d_resource_governor_internal_reserve_capability( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceSnapshot *snapshot, + const char *task_kind, + uint32_t task_kind_version, + const Lardon3DTaskCapabilityEnvelope *envelope, + Lardon3DResourceCapabilitySelection *selection, + Lardon3DResourceReservation **reservation +) +{ + if (!governor || !snapshot || !selection || !reservation + || !valid_capability_envelope(envelope)) { + return false; + } + *reservation = NULL; + const char *feedback_kind = task_kind && task_kind[0] + ? task_kind : "task.untyped"; + Lardon3DResourceReservation *created = calloc(1, sizeof(*created)); + if (!created) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + if (!valid_snapshot(&governor->profile, snapshot)) { + (void)pthread_mutex_unlock(&governor->mutex); + free(created); + return false; + } + uint64_t soft_floor; + uint64_t hard_floor; + internal_pressure_floors_locked(governor, &soft_floor, &hard_floor); + bool pressure_updated = false; + bool saw_wait = false; + bool evaluated = false; + Lardon3DResourceCapabilitySelection last = {0}; + for (unsigned int preference = 0; preference < 2; ++preference) { + for (size_t index = 0; index < envelope->count; ++index) { + const Lardon3DTaskCapability *capability = + &envelope->capabilities[index]; + if (capability->preferred != (preference == 0)) { + continue; + } + if (!runtime_capability_available_locked(governor, capability)) { + continue; + } + Lardon3DResourceCapabilitySelection candidate; + if (!try_capability_locked( + governor, + snapshot, + feedback_kind, + task_kind_version, + capability, + index, + !pressure_updated, + soft_floor, + hard_floor, + &candidate + )) { + (void)pthread_mutex_unlock(&governor->mutex); + free(created); + return false; + } + pressure_updated = true; + evaluated = true; + last = candidate; + if (candidate.decision.kind == LARDON3D_RESOURCE_WAIT) { + saw_wait = true; + } + if (candidate.decision.kind != LARDON3D_RESOURCE_START + && candidate.decision.kind + != LARDON3D_RESOURCE_REDUCE_BATCH) { + continue; + } + if (candidate.capability.backend + == LARDON3D_RESOURCE_BACKEND_CPU + && envelope->count > 1 && index != 0) { + selection_reason( + &candidate, + "cpu-fallback-gpu-non-admitted" + ); + } + if (!activate_reservation_locked( + governor, + &candidate.reservation_estimate, + &candidate.decision, + created + )) { + (void)pthread_mutex_unlock(&governor->mutex); + free(created); + return false; + } + Lardon3DCapabilityFeedback *feedback = capability_feedback_locked( + governor, + feedback_kind, + task_kind_version, + candidate.capability.backend, + true + ); + uint64_t previous_wall_time_ns = + feedback->diagnostic.previous_wall_time_ns; + size_t previous_items_completed = + feedback->diagnostic.items_completed; + uint64_t previous_rate_milli = + feedback->diagnostic.durable_items_per_second_milli; + Lardon3DResourceExecutionMetrics previous_execution = + feedback->diagnostic.execution; + feedback->diagnostic = (Lardon3DResourceSequenceDiagnostic) { + .task_kind_version = task_kind_version, + .backend = candidate.capability.backend, + .actual_backend = candidate.capability.backend, + .previous_wall_time_ns = previous_wall_time_ns, + .items_completed = previous_items_completed, + .durable_items_per_second_milli = previous_rate_milli, + .execution = previous_execution, + .batch_size = candidate.decision.batch_size, + .inflight_limit = candidate.inflight_limit, + .helper_limit = candidate.capability.helper_limit, + .memory_bytes = created->information.memory_bytes, + .gpu_memory_bytes = created->information.gpu_memory_bytes, + .cpu_threads = candidate.decision.cpu_threads, + .gpu_slots = candidate.decision.gpu_slots, + .io_slots = candidate.decision.io_slots, + .pressure = candidate.pressure, + .host = governor->host_telemetry, + }; + (void)snprintf(feedback->diagnostic.task_kind, + sizeof(feedback->diagnostic.task_kind), "%s", feedback_kind); + (void)snprintf(feedback->diagnostic.reason, + sizeof(feedback->diagnostic.reason), "%s", candidate.reason); + (void)snprintf(feedback->diagnostic.backend_reason, + sizeof(feedback->diagnostic.backend_reason), + "selected-backend-executing"); + /* Admission changes and completed-sequence totals are deliberately + * separate: a failed/cancelled callback may have a contract but no + * durable work observation. Both remain fixed-size operational + * evidence owned by this Governor instance. */ + aggregate_admission(governor, feedback, &candidate); + advance_diagnostic_serial_locked(governor, feedback); + *selection = candidate; + *reservation = created; + (void)pthread_mutex_unlock(&governor->mutex); + return true; + } + } + if (!evaluated) { + set_decision(&last.decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, + "Aucune capacité d'exécution disponible."); + selection_reason(&last, "no-runtime-capability"); + } else if (saw_wait) { + set_decision(&last.decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, + "Aucune capacité n'est actuellement admissible."); + selection_reason(&last, "all-capabilities-wait"); + } + last.pressure = governor->pressure; + *selection = last; + (void)pthread_mutex_unlock(&governor->mutex); + free(created); + return true; +} + +bool +lardon3d_resource_governor_internal_reserve_capability_available( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + const Lardon3DTaskCapabilityEnvelope *envelope, + Lardon3DResourceCapabilitySelection *selection, + Lardon3DResourceReservation **reservation +) +{ + if (!governor) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + Lardon3DHardwareProfile profile = governor->profile; + bool force_capture_failure = governor->internal_force_capture_failure; + (void)pthread_mutex_unlock(&governor->mutex); + if (force_capture_failure) { + return false; + } + /* Optional telemetry is sampled before the stable Gate G snapshot so a + * slow/missing proc/sysfs reader cannot age an otherwise valid admission + * snapshot. Its failure leaves private fields unknown only. */ + capture_host_telemetry(governor); + Lardon3DResourceSnapshot snapshot; + if (!lardon3d_resource_snapshot_capture(&profile, &snapshot, NULL, 0)) { + return false; + } + return lardon3d_resource_governor_internal_reserve_capability( + governor, + &snapshot, + task_kind, + task_kind_version, + envelope, + selection, + reservation + ); +} + +bool +lardon3d_resource_governor_internal_record_sequence_execution( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + const Lardon3DResourceCapabilitySelection *selection, + uint64_t wall_time_ns, + size_t items_completed, + Lardon3DResourceBackend actual_backend, + const char *backend_reason +) +{ + return lardon3d_resource_governor_internal_record_sequence_execution_metrics( + governor, task_kind, task_kind_version, selection, wall_time_ns, + items_completed, actual_backend, backend_reason, NULL); +} + +bool +lardon3d_resource_governor_internal_record_sequence_execution_metrics( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + const Lardon3DResourceCapabilitySelection *selection, + uint64_t wall_time_ns, + size_t items_completed, + Lardon3DResourceBackend actual_backend, + const char *backend_reason, + const Lardon3DResourceExecutionMetrics *metrics +) +{ + if (!governor || !selection || !task_kind || !task_kind[0] + || wall_time_ns == 0 || !backend_reason + || actual_backend < LARDON3D_RESOURCE_BACKEND_FIXED + || actual_backend > LARDON3D_RESOURCE_BACKEND_MIXED) { + return false; + } + capture_host_telemetry(governor); + (void)pthread_mutex_lock(&governor->mutex); + Lardon3DCapabilityFeedback *feedback = capability_feedback_locked( + governor, + task_kind, + task_kind_version, + selection->capability.backend, + true + ); + if (!feedback) { + (void)pthread_mutex_unlock(&governor->mutex); + return false; + } + feedback->diagnostic.previous_wall_time_ns = wall_time_ns; + feedback->diagnostic.items_completed = items_completed; + feedback->diagnostic.actual_backend = actual_backend; + feedback->diagnostic.backend_fallback = + actual_backend != selection->capability.backend; + feedback->diagnostic.host = governor->host_telemetry; + feedback->diagnostic.execution = metrics + ? *metrics : (Lardon3DResourceExecutionMetrics){0}; + (void)snprintf(feedback->diagnostic.backend_reason, + sizeof(feedback->diagnostic.backend_reason), "%s", backend_reason); + if (items_completed > 0 + && items_completed <= UINT64_MAX / 1000000000000ULL) { + feedback->diagnostic.durable_items_per_second_milli = + (uint64_t)items_completed * 1000000000000ULL / wall_time_ns; + } else { + feedback->diagnostic.durable_items_per_second_milli = 0; + } + bool adaptive = selection->capability.cpu_reducible + || selection->capability.batch_adaptive + || selection->capability.inflight_adaptive; + if (adaptive) { + if (selection->pressure != LARDON3D_RESOURCE_PRESSURE_GREEN) { + reset_capability_throughput_feedback( + feedback, &selection->capability); + (void)snprintf(feedback->diagnostic.reason, + sizeof(feedback->diagnostic.reason), "pressure-decrease"); + } else if (selection->capability.backend + == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN + && actual_backend != LARDON3D_RESOURCE_BACKEND_ORB_VULKAN) { + /* Complete CPU fallback is valid scientific output, but cannot be + * a pure GPU observation. Abandon the trial and rebuild the full + * capability-owned baseline window on later pure sequences. */ + feedback->adaptive_cpu_limit = feedback->accepted_cpu_limit; + feedback->adaptive_batch_limit = feedback->accepted_batch_limit; + feedback->adaptive_inflight_limit = + feedback->accepted_inflight_limit; + feedback->trial_dimension = LARDON3D_CAPABILITY_TRIAL_NONE; + clear_baseline_rate_observations(feedback); + feedback->baseline_rate_milli = 0; + clear_trial_rate_observations(feedback); + (void)snprintf(feedback->diagnostic.reason, + sizeof(feedback->diagnostic.reason), "backend-fallback-hold"); + } else if (feedback->diagnostic.durable_items_per_second_milli == 0) { + if (selection->capability.sustained_gpu_batch_feedback) { + clear_baseline_rate_observations(feedback); + clear_trial_rate_observations(feedback); + } + (void)snprintf(feedback->diagnostic.reason, + sizeof(feedback->diagnostic.reason), "throughput-no-work"); + } else if (feedback->trial_dimension + != LARDON3D_CAPABILITY_TRIAL_NONE) { + uint64_t rate = + feedback->diagnostic.durable_items_per_second_milli; + bool exercised = feedback->trial_dimension + == LARDON3D_CAPABILITY_TRIAL_CPU + ? selection->decision.cpu_threads + == feedback->adaptive_cpu_limit + : feedback->trial_dimension + == LARDON3D_CAPABILITY_TRIAL_INFLIGHT + ? selection->inflight_limit + == feedback->adaptive_inflight_limit + : selection->decision.batch_size + == feedback->adaptive_batch_limit; + if (!exercised) { + /* A reduced contract is not part of a consecutive GPU batch + * trial. Generic CPU/inflight behavior remains unchanged. */ + if (selection->capability.sustained_gpu_batch_feedback + && feedback->trial_dimension + == LARDON3D_CAPABILITY_TRIAL_BATCH) { + clear_trial_rate_observations(feedback); + } + (void)snprintf(feedback->diagnostic.reason, + sizeof(feedback->diagnostic.reason), "%s", + trial_observation_reason( + &selection->capability, + feedback->trial_dimension)); + } else { + unsigned int required = required_throughput_observations( + &selection->capability, feedback->trial_dimension); + if (feedback->trial_observations >= required) { + clear_trial_rate_observations(feedback); + } + feedback->trial_rates_milli[ + feedback->trial_observations++] = rate; + if (feedback->trial_observations < required) { + (void)snprintf(feedback->diagnostic.reason, + sizeof(feedback->diagnostic.reason), "%s", + trial_observation_reason( + &selection->capability, + feedback->trial_dimension)); + } else { + /* Generic adaptation retains the conservative minimum of + * two. Only the explicit ORB GPU batch property uses an + * overflow-safe eight-observation arithmetic mean. */ + uint64_t trial_rate = + selection->capability.sustained_gpu_batch_feedback + && feedback->trial_dimension + == LARDON3D_CAPABILITY_TRIAL_BATCH + ? average_rate_observations( + feedback->trial_rates_milli, required) + : minimum_uint64( + feedback->trial_rates_milli[0], + feedback->trial_rates_milli[1]); + clear_trial_rate_observations(feedback); + Lardon3DCapabilityTrialDimension completed_dimension = + feedback->trial_dimension; + feedback->trial_dimension = + LARDON3D_CAPABILITY_TRIAL_NONE; + bool improved = throughput_materially_improved( + trial_rate, feedback->baseline_rate_milli); + if (improved) { + if (completed_dimension + == LARDON3D_CAPABILITY_TRIAL_CPU) { + feedback->accepted_cpu_limit = + feedback->adaptive_cpu_limit; + } else if (completed_dimension + == LARDON3D_CAPABILITY_TRIAL_INFLIGHT) { + feedback->accepted_inflight_limit = + feedback->adaptive_inflight_limit; + } else { + feedback->accepted_batch_limit = + feedback->adaptive_batch_limit; + } + feedback->baseline_rate_milli = trial_rate; + feedback->baseline_observations = required; + } else if (completed_dimension + == LARDON3D_CAPABILITY_TRIAL_CPU) { + feedback->adaptive_cpu_limit = + feedback->accepted_cpu_limit; + feedback->cpu_growth_stopped = true; + } else if (completed_dimension + == LARDON3D_CAPABILITY_TRIAL_INFLIGHT) { + feedback->adaptive_inflight_limit = + feedback->accepted_inflight_limit; + feedback->inflight_growth_stopped = true; + } else { + feedback->adaptive_batch_limit = + feedback->accepted_batch_limit; + feedback->batch_growth_stopped = true; + } + open_next_capability_trial_locked( + governor, feedback, &selection->capability); + (void)snprintf(feedback->diagnostic.reason, + sizeof(feedback->diagnostic.reason), "%s", + trial_result_reason( + &selection->capability, completed_dimension, + improved)); + } + } + } else { + uint64_t rate = + feedback->diagnostic.durable_items_per_second_milli; + if (feedback->baseline_rate_milli == 0) { + unsigned int required = required_throughput_observations( + &selection->capability, + LARDON3D_CAPABILITY_TRIAL_NONE); + bool exercised = + !selection->capability.sustained_gpu_batch_feedback + || selection->decision.batch_size + == feedback->accepted_batch_limit; + if (!exercised) { + clear_baseline_rate_observations(feedback); + } else { + if (feedback->baseline_observations >= required) { + clear_baseline_rate_observations(feedback); + } + feedback->baseline_rates_milli[ + feedback->baseline_observations++] = rate; + } + bool complete = exercised + && feedback->baseline_observations == required; + if (complete) { + feedback->baseline_rate_milli = + average_rate_observations( + feedback->baseline_rates_milli, required); + open_next_capability_trial_locked( + governor, feedback, &selection->capability); + } + (void)snprintf(feedback->diagnostic.reason, + sizeof(feedback->diagnostic.reason), "%s", + complete + ? trial_observation_reason( + &selection->capability, + feedback->trial_dimension) + : selection->capability.sustained_gpu_batch_feedback + ? "gpu-batch-baseline" : "hold-deadband"); + } else { + (void)snprintf(feedback->diagnostic.reason, + sizeof(feedback->diagnostic.reason), "hold-deadband"); + } + } + } + aggregate_sequence(feedback, wall_time_ns, items_completed, actual_backend, + backend_reason, metrics); + advance_diagnostic_serial_locked(governor, feedback); + ++governor->generation; + (void)pthread_cond_broadcast(&governor->cond); + (void)pthread_mutex_unlock(&governor->mutex); + return true; +} + +bool +lardon3d_resource_governor_internal_record_fallback_items( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + const Lardon3DResourceCapabilitySelection *selection, + Lardon3DResourceFallbackItemCause cause, + uint64_t item_count +) +{ + if (!governor || !selection || !task_kind || !task_kind[0] + || selection->capability.backend + != LARDON3D_RESOURCE_BACKEND_ORB_VULKAN + || cause < LARDON3D_RESOURCE_FALLBACK_ITEM_LOCAL_INELIGIBLE + || cause > LARDON3D_RESOURCE_FALLBACK_ITEM_OTHER) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + Lardon3DCapabilityFeedback *feedback = capability_feedback_locked( + governor, task_kind, task_kind_version, + selection->capability.backend, true); + if (!feedback) { + (void)pthread_mutex_unlock(&governor->mutex); + return false; + } + uint64_t *counter = NULL; + switch (cause) { + case LARDON3D_RESOURCE_FALLBACK_ITEM_LOCAL_INELIGIBLE: + counter = &feedback->aggregate.local_ineligible_fallback_items; + break; + case LARDON3D_RESOURCE_FALLBACK_ITEM_BACKEND_FAILURE: + counter = &feedback->aggregate.backend_failure_fallback_items; + break; + case LARDON3D_RESOURCE_FALLBACK_ITEM_OTHER: + counter = &feedback->aggregate.backend_other_fallback_items; + break; + } + /* CONTRACT: an item becomes evidence at its own durable publication, + * even if a later item aborts the sequence. This fixed counter update is + * deliberately isolated from diagnostics, serials, pressure, trials and + * throughput observations; it cannot mutate the frozen execution + * selection or teach the Governor from a failed sequence. */ + aggregate_add(counter, item_count, &feedback->aggregate.saturated); + (void)pthread_mutex_unlock(&governor->mutex); + return true; +} + +bool +lardon3d_resource_governor_internal_record_sequence( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + const Lardon3DResourceCapabilitySelection *selection, + uint64_t wall_time_ns, + size_t items_completed +) +{ + if (!selection) { + return false; + } + return lardon3d_resource_governor_internal_record_sequence_execution( + governor, + task_kind, + task_kind_version, + selection, + wall_time_ns, + items_completed, + selection->capability.backend, + "selected-backend-completed" + ); +} + +bool +lardon3d_resource_governor_internal_last_diagnostic( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + Lardon3DResourceSequenceDiagnostic *diagnostic +) +{ + if (!governor || !task_kind || !diagnostic) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + Lardon3DCapabilityFeedback *best = NULL; + for (size_t index = 0; index < LARDON3D_CAPABILITY_FEEDBACK_CAPACITY; + ++index) { + Lardon3DCapabilityFeedback *candidate = + &governor->capability_feedback[index]; + if (candidate->used && candidate->task_kind_version == task_kind_version + && strcmp(candidate->task_kind, task_kind) == 0 + && candidate->diagnostic.task_kind[0] + && (!best + || candidate->diagnostic_serial > best->diagnostic_serial + || (candidate->diagnostic_serial == best->diagnostic_serial + && candidate->diagnostic_update_order + > best->diagnostic_update_order))) { + best = candidate; + } + } + if (best) { + *diagnostic = best->diagnostic; + } + (void)pthread_mutex_unlock(&governor->mutex); + return best != NULL; +} + +bool +lardon3d_resource_governor_internal_diagnostic_since( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + uint64_t after_serial, + Lardon3DResourceSequenceDiagnostic *diagnostic +) +{ + return lardon3d_resource_governor_internal_last_diagnostic( + governor, task_kind, task_kind_version, diagnostic) + && diagnostic->serial > after_serial; +} + +static void +aggregate_merge_counter(uint64_t *target, uint64_t source, bool *saturated) +{ + aggregate_add(target, source, saturated); +} + +bool +lardon3d_resource_governor_internal_sequence_aggregate( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + Lardon3DResourceSequenceAggregate *aggregate +) +{ + if (!governor || !task_kind || !task_kind[0] || !aggregate) { + return false; + } + Lardon3DResourceSequenceAggregate result = {0}; + bool found = false; + (void)pthread_mutex_lock(&governor->mutex); + for (size_t index = 0; index < LARDON3D_CAPABILITY_FEEDBACK_CAPACITY; + ++index) { + const Lardon3DCapabilityFeedback *feedback = + &governor->capability_feedback[index]; + if (!feedback->used || feedback->task_kind_version != task_kind_version + || strcmp(feedback->task_kind, task_kind) != 0) { + continue; + } + found = true; + const Lardon3DResourceSequenceAggregate *source = &feedback->aggregate; + result.saturated = result.saturated || source->saturated; +#define MERGE_COUNTER(field) \ + aggregate_merge_counter(&result.field, source->field, &result.saturated) + MERGE_COUNTER(admission_count); + MERGE_COUNTER(sequence_count); + MERGE_COUNTER(durable_items); + MERGE_COUNTER(total_wall_time_ns); + MERGE_COUNTER(backend_fallback_sequences); + MERGE_COUNTER(backend_ineligible_fallback_sequences); + MERGE_COUNTER(backend_failure_fallback_sequences); + MERGE_COUNTER(backend_other_fallback_sequences); + MERGE_COUNTER(local_ineligible_fallback_items); + MERGE_COUNTER(backend_failure_fallback_items); + MERGE_COUNTER(backend_other_fallback_items); + MERGE_COUNTER(contract_change_count); + MERGE_COUNTER(vulkan_submits); + MERGE_COUNTER(vulkan_completions); + MERGE_COUNTER(vulkan_submit_cpu_ns); + MERGE_COUNTER(vulkan_fence_wait_ns); + MERGE_COUNTER(vulkan_readback_ns); + MERGE_COUNTER(vulkan_gpu_known_sequences); + MERGE_COUNTER(vulkan_gpu_ns); + MERGE_COUNTER(vulkan_starvation_ns); + MERGE_COUNTER(matcher_cpu_ns); + MERGE_COUNTER(publication_ns); +#undef MERGE_COUNTER + for (size_t backend = 0; + backend <= LARDON3D_RESOURCE_BACKEND_MIXED; ++backend) { + aggregate_merge_counter( + &result.selected_backend_admissions[backend], + source->selected_backend_admissions[backend], + &result.saturated); + aggregate_merge_counter( + &result.actual_backend_sequences[backend], + source->actual_backend_sequences[backend], + &result.saturated); + } + if (source->memory_available_known + && (!result.memory_available_known + || source->minimum_memory_available_bytes + < result.minimum_memory_available_bytes)) { + result.memory_available_known = true; + result.minimum_memory_available_bytes = + source->minimum_memory_available_bytes; + } + if (source->gpu_busy_known + && (!result.gpu_busy_known + || source->maximum_gpu_busy_basis_points + > result.maximum_gpu_busy_basis_points)) { + result.gpu_busy_known = true; + result.maximum_gpu_busy_basis_points = + source->maximum_gpu_busy_basis_points; + } + if (source->process_rss_known + && (!result.process_rss_known + || source->maximum_process_rss_bytes + > result.maximum_process_rss_bytes)) { + result.process_rss_known = true; + result.maximum_process_rss_bytes = + source->maximum_process_rss_bytes; + } + if (source->process_peak_rss_known + && (!result.process_peak_rss_known + || source->maximum_process_peak_rss_bytes + > result.maximum_process_peak_rss_bytes)) { + result.process_peak_rss_known = true; + result.maximum_process_peak_rss_bytes = + source->maximum_process_peak_rss_bytes; + } + } + (void)pthread_mutex_unlock(&governor->mutex); + if (found) *aggregate = result; + return found; +} + +bool +lardon3d_resource_governor_internal_format_diagnostic( + const Lardon3DResourceSequenceDiagnostic *diagnostic, + char *text, + size_t capacity +) +{ + if (!diagnostic || !text || capacity == 0) return false; + /* Explicit pull formatting is bounded and intentionally not wired to + * ncurses or an unbounded logger. Unknown telemetry is paired with its + * known bit so zero never masquerades as an observed idle system. */ + int written = snprintf(text, capacity, + "serial=%llu kind=%s/%u selected=%d actual=%d fallback=%u " + "pressure=%d cpu=%u gpu=%u batch=%zu inflight=%zu helpers=%u " + "memory=%llu gpu_memory=%llu io=%u wall_ns=%llu items=%zu " + "rate_milli=%llu mem_known=%u mem_available=%llu " + "mem_psi_some=%u/%u mem_psi_full=%u/%u io_psi_some=%u/%u " + "io_psi_full=%u/%u swap_delta=%u/%llu/%llu pool_util=%u/%u " + "gpu_busy=%u/%u rss=%u/%llu peak_rss=%u/%llu " + "vk_submit=%llu vk_complete=%llu vk_submit_cpu_ns=%llu " + "vk_fence_ns=%llu vk_readback_ns=%llu vk_gpu_known=%u vk_gpu_ns=%llu " + "vk_starvation_ns=%llu matcher_cpu_ns=%llu publication_ns=%llu " + "fallback_items=%llu/%llu/%llu fallback_items_saturated=%u " + "reason=%s backend_reason=%s", + (unsigned long long)diagnostic->serial, + diagnostic->task_kind, diagnostic->task_kind_version, + (int)diagnostic->backend, (int)diagnostic->actual_backend, + diagnostic->backend_fallback ? 1U : 0U, + (int)diagnostic->pressure, diagnostic->cpu_threads, + diagnostic->gpu_slots, diagnostic->batch_size, + diagnostic->inflight_limit, diagnostic->helper_limit, + (unsigned long long)diagnostic->memory_bytes, + (unsigned long long)diagnostic->gpu_memory_bytes, + diagnostic->io_slots, + (unsigned long long)diagnostic->previous_wall_time_ns, + diagnostic->items_completed, + (unsigned long long)diagnostic->durable_items_per_second_milli, + diagnostic->host.memory_available_known ? 1U : 0U, + (unsigned long long)diagnostic->host.memory_available_bytes, + diagnostic->host.memory_psi_some_known ? 1U : 0U, + diagnostic->host.memory_psi_some_basis_points, + diagnostic->host.memory_psi_full_known ? 1U : 0U, + diagnostic->host.memory_psi_full_basis_points, + diagnostic->host.io_psi_some_known ? 1U : 0U, + diagnostic->host.io_psi_some_basis_points, + diagnostic->host.io_psi_full_known ? 1U : 0U, + diagnostic->host.io_psi_full_basis_points, + diagnostic->host.swap_delta_known ? 1U : 0U, + (unsigned long long)diagnostic->host.swap_pages_in_delta, + (unsigned long long)diagnostic->host.swap_pages_out_delta, + diagnostic->host.compute_pool_utilization_known ? 1U : 0U, + diagnostic->host.compute_pool_utilization_basis_points, + diagnostic->host.gpu_busy_known ? 1U : 0U, + diagnostic->host.gpu_busy_basis_points, + diagnostic->host.process_rss_known ? 1U : 0U, + (unsigned long long)diagnostic->host.process_rss_bytes, + diagnostic->host.process_peak_rss_known ? 1U : 0U, + (unsigned long long)diagnostic->host.process_peak_rss_bytes, + (unsigned long long)diagnostic->execution.vulkan_submits, + (unsigned long long)diagnostic->execution.vulkan_completions, + (unsigned long long)diagnostic->execution.vulkan_submit_cpu_ns, + (unsigned long long)diagnostic->execution.vulkan_fence_wait_ns, + (unsigned long long)diagnostic->execution.vulkan_readback_ns, + diagnostic->execution.vulkan_gpu_time_known ? 1U : 0U, + (unsigned long long)diagnostic->execution.vulkan_gpu_ns, + (unsigned long long)diagnostic->execution.vulkan_starvation_ns, + (unsigned long long)diagnostic->execution.matcher_cpu_ns, + (unsigned long long)diagnostic->execution.publication_ns, + (unsigned long long) + diagnostic->execution.local_ineligible_fallback_items, + (unsigned long long) + diagnostic->execution.backend_failure_fallback_items, + (unsigned long long) + diagnostic->execution.backend_other_fallback_items, + diagnostic->execution.fallback_items_saturated ? 1U : 0U, + diagnostic->reason, diagnostic->backend_reason); + return written >= 0 && (size_t)written < capacity; +} + +bool +lardon3d_resource_governor_internal_set_backend_available( + Lardon3DResourceGovernor *governor, + Lardon3DResourceBackend backend, + bool available +) +{ + if (!governor || backend != LARDON3D_RESOURCE_BACKEND_ORB_VULKAN) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + governor->orb_vulkan_backend_available = available; + ++governor->generation; + (void)pthread_cond_broadcast(&governor->cond); + (void)pthread_mutex_unlock(&governor->mutex); + return true; +} + +bool +lardon3d_resource_governor_internal_capability_hardware_safe( + Lardon3DResourceGovernor *governor, + const Lardon3DTaskCapability *capability +) +{ + if (!governor || !capability) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + bool safe = capability_hardware_safe_locked(governor, capability); + (void)pthread_mutex_unlock(&governor->mutex); + return safe; +} + +bool +lardon3d_resource_governor_internal_cpu_policy( + Lardon3DResourceGovernor *governor, + Lardon3DResourceCpuPolicyDiagnostic *diagnostic +) +{ + if (!governor || !diagnostic) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + *diagnostic = governor->cpu_policy; + (void)pthread_mutex_unlock(&governor->mutex); + return true; +} + +bool +lardon3d_resource_governor_internal_configure_cpu_topology( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceCpuTopologyInput *input +) +{ + if (!governor || !input) { + return false; + } + uint64_t allowed[LARDON3D_RESOURCE_CPU_MASK_WORDS]; + unsigned int packages[LARDON3D_RESOURCE_CPU_MAX] = {0}; + unsigned int cores[LARDON3D_RESOURCE_CPU_MAX] = {0}; + if (input->affinity_available + && !cpu_topology_input_masks(input, allowed, packages, cores)) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + bool accepted = governor->cpu_reserved == 0; + if (accepted) { + Lardon3DResourceCpuPolicyDiagnostic policy; + build_cpu_policy(&governor->profile, &governor->policy, input, + &policy); + if (policy.compute_cpu_count == 0) { + accepted = false; + } else { + /* This is ephemeral host policy only: no durable Task estimate or + * public resource contract is rewritten by topology discovery. */ + governor->cpu_topology = *input; + governor->cpu_policy = policy; + ++governor->generation; + (void)pthread_cond_broadcast(&governor->cond); + } + } + (void)pthread_mutex_unlock(&governor->mutex); + return accepted; +} + +void +lardon3d_resource_governor_internal_force_worker_affinity_failure( + Lardon3DResourceGovernor *governor, + bool force_failure +) +{ + if (!governor) { + return; + } + (void)pthread_mutex_lock(&governor->mutex); + governor->internal_force_worker_affinity_failure = force_failure; + (void)pthread_mutex_unlock(&governor->mutex); +} + +#ifdef __linux__ +static bool +cpu_set_from_policy( + const Lardon3DResourceCpuPolicyDiagnostic *policy, + cpu_set_t *requested +) +{ + CPU_ZERO(requested); + for (unsigned int cpu = 0; cpu < LARDON3D_RESOURCE_CPU_MAX; ++cpu) { + if (!cpu_mask_test(policy->compute_mask, cpu)) continue; + if (cpu >= CPU_SETSIZE) return false; + CPU_SET((size_t)cpu, requested); + } + return policy->compute_cpu_count > 0; +} + +#endif + +bool +lardon3d_resource_governor_internal_apply_worker_affinity( + Lardon3DResourceGovernor *governor +) +{ + if (!governor) { + return false; + } + Lardon3DResourceCpuPolicyDiagnostic policy; + bool force_failure; + (void)pthread_mutex_lock(&governor->mutex); + policy = governor->cpu_policy; + force_failure = governor->internal_force_worker_affinity_failure; + (void)pthread_mutex_unlock(&governor->mutex); + + bool applied = true; + const char *reason = policy.reason; + if (policy.affinity_configured) { + applied = false; + reason = "worker-affinity-apply-failed"; +#ifdef __linux__ + cpu_set_t previous; + cpu_set_t requested; + cpu_set_t verified; + CPU_ZERO(&previous); + CPU_ZERO(&verified); + bool representable = cpu_set_from_policy(&policy, &requested); + bool previous_known = sched_getaffinity( + 0, sizeof(previous), &previous) == 0; + if (!force_failure && representable && previous_known + && sched_setaffinity(0, sizeof(requested), &requested) == 0) { + bool verified_known = sched_getaffinity( + 0, sizeof(verified), &verified) == 0; + applied = verified_known && CPU_EQUAL(&requested, &verified); + if (!applied) { + /* Verification failure may follow a successful mutation. + * Restore the worker's prior mask before any Task callback. */ + (void)sched_setaffinity(0, sizeof(previous), &previous); + } + } +#else + (void)force_failure; +#endif + if (applied) { + reason = policy.externally_constrained + ? "worker-affinity-active-external-mask" + : "worker-affinity-active-compute-mask"; + } + } + (void)pthread_mutex_lock(&governor->mutex); + governor->cpu_policy.affinity_attempted = true; + governor->cpu_policy.affinity_active = + policy.affinity_configured && applied; + (void)snprintf(governor->cpu_policy.reason, + sizeof(governor->cpu_policy.reason), "%s", reason); + (void)pthread_mutex_unlock(&governor->mutex); + return applied; +} + bool lardon3d_resource_governor_internal_set_next_reservation_id( Lardon3DResourceGovernor *governor, @@ -1259,6 +3955,19 @@ lardon3d_resource_governor_internal_set_next_reservation_id( return true; } +bool +lardon3d_resource_governor_internal_set_diagnostic_serial( + Lardon3DResourceGovernor *governor, + uint64_t diagnostic_serial +) +{ + if (!governor) return false; + (void)pthread_mutex_lock(&governor->mutex); + governor->capability_diagnostic_serial = diagnostic_serial; + (void)pthread_mutex_unlock(&governor->mutex); + return true; +} + bool lardon3d_resource_governor_internal_set_counters( Lardon3DResourceGovernor *governor, diff --git a/src/resource_governor_internal.h b/src/resource_governor_internal.h index b2277e6..73e7a23 100644 --- a/src/resource_governor_internal.h +++ b/src/resource_governor_internal.h @@ -2,10 +2,396 @@ #define LARDON3D_RESOURCE_GOVERNOR_INTERNAL_H #include +#include #include #include #include +#include + +#ifdef __cplusplus +extern "C" { +#endif + +enum { + LARDON3D_RESOURCE_CAPABILITY_MAX = 4, + LARDON3D_RESOURCE_DIAGNOSTIC_REASON_CAPACITY = 96, + LARDON3D_RESOURCE_CPU_MAX = 1024, + LARDON3D_RESOURCE_CPU_MASK_WORDS = LARDON3D_RESOURCE_CPU_MAX / 64, +}; + +typedef enum { + LARDON3D_RESOURCE_BACKEND_FIXED = 0, + LARDON3D_RESOURCE_BACKEND_CPU = 1, + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN = 2, + /* Diagnostics only: one admitted GPU sequence completed whole pairs on + * both backends after an operational Vulkan failure. Never a capability. */ + LARDON3D_RESOURCE_BACKEND_MIXED = 3, +} Lardon3DResourceBackend; + +typedef struct { + Lardon3DResourceEstimate estimate; + Lardon3DResourceBackend backend; + /* Maximum validated simultaneous requests. A zero minimum means the + * established fixed-capability shorthand minimum==maximum; adaptive + * capabilities state the lower bound explicitly. The canonical durable + * estimate remains untouched while per-inflight operational GPU bytes are + * reconstructed privately for each sequence. */ + size_t inflight_limit; + size_t minimum_inflight_limit; + uint64_t gpu_memory_bytes_per_inflight; + unsigned int helper_limit; + bool preferred; + bool cpu_reducible; + bool batch_adaptive; + /* Current ORB Vulkan batch trials use a longer observation window than + * generic CPU adaptation. This private operational property is + * reconstructed with the capability and is never durable/scientific. */ + bool sustained_gpu_batch_feedback; + bool inflight_adaptive; + bool requires_runtime_backend; +} Lardon3DTaskCapability; + +typedef struct { + size_t count; + Lardon3DTaskCapability capabilities[LARDON3D_RESOURCE_CAPABILITY_MAX]; +} Lardon3DTaskCapabilityEnvelope; + +typedef struct { + size_t capability_index; + Lardon3DTaskCapability capability; + /* Exact per-sequence estimate used only for reservation. The envelope's + * canonical maximums stay intact for next-sequence feedback. */ + Lardon3DResourceEstimate reservation_estimate; + Lardon3DResourceDecision decision; + /* Frozen operational value selected for this sequence. */ + size_t inflight_limit; + Lardon3DResourcePressure pressure; + char reason[LARDON3D_RESOURCE_DIAGNOSTIC_REASON_CAPACITY]; +} Lardon3DResourceCapabilitySelection; + +typedef struct { + bool memory_available_known; + uint64_t memory_available_bytes; + bool memory_psi_some_known; + uint32_t memory_psi_some_basis_points; + bool memory_psi_full_known; + uint32_t memory_psi_full_basis_points; + bool io_psi_some_known; + uint32_t io_psi_some_basis_points; + bool io_psi_full_known; + uint32_t io_psi_full_basis_points; + bool swap_delta_known; + uint64_t swap_pages_in_delta; + uint64_t swap_pages_out_delta; + bool compute_pool_utilization_known; + uint32_t compute_pool_utilization_basis_points; + bool gpu_busy_known; + uint32_t gpu_busy_basis_points; + bool process_rss_known; + uint64_t process_rss_bytes; + bool process_peak_rss_known; + uint64_t process_peak_rss_bytes; +} Lardon3DResourceHostTelemetry; + +/* Private raw-input seam used by production's bounded proc/sysfs readers and + * deterministic tests. Pointers are borrowed only for the call. A missing + * optional input produces an unknown metric, never a Task failure. */ +typedef struct { + const char *proc_stat; + const char *meminfo; + const char *memory_psi; + const char *io_psi; + const char *vmstat; + const char *process_status; + const char *gpu_busy_percent; +} Lardon3DResourceTelemetryRaw; + +typedef struct { + uint64_t vulkan_submits; + uint64_t vulkan_completions; + uint64_t vulkan_submit_cpu_ns; + uint64_t vulkan_fence_wait_ns; + uint64_t vulkan_readback_ns; + bool vulkan_gpu_time_known; + uint64_t vulkan_gpu_ns; + uint64_t vulkan_starvation_ns; + uint64_t matcher_cpu_ns; + uint64_t publication_ns; + /* A Vulkan-selected Matcher pair is classified only after its complete + * CPU fallback is durably published. Normal CPU-selected work is not a + * fallback. The sequence bound keeps these counters small; the explicit + * flag still makes injected/overflowed private telemetry fail closed. */ + bool fallback_items_saturated; + uint64_t local_ineligible_fallback_items; + uint64_t backend_failure_fallback_items; + uint64_t backend_other_fallback_items; +} Lardon3DResourceExecutionMetrics; + +/* Item-level fallback evidence is committed at the exact durable publication + * boundary, independently of end-of-sequence throughput feedback. The enum is + * private because these are operational benchmark classes, not scientific or + * persisted Matcher identities. */ +typedef enum { + LARDON3D_RESOURCE_FALLBACK_ITEM_LOCAL_INELIGIBLE = 1, + LARDON3D_RESOURCE_FALLBACK_ITEM_BACKEND_FAILURE, + LARDON3D_RESOURCE_FALLBACK_ITEM_OTHER, +} Lardon3DResourceFallbackItemCause; + +typedef struct { + uint64_t serial; + char task_kind[LARDON3D_TASK_KIND_CAPACITY]; + uint32_t task_kind_version; + /* `backend` is the immutable selected capability. `actual_backend` is + * execution feedback and may differ only through complete CPU fallback. */ + Lardon3DResourceBackend backend; + Lardon3DResourceBackend actual_backend; + bool backend_fallback; + uint64_t previous_wall_time_ns; + size_t items_completed; + uint64_t durable_items_per_second_milli; + size_t batch_size; + size_t inflight_limit; + unsigned int helper_limit; + uint64_t memory_bytes; + uint64_t gpu_memory_bytes; + unsigned int cpu_threads; + unsigned int gpu_slots; + unsigned int io_slots; + Lardon3DResourcePressure pressure; + Lardon3DResourceHostTelemetry host; + Lardon3DResourceExecutionMetrics execution; + char reason[LARDON3D_RESOURCE_DIAGNOSTIC_REASON_CAPACITY]; + char backend_reason[LARDON3D_RESOURCE_DIAGNOSTIC_REASON_CAPACITY]; +} Lardon3DResourceSequenceDiagnostic; + +/* Fixed-size Governor-lifetime evidence. This aggregate makes fast sequences + * countable even when a polling runner observes only the latest diagnostic + * change. Counters saturate, gauges retain extrema, and no per-sequence history + * or scientific state is persisted. */ +typedef struct { + bool saturated; + uint64_t admission_count; + uint64_t sequence_count; + uint64_t durable_items; + uint64_t total_wall_time_ns; + uint64_t backend_fallback_sequences; + /* Benchmark evidence must distinguish the scientifically valid complete + * CPU handling of a locally Vulkan-ineligible pair from an unhealthy + * backend. These bounded counters classify exact execution reasons; they + * remain operational Governor telemetry and are never persisted. */ + uint64_t backend_ineligible_fallback_sequences; + uint64_t backend_failure_fallback_sequences; + uint64_t backend_other_fallback_sequences; + /* Item counters, unlike the sequence classifiers above, are invariant to + * benchmark batch regrouping. They are summed with the aggregate's shared + * saturation flag and remain fixed-size, operational, and non-persistent. */ + uint64_t local_ineligible_fallback_items; + uint64_t backend_failure_fallback_items; + uint64_t backend_other_fallback_items; + uint64_t selected_backend_admissions[LARDON3D_RESOURCE_BACKEND_MIXED + 1]; + uint64_t actual_backend_sequences[LARDON3D_RESOURCE_BACKEND_MIXED + 1]; + uint64_t contract_change_count; + bool memory_available_known; + uint64_t minimum_memory_available_bytes; + bool gpu_busy_known; + uint32_t maximum_gpu_busy_basis_points; + bool process_rss_known; + uint64_t maximum_process_rss_bytes; + bool process_peak_rss_known; + uint64_t maximum_process_peak_rss_bytes; + uint64_t vulkan_submits; + uint64_t vulkan_completions; + uint64_t vulkan_submit_cpu_ns; + uint64_t vulkan_fence_wait_ns; + uint64_t vulkan_readback_ns; + uint64_t vulkan_gpu_known_sequences; + uint64_t vulkan_gpu_ns; + uint64_t vulkan_starvation_ns; + uint64_t matcher_cpu_ns; + uint64_t publication_ns; +} Lardon3DResourceSequenceAggregate; + +typedef struct { + unsigned int cpu_id; + unsigned int package_id; + unsigned int core_id; +} Lardon3DResourceCpuTopologyEntry; + +typedef struct { + bool affinity_available; + bool topology_available; + size_t allowed_cpu_count; + unsigned int allowed_cpu_ids[LARDON3D_RESOURCE_CPU_MAX]; + size_t topology_entry_count; + Lardon3DResourceCpuTopologyEntry + topology_entries[LARDON3D_RESOURCE_CPU_MAX]; +} Lardon3DResourceCpuTopologyInput; + +typedef struct { + bool affinity_configured; + bool affinity_attempted; + bool affinity_active; + bool runtime_thread_policy_active; + bool mesa_shader_cache_disabled; + bool externally_constrained; + unsigned int compute_cpu_count; + unsigned int reserved_cpu_count; + uint64_t allowed_mask[LARDON3D_RESOURCE_CPU_MASK_WORDS]; + uint64_t compute_mask[LARDON3D_RESOURCE_CPU_MASK_WORDS]; + uint64_t reserved_mask[LARDON3D_RESOURCE_CPU_MASK_WORDS]; + char reason[LARDON3D_RESOURCE_DIAGNOSTIC_REASON_CAPACITY]; + char runtime_thread_policy_reason[ + LARDON3D_RESOURCE_DIAGNOSTIC_REASON_CAPACITY]; +} Lardon3DResourceCpuPolicyDiagnostic; + +typedef enum { + LARDON3D_RESOURCE_DRIVER_POLICY_FAILED = 0, + LARDON3D_RESOURCE_DRIVER_POLICY_DEFAULTED, + LARDON3D_RESOURCE_DRIVER_POLICY_INHERITED_SAFE, + LARDON3D_RESOURCE_DRIVER_POLICY_REJECTED_UNSAFE, +} Lardon3DResourceDriverPolicyResult; + +bool lardon3d_resource_governor_internal_reserve_capability_available( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + const Lardon3DTaskCapabilityEnvelope *envelope, + Lardon3DResourceCapabilitySelection *selection, + Lardon3DResourceReservation **reservation +); +bool lardon3d_resource_governor_internal_reserve_capability( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceSnapshot *snapshot, + const char *task_kind, + uint32_t task_kind_version, + const Lardon3DTaskCapabilityEnvelope *envelope, + Lardon3DResourceCapabilitySelection *selection, + Lardon3DResourceReservation **reservation +); +bool lardon3d_resource_governor_internal_record_sequence( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + const Lardon3DResourceCapabilitySelection *selection, + uint64_t wall_time_ns, + size_t items_completed +); +bool lardon3d_resource_governor_internal_record_sequence_execution( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + const Lardon3DResourceCapabilitySelection *selection, + uint64_t wall_time_ns, + size_t items_completed, + Lardon3DResourceBackend actual_backend, + const char *backend_reason +); +bool lardon3d_resource_governor_internal_record_sequence_execution_metrics( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + const Lardon3DResourceCapabilitySelection *selection, + uint64_t wall_time_ns, + size_t items_completed, + Lardon3DResourceBackend actual_backend, + const char *backend_reason, + const Lardon3DResourceExecutionMetrics *metrics +); +/* Adds fixed-size ephemeral evidence only. This operation never creates a + * sequence observation, trains throughput feedback, or changes admission. + * `item_count` permits deterministic overflow testing; Task execution passes + * exactly one after each durable fallback publication. */ +bool lardon3d_resource_governor_internal_record_fallback_items( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + const Lardon3DResourceCapabilitySelection *selection, + Lardon3DResourceFallbackItemCause cause, + uint64_t item_count +); +bool lardon3d_resource_governor_internal_last_diagnostic( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + Lardon3DResourceSequenceDiagnostic *diagnostic +); +bool lardon3d_resource_governor_internal_diagnostic_since( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + uint64_t after_serial, + Lardon3DResourceSequenceDiagnostic *diagnostic +); +bool lardon3d_resource_governor_internal_sequence_aggregate( + Lardon3DResourceGovernor *governor, + const char *task_kind, + uint32_t task_kind_version, + Lardon3DResourceSequenceAggregate *aggregate +); +bool lardon3d_resource_governor_internal_format_diagnostic( + const Lardon3DResourceSequenceDiagnostic *diagnostic, + char *text, + size_t capacity +); +bool lardon3d_resource_governor_internal_sample_telemetry_raw( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceTelemetryRaw *raw, + Lardon3DResourceHostTelemetry *telemetry +); +/* Reads only the retained DRM card identity below root. No card scan or + * fallback is permitted; O_NOFOLLOW and the production strict parser apply. */ +bool lardon3d_resource_governor_internal_read_gpu_busy_at_root( + const char *root, + unsigned int card_index, + uint32_t *basis_points +); +bool lardon3d_resource_governor_internal_set_backend_available( + Lardon3DResourceGovernor *governor, + Lardon3DResourceBackend backend, + bool available +); +bool lardon3d_resource_governor_internal_capability_hardware_safe( + Lardon3DResourceGovernor *governor, + const Lardon3DTaskCapability *capability +); + +/* Governor owns the bounded topology snapshot and compute mask. Queue calls + * apply only from its sole heavy-compute worker; callers/main threads must not + * be constrained. Failure is observable and leaves Task scientific state + * untouched. */ +bool lardon3d_resource_governor_internal_cpu_policy( + Lardon3DResourceGovernor *governor, + Lardon3DResourceCpuPolicyDiagnostic *diagnostic +); +bool lardon3d_resource_governor_internal_apply_worker_affinity( + Lardon3DResourceGovernor *governor +); +/* Must run on the startup thread before any application pthread is created. + * An absent Mesa setting is defaulted to the safe value; an explicit safe + * value is retained, while false/malformed values are rejected rather than + * overwritten. The policy is harmless on non-Mesa drivers and is operational, + * never scientific identity or persistence. */ +Lardon3DResourceDriverPolicyResult +lardon3d_resource_governor_internal_configure_driver_policy(void); + +/* Deterministic private injection seam. It replaces only ephemeral topology + * policy and is rejected while CPU reservations are active. */ +bool lardon3d_resource_governor_internal_configure_cpu_topology( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceCpuTopologyInput *input +); +void lardon3d_resource_governor_internal_force_worker_affinity_failure( + Lardon3DResourceGovernor *governor, + bool force_failure +); +/* Private parser seam for the exact bounded sysfs reader. The file must + * contain one unsigned decimal token followed only by ASCII whitespace and + * EOF; signs, overflow, tails, and capacity truncation are rejected. */ +bool lardon3d_resource_governor_internal_read_topology_value_file( + const char *path, + unsigned int *value +); typedef struct { unsigned int pressure_streak; @@ -17,6 +403,10 @@ bool lardon3d_resource_governor_internal_set_next_reservation_id( Lardon3DResourceGovernor *governor, uint64_t next_reservation_id ); +bool lardon3d_resource_governor_internal_set_diagnostic_serial( + Lardon3DResourceGovernor *governor, + uint64_t diagnostic_serial +); bool lardon3d_resource_governor_internal_set_counters( Lardon3DResourceGovernor *governor, const Lardon3DResourceGovernorInternalCounters *counters @@ -34,4 +424,8 @@ void lardon3d_resource_governor_internal_force_capture_failure( bool force_failure ); +#ifdef __cplusplus +} +#endif + #endif diff --git a/src/resource_snapshot.c b/src/resource_snapshot.c index 296ff72..5cc90d4 100644 --- a/src/resource_snapshot.c +++ b/src/resource_snapshot.c @@ -49,6 +49,57 @@ read_file(const char *path, char *buffer, size_t capacity) return success; } +static bool +read_exact_decimal_u64(const char *path, uint64_t *value) +{ + if (!path || !value) return false; + int descriptor = open(path, O_RDONLY | O_CLOEXEC | O_NOFOLLOW); + if (descriptor < 0) return false; + char buffer[64]; + size_t total = 0; + bool success = true; + while (total + 1 < sizeof(buffer)) { + ssize_t count = read(descriptor, buffer + total, + sizeof(buffer) - total - 1); + if (count < 0 && errno == EINTR) continue; + if (count < 0) { + success = false; + break; + } + if (count == 0) break; + total += (size_t)count; + } + if (success && total + 1 == sizeof(buffer)) { + char extra; + ssize_t count; + do { + count = read(descriptor, &extra, 1); + } while (count < 0 && errno == EINTR); + success = count == 0; + } + if (close(descriptor) != 0) success = false; + if (!success || total == 0) return false; + buffer[total] = '\0'; + const unsigned char *cursor = (const unsigned char *)buffer; + uint64_t parsed = 0; + size_t digits = 0; + while (*cursor >= '0' && *cursor <= '9') { + uint64_t digit = (uint64_t)(*cursor - '0'); + if (parsed > (UINT64_MAX - digit) / 10U) return false; + parsed = parsed * 10U + digit; + ++cursor; + ++digits; + } + if (digits == 0) return false; + while (*cursor == ' ' || *cursor == '\t' || *cursor == '\n' + || *cursor == '\r' || *cursor == '\f' || *cursor == '\v') { + ++cursor; + } + if (*cursor) return false; + *value = parsed; + return true; +} + static bool meminfo_bytes(const char *buffer, const char *key, uint64_t *bytes) { @@ -163,18 +214,12 @@ lardon3d_resource_snapshot_capture_gpu_at_root( if (written < 0 || (size_t)written >= sizeof(path)) { return; } - char buffer[64]; - if (!read_file(path, buffer, sizeof(buffer))) { - return; - } - errno = 0; - char *end; - unsigned long long used = strtoull(buffer, &end, 10); - if (errno == 0 && end != buffer - && (uint64_t)used <= profile->gpu_memory_total_bytes) { + uint64_t used; + if (read_exact_decimal_u64(path, &used) + && used <= profile->gpu_memory_total_bytes) { snapshot->gpu_memory_available_known = true; snapshot->gpu_memory_available_bytes = - profile->gpu_memory_total_bytes - (uint64_t)used; + profile->gpu_memory_total_bytes - used; } } diff --git a/src/sift_task.c b/src/sift_task.c index 31d1dd9..5c5deb8 100644 --- a/src/sift_task.c +++ b/src/sift_task.c @@ -13,6 +13,9 @@ #include #include +#include "opencv_task_thread_control.h" +#include "task_internal.h" + typedef struct { char project_path[PATH_MAX]; Lardon3DProjectDb *database; @@ -92,8 +95,9 @@ static bool source_path(const SiftTaskContext *context, char path[PATH_MAX]) { memcmp(actual, asset.sha256, 32) == 0; } -static bool run(Lardon3DTask *task, void *userdata) { +static bool run_body(Lardon3DTask *task, void *userdata, size_t *durable_items) { SiftTaskContext *context = userdata; + *durable_items = 0; if (!lardon3d_task_checkpoint(task)) return false; Lardon3DProjectDbFeatureSet existing; Lardon3DProjectDbResult found = lardon3d_project_db_find_feature_set( @@ -111,8 +115,8 @@ static bool run(Lardon3DTask *task, void *userdata) { } if (found != LARDON3D_PROJECT_DB_NOT_FOUND) return lardon3d_task_fail(task, "Recherche Feature Store SIFT impossible."); - struct timespec begin; - (void)clock_gettime(CLOCK_MONOTONIC, &begin); + struct timespec begin = {0}; + bool timing_known = clock_gettime(CLOCK_MONOTONIC, &begin) == 0; char path[PATH_MAX]; if (!source_path(context, path)) return lardon3d_task_fail(task, "Asset image corrompu."); Lardon3DSiftExtractorParameters parameters = extract_parameters(context); @@ -147,13 +151,46 @@ static bool run(Lardon3DTask *task, void *userdata) { published != LARDON3D_FEATURE_STORE_ALREADY_PRESENT && published != LARDON3D_FEATURE_STORE_PUBLISHED_NOT_DURABLE) return lardon3d_task_fail(task, "Publication SIFT impossible."); - struct timespec end; - (void)clock_gettime(CLOCK_MONOTONIC, &end); - (void)lardon3d_resource_governor_record_batch( - context->governor, LARDON3D_RESOURCE_TASK_CPU, 1, elapsed_ns(begin, end), 0); + *durable_items = published == LARDON3D_FEATURE_STORE_OK ? 1 : 0; + struct timespec end = {0}; + timing_known = timing_known && clock_gettime(CLOCK_MONOTONIC, &end) == 0; + if (*durable_items > 0 && timing_known) { + (void)lardon3d_resource_governor_record_batch( + context->governor, LARDON3D_RESOURCE_TASK_CPU, *durable_items, + elapsed_ns(begin, end), 0); + } return lardon3d_task_set_progress(task, 100, "Feature Set SIFT publié."); } +static bool run(Lardon3DTask *task, void *userdata) { + struct timespec begin = {0}; + struct timespec end = {0}; + bool timing_known = clock_gettime(CLOCK_MONOTONIC, &begin) == 0; + Lardon3DOpenCvTaskThreadControl threads; + if (!lardon3d_opencv_task_threads_begin(task, 12, &threads)) { + return lardon3d_task_fail(task, "Contrat CPU OpenCV SIFT invalide."); + } + /* SIFT/RootSIFT consume the immutable admitted 1..12 OpenCV count. Queue's + * single Task owner makes the process-wide set/restore deterministic and + * race-free without changing extractor identity or output semantics. */ + size_t durable_items = 0; + bool result = run_body(task, userdata, &durable_items); + if (!lardon3d_opencv_task_threads_end(&threads)) { + (void)lardon3d_task_fail(task, "Restauration OpenCV SIFT impossible."); + return false; + } + timing_known = timing_known && clock_gettime(CLOCK_MONOTONIC, &end) == 0; + if (result && timing_known) { + /* SIFT and RootSIFT retain distinct histories. READY reuse, a durable + * ALREADY_PRESENT collision, and uncertain directory durability are + * successful no-work observations, so they cannot advance either kind's + * next-sequence CPU baseline or trial. */ + (void)lardon3d_task_internal_record_sequence( + task, elapsed_ns(begin, end), durable_items); + } + return result; +} + static void finished(const Lardon3DTask *task, void *userdata) { #ifdef LARDON3D_FEATURE_TASK_TESTING const char *skip = getenv("LARDON3D_TEST_SIFT_SKIP_FINISHED_CHECKPOINT"); @@ -252,9 +289,8 @@ Lardon3DTask *lardon3d_project_create_sift_extract_task( .memory_bytes_per_item = 1024ULL * 1024 * 1024, .minimum_batch_size = 1, .maximum_batch_size = 1, - /* Keep the immutable estimate independent of Matcher's - * temporary process-wide single-thread setting. Governor - * reduction makes this ceiling equal the startup pool. */ + /* Canonical durable maximum; the admitted OpenCV count may + * adapt within the validated 1..12 range. */ .desired_cpu_threads = 12, .desired_io_slots = 1, .task_class = LARDON3D_RESOURCE_TASK_CPU}; diff --git a/src/task.c b/src/task.c index a661d49..b90ee5a 100644 --- a/src/task.c +++ b/src/task.c @@ -6,6 +6,8 @@ #include +#include "task_internal.h" + struct Lardon3DTask { pthread_mutex_t mutex; pthread_cond_t condition; @@ -26,6 +28,11 @@ struct Lardon3DTask { char task_kind[LARDON3D_TASK_KIND_CAPACITY]; uint32_t task_kind_version; Lardon3DResourceEstimate estimate; + Lardon3DTaskCapabilityEnvelope capability_envelope; + Lardon3DResourceCapabilitySelection pending_selection; + Lardon3DResourceCapabilitySelection selected_capability; + Lardon3DResourceReservation *pending_reservation; + bool has_selected_capability; Lardon3DTaskExecutionContract contract; bool has_contract; bool pause_requested; @@ -34,6 +41,18 @@ struct Lardon3DTask { Lardon3DResourceGovernor *governor; Lardon3DResourceReservation *current_reservation; unsigned int sequence_count; + /* Matcher candidate-pair publication is strictly ascending within one + * Task. This private, non-persisted watermark makes item telemetry + * idempotent in-process without turning it into checkpoint state. */ + uint64_t fallback_item_high_water; + bool fallback_items_saturated; + uint64_t local_ineligible_fallback_items; + uint64_t backend_failure_fallback_items; + uint64_t backend_other_fallback_items; +#ifdef LARDON3D_TASK_TESTING + bool test_force_sequence_association_mismatch; + unsigned int test_association_failure_releases; +#endif }; static bool @@ -83,6 +102,29 @@ copy_text(char *destination, size_t capacity, const char *text) (void)snprintf(destination, capacity, "%s", text ? text : ""); } +static bool +kind_has_validated_cpu_range(const char *task_kind, uint32_t task_kind_version) +{ + if (!task_kind || task_kind_version != 1) { + return false; + } + return strcmp(task_kind, "features.extract") == 0 + || strcmp(task_kind, "features.extract.sift") == 0 + || strcmp(task_kind, "features.extract.rootsift") == 0 + || strcmp(task_kind, "visual_index.update") == 0 + || strcmp(task_kind, "candidate_pair.generate") == 0; +} + +static bool +kind_has_validated_batch_range(const char *task_kind, uint32_t version) +{ + /* Candidate generation already executes/publishes at every canonical + * 1..64 sequence size. This private bit changes operational pacing only; + * it does not add a scientific or durable identity dimension. */ + return task_kind && version == 1 + && strcmp(task_kind, "candidate_pair.generate") == 0; +} + static void finish_locked( Lardon3DTask *task, @@ -179,10 +221,355 @@ lardon3d_task_create_typed( task->task_kind_version = task_kind_version; } task->estimate = *estimate; + /* Every Task starts with one honest capability identical to its canonical + * durable estimate. It remains fixed unless kind/version proves a bounded + * operational range or installs alternatives before admission; the + * durable estimate is never mutated. */ + task->capability_envelope = (Lardon3DTaskCapabilityEnvelope) { + .count = 1, + .capabilities = {{ + .estimate = *estimate, + .backend = LARDON3D_RESOURCE_BACKEND_FIXED, + .inflight_limit = 1, + /* A durable estimate is a fixed operational envelope unless its + * registered kind has proved that its callback consumes a CPU + * range without changing scientific identity. This private bit + * is reconstructed from kind/version and is never persisted. */ + .cpu_reducible = typed + && kind_has_validated_cpu_range(task_kind, task_kind_version), + .batch_adaptive = typed + && kind_has_validated_batch_range(task_kind, task_kind_version), + }}, + }; copy_text(task->message, sizeof(task->message), "En attente."); return task; } +bool +lardon3d_task_internal_set_capability_envelope( + Lardon3DTask *task, + const Lardon3DTaskCapabilityEnvelope *envelope +) +{ + if (!task || !envelope || envelope->count == 0 + || envelope->count > LARDON3D_RESOURCE_CAPABILITY_MAX) { + return false; + } + for (size_t index = 0; index < envelope->count; ++index) { + const Lardon3DResourceEstimate *estimate = + &envelope->capabilities[index].estimate; + const Lardon3DTaskCapability *capability = + &envelope->capabilities[index]; + size_t minimum_inflight = capability->minimum_inflight_limit != 0 + ? capability->minimum_inflight_limit : capability->inflight_limit; + if (estimate->minimum_batch_size == 0 + || estimate->maximum_batch_size < estimate->minimum_batch_size + || estimate->desired_cpu_threads == 0 + || capability->inflight_limit == 0 + || minimum_inflight > capability->inflight_limit + || (capability->sustained_gpu_batch_feedback + && (!capability->batch_adaptive + || capability->backend + != LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)) + || (capability->inflight_adaptive + && (capability->minimum_inflight_limit == 0 + || capability->gpu_memory_bytes_per_inflight == 0)) + || (capability->gpu_memory_bytes_per_inflight != 0 + && capability->inflight_limit + > UINT64_MAX / capability->gpu_memory_bytes_per_inflight)) { + return false; + } + } + (void)pthread_mutex_lock(&task->mutex); + bool accepted = !task->executing && task->state == TASK_PENDING + && !task->current_reservation && !task->pending_reservation; + if (accepted) { + /* Task owns this bounded copy for its lifetime. Hardware/backend + * availability remains Governor-owned and is checked at admission. */ + task->capability_envelope = *envelope; + } + (void)pthread_mutex_unlock(&task->mutex); + return accepted; +} + +bool +lardon3d_task_internal_enable_known_capabilities(Lardon3DTask *task) +{ + if (!task) { + return false; + } + (void)pthread_mutex_lock(&task->mutex); + /* Recovery repeats the same private kind/version derivation used at fresh + * creation. Matcher and acquisition hooks may replace this one-capability + * default immediately afterward; the canonical durable estimate remains + * untouched in every case. */ + if (task->capability_envelope.count == 1 + && task->capability_envelope.capabilities[0].backend + == LARDON3D_RESOURCE_BACKEND_FIXED) { + task->capability_envelope.capabilities[0].cpu_reducible = + kind_has_validated_cpu_range( + task->task_kind, task->task_kind_version); + task->capability_envelope.capabilities[0].batch_adaptive = + kind_has_validated_batch_range( + task->task_kind, task->task_kind_version); + } + (void)pthread_mutex_unlock(&task->mutex); + return true; +} + +bool +lardon3d_task_internal_reserve_available( + Lardon3DTask *task, + Lardon3DResourceGovernor *governor, + Lardon3DResourceDecision *decision, + Lardon3DResourceReservation **reservation +) +{ + if (!task || !governor || !decision || !reservation) { + return false; + } + (void)pthread_mutex_lock(&task->mutex); + Lardon3DTaskCapabilityEnvelope envelope = task->capability_envelope; + char task_kind[LARDON3D_TASK_KIND_CAPACITY]; + copy_text(task_kind, sizeof(task_kind), task->task_kind[0] + ? task->task_kind : "task.untyped"); + uint32_t task_kind_version = task->task_kind_version; + (void)pthread_mutex_unlock(&task->mutex); + + Lardon3DResourceCapabilitySelection selection; + Lardon3DResourceReservation *created = NULL; + if (!lardon3d_resource_governor_internal_reserve_capability_available( + governor, + task_kind, + task_kind_version, + &envelope, + &selection, + &created + )) { + return false; + } + *decision = selection.decision; + *reservation = created; + if (created) { + (void)pthread_mutex_lock(&task->mutex); + if (task->pending_reservation) { + (void)pthread_mutex_unlock(&task->mutex); + (void)lardon3d_resource_governor_release(governor, created); + *reservation = NULL; + return false; + } + task->pending_selection = selection; + task->pending_reservation = created; + (void)pthread_mutex_unlock(&task->mutex); + } + return true; +} + +bool +lardon3d_task_internal_record_sequence_execution( + Lardon3DTask *task, + uint64_t wall_time_ns, + size_t items_completed, + Lardon3DResourceBackend actual_backend, + const char *backend_reason +) +{ + return lardon3d_task_internal_record_sequence_execution_metrics( + task, wall_time_ns, items_completed, actual_backend, backend_reason, + NULL); +} + +bool +lardon3d_task_internal_record_sequence_execution_metrics( + Lardon3DTask *task, + uint64_t wall_time_ns, + size_t items_completed, + Lardon3DResourceBackend actual_backend, + const char *backend_reason, + const Lardon3DResourceExecutionMetrics *metrics +) +{ + if (!task || wall_time_ns == 0 || !backend_reason) { + return false; + } + (void)pthread_mutex_lock(&task->mutex); + bool available = task->executing && task->governor + && task->has_selected_capability; + Lardon3DResourceGovernor *governor = task->governor; + Lardon3DResourceCapabilitySelection selection = task->selected_capability; + char task_kind[LARDON3D_TASK_KIND_CAPACITY]; + copy_text(task_kind, sizeof(task_kind), task->task_kind[0] + ? task->task_kind : "task.untyped"); + uint32_t task_kind_version = task->task_kind_version; + (void)pthread_mutex_unlock(&task->mutex); + return available + && lardon3d_resource_governor_internal_record_sequence_execution_metrics( + governor, + task_kind, + task_kind_version, + &selection, + wall_time_ns, + items_completed, + actual_backend, + backend_reason, + metrics + ); +} + +static void +increment_task_fallback_counter(uint64_t *counter, bool *saturated) +{ + if (*counter == UINT64_MAX) { + *saturated = true; + } else { + ++*counter; + } +} + +bool +lardon3d_task_internal_record_fallback_item( + Lardon3DTask *task, + uint64_t candidate_pair_id, + Lardon3DResourceFallbackItemCause cause +) +{ + if (!task || candidate_pair_id == 0 + || cause < LARDON3D_RESOURCE_FALLBACK_ITEM_LOCAL_INELIGIBLE + || cause > LARDON3D_RESOURCE_FALLBACK_ITEM_OTHER) { + return false; + } + (void)pthread_mutex_lock(&task->mutex); + bool available = task->executing && task->governor + && task->has_selected_capability + && task->selected_capability.capability.backend + == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN; + if (!available || candidate_pair_id <= task->fallback_item_high_water) { + bool duplicate = available + && candidate_pair_id <= task->fallback_item_high_water; + (void)pthread_mutex_unlock(&task->mutex); + return duplicate; + } + Lardon3DResourceGovernor *governor = task->governor; + Lardon3DResourceCapabilitySelection selection = task->selected_capability; + char task_kind[LARDON3D_TASK_KIND_CAPACITY]; + copy_text(task_kind, sizeof(task_kind), task->task_kind[0] + ? task->task_kind : "task.untyped"); + uint32_t task_kind_version = task->task_kind_version; + (void)pthread_mutex_unlock(&task->mutex); + + if (!lardon3d_resource_governor_internal_record_fallback_items( + governor, task_kind, task_kind_version, &selection, cause, 1)) { + return false; + } + + (void)pthread_mutex_lock(&task->mutex); + /* Queue owns the single callback, so no second recorder can pass the + * watermark concurrently. Retain the check to make the private operation + * idempotent even if a caller retries after a successful commit. */ + if (candidate_pair_id > task->fallback_item_high_water) { + task->fallback_item_high_water = candidate_pair_id; + switch (cause) { + case LARDON3D_RESOURCE_FALLBACK_ITEM_LOCAL_INELIGIBLE: + increment_task_fallback_counter( + &task->local_ineligible_fallback_items, + &task->fallback_items_saturated); + break; + case LARDON3D_RESOURCE_FALLBACK_ITEM_BACKEND_FAILURE: + increment_task_fallback_counter( + &task->backend_failure_fallback_items, + &task->fallback_items_saturated); + break; + case LARDON3D_RESOURCE_FALLBACK_ITEM_OTHER: + increment_task_fallback_counter( + &task->backend_other_fallback_items, + &task->fallback_items_saturated); + break; + } + } + (void)pthread_mutex_unlock(&task->mutex); + return true; +} + +bool +lardon3d_task_internal_record_sequence( + Lardon3DTask *task, + uint64_t wall_time_ns, + size_t items_completed +) +{ + if (!task) { + return false; + } + (void)pthread_mutex_lock(&task->mutex); + Lardon3DResourceBackend selected = task->has_selected_capability + ? task->selected_capability.capability.backend + : LARDON3D_RESOURCE_BACKEND_FIXED; + (void)pthread_mutex_unlock(&task->mutex); + return lardon3d_task_internal_record_sequence_execution( + task, + wall_time_ns, + items_completed, + selected, + "selected-backend-completed" + ); +} + +bool +lardon3d_task_internal_execution_selection( + const Lardon3DTask *task, + Lardon3DResourceCapabilitySelection *selection +) +{ + if (!task || !selection) return false; + Lardon3DTask *mutable_task = (Lardon3DTask *)task; + (void)pthread_mutex_lock(&mutable_task->mutex); + bool available = task->executing && task->has_selected_capability + && task->has_contract; + if (available) { + *selection = task->selected_capability; + } + (void)pthread_mutex_unlock(&mutable_task->mutex); + return available; +} + +#ifdef LARDON3D_TASK_TESTING +bool +lardon3d_task_internal_test_force_sequence_association_mismatch( + Lardon3DTask *task +) +{ + if (!task) return false; + (void)pthread_mutex_lock(&task->mutex); + bool accepted = task->executing && !is_terminal(task->state); + if (accepted) { + task->test_force_sequence_association_mismatch = true; + task->test_association_failure_releases = 0; + } + (void)pthread_mutex_unlock(&task->mutex); + return accepted; +} + +bool +lardon3d_task_internal_test_has_reservation_ownership(Lardon3DTask *task) +{ + if (!task) return false; + (void)pthread_mutex_lock(&task->mutex); + bool owned = task->current_reservation || task->pending_reservation; + (void)pthread_mutex_unlock(&task->mutex); + return owned; +} + +unsigned int +lardon3d_task_internal_test_association_failure_releases(Lardon3DTask *task) +{ + if (!task) return 0; + (void)pthread_mutex_lock(&task->mutex); + unsigned int count = task->test_association_failure_releases; + (void)pthread_mutex_unlock(&task->mutex); + return count; +} +#endif + void lardon3d_task_destroy(Lardon3DTask *task) { @@ -232,13 +619,46 @@ lardon3d_task_start( return false; } (void)pthread_mutex_lock(&task->mutex); - if (task->executing || is_terminal(task->state)) { + if (task->executing || is_terminal(task->state) + || (task->pending_reservation + && task->pending_reservation != reservation)) { (void)pthread_mutex_unlock(&task->mutex); return false; } task->executing = true; task->governor = governor; task->current_reservation = (Lardon3DResourceReservation *)reservation; + if (task->pending_reservation == reservation) { + task->selected_capability = task->pending_selection; + task->has_selected_capability = true; + task->pending_reservation = NULL; + } else { + /* Direct public callers remain compatible. Their reservation is an + * immutable fixed selection for this sequence. They did not ask the + * private Governor capability chooser, so an adaptive envelope must + * use its durable/minimum inflight rather than advertise an + * unreserved operational maximum. Only Queue-owned admission may + * install a higher adaptive depth. */ + const Lardon3DTaskCapability *direct_capability = + &task->capability_envelope.capabilities[0]; + size_t direct_inflight = direct_capability->inflight_adaptive + ? direct_capability->minimum_inflight_limit + : direct_capability->inflight_limit; + task->selected_capability = (Lardon3DResourceCapabilitySelection) { + .capability = *direct_capability, + .reservation_estimate = direct_capability->estimate, + .decision = { + .kind = LARDON3D_RESOURCE_START, + .batch_size = information.batch_size, + .cpu_threads = information.cpu_threads, + .gpu_slots = information.gpu_slots, + .io_slots = information.io_slots, + }, + .inflight_limit = direct_inflight, + .pressure = lardon3d_resource_governor_pressure(governor), + }; + task->has_selected_capability = true; + } task->contract = (Lardon3DTaskExecutionContract) { .batch_size = information.batch_size, .memory_bytes = information.memory_bytes, @@ -482,7 +902,6 @@ lardon3d_task_sequence_break( if (previous) { (void)lardon3d_resource_governor_release(governor, previous); } - for (;;) { /* Vérifier pause et annulation avant chaque tentative d'admission. */ (void)pthread_mutex_lock(&task->mutex); @@ -508,9 +927,9 @@ lardon3d_task_sequence_break( uint64_t generation = lardon3d_resource_governor_generation(governor); Lardon3DResourceDecision decision; Lardon3DResourceReservation *next = NULL; - bool admitted = lardon3d_resource_governor_reserve_available( + bool admitted = lardon3d_task_internal_reserve_available( + task, governor, - &task->estimate, &decision, &next ); @@ -561,7 +980,44 @@ lardon3d_task_sequence_break( return false; } (void)pthread_mutex_lock(&task->mutex); +#ifdef LARDON3D_TASK_TESTING + if (task->test_force_sequence_association_mismatch) { + task->pending_reservation = NULL; + task->test_force_sequence_association_mismatch = false; + } +#endif task->current_reservation = next; + if (task->pending_reservation != next) { + /* Association validation remains strict. Before releasing the + * rejected reservation, remove every Task ownership marker so + * task_start's common epilogue cannot release it a second time + * or expose a capability without its reservation. */ + task->current_reservation = NULL; + task->pending_reservation = NULL; + task->pending_selection = (Lardon3DResourceCapabilitySelection) {0}; + task->selected_capability = (Lardon3DResourceCapabilitySelection) {0}; + task->has_selected_capability = false; + task->contract = (Lardon3DTaskExecutionContract) {0}; + task->has_contract = false; + finish_locked( + task, + TASK_FAILED, + "Sélection de capacité incohérente." + ); + (void)pthread_mutex_unlock(&task->mutex); + bool released = lardon3d_resource_governor_release(governor, next); +#ifdef LARDON3D_TASK_TESTING + (void)pthread_mutex_lock(&task->mutex); + if (released) ++task->test_association_failure_releases; + (void)pthread_mutex_unlock(&task->mutex); +#else + (void)released; +#endif + return false; + } + task->selected_capability = task->pending_selection; + task->has_selected_capability = true; + task->pending_reservation = NULL; task->contract = (Lardon3DTaskExecutionContract) { .batch_size = information.batch_size, .memory_bytes = information.memory_bytes, diff --git a/src/task_internal.h b/src/task_internal.h new file mode 100644 index 0000000..0fb184c --- /dev/null +++ b/src/task_internal.h @@ -0,0 +1,98 @@ +#ifndef LARDON3D_TASK_INTERNAL_H +#define LARDON3D_TASK_INTERNAL_H + +#include +#include +#include + +#include + +#include "resource_governor_internal.h" + +#ifdef __cplusplus +extern "C" { +#endif + +/* The envelope is operation-owned policy reconstructed from Task kind/version + * and runtime support. Task copies it; it is neither durable state nor part of + * the public Task ABI. The selected capability is immutable until the Task + * crosses its next sequence boundary. */ +bool lardon3d_task_internal_set_capability_envelope( + Lardon3DTask *task, + const Lardon3DTaskCapabilityEnvelope *envelope +); + +/* Confirms the universal fixed default after generic reconstruction. Optional + * kind-owned private hooks may then replace it using runtime/business context; + * this avoids guessing scientific eligibility from a resource estimate. */ +bool lardon3d_task_internal_enable_known_capabilities(Lardon3DTask *task); + +/* Queue and sequence_break are the only admission owners. A successful call + * records the one selection associated with reservation so task_start cannot + * install a contract from a different capability. */ +bool lardon3d_task_internal_reserve_available( + Lardon3DTask *task, + Lardon3DResourceGovernor *governor, + Lardon3DResourceDecision *decision, + Lardon3DResourceReservation **reservation +); + +bool lardon3d_task_internal_record_sequence( + Lardon3DTask *task, + uint64_t wall_time_ns, + size_t items_completed +); +bool lardon3d_task_internal_record_sequence_execution( + Lardon3DTask *task, + uint64_t wall_time_ns, + size_t items_completed, + Lardon3DResourceBackend actual_backend, + const char *backend_reason +); +bool lardon3d_task_internal_record_sequence_execution_metrics( + Lardon3DTask *task, + uint64_t wall_time_ns, + size_t items_completed, + Lardon3DResourceBackend actual_backend, + const char *backend_reason, + const Lardon3DResourceExecutionMetrics *metrics +); + +/* Matcher calls this only after the exact candidate pair is durable. Task + * owns a bounded current-run high-water mark so an in-process retry cannot + * count the same ordered pair twice; restart intentionally reconstructs no + * operational telemetry. The operation does not create throughput feedback. */ +bool lardon3d_task_internal_record_fallback_item( + Lardon3DTask *task, + uint64_t candidate_pair_id, + Lardon3DResourceFallbackItemCause cause +); + +/* Copies the immutable operational selection installed for the executing + * sequence. Task retains ownership; callbacks use the copy only to honor + * private dimensions (currently Matcher inflight) absent from the stable + * public execution-contract ABI. Observation never changes admission. */ +bool lardon3d_task_internal_execution_selection( + const Lardon3DTask *task, + Lardon3DResourceCapabilitySelection *selection +); + +#ifdef LARDON3D_TASK_TESTING +/* Deterministically exercises the post-reserve association failure that is + * otherwise unreachable without corrupting private Task state. */ +bool lardon3d_task_internal_test_force_sequence_association_mismatch( + Lardon3DTask *task +); +bool lardon3d_task_internal_test_has_reservation_ownership( + Lardon3DTask *task +); +unsigned int lardon3d_task_internal_test_association_failure_releases( + Lardon3DTask *task +); +#endif + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/src/task_kind_registry.c b/src/task_kind_registry.c index 6702983..39f23cc 100644 --- a/src/task_kind_registry.c +++ b/src/task_kind_registry.c @@ -4,6 +4,22 @@ #include +#include "task_internal.h" + +/* Production Matcher provides this private post-reconstruction hook. Minimal + * Registry-only targets intentionally omit it; weak absence preserves the + * universal fixed-envelope default without adding a public descriptor field. */ +#if defined(__GNUC__) || defined(__clang__) +extern bool lardon3d_matcher_task_internal_configure_restored( + Lardon3DTask *task, + void *userdata +) __attribute__((weak)); +extern bool lardon3d_acquisition_campaign_task_internal_configure_restored( + Lardon3DTask *task, + void *userdata +) __attribute__((weak)); +#endif + enum { CANDIDATE_LEGACY_FIXED_BYTES = 128 * 1024, CANDIDATE_CURRENT_FIXED_BYTES = 256 * 1024, @@ -37,6 +53,8 @@ normalize_known_legacy_estimate(const char *kind, { Lardon3DResourceEstimate current = {0}; Lardon3DResourceEstimate historical = {0}; + Lardon3DResourceEstimate oldest = {0}; + bool has_oldest = false; if (strcmp(kind, "candidate_pair.generate") == 0) { current = (Lardon3DResourceEstimate) { .memory_fixed_bytes = CANDIDATE_CURRENT_FIXED_BYTES, @@ -56,16 +74,33 @@ normalize_known_legacy_estimate(const char *kind, .gpu_memory_fixed_bytes = gpu ? MATCHER_GPU_FIXED_BYTES : 0, .memory_bytes_per_item = MATCHER_CURRENT_PER_ITEM_BYTES, .minimum_batch_size = 1, - .maximum_batch_size = 8, - .desired_cpu_threads = gpu ? 1U : 8U, + .maximum_batch_size = 12, + .desired_cpu_threads = gpu ? 1U : 12U, .desired_gpu_slots = gpu ? 1U : 0U, .desired_io_slots = 1, .task_class = LARDON3D_RESOURCE_TASK_CPU, }; historical = current; - historical.memory_fixed_bytes = MATCHER_LEGACY_FIXED_BYTES; - historical.memory_bytes_per_item = 0; - historical.desired_cpu_threads = 12; + /* CPU8/GPU0 and CPU1/GPU1 are the immediately preceding durable + * operational forms. They are normalized only in memory: thread + * count is not Matcher scientific identity and no estimate-only + * checkpoint is published during recovery. */ + historical.maximum_batch_size = 8; + historical.desired_cpu_threads = gpu ? 1U : 8U; + oldest = current; + oldest.memory_fixed_bytes = MATCHER_LEGACY_FIXED_BYTES; + oldest.memory_bytes_per_item = 0; + oldest.maximum_batch_size = 8; + oldest.desired_cpu_threads = 12; + has_oldest = true; + Lardon3DResourceEstimate automatic = current; + automatic.task_class = LARDON3D_RESOURCE_TASK_MIXED; + if (!gpu && estimate_equals(durable, &automatic)) { + /* MIXED is the truthful durable signature for new normal AUTO: + * execution may consume either its CPU or Vulkan capability. */ + *effective = *durable; + return true; + } } else if (strcmp(kind, "features.extract.sift") == 0 || strcmp(kind, "features.extract.rootsift") == 0) { current = (Lardon3DResourceEstimate) { @@ -87,7 +122,8 @@ normalize_known_legacy_estimate(const char *kind, *effective = *durable; return true; } - if (!estimate_equals(durable, &historical)) { + if (!estimate_equals(durable, &historical) + && (!has_oldest || !estimate_equals(durable, &oldest))) { return false; } /* These exact signatures are historical operational policy, not scientific @@ -214,6 +250,28 @@ lardon3d_task_kind_registry_restore( } return LARDON3D_TASK_KIND_RESTORE_FAILED; } + bool capabilities_configured = + lardon3d_task_internal_enable_known_capabilities(*task); +#if defined(__GNUC__) || defined(__clang__) + if (capabilities_configured && strcmp(kind, "matcher.run") == 0 + && lardon3d_matcher_task_internal_configure_restored) { + capabilities_configured = + lardon3d_matcher_task_internal_configure_restored( + *task, binding.userdata); + } + if (capabilities_configured + && strcmp(kind, "acquisition_campaign.run") == 0 + && lardon3d_acquisition_campaign_task_internal_configure_restored) { + capabilities_configured = + lardon3d_acquisition_campaign_task_internal_configure_restored( + *task, binding.userdata); + } +#endif + if (!capabilities_configured) { + lardon3d_task_destroy(*task); + *task = NULL; + return LARDON3D_TASK_KIND_RESTORE_FAILED; + } if (binding.finished_callback && !lardon3d_task_set_finished_callback(*task, binding.finished_callback, binding.finished_userdata)) { diff --git a/src/task_queue.c b/src/task_queue.c index 3934e67..c2a5c72 100644 --- a/src/task_queue.c +++ b/src/task_queue.c @@ -6,6 +6,8 @@ #include +#include "task_internal.h" + typedef struct TaskNode { Lardon3DTask *task; struct TaskNode *next_all; @@ -26,6 +28,7 @@ struct Lardon3DTaskQueue { pthread_cond_t not_full; pthread_t worker; bool worker_started; + bool worker_ready; bool stopping; Lardon3DResourceGovernor *governor; uint64_t next_id; @@ -94,16 +97,14 @@ select_admissible( node = next; continue; } - Lardon3DResourceEstimate estimate; Lardon3DResourceDecision decision; Lardon3DResourceReservation *candidate = NULL; - bool evaluated = lardon3d_task_resource_estimate(node->task, &estimate) - && lardon3d_resource_governor_reserve_available( - queue->governor, - &estimate, - &decision, - &candidate - ); + bool evaluated = lardon3d_task_internal_reserve_available( + node->task, + queue->governor, + &decision, + &candidate + ); if (!evaluated) { (void)lardon3d_task_reject( node->task, @@ -160,6 +161,15 @@ static void * queue_worker(void *context) { Lardon3DTaskQueue *queue = context; + /* Only this thread owns heavy Task callbacks. Applying the Governor mask + * here keeps the caller/main/TUI thread unconstrained; children created by + * OpenCV, Matcher, or the Vulkan driver inherit the bounded worker mask. */ + (void)lardon3d_resource_governor_internal_apply_worker_affinity( + queue->governor); + (void)pthread_mutex_lock(&queue->mutex); + queue->worker_ready = true; + (void)pthread_cond_broadcast(&queue->not_empty); + (void)pthread_mutex_unlock(&queue->mutex); /* Single worker only; execution is serialized by design. Queue preserves * pending FIFO order with adaptive dispatch/backpressure; Governor decides * admission. @@ -188,6 +198,12 @@ queue_worker(void *context) queue->active = selected; (void)pthread_mutex_unlock(&queue->mutex); + /* Policy may change between Tasks. Reapply and verify only on this + * worker; failure is recorded by the Governor and execution continues + * under the conservative compute-count admission without corrupting + * Queue ownership or durable scientific state. */ + (void)lardon3d_resource_governor_internal_apply_worker_affinity( + queue->governor); if (!lardon3d_task_start(selected, queue->governor, reservation)) { (void)lardon3d_task_reject( selected, @@ -261,6 +277,11 @@ lardon3d_task_queue_create(Lardon3DResourceGovernor *governor, size_t capacity) return NULL; } queue->worker_started = true; + (void)pthread_mutex_lock(&queue->mutex); + while (!queue->worker_ready) { + (void)pthread_cond_wait(&queue->not_empty, &queue->mutex); + } + (void)pthread_mutex_unlock(&queue->mutex); return queue; } diff --git a/src/visual_index_task.c b/src/visual_index_task.c index d9c34af..aa22ba7 100644 --- a/src/visual_index_task.c +++ b/src/visual_index_task.c @@ -12,6 +12,7 @@ #include #include "visual_index_internal.h" +#include "task_internal.h" enum { VISUAL_INDEX_TASK_CPU_THREADS = 12 }; @@ -72,16 +73,16 @@ static bool run(Lardon3DTask *task, void *userdata) { contract.cpu_threads == 0 || contract.cpu_threads > VISUAL_INDEX_TASK_CPU_THREADS) { return lardon3d_task_fail(task, "Contrat Visual Index invalide."); } - struct timespec begin; - struct timespec end; - clock_gettime(CLOCK_MONOTONIC, &begin); + struct timespec begin = {0}; + struct timespec end = {0}; + bool timing_known = clock_gettime(CLOCK_MONOTONIC, &begin) == 0; uint64_t last = context->parameters.after_feature_set_id; size_t indexed = 0; Lardon3DVisualIndexResult result = lardon3d_visual_index_update_once_parallel( context->project_path, context->database, context->parameters.visual_index_id, lardon3d_task_id(task), context->parameters.after_feature_set_id, contract.batch_size, contract.cpu_threads, &last, &indexed); - clock_gettime(CLOCK_MONOTONIC, &end); + timing_known = timing_known && clock_gettime(CLOCK_MONOTONIC, &end) == 0; if (result == LARDON3D_VISUAL_INDEX_NO_CHANGE) { return lardon3d_task_set_progress(task, 100, "Visual Index à jour."); } @@ -90,8 +91,21 @@ static bool run(Lardon3DTask *task, void *userdata) { return lardon3d_task_fail(task, "Mise à jour Visual Index impossible."); } context->parameters.after_feature_set_id = last; - lardon3d_resource_governor_record_batch(context->governor, LARDON3D_RESOURCE_TASK_CPU, - indexed, elapsed_ns(begin, end), 0); + size_t durable_indexed = result == LARDON3D_VISUAL_INDEX_OK ? indexed : 0; + uint64_t duration_ns = timing_known ? elapsed_ns(begin, end) : 0; + if (durable_indexed > 0 && duration_ns > 0) { + lardon3d_resource_governor_record_batch( + context->governor, LARDON3D_RESOURCE_TASK_CPU, durable_indexed, + duration_ns, 0); + } + /* Visual Index already consumes the immutable admitted CPU count. A + * segment whose directory publication is not durable remains visible for + * restart semantics but is zero operational work, so it cannot train the + * next sequence's 1/2/4/8/12 CPU trial. */ + if (duration_ns > 0) { + (void)lardon3d_task_internal_record_sequence( + task, duration_ns, durable_indexed); + } if (!lardon3d_task_set_progress(task, 99, "Segment Visual Index publié.") || lardon3d_project_checkpoint_visual_index_update_task(&state, task, &context->parameters) != diff --git a/tests/benchmark_orb_vulkan.cpp b/tests/benchmark_orb_vulkan.cpp index 754f8c6..d8f6fc1 100644 --- a/tests/benchmark_orb_vulkan.cpp +++ b/tests/benchmark_orb_vulkan.cpp @@ -11,6 +11,8 @@ #include #include +#include "vulkan_process_startup.h" + namespace { constexpr uint32_t kDescriptorBytes = 32; @@ -124,6 +126,11 @@ static bool benchmark_sustained(Lardon3DOrbVulkanBackend *backend) { } // namespace int main(int argc, char **argv) { + if (!lardon3d_vulkan_evidence_process_startup()) { + std::fprintf(stderr, + "MESA_SHADER_CACHE_DISABLE must be true for safe CPU affinity\n"); + return 1; + } Lardon3DOrbVulkanBackend *backend = lardon3d_orb_vulkan_backend_create(); if (!backend) { return 1; diff --git a/tests/benchmark_sift_vulkan.cpp b/tests/benchmark_sift_vulkan.cpp index 476e61c..c807bd0 100644 --- a/tests/benchmark_sift_vulkan.cpp +++ b/tests/benchmark_sift_vulkan.cpp @@ -10,6 +10,8 @@ #include #include +#include "vulkan_process_startup.h" + static uint32_t random_u32(uint32_t *state) { uint32_t value = *state; value ^= value << 13; @@ -165,6 +167,11 @@ static bool benchmark(uint32_t count_a, uint32_t count_b, bool rootsift) { } int main() { + if (!lardon3d_vulkan_evidence_process_startup()) { + std::fprintf(stderr, + "MESA_SHADER_CACHE_DISABLE must be true for safe CPU affinity\n"); + return EXIT_FAILURE; + } cv::setNumThreads(12); std::printf("kind,count_a,count_b,cpu_ms,cold_ms,after_orb_ms,warm_ms,gpu_ms," "finalization_ms,gain_with_finalization,final_exact\n"); diff --git a/tests/pre_sfm_real_execution.cpp b/tests/pre_sfm_real_execution.cpp index bb44dbb..363e663 100644 --- a/tests/pre_sfm_real_execution.cpp +++ b/tests/pre_sfm_real_execution.cpp @@ -10,6 +10,7 @@ #include #include #include +#include #include #include #include @@ -22,7 +23,9 @@ extern "C" { #include #include #include +#include #include +#include #include #include #include @@ -31,11 +34,17 @@ extern "C" { #include #include #include + +#include "../src/matcher_task_benchmark_internal.h" +#include "../src/orb_vulkan_backend_internal.h" +#include "../src/resource_governor_internal.h" } namespace { enum class Mode { kA6000, kS21 }; +enum class MatcherMode { kAuto, kCpu, kVulkan }; +enum class MatcherPipeline { kRolling, kSynchronous }; enum class RestartBoundary { kNone, kRepresentations, kFeatures, kGeometry }; struct Options { @@ -45,6 +54,17 @@ struct Options { bool resume_pre_gv_existing{}; bool resume_candidate_existing{}; unsigned int cpu_budget{}; + unsigned int gpu_budget{}; + bool has_gpu_budget{}; + MatcherMode matcher_mode{MatcherMode::kAuto}; + bool has_matcher_mode{}; + MatcherPipeline matcher_pipeline{MatcherPipeline::kRolling}; + bool has_matcher_pipeline{}; + unsigned int matcher_inflight_override{}; + bool has_matcher_inflight_override{}; + unsigned int matcher_batch_override{}; + bool has_matcher_batch_override{}; + bool stop_after_matcher{}; std::filesystem::path project_dir; std::vector roots; size_t limit{LARDON3D_VISUAL_INDEX_CANDIDATE_MAX}; @@ -61,6 +81,20 @@ struct Runtime { std::string project_path; std::string database_path; unsigned int cpu_budget{}; + unsigned int gpu_budget{}; + bool has_gpu_budget{}; + MatcherMode matcher_mode{MatcherMode::kAuto}; + MatcherPipeline matcher_pipeline{MatcherPipeline::kRolling}; + unsigned int matcher_inflight_override{}; + unsigned int matcher_batch_override{}; + bool matcher_needed{}; + bool stop_after_matcher{}; + bool matcher_evidence_active{}; + uint64_t matcher_diagnostic_serial{}; + uint64_t matcher_diagnostic_samples{}; + std::chrono::steady_clock::time_point matcher_wall_begin{}; + bool matcher_backend_before_known{}; + Lardon3DOrbVulkanTelemetry matcher_backend_before{}; }; struct Evidence { @@ -73,18 +107,30 @@ struct Evidence { size_t tracks{}; }; +struct MatchAudit { + char digest_hex[65]{}; + size_t match_result_count{}; + size_t match_asset_count{}; + size_t duplicate_candidate_pair_mappings{}; + bool candidate_mapping_contiguous{}; + bool matcher_cursor_complete{}; +}; + void usage(const char *program) { std::fprintf( stderr, "Usage: %s --mode a6000|s21 --project-dir ABSOLUTE_EMPTY_DIR " "--root ABSOLUTE_DIR [--root ABSOLUTE_DIR ...] [--limit 1..4096] " "[--restart-boundary representations|features|geometry]\n" - " %s --resume-geometry-existing --project-dir " - "ABSOLUTE_EXISTING_DIR\n" + " %s --resume-geometry-existing --project-dir ABSOLUTE_EXISTING_DIR\n" " %s --resume-pre-gv-existing --project-dir " - "ABSOLUTE_EXISTING_DIR\n" + "ABSOLUTE_EXISTING_DIR [--cpu-budget 1..12] [--gpu-budget 0..1] " + "[--matcher-mode auto|cpu|vulkan] " + "[--matcher-pipeline rolling|synchronous] [--matcher-inflight 1|2] " + "[--matcher-batch 2|4|8|12] " + "[--stop-after-matcher]\n" " %s --resume-candidate-existing --project-dir " - "ABSOLUTE_EXISTING_DIR [--cpu-budget 1..12]\n", + "ABSOLUTE_EXISTING_DIR [--cpu-budget 1..12] [--gpu-budget 0..1]\n", program, program, program, @@ -109,6 +155,32 @@ bool parse_cpu_budget(const char *text, unsigned int &value) { return true; } +bool parse_gpu_budget(const char *text, unsigned int &value) { + char *end = nullptr; + errno = 0; + const unsigned long parsed = std::strtoul(text, &end, 10); + if (errno != 0 || end == text || *end != '\0' || parsed > 1) return false; + value = static_cast(parsed); + return true; +} + +bool parse_matcher_inflight(const char *text, unsigned int &value) { + if (!text || (std::strcmp(text, "1") != 0 && std::strcmp(text, "2") != 0)) + return false; + value = text[0] == '1' ? 1U : 2U; + return true; +} + +bool parse_matcher_batch(const char *text, unsigned int &value) { + if (!text) return false; + if (std::strcmp(text, "2") == 0) value = 2; + else if (std::strcmp(text, "4") == 0) value = 4; + else if (std::strcmp(text, "8") == 0) value = 8; + else if (std::strcmp(text, "12") == 0) value = 12; + else return false; + return true; +} + bool parse_options(int argc, char **argv, Options &options) { for (int index = 1; index < argc; ++index) { const std::string argument(argv[index]); @@ -130,7 +202,9 @@ bool parse_options(int argc, char **argv, Options &options) { } if ((argument == "--mode" || argument == "--project-dir" || argument == "--root" || argument == "--limit" || argument == "--restart-boundary" || - argument == "--cpu-budget") && + argument == "--cpu-budget" || argument == "--gpu-budget" || + argument == "--matcher-mode" || argument == "--matcher-pipeline" || + argument == "--matcher-inflight" || argument == "--matcher-batch") && index + 1 >= argc) return false; if (argument == "--mode") { @@ -153,6 +227,33 @@ bool parse_options(int argc, char **argv, Options &options) { else return false; } else if (argument == "--cpu-budget") { if (!parse_cpu_budget(argv[++index], options.cpu_budget)) return false; + } else if (argument == "--gpu-budget") { + if (!parse_gpu_budget(argv[++index], options.gpu_budget)) return false; + options.has_gpu_budget = true; + } else if (argument == "--matcher-mode") { + const std::string value(argv[++index]); + if (value == "auto") options.matcher_mode = MatcherMode::kAuto; + else if (value == "cpu") options.matcher_mode = MatcherMode::kCpu; + else if (value == "vulkan") options.matcher_mode = MatcherMode::kVulkan; + else return false; + options.has_matcher_mode = true; + } else if (argument == "--matcher-pipeline") { + const std::string value(argv[++index]); + if (value == "rolling") options.matcher_pipeline = MatcherPipeline::kRolling; + else if (value == "synchronous") + options.matcher_pipeline = MatcherPipeline::kSynchronous; + else return false; + options.has_matcher_pipeline = true; + } else if (argument == "--matcher-inflight") { + if (!parse_matcher_inflight( + argv[++index], options.matcher_inflight_override)) return false; + options.has_matcher_inflight_override = true; + } else if (argument == "--matcher-batch") { + if (!parse_matcher_batch(argv[++index], options.matcher_batch_override)) + return false; + options.has_matcher_batch_override = true; + } else if (argument == "--stop-after-matcher") { + options.stop_after_matcher = true; } else { return false; } @@ -160,16 +261,91 @@ bool parse_options(int argc, char **argv, Options &options) { const unsigned int resume_mode_count = options.resume_geometry_existing + options.resume_pre_gv_existing + options.resume_candidate_existing; - if (resume_mode_count != 0) - return !options.has_mode && !options.project_dir.empty() && options.roots.empty() && - options.restart == RestartBoundary::kNone && - resume_mode_count == 1 && - (options.resume_candidate_existing || options.cpu_budget == 0); - if (options.cpu_budget != 0) return false; + if (resume_mode_count != 0) { + if (options.has_mode || options.project_dir.empty() || !options.roots.empty() || + options.restart != RestartBoundary::kNone || resume_mode_count != 1) + return false; + if (options.resume_pre_gv_existing) { + return true; + } + if (options.resume_candidate_existing) + return !options.has_matcher_mode && !options.has_matcher_pipeline && + !options.has_matcher_inflight_override && + !options.has_matcher_batch_override && + !options.stop_after_matcher; + return options.cpu_budget == 0 && !options.has_gpu_budget && + !options.has_matcher_mode && !options.has_matcher_pipeline && + !options.has_matcher_inflight_override && + !options.has_matcher_batch_override && + !options.stop_after_matcher; + } + if (options.cpu_budget != 0 || options.has_gpu_budget || + options.has_matcher_mode || options.has_matcher_pipeline || + options.has_matcher_inflight_override || + options.has_matcher_batch_override || + options.stop_after_matcher) return false; return options.has_mode && !options.project_dir.empty() && !options.roots.empty() && options.roots.size() <= LARDON3D_ACQUISITION_CAMPAIGN_MAX_ROOTS; } +class ScopedEnvironmentValue { + public: + explicit ScopedEnvironmentValue(const char *name) : name_(name) { + const char *value = std::getenv(name_); + existed_ = value != nullptr; + if (value) value_ = value; + } + + ScopedEnvironmentValue(const ScopedEnvironmentValue &) = delete; + ScopedEnvironmentValue &operator=(const ScopedEnvironmentValue &) = delete; + + bool replace(const char *value) { + changed_ = true; + return value ? setenv(name_, value, 1) == 0 : unsetenv(name_) == 0; + } + + ~ScopedEnvironmentValue() { + if (!changed_) return; + /* Process-owned benchmark controls must not leak into a later in-process + * invocation. POSIX setenv/unsetenv are used only before runtime threads; + * destruction is the all-exit restoration boundary and never throws. */ + if (existed_) (void)setenv(name_, value_.c_str(), 1); + else (void)unsetenv(name_); + } + + private: + const char *name_; + bool existed_{}; + bool changed_{}; + std::string value_; +}; + +class ScopedMatcherBenchmarkEnvironment { + public: + ScopedMatcherBenchmarkEnvironment() + : pipeline_(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV), + inflight_(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV), + batch_(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV) {} + + bool configure(const Options &options) { + const char *pipeline = options.matcher_pipeline == + MatcherPipeline::kSynchronous ? "1" : nullptr; + const char *inflight = !options.has_matcher_inflight_override + ? nullptr : options.matcher_inflight_override == 1 ? "1" : "2"; + const char *batch = !options.has_matcher_batch_override + ? nullptr : options.matcher_batch_override == 2 ? "2" + : options.matcher_batch_override == 4 ? "4" + : options.matcher_batch_override == 8 ? "8" : "12"; + return pipeline_.replace(pipeline) && inflight_.replace(inflight) && + batch_.replace(batch); + } + + private: + ScopedEnvironmentValue pipeline_; + ScopedEnvironmentValue inflight_; + ScopedEnvironmentValue batch_; +}; + bool prepare_existing_project(const std::filesystem::path &input, Runtime &runtime) { if (!input.is_absolute() || input.lexically_normal() != input) return false; std::error_code error; @@ -320,10 +496,13 @@ void stop_runtime(Runtime &runtime) { if (runtime.state.task_queue) lardon3d_task_queue_destroy(runtime.state.task_queue); if (runtime.state.resource_governor) lardon3d_resource_governor_destroy(runtime.state.resource_governor); + if (runtime.state.orb_vulkan_backend) + lardon3d_orb_vulkan_backend_destroy(runtime.state.orb_vulkan_backend); if (runtime.state.project_db) lardon3d_project_db_close(runtime.state.project_db); runtime.state.task_queue = nullptr; runtime.state.resource_governor = nullptr; runtime.state.project_db = nullptr; + runtime.state.orb_vulkan_backend = nullptr; } bool start_runtime(Runtime &runtime) { @@ -356,6 +535,14 @@ bool start_runtime(Runtime &runtime) { policy.system_cpu_reserve = runtime.state.hardware_profile.logical_cpu_count - runtime.cpu_budget; } + if (runtime.has_gpu_budget) { + if (runtime.gpu_budget != 0 && !runtime.state.hardware_profile.gpu_available) { + std::fprintf(stderr, "requested GPU budget requires a detected GPU\n"); + stop_runtime(runtime); + return false; + } + policy.gpu_slot_capacity = runtime.gpu_budget; + } unsigned int feature_threads = runtime.state.hardware_profile.logical_cpu_count - policy.system_cpu_reserve; if (feature_threads > 12) feature_threads = 12; @@ -371,6 +558,12 @@ bool start_runtime(Runtime &runtime) { } runtime.state.resource_governor = lardon3d_resource_governor_create(&runtime.state.hardware_profile, &policy); + /* AUTO and explicit Vulkan receive only an uninitialized backend object. + * Metadata inspection here cannot start driver threads: first initialization + * remains on Queue's affinity-constrained worker. Portable AUTO receives the + * null stub result and exposes only CPU without a GPU side effect. */ + if (runtime.matcher_needed && runtime.matcher_mode != MatcherMode::kCpu) + runtime.state.orb_vulkan_backend = lardon3d_orb_vulkan_backend_create(); runtime.state.task_queue = runtime.state.resource_governor ? lardon3d_task_queue_create(runtime.state.resource_governor, 2) : nullptr; @@ -388,8 +581,470 @@ bool start_runtime(Runtime &runtime) { return true; } +const char *backend_name(Lardon3DResourceBackend backend) { + switch (backend) { + case LARDON3D_RESOURCE_BACKEND_FIXED: return "fixed"; + case LARDON3D_RESOURCE_BACKEND_CPU: return "cpu"; + case LARDON3D_RESOURCE_BACKEND_ORB_VULKAN: return "orb-vulkan"; + case LARDON3D_RESOURCE_BACKEND_MIXED: return "mixed"; + } + return "invalid"; +} + +const char *pressure_name(Lardon3DResourcePressure pressure) { + switch (pressure) { + case LARDON3D_RESOURCE_PRESSURE_GREEN: return "green"; + case LARDON3D_RESOURCE_PRESSURE_YELLOW: return "yellow"; + case LARDON3D_RESOURCE_PRESSURE_RED: return "red"; + } + return "invalid"; +} + +void print_json_string(const char *text) { + std::putchar('"'); + for (const unsigned char *cursor = + reinterpret_cast(text ? text : ""); + *cursor; ++cursor) { + switch (*cursor) { + case '"': std::fputs("\\\"", stdout); break; + case '\\': std::fputs("\\\\", stdout); break; + case '\b': std::fputs("\\b", stdout); break; + case '\f': std::fputs("\\f", stdout); break; + case '\n': std::fputs("\\n", stdout); break; + case '\r': std::fputs("\\r", stdout); break; + case '\t': std::fputs("\\t", stdout); break; + default: + if (*cursor < 0x20) + std::printf("\\u%04x", static_cast(*cursor)); + else + std::putchar(*cursor); + } + } + std::putchar('"'); +} + +void print_known_u64(bool known, uint64_t value) { + if (known) std::printf("%llu", static_cast(value)); + else std::fputs("null", stdout); +} + +void print_known_u32(bool known, uint32_t value) { + if (known) std::printf("%u", value); + else std::fputs("null", stdout); +} + +void emit_matcher_diagnostic_change(Runtime &runtime) { + if (!runtime.matcher_evidence_active || !runtime.state.resource_governor) return; + Lardon3DResourceSequenceDiagnostic diagnostic{}; + if (!lardon3d_resource_governor_internal_diagnostic_since( + runtime.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, runtime.matcher_diagnostic_serial, + &diagnostic)) + return; + runtime.matcher_diagnostic_serial = diagnostic.serial; + ++runtime.matcher_diagnostic_samples; + std::printf( + "{\"record\":\"matcher_diagnostic_sample\",\"sampling\":" + "\"latest-change-coalescing\",\"serial\":%llu,\"selected_backend\":\"%s\"," + "\"actual_backend\":\"%s\",\"fallback\":%s,\"pressure\":\"%s\"," + "\"cpu\":%u,\"gpu\":%u,\"batch\":%zu,\"inflight\":%zu," + "\"helpers\":%u,\"io\":%u,\"host_memory_bytes\":%llu," + "\"gpu_memory_bytes\":%llu,\"uma\":%s,\"wall_ns\":%llu," + "\"items\":%zu,\"durable_rate_milli\":%llu,\"mem_available_bytes\":" + , static_cast(diagnostic.serial), + backend_name(diagnostic.backend), backend_name(diagnostic.actual_backend), + diagnostic.backend_fallback ? "true" : "false", + pressure_name(diagnostic.pressure), diagnostic.cpu_threads, + diagnostic.gpu_slots, diagnostic.batch_size, diagnostic.inflight_limit, + diagnostic.helper_limit, diagnostic.io_slots, + static_cast(diagnostic.memory_bytes), + static_cast(diagnostic.gpu_memory_bytes), + runtime.state.hardware_profile.gpu_uses_shared_memory ? "true" : "false", + static_cast(diagnostic.previous_wall_time_ns), + diagnostic.items_completed, + static_cast(diagnostic.durable_items_per_second_milli)); + print_known_u64(diagnostic.host.memory_available_known, + diagnostic.host.memory_available_bytes); + std::fputs(",\"memory_psi_some_basis_points\":", stdout); + print_known_u32(diagnostic.host.memory_psi_some_known, + diagnostic.host.memory_psi_some_basis_points); + std::fputs(",\"memory_psi_full_basis_points\":", stdout); + print_known_u32(diagnostic.host.memory_psi_full_known, + diagnostic.host.memory_psi_full_basis_points); + std::fputs(",\"io_psi_some_basis_points\":", stdout); + print_known_u32(diagnostic.host.io_psi_some_known, + diagnostic.host.io_psi_some_basis_points); + std::fputs(",\"io_psi_full_basis_points\":", stdout); + print_known_u32(diagnostic.host.io_psi_full_known, + diagnostic.host.io_psi_full_basis_points); + std::printf( + ",\"swap_delta_known\":%s,\"swap_pages_in_delta\":%llu," + "\"swap_pages_out_delta\":%llu,\"compute_pool_utilization_basis_points\":", + diagnostic.host.swap_delta_known ? "true" : "false", + static_cast(diagnostic.host.swap_pages_in_delta), + static_cast(diagnostic.host.swap_pages_out_delta)); + print_known_u32(diagnostic.host.compute_pool_utilization_known, + diagnostic.host.compute_pool_utilization_basis_points); + std::fputs(",\"gpu_busy_basis_points\":", stdout); + print_known_u32(diagnostic.host.gpu_busy_known, + diagnostic.host.gpu_busy_basis_points); + std::fputs(",\"process_rss_bytes\":", stdout); + print_known_u64(diagnostic.host.process_rss_known, + diagnostic.host.process_rss_bytes); + std::fputs(",\"process_peak_rss_bytes\":", stdout); + print_known_u64(diagnostic.host.process_peak_rss_known, + diagnostic.host.process_peak_rss_bytes); + std::printf( + ",\"vulkan_submits\":%llu,\"vulkan_completions\":%llu," + "\"vulkan_submit_cpu_ns\":%llu,\"vulkan_fence_wait_ns\":%llu," + "\"vulkan_readback_ns\":%llu,\"vulkan_gpu_time_known\":%s," + "\"vulkan_gpu_ns\":%llu,\"vulkan_starvation_ns\":%llu," + "\"matcher_cpu_ns\":%llu,\"publication_ns\":%llu," + "\"local_ineligible_fallback_items\":%llu," + "\"backend_failure_fallback_items\":%llu," + "\"backend_other_fallback_items\":%llu," + "\"fallback_items_saturated\":%s,\"reason\":", + static_cast(diagnostic.execution.vulkan_submits), + static_cast(diagnostic.execution.vulkan_completions), + static_cast(diagnostic.execution.vulkan_submit_cpu_ns), + static_cast(diagnostic.execution.vulkan_fence_wait_ns), + static_cast(diagnostic.execution.vulkan_readback_ns), + diagnostic.execution.vulkan_gpu_time_known ? "true" : "false", + static_cast(diagnostic.execution.vulkan_gpu_ns), + static_cast(diagnostic.execution.vulkan_starvation_ns), + static_cast(diagnostic.execution.matcher_cpu_ns), + static_cast(diagnostic.execution.publication_ns), + static_cast( + diagnostic.execution.local_ineligible_fallback_items), + static_cast( + diagnostic.execution.backend_failure_fallback_items), + static_cast( + diagnostic.execution.backend_other_fallback_items), + diagnostic.execution.fallback_items_saturated ? "true" : "false"); + print_json_string(diagnostic.reason); + std::fputs(",\"backend_reason\":", stdout); + print_json_string(diagnostic.backend_reason); + std::fputs("}\n", stdout); +} + +void begin_matcher_evidence(Runtime &runtime) { + runtime.matcher_evidence_active = true; + runtime.matcher_diagnostic_serial = 0; + runtime.matcher_diagnostic_samples = 0; + runtime.matcher_wall_begin = std::chrono::steady_clock::now(); + runtime.matcher_backend_before_known = runtime.state.orb_vulkan_backend && + lardon3d_orb_vulkan_internal_telemetry( + runtime.state.orb_vulkan_backend, &runtime.matcher_backend_before); +} + +uint64_t counter_delta(uint64_t before, uint64_t after) { + return after >= before ? after - before : 0; +} + +struct MatcherExperimentValidation { + bool applicable{}; + bool valid{}; + uint64_t local_ineligible_fallback_items{}; + const char *reason{"not-forced"}; +}; + +bool counter_partition(uint64_t first, uint64_t second, uint64_t third, + uint64_t fourth, uint64_t total) { + if (first > total) return false; + total -= first; + if (second > total) return false; + total -= second; + if (third > total) return false; + total -= third; + return fourth == total; +} + +MatcherExperimentValidation validate_forced_matcher_experiment( + const Runtime &runtime, bool aggregate_known, + const Lardon3DResourceSequenceAggregate &aggregate, bool last_known, + const Lardon3DResourceSequenceDiagnostic &last, + bool backend_delta_known, uint64_t backend_failures, + uint64_t backend_discards, bool backend_slot_pending) { + MatcherExperimentValidation result{}; + result.applicable = runtime.matcher_inflight_override != 0; + if (!result.applicable) return result; + const uint64_t depth = runtime.matcher_inflight_override; + const uint64_t batch = runtime.matcher_batch_override != 0 + ? runtime.matcher_batch_override : 2; + const uint64_t payload = depth * LARDON3D_ORB_VULKAN_PER_SLOT_BYTES; + result.local_ineligible_fallback_items = + aggregate.local_ineligible_fallback_items; +#define INVALID_EXPERIMENT(why) \ + do { \ + result.reason = (why); \ + return result; \ + } while (false) + if (!aggregate_known) INVALID_EXPERIMENT("aggregate-unavailable"); + if (aggregate.saturated) INVALID_EXPERIMENT("aggregate-saturated"); + if (aggregate.admission_count == 0 || aggregate.sequence_count == 0) + INVALID_EXPERIMENT("no-completed-vulkan-sequence"); + if (aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_FIXED] != 0 || + aggregate.selected_backend_admissions[LARDON3D_RESOURCE_BACKEND_CPU] != 0 || + aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_MIXED] != 0 || + aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN] != aggregate.admission_count) + INVALID_EXPERIMENT("selected-contract-not-exclusively-vulkan"); + if (aggregate.admission_count != aggregate.sequence_count) + INVALID_EXPERIMENT("admission-sequence-count-mismatch"); + if (aggregate.contract_change_count != 0) + INVALID_EXPERIMENT("forced-contract-changed"); + if (!last_known || last.backend != LARDON3D_RESOURCE_BACKEND_ORB_VULKAN || + last.cpu_threads != 1 || last.gpu_slots != 1 || + last.batch_size != batch || + last.inflight_limit != depth || last.helper_limit != 0 || + last.io_slots != 1 || + last.memory_bytes != batch * UINT64_C(10) * 1024 * 1024 || + last.gpu_memory_bytes != payload) + INVALID_EXPERIMENT("forced-contract-mismatch"); + if (!backend_delta_known) INVALID_EXPERIMENT("backend-telemetry-unavailable"); + if (backend_failures != 0 || + aggregate.backend_failure_fallback_sequences != 0 || + aggregate.backend_failure_fallback_items != 0) + INVALID_EXPERIMENT("backend-failure"); + if (backend_discards != 0) INVALID_EXPERIMENT("backend-discard"); + if (backend_slot_pending) INVALID_EXPERIMENT("backend-slot-pending"); + if (aggregate.backend_other_fallback_sequences != 0 || + aggregate.backend_other_fallback_items != 0) + INVALID_EXPERIMENT("unclassified-backend-fallback"); + if (aggregate.local_ineligible_fallback_items > aggregate.durable_items || + aggregate.backend_ineligible_fallback_sequences > + aggregate.local_ineligible_fallback_items || + ((aggregate.local_ineligible_fallback_items == 0) != + (aggregate.backend_ineligible_fallback_sequences == 0))) + INVALID_EXPERIMENT("fallback-item-classification-mismatch"); + if (!counter_partition( + aggregate.backend_ineligible_fallback_sequences, + aggregate.backend_failure_fallback_sequences, + aggregate.backend_other_fallback_sequences, 0, + aggregate.backend_fallback_sequences)) + INVALID_EXPERIMENT("fallback-classification-mismatch"); + if (!counter_partition( + aggregate.actual_backend_sequences[LARDON3D_RESOURCE_BACKEND_FIXED], + aggregate.actual_backend_sequences[LARDON3D_RESOURCE_BACKEND_CPU], + aggregate.actual_backend_sequences[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN], + aggregate.actual_backend_sequences[LARDON3D_RESOURCE_BACKEND_MIXED], + aggregate.sequence_count)) + INVALID_EXPERIMENT("actual-backend-count-mismatch"); + if (aggregate.actual_backend_sequences[LARDON3D_RESOURCE_BACKEND_FIXED] != 0 || + !counter_partition( + aggregate.actual_backend_sequences[LARDON3D_RESOURCE_BACKEND_CPU], + aggregate.actual_backend_sequences[LARDON3D_RESOURCE_BACKEND_MIXED], + 0, 0, aggregate.backend_ineligible_fallback_sequences) || + !counter_partition( + aggregate.actual_backend_sequences[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN], + aggregate.backend_ineligible_fallback_sequences, 0, 0, + aggregate.sequence_count)) + INVALID_EXPERIMENT("nonlocal-cpu-fallback"); + if (aggregate.vulkan_submits != aggregate.vulkan_completions) + INVALID_EXPERIMENT("vulkan-submit-completion-mismatch"); + result.valid = true; + result.reason = "valid-forced-vulkan-cohort"; +#undef INVALID_EXPERIMENT + return result; +} + +void print_cpu_mask(const uint64_t mask[LARDON3D_RESOURCE_CPU_MASK_WORDS]) { + std::putchar('['); + bool first = true; + for (unsigned int cpu = 0; cpu < LARDON3D_RESOURCE_CPU_MAX; ++cpu) { + if ((mask[cpu / 64] & (UINT64_C(1) << (cpu % 64))) == 0) continue; + std::printf("%s%u", first ? "" : ",", cpu); + first = false; + } + std::putchar(']'); +} + +bool end_matcher_evidence(Runtime &runtime) { + if (!runtime.matcher_evidence_active) + return runtime.matcher_inflight_override == 0; + emit_matcher_diagnostic_change(runtime); + const uint64_t wall_ns = static_cast( + std::chrono::duration_cast( + std::chrono::steady_clock::now() - runtime.matcher_wall_begin).count()); + Lardon3DResourceSequenceAggregate aggregate{}; + const bool aggregate_known = + lardon3d_resource_governor_internal_sequence_aggregate( + runtime.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &aggregate); + Lardon3DResourceSequenceDiagnostic last{}; + const bool last_known = lardon3d_resource_governor_internal_last_diagnostic( + runtime.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &last); + Lardon3DResourceCpuPolicyDiagnostic cpu_policy{}; + const bool cpu_policy_known = lardon3d_resource_governor_internal_cpu_policy( + runtime.state.resource_governor, &cpu_policy); + Lardon3DOrbVulkanTelemetry backend_after{}; + const bool backend_after_known = runtime.state.orb_vulkan_backend && + lardon3d_orb_vulkan_internal_telemetry(runtime.state.orb_vulkan_backend, + &backend_after); + const bool backend_delta_known = runtime.matcher_backend_before_known && + backend_after_known; + const uint64_t backend_failures = backend_delta_known + ? counter_delta(runtime.matcher_backend_before.failures, + backend_after.failures) : 0; + const uint64_t backend_discards = backend_delta_known + ? counter_delta(runtime.matcher_backend_before.discards, + backend_after.discards) : 0; + const bool backend_slot_pending = backend_after_known && + backend_after.slot_pending; + const MatcherExperimentValidation experiment = + validate_forced_matcher_experiment( + runtime, aggregate_known, aggregate, last_known, last, + backend_delta_known, backend_failures, backend_discards, + backend_slot_pending); + uint64_t durable_rate_milli = 0; + if (aggregate_known && aggregate.durable_items > 0 && wall_ns > 0 && + aggregate.durable_items <= UINT64_MAX / UINT64_C(1000000000000)) + durable_rate_milli = aggregate.durable_items * UINT64_C(1000000000000) / wall_ns; + std::printf( + "{\"record\":\"matcher_evidence_aggregate\",\"aggregate_scope\":" + "\"governor-recorded-sequences\",\"diagnostic_sampling\":" + "\"latest-change-coalescing\",\"diagnostic_samples\":%llu," + "\"wall_ns\":%llu,\"durable_pairs\":%llu," + "\"durable_pairs_per_second_milli\":%llu,\"admissions\":%llu," + "\"sequences\":%llu,\"contract_changes\":%llu," + "\"selected_cpu_admissions\":%llu,\"selected_vulkan_admissions\":%llu," + "\"actual_cpu_sequences\":%llu,\"actual_vulkan_sequences\":%llu," + "\"actual_mixed_sequences\":%llu,\"fallback_sequences\":%llu," + "\"local_ineligible_fallback_sequences\":%llu," + "\"backend_failure_fallback_sequences\":%llu," + "\"backend_other_fallback_sequences\":%llu," + "\"local_ineligible_fallback_items\":%llu," + "\"backend_failure_fallback_items\":%llu," + "\"backend_other_fallback_items\":%llu," + "\"sequence_wall_ns\":%llu,\"min_mem_available_bytes\":", + static_cast(runtime.matcher_diagnostic_samples), + static_cast(wall_ns), + static_cast(aggregate.durable_items), + static_cast(durable_rate_milli), + static_cast(aggregate.admission_count), + static_cast(aggregate.sequence_count), + static_cast(aggregate.contract_change_count), + static_cast(aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_CPU]), + static_cast(aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN]), + static_cast(aggregate.actual_backend_sequences[ + LARDON3D_RESOURCE_BACKEND_CPU]), + static_cast(aggregate.actual_backend_sequences[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN]), + static_cast(aggregate.actual_backend_sequences[ + LARDON3D_RESOURCE_BACKEND_MIXED]), + static_cast(aggregate.backend_fallback_sequences), + static_cast( + aggregate.backend_ineligible_fallback_sequences), + static_cast( + aggregate.backend_failure_fallback_sequences), + static_cast( + aggregate.backend_other_fallback_sequences), + static_cast( + aggregate.local_ineligible_fallback_items), + static_cast( + aggregate.backend_failure_fallback_items), + static_cast( + aggregate.backend_other_fallback_items), + static_cast(aggregate.total_wall_time_ns)); + print_known_u64(aggregate_known && aggregate.memory_available_known, + aggregate.minimum_memory_available_bytes); + std::fputs(",\"max_gpu_busy_basis_points\":", stdout); + print_known_u32(aggregate_known && aggregate.gpu_busy_known, + aggregate.maximum_gpu_busy_basis_points); + std::fputs(",\"max_process_rss_bytes\":", stdout); + print_known_u64(aggregate_known && aggregate.process_rss_known, + aggregate.maximum_process_rss_bytes); + std::fputs(",\"max_process_peak_rss_bytes\":", stdout); + print_known_u64(aggregate_known && aggregate.process_peak_rss_known, + aggregate.maximum_process_peak_rss_bytes); + std::printf( + ",\"publication_ns\":%llu,\"vulkan_submits\":%llu," + "\"vulkan_completions\":%llu,\"vulkan_submit_cpu_ns\":%llu," + "\"vulkan_fence_wait_ns\":%llu," + "\"vulkan_readback_ns\":%llu,\"known_vulkan_gpu_ns\":%llu," + "\"vulkan_gpu_known_sequences\":%llu,\"vulkan_starvation_ns\":%llu," + "\"matcher_cpu_ns\":%llu,\"aggregate_saturated\":%s," + "\"backend_counter_delta_known\":%s,\"backend_failures\":%llu," + "\"backend_discards\":%llu,\"backend_slot_pending\":%s," + "\"affinity_known\":%s,\"affinity_active\":%s," + "\"runtime_thread_policy_active\":%s," + "\"mesa_shader_cache_disabled\":%s," + "\"compute_cpu_count\":%u,\"reserved_cpu_count\":%u," + "\"compute_mask\":", + static_cast(aggregate.publication_ns), + static_cast(aggregate.vulkan_submits), + static_cast(aggregate.vulkan_completions), + static_cast(aggregate.vulkan_submit_cpu_ns), + static_cast(aggregate.vulkan_fence_wait_ns), + static_cast(aggregate.vulkan_readback_ns), + static_cast(aggregate.vulkan_gpu_ns), + static_cast(aggregate.vulkan_gpu_known_sequences), + static_cast(aggregate.vulkan_starvation_ns), + static_cast(aggregate.matcher_cpu_ns), + aggregate.saturated ? "true" : "false", + backend_delta_known ? "true" : "false", + static_cast(backend_failures), + static_cast(backend_discards), + backend_slot_pending ? "true" : "false", + cpu_policy_known ? "true" : "false", + cpu_policy_known && cpu_policy.affinity_active ? "true" : "false", + cpu_policy_known && cpu_policy.runtime_thread_policy_active + ? "true" : "false", + cpu_policy_known && cpu_policy.mesa_shader_cache_disabled + ? "true" : "false", + cpu_policy.compute_cpu_count, cpu_policy.reserved_cpu_count); + print_cpu_mask(cpu_policy.compute_mask); + std::fputs(",\"reserved_mask\":", stdout); + print_cpu_mask(cpu_policy.reserved_mask); + std::fputs(",\"runtime_thread_policy_reason\":", stdout); + print_json_string(cpu_policy_known + ? cpu_policy.runtime_thread_policy_reason : "unknown"); + std::fputs(",\"matcher_inflight_override\":", stdout); + print_known_u32(runtime.matcher_inflight_override != 0, + runtime.matcher_inflight_override); + std::fputs(",\"matcher_batch_override\":", stdout); + print_known_u32(runtime.matcher_batch_override != 0, + runtime.matcher_batch_override); + std::fputs(",\"experiment_valid\":", stdout); + if (experiment.applicable) + std::fputs(experiment.valid ? "true" : "false", stdout); + else + std::fputs("null", stdout); + std::fputs(",\"experiment_reason\":", stdout); + if (experiment.applicable) print_json_string(experiment.reason); + else std::fputs("null", stdout); + std::fputs( + ",\"comparison_requires_equal_local_ineligible_fallback_items\":", + stdout); + if (experiment.applicable) std::fputs("true", stdout); + else std::fputs("null", stdout); + std::printf(",\"uma\":%s,\"last_contract_known\":%s", + runtime.state.hardware_profile.gpu_uses_shared_memory ? "true" : "false", + last_known ? "true" : "false"); + if (last_known) { + std::printf(",\"last_selected_backend\":\"%s\",\"last_actual_backend\":\"%s\"," + "\"last_cpu\":%u,\"last_gpu\":%u,\"last_batch\":%zu," + "\"last_inflight\":%zu,\"last_helpers\":%u,\"last_reason\":", + backend_name(last.backend), backend_name(last.actual_backend), + last.cpu_threads, last.gpu_slots, last.batch_size, + last.inflight_limit, last.helper_limit); + print_json_string(last.reason); + } + std::fputs("}\n", stdout); + runtime.matcher_evidence_active = false; + return !experiment.applicable || experiment.valid; +} + bool wait_completed(Runtime &runtime, uint64_t task_id, const char *phase) { for (;;) { + emit_matcher_diagnostic_change(runtime); Lardon3DTaskSnapshot snapshot{}; if (!lardon3d_task_queue_get(runtime.state.task_queue, task_id, &snapshot)) return false; if (snapshot.state == TASK_COMPLETED || snapshot.state == TASK_FAILED || @@ -404,6 +1059,7 @@ bool wait_completed(Runtime &runtime, uint64_t task_id, const char *phase) { lardon3d_task_state_name(snapshot.state), snapshot.message); return false; } + emit_matcher_diagnostic_change(runtime); return lardon3d_task_queue_remove(runtime.state.task_queue, task_id); } } @@ -665,6 +1321,7 @@ bool collect_verified_ids(Runtime &runtime, const unsigned char verifier_fingerp bool downstream(Runtime &runtime, const std::vector &feature_sets, const Lardon3DFeatureExtractorParameters &orb, uint64_t &geometry_task_id, + uint64_t &matcher_task_id, std::vector &verified_ids) { Lardon3DVisualIndexConfiguration index_configuration{ LARDON3D_VISUAL_INDEX_VERSION, 1024, 256}; @@ -693,9 +1350,14 @@ bool downstream(Runtime &runtime, const std::vector lardon3d_feature_extractor_parameter_fingerprint(&orb, matcher.feature_parameter_fingerprint); matcher.matcher.kind = LARDON3D_MATCHER_ORB_BF; matcher.matcher.ratio_threshold = lardon3d_matcher_default_ratio(LARDON3D_MATCHER_ORB_BF); - if (!lardon3d_project_enqueue_matcher_task(&runtime.state, &matcher, &task_id) || - !wait_completed(runtime, task_id, "matcher.run")) + begin_matcher_evidence(runtime); + const bool matcher_ok = + lardon3d_project_enqueue_matcher_task(&runtime.state, &matcher, &task_id) && + wait_completed(runtime, task_id, "matcher.run"); + const bool matcher_experiment_valid = end_matcher_evidence(runtime); + if (!matcher_ok || !matcher_experiment_valid) return false; + matcher_task_id = task_id; Lardon3DGeometricVerifierTaskConfiguration verifier{ lardon3d_geometric_verifier_default_parameters()}; if (!lardon3d_project_enqueue_geometric_verifier_task(&runtime.state, &verifier, @@ -839,6 +1501,260 @@ bool collect_evidence(Runtime &runtime, uint64_t execution_id, Evidence &evidenc return true; } +bool digest_update(EVP_MD_CTX *context, const void *bytes, size_t size) { + return EVP_DigestUpdate(context, bytes, size) == 1; +} + +bool digest_u32(EVP_MD_CTX *context, uint32_t value) { + unsigned char encoded[4]; + for (size_t index = 0; index < sizeof(encoded); ++index) { + encoded[index] = static_cast(value & 0xffU); + value >>= 8U; + } + return digest_update(context, encoded, sizeof(encoded)); +} + +bool digest_u64(EVP_MD_CTX *context, uint64_t value) { + unsigned char encoded[8]; + for (size_t index = 0; index < sizeof(encoded); ++index) { + encoded[index] = static_cast(value & 0xffU); + value >>= 8U; + } + return digest_update(context, encoded, sizeof(encoded)); +} + +bool digest_string(EVP_MD_CTX *context, const char *text, size_t capacity) { + const size_t length = strnlen(text, capacity); + return length < capacity && length <= UINT32_MAX && + digest_u32(context, static_cast(length)) && + digest_update(context, text, length); +} + +struct CandidateStream { + uint64_t cursor{}; + Lardon3DProjectDbCandidatePair page[64]{}; + size_t count{}; + size_t index{}; + bool exhausted{}; +}; + +bool next_candidate(Lardon3DProjectDb *database, CandidateStream &stream, + Lardon3DProjectDbCandidatePair &candidate, bool &has_value) { + has_value = false; + if (stream.exhausted) return true; + if (stream.index == stream.count) { + stream.index = 0; + stream.count = 0; + if (lardon3d_project_db_list_candidate_pairs(database, stream.cursor, + stream.page, 64, + &stream.count) != + LARDON3D_PROJECT_DB_OK) + return false; + if (stream.count == 0) { + stream.exhausted = true; + return true; + } + } + candidate = stream.page[stream.index++]; + if (candidate.candidate_pair_id <= stream.cursor) return false; + stream.cursor = candidate.candidate_pair_id; + has_value = true; + return true; +} + +struct MatchStream { + uint64_t cursor{}; + Lardon3DProjectDbMatchResult page[64]{}; + size_t count{}; + size_t index{}; + bool exhausted{}; +}; + +bool next_match(Lardon3DProjectDb *database, MatchStream &stream, + Lardon3DProjectDbMatchResult &match, bool &has_value) { + has_value = false; + if (stream.exhausted) return true; + if (stream.index == stream.count) { + stream.index = 0; + stream.count = 0; + if (lardon3d_project_db_list_match_results(database, stream.cursor, + stream.page, 64, + &stream.count) != + LARDON3D_PROJECT_DB_OK) + return false; + if (stream.count == 0) { + stream.exhausted = true; + return true; + } + } + match = stream.page[stream.index++]; + if (match.match_result_id <= stream.cursor) return false; + stream.cursor = match.match_result_id; + has_value = true; + return true; +} + +bool query_match_audit_sql(const Runtime &runtime, uint64_t &latest_matcher_task_id, + size_t &duplicate_mappings) { + latest_matcher_task_id = 0; + duplicate_mappings = 0; + sqlite3 *database = nullptr; + sqlite3_stmt *statement = nullptr; + if (sqlite3_open_v2(runtime.database_path.c_str(), &database, + SQLITE_OPEN_READONLY | SQLITE_OPEN_NOMUTEX, nullptr) != SQLITE_OK) { + if (database) sqlite3_close(database); + return false; + } + bool ok = sqlite3_prepare_v2( + database, + "SELECT task_id FROM matcher_tasks ORDER BY task_id DESC LIMIT 1", + -1, &statement, nullptr) == SQLITE_OK && + sqlite3_step(statement) == SQLITE_ROW; + if (ok) { + const sqlite3_int64 task_id = sqlite3_column_int64(statement, 0); + ok = task_id > 0; + if (ok) latest_matcher_task_id = static_cast(task_id); + } + if (statement) sqlite3_finalize(statement); + statement = nullptr; + ok = ok && sqlite3_prepare_v2( + database, + "SELECT COALESCE(SUM(n-1),0) FROM (SELECT COUNT(*) AS n " + "FROM match_results GROUP BY candidate_pair_id HAVING COUNT(*)>1)", + -1, &statement, nullptr) == SQLITE_OK && + sqlite3_step(statement) == SQLITE_ROW; + if (ok) { + const sqlite3_int64 duplicates = sqlite3_column_int64(statement, 0); + ok = duplicates >= 0 && static_cast(duplicates) <= SIZE_MAX; + if (ok) duplicate_mappings = static_cast(duplicates); + } + if (statement) sqlite3_finalize(statement); + return sqlite3_close(database) == SQLITE_OK && ok; +} + +bool validate_match_asset(const Runtime &runtime, + const Lardon3DProjectDbMatchResult &match) { + if (match.result_status == LARDON3D_MATCH_RESULT_STATUS_NO_MATCH) + return match.match_count == 0 && !match.has_match_asset && + match.match_asset_path[0] == '\0' && match.match_asset_size_bytes == 0; + if (match.result_status != LARDON3D_MATCH_RESULT_STATUS_MATCHED || + match.match_count == 0 || !match.has_match_asset || + match.match_asset_path[0] == '\0' || match.match_asset_size_bytes == 0) + return false; + const std::filesystem::path relative(match.match_asset_path); + if (relative.is_absolute() || relative.lexically_normal() != relative) return false; + for (const auto &component : relative) + if (component == "..") return false; + Lardon3DProjectDbFeatureSet feature_a{}; + Lardon3DProjectDbFeatureSet feature_b{}; + if (lardon3d_project_db_load_feature_set(runtime.state.project_db, + match.feature_set_id_a, + &feature_a) != LARDON3D_PROJECT_DB_OK || + lardon3d_project_db_load_feature_set(runtime.state.project_db, + match.feature_set_id_b, + &feature_b) != LARDON3D_PROJECT_DB_OK) + return false; + const std::string full_path = + (std::filesystem::path(runtime.project_path) / relative).string(); + Lardon3DMatchFileHeader header{}; + return lardon3d_match_file_validate_asset( + full_path.c_str(), match.match_asset_sha256, + match.match_asset_size_bytes, &header, match.feature_set_id_a, + match.feature_set_id_b, feature_a.feature_count, + feature_b.feature_count) == LARDON3D_MATCH_FILE_OK && + header.match_count == match.match_count; +} + +bool audit_match_results(Runtime &runtime, uint64_t &matcher_task_id, + MatchAudit &audit) { + uint64_t latest_matcher_task_id = 0; + if (!query_match_audit_sql(runtime, latest_matcher_task_id, + audit.duplicate_candidate_pair_mappings) || + audit.duplicate_candidate_pair_mappings != 0) + return false; + if (matcher_task_id == 0) matcher_task_id = latest_matcher_task_id; + if (matcher_task_id == 0) return false; + Lardon3DProjectDbTask durable_task{}; + Lardon3DProjectDbMatcherTask durable_matcher{}; + if (lardon3d_project_db_load_task(runtime.state.project_db, matcher_task_id, + &durable_task) != LARDON3D_PROJECT_DB_OK || + durable_task.saved_state != TASK_COMPLETED || + lardon3d_project_db_load_matcher_task(runtime.state.project_db, + matcher_task_id, + &durable_matcher) != + LARDON3D_PROJECT_DB_OK) + return false; + + using DigestContext = std::unique_ptr; + DigestContext digest(EVP_MD_CTX_new(), EVP_MD_CTX_free); + static constexpr unsigned char format_tag[8] = { + 'L', '3', 'D', 'M', 'R', 'D', '1', '\0'}; + if (!digest || EVP_DigestInit_ex(digest.get(), EVP_sha256(), nullptr) != 1 || + !digest_update(digest.get(), format_tag, sizeof(format_tag))) + return false; + + CandidateStream candidates; + MatchStream matches; + uint64_t last_candidate_id = 0; + for (;;) { + Lardon3DProjectDbCandidatePair candidate{}; + Lardon3DProjectDbMatchResult match{}; + bool has_candidate = false; + bool has_match = false; + if (!next_candidate(runtime.state.project_db, candidates, candidate, + has_candidate) || + !next_match(runtime.state.project_db, matches, match, has_match) || + has_candidate != has_match) + return false; + if (!has_candidate) break; + if (candidate.candidate_pair_id != match.candidate_pair_id || + !validate_match_asset(runtime, match)) + return false; + const size_t matcher_kind_length = + strnlen(match.matcher_kind, sizeof(match.matcher_kind)); + if (matcher_kind_length == sizeof(match.matcher_kind)) return false; + const unsigned char has_asset = match.has_match_asset ? 1U : 0U; + unsigned char zero_sha[32]{}; + if (!digest_u64(digest.get(), match.candidate_pair_id) || + !digest_u64(digest.get(), match.feature_set_id_a) || + !digest_u64(digest.get(), match.feature_set_id_b) || + !digest_string(digest.get(), match.matcher_kind, + sizeof(match.matcher_kind)) || + !digest_u32(digest.get(), match.matcher_version) || + !digest_update(digest.get(), match.parameter_fingerprint, + sizeof(match.parameter_fingerprint)) || + !digest_u32(digest.get(), static_cast(match.result_status)) || + !digest_u32(digest.get(), match.match_count) || + !digest_update(digest.get(), &has_asset, sizeof(has_asset)) || + !digest_update(digest.get(), match.has_match_asset + ? match.match_asset_sha256 : zero_sha, + sizeof(zero_sha)) || + !digest_u64(digest.get(), match.match_asset_size_bytes)) + return false; + ++audit.match_result_count; + if (match.has_match_asset) ++audit.match_asset_count; + last_candidate_id = candidate.candidate_pair_id; + } + audit.candidate_mapping_contiguous = true; + audit.matcher_cursor_complete = + durable_matcher.after_candidate_pair_id == last_candidate_id; + if (!audit.matcher_cursor_complete || + !digest_u64(digest.get(), static_cast(audit.match_result_count))) + return false; + unsigned char output[EVP_MAX_MD_SIZE]{}; + unsigned int output_size = 0; + if (EVP_DigestFinal_ex(digest.get(), output, &output_size) != 1 || + output_size != 32) + return false; + static constexpr char digits[] = "0123456789abcdef"; + for (size_t index = 0; index < 32; ++index) { + audit.digest_hex[2 * index] = digits[output[index] >> 4U]; + audit.digest_hex[2 * index + 1] = digits[output[index] & 0x0fU]; + } + audit.digest_hex[64] = '\0'; + return true; +} + bool count_exact_gvrs(Runtime &runtime, const unsigned char fingerprint[32], size_t &count) { count = 0; @@ -1060,13 +1976,29 @@ bool run_existing_pre_gv(Runtime &runtime) { } if (pending_matcher_task_id != 0) matcher_task_id = pending_matcher_task_id; + if ((runtime.matcher_pipeline == MatcherPipeline::kSynchronous || + runtime.matcher_inflight_override != 0 || + runtime.matcher_batch_override != 0) && + matcher_task_id != 0) { + /* Pipeline is deliberately absent from Task identity/checkpoints. Refuse a + * recovered Matcher rather than pretending a non-persisted benchmark + * control survived restart. Rolling production recovery remains normal. */ + std::fprintf(stderr, + "benchmark Matcher pipeline/inflight controls require a new " + "Matcher task; a pending Matcher cannot retain them\n"); + stop_runtime(runtime); + return false; + } + Lardon3DProjectRecoverySummary matcher_recovery{}; const char *matcher_action = "already_complete"; + begin_matcher_evidence(runtime); if (after_candidate.matches < after_candidate.pairs) { if (matcher_task_id != 0) { if (!recover_one_pre_gv_task(runtime, matcher_task_id, "matcher.run recovered", matcher_recovery)) { + end_matcher_evidence(runtime); stop_runtime(runtime); return false; } @@ -1083,9 +2015,21 @@ bool run_existing_pre_gv(Runtime &runtime) { matcher.matcher.kind = LARDON3D_MATCHER_ORB_BF; matcher.matcher.ratio_threshold = lardon3d_matcher_default_ratio(LARDON3D_MATCHER_ORB_BF); - if (!lardon3d_project_enqueue_matcher_task(&runtime.state, &matcher, - &matcher_task_id) || + bool enqueued = false; + if (runtime.matcher_mode == MatcherMode::kAuto) { + enqueued = lardon3d_project_enqueue_matcher_task( + &runtime.state, &matcher, &matcher_task_id); + } else { + const Lardon3DMatcherTaskMode matcher_mode = + runtime.matcher_mode == MatcherMode::kVulkan + ? LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN + : LARDON3D_MATCHER_TASK_MODE_CPU_PARALLEL; + enqueued = lardon3d_project_enqueue_matcher_task_with_mode( + &runtime.state, &matcher, matcher_mode, &matcher_task_id); + } + if (!enqueued || !wait_completed(runtime, matcher_task_id, "matcher.run existing")) { + end_matcher_evidence(runtime); stop_runtime(runtime); return false; } @@ -1094,17 +2038,27 @@ bool run_existing_pre_gv(Runtime &runtime) { } else if (matcher_task_id != 0) { std::fprintf(stderr, "complete Match Results coexist with a pending Matcher task\n"); + end_matcher_evidence(runtime); stop_runtime(runtime); return false; } + const bool matcher_experiment_valid = end_matcher_evidence(runtime); Evidence after{}; - const bool ok = collect_evidence(runtime, 0, after) && + MatchAudit match_audit{}; + const bool ok = matcher_experiment_valid && + collect_evidence(runtime, 0, after) && after.features == before.features && after.pairs >= before.pairs && after.matches == after.pairs && after.verified == before.verified && after.rejected == before.rejected && - after.tracks == before.tracks; + after.tracks == before.tracks && + audit_match_results(runtime, matcher_task_id, match_audit) && + match_audit.match_result_count == after.matches; + const char *matcher_mode_name = runtime.matcher_mode == MatcherMode::kAuto + ? "auto" : runtime.matcher_mode == MatcherMode::kCpu ? "cpu" : "vulkan"; + const char *pipeline_name = runtime.matcher_pipeline == MatcherPipeline::kRolling + ? "rolling" : "synchronous"; std::printf( "{\"record\":\"existing_pre_gv_summary\",\"ok\":%s," "\"candidate_task_id\":%llu,\"candidate_action\":\"%s\"," @@ -1116,6 +2070,12 @@ bool run_existing_pre_gv(Runtime &runtime) { "\"match_results_before\":%zu,\"match_results_after\":%zu," "\"gvrs_before\":%zu,\"gvrs_after\":%zu," "\"tracks_before\":%zu,\"tracks_after\":%zu," + "\"matcher_mode\":\"%s\",\"matcher_pipeline\":\"%s\"," + "\"matcher_inflight_override\":%s,\"matcher_batch_override\":%s," + "\"match_output_digest\":\"%s\",\"match_digest_format\":\"L3DMRD1\"," + "\"match_asset_validation\":\"sha-size-header-and-entries\"," + "\"match_asset_count\":%zu,\"duplicate_candidate_pair_mappings\":%zu," + "\"candidate_mapping_contiguous\":%s,\"matcher_cursor_complete\":%s," "\"prior_stages_replayed\":false,\"gv_enqueued\":false," "\"track_builder_enqueued\":false,\"sparse_sfm_run\":false}\n", ok ? "true" : "false", @@ -1126,7 +2086,17 @@ bool run_existing_pre_gv(Runtime &runtime) { matcher_recovery.resumed, before.features, after.features, before.pairs, after.pairs, before.matches, after.matches, before.verified + before.rejected, after.verified + after.rejected, - before.tracks, after.tracks); + before.tracks, after.tracks, matcher_mode_name, pipeline_name, + runtime.matcher_inflight_override == 0 ? "null" : + runtime.matcher_inflight_override == 1 ? "1" : "2", + runtime.matcher_batch_override == 0 ? "null" : + runtime.matcher_batch_override == 2 ? "2" : + runtime.matcher_batch_override == 4 ? "4" : + runtime.matcher_batch_override == 8 ? "8" : "12", + match_audit.digest_hex, match_audit.match_asset_count, + match_audit.duplicate_candidate_pair_mappings, + match_audit.candidate_mapping_contiguous ? "true" : "false", + match_audit.matcher_cursor_complete ? "true" : "false"); stop_runtime(runtime); return ok; } @@ -1208,11 +2178,76 @@ bool run_existing_geometry(Runtime &runtime) { } // namespace int main(int argc, char **argv) { + /* The runner is a production-runtime consumer even though its evidence + * controls are benchmark-only. Establish Mesa's no-disk-worker policy before + * argument handling or any application pthread creation. */ + const Lardon3DResourceDriverPolicyResult driver_policy = + lardon3d_resource_governor_internal_configure_driver_policy(); + if (driver_policy == LARDON3D_RESOURCE_DRIVER_POLICY_FAILED || + driver_policy == LARDON3D_RESOURCE_DRIVER_POLICY_REJECTED_UNSAFE) { + std::fprintf(stderr, + "MESA_SHADER_CACHE_DISABLE must be true for safe CPU affinity\n"); + return 2; + } Options options; if (!parse_options(argc, argv, options)) { usage(argv[0]); return 2; } + /* Reject this contradictory operational request before opening a project or + * allocating a durable Task ID. A Vulkan Matcher has an immutable GPU=1 + * estimate, while GPU budget zero expressly removes GPU admission. */ + if (options.matcher_mode == MatcherMode::kVulkan && options.has_gpu_budget && + options.gpu_budget == 0) { + std::fprintf(stderr, "--matcher-mode vulkan requires --gpu-budget 1\n"); + return 2; + } + if (options.has_matcher_inflight_override && options.has_gpu_budget && + options.gpu_budget == 0) { + /* A forced A/B cohort has no CPU admission alternative. Reject the + * contradictory budget before project inspection so a typo cannot create + * a durable Task that merely waits or silently measures CPU. */ + std::fprintf(stderr, "--matcher-inflight requires --gpu-budget 1\n"); + return 2; + } + if (options.matcher_mode == MatcherMode::kCpu && + options.matcher_pipeline == MatcherPipeline::kSynchronous) { + std::fprintf(stderr, + "--matcher-pipeline synchronous requires auto or vulkan; " + "CPU has no Vulkan fence baseline\n"); + return 2; + } + if (options.has_matcher_inflight_override && + options.matcher_mode != MatcherMode::kAuto) { + std::fprintf(stderr, + "--matcher-inflight requires normal AUTO Matcher mode\n"); + return 2; + } + if (options.has_matcher_batch_override && + !options.has_matcher_inflight_override) { + std::fprintf(stderr, + "--matcher-batch requires --matcher-inflight 1 or 2\n"); + return 2; + } + if (options.has_matcher_batch_override && + options.matcher_pipeline != MatcherPipeline::kRolling) { + std::fprintf(stderr, "--matcher-batch requires rolling pipeline\n"); + return 2; + } + if (options.matcher_pipeline == MatcherPipeline::kSynchronous && + options.matcher_inflight_override == 2) { + std::fprintf(stderr, + "--matcher-pipeline synchronous supports inflight 1 only\n"); + return 2; + } + /* These environment seams are compiled only into this runner's private + * Matcher Task object. Defaults actively clear inherited controls, while the + * scoped owner restores their exact prior values on every return path. */ + ScopedMatcherBenchmarkEnvironment benchmark_environment; + if (!benchmark_environment.configure(options)) { + std::fprintf(stderr, "unable to configure benchmark Matcher controls\n"); + return 2; + } Runtime runtime; if (options.resume_geometry_existing || options.resume_pre_gv_existing || options.resume_candidate_existing) { @@ -1221,6 +2256,14 @@ int main(int argc, char **argv) { return 2; } runtime.cpu_budget = options.cpu_budget; + runtime.gpu_budget = options.gpu_budget; + runtime.has_gpu_budget = options.has_gpu_budget; + runtime.matcher_mode = options.matcher_mode; + runtime.matcher_pipeline = options.matcher_pipeline; + runtime.matcher_inflight_override = options.matcher_inflight_override; + runtime.matcher_batch_override = options.matcher_batch_override; + runtime.matcher_needed = options.resume_pre_gv_existing; + runtime.stop_after_matcher = options.stop_after_matcher; if (options.resume_candidate_existing) return run_existing_candidate(runtime) ? 0 : 1; return (options.resume_pre_gv_existing ? run_existing_pre_gv(runtime) @@ -1231,6 +2274,7 @@ int main(int argc, char **argv) { return 2; } runtime.database_path = (options.project_dir / "project.lardon3d").string(); + runtime.matcher_needed = true; Campaign campaign; if (!discover_campaign(options, campaign) || !start_runtime(runtime)) { std::fprintf(stderr, "campaign discovery or runtime setup failed\n"); @@ -1276,11 +2320,14 @@ int main(int argc, char **argv) { } uint64_t geometry_task_id = 0; + uint64_t matcher_task_id = 0; std::vector verified_ids; Lardon3DGeometricVerifierParameters verifier = lardon3d_geometric_verifier_default_parameters(); unsigned char verifier_fingerprint[32]{}; lardon3d_geometric_verifier_fingerprint(&verifier, verifier_fingerprint); - if (ok) ok = downstream(runtime, features, orb, geometry_task_id, verified_ids); + if (ok) + ok = downstream(runtime, features, orb, geometry_task_id, matcher_task_id, + verified_ids); if (ok && options.restart == RestartBoundary::kGeometry) { ok = restart_runtime(runtime, execution.execution_id, "geometry"); if (ok) ok = collect_verified_ids(runtime, verifier_fingerprint, verified_ids); @@ -1298,12 +2345,26 @@ int main(int argc, char **argv) { ok = start_runtime(runtime); } Evidence evidence{}; - if (ok) ok = collect_evidence(runtime, execution.execution_id, evidence); + MatchAudit match_audit{}; + if (ok) + ok = collect_evidence(runtime, execution.execution_id, evidence) && + audit_match_results(runtime, matcher_task_id, match_audit) && + match_audit.match_result_count == evidence.matches; if (ok) { std::printf("{\"record\":\"summary\",\"mode\":\"%s\"," "\"execution_id\":%llu,\"selected\":%zu,\"feature_sets\":%zu," "\"candidate_pairs\":%zu,\"matches\":%zu,\"verified\":%zu," "\"rejected\":%zu,\"tracks\":%zu," + "\"matcher_mode\":\"auto\",\"matcher_pipeline\":\"rolling\"," + "\"matcher_inflight_override\":null," + "\"matcher_batch_override\":null," + "\"match_output_digest\":\"%s\"," + "\"match_digest_format\":\"L3DMRD1\"," + "\"match_asset_validation\":\"sha-size-header-and-entries\"," + "\"match_asset_count\":%zu," + "\"duplicate_candidate_pair_mappings\":%zu," + "\"candidate_mapping_contiguous\":%s," + "\"matcher_cursor_complete\":%s," "\"orb_max_features_per_set\":8192," "\"visual_index_max_features_per_set\":1024," "\"visual_index_feature_set_limit\":4096," @@ -1311,7 +2372,11 @@ int main(int argc, char **argv) { "\"calibration_attached\":false,\"sparse_sfm_run\":false}\n", mode_name, static_cast(execution.execution_id), evidence.selected, evidence.features, evidence.pairs, evidence.matches, - evidence.verified, evidence.rejected, evidence.tracks); + evidence.verified, evidence.rejected, evidence.tracks, + match_audit.digest_hex, match_audit.match_asset_count, + match_audit.duplicate_candidate_pair_mappings, + match_audit.candidate_mapping_contiguous ? "true" : "false", + match_audit.matcher_cursor_complete ? "true" : "false"); } stop_runtime(runtime); return ok ? 0 : 1; diff --git a/tests/test_acquisition_campaign_task.cpp b/tests/test_acquisition_campaign_task.cpp index 6940c7b..2736de0 100644 --- a/tests/test_acquisition_campaign_task.cpp +++ b/tests/test_acquisition_campaign_task.cpp @@ -7,8 +7,13 @@ extern "C" { #include #include +#include "../src/task_internal.h" } +extern "C" bool +lardon3d_acquisition_campaign_task_internal_configure_restored( + Lardon3DTask *, void *); + #define CHECK(x) \ do { \ if (!(x)) { \ @@ -173,6 +178,45 @@ int main() { CHECK(loaded_task.next_group_id == 1 && loaded_task.group_count == 2 && loaded_task.request_size == encoded.size() && std::memcmp(loaded.data(), encoded.data(), encoded.size()) == 0); + snapshot.estimate = Lardon3DResourceEstimate{ + 256 * 1024, 0, 64 * 1024, 0, 1, 1, 1, 0, 1, + LARDON3D_RESOURCE_TASK_IMPORT}; + Lardon3DHardwareProfile profile{16, 4096, UINT64_MAX, false, 0, false, + false, 0, "test", ""}; + Lardon3DResourcePolicy policy{}; + policy.maximum_cpu_load_ratio = 1.0; + policy.maximum_io_pressure_avg10 = 100.0; + policy.io_slot_capacity = 1; + Lardon3DResourceGovernor *governor = + lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor != nullptr); + Lardon3DTaskReconstructionContext valid_reconstruction{ + "/tmp", database, governor, nullptr}; + Lardon3DTaskKindBinding recovered{}; + CHECK(lardon3d_acquisition_campaign_task_reconstruct( + &snapshot, &valid_reconstruction, &recovered)); + Lardon3DTask *restored = lardon3d_task_restore_typed( + &snapshot, LARDON3D_ACQUISITION_CAMPAIGN_TASK_KIND, + LARDON3D_ACQUISITION_CAMPAIGN_TASK_KIND_VERSION, recovered.callback, + recovered.userdata, recovered.userdata_destroy); + CHECK(restored && + lardon3d_acquisition_campaign_task_internal_configure_restored( + restored, recovered.userdata)); + Lardon3DResourceDecision decision{}; + Lardon3DResourceReservation *reservation = nullptr; + CHECK(lardon3d_task_internal_reserve_available( + restored, governor, &decision, &reservation)); + Lardon3DResourceReservationInfo admitted{}; + CHECK(reservation && lardon3d_resource_reservation_get_active( + governor, reservation, &admitted)); + CHECK(admitted.memory_bytes > 320 * 1024); + Lardon3DTaskDurableSnapshot unchanged{}; + CHECK(lardon3d_task_durable_snapshot(restored, &unchanged)); + CHECK(unchanged.estimate.memory_fixed_bytes == 256 * 1024 && + unchanged.estimate.memory_bytes_per_item == 64 * 1024); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + lardon3d_task_destroy(restored); + lardon3d_resource_governor_destroy(governor); std::vector overlong_project_path(LARDON3D_APP_STATE_PATH_CAPACITY + 1, 'x'); overlong_project_path.back() = '\0'; diff --git a/tests/test_candidate_pair_task.c b/tests/test_candidate_pair_task.c index 3f3aaa5..7e13bfe 100644 --- a/tests/test_candidate_pair_task.c +++ b/tests/test_candidate_pair_task.c @@ -338,7 +338,10 @@ static bool run_test(void) { &state, visual_index_id, &qopts, &multi_sequence_task_id) && wait_state(state.task_queue, multi_sequence_task_id, TASK_COMPLETED, &snap) && snap.progress == 100 && - lardon3d_candidate_pair_task_test_started_participants() >= 6 && + /* A fresh kind history begins at CPU1. This small fixture proves + * every durable item executes; synthetic throughput is not allowed + * to assume that CPU2 was already demonstrated beneficial. */ + lardon3d_candidate_pair_task_test_started_participants() >= 1 && lardon3d_candidate_pair_task_test_computed_work_items() >= 6); Lardon3DProjectDbTask multi_sequence_task; CHECK(lardon3d_project_db_load_task(state.project_db, multi_sequence_task_id, diff --git a/tests/test_driver_policy_startup.py b/tests/test_driver_policy_startup.py new file mode 100644 index 0000000..193db58 --- /dev/null +++ b/tests/test_driver_policy_startup.py @@ -0,0 +1,37 @@ +import os +import subprocess +import sys + + +def main() -> int: + if len(sys.argv) != 2: + return 2 + environment = os.environ.copy() + environment["MESA_SHADER_CACHE_DISABLE"] = "false" + try: + result = subprocess.run( + [sys.argv[1]], + check=False, + env=environment, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + timeout=4, + ) + except (OSError, subprocess.TimeoutExpired) as error: + print(f"unable to execute production boundary: {error}", file=sys.stderr) + return 1 + + expected = b"MESA_SHADER_CACHE_DISABLE must be true for safe CPU affinity\n" + if result.returncode != 1 or result.stdout or result.stderr != expected: + print( + "unexpected production boundary result: " + f"status={result.returncode} stdout={result.stdout!r} " + f"stderr={result.stderr!r}", + file=sys.stderr, + ) + return 1 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/test_feature_task.c b/tests/test_feature_task.c index 8d7d83e..93f8616 100644 --- a/tests/test_feature_task.c +++ b/tests/test_feature_task.c @@ -1,3 +1,7 @@ +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif + #include #include #include @@ -24,6 +28,10 @@ #include #include +#include "../src/opencv_task_thread_control.h" +#include "../src/resource_governor_internal.h" +#include "../src/task_internal.h" + #define CHECK(x) \ do { \ if (!(x)) { \ @@ -95,6 +103,196 @@ static bool write_uniform_pgm(const char *path) { for (size_t y = 0; ok && y < 192; ++y) ok = fwrite(row, 1, sizeof(row), file) == sizeof(row); return fclose(file) == 0 && ok; } + +static bool wait_state(Lardon3DTaskQueue *q, uint64_t id, + Lardon3DTaskState wanted, + Lardon3DTaskSnapshot *out); + +static bool extracted_equal(const Lardon3DExtractedFeatures *left, + const Lardon3DExtractedFeatures *right) { + if (!left || !right || left->image_width != right->image_width || + left->image_height != right->image_height || + left->feature_count != right->feature_count || + left->descriptor_bytes != right->descriptor_bytes || + memcmp(&left->quality, &right->quality, sizeof(left->quality)) != 0 || + (left->descriptor_bytes > 0 && + memcmp(left->descriptors, right->descriptors, + left->descriptor_bytes) != 0)) { + return false; + } + for (uint32_t index = 0; index < left->feature_count; ++index) { + const Lardon3DFeatureKeypoint *a = &left->keypoints[index]; + const Lardon3DFeatureKeypoint *b = &right->keypoints[index]; + if (memcmp(&a->x, &b->x, sizeof(a->x)) != 0 || + memcmp(&a->y, &b->y, sizeof(a->y)) != 0 || + memcmp(&a->size, &b->size, sizeof(a->size)) != 0 || + memcmp(&a->angle_degrees, &b->angle_degrees, + sizeof(a->angle_degrees)) != 0 || + memcmp(&a->response, &b->response, sizeof(a->response)) != 0 || + a->octave != b->octave) { + return false; + } + } + return true; +} + +typedef enum { + TEST_EXTRACT_ORB = 0, + TEST_EXTRACT_SIFT = 1, + TEST_EXTRACT_ROOTSIFT = 2, + TEST_EXTRACT_COUNT = 3, +} TestExtractKind; + +typedef struct { + const char *path; + unsigned int expected_threads; + TestExtractKind kind; + Lardon3DExtractedFeatures output; +} AdaptiveExtractWork; + +static bool adaptive_extract_callback(Lardon3DTask *task, void *userdata) { + AdaptiveExtractWork *work = userdata; + Lardon3DOpenCvTaskThreadControl control; + if (!lardon3d_opencv_task_threads_begin(task, 12, &control) || + lardon3d_feature_opencv_thread_count() != work->expected_threads) { + return false; + } + Lardon3DFeatureExtractResult result; + if (work->kind == TEST_EXTRACT_ORB) { + const Lardon3DFeatureExtractorParameters parameters = {512, 4, 10}; + result = lardon3d_feature_extract_orb(work->path, ¶meters, + &work->output); + } else { + Lardon3DSiftExtractorParameters parameters = + lardon3d_sift_precision_classic_v1( + work->kind == TEST_EXTRACT_ROOTSIFT); + parameters.max_features = 512; + result = lardon3d_feature_extract_sift(work->path, ¶meters, + &work->output); + } + bool restored = lardon3d_opencv_task_threads_end(&control); + return result == LARDON3D_FEATURE_EXTRACT_OK && restored && + lardon3d_task_set_progress(task, 100, "Extraction adaptative testée."); +} + +static bool run_adaptive_output_equivalence(const char *path) { + cpu_set_t allowed_mask; + CPU_ZERO(&allowed_mask); + if (sched_getaffinity(0, sizeof(allowed_mask), &allowed_mask) != 0) { + return true; + } + unsigned int allowed_ids[LARDON3D_RESOURCE_CPU_MAX]; + size_t allowed_count = 0; + for (unsigned int cpu = 0; cpu < CPU_SETSIZE && + cpu < LARDON3D_RESOURCE_CPU_MAX; ++cpu) { + if (CPU_ISSET((size_t)cpu, &allowed_mask)) { + allowed_ids[allowed_count++] = cpu; + } + } + if (allowed_count == 0) return true; + const unsigned int requested[] = {1, 2, 4, 8, 12}; + Lardon3DExtractedFeatures baseline[TEST_EXTRACT_COUNT] = {0}; + bool have_baseline[TEST_EXTRACT_COUNT] = {false}; + unsigned int previous_threads = lardon3d_feature_opencv_thread_count(); + bool ok = true; + for (size_t request_index = 0; + request_index < sizeof(requested) / sizeof(requested[0]) && ok; + ++request_index) { + unsigned int threads = requested[request_index]; + if (threads > allowed_count) continue; + Lardon3DHardwareProfile profile = { + .logical_cpu_count = threads + 4, + .page_size_bytes = 4096, + .memory_total_bytes = UINT64_C(16) * 1024 * 1024 * 1024, + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy = { + .system_cpu_reserve = 4, + .maximum_cpu_load_ratio = 1.0, + .maximum_io_pressure_avg10 = 100.0, + .io_slot_capacity = 1, + }; + Lardon3DResourceGovernor *governor = + lardon3d_resource_governor_create(&profile, &policy); + Lardon3DResourceCpuTopologyInput topology = { + .affinity_available = true, + .topology_available = false, + .allowed_cpu_count = threads, + }; + for (unsigned int index = 0; index < threads; ++index) { + topology.allowed_cpu_ids[index] = allowed_ids[index]; + } + Lardon3DTaskQueue *queue = NULL; + if (!governor || + !lardon3d_resource_governor_internal_configure_cpu_topology( + governor, &topology) || + !(queue = lardon3d_task_queue_create(governor, TEST_EXTRACT_COUNT))) { + lardon3d_task_queue_destroy(queue); + lardon3d_resource_governor_destroy(governor); + ok = false; + break; + } + AdaptiveExtractWork work[TEST_EXTRACT_COUNT] = {0}; + uint64_t ids[TEST_EXTRACT_COUNT] = {0}; + Lardon3DResourceEstimate estimate = { + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 12, + .task_class = LARDON3D_RESOURCE_TASK_CPU, + }; + static const char *const task_kinds[TEST_EXTRACT_COUNT] = { + "features.extract", + "features.extract.sift", + "features.extract.rootsift", + }; + for (size_t kind = 0; kind < TEST_EXTRACT_COUNT && ok; ++kind) { + work[kind] = (AdaptiveExtractWork) { + .path = path, + .expected_threads = threads, + .kind = (TestExtractKind)kind, + }; + Lardon3DTask *task = lardon3d_task_create_typed( + "Extraction adaptative", &estimate, task_kinds[kind], 1, + adaptive_extract_callback, &work[kind], NULL); + Lardon3DTaskCapabilityEnvelope envelope = { + .count = 1, + .capabilities = {{ + .estimate = estimate, + .backend = LARDON3D_RESOURCE_BACKEND_FIXED, + .inflight_limit = 1, + }}, + }; + envelope.capabilities[0].estimate.desired_cpu_threads = threads; + /* Scientific equality is tested at exact admitted counts independently + * of feedback timing; Governor progression/deadband has its own fully + * deterministic rate-injection regression. */ + ok = task && lardon3d_task_internal_set_capability_envelope( + task, &envelope) && lardon3d_task_queue_add(queue, task, &ids[kind]); + if (!ok && task) lardon3d_task_destroy(task); + } + Lardon3DTaskSnapshot snapshot; + for (size_t kind = 0; kind < TEST_EXTRACT_COUNT && ok; ++kind) { + ok = wait_state(queue, ids[kind], TASK_COMPLETED, &snapshot); + if (!ok) break; + if (!have_baseline[kind]) { + baseline[kind] = work[kind].output; + work[kind].output = (Lardon3DExtractedFeatures) {0}; + have_baseline[kind] = true; + } else { + ok = extracted_equal(&baseline[kind], &work[kind].output); + } + } + for (size_t kind = 0; kind < TEST_EXTRACT_COUNT; ++kind) { + lardon3d_extracted_features_destroy(&work[kind].output); + } + lardon3d_task_queue_destroy(queue); + lardon3d_resource_governor_destroy(governor); + } + for (size_t kind = 0; kind < TEST_EXTRACT_COUNT; ++kind) { + lardon3d_extracted_features_destroy(&baseline[kind]); + } + return lardon3d_feature_opencv_configure_threads(previous_threads) && ok; +} static bool runtime(Lardon3DAppState *s) { s->hardware_profile = (Lardon3DHardwareProfile){.logical_cpu_count = 64, .page_size_bytes = 4096, @@ -110,7 +308,10 @@ static bool wait_state(Lardon3DTaskQueue *q, uint64_t id, Lardon3DTaskState want Lardon3DTaskSnapshot *out) { struct timespec deadline; if (clock_gettime(CLOCK_MONOTONIC, &deadline) != 0) return false; - deadline.tv_sec += 5; + /* The equality fixture deliberately executes ORB/SIFT/RootSIFT at five CPU + * contracts. Instrumented OpenCV is slower than the normal build; keep the + * wait bounded without mistaking sanitizer overhead for scientific drift. */ + deadline.tv_sec += 30; for (;;) { if (lardon3d_task_queue_get(q, id, out) && out->state == wanted) { return true; @@ -124,6 +325,16 @@ static bool wait_state(Lardon3DTaskQueue *q, uint64_t id, Lardon3DTaskState want } } +static bool last_sequence_is_no_work(Lardon3DAppState *state, const char *task_kind) { + Lardon3DResourceSequenceDiagnostic diagnostic; + return lardon3d_resource_governor_internal_last_diagnostic( + state->resource_governor, task_kind, 1, &diagnostic) && + diagnostic.items_completed == 0 && + diagnostic.durable_items_per_second_milli == 0 && + diagnostic.cpu_threads == 1 && + strcmp(diagnostic.reason, "throughput-no-work") == 0; +} + typedef struct { Lardon3DAppState *state; uint64_t orb_feature_set_id; @@ -146,9 +357,11 @@ static bool run_test(void) { CHECK(mkdtemp(root) && setenv("LARDON3D_PROJECTS_ROOT", root, 1) == 0); char source[PATH_MAX]; CHECK(join_path(source, root, "source.pgm") && write_pgm(source)); + CHECK(run_adaptive_output_equivalence(source)); Lardon3DAppState state; lardon3d_app_state_init(&state); CHECK(runtime(&state) && lardon3d_project_create(&state, "Features")); + CHECK(lardon3d_feature_opencv_configure_threads(3)); Lardon3DProjectDbScanSet scanset; CHECK(lardon3d_image_catalog_create_scanset(&state, "A", &scanset)); Lardon3DProjectDbImage image; @@ -164,6 +377,7 @@ static bool run_test(void) { CHECK(wait_state(state.task_queue, task_id, TASK_PAUSED, &snapshot)); lardon3d_task_queue_destroy(state.task_queue); state.task_queue = NULL; + CHECK(lardon3d_feature_opencv_thread_count() == 3); lardon3d_project_close(&state); lardon3d_resource_governor_destroy(state.resource_governor); state.resource_governor = NULL; @@ -171,10 +385,12 @@ static bool run_test(void) { unsetenv("LARDON3D_TEST_FEATURE_SKIP_FINISHED_CHECKPOINT") == 0); lardon3d_app_state_init(&state); CHECK(runtime(&state) && lardon3d_project_open(&state, "Features")); + CHECK(lardon3d_feature_opencv_configure_threads(3)); Lardon3DProjectRecoverySummary summary; CHECK(lardon3d_project_last_recovery_summary(&state, &summary) && summary.resumed == 1); CHECK(wait_state(state.task_queue, task_id, TASK_COMPLETED, &snapshot) && snapshot.progress == 100); + CHECK(lardon3d_feature_opencv_thread_count() == 3); unsigned char fp[32]; lardon3d_feature_extractor_parameter_fingerprint(¶meters, fp); Lardon3DProjectDbFeatureSet set; @@ -198,6 +414,60 @@ static bool run_test(void) { LARDON3D_FEATURE_STORE_OK); lardon3d_feature_reader_close(reader); + uint64_t duplicate_orb_task = 0; + CHECK(lardon3d_project_enqueue_feature_extract( + &state, image.image_id, ¶meters, &duplicate_orb_task) && + wait_state(state.task_queue, duplicate_orb_task, TASK_COMPLETED, &snapshot) && + last_sequence_is_no_work(&state, LARDON3D_FEATURE_EXTRACT_TASK_KIND)); + + /* The asset and DB row are visible, but forced directory-sync uncertainty + * means this extraction is not durable throughput evidence. */ + Lardon3DFeatureExtractorParameters nondurable_orb_parameters = {508, 4, 10}; + uint64_t nondurable_orb_task = 0; + CHECK(setenv("LARDON3D_TEST_FEATURE_FAIL_DIRECTORY_SYNC", "1", 1) == 0 && + lardon3d_project_enqueue_feature_extract( + &state, image.image_id, &nondurable_orb_parameters, + &nondurable_orb_task) && + wait_state(state.task_queue, nondurable_orb_task, TASK_COMPLETED, &snapshot) && + last_sequence_is_no_work(&state, LARDON3D_FEATURE_EXTRACT_TASK_KIND) && + unsetenv("LARDON3D_TEST_FEATURE_FAIL_DIRECTORY_SYNC") == 0); + uint64_t post_no_work_orb_task = 0; + CHECK(lardon3d_project_enqueue_feature_extract( + &state, image.image_id, ¶meters, &post_no_work_orb_task) && + wait_state(state.task_queue, post_no_work_orb_task, TASK_COMPLETED, + &snapshot) && + last_sequence_is_no_work(&state, LARDON3D_FEATURE_EXTRACT_TASK_KIND)); + + /* Configuration failure is injected after OpenCV mutates its process-wide + * pool. Feature, SIFT, and RootSIFT must each restore the captured value. */ + CHECK(lardon3d_feature_opencv_configure_threads(3)); + CHECK(setenv("LARDON3D_TEST_OPENCV_CONFIGURE_FAILURE_THREADS", "1", 1) == 0); + Lardon3DFeatureExtractorParameters failed_orb_parameters = {509, 4, 10}; + uint64_t failed_orb_task = 0; + CHECK(lardon3d_project_enqueue_feature_extract( + &state, image.image_id, &failed_orb_parameters, &failed_orb_task) && + wait_state(state.task_queue, failed_orb_task, TASK_FAILED, &snapshot) && + lardon3d_feature_opencv_thread_count() == 3); + + Lardon3DSiftExtractorParameters failed_sift_parameters = + lardon3d_sift_precision_classic_v1(false); + failed_sift_parameters.max_features = 511; + uint64_t failed_sift_task = 0; + CHECK(lardon3d_project_enqueue_sift_extract( + &state, image.image_id, &failed_sift_parameters, + &failed_sift_task) && + wait_state(state.task_queue, failed_sift_task, TASK_FAILED, &snapshot) && + lardon3d_feature_opencv_thread_count() == 3); + failed_sift_parameters.rootsift = true; + failed_sift_parameters.max_features = 510; + uint64_t failed_rootsift_task = 0; + CHECK(lardon3d_project_enqueue_sift_extract( + &state, image.image_id, &failed_sift_parameters, + &failed_rootsift_task) && + wait_state(state.task_queue, failed_rootsift_task, TASK_FAILED, &snapshot) && + lardon3d_feature_opencv_thread_count() == 3 && + unsetenv("LARDON3D_TEST_OPENCV_CONFIGURE_FAILURE_THREADS") == 0); + Lardon3DSiftExtractorParameters sift_parameters = lardon3d_sift_precision_classic_v1(false); sift_parameters.max_features = 512; uint64_t sift_task_id = 0, concurrent_identical_sift_task = 0; @@ -207,7 +477,8 @@ static bool run_test(void) { &concurrent_identical_sift_task)); CHECK(sift_task_id != concurrent_identical_sift_task && wait_state(state.task_queue, sift_task_id, TASK_COMPLETED, &snapshot) && - wait_state(state.task_queue, concurrent_identical_sift_task, TASK_COMPLETED, &snapshot)); + wait_state(state.task_queue, concurrent_identical_sift_task, TASK_COMPLETED, &snapshot) && + last_sequence_is_no_work(&state, LARDON3D_SIFT_EXTRACT_TASK_KIND)); unsigned char sift_fingerprint[32]; lardon3d_sift_extractor_parameter_fingerprint(&sift_parameters, sift_fingerprint); unsigned char same_sift_fingerprint[32], changed_sift_fingerprint[32]; @@ -281,6 +552,31 @@ static bool run_test(void) { rootsift_fingerprint, &rootsift_set) == LARDON3D_PROJECT_DB_OK && rootsift_set.feature_set_id != sift_set.feature_set_id); + uint64_t duplicate_rootsift_task = 0; + CHECK(lardon3d_project_enqueue_sift_extract( + &state, image.image_id, &rootsift_parameters, + &duplicate_rootsift_task) && + wait_state(state.task_queue, duplicate_rootsift_task, TASK_COMPLETED, &snapshot) && + last_sequence_is_no_work(&state, LARDON3D_ROOTSIFT_EXTRACT_TASK_KIND)); + + Lardon3DSiftExtractorParameters nondurable_sift_parameters = sift_parameters; + nondurable_sift_parameters.max_features = 508; + uint64_t nondurable_sift_task = 0; + CHECK(setenv("LARDON3D_TEST_FEATURE_FAIL_DIRECTORY_SYNC", "1", 1) == 0 && + lardon3d_project_enqueue_sift_extract( + &state, image.image_id, &nondurable_sift_parameters, + &nondurable_sift_task) && + wait_state(state.task_queue, nondurable_sift_task, TASK_COMPLETED, &snapshot) && + last_sequence_is_no_work(&state, LARDON3D_SIFT_EXTRACT_TASK_KIND)); + nondurable_sift_parameters.rootsift = true; + nondurable_sift_parameters.max_features = 509; + uint64_t nondurable_rootsift_task = 0; + CHECK(lardon3d_project_enqueue_sift_extract( + &state, image.image_id, &nondurable_sift_parameters, + &nondurable_rootsift_task) && + wait_state(state.task_queue, nondurable_rootsift_task, TASK_COMPLETED, &snapshot) && + last_sequence_is_no_work(&state, LARDON3D_ROOTSIFT_EXTRACT_TASK_KIND) && + unsetenv("LARDON3D_TEST_FEATURE_FAIL_DIRECTORY_SYNC") == 0); /* SIFT and RootSIFT CPU1 checkpoints are exact historical operational * signatures. Recovery uses CPU12 in memory without publishing an * estimate-only checkpoint; a neighboring CPU2 shape is corruption. */ @@ -507,6 +803,26 @@ static bool run_test(void) { lardon3d_feature_reader_close(reader); Lardon3DVisualIndexConfiguration index_configuration = {1, 256, 128}; + Lardon3DVisualIndexConfiguration nondurable_index_configuration = {1, 255, 128}; + uint64_t nondurable_visual_index_id = 0; + uint64_t nondurable_visual_task_id = 0; + CHECK(lardon3d_visual_index_create( + state.project_db, &set, &nondurable_index_configuration, + &nondurable_visual_index_id) == LARDON3D_VISUAL_INDEX_OK); + CHECK(setenv("LARDON3D_TEST_VISUAL_INDEX_FAIL_DIR_FSYNC", "1", 1) == 0 && + setenv("LARDON3D_TEST_VISUAL_INDEX_PAUSE_AFTER_SEGMENT", "1", 1) == 0); + CHECK(lardon3d_project_enqueue_visual_index_update( + &state, nondurable_visual_index_id, &nondurable_visual_task_id)); + CHECK(wait_state(state.task_queue, nondurable_visual_task_id, TASK_PAUSED, + &snapshot)); + CHECK(last_sequence_is_no_work( + &state, LARDON3D_VISUAL_INDEX_UPDATE_TASK_KIND)); + CHECK(unsetenv("LARDON3D_TEST_VISUAL_INDEX_FAIL_DIR_FSYNC") == 0 && + unsetenv("LARDON3D_TEST_VISUAL_INDEX_PAUSE_AFTER_SEGMENT") == 0 && + lardon3d_task_queue_resume( + state.task_queue, nondurable_visual_task_id) && + wait_state(state.task_queue, nondurable_visual_task_id, TASK_COMPLETED, + &snapshot)); uint64_t visual_index_id = 0; CHECK(lardon3d_visual_index_create(state.project_db, &set, &index_configuration, &visual_index_id) == LARDON3D_VISUAL_INDEX_OK); diff --git a/tests/test_hardware_profile.c b/tests/test_hardware_profile.c index c1a5dfa..f675ae6 100644 --- a/tests/test_hardware_profile.c +++ b/tests/test_hardware_profile.c @@ -1,10 +1,17 @@ +#include +#include +#include #include #include #include #include +#include +#include #include +#include "../src/hardware_profile_internal.h" + #define CHECK(condition) \ do { \ if (!(condition)) { \ @@ -13,6 +20,138 @@ } \ } while (0) +#define GIBIBYTES(value) ((uint64_t)(value) * 1024 * 1024 * 1024) + +static bool +write_text(const char *path, const char *text) +{ + int descriptor = open(path, O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, + 0600); + if (descriptor < 0) return false; + size_t length = strlen(text); + ssize_t written = write(descriptor, text, length); + return close(descriptor) == 0 && written == (ssize_t)length; +} + +static bool +read_exact_text(const char *path, const char *expected) +{ + char buffer[64]; + int descriptor = open(path, O_RDONLY | O_CLOEXEC | O_NOFOLLOW); + if (descriptor < 0) return false; + ssize_t count = read(descriptor, buffer, sizeof(buffer)); + bool eof = count >= 0 && count < (ssize_t)sizeof(buffer) + && read(descriptor, buffer + count, 1) == 0; + bool equal = eof && (size_t)count == strlen(expected) + && memcmp(buffer, expected, (size_t)count) == 0; + return close(descriptor) == 0 && equal; +} + +static bool +card_path(char path[PATH_MAX], const char *root, unsigned int index, + const char *name) +{ + int written = snprintf(path, PATH_MAX, "%s/card%u/device/%s", root, + index, name); + return written > 0 && written < PATH_MAX; +} + +static bool +create_card(const char *root, unsigned int index, const char *vram, + const char *gtt) +{ + char card[PATH_MAX]; + char device[PATH_MAX]; + int card_written = snprintf(card, sizeof(card), "%s/card%u", root, index); + int device_written = snprintf(device, sizeof(device), "%s/device", card); + if (card_written <= 0 || (size_t)card_written >= sizeof(card) + || device_written <= 0 || (size_t)device_written >= sizeof(device) + || mkdir(card, 0700) != 0 || mkdir(device, 0700) != 0) { + return false; + } + char path[PATH_MAX]; + return card_path(path, root, index, "vendor") + && write_text(path, "0x1002\n") + && card_path(path, root, index, "mem_info_vram_total") + && write_text(path, vram) + && (!gtt || (card_path(path, root, index, "mem_info_gtt_total") + && write_text(path, gtt))); +} + +static bool +remove_card(const char *root, unsigned int index) +{ + char path[PATH_MAX]; + const char *files[] = { + "vendor", "mem_info_vram_total", "mem_info_gtt_total", + }; + for (size_t file = 0; file < sizeof(files) / sizeof(files[0]); ++file) { + if (!card_path(path, root, index, files[file])) return false; + if (unlink(path) != 0 && errno != ENOENT) return false; + } + int written = snprintf(path, sizeof(path), "%s/card%u/device", root, + index); + if (written <= 0 || (size_t)written >= sizeof(path) || rmdir(path) != 0) + return false; + written = snprintf(path, sizeof(path), "%s/card%u", root, index); + return written > 0 && (size_t)written < sizeof(path) && rmdir(path) == 0; +} + +static bool +run_fake_gpu_test(void) +{ + char root[] = "/tmp/lardon3d-hardware-profile-XXXXXX"; + CHECK(mkdtemp(root)); + Lardon3DHardwareProfile profile = { + .memory_total_bytes = GIBIBYTES(16), + }; + + /* Exact current-host evidence: the small 512 MiB aperture and system-scale + * GTT are UMA, while the payload capacity remains observable. */ + CHECK(create_card(root, 1, "536870912\n", "7986020352\n")); + lardon3d_hardware_profile_detect_gpu_at_root(&profile, root); + CHECK(profile.gpu_available && profile.gpu_drm_card_index == 1 + && profile.gpu_memory_known + && profile.gpu_memory_total_bytes == UINT64_C(536870912) + && profile.gpu_uses_shared_memory); + CHECK(remove_card(root, 1)); + + CHECK(create_card(root, 0, "8589934592\n", "8589934592\n")); + lardon3d_hardware_profile_detect_gpu_at_root(&profile, root); + CHECK(profile.gpu_available && profile.gpu_memory_known + && profile.gpu_memory_total_bytes == GIBIBYTES(8) + && !profile.gpu_uses_shared_memory); + CHECK(remove_card(root, 0)); + + /* Missing GTT with a low aperture is intentionally conservative; malformed + * GTT cannot turn a large known VRAM device into UMA. */ + CHECK(create_card(root, 2, "536870912\n", NULL)); + lardon3d_hardware_profile_detect_gpu_at_root(&profile, root); + CHECK(profile.gpu_memory_known && profile.gpu_uses_shared_memory); + CHECK(remove_card(root, 2)); + CHECK(create_card(root, 3, "8589934592\n", "-1\n")); + lardon3d_hardware_profile_detect_gpu_at_root(&profile, root); + CHECK(profile.gpu_memory_known && !profile.gpu_uses_shared_memory); + CHECK(remove_card(root, 3)); + + CHECK(create_card(root, 4, "+536870912\n", "7986020352\n")); + lardon3d_hardware_profile_detect_gpu_at_root(&profile, root); + CHECK(profile.gpu_available && !profile.gpu_memory_known + && profile.gpu_uses_shared_memory); + CHECK(remove_card(root, 4)); + + profile.gpu_available = true; + profile.gpu_memory_known = true; + profile.gpu_uses_shared_memory = true; + profile.gpu_memory_total_bytes = 1; + lardon3d_hardware_profile_detect_gpu_at_root(&profile, root); + CHECK(!profile.gpu_available && !profile.gpu_memory_known + && !profile.gpu_uses_shared_memory + && profile.gpu_memory_total_bytes == 0 && profile.gpu_name[0] == '\0'); + CHECK(rmdir(root) == 0); + return true; +} + static bool run_test(void) { @@ -34,10 +173,25 @@ run_test(void) CHECK(profile.gpu_name[0]); CHECK(profile.gpu_drm_card_index < 64); } + /* Skip-safe current-host integration evidence. Exact sysfs values identify + * the validated 780M configuration without making its PCI device ID part + * of production policy or a portable test requirement. */ + if (read_exact_text("/sys/class/drm/card1/device/vendor", "0x1002\n") + && read_exact_text("/sys/class/drm/card1/device/device", "0x1900\n") + && read_exact_text("/sys/class/drm/card1/device/mem_info_vram_total", + "536870912\n") + && read_exact_text("/sys/class/drm/card1/device/mem_info_gtt_total", + "7986020352\n")) { + CHECK(profile.gpu_available && profile.gpu_drm_card_index == 1 + && profile.gpu_memory_known + && profile.gpu_memory_total_bytes == UINT64_C(536870912) + && profile.gpu_uses_shared_memory); + } Lardon3DHardwareProfile second; CHECK(lardon3d_hardware_profile_detect(&second, NULL, 0)); CHECK(second.logical_cpu_count == profile.logical_cpu_count); CHECK(second.memory_total_bytes == profile.memory_total_bytes); + CHECK(run_fake_gpu_test()); return true; } diff --git a/tests/test_matcher.c b/tests/test_matcher.c index 761d19f..58587be 100644 --- a/tests/test_matcher.c +++ b/tests/test_matcher.c @@ -11,6 +11,9 @@ #include #include +#include + +#include "../src/matcher_internal.h" #define CHECK(condition) \ do { \ @@ -495,6 +498,35 @@ static bool test_matcher_default_ratio(void) { return true; } +static bool test_matcher_private_path_bound(void) { + char project_path[4097]; + memset(project_path, 'x', sizeof(project_path) - 1); + project_path[sizeof(project_path) - 1] = '\0'; + Lardon3DProjectDbFeatureSet feature_set = { + .feature_set_id = 1, + .feature_count = 769, + .descriptor_type = LARDON3D_FEATURE_DESCRIPTOR_U8, + .descriptor_dimension = 32, + }; + Lardon3DMatcherParams parameters = { + .kind = LARDON3D_MATCHER_ORB_BF, + .ratio_threshold = 0.75F, + }; + Lardon3DOrbVulkanBackend *backend = lardon3d_orb_vulkan_backend_create(); + Lardon3DMatcherPendingVulkanStage *pending = + (Lardon3DMatcherPendingVulkanStage *)(uintptr_t)1; + bool backend_fault = true; + CHECK(backend); + Lardon3DMatcherResult expected = lardon3d_orb_vulkan_should_use(769, 769) + ? LARDON3D_MATCHER_IO_ERROR : LARDON3D_MATCHER_INVALID_ARGUMENT; + CHECK(lardon3d_matcher_begin_vulkan_stage( + project_path, &feature_set, &feature_set, ¶meters, backend, + &pending, &backend_fault) == expected && + pending == NULL && !backend_fault); + lardon3d_orb_vulkan_backend_destroy(backend); + return true; +} + static bool run_tests(void) { CHECK(test_match_file_write_read()); CHECK(test_match_file_empty()); @@ -510,6 +542,7 @@ static bool run_tests(void) { CHECK(test_matcher_kind_string()); CHECK(test_matcher_fingerprint()); CHECK(test_matcher_default_ratio()); + CHECK(test_matcher_private_path_bound()); return true; } diff --git a/tests/test_matcher_e2e.c b/tests/test_matcher_e2e.c index cf8f6d1..f87f17f 100644 --- a/tests/test_matcher_e2e.c +++ b/tests/test_matcher_e2e.c @@ -179,6 +179,7 @@ static bool read_result_asset(const Fixture *fixture, const Lardon3DProjectDbFea return read_result == LARDON3D_MATCH_FILE_OK; } +#ifdef LARDON3D_MATCHER_E2E_VULKAN static bool files_equal(const char *path_a, const char *path_b) { int fd_a = open(path_a, O_RDONLY | O_CLOEXEC); int fd_b = open(path_b, O_RDONLY | O_CLOEXEC); @@ -235,7 +236,6 @@ static uint32_t deterministic_random(uint32_t *state) { return *state; } -#ifdef LARDON3D_MATCHER_E2E_VULKAN static bool test_cpu_vulkan_match_file_parity(void) { const uint32_t feature_count = 768; const size_t bytes = (size_t)feature_count * 32; diff --git a/tests/test_matcher_task.c b/tests/test_matcher_task.c index b28f98f..28fcd84 100644 --- a/tests/test_matcher_task.c +++ b/tests/test_matcher_task.c @@ -1,5 +1,10 @@ +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif + #include #include +#include #include #include #include @@ -9,14 +14,23 @@ #include #include #include +#ifdef __linux__ +#include +#endif #include +#include #include +#include #include #include #include #include +#include "../src/matcher_task_benchmark_internal.h" +#include "../src/orb_vulkan_backend_internal.h" +#include "../src/resource_governor_internal.h" + #define CHECK(condition) \ do { \ if (!(condition)) { \ @@ -29,11 +43,30 @@ enum { IMAGE_COUNT = 42, PAIR_COUNT = IMAGE_COUNT - 1, PERSISTED_PAIR_COUNT = PAIR_COUNT - 1, + /* The fixture makes twelve adjacent pairs eligible, then deliberately + * deletes eligible pair index two to preserve sparse Candidate Pair IDs. */ + PIPELINE_ELIGIBLE_PAIR_COUNT = 11, }; void lardon3d_matcher_task_test_reset_backend_counters(void); size_t lardon3d_matcher_task_test_vulkan_uses(void); size_t lardon3d_matcher_task_test_forced_fallbacks(void); +size_t lardon3d_matcher_task_test_overlap_publications(void); +uint64_t lardon3d_matcher_task_test_max_retained_vulkan_payload(void); +size_t lardon3d_matcher_task_test_event_count(void); +bool lardon3d_matcher_task_test_event(size_t index, int *kind, + uint64_t *candidate_pair_id, + size_t *order); +bool lardon3d_matcher_task_test_auto_capability_envelope( + size_t benchmark_inflight_override, size_t benchmark_batch_override, + Lardon3DTaskCapabilityEnvelope *envelope); + +enum { + TEST_EVENT_GPU_SUBMIT = 1, + TEST_EVENT_GPU_FINISH = 2, + TEST_EVENT_PUBLICATION_START = 3, + TEST_EVENT_PUBLICATION_FINISH = 4, +}; typedef struct { char root[PATH_MAX]; @@ -58,6 +91,160 @@ static Lardon3DResourcePolicy interactive_policy(void) { }; } +#ifdef LARDON3D_MATCHER_TASK_VULKAN +static bool matcher_capability_boundary_case( + const Lardon3DTaskCapabilityEnvelope *envelope, uint64_t total_bytes, + uint64_t available_bytes, Lardon3DResourceDecisionKind expected_decision, + Lardon3DResourceBackend expected_backend, size_t expected_batch, + uint64_t expected_memory_bytes, uint64_t expected_gpu_memory_bytes) { + const uint64_t gib = UINT64_C(1024) * 1024 * 1024; + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 16, + .page_size_bytes = 4096, + .memory_total_bytes = total_bytes, + .gpu_available = true, + .gpu_uses_shared_memory = true, + .cpu_architecture = "boundary-test", + }; + Lardon3DResourcePolicy policy = { + .system_memory_reserve_bytes = 3 * gib, + .emergency_memory_floor_bytes = 2 * gib, + .system_cpu_reserve = 0, + .maximum_cpu_load_ratio = 1.0, + .maximum_cpu_pressure_avg10 = 100.0, + .maximum_memory_pressure_avg10 = 100.0, + .maximum_io_pressure_avg10 = 100.0, + .gpu_slot_capacity = 1, + .io_slot_capacity = 1, + }; + Lardon3DResourceGovernor *governor = + lardon3d_resource_governor_create(&profile, &policy); + if (!governor || + !lardon3d_resource_governor_internal_set_backend_available( + governor, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, true)) { + lardon3d_resource_governor_destroy(governor); + return false; + } + Lardon3DResourceSnapshot snapshot = { + .memory_available_bytes = available_bytes, + .swap_activity_known = true, + }; + if (clock_gettime(CLOCK_MONOTONIC, &snapshot.captured_at) != 0) { + lardon3d_resource_governor_destroy(governor); + return false; + } + Lardon3DResourceCapabilitySelection selection = {0}; + Lardon3DResourceReservation *reservation = NULL; + bool success = lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, envelope, &selection, &reservation); + bool admitting = expected_decision == LARDON3D_RESOURCE_START || + expected_decision == LARDON3D_RESOURCE_REDUCE_BATCH; + success = success && selection.decision.kind == expected_decision && + ((admitting && reservation != NULL && + selection.capability.backend == expected_backend && + selection.decision.batch_size == expected_batch) || + (!admitting && reservation == NULL)); + if (!success) { + fprintf(stderr, + "boundary actual decision=%d backend=%d batch=%zu reservation=%s; " + "expected decision=%d backend=%d batch=%zu\n", + (int)selection.decision.kind, (int)selection.capability.backend, + selection.decision.batch_size, reservation ? "yes" : "no", + (int)expected_decision, (int)expected_backend, expected_batch); + } + if (success && reservation) { + Lardon3DResourceReservationInfo information; + success = lardon3d_resource_reservation_get_active( + governor, reservation, &information) && + information.memory_bytes == expected_memory_bytes && + information.gpu_memory_bytes == expected_gpu_memory_bytes && + lardon3d_resource_governor_release(governor, reservation); + reservation = NULL; + } + if (reservation) { + (void)lardon3d_resource_governor_release(governor, reservation); + } + lardon3d_resource_governor_destroy(governor); + return success; +} +#endif + +static bool matcher_exact_memory_boundary_test(void) { +#ifdef LARDON3D_MATCHER_TASK_VULKAN + const uint64_t gib = UINT64_C(1024) * 1024 * 1024; + const uint64_t per_pair = UINT64_C(10) * 1024 * 1024; + const uint64_t per_slot = UINT64_C(640) * 1024; + const uint64_t normal_window = 8 * per_pair + per_slot; + const uint64_t normal_total = 3 * gib + normal_window; + const uint64_t normal_gpu_minimum = 3 * gib + per_pair + per_slot; + const uint64_t normal_cpu_minimum = 3 * gib + per_pair; + Lardon3DTaskCapabilityEnvelope normal; + CHECK(lardon3d_matcher_task_test_auto_capability_envelope(0, 0, &normal)); + CHECK(normal.count == 2 && + normal.capabilities[0].backend == + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + normal.capabilities[0].estimate.maximum_batch_size == 8 && + normal.capabilities[0].sustained_gpu_batch_feedback && + normal.capabilities[0].inflight_limit == 1 && + normal.capabilities[0].minimum_inflight_limit == 1 && + normal.capabilities[0].gpu_memory_bytes_per_inflight == per_slot && + normal_total < 3 * gib + 12 * per_pair + 2 * per_slot); + /* The exact maximum proves that the old depth-two/batch-twelve prefilter is + * gone. Falling below that maximum must not reject an adaptive capability: + * AUTO can still admit the immutable batch-one contract. Only the exact + * minimum boundaries select CPU fallback and finally reject both arms. */ + CHECK(matcher_capability_boundary_case( + &normal, normal_total, normal_total, LARDON3D_RESOURCE_START, + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, 1, per_pair, per_slot)); + CHECK(matcher_capability_boundary_case( + &normal, normal_total - 1, normal_total - 1, + LARDON3D_RESOURCE_START, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, 1, + per_pair, per_slot)); + CHECK(matcher_capability_boundary_case( + &normal, normal_gpu_minimum - 1, normal_gpu_minimum - 1, + LARDON3D_RESOURCE_START, LARDON3D_RESOURCE_BACKEND_CPU, 1, per_pair, 0)); + CHECK(matcher_capability_boundary_case( + &normal, normal_cpu_minimum - 1, normal_cpu_minimum - 1, + LARDON3D_RESOURCE_REJECT, LARDON3D_RESOURCE_BACKEND_FIXED, 0, 0, 0)); +#if defined(LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE) + const uint64_t forced_window = 2 * per_pair + per_slot; + const uint64_t forced_total = 3 * gib + forced_window; + Lardon3DTaskCapabilityEnvelope forced; + CHECK(lardon3d_matcher_task_test_auto_capability_envelope(1, 2, &forced)); + CHECK(forced.count == 1 && + forced.capabilities[0].backend == + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + forced.capabilities[0].estimate.minimum_batch_size == 2 && + forced.capabilities[0].estimate.maximum_batch_size == 2 && + !forced.capabilities[0].sustained_gpu_batch_feedback && + forced.capabilities[0].inflight_limit == 1 && + forced.capabilities[0].minimum_inflight_limit == 1 && + forced_total < 3 * gib + 12 * per_pair + 2 * per_slot); + CHECK(matcher_capability_boundary_case( + &forced, forced_total, forced_total, LARDON3D_RESOURCE_START, + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, 2, 2 * per_pair, per_slot)); + CHECK(matcher_capability_boundary_case( + &forced, forced_total - 1, forced_total - 1, + LARDON3D_RESOURCE_REJECT, LARDON3D_RESOURCE_BACKEND_FIXED, 0, 0, 0)); + /* Exact sizing never weakens the 2 GiB hard floor: an otherwise valid + * forced capability waits and owns no reservation at that current snapshot. */ + CHECK(matcher_capability_boundary_case( + &forced, forced_total, 2 * gib, LARDON3D_RESOURCE_WAIT, + LARDON3D_RESOURCE_BACKEND_FIXED, 0, 0, 0)); + Lardon3DTaskCapabilityEnvelope forced_twelve; + CHECK(lardon3d_matcher_task_test_auto_capability_envelope( + 1, 12, &forced_twelve) && + forced_twelve.count == 1 && + forced_twelve.capabilities[0].estimate.minimum_batch_size == 12 && + forced_twelve.capabilities[0].estimate.maximum_batch_size == 12 && + !forced_twelve.capabilities[0].batch_adaptive && + !forced_twelve.capabilities[0].sustained_gpu_batch_feedback); +#endif +#endif + return true; +} + static bool join_path(char output[PATH_MAX], const char *left, const char *right) { int written = snprintf(output, PATH_MAX, "%s/%s", left, right); @@ -97,6 +284,71 @@ static bool candidate_results_have_one_evidence(const char *path, query_integer(path, sql, 0); } +#ifdef LARDON3D_MATCHER_TASK_VULKAN +static bool wait_candidate_results_have_one_evidence( + const char *path, uint64_t candidate_pair_id) { + /* Queue state may become terminal immediately before its finished callback + * releases the DB transaction. Retry the observable read boundary without + * timing sleeps; a real duplicate never converges to one scientific value. */ + for (size_t attempt = 0; attempt < 2000000; ++attempt) { + if (candidate_results_have_one_evidence(path, candidate_pair_id)) { + return true; + } + sched_yield(); + } + return false; +} + +static bool find_test_event(int wanted_kind, uint64_t wanted_pair, + size_t *order) { + if (!order) return false; + size_t count = lardon3d_matcher_task_test_event_count(); + for (size_t index = 0; index < count; ++index) { + int kind = 0; + uint64_t pair = 0; + size_t candidate_order = 0; + if (lardon3d_matcher_task_test_event(index, &kind, &pair, + &candidate_order) && + kind == wanted_kind && pair == wanted_pair) { + *order = candidate_order; + return true; + } + } + return false; +} + +static bool staged_match_temporaries_absent_once(const Fixture *fixture) { + char assets[PATH_MAX]; + char matches[PATH_MAX]; + if (!join_path(assets, fixture->state.project_path, "assets") || + !join_path(matches, assets, "matches")) { + return false; + } + DIR *directory = opendir(matches); + if (!directory) return false; + bool clean = true; + for (struct dirent *entry = readdir(directory); entry; + entry = readdir(directory)) { + if (strncmp(entry->d_name, ".match-", 7) == 0) { + clean = false; + break; + } + } + return closedir(directory) == 0 && clean; +} + +static bool no_staged_match_temporaries(const Fixture *fixture) { + /* Task state may become FAILED before the callback unwinds its private stage + * cleanup. Poll the observable filesystem boundary without sleeps; Queue's + * single owner must converge before another Task can consume the slot. */ + for (size_t attempt = 0; attempt < 2000000; ++attempt) { + if (staged_match_temporaries_absent_once(fixture)) return true; + sched_yield(); + } + return false; +} +#endif + static bool downgrade_project_to_historical_v10(const char *database_path) { sqlite3 *connection = NULL; if (sqlite3_open_v2(database_path, &connection, SQLITE_OPEN_READWRITE, @@ -188,28 +440,47 @@ static bool create_runtime(Lardon3DAppState *state) { static bool wait_state(Lardon3DTaskQueue *queue, uint64_t task_id, Lardon3DTaskState wanted, Lardon3DTaskSnapshot *snapshot) { - for (size_t attempt = 0; attempt < 2000000; ++attempt) { + struct timespec deadline; + if (clock_gettime(CLOCK_MONOTONIC, &deadline) != 0) return false; + /* ASan/UBSan intentionally makes the real Vulkan fixture much slower. A + * wall-clock bound tests eventual Queue state without turning main-thread + * polling speed into an accidental timeout contract. */ + deadline.tv_sec += 30; + for (;;) { if (lardon3d_task_queue_get(queue, task_id, snapshot) && snapshot->state == wanted) { return true; } + struct timespec now; + if (clock_gettime(CLOCK_MONOTONIC, &now) != 0 || + now.tv_sec > deadline.tv_sec || + (now.tv_sec == deadline.tv_sec && now.tv_nsec >= deadline.tv_nsec)) { + return false; + } sched_yield(); } - return false; } static bool wait_durable_state(Lardon3DProjectDb *database, uint64_t task_id, Lardon3DTaskState wanted, Lardon3DProjectDbTask *durable_task) { - for (size_t attempt = 0; attempt < 2000000; ++attempt) { + struct timespec deadline; + if (clock_gettime(CLOCK_MONOTONIC, &deadline) != 0) return false; + deadline.tv_sec += 30; + for (;;) { if (lardon3d_project_db_load_task(database, task_id, durable_task) == LARDON3D_PROJECT_DB_OK && durable_task->saved_state == wanted) { return true; } + struct timespec now; + if (clock_gettime(CLOCK_MONOTONIC, &now) != 0 || + now.tv_sec > deadline.tv_sec || + (now.tv_sec == deadline.tv_sec && now.tv_nsec >= deadline.tv_nsec)) { + return false; + } sched_yield(); } - return false; } static void image_asset_path(const unsigned char hash[32], char path[4096]) { @@ -236,7 +507,10 @@ static bool register_image(Fixture *fixture, unsigned char seed, size_t index) { } static bool publish_features(Fixture *fixture, size_t image_index) { - uint32_t feature_count = image_index < 2 ? 769U : 1U; + const char *pipeline_fixture = getenv("LARDON3D_TEST_MATCHER_PIPELINE_FIXTURE"); + const size_t eligible_image_count = + pipeline_fixture && strcmp(pipeline_fixture, "1") == 0 ? 13U : 3U; + uint32_t feature_count = image_index < eligible_image_count ? 769U : 1U; Lardon3DFeatureKeypoint *keypoints = calloc(feature_count, sizeof(*keypoints)); unsigned char *descriptors = calloc(feature_count, 32); @@ -247,10 +521,11 @@ static bool publish_features(Fixture *fixture, size_t image_index) { } for (uint32_t index = 0; index < feature_count; ++index) { keypoints[index].size = 1.0F; - /* The first pair is identical and has more than 768 descriptors per side, - * forcing the audited Vulkan dispatch boundary while retaining a unique - * zero-distance nearest neighbor for exact CPU/file parity. */ - uint32_t value = image_index < 2 ? index : (uint32_t)image_index; + /* The first two pairs have more than 768 descriptors per side, forcing + * consecutive audited Vulkan submissions while retaining bounded fixture + * memory and deterministic CPU/file parity. */ + uint32_t value = image_index < eligible_image_count + ? index : (uint32_t)image_index; for (size_t byte = 0; byte < 32; ++byte) { descriptors[(size_t)index * 32 + byte] = (unsigned char)((value >> (8U * (byte % 4))) ^ (uint32_t)(byte * 29)); @@ -412,8 +687,9 @@ static bool run_completed_with_threads(Fixture *fixture, uint64_t task_id = 0; Lardon3DTaskSnapshot snapshot; Lardon3DProjectDbTask durable; - bool success = lardon3d_project_enqueue_matcher_task(&fixture->state, &settings, - &task_id) && + bool success = lardon3d_project_enqueue_matcher_task_with_mode( + &fixture->state, &settings, + LARDON3D_MATCHER_TASK_MODE_CPU_PARALLEL, &task_id) && wait_state(fixture->state.task_queue, task_id, TASK_COMPLETED, &snapshot) && snapshot.progress == 100 && @@ -437,8 +713,9 @@ static bool run_failed_at_pair(Fixture *fixture, } uint64_t task_id = 0; Lardon3DTaskSnapshot snapshot; - bool failed = lardon3d_project_enqueue_matcher_task(&fixture->state, &settings, - &task_id) && + bool failed = lardon3d_project_enqueue_matcher_task_with_mode( + &fixture->state, &settings, + LARDON3D_MATCHER_TASK_MODE_CPU_PARALLEL, &task_id) && wait_state(fixture->state.task_queue, task_id, TASK_FAILED, &snapshot); (void)unsetenv(failure_variable); @@ -460,7 +737,841 @@ static bool run_failed_at_pair(Fixture *fixture, saved.after_candidate_pair_id == expected_cursor; } +#if defined(LARDON3D_MATCHER_TASK_VULKAN) && \ + defined(LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE) +typedef struct { + size_t count; + Lardon3DProjectDbMatchResult results[PAIR_COUNT]; + Lardon3DResourceSequenceDiagnostic diagnostic; + Lardon3DResourceSequenceAggregate aggregate; + size_t overlap_publications; +} PipelineEvidence; + +#ifdef __linux__ +enum { PROCESS_THREAD_SNAPSHOT_CAPACITY = 1024 }; + +typedef struct { + size_t count; + pid_t tids[PROCESS_THREAD_SNAPSHOT_CAPACITY]; +} ProcessThreadSnapshot; + +static bool parse_task_tid(const char *text, pid_t *tid) { + if (!text || !tid || text[0] < '0' || text[0] > '9') return false; + unsigned long value = 0; + const char *cursor = text; + do { + unsigned int digit = (unsigned int)(*cursor - '0'); + if (value > (unsigned long)(INT_MAX - (int)digit) / 10) return false; + value = value * 10 + digit; + ++cursor; + } while (*cursor >= '0' && *cursor <= '9'); + if (*cursor != '\0' || value == 0) return false; + *tid = (pid_t)value; + return true; +} + +static bool capture_process_thread_snapshot(ProcessThreadSnapshot *snapshot) { + if (!snapshot) return false; + *snapshot = (ProcessThreadSnapshot){0}; + DIR *directory = opendir("/proc/self/task"); + if (!directory) return false; + bool valid = true; + for (struct dirent *entry = readdir(directory); entry; + entry = readdir(directory)) { + pid_t tid = 0; + if (!parse_task_tid(entry->d_name, &tid)) continue; + if (snapshot->count == PROCESS_THREAD_SNAPSHOT_CAPACITY) { + valid = false; + break; + } + snapshot->tids[snapshot->count++] = tid; + } + return closedir(directory) == 0 && valid; +} + +static bool thread_snapshot_contains( + const ProcessThreadSnapshot *snapshot, pid_t tid) { + for (size_t index = 0; index < snapshot->count; ++index) { + if (snapshot->tids[index] == tid) return true; + } + return false; +} + +static bool read_thread_comm(pid_t tid, char comm[64]) { + char path[128]; + int written = snprintf(path, sizeof(path), "/proc/self/task/%ld/comm", + (long)tid); + if (written <= 0 || (size_t)written >= sizeof(path)) return false; + int descriptor = open(path, O_RDONLY | O_CLOEXEC | O_NOFOLLOW); + if (descriptor < 0) return false; + ssize_t count = read(descriptor, comm, 63); + bool valid = count > 0 && count < 63 && read(descriptor, comm + count, 1) == 0; + if (close(descriptor) != 0) valid = false; + if (!valid) return false; + size_t length = (size_t)count; + while (length > 0 && + (comm[length - 1] == '\n' || comm[length - 1] == '\r')) { + --length; + } + comm[length] = '\0'; + return true; +} + +static bool read_thread_cpu_list(pid_t tid, char output[256]) { + char path[128]; + int written = snprintf(path, sizeof(path), "/proc/self/task/%ld/status", + (long)tid); + if (written <= 0 || (size_t)written >= sizeof(path)) return false; + int descriptor = open(path, O_RDONLY | O_CLOEXEC | O_NOFOLLOW); + if (descriptor < 0) return false; + char status[16384]; + ssize_t count = read(descriptor, status, sizeof(status) - 1); + bool valid = count > 0 && count < (ssize_t)(sizeof(status) - 1) && + read(descriptor, status + count, 1) == 0; + int saved_error = errno; + if (close(descriptor) != 0) valid = false; + if (!valid) { + errno = saved_error; + return false; + } + status[count] = '\0'; + static const char prefix[] = "Cpus_allowed_list:"; + char *line = strstr(status, prefix); + if (!line || (line != status && line[-1] != '\n')) return false; + line += sizeof(prefix) - 1; + while (*line == ' ' || *line == '\t') ++line; + char *end = strchr(line, '\n'); + if (!end) return false; + while (end > line && (end[-1] == ' ' || end[-1] == '\t' || + end[-1] == '\r')) { + --end; + } + size_t length = (size_t)(end - line); + if (length == 0 || length >= 256) return false; + memcpy(output, line, length); + output[length] = '\0'; + return true; +} + +static bool format_cpu_list( + const uint64_t mask[LARDON3D_RESOURCE_CPU_MASK_WORDS], char output[256]) { + size_t used = 0; + bool first = true; + for (unsigned int cpu = 0; cpu < LARDON3D_RESOURCE_CPU_MAX;) { + if ((mask[cpu / 64] & (UINT64_C(1) << (cpu % 64))) == 0) { + ++cpu; + continue; + } + unsigned int begin = cpu; + while (cpu + 1 < LARDON3D_RESOURCE_CPU_MAX && + (mask[(cpu + 1) / 64] & + (UINT64_C(1) << ((cpu + 1) % 64))) != 0) { + ++cpu; + } + unsigned int end = cpu; + int written = snprintf(output + used, 256 - used, + first ? begin == end ? "%u" : "%u-%u" + : begin == end ? ",%u" : ",%u-%u", + begin, end); + if (written <= 0 || (size_t)written >= 256 - used) return false; + used += (size_t)written; + first = false; + ++cpu; + } + return !first; +} + +static bool verify_process_heavy_affinity( + Lardon3DResourceGovernor *governor, + const ProcessThreadSnapshot *before_threads, + const cpu_set_t *main_before) { + Lardon3DResourceCpuPolicyDiagnostic diagnostic; + if (!governor || !before_threads || !main_before + || !lardon3d_resource_governor_internal_cpu_policy( + governor, &diagnostic) + || !diagnostic.affinity_active + || !diagnostic.runtime_thread_policy_active + || !diagnostic.mesa_shader_cache_disabled) { + return false; + } + char expected_cpu_list[256]; + if (!format_cpu_list(diagnostic.compute_mask, expected_cpu_list)) { + return false; + } + DIR *directory = opendir("/proc/self/task"); + if (!directory) return false; + bool valid = true; + size_t non_main = 0; + size_t mesa_disk_threads = 0; + pid_t main_tid = getpid(); + char main_comm[64]; + char main_cpu_list[256]; + if (!read_thread_comm(main_tid, main_comm) + || !read_thread_cpu_list(main_tid, main_cpu_list)) { + valid = false; + } else { + (void)fprintf(stdout, + "thread_affinity_evidence tid=%ld comm=%s cpus=%s " + "role=main\n", + (long)main_tid, main_comm, main_cpu_list); + } + for (struct dirent *entry = readdir(directory); entry; + entry = readdir(directory)) { + pid_t tid = 0; + if (!parse_task_tid(entry->d_name, &tid) || tid == main_tid) continue; + char comm[64]; + char cpu_list[256]; + errno = 0; + if (!read_thread_comm(tid, comm) || !read_thread_cpu_list(tid, cpu_list)) { + if (errno == ESRCH || errno == ENOENT) continue; + valid = false; + break; + } + ++non_main; + bool created_after_snapshot = !thread_snapshot_contains(before_threads, tid); + (void)fprintf(stdout, + "thread_affinity_evidence tid=%ld comm=%s cpus=%s " + "role=runtime new=%s\n", + (long)tid, comm, cpu_list, + created_after_snapshot ? "true" : "false"); + if (strcmp(cpu_list, expected_cpu_list) != 0) valid = false; + if (strstr(comm, ":disk$") != NULL) ++mesa_disk_threads; + if (!valid) break; + } + if (closedir(directory) != 0) valid = false; + cpu_set_t main_after; + CPU_ZERO(&main_after); + if (sched_getaffinity(0, sizeof(main_after), &main_after) != 0 + || !CPU_EQUAL(main_before, &main_after)) { + valid = false; + } + /* The Queue worker is always present. On the verified 780M host, disabling + * Mesa's disk cache must eliminate the known affinity-widening disk helpers; + * every remaining live runtime thread must retain the worker compute pool. */ + Lardon3DHardwareProfile host; + bool current_780m = lardon3d_hardware_profile_detect(&host, NULL, 0) + && host.gpu_drm_card_index == 1 && host.gpu_uses_shared_memory + && host.gpu_memory_known + && host.gpu_memory_total_bytes == UINT64_C(536870912); + if (current_780m && mesa_disk_threads != 0) { + valid = false; + } + return valid && non_main >= 1; +} +#else +typedef struct { + bool unused; +} ProcessThreadSnapshot; + +static bool capture_process_thread_snapshot(ProcessThreadSnapshot *snapshot) { + if (!snapshot) return false; + snapshot->unused = false; + return true; +} + +static bool verify_process_heavy_affinity( + Lardon3DResourceGovernor *governor, + const ProcessThreadSnapshot *before_threads, + const cpu_set_t *main_before) { + (void)governor; + (void)before_threads; + (void)main_before; + return true; +} +#endif + +static bool capture_pipeline_evidence(bool synchronous, + unsigned int inflight_override, + unsigned int batch_override, + PipelineEvidence *evidence) { + if (!evidence || (inflight_override != 1 && inflight_override != 2) || + (batch_override != 0 && batch_override != 2 && batch_override != 4 && + batch_override != 8 && batch_override != 12) || + (synchronous && (inflight_override != 1 || batch_override != 0))) + return false; + Fixture fixture; + cpu_set_t main_before; + CPU_ZERO(&main_before); + if (sched_getaffinity(0, sizeof(main_before), &main_before) != 0) return false; + /* Keep enough consecutive eligible pairs for AUTO slow-start to reach a + * rolling batch greater than one without making the main CPU fixture heavy. */ + if (setenv("LARDON3D_TEST_MATCHER_PIPELINE_FIXTURE", "1", 1) != 0 || + !fixture_create(&fixture) || + unsetenv("LARDON3D_TEST_MATCHER_PIPELINE_FIXTURE") != 0) + return false; + ProcessThreadSnapshot before_threads; + if (!capture_process_thread_snapshot(&before_threads)) { + stop_runtime(&fixture); + (void)remove_tree(fixture.root); + return false; + } + const char *inflight = inflight_override == 1 ? "1" : "2"; + char batch[3]; + int batch_length = snprintf(batch, sizeof(batch), "%u", batch_override); + bool configured = + (synchronous + ? setenv(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV, "1", 1) + : unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV)) == 0 && + setenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV, inflight, 1) == 0 && + (batch_override != 0 + ? batch_length > 0 && (size_t)batch_length < sizeof(batch) && + setenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV, batch, 1) == 0 + : unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV) == 0) && + unsetenv("LARDON3D_TEST_MATCHER_INFLIGHT_LIMIT") == 0; + lardon3d_matcher_task_test_reset_backend_counters(); + Lardon3DMatcherTaskConfiguration settings = configuration(&fixture); + uint64_t task_id = 0; + Lardon3DTaskSnapshot snapshot; + bool completed = configured && lardon3d_project_enqueue_matcher_task( + &fixture.state, &settings, &task_id) && + wait_state(fixture.state.task_queue, task_id, + TASK_COMPLETED, &snapshot); + bool environment_restored = + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV) == 0 && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV) == 0 && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV) == 0; + completed = environment_restored && completed; + uint64_t cursor = 0; + while (completed && evidence->count < PAIR_COUNT) { + size_t count = 0; + completed = lardon3d_project_db_list_match_results( + fixture.state.project_db, cursor, + evidence->results + evidence->count, + PAIR_COUNT - evidence->count, &count) == + LARDON3D_PROJECT_DB_OK; + if (!completed || count == 0) break; + evidence->count += count; + cursor = evidence->results[evidence->count - 1].match_result_id; + } + evidence->overlap_publications = + lardon3d_matcher_task_test_overlap_publications(); + completed = completed && evidence->count == PERSISTED_PAIR_COUNT && + lardon3d_matcher_task_test_vulkan_uses() >= 2 && + lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &evidence->diagnostic) && + lardon3d_resource_governor_internal_sequence_aggregate( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &evidence->aggregate) && + evidence->diagnostic.backend == + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + evidence->diagnostic.cpu_threads == 1 && + evidence->diagnostic.gpu_slots == 1 && + evidence->diagnostic.helper_limit == 0 && + evidence->diagnostic.io_slots == 1 && + evidence->diagnostic.batch_size == + (batch_override != 0 ? batch_override : 2) && + evidence->diagnostic.memory_bytes == + (uint64_t)(batch_override != 0 ? batch_override : 2) * + 10U * 1024U * 1024U && + evidence->diagnostic.inflight_limit == inflight_override && + evidence->diagnostic.gpu_memory_bytes == + inflight_override * 640U * 1024U && + evidence->aggregate.admission_count > 0 && + evidence->aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_CPU] == 0 && + evidence->aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN] == + evidence->aggregate.admission_count && + evidence->aggregate.contract_change_count == 0 && + evidence->aggregate.backend_failure_fallback_sequences == 0 && + evidence->aggregate.backend_other_fallback_sequences == 0 && + evidence->aggregate.local_ineligible_fallback_items == + PERSISTED_PAIR_COUNT - PIPELINE_ELIGIBLE_PAIR_COUNT && + evidence->aggregate.backend_failure_fallback_items == 0 && + evidence->aggregate.backend_other_fallback_items == 0 && + verify_process_heavy_affinity( + fixture.state.resource_governor, &before_threads, &main_before); + stop_runtime(&fixture); + return remove_tree(fixture.root) && completed; +} + +static bool same_pipeline_scientific_output( + const PipelineEvidence *rolling, const PipelineEvidence *synchronous) { + if (rolling->count != synchronous->count) return false; + for (size_t index = 0; index < rolling->count; ++index) { + const Lardon3DProjectDbMatchResult *left = &rolling->results[index]; + const Lardon3DProjectDbMatchResult *right = &synchronous->results[index]; + if (left->candidate_pair_id != right->candidate_pair_id || + left->feature_set_id_a != right->feature_set_id_a || + left->feature_set_id_b != right->feature_set_id_b || + strcmp(left->matcher_kind, right->matcher_kind) != 0 || + left->matcher_version != right->matcher_version || + memcmp(left->parameter_fingerprint, right->parameter_fingerprint, + sizeof(left->parameter_fingerprint)) != 0 || + left->result_status != right->result_status || + left->match_count != right->match_count || + left->has_match_asset != right->has_match_asset || + memcmp(left->match_asset_sha256, right->match_asset_sha256, + sizeof(left->match_asset_sha256)) != 0 || + left->match_asset_size_bytes != right->match_asset_size_bytes) + return false; + } + return true; +} + +static bool same_forced_contract_except_depth( + const PipelineEvidence *depth_one, const PipelineEvidence *depth_two) { + const Lardon3DResourceSequenceDiagnostic *one = &depth_one->diagnostic; + const Lardon3DResourceSequenceDiagnostic *two = &depth_two->diagnostic; + return one->backend == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + two->backend == one->backend && one->cpu_threads == 1 && + two->cpu_threads == one->cpu_threads && one->gpu_slots == 1 && + two->gpu_slots == one->gpu_slots && one->batch_size == 2 && + two->batch_size == one->batch_size && one->helper_limit == 0 && + two->helper_limit == one->helper_limit && + two->io_slots == one->io_slots && two->memory_bytes == one->memory_bytes && + one->inflight_limit == 1 && two->inflight_limit == 2 && + one->gpu_memory_bytes == 640U * 1024U && + two->gpu_memory_bytes == 2U * 640U * 1024U; +} + +static bool same_forced_contract_except_batch( + const PipelineEvidence *batch_two, const PipelineEvidence *other, + size_t expected_batch) { + const Lardon3DResourceSequenceDiagnostic *two = &batch_two->diagnostic; + const Lardon3DResourceSequenceDiagnostic *candidate = &other->diagnostic; + return two->backend == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + candidate->backend == two->backend && two->cpu_threads == 1 && + candidate->cpu_threads == two->cpu_threads && two->gpu_slots == 1 && + candidate->gpu_slots == two->gpu_slots && two->batch_size == 2 && + candidate->batch_size == expected_batch && two->helper_limit == 0 && + candidate->helper_limit == two->helper_limit && + candidate->io_slots == two->io_slots && two->inflight_limit == 1 && + candidate->inflight_limit == two->inflight_limit && + two->gpu_memory_bytes == 640U * 1024U && + candidate->gpu_memory_bytes == two->gpu_memory_bytes && + two->memory_bytes == 2U * 10U * 1024U * 1024U && + candidate->memory_bytes == expected_batch * 10U * 1024U * 1024U; +} + +static bool forced_benchmark_fail_closed(void) { + if (unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV) != 0 || + setenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV, "2", 1) != 0) + return false; + Fixture fixture; + if (!fixture_create(&fixture)) { + (void)unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV); + return false; + } + Lardon3DMatcherTaskConfiguration settings = configuration(&fixture); + uint64_t rejected_id = 0; + /* Build/GPU/backend are the only creation metadata. Memory failure belongs + * to the exact Governor boundary test above, not a divergent AppState copy. */ + fixture.state.hardware_profile.gpu_available = false; + Lardon3DTask *gpu_rejected = lardon3d_project_create_matcher_task( + &fixture.state, &settings, &rejected_id); + fixture.state.hardware_profile.gpu_available = true; + bool ok = !gpu_rejected; + if (!ok) fprintf(stderr, "forced benchmark pre-admission rejection failed\n"); + if (gpu_rejected) lardon3d_task_destroy(gpu_rejected); + + /* Backend availability can disappear after creation but before admission. + * The one-capability forced envelope must be rejected, never admitted CPU. */ + uint64_t unavailable_id = 0; + Lardon3DTask *unavailable = ok ? lardon3d_project_create_matcher_task( + &fixture.state, &settings, &unavailable_id) : NULL; + Lardon3DTaskSnapshot snapshot; + bool unavailable_added = unavailable && + lardon3d_resource_governor_internal_set_backend_available( + fixture.state.resource_governor, + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, false) && + lardon3d_task_queue_add(fixture.state.task_queue, unavailable, NULL); + if (unavailable_added) unavailable = NULL; + ok = ok && unavailable_added && + wait_state(fixture.state.task_queue, unavailable_id, TASK_FAILED, + &snapshot); + if (!ok) fprintf(stderr, "forced benchmark unavailable backend was not rejected\n"); + if (unavailable) lardon3d_task_destroy(unavailable); + + /* A failure after GPU admission may leave exact whole-pair CPU publications + * durable, but the benchmark Task itself must fail and classify the cohort + * as backend failure rather than local ineligibility. */ + char failed_pair[32]; + Lardon3DMatcherTaskConfiguration failing = settings; + failing.matcher.ratio_threshold = 0.74F; + uint64_t failing_id = 0; + ok = ok && + lardon3d_resource_governor_internal_set_backend_available( + fixture.state.resource_governor, + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, true) && + snprintf(failed_pair, sizeof(failed_pair), "%lu", + (unsigned long)fixture.pairs[0].candidate_pair_id) > 0 && + setenv("LARDON3D_TEST_MATCHER_FAIL_VULKAN_BEGIN_PAIR_ID", + failed_pair, 1) == 0 && + lardon3d_project_enqueue_matcher_task( + &fixture.state, &failing, &failing_id) && + wait_state(fixture.state.task_queue, failing_id, TASK_FAILED, &snapshot); + (void)unsetenv("LARDON3D_TEST_MATCHER_FAIL_VULKAN_BEGIN_PAIR_ID"); + Lardon3DResourceSequenceAggregate aggregate; + Lardon3DResourceSequenceDiagnostic failing_diagnostic; + size_t failed_results = 0; + ok = ok && count_results(&fixture, &failed_results) && failed_results == 2 && + lardon3d_resource_governor_internal_sequence_aggregate( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &aggregate) && + lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &failing_diagnostic) && + aggregate.admission_count == 1 && aggregate.sequence_count == 1 && + aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_CPU] == 0 && + aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN] == 1 && + aggregate.backend_failure_fallback_sequences == 1 && + aggregate.backend_ineligible_fallback_sequences == 0 && + aggregate.backend_other_fallback_sequences == 0 && + aggregate.local_ineligible_fallback_items == 0 && + aggregate.backend_failure_fallback_items == failed_results && + aggregate.backend_other_fallback_items == 0 && + failing_diagnostic.execution.backend_failure_fallback_items == + failed_results && + failing_diagnostic.execution.local_ineligible_fallback_items == 0 && + failing_diagnostic.execution.backend_other_fallback_items == 0; + if (!ok) { + fprintf(stderr, + "forced benchmark failure classification failed results=%zu " + "admissions=%lu sequences=%lu cpu=%lu vk=%lu actual_cpu=%lu " + "actual_vk=%lu actual_mixed=%lu fallbacks=%lu local=%lu " + "failure=%lu other=%lu local_items=%lu failure_items=%lu " + "other_items=%lu reason=%s\n", + failed_results, (unsigned long)aggregate.admission_count, + (unsigned long)aggregate.sequence_count, + (unsigned long)aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_CPU], + (unsigned long)aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN], + (unsigned long)aggregate.actual_backend_sequences[ + LARDON3D_RESOURCE_BACKEND_CPU], + (unsigned long)aggregate.actual_backend_sequences[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN], + (unsigned long)aggregate.actual_backend_sequences[ + LARDON3D_RESOURCE_BACKEND_MIXED], + (unsigned long)aggregate.backend_fallback_sequences, + (unsigned long)aggregate.backend_ineligible_fallback_sequences, + (unsigned long)aggregate.backend_failure_fallback_sequences, + (unsigned long)aggregate.backend_other_fallback_sequences, + (unsigned long)aggregate.local_ineligible_fallback_items, + (unsigned long)aggregate.backend_failure_fallback_items, + (unsigned long)aggregate.backend_other_fallback_items, + failing_diagnostic.backend_reason); + } + + Lardon3DMatcherTaskConfiguration following = settings; + following.matcher.ratio_threshold = 0.73F; + uint64_t following_id = 0; + bool availability_restored = + lardon3d_resource_governor_internal_set_backend_available( + fixture.state.resource_governor, + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, true); + bool following_enqueued = availability_restored && + lardon3d_project_enqueue_matcher_task( + &fixture.state, &following, &following_id); + bool following_completed = following_enqueued && + wait_state(fixture.state.task_queue, following_id, TASK_COMPLETED, + &snapshot); + bool following_aggregate = following_completed && + lardon3d_resource_governor_internal_sequence_aggregate( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &aggregate); + ok = ok && following_aggregate && aggregate.admission_count > 1 && + aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_CPU] == 0 && + aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN] == + aggregate.admission_count && + aggregate.backend_failure_fallback_sequences == 1 && + aggregate.backend_other_fallback_sequences == 0 && + aggregate.backend_failure_fallback_items == failed_results && + aggregate.backend_other_fallback_items == 0; + if (!ok) { + fprintf(stderr, + "forced benchmark backend reuse failed available=%d enqueue=%d " + "completed=%d aggregate=%d state=%d admissions=%lu cpu=%lu vk=%lu\n", + availability_restored, following_enqueued, following_completed, + following_aggregate, (int)snapshot.state, + (unsigned long)aggregate.admission_count, + (unsigned long)aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_CPU], + (unsigned long)aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN]); + } + bool environment_restored = + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV) == 0 && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV) == 0; + stop_runtime(&fixture); + return remove_tree(fixture.root) && environment_restored && ok; +} + +static bool pair_local_neighbor_failure_case(const char *failure_variable) { + Fixture fixture; + if (!failure_variable || !fixture_create(&fixture)) return false; + char local_pair[32]; + char failed_pair[32]; + bool configured = + snprintf(local_pair, sizeof(local_pair), "%lu", + (unsigned long)fixture.pairs[0].candidate_pair_id) > 0 && + snprintf(failed_pair, sizeof(failed_pair), "%lu", + (unsigned long)fixture.pairs[1].candidate_pair_id) > 0 && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV) == 0 && + setenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV, "2", 1) == 0 && + setenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV, "2", 1) == 0 && + setenv("LARDON3D_TEST_MATCHER_INELIGIBLE_VULKAN_BEGIN_PAIR_ID", + local_pair, 1) == 0 && + setenv(failure_variable, failed_pair, 1) == 0; + Lardon3DMatcherTaskConfiguration settings = configuration(&fixture); + uint64_t task_id = 0; + Lardon3DTaskSnapshot snapshot; + bool failed = configured && lardon3d_project_enqueue_matcher_task( + &fixture.state, &settings, &task_id) && + wait_state(fixture.state.task_queue, task_id, TASK_FAILED, &snapshot); + Lardon3DResourceSequenceAggregate aggregate = {0}; + Lardon3DResourceSequenceDiagnostic diagnostic = {0}; + size_t result_count = 0; + bool exact = failed && count_results(&fixture, &result_count) && + result_count == 2 && + lardon3d_resource_governor_internal_sequence_aggregate( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &aggregate) && + lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &diagnostic) && + aggregate.sequence_count == 1 && aggregate.durable_items == 2 && + aggregate.local_ineligible_fallback_items == 1 && + aggregate.backend_failure_fallback_items == 1 && + aggregate.backend_other_fallback_items == 0 && + diagnostic.execution.local_ineligible_fallback_items == 1 && + diagnostic.execution.backend_failure_fallback_items == 1 && + diagnostic.execution.backend_other_fallback_items == 0; + (void)unsetenv(failure_variable); + (void)unsetenv("LARDON3D_TEST_MATCHER_INELIGIBLE_VULKAN_BEGIN_PAIR_ID"); + (void)unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV); + (void)unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV); + stop_runtime(&fixture); + return remove_tree(fixture.root) && exact; +} + +static bool local_vulkan_failure_preserves_backend_case( + const char *failure_variable, bool pre_submit) { + Fixture fixture; + if (!failure_variable || !fixture_create(&fixture)) return false; + char first_pair[32]; + bool configured = + snprintf(first_pair, sizeof(first_pair), "%lu", + (unsigned long)fixture.pairs[0].candidate_pair_id) > 0 && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV) == 0 && + setenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV, "2", 1) == 0 && + setenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV, "2", 1) == 0 && + setenv("LARDON3D_TEST_MATCHER_PAUSE_AFTER_BATCH", "1", 1) == 0 && + setenv(failure_variable, first_pair, 1) == 0; + Lardon3DOrbVulkanTelemetry telemetry_before = {0}; + Lardon3DOrbVulkanTelemetry telemetry_after = {0}; + configured = configured && lardon3d_orb_vulkan_internal_telemetry( + fixture.state.orb_vulkan_backend, &telemetry_before); + lardon3d_matcher_task_test_reset_backend_counters(); + Lardon3DMatcherTaskConfiguration settings = configuration(&fixture); + settings.matcher.ratio_threshold = pre_submit ? 0.611F : 0.612F; + uint64_t task_id = 0; + Lardon3DTaskSnapshot snapshot; + bool paused = configured && lardon3d_project_enqueue_matcher_task( + &fixture.state, &settings, &task_id) && + wait_state(fixture.state.task_queue, task_id, TASK_PAUSED, &snapshot); + Lardon3DResourceSequenceAggregate aggregate = {0}; + Lardon3DResourceSequenceDiagnostic diagnostic = {0}; + Lardon3DOrbVulkanInfo backend_info = {0}; + size_t first_submit = 0; + size_t first_finish = 0; + size_t successor_submit = 0; + size_t successor_finish = 0; + bool first_submitted = find_test_event( + TEST_EVENT_GPU_SUBMIT, fixture.pairs[0].candidate_pair_id, + &first_submit); + bool first_finished = find_test_event( + TEST_EVENT_GPU_FINISH, fixture.pairs[0].candidate_pair_id, + &first_finish); + bool successor_submitted = find_test_event( + TEST_EVENT_GPU_SUBMIT, fixture.pairs[1].candidate_pair_id, + &successor_submit); + bool successor_finished = find_test_event( + TEST_EVENT_GPU_FINISH, fixture.pairs[1].candidate_pair_id, + &successor_finish); + /* The injected fault is deliberately outside Vulkan. The first durable + * sequence must use a complete CPU fallback, retain an unrelated submitted + * successor, and classify exactly one OTHER item without poisoning shared + * backend health or leaking a request-bound slot. */ + bool exact = paused && + lardon3d_resource_governor_internal_sequence_aggregate( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &aggregate) && + lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &diagnostic) && + lardon3d_orb_vulkan_internal_telemetry( + fixture.state.orb_vulkan_backend, &telemetry_after) && + lardon3d_orb_vulkan_backend_info( + fixture.state.orb_vulkan_backend, &backend_info) && + aggregate.sequence_count == 1 && aggregate.durable_items == 2 && + aggregate.backend_failure_fallback_sequences == 0 && + aggregate.backend_other_fallback_sequences == 1 && + aggregate.local_ineligible_fallback_items == 0 && + aggregate.backend_failure_fallback_items == 0 && + aggregate.backend_other_fallback_items == 1 && + diagnostic.backend == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + diagnostic.actual_backend == LARDON3D_RESOURCE_BACKEND_MIXED && + diagnostic.backend_fallback && + diagnostic.execution.local_ineligible_fallback_items == 0 && + diagnostic.execution.backend_failure_fallback_items == 0 && + diagnostic.execution.backend_other_fallback_items == 1 && + strcmp(diagnostic.backend_reason, + "vulkan-and-local-failure-cpu-fallback") == 0 && + successor_submitted && successor_finished && + (pre_submit ? !first_submitted && !first_finished + : first_submitted && first_finished) && + telemetry_after.submits == telemetry_before.submits + + (pre_submit ? 1U : 2U) && + telemetry_after.completions == telemetry_before.completions + + (pre_submit ? 1U : 2U) && + telemetry_after.failures == telemetry_before.failures && + telemetry_after.discards == telemetry_before.discards && + telemetry_after.pending_slots == 0 && !telemetry_after.slot_pending && + backend_info.available && + lardon3d_matcher_task_test_overlap_publications() >= 1 && + no_staged_match_temporaries(&fixture); + + bool resumed = unsetenv(failure_variable) == 0 && + unsetenv("LARDON3D_TEST_MATCHER_PAUSE_AFTER_BATCH") == 0 && + lardon3d_task_queue_resume(fixture.state.task_queue, task_id) && + wait_state(fixture.state.task_queue, task_id, TASK_COMPLETED, &snapshot); + size_t first_task_results = 0; + exact = exact && resumed && count_results(&fixture, &first_task_results) && + first_task_results == PERSISTED_PAIR_COUNT; + + /* A fresh AUTO Task is the Governor-level health proof. It must still + * select the Vulkan arm and reuse the exact slots after the local fault. */ + bool overrides_cleared = + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV) == 0 && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV) == 0; + lardon3d_matcher_task_test_reset_backend_counters(); + Lardon3DMatcherTaskConfiguration following = configuration(&fixture); + following.matcher.ratio_threshold = pre_submit ? 0.613F : 0.614F; + uint64_t following_id = 0; + size_t final_results = 0; + exact = exact && overrides_cleared && + lardon3d_project_enqueue_matcher_task( + &fixture.state, &following, &following_id) && + wait_state(fixture.state.task_queue, following_id, + TASK_COMPLETED, &snapshot) && + lardon3d_matcher_task_test_vulkan_uses() >= 2 && + lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &diagnostic) && + diagnostic.backend == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + diagnostic.gpu_slots == 1 && + lardon3d_resource_governor_internal_sequence_aggregate( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &aggregate) && + aggregate.backend_failure_fallback_sequences == 0 && + aggregate.backend_failure_fallback_items == 0 && + aggregate.backend_other_fallback_items == 1 && + count_results(&fixture, &final_results) && + final_results == 2 * PERSISTED_PAIR_COUNT && + lardon3d_orb_vulkan_backend_info( + fixture.state.orb_vulkan_backend, &backend_info) && + backend_info.available && no_staged_match_temporaries(&fixture); + + (void)unsetenv(failure_variable); + (void)unsetenv("LARDON3D_TEST_MATCHER_PAUSE_AFTER_BATCH"); + (void)unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV); + (void)unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV); + (void)unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV); + stop_runtime(&fixture); + return remove_tree(fixture.root) && exact; +} + +typedef enum { + PREFIX_EXIT_CANCEL = 0, + PREFIX_EXIT_COMPUTE_FAILURE, + PREFIX_EXIT_PUBLICATION_FAILURE, +} PrefixExit; + +static bool durable_fallback_prefix_case(bool synchronous, PrefixExit exit) { + Fixture fixture; + if (!fixture_create(&fixture)) return false; + char first_pair[32]; + char second_pair[32]; + bool configured = + snprintf(first_pair, sizeof(first_pair), "%lu", + (unsigned long)fixture.pairs[0].candidate_pair_id) > 0 && + snprintf(second_pair, sizeof(second_pair), "%lu", + (unsigned long)fixture.pairs[1].candidate_pair_id) > 0 && + (synchronous + ? setenv(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV, "1", 1) + : unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV)) == 0 && + setenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV, + synchronous ? "1" : "2", 1) == 0 && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV) == 0 && + setenv("LARDON3D_TEST_MATCHER_INELIGIBLE_VULKAN_BEGIN_PAIR_ID", + first_pair, 1) == 0; + if (configured && exit == PREFIX_EXIT_CANCEL) { + configured = setenv( + "LARDON3D_TEST_MATCHER_PAUSE_AFTER_PUBLICATION", "1", 1) == 0; + } else if (configured && exit == PREFIX_EXIT_COMPUTE_FAILURE) { + configured = + setenv("LARDON3D_TEST_MATCHER_FAIL_VULKAN_BEGIN_PAIR_ID", + second_pair, 1) == 0 && + setenv("LARDON3D_TEST_MATCHER_FAIL_COMPUTE_PAIR_ID", + second_pair, 1) == 0; + } else if (configured) { + configured = + setenv("LARDON3D_TEST_MATCHER_FAIL_VULKAN_BEGIN_PAIR_ID", + second_pair, 1) == 0 && + setenv("LARDON3D_TEST_MATCHER_FAIL_PUBLISH_PAIR_ID", + second_pair, 1) == 0; + } + Lardon3DMatcherTaskConfiguration settings = configuration(&fixture); + uint64_t task_id = 0; + Lardon3DTaskSnapshot snapshot; + bool ended = configured && lardon3d_project_enqueue_matcher_task( + &fixture.state, &settings, &task_id); + if (ended && exit == PREFIX_EXIT_CANCEL) { + ended = wait_state(fixture.state.task_queue, task_id, TASK_PAUSED, + &snapshot) && + lardon3d_task_queue_cancel(fixture.state.task_queue, task_id) && + wait_state(fixture.state.task_queue, task_id, TASK_CANCELLED, + &snapshot); + } else if (ended) { + ended = wait_state(fixture.state.task_queue, task_id, TASK_FAILED, + &snapshot); + } + Lardon3DResourceSequenceAggregate aggregate = {0}; + size_t result_count = 0; + /* The first fallback is durable; the second pair never reaches that + * boundary. Failed/cancelled work must retain one item without manufacturing + * a successful throughput sequence or durable-item rate observation. */ + bool exact = ended && count_results(&fixture, &result_count) && + result_count == 1 && + lardon3d_resource_governor_internal_sequence_aggregate( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &aggregate) && + aggregate.local_ineligible_fallback_items == 1 && + aggregate.backend_failure_fallback_items == 0 && + aggregate.backend_other_fallback_items == 0 && + aggregate.sequence_count == 0 && aggregate.durable_items == 0; + (void)unsetenv("LARDON3D_TEST_MATCHER_PAUSE_AFTER_PUBLICATION"); + (void)unsetenv("LARDON3D_TEST_MATCHER_FAIL_VULKAN_BEGIN_PAIR_ID"); + (void)unsetenv("LARDON3D_TEST_MATCHER_FAIL_COMPUTE_PAIR_ID"); + (void)unsetenv("LARDON3D_TEST_MATCHER_FAIL_PUBLISH_PAIR_ID"); + (void)unsetenv("LARDON3D_TEST_MATCHER_INELIGIBLE_VULKAN_BEGIN_PAIR_ID"); + (void)unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV); + (void)unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV); + (void)unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV); + stop_runtime(&fixture); + return remove_tree(fixture.root) && exact; +} +#endif + static bool run_test(void) { + CHECK(matcher_exact_memory_boundary_test()); Fixture fixture; CHECK(fixture_create(&fixture)); char database_path[PATH_MAX]; @@ -498,6 +1609,31 @@ static bool run_test(void) { descriptor != NULL); Lardon3DMatcherTaskConfiguration settings = configuration(&fixture); +#ifdef LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE + /* Malformed runner-private controls must fail Task construction rather than + * silently selecting an adaptive or partially tagged capability. */ + uint64_t invalid_benchmark_task_id = 99; + CHECK(setenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV, "01", 1) == 0); + Lardon3DTask *invalid_benchmark_task = + lardon3d_project_create_matcher_task( + &fixture.state, &settings, &invalid_benchmark_task_id); + CHECK(invalid_benchmark_task == NULL && invalid_benchmark_task_id == 0 && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV) == 0); + CHECK(setenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV, "1", 1) == 0 && + setenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV, "3", 1) == 0); + invalid_benchmark_task_id = 99; + invalid_benchmark_task = lardon3d_project_create_matcher_task( + &fixture.state, &settings, &invalid_benchmark_task_id); + CHECK(invalid_benchmark_task == NULL && invalid_benchmark_task_id == 0 && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV) == 0 && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV) == 0); + CHECK(setenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV, "4", 1) == 0); + invalid_benchmark_task_id = 99; + invalid_benchmark_task = lardon3d_project_create_matcher_task( + &fixture.state, &settings, &invalid_benchmark_task_id); + CHECK(invalid_benchmark_task == NULL && invalid_benchmark_task_id == 0 && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV) == 0); +#endif Lardon3DMatcherTaskConfiguration invalid = settings; invalid.matcher.kind = LARDON3D_MATCHER_SIFT_BF; uint64_t invalid_task_id = 0; @@ -507,10 +1643,34 @@ static bool run_test(void) { CHECK(setenv("LARDON3D_TEST_MATCHER_PAUSE_AFTER_PUBLICATION", "1", 1) == 0); CHECK(setenv("LARDON3D_TEST_MATCHER_SKIP_FINISHED_CHECKPOINT", "1", 1) == 0); uint64_t task_id = 0; - CHECK(lardon3d_project_enqueue_matcher_task(&fixture.state, &settings, - &task_id)); + const uint64_t original_memory_total = + fixture.state.hardware_profile.memory_total_bytes; + const uint64_t depth_one_maximum_total = + UINT64_C(3) * 1024 * 1024 * 1024 + + UINT64_C(8) * 10 * 1024 * 1024 + UINT64_C(640) * 1024; + fixture.state.hardware_profile.memory_total_bytes = depth_one_maximum_total; + Lardon3DTask *first_auto = lardon3d_project_create_matcher_task( + &fixture.state, &settings, &task_id); + fixture.state.hardware_profile.memory_total_bytes = original_memory_total; + Lardon3DOrbVulkanInfo before_auto; + CHECK(first_auto && lardon3d_orb_vulkan_backend_info( + fixture.state.orb_vulkan_backend, &before_auto) + && !before_auto.initialized); + /* Normal creation at the exact depth-1/batch-8 UMA boundary has exposed + * metadata only. The removed depth-2 caller guess would suppress it here; + * the first possible initialization remains begin() on the Queue worker. */ + CHECK(lardon3d_task_queue_add( + fixture.state.task_queue, first_auto, NULL)); Lardon3DTaskSnapshot snapshot; CHECK(wait_state(fixture.state.task_queue, task_id, TASK_PAUSED, &snapshot)); + Lardon3DOrbVulkanInfo after_auto; + CHECK(lardon3d_orb_vulkan_backend_info( + fixture.state.orb_vulkan_backend, &after_auto)); +#ifdef LARDON3D_MATCHER_TASK_VULKAN + CHECK(after_auto.initialized); +#else + CHECK(!after_auto.initialized); +#endif size_t result_count = 0; CHECK(count_results(&fixture, &result_count) && result_count == 1); @@ -545,9 +1705,11 @@ static bool run_test(void) { fixture.pairs[PAIR_COUNT - 1].candidate_pair_id); CHECK(setenv("LARDON3D_TEST_MATCHER_PAUSE_AFTER_PUBLICATION", "1", 1) == 0); + CHECK(setenv("LARDON3D_TEST_MATCHER_INFLIGHT_LIMIT", "2", 1) == 0); uint64_t cancelled_id = 0; CHECK(lardon3d_project_enqueue_matcher_task(&fixture.state, &settings, &cancelled_id)); + CHECK(unsetenv("LARDON3D_TEST_MATCHER_INFLIGHT_LIMIT") == 0); CHECK(wait_state(fixture.state.task_queue, cancelled_id, TASK_PAUSED, &snapshot)); CHECK(lardon3d_task_queue_cancel(fixture.state.task_queue, cancelled_id)); @@ -574,7 +1736,10 @@ static bool run_test(void) { &snapshot)); Lardon3DResourcePolicy pressure_policy = interactive_policy(); pressure_policy.system_memory_reserve_bytes = - fixture.state.hardware_profile.memory_total_bytes - 96ULL * 1024 * 1024; + /* CPU12 reserves at most twelve 10 MiB staged-pair working sets; leave + * less than that available so the paused task must wait for a safe + * re-admission rather than silently exceeding its Governor contract. */ + fixture.state.hardware_profile.memory_total_bytes - 128ULL * 1024 * 1024; pressure_policy.emergency_memory_floor_bytes = pressure_policy.system_memory_reserve_bytes; CHECK(lardon3d_resource_governor_set_policy(fixture.state.resource_governor, @@ -596,8 +1761,9 @@ static bool run_test(void) { CHECK(wait_state(fixture.state.task_queue, pressure_id, TASK_COMPLETED, &snapshot)); - /* Thread-count changes are operational only: all three runs must publish + /* Thread-count changes are operational only: all five runs must publish * the same Candidate Pair cardinality and identical raw Match evidence. */ + CHECK(lardon3d_feature_opencv_configure_threads(3)); size_t equivalence_before = 0; CHECK(count_results(&fixture, &equivalence_before)); Lardon3DMatcherTaskConfiguration one = settings; @@ -612,9 +1778,22 @@ static bool run_test(void) { Lardon3DMatcherTaskConfiguration eight = settings; eight.matcher.ratio_threshold = 0.84F; CHECK(run_completed_with_threads(&fixture, eight, 8)); + Lardon3DMatcherTaskConfiguration twelve = settings; + twelve.matcher.ratio_threshold = 0.85F; + CHECK(run_completed_with_threads(&fixture, twelve, 12)); size_t equivalence_after = 0; CHECK(count_results(&fixture, &equivalence_after)); - CHECK(equivalence_after == equivalence_before + 4 * PERSISTED_PAIR_COUNT); + CHECK(equivalence_after == equivalence_before + 5 * PERSISTED_PAIR_COUNT); + CHECK(lardon3d_feature_opencv_thread_count() == 3); + Lardon3DResourceSequenceDiagnostic cpu_item_diagnostic; + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &cpu_item_diagnostic) && + cpu_item_diagnostic.backend == LARDON3D_RESOURCE_BACKEND_CPU && + cpu_item_diagnostic.actual_backend == LARDON3D_RESOURCE_BACKEND_CPU && + cpu_item_diagnostic.execution.local_ineligible_fallback_items == 0 && + cpu_item_diagnostic.execution.backend_failure_fallback_items == 0 && + cpu_item_diagnostic.execution.backend_other_fallback_items == 0); CHECK(candidate_results_have_one_evidence( database_path, fixture.pairs[0].candidate_pair_id)); @@ -622,10 +1801,219 @@ static bool run_test(void) { * reserve the exact GPU shape and dispatch the first 769x769 ORB pair to * Vulkan; all Match evidence remains byte-identical to the CPU runs above. */ #ifdef LARDON3D_MATCHER_TASK_VULKAN + /* Normal ORB is AUTO. The validated runtime must select GPU first, while + * retaining the canonical CPU durable estimate and identical evidence. */ + lardon3d_matcher_task_test_reset_backend_counters(); + Lardon3DMatcherTaskConfiguration automatic = settings; + automatic.matcher.ratio_threshold = 0.69F; + uint64_t automatic_id = 0; + CHECK(lardon3d_project_enqueue_matcher_task( + &fixture.state, &automatic, &automatic_id)); + CHECK(wait_state(fixture.state.task_queue, automatic_id, TASK_COMPLETED, + &snapshot)); + CHECK(lardon3d_matcher_task_test_vulkan_uses() >= 1); + Lardon3DResourceSequenceDiagnostic automatic_diagnostic; + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &automatic_diagnostic)); + CHECK(automatic_diagnostic.backend == + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + automatic_diagnostic.helper_limit == 0 && + automatic_diagnostic.gpu_slots == 1 && + automatic_diagnostic.inflight_limit == 1 && + automatic_diagnostic.gpu_memory_bytes == 640U * 1024U && + automatic_diagnostic.reason[0] != '\0' && + automatic_diagnostic.backend_reason[0] != '\0'); + if (automatic_diagnostic.actual_backend == + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN) { + CHECK(!automatic_diagnostic.backend_fallback && + automatic_diagnostic.execution.vulkan_submits >= 1 && + automatic_diagnostic.execution.vulkan_completions >= 1 && + automatic_diagnostic.execution.publication_ns > 0 && + strcmp(automatic_diagnostic.backend_reason, + "vulkan-completed") == 0); + } else { + /* Tiny fixture pairs are below the validated Vulkan threshold. They are + * complete CPU pairs inside a GPU-selected sequence, not backend failure. */ + CHECK(automatic_diagnostic.backend_fallback && + (automatic_diagnostic.actual_backend == + LARDON3D_RESOURCE_BACKEND_CPU || + automatic_diagnostic.actual_backend == + LARDON3D_RESOURCE_BACKEND_MIXED)); + } + + /* A successor rejected as locally ineligible has no pending handle. The + * first sequence must finish that complete pair on CPU, retain backend + * health, and leave an already-created AUTO Task eligible for GPU admission. */ + char ineligible_pair[32]; + CHECK(snprintf(ineligible_pair, sizeof(ineligible_pair), "%lu", + (unsigned long)fixture.pairs[1].candidate_pair_id) > 0 && + setenv("LARDON3D_TEST_MATCHER_INELIGIBLE_VULKAN_BEGIN_PAIR_ID", + ineligible_pair, 1) == 0 && + setenv("LARDON3D_TEST_MATCHER_PAUSE_AFTER_BATCH", "1", 1) == 0); + Lardon3DMatcherTaskConfiguration local_ineligible = settings; + local_ineligible.matcher.ratio_threshold = 0.687F; + uint64_t local_ineligible_id = 0; + Lardon3DTask *local_ineligible_task = + lardon3d_project_create_matcher_task_with_mode( + &fixture.state, &local_ineligible, + LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN, &local_ineligible_id); + Lardon3DMatcherTaskConfiguration auto_after_ineligible = settings; + auto_after_ineligible.matcher.ratio_threshold = 0.686F; + uint64_t auto_after_ineligible_id = 0; + Lardon3DTask *auto_after_ineligible_task = + lardon3d_project_create_matcher_task( + &fixture.state, &auto_after_ineligible, + &auto_after_ineligible_id); + CHECK(local_ineligible_task && auto_after_ineligible_task && + lardon3d_task_queue_add(fixture.state.task_queue, + local_ineligible_task, NULL) && + wait_state(fixture.state.task_queue, local_ineligible_id, + TASK_PAUSED, &snapshot)); + Lardon3DResourceSequenceDiagnostic ineligible_diagnostic; + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &ineligible_diagnostic) && + ineligible_diagnostic.backend == + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + ineligible_diagnostic.actual_backend == + LARDON3D_RESOURCE_BACKEND_MIXED && + ineligible_diagnostic.backend_fallback && + ineligible_diagnostic.execution.local_ineligible_fallback_items > 0 && + ineligible_diagnostic.execution.backend_failure_fallback_items == 0 && + ineligible_diagnostic.execution.backend_other_fallback_items == 0 && + strcmp(ineligible_diagnostic.backend_reason, + "vulkan-and-ineligible-pair-cpu-fallback") == 0); + CHECK(unsetenv("LARDON3D_TEST_MATCHER_INELIGIBLE_VULKAN_BEGIN_PAIR_ID") == 0 && + unsetenv("LARDON3D_TEST_MATCHER_PAUSE_AFTER_BATCH") == 0 && + lardon3d_task_queue_resume(fixture.state.task_queue, + local_ineligible_id) && + wait_state(fixture.state.task_queue, local_ineligible_id, + TASK_COMPLETED, &snapshot) && + lardon3d_task_queue_add(fixture.state.task_queue, + auto_after_ineligible_task, NULL) && + wait_state(fixture.state.task_queue, auto_after_ineligible_id, + TASK_COMPLETED, &snapshot)); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &ineligible_diagnostic) && + ineligible_diagnostic.backend == + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + ineligible_diagnostic.gpu_slots == 1); + + /* Genuine begin failure is learned before publication. Even when that + * publication then fails, shared AUTO admission must already be disabled; + * the pre-created probe therefore selects its complete CPU capability. */ + char backend_failure_pair[32]; + char early_publication_pair[32]; + CHECK(snprintf(backend_failure_pair, sizeof(backend_failure_pair), "%lu", + (unsigned long)fixture.pairs[1].candidate_pair_id) > 0 && + snprintf(early_publication_pair, sizeof(early_publication_pair), "%lu", + (unsigned long)fixture.pairs[0].candidate_pair_id) > 0 && + setenv("LARDON3D_TEST_MATCHER_FAIL_VULKAN_BEGIN_PAIR_ID", + backend_failure_pair, 1) == 0 && + setenv("LARDON3D_TEST_MATCHER_FAIL_PUBLISH_PAIR_ID", + early_publication_pair, 1) == 0); + Lardon3DMatcherTaskConfiguration early_failure = settings; + early_failure.matcher.ratio_threshold = 0.685F; + uint64_t early_failure_id = 0; + Lardon3DTask *early_failure_task = + lardon3d_project_create_matcher_task_with_mode( + &fixture.state, &early_failure, + LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN, &early_failure_id); + Lardon3DMatcherTaskConfiguration auto_after_failure = settings; + auto_after_failure.matcher.ratio_threshold = 0.684F; + uint64_t auto_after_failure_id = 0; + Lardon3DTask *auto_after_failure_task = + lardon3d_project_create_matcher_task( + &fixture.state, &auto_after_failure, &auto_after_failure_id); + size_t early_failure_results_before = 0; + size_t early_failure_results_after = 0; + CHECK(early_failure_task && auto_after_failure_task && + count_results(&fixture, &early_failure_results_before) && + lardon3d_task_queue_add(fixture.state.task_queue, + early_failure_task, NULL) && + wait_state(fixture.state.task_queue, early_failure_id, + TASK_FAILED, &snapshot) && + unsetenv("LARDON3D_TEST_MATCHER_FAIL_VULKAN_BEGIN_PAIR_ID") == 0 && + unsetenv("LARDON3D_TEST_MATCHER_FAIL_PUBLISH_PAIR_ID") == 0 && + count_results(&fixture, &early_failure_results_after) && + early_failure_results_after == early_failure_results_before && + no_staged_match_temporaries(&fixture) && + lardon3d_task_queue_add(fixture.state.task_queue, + auto_after_failure_task, NULL) && + wait_state(fixture.state.task_queue, auto_after_failure_id, + TASK_COMPLETED, &snapshot)); + Lardon3DResourceSequenceDiagnostic after_failure_diagnostic; + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &after_failure_diagnostic) && + after_failure_diagnostic.backend == LARDON3D_RESOURCE_BACKEND_CPU && + after_failure_diagnostic.gpu_slots == 0 && + lardon3d_resource_governor_internal_set_backend_available( + fixture.state.resource_governor, + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, true)); + + /* Force only the private admission value, not the execution algorithm. The + * production rolling owner must submit two exact requests before finishing + * the oldest, then preserve ordered one-evidence publication and output + * cardinality without timing sleeps. */ + CHECK(setenv("LARDON3D_TEST_MATCHER_INFLIGHT_LIMIT", "2", 1) == 0); + lardon3d_matcher_task_test_reset_backend_counters(); + Lardon3DMatcherTaskConfiguration depth_two = settings; + depth_two.matcher.ratio_threshold = 0.683F; + uint64_t depth_two_id = 0; + size_t depth_two_before = 0; + size_t depth_two_after = 0; + CHECK(count_results(&fixture, &depth_two_before) && + lardon3d_project_enqueue_matcher_task( + &fixture.state, &depth_two, &depth_two_id) && + wait_state(fixture.state.task_queue, depth_two_id, TASK_COMPLETED, + &snapshot) && + unsetenv("LARDON3D_TEST_MATCHER_INFLIGHT_LIMIT") == 0 && + count_results(&fixture, &depth_two_after) && + depth_two_after == depth_two_before + PERSISTED_PAIR_COUNT); + size_t depth_submit_first = 0; + size_t depth_submit_second = 0; + size_t depth_finish_first = 0; + size_t depth_publication_first = 0; + CHECK(find_test_event(TEST_EVENT_GPU_SUBMIT, + fixture.pairs[0].candidate_pair_id, + &depth_submit_first) && + find_test_event(TEST_EVENT_GPU_SUBMIT, + fixture.pairs[1].candidate_pair_id, + &depth_submit_second) && + find_test_event(TEST_EVENT_GPU_FINISH, + fixture.pairs[0].candidate_pair_id, + &depth_finish_first) && + find_test_event(TEST_EVENT_PUBLICATION_START, + fixture.pairs[0].candidate_pair_id, + &depth_publication_first)); + Lardon3DResourceSequenceDiagnostic depth_diagnostic; + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &depth_diagnostic) && + depth_diagnostic.backend == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + depth_diagnostic.inflight_limit == 2 && + depth_diagnostic.gpu_memory_bytes == 2 * 640 * 1024); + Lardon3DOrbVulkanInfo depth_backend_info; + CHECK(lardon3d_matcher_task_test_max_retained_vulkan_payload() == + 2 * 640 * 1024 + && lardon3d_orb_vulkan_backend_info( + fixture.state.orb_vulkan_backend, &depth_backend_info) + && depth_backend_info.permanent_payload_bytes == 640 * 1024); + CHECK(depth_submit_first < depth_submit_second); + CHECK(depth_submit_second < depth_finish_first); + CHECK(depth_finish_first < depth_publication_first); + CHECK(wait_candidate_results_have_one_evidence( + database_path, fixture.pairs[0].candidate_pair_id)); + lardon3d_matcher_task_test_reset_backend_counters(); Lardon3DMatcherTaskConfiguration gpu = settings; gpu.matcher.ratio_threshold = 0.68F; uint64_t gpu_id = 0; + size_t gpu_results_before = 0; + CHECK(count_results(&fixture, &gpu_results_before)); CHECK(lardon3d_project_enqueue_matcher_task_with_mode( &fixture.state, &gpu, LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN, &gpu_id)); CHECK(wait_state(fixture.state.task_queue, gpu_id, TASK_COMPLETED, &snapshot)); @@ -633,8 +2021,202 @@ static bool run_test(void) { CHECK(wait_durable_state(fixture.state.project_db, gpu_id, TASK_COMPLETED, &gpu_durable)); CHECK(lardon3d_matcher_task_test_vulkan_uses() >= 1); + Lardon3DOrbVulkanInfo fixed_backend_info; + CHECK(lardon3d_matcher_task_test_max_retained_vulkan_payload() == + 640 * 1024 + && lardon3d_orb_vulkan_backend_info( + fixture.state.orb_vulkan_backend, &fixed_backend_info) + && fixed_backend_info.permanent_payload_bytes == 640 * 1024); + size_t gpu_results_after = 0; + CHECK(count_results(&fixture, &gpu_results_after) && + gpu_results_after == gpu_results_before + PERSISTED_PAIR_COUNT); + size_t submit_first = 0, finish_first = 0, submit_second = 0; + size_t publication_start = 0, publication_finish = 0; + CHECK(lardon3d_matcher_task_test_overlap_publications() >= 1 && + find_test_event(TEST_EVENT_GPU_SUBMIT, + fixture.pairs[0].candidate_pair_id, &submit_first) && + find_test_event(TEST_EVENT_GPU_FINISH, + fixture.pairs[0].candidate_pair_id, &finish_first) && + find_test_event(TEST_EVENT_GPU_SUBMIT, + fixture.pairs[1].candidate_pair_id, &submit_second) && + find_test_event(TEST_EVENT_PUBLICATION_START, + fixture.pairs[0].candidate_pair_id, + &publication_start) && + find_test_event(TEST_EVENT_PUBLICATION_FINISH, + fixture.pairs[0].candidate_pair_id, + &publication_finish)); + CHECK(submit_first < finish_first && finish_first < submit_second && + submit_second < publication_start && + publication_start < publication_finish); CHECK(candidate_results_have_one_evidence( database_path, fixture.pairs[0].candidate_pair_id)); + + /* A rejected successor begin has no GPU event or partial evidence; that + * pair runs wholly on CPU and every exact backend slot remains reusable. */ + char failed_begin_pair[32]; + CHECK(snprintf(failed_begin_pair, sizeof(failed_begin_pair), "%lu", + (unsigned long)fixture.pairs[1].candidate_pair_id) > 0 && + setenv("LARDON3D_TEST_MATCHER_FAIL_VULKAN_BEGIN_PAIR_ID", + failed_begin_pair, 1) == 0); + lardon3d_matcher_task_test_reset_backend_counters(); + Lardon3DMatcherTaskConfiguration begin_fallback = settings; + begin_fallback.matcher.ratio_threshold = 0.665F; + uint64_t begin_fallback_id = 0; + CHECK(count_results(&fixture, &gpu_results_before) && + lardon3d_project_enqueue_matcher_task_with_mode( + &fixture.state, &begin_fallback, + LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN, &begin_fallback_id) && + wait_state(fixture.state.task_queue, begin_fallback_id, + TASK_COMPLETED, &snapshot) && + unsetenv("LARDON3D_TEST_MATCHER_FAIL_VULKAN_BEGIN_PAIR_ID") == 0 && + count_results(&fixture, &gpu_results_after) && + gpu_results_after == gpu_results_before + PERSISTED_PAIR_COUNT); + CHECK(!find_test_event(TEST_EVENT_GPU_SUBMIT, + fixture.pairs[1].candidate_pair_id, &submit_second)); + CHECK(lardon3d_matcher_task_test_overlap_publications() == 0); + + /* Publication failure occurs after successor submission. Failure cleanup + * must discard that private request so a following Task can reuse the slot. */ + char failed_publish_pair[32]; + CHECK(snprintf(failed_publish_pair, sizeof(failed_publish_pair), "%lu", + (unsigned long)fixture.pairs[0].candidate_pair_id) > 0 && + setenv("LARDON3D_TEST_MATCHER_FAIL_PUBLISH_PAIR_ID", + failed_publish_pair, 1) == 0 && + setenv("LARDON3D_TEST_MATCHER_INFLIGHT_LIMIT", "2", 1) == 0); + lardon3d_matcher_task_test_reset_backend_counters(); + Lardon3DMatcherTaskConfiguration failed_publication = settings; + failed_publication.matcher.ratio_threshold = 0.655F; + uint64_t failed_publication_id = 0; + CHECK(count_results(&fixture, &gpu_results_before)); + CHECK(lardon3d_project_enqueue_matcher_task( + &fixture.state, &failed_publication, &failed_publication_id)); + CHECK(wait_state(fixture.state.task_queue, failed_publication_id, + TASK_FAILED, &snapshot)); + CHECK(unsetenv("LARDON3D_TEST_MATCHER_FAIL_PUBLISH_PAIR_ID") == 0); + CHECK(unsetenv("LARDON3D_TEST_MATCHER_INFLIGHT_LIMIT") == 0); + CHECK(count_results(&fixture, &gpu_results_after)); + CHECK(gpu_results_after == gpu_results_before); + CHECK(lardon3d_matcher_task_test_overlap_publications() == 1); + CHECK(no_staged_match_temporaries(&fixture)); + lardon3d_matcher_task_test_reset_backend_counters(); + Lardon3DMatcherTaskConfiguration after_publish_failure = settings; + after_publish_failure.matcher.ratio_threshold = 0.645F; + uint64_t after_publish_failure_id = 0; + CHECK(lardon3d_project_enqueue_matcher_task_with_mode( + &fixture.state, &after_publish_failure, + LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN, + &after_publish_failure_id) && + wait_state(fixture.state.task_queue, after_publish_failure_id, + TASK_COMPLETED, &snapshot) && + lardon3d_matcher_task_test_vulkan_uses() >= 2); + + /* At depth two, local ineligibility of the oldest pair consumes no slot, so + * the successor is already pending when its whole-pair CPU fallback fails. + * The Task owner must discard that successor before sequence cleanup can + * shrink the mapped payload and release the Governor reservation. */ + char failed_cpu_fallback_pair[32]; + CHECK(snprintf(failed_cpu_fallback_pair, + sizeof(failed_cpu_fallback_pair), "%lu", + (unsigned long)fixture.pairs[0].candidate_pair_id) > 0); + Lardon3DOrbVulkanTelemetry cleanup_before = {0}; + Lardon3DOrbVulkanTelemetry cleanup_after = {0}; + CHECK(lardon3d_orb_vulkan_internal_telemetry( + fixture.state.orb_vulkan_backend, &cleanup_before) && + setenv("LARDON3D_TEST_MATCHER_INELIGIBLE_VULKAN_BEGIN_PAIR_ID", + failed_cpu_fallback_pair, 1) == 0 && + setenv("LARDON3D_TEST_MATCHER_FAIL_COMPUTE_PAIR_ID", + failed_cpu_fallback_pair, 1) == 0 && + setenv("LARDON3D_TEST_MATCHER_INFLIGHT_LIMIT", "2", 1) == 0); + lardon3d_matcher_task_test_reset_backend_counters(); + Lardon3DMatcherTaskConfiguration failed_cpu_fallback = settings; + failed_cpu_fallback.matcher.ratio_threshold = 0.643F; + uint64_t failed_cpu_fallback_id = 0; + size_t failed_cpu_fallback_before = 0; + size_t failed_cpu_fallback_after = 0; + CHECK(count_results(&fixture, &failed_cpu_fallback_before) && + lardon3d_project_enqueue_matcher_task( + &fixture.state, &failed_cpu_fallback, + &failed_cpu_fallback_id) && + wait_state(fixture.state.task_queue, failed_cpu_fallback_id, + TASK_FAILED, &snapshot) && + unsetenv( + "LARDON3D_TEST_MATCHER_INELIGIBLE_VULKAN_BEGIN_PAIR_ID") == 0 && + unsetenv("LARDON3D_TEST_MATCHER_FAIL_COMPUTE_PAIR_ID") == 0 && + unsetenv("LARDON3D_TEST_MATCHER_INFLIGHT_LIMIT") == 0 && + count_results(&fixture, &failed_cpu_fallback_after) && + failed_cpu_fallback_after == failed_cpu_fallback_before && + no_staged_match_temporaries(&fixture)); + size_t failed_cpu_successor_submit = 0; + Lardon3DOrbVulkanInfo cleaned_backend_info = {0}; + CHECK(find_test_event(TEST_EVENT_GPU_SUBMIT, + fixture.pairs[1].candidate_pair_id, + &failed_cpu_successor_submit) && + lardon3d_matcher_task_test_max_retained_vulkan_payload() == + 2 * 640 * 1024 && + lardon3d_orb_vulkan_internal_telemetry( + fixture.state.orb_vulkan_backend, &cleanup_after) && + cleanup_after.discards == cleanup_before.discards + 1 && + cleanup_after.pending_slots == 0 && + !cleanup_after.slot_pending && + cleanup_after.retained_capacity == 1 && + cleanup_after.retained_payload_bytes == 640 * 1024 && + !cleanup_after.sequence_capacity_active && + lardon3d_orb_vulkan_backend_info( + fixture.state.orb_vulkan_backend, &cleaned_backend_info) && + cleaned_backend_info.available && + cleaned_backend_info.permanent_payload_bytes == 640 * 1024); + + lardon3d_matcher_task_test_reset_backend_counters(); + Lardon3DMatcherTaskConfiguration after_cpu_fallback_failure = settings; + after_cpu_fallback_failure.matcher.ratio_threshold = 0.642F; + uint64_t after_cpu_fallback_failure_id = 0; + CHECK(lardon3d_project_enqueue_matcher_task_with_mode( + &fixture.state, &after_cpu_fallback_failure, + LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN, + &after_cpu_fallback_failure_id) && + wait_state(fixture.state.task_queue, + after_cpu_fallback_failure_id, TASK_COMPLETED, &snapshot) && + lardon3d_matcher_task_test_vulkan_uses() >= 2 && + count_results(&fixture, &gpu_results_after) && + gpu_results_after == + failed_cpu_fallback_before + PERSISTED_PAIR_COUNT && + wait_candidate_results_have_one_evidence( + database_path, fixture.pairs[0].candidate_pair_id) && + lardon3d_orb_vulkan_internal_telemetry( + fixture.state.orb_vulkan_backend, &cleanup_after) && + cleanup_after.pending_slots == 0 && + cleanup_after.retained_capacity == 1 && + !cleanup_after.sequence_capacity_active); + + /* Cancellation while predecessor publication is paused owns a submitted + * successor. The sequence-exit path discards it, and reuse is immediate. */ + CHECK(setenv("LARDON3D_TEST_MATCHER_PAUSE_AFTER_PUBLICATION", "1", 1) == 0); + lardon3d_matcher_task_test_reset_backend_counters(); + Lardon3DMatcherTaskConfiguration cancelled_gpu = settings; + cancelled_gpu.matcher.ratio_threshold = 0.635F; + uint64_t cancelled_gpu_id = 0; + CHECK(lardon3d_project_enqueue_matcher_task_with_mode( + &fixture.state, &cancelled_gpu, + LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN, &cancelled_gpu_id) && + wait_state(fixture.state.task_queue, cancelled_gpu_id, TASK_PAUSED, + &snapshot) && + lardon3d_task_queue_cancel(fixture.state.task_queue, + cancelled_gpu_id) && + wait_state(fixture.state.task_queue, cancelled_gpu_id, + TASK_CANCELLED, &snapshot) && + unsetenv("LARDON3D_TEST_MATCHER_PAUSE_AFTER_PUBLICATION") == 0 && + lardon3d_matcher_task_test_overlap_publications() == 1 && + no_staged_match_temporaries(&fixture)); + lardon3d_matcher_task_test_reset_backend_counters(); + Lardon3DMatcherTaskConfiguration after_cancel = settings; + after_cancel.matcher.ratio_threshold = 0.625F; + uint64_t after_cancel_id = 0; + CHECK(lardon3d_project_enqueue_matcher_task_with_mode( + &fixture.state, &after_cancel, + LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN, &after_cancel_id) && + wait_state(fixture.state.task_queue, after_cancel_id, TASK_COMPLETED, + &snapshot) && + lardon3d_matcher_task_test_vulkan_uses() >= 2); /* Runtime backend failure after valid GPU admission falls back through the * exact CPU implementation; it never changes the immutable estimate or the * scientific Match identity. */ @@ -653,6 +2235,17 @@ static bool run_test(void) { TASK_COMPLETED, &fallback_durable)); CHECK(lardon3d_matcher_task_test_forced_fallbacks() >= 1); CHECK(lardon3d_matcher_task_test_vulkan_uses() == 0); + Lardon3DResourceSequenceDiagnostic fallback_diagnostic; + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &fallback_diagnostic) && + fallback_diagnostic.backend == + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + fallback_diagnostic.actual_backend == LARDON3D_RESOURCE_BACKEND_CPU && + fallback_diagnostic.backend_fallback && + fallback_diagnostic.helper_limit == 0 && + strcmp(fallback_diagnostic.backend_reason, + "vulkan-failed-whole-pair-cpu-fallback") == 0); CHECK(unsetenv("LARDON3D_TEST_MATCHER_FORCE_FALLBACK") == 0); CHECK(candidate_results_have_one_evidence( database_path, fixture.pairs[0].candidate_pair_id)); @@ -677,8 +2270,9 @@ static bool run_test(void) { Lardon3DMatcherTaskConfiguration reduced = settings; reduced.matcher.ratio_threshold = 0.83F; uint64_t reduced_id = 0; - CHECK(lardon3d_project_enqueue_matcher_task(&fixture.state, &reduced, - &reduced_id)); + CHECK(lardon3d_project_enqueue_matcher_task_with_mode( + &fixture.state, &reduced, LARDON3D_MATCHER_TASK_MODE_CPU_PARALLEL, + &reduced_id)); CHECK(wait_state(fixture.state.task_queue, reduced_id, TASK_COMPLETED, &snapshot)); Lardon3DResourcePolicy restored_policy = interactive_policy(); @@ -730,12 +2324,17 @@ static bool run_test(void) { Lardon3DTaskDurableSnapshot estimate_snapshot; CHECK(estimate_task != NULL && estimate_task_id != 0); CHECK(lardon3d_task_durable_snapshot(estimate_task, &estimate_snapshot)); - /* Parallel Matcher execution is permanently CPU-only even if admission - * later grants one thread; its immutable estimate must reserve no GPU. */ + /* AUTO backend choice is private runtime state. MIXED truthfully records + * that the operation may consume CPU or Vulkan without persisting either + * backend choice. Explicit CPU retains the exact CPU-class signature. */ CHECK(estimate_snapshot.estimate.desired_cpu_threads > 1); CHECK(estimate_snapshot.estimate.gpu_memory_fixed_bytes == 0); CHECK(estimate_snapshot.estimate.desired_gpu_slots == 0); - Lardon3DTaskDurableSnapshot current_cpu = estimate_snapshot; + CHECK(estimate_snapshot.estimate.task_class == + LARDON3D_RESOURCE_TASK_MIXED); + Lardon3DTaskDurableSnapshot current_auto = estimate_snapshot; + Lardon3DTaskDurableSnapshot current_cpu = current_auto; + current_cpu.estimate.task_class = LARDON3D_RESOURCE_TASK_CPU; Lardon3DTaskDurableSnapshot current_gpu = current_cpu; current_gpu.estimate.gpu_memory_fixed_bytes = LARDON3D_ORB_VULKAN_PERMANENT_BUFFER_BYTES; @@ -744,6 +2343,25 @@ static bool run_test(void) { lardon3d_task_destroy(estimate_task); estimate_task = NULL; + uint64_t explicit_cpu_id = 0; + Lardon3DTask *explicit_cpu_task = + lardon3d_project_create_matcher_task_with_mode( + &fixture.state, &settings, LARDON3D_MATCHER_TASK_MODE_CPU_PARALLEL, + &explicit_cpu_id); + Lardon3DTaskDurableSnapshot explicit_cpu_snapshot; + CHECK(explicit_cpu_task && explicit_cpu_id != current_auto.id && + lardon3d_task_durable_snapshot(explicit_cpu_task, + &explicit_cpu_snapshot)); + CHECK(same_estimate(&explicit_cpu_snapshot.estimate, + ¤t_cpu.estimate)); + current_cpu = explicit_cpu_snapshot; + current_gpu = current_cpu; + current_gpu.estimate.gpu_memory_fixed_bytes = + LARDON3D_ORB_VULKAN_PERMANENT_BUFFER_BYTES; + current_gpu.estimate.desired_cpu_threads = 1; + current_gpu.estimate.desired_gpu_slots = 1; + lardon3d_task_destroy(explicit_cpu_task); + #ifdef LARDON3D_MATCHER_TASK_VULKAN estimate_task = lardon3d_project_create_matcher_task_with_mode( &fixture.state, &settings, LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN, @@ -751,10 +2369,11 @@ static bool run_test(void) { CHECK(estimate_task != NULL && lardon3d_task_durable_snapshot(estimate_task, &estimate_snapshot)); CHECK(same_estimate(&estimate_snapshot.estimate, ¤t_gpu.estimate)); + current_gpu = estimate_snapshot; #endif - /* Selection failures occur before Task-ID allocation and cannot silently - * become CPU work. The existing create API remains CPU-parallel. */ + /* Explicit selection failures occur before Task-ID allocation and cannot + * silently become CPU work. Normal create remains AUTO. */ Lardon3DMatcherTaskConfiguration wrong_kind = settings; wrong_kind.matcher.kind = LARDON3D_MATCHER_SIFT_BF; (void)snprintf(wrong_kind.feature_extractor_kind, @@ -776,6 +2395,21 @@ static bool run_test(void) { &fixture.state, &settings, LARDON3D_MATCHER_TASK_MODE_ORB_VULKAN, &rejected_id) == NULL && rejected_id == 0); + lardon3d_matcher_task_test_reset_backend_counters(); + Lardon3DMatcherTaskConfiguration no_gpu_auto = settings; + no_gpu_auto.matcher.ratio_threshold = 0.63F; + uint64_t no_gpu_id = 0; + CHECK(lardon3d_project_enqueue_matcher_task( + &fixture.state, &no_gpu_auto, &no_gpu_id)); + CHECK(wait_state(fixture.state.task_queue, no_gpu_id, TASK_COMPLETED, + &snapshot)); + CHECK(lardon3d_matcher_task_test_vulkan_uses() == 0); + Lardon3DResourceSequenceDiagnostic cpu_diagnostic; + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &cpu_diagnostic)); + CHECK(cpu_diagnostic.backend == LARDON3D_RESOURCE_BACKEND_CPU && + cpu_diagnostic.gpu_slots == 0); fixture.state.hardware_profile.gpu_available = gpu_available; Lardon3DOrbVulkanBackend *backend = fixture.state.orb_vulkan_backend; fixture.state.orb_vulkan_backend = NULL; @@ -786,23 +2420,94 @@ static bool run_test(void) { rejected_id == 0); fixture.state.orb_vulkan_backend = backend; - /* Reconstruction accepts only the two current and two historical complete - * shapes. CPU12 signatures are normalized ephemerally for Task admission; - * neighboring shapes are corruption, not mode inference. */ + /* Reconstruction accepts exact current CPU12 forms plus the preceding + * CPU8-per-pair and oldest fixed-memory CPU12 forms. All historic resource + * policies normalize only in memory; neighboring shapes stay corruption. */ Lardon3DTaskReconstructionContext reconstruction = { .project_path = fixture.state.project_path, .project_db = fixture.state.project_db, .resource_governor = fixture.state.resource_governor, .orb_vulkan_backend = fixture.state.orb_vulkan_backend, }; - Lardon3DTaskDurableSnapshot historical_cpu = current_cpu; - historical_cpu.estimate.memory_fixed_bytes = 10U * 1024U * 1024U; - historical_cpu.estimate.memory_bytes_per_item = 0; - historical_cpu.estimate.desired_cpu_threads = 12; - Lardon3DTaskDurableSnapshot historical_gpu = current_gpu; - historical_gpu.estimate.memory_fixed_bytes = 10U * 1024U * 1024U; - historical_gpu.estimate.memory_bytes_per_item = 0; - historical_gpu.estimate.desired_cpu_threads = 12; + + /* Restart preserves the two normal/override policies without a schema or + * codec field. On a Vulkan build the restored AUTO signature selects GPU, + * while the restored explicit CPU signature remains fixed CPU. */ + Lardon3DTask *restarted_auto = NULL; + CHECK(lardon3d_task_kind_registry_restore( + lardon3d_task_kind_registry_production(), + LARDON3D_MATCHER_TASK_KIND, LARDON3D_MATCHER_TASK_KIND_VERSION, + ¤t_auto, &reconstruction, &restarted_auto) == + LARDON3D_TASK_KIND_OK && + restarted_auto); + Lardon3DTaskDurableSnapshot order_historical_cpu = current_cpu; + order_historical_cpu.estimate.memory_fixed_bytes = 10U * 1024U * 1024U; + order_historical_cpu.estimate.memory_bytes_per_item = 0; + order_historical_cpu.estimate.desired_cpu_threads = 12; + order_historical_cpu.estimate.maximum_batch_size = 8; + Lardon3DTaskDurableSnapshot order_historical_gpu = current_gpu; + order_historical_gpu.estimate.memory_fixed_bytes = 10U * 1024U * 1024U; + order_historical_gpu.estimate.memory_bytes_per_item = 0; + order_historical_gpu.estimate.desired_cpu_threads = 12; + order_historical_gpu.estimate.maximum_batch_size = 8; + const Lardon3DTaskDurableSnapshot *fixed_restore_order[] = { + ¤t_cpu, + ¤t_gpu, + &order_historical_cpu, + &order_historical_gpu, + }; + /* AUTO establishes shared eligibility. Later co-restoration of fixed CPU, + * fixed Vulkan, and historical forms is order-independent and must not + * clear it before the already-restored AUTO Task reaches admission. */ + for (size_t index = 0; + index < sizeof(fixed_restore_order) / sizeof(fixed_restore_order[0]); + ++index) { + Lardon3DTask *fixed_restored = NULL; + CHECK(lardon3d_task_kind_registry_restore( + lardon3d_task_kind_registry_production(), + LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, fixed_restore_order[index], + &reconstruction, &fixed_restored) == LARDON3D_TASK_KIND_OK && + fixed_restored); + lardon3d_task_destroy(fixed_restored); + } + CHECK(lardon3d_task_queue_add(fixture.state.task_queue, restarted_auto, NULL)); + CHECK(wait_state(fixture.state.task_queue, current_auto.id, TASK_COMPLETED, + &snapshot)); + Lardon3DResourceSequenceDiagnostic restart_diagnostic; + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &restart_diagnostic)); +#ifdef LARDON3D_MATCHER_TASK_VULKAN + CHECK(restart_diagnostic.backend == + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + restart_diagnostic.gpu_slots == 1); +#else + CHECK(restart_diagnostic.backend == LARDON3D_RESOURCE_BACKEND_CPU && + restart_diagnostic.gpu_slots == 0); +#endif + Lardon3DTask *restarted_cpu = NULL; + CHECK(lardon3d_task_kind_registry_restore( + lardon3d_task_kind_registry_production(), + LARDON3D_MATCHER_TASK_KIND, LARDON3D_MATCHER_TASK_KIND_VERSION, + ¤t_cpu, &reconstruction, &restarted_cpu) == + LARDON3D_TASK_KIND_OK && + restarted_cpu); + CHECK(lardon3d_task_queue_add(fixture.state.task_queue, restarted_cpu, NULL)); + CHECK(wait_state(fixture.state.task_queue, current_cpu.id, TASK_COMPLETED, + &snapshot)); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + fixture.state.resource_governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &restart_diagnostic)); + CHECK(restart_diagnostic.backend == LARDON3D_RESOURCE_BACKEND_CPU && + restart_diagnostic.gpu_slots == 0); + Lardon3DTaskDurableSnapshot historical_cpu = order_historical_cpu; + Lardon3DTaskDurableSnapshot historical_gpu = order_historical_gpu; + Lardon3DTaskDurableSnapshot previous_cpu = current_cpu; + previous_cpu.estimate.maximum_batch_size = 8; + previous_cpu.estimate.desired_cpu_threads = 8; + Lardon3DTaskDurableSnapshot previous_gpu = current_gpu; + previous_gpu.estimate.maximum_batch_size = 8; char reconciled_path[PATH_MAX]; CHECK(snprintf(reconciled_path, sizeof(reconciled_path), "%s/.lardon3d/checkpoints/%lu.chk", @@ -811,7 +2516,8 @@ static bool run_test(void) { lardon3d_task_checkpoint_save(reconciled_path, &historical_gpu) == LARDON3D_TASK_CHECKPOINT_OK); const Lardon3DTaskDurableSnapshot *accepted[] = { - ¤t_cpu, ¤t_gpu, &historical_cpu, &historical_gpu}; + ¤t_auto, ¤t_cpu, ¤t_gpu, &previous_cpu, &previous_gpu, + &historical_cpu, &historical_gpu}; for (size_t index = 0; index < sizeof(accepted) / sizeof(accepted[0]); ++index) { Lardon3DTask *restored = NULL; CHECK(lardon3d_task_kind_registry_restore( @@ -823,9 +2529,9 @@ static bool run_test(void) { Lardon3DResourceEstimate restored_estimate; CHECK(lardon3d_task_resource_estimate(restored, &restored_estimate)); const Lardon3DResourceEstimate *expected = - index == 2 ? ¤t_cpu.estimate - : index == 3 ? ¤t_gpu.estimate - : &accepted[index]->estimate; + index == 3 || index == 5 ? ¤t_cpu.estimate + : index == 4 || index == 6 ? ¤t_gpu.estimate + : &accepted[index]->estimate; CHECK(same_estimate(&restored_estimate, expected)); lardon3d_task_destroy(restored); } @@ -840,12 +2546,16 @@ static bool run_test(void) { CHECK(snprintf(staged_path, sizeof(staged_path), "%s.next", reconciled_path) > 0 && access(staged_path, F_OK) != 0 && errno == ENOENT); - Lardon3DTaskDurableSnapshot malformed[4] = { - historical_cpu, historical_cpu, historical_gpu, historical_gpu}; + Lardon3DTaskDurableSnapshot malformed[5] = { + historical_cpu, historical_cpu, historical_gpu, historical_gpu, + current_gpu}; malformed[0].estimate.memory_fixed_bytes--; malformed[1].estimate.memory_bytes_per_item = 1; malformed[2].estimate.gpu_memory_fixed_bytes++; malformed[3].estimate.desired_cpu_threads = 11; + /* MIXED is truthful only for the CPU-shaped durable AUTO policy. A Vulkan + * resource shape cannot use the class as a synthetic backend tag. */ + malformed[4].estimate.task_class = LARDON3D_RESOURCE_TASK_MIXED; for (size_t index = 0; index < sizeof(malformed) / sizeof(malformed[0]); ++index) { Lardon3DTask *restored = NULL; CHECK(lardon3d_task_kind_registry_restore( @@ -859,7 +2569,88 @@ static bool run_test(void) { stop_runtime(&fixture); CHECK(remove_tree(fixture.root)); +#if defined(LARDON3D_MATCHER_TASK_VULKAN) && \ + defined(LARDON3D_MATCHER_TASK_BENCHMARK_PIPELINE) + /* Distinct fresh Projects prevent identity reuse from hiding either control. + * Runner-private tokens fix one normal Governor-admitted Vulkan capability + * for the depth and batch matrices. Synchronous remains the historical + * depth-one/batch-two fence baseline; all scientific fields stay exact. */ + PipelineEvidence rolling_one = {0}; + PipelineEvidence rolling_two = {0}; + PipelineEvidence batch_four = {0}; + PipelineEvidence batch_eight = {0}; + PipelineEvidence batch_twelve = {0}; + PipelineEvidence synchronous = {0}; + CHECK(capture_pipeline_evidence(false, 1, 2, &rolling_one)); + CHECK(capture_pipeline_evidence(false, 2, 2, &rolling_two)); + CHECK(capture_pipeline_evidence(false, 1, 4, &batch_four)); + CHECK(capture_pipeline_evidence(false, 1, 8, &batch_eight)); + CHECK(capture_pipeline_evidence(false, 1, 12, &batch_twelve)); + CHECK(capture_pipeline_evidence(true, 1, 0, &synchronous)); + CHECK(rolling_one.overlap_publications >= 1); + CHECK(rolling_two.overlap_publications >= 1); + CHECK(batch_four.overlap_publications >= 1); + CHECK(batch_eight.overlap_publications >= 1); + CHECK(batch_twelve.overlap_publications >= 1); + CHECK(synchronous.overlap_publications == 0); + CHECK(same_forced_contract_except_depth(&rolling_one, &rolling_two)); + CHECK(same_forced_contract_except_batch(&rolling_one, &batch_four, 4)); + CHECK(same_forced_contract_except_batch(&rolling_one, &batch_eight, 8)); + CHECK(same_forced_contract_except_batch(&rolling_one, &batch_twelve, 12)); + /* Sequence grouping is deliberately different at each fixed batch. The + * production Matcher-owned item comparator must nevertheless count the same + * 29 locally ineligible pairs exactly once in every cohort. */ + CHECK(rolling_one.aggregate.local_ineligible_fallback_items == + PERSISTED_PAIR_COUNT - PIPELINE_ELIGIBLE_PAIR_COUNT); + CHECK(rolling_two.aggregate.local_ineligible_fallback_items == + rolling_one.aggregate.local_ineligible_fallback_items); + CHECK(batch_four.aggregate.local_ineligible_fallback_items == + rolling_one.aggregate.local_ineligible_fallback_items); + CHECK(batch_eight.aggregate.local_ineligible_fallback_items == + rolling_one.aggregate.local_ineligible_fallback_items); + CHECK(batch_twelve.aggregate.local_ineligible_fallback_items == + rolling_one.aggregate.local_ineligible_fallback_items); + CHECK(synchronous.aggregate.local_ineligible_fallback_items == + rolling_one.aggregate.local_ineligible_fallback_items); + CHECK(rolling_one.aggregate.backend_ineligible_fallback_sequences != + batch_four.aggregate.backend_ineligible_fallback_sequences); + CHECK(batch_four.aggregate.backend_ineligible_fallback_sequences != + batch_eight.aggregate.backend_ineligible_fallback_sequences); + CHECK(same_pipeline_scientific_output(&rolling_one, &rolling_two)); + CHECK(same_pipeline_scientific_output(&rolling_one, &batch_four)); + CHECK(same_pipeline_scientific_output(&rolling_one, &batch_eight)); + CHECK(same_pipeline_scientific_output(&rolling_one, &batch_twelve)); + CHECK(same_pipeline_scientific_output(&rolling_two, &synchronous)); + CHECK(forced_benchmark_fail_closed()); + CHECK(pair_local_neighbor_failure_case( + "LARDON3D_TEST_MATCHER_FAIL_VULKAN_BEGIN_PAIR_ID")); + CHECK(pair_local_neighbor_failure_case( + "LARDON3D_TEST_MATCHER_FAIL_VULKAN_FINISH_PAIR_ID")); + CHECK(local_vulkan_failure_preserves_backend_case( + "LARDON3D_TEST_MATCHER_FAIL_LOCAL_VULKAN_BEGIN_PAIR_ID", true)); + CHECK(local_vulkan_failure_preserves_backend_case( + "LARDON3D_TEST_MATCHER_FAIL_LOCAL_VULKAN_FINISH_PAIR_ID", false)); + CHECK(durable_fallback_prefix_case(false, PREFIX_EXIT_CANCEL)); + CHECK(durable_fallback_prefix_case( + false, PREFIX_EXIT_COMPUTE_FAILURE)); + CHECK(durable_fallback_prefix_case( + false, PREFIX_EXIT_PUBLICATION_FAILURE)); + CHECK(durable_fallback_prefix_case(true, PREFIX_EXIT_CANCEL)); + CHECK(durable_fallback_prefix_case( + true, PREFIX_EXIT_COMPUTE_FAILURE)); + CHECK(durable_fallback_prefix_case( + true, PREFIX_EXIT_PUBLICATION_FAILURE)); +#endif return true; } -int main(void) { return run_test() ? EXIT_SUCCESS : EXIT_FAILURE; } +int main(void) { + Lardon3DResourceDriverPolicyResult driver_policy = + lardon3d_resource_governor_internal_configure_driver_policy(); + return driver_policy != LARDON3D_RESOURCE_DRIVER_POLICY_FAILED && + driver_policy != + LARDON3D_RESOURCE_DRIVER_POLICY_REJECTED_UNSAFE && + run_test() + ? EXIT_SUCCESS + : EXIT_FAILURE; +} diff --git a/tests/test_orb_vulkan_backend.cpp b/tests/test_orb_vulkan_backend.cpp index 7e32fd5..5c575de 100644 --- a/tests/test_orb_vulkan_backend.cpp +++ b/tests/test_orb_vulkan_backend.cpp @@ -1,9 +1,12 @@ #include +#include "../src/orb_vulkan_backend_internal.h" + #include #include #include +#include #include #include #include @@ -115,24 +118,114 @@ static bool check_serialized_threads(Lardon3DOrbVulkanBackend *backend) { std::vector b = make_descriptors(1024, 0x22222222U); std::vector expected_a = opencv_top2(a, 1024, b, 1024); std::vector expected_b = opencv_top2(b, 1024, a, 1024); - std::vector actual_a(1024); - std::vector actual_b(1024); - Lardon3DOrbVulkanResult result_a = LARDON3D_ORB_VULKAN_FAILED; - Lardon3DOrbVulkanResult result_b = LARDON3D_ORB_VULKAN_FAILED; - std::thread thread_a([&] { - result_a = lardon3d_orb_vulkan_top2(backend, a.data(), 1024, b.data(), 1024, - actual_a.data(), actual_a.size()); - }); - std::thread thread_b([&] { - result_b = lardon3d_orb_vulkan_top2(backend, b.data(), 1024, a.data(), 1024, - actual_b.data(), actual_b.size()); - }); - thread_a.join(); - thread_b.join(); - return result_a == LARDON3D_ORB_VULKAN_OK && - result_b == LARDON3D_ORB_VULKAN_OK && - std::equal(expected_a.begin(), expected_a.end(), actual_a.begin(), equal_top2) && - std::equal(expected_b.begin(), expected_b.end(), actual_b.begin(), equal_top2); + for (unsigned int iteration = 0; iteration < 32; ++iteration) { + std::vector actual_a(1024); + std::vector actual_b(1024); + Lardon3DOrbVulkanResult result_a = LARDON3D_ORB_VULKAN_FAILED; + Lardon3DOrbVulkanResult result_b = LARDON3D_ORB_VULKAN_FAILED; + std::atomic ready{0}; + std::atomic start{false}; + auto await_start = [&] { + ready.fetch_add(1, std::memory_order_release); + while (!start.load(std::memory_order_acquire)) { + std::this_thread::yield(); + } + }; + std::thread thread_a([&] { + await_start(); + result_a = lardon3d_orb_vulkan_top2( + backend, a.data(), 1024, b.data(), 1024, actual_a.data(), + actual_a.size()); + }); + std::thread thread_b([&] { + await_start(); + result_b = lardon3d_orb_vulkan_top2( + backend, b.data(), 1024, a.data(), 1024, actual_b.data(), + actual_b.size()); + }); + while (ready.load(std::memory_order_acquire) != 2) { + std::this_thread::yield(); + } + start.store(true, std::memory_order_release); + thread_a.join(); + thread_b.join(); + /* Both calls start from the same gate and repeat enough times to exercise + * the public transaction boundary without timing sleeps. Each result is + * compared with its own directional CPU reference, catching cross-call + * request/output substitution as well as spurious pending-slot failure. */ + if (result_a != LARDON3D_ORB_VULKAN_OK || + result_b != LARDON3D_ORB_VULKAN_OK || + !std::equal(expected_a.begin(), expected_a.end(), actual_a.begin(), + equal_top2) || + !std::equal(expected_b.begin(), expected_b.end(), actual_b.begin(), + equal_top2)) { + return false; + } + } + return true; +} + +static bool check_driver_policy_case(const char *value, bool private_begin) { + if ((value && setenv("MESA_SHADER_CACHE_DISABLE", value, 1) != 0) || + (!value && unsetenv("MESA_SHADER_CACHE_DISABLE") != 0)) { + return false; + } + Lardon3DOrbVulkanBackend *backend = lardon3d_orb_vulkan_backend_create(); + if (!backend) return false; + unsigned char descriptor[kDescriptorBytes]{}; + Lardon3DOrbTop2 output{7, 11, 13, 17, 19}; + const Lardon3DOrbTop2 unchanged = output; + Lardon3DOrbVulkanInfo before{}; + Lardon3DOrbVulkanInfo after{}; + Lardon3DOrbVulkanRequest request{}; + bool ok = lardon3d_orb_vulkan_backend_info(backend, &before) && + !before.initialized && !before.available; + Lardon3DOrbVulkanResult result = private_begin + ? lardon3d_orb_vulkan_internal_top2_begin( + backend, descriptor, 1, descriptor, 1, &request) + : lardon3d_orb_vulkan_top2( + backend, descriptor, 1, descriptor, 1, &output, 1); + ok = ok && result == LARDON3D_ORB_VULKAN_UNAVAILABLE && + std::memcmp(&output, &unchanged, sizeof(output)) == 0 && + lardon3d_orb_vulkan_backend_info(backend, &after) && after.initialized && + !after.available; + const char *retained = std::getenv("MESA_SHADER_CACHE_DISABLE"); + ok = ok && ((!value && !retained) || + (value && retained && std::strcmp(value, retained) == 0)); + lardon3d_orb_vulkan_backend_destroy(backend); + return ok; +} + +static bool check_driver_policy_gate() { + /* Meson's safe test environment must not mask these late-boundary cases. + * Every case uses a fresh backend before any successful Vulkan request, so a + * failure proves rejection precedes Mesa rather than observing cached state. */ + bool ok = check_driver_policy_case(nullptr, false) && + check_driver_policy_case("false", true) && + check_driver_policy_case("malformed", false); + return setenv("MESA_SHADER_CACHE_DISABLE", "true", 1) == 0 && ok; +} + +static bool check_current_driver_policy_rejection() { + const char *value = std::getenv("MESA_SHADER_CACHE_DISABLE"); + if (value && (std::strcmp(value, "true") == 0 || + std::strcmp(value, "1") == 0)) { + return false; + } + Lardon3DOrbVulkanBackend *backend = lardon3d_orb_vulkan_backend_create(); + if (!backend) return false; + unsigned char descriptor[kDescriptorBytes]{}; + Lardon3DOrbTop2 output{7, 11, 13, 17, 19}; + const Lardon3DOrbTop2 unchanged = output; + Lardon3DOrbVulkanResult result = lardon3d_orb_vulkan_top2( + backend, descriptor, 1, descriptor, 1, &output, 1); + Lardon3DOrbVulkanInfo info{}; + bool ok = result == LARDON3D_ORB_VULKAN_UNAVAILABLE && + std::memcmp(&output, &unchanged, sizeof(output)) == 0 && + lardon3d_orb_vulkan_backend_info(backend, &info) && + info.initialized && !info.available; + lardon3d_orb_vulkan_backend_destroy(backend); + return ok; } static bool check_cached_unavailable() { @@ -165,7 +258,299 @@ static bool check_invalid_inputs(Lardon3DOrbVulkanBackend *backend) { 0) == LARDON3D_ORB_VULKAN_INVALID_ARGUMENT; } +static bool check_private_slot_contract(Lardon3DOrbVulkanBackend *backend) { + constexpr uint32_t count = 1024; + std::vector a = make_descriptors(count, 0x30303030U); + std::vector b = make_descriptors(count, 0x40404040U); + std::vector expected = opencv_top2(a, count, b, count); + std::vector actual(count); + Lardon3DOrbVulkanRequest invalid{}; + + if (lardon3d_orb_vulkan_internal_top2_finish( + backend, &invalid, actual.data(), actual.size()) != + LARDON3D_ORB_VULKAN_INVALID_ARGUMENT) { + return false; + } + Lardon3DOrbVulkanRequest first{}; + if (lardon3d_orb_vulkan_internal_top2_begin( + backend, a.data(), count, b.data(), count, &first) != + LARDON3D_ORB_VULKAN_OK) { + return false; + } + /* Invalid finish output still consumes its exact request. A following + * submit proves that slot was not stranded by argument validation. */ + if (lardon3d_orb_vulkan_internal_top2_finish( + backend, &first, nullptr, 0) != + LARDON3D_ORB_VULKAN_INVALID_ARGUMENT || + lardon3d_orb_vulkan_internal_top2_finish( + backend, &first, actual.data(), actual.size()) != + LARDON3D_ORB_VULKAN_FAILED) { + return false; + } + Lardon3DOrbVulkanRequest second{}; + if ( + lardon3d_orb_vulkan_internal_top2_begin( + backend, a.data(), count, b.data(), count, &second) != + LARDON3D_ORB_VULKAN_OK || + lardon3d_orb_vulkan_internal_top2_finish( + backend, &second, actual.data(), actual.size()) != + LARDON3D_ORB_VULKAN_OK) { + return false; + } + if (!std::equal(expected.begin(), expected.end(), actual.begin(), equal_top2)) { + return false; + } + Lardon3DOrbVulkanRequest discarded{}; + Lardon3DOrbVulkanRequest reused{}; + if (lardon3d_orb_vulkan_internal_top2_begin( + backend, a.data(), count, b.data(), count, &discarded) != + LARDON3D_ORB_VULKAN_OK || + lardon3d_orb_vulkan_internal_top2_discard(backend, &discarded) != + LARDON3D_ORB_VULKAN_OK || + lardon3d_orb_vulkan_internal_top2_begin( + backend, a.data(), count, b.data(), count, &reused) != + LARDON3D_ORB_VULKAN_OK || + reused.slot != discarded.slot || reused.generation == discarded.generation || + lardon3d_orb_vulkan_internal_top2_finish( + backend, &reused, actual.data(), actual.size()) != + LARDON3D_ORB_VULKAN_OK) { + return false; + } + return std::equal(expected.begin(), expected.end(), actual.begin(), equal_top2); +} + +static bool check_two_request_identity(Lardon3DOrbVulkanBackend *backend) { + constexpr uint32_t count = 1024; + std::vector a = make_descriptors(count, 0x71717171U); + std::vector b = make_descriptors(count, 0x72727272U); + std::vector c = make_descriptors(count, 0x73737373U); + std::vector expected_ab = opencv_top2(a, count, b, count); + std::vector expected_cb = opencv_top2(c, count, b, count); + std::vector actual_ab(count); + std::vector actual_cb(count); + Lardon3DOrbVulkanRequest request_ab{}; + Lardon3DOrbVulkanRequest request_cb{}; + Lardon3DOrbVulkanRequest third{}; + if (!lardon3d_orb_vulkan_internal_begin_sequence( + backend, LARDON3D_ORB_VULKAN_MAX_INFLIGHT)) { + return false; + } + bool ok = lardon3d_orb_vulkan_internal_top2_begin( + backend, a.data(), count, b.data(), count, &request_ab) == + LARDON3D_ORB_VULKAN_OK && + lardon3d_orb_vulkan_internal_top2_begin( + backend, c.data(), count, b.data(), count, &request_cb) == + LARDON3D_ORB_VULKAN_OK && + request_ab.slot != request_cb.slot && + lardon3d_orb_vulkan_internal_top2_begin( + backend, a.data(), count, c.data(), count, &third) == + LARDON3D_ORB_VULKAN_FAILED; + Lardon3DOrbVulkanRequest mismatched = request_ab; + mismatched.generation = request_cb.generation + 1; + if (mismatched.generation == 0) mismatched.generation = 1; + if (ok) { + ok = lardon3d_orb_vulkan_internal_top2_finish( + backend, &mismatched, actual_ab.data(), actual_ab.size()) == + LARDON3D_ORB_VULKAN_FAILED + && lardon3d_orb_vulkan_internal_top2_finish( + backend, &request_cb, actual_cb.data(), actual_cb.size()) == + LARDON3D_ORB_VULKAN_OK + && lardon3d_orb_vulkan_internal_top2_finish( + backend, &request_ab, actual_ab.data(), actual_ab.size()) == + LARDON3D_ORB_VULKAN_OK; + } else { + if (request_ab.generation != 0) { + (void)lardon3d_orb_vulkan_internal_top2_discard(backend, &request_ab); + } + if (request_cb.generation != 0) { + (void)lardon3d_orb_vulkan_internal_top2_discard(backend, &request_cb); + } + } + bool ended = lardon3d_orb_vulkan_internal_end_sequence(backend); + return ok && ended + && std::equal(expected_ab.begin(), expected_ab.end(), actual_ab.begin(), + equal_top2) + && std::equal(expected_cb.begin(), expected_cb.end(), actual_cb.begin(), + equal_top2); +} + +static bool check_private_telemetry_lifecycle() { + Lardon3DOrbVulkanBackend *backend = lardon3d_orb_vulkan_backend_create(); + if (!backend) return false; + constexpr uint32_t count = 1024; + std::vector descriptors = + make_descriptors(count, 0x60606060U); + std::vector output(count); + Lardon3DOrbVulkanTelemetry telemetry{}; + Lardon3DOrbVulkanRequest first{}; + bool ok = lardon3d_orb_vulkan_internal_telemetry(backend, &telemetry) + && telemetry.submits == 0 && telemetry.completions == 0 + && !telemetry.slot_pending && telemetry.pending_slots == 0; + ok = ok && lardon3d_orb_vulkan_internal_top2_begin( + backend, descriptors.data(), count, descriptors.data(), count, &first) + == LARDON3D_ORB_VULKAN_OK; + ok = ok && lardon3d_orb_vulkan_internal_telemetry(backend, &telemetry) + && telemetry.submits == 1 && telemetry.completions == 0 + && telemetry.slot_pending && telemetry.pending_slots == 1 + && telemetry.serial > 0; + ok = ok && lardon3d_orb_vulkan_internal_top2_finish( + backend, &first, output.data(), output.size()) == LARDON3D_ORB_VULKAN_OK; + ok = ok && lardon3d_orb_vulkan_internal_telemetry(backend, &telemetry) + && telemetry.submits == 1 && telemetry.completions == 1 + && !telemetry.slot_pending && telemetry.pending_slots == 0; + Lardon3DOrbVulkanRequest second{}; + ok = ok && lardon3d_orb_vulkan_internal_top2_begin( + backend, descriptors.data(), count, descriptors.data(), count, &second) + == LARDON3D_ORB_VULKAN_OK; + ok = ok && lardon3d_orb_vulkan_internal_top2_discard(backend, &second) + == LARDON3D_ORB_VULKAN_OK; + ok = ok && lardon3d_orb_vulkan_internal_telemetry(backend, &telemetry) + && telemetry.submits == 2 && telemetry.completions == 1 + && telemetry.discards == 1 && !telemetry.slot_pending + && telemetry.pending_slots == 0; + /* Slot reuse after discard remains operational and metrics remain + * cumulative; no history allocation or public info-struct change is used. */ + Lardon3DOrbVulkanRequest third{}; + ok = ok && lardon3d_orb_vulkan_internal_top2_begin( + backend, descriptors.data(), count, descriptors.data(), count, &third) + == LARDON3D_ORB_VULKAN_OK; + ok = ok && lardon3d_orb_vulkan_internal_top2_finish( + backend, &third, output.data(), output.size()) == LARDON3D_ORB_VULKAN_OK; + ok = ok && lardon3d_orb_vulkan_internal_telemetry(backend, &telemetry) + && telemetry.submits == 3 && telemetry.completions == 2 + && telemetry.discards == 1 && !telemetry.slot_pending; + lardon3d_orb_vulkan_backend_destroy(backend); + return ok; +} + #ifdef LARDON3D_ORB_VULKAN_TESTING +static bool check_capacity_lifecycle() { + Lardon3DOrbVulkanBackend *backend = lardon3d_orb_vulkan_backend_create(); + if (!backend) return false; + std::vector descriptors = make_descriptors(1024, 0x81818181U); + std::vector output(1024); + Lardon3DOrbVulkanInfo info{}; + Lardon3DOrbVulkanTelemetry telemetry{}; + bool ok = lardon3d_orb_vulkan_backend_info(backend, &info) + && !info.initialized && info.permanent_payload_bytes == 0 + && lardon3d_orb_vulkan_top2( + backend, descriptors.data(), 1024, descriptors.data(), 1024, + output.data(), output.size()) == LARDON3D_ORB_VULKAN_OK + && lardon3d_orb_vulkan_backend_info(backend, &info) + && info.permanent_payload_bytes == + LARDON3D_ORB_VULKAN_PER_SLOT_BYTES + && lardon3d_orb_vulkan_internal_telemetry(backend, &telemetry) + && telemetry.retained_capacity == 1 + && telemetry.retained_payload_bytes == + LARDON3D_ORB_VULKAN_PER_SLOT_BYTES + && !telemetry.sequence_capacity_active; + + if (ok) { + ok = setenv("LARDON3D_TEST_VULKAN_SLOT_ALLOCATION_FAILURE", "1", 1) == 0 + && !lardon3d_orb_vulkan_internal_begin_sequence(backend, 2) + && lardon3d_orb_vulkan_backend_info(backend, &info) + && info.permanent_payload_bytes == + LARDON3D_ORB_VULKAN_PER_SLOT_BYTES + && lardon3d_orb_vulkan_internal_telemetry(backend, &telemetry) + && telemetry.retained_capacity == 1 + && !telemetry.sequence_capacity_active; + } + bool environment_restored = + unsetenv("LARDON3D_TEST_VULKAN_SLOT_ALLOCATION_FAILURE") == 0; + if (ok) { + ok = lardon3d_orb_vulkan_internal_begin_sequence(backend, 2) + && lardon3d_orb_vulkan_backend_info(backend, &info) + && info.permanent_payload_bytes == + 2 * LARDON3D_ORB_VULKAN_PER_SLOT_BYTES + && lardon3d_orb_vulkan_internal_telemetry(backend, &telemetry) + && telemetry.retained_capacity == 2 + && telemetry.sequence_capacity_active + && lardon3d_orb_vulkan_internal_end_sequence(backend) + && lardon3d_orb_vulkan_backend_info(backend, &info) + && info.permanent_payload_bytes == + LARDON3D_ORB_VULKAN_PER_SLOT_BYTES; + } + + Lardon3DOrbVulkanRequest pending{}; + if (ok) { + ok = lardon3d_orb_vulkan_internal_begin_sequence(backend, 1) + && lardon3d_orb_vulkan_internal_top2_begin( + backend, descriptors.data(), 1024, descriptors.data(), 1024, + &pending) == LARDON3D_ORB_VULKAN_OK + && !lardon3d_orb_vulkan_internal_end_sequence(backend) + && lardon3d_orb_vulkan_internal_top2_finish( + backend, &pending, output.data(), output.size()) == + LARDON3D_ORB_VULKAN_OK + && lardon3d_orb_vulkan_internal_end_sequence(backend); + } + lardon3d_orb_vulkan_backend_destroy(backend); + return environment_restored && ok; +} + +static bool check_generation_saturation() { + Lardon3DOrbVulkanBackend *backend = lardon3d_orb_vulkan_backend_create(); + if (!backend) return false; + std::vector descriptors = make_descriptors(1024, 0x91919191U); + std::vector output(1024); + Lardon3DOrbVulkanRequest ancient{}; + Lardon3DOrbVulkanRequest terminal{}; + Lardon3DOrbVulkanRequest future{}; + bool ok = lardon3d_orb_vulkan_internal_begin_sequence(backend, 2) + && lardon3d_orb_vulkan_internal_top2_begin( + backend, descriptors.data(), 1024, descriptors.data(), 1024, + &ancient) == LARDON3D_ORB_VULKAN_OK + && ancient.generation == 1 + && lardon3d_orb_vulkan_internal_top2_finish( + backend, &ancient, output.data(), output.size()) == + LARDON3D_ORB_VULKAN_OK + && lardon3d_orb_vulkan_internal_test_set_slot_generation( + backend, ancient.slot, UINT64_MAX - 1) + /* UINT64_MAX is the terminal valid request identity. The following + * begin must issue it exactly once; only the subsequent begin retires + * this slot, before any submission could wrap to generation one. */ + && lardon3d_orb_vulkan_internal_top2_begin( + backend, descriptors.data(), 1024, descriptors.data(), 1024, + &terminal) == LARDON3D_ORB_VULKAN_OK + && terminal.slot == ancient.slot && terminal.generation == UINT64_MAX + && lardon3d_orb_vulkan_internal_top2_finish( + backend, &terminal, output.data(), output.size()) == + LARDON3D_ORB_VULKAN_OK + && lardon3d_orb_vulkan_internal_top2_begin( + backend, descriptors.data(), 1024, descriptors.data(), 1024, + &future) == LARDON3D_ORB_VULKAN_OK + && future.slot != ancient.slot && future.generation == 1 + && lardon3d_orb_vulkan_internal_top2_finish( + backend, &ancient, output.data(), output.size()) == + LARDON3D_ORB_VULKAN_FAILED; + Lardon3DOrbVulkanTelemetry telemetry{}; + if (ok) { + ok = lardon3d_orb_vulkan_internal_telemetry(backend, &telemetry) + && telemetry.pending_slots == 1 + && lardon3d_orb_vulkan_internal_top2_finish( + backend, &future, output.data(), output.size()) == + LARDON3D_ORB_VULKAN_OK + && lardon3d_orb_vulkan_internal_end_sequence(backend); + } + Lardon3DOrbVulkanRequest after_boundary{}; + if (ok) { + ok = lardon3d_orb_vulkan_internal_begin_sequence(backend, 1) + && lardon3d_orb_vulkan_internal_top2_begin( + backend, descriptors.data(), 1024, descriptors.data(), 1024, + &after_boundary) == LARDON3D_ORB_VULKAN_OK + && after_boundary.slot == future.slot + && after_boundary.generation == 2 + && lardon3d_orb_vulkan_internal_top2_finish( + backend, &ancient, output.data(), output.size()) == + LARDON3D_ORB_VULKAN_FAILED + && lardon3d_orb_vulkan_internal_top2_finish( + backend, &after_boundary, output.data(), output.size()) == + LARDON3D_ORB_VULKAN_OK + && lardon3d_orb_vulkan_internal_end_sequence(backend); + } + lardon3d_orb_vulkan_backend_destroy(backend); + return ok; +} + static bool check_cached_device_failure() { Lardon3DOrbVulkanBackend *backend = lardon3d_orb_vulkan_backend_create(); std::vector descriptors = make_descriptors(1024, 0x20202020U); @@ -184,11 +569,92 @@ static bool check_cached_device_failure() { return first == LARDON3D_ORB_VULKAN_FAILED && second == LARDON3D_ORB_VULKAN_UNAVAILABLE; } + + +static bool check_discard_wait_failure() { + Lardon3DOrbVulkanBackend *backend = lardon3d_orb_vulkan_backend_create(); + if (!backend) { + return false; + } + std::vector descriptors = make_descriptors(1024, 0x50505050U); + Lardon3DOrbVulkanRequest request{}; + bool ok = lardon3d_orb_vulkan_internal_top2_begin( + backend, descriptors.data(), 1024, descriptors.data(), 1024, + &request) == + LARDON3D_ORB_VULKAN_OK; + if (ok && setenv("LARDON3D_TEST_VULKAN_WAIT_FAILURE", "1", 1) != 0) { + ok = false; + } + if (ok) { + ok = lardon3d_orb_vulkan_internal_top2_discard(backend, &request) == + LARDON3D_ORB_VULKAN_FAILED; + } + unsetenv("LARDON3D_TEST_VULKAN_WAIT_FAILURE"); + if (ok) { + ok = lardon3d_orb_vulkan_internal_top2_begin( + backend, descriptors.data(), 1024, descriptors.data(), 1024, + &request) == + LARDON3D_ORB_VULKAN_UNAVAILABLE; + } + Lardon3DOrbVulkanTelemetry telemetry{}; + if (ok) { + ok = lardon3d_orb_vulkan_internal_telemetry(backend, &telemetry) + && telemetry.submits == 1 && telemetry.discards == 1 + && telemetry.failures == 1 && !telemetry.slot_pending; + } + lardon3d_orb_vulkan_backend_destroy(backend); + return ok; +} + +static bool check_finish_failure(const char *failure_variable, + uint64_t expected_completions) { + Lardon3DOrbVulkanBackend *backend = lardon3d_orb_vulkan_backend_create(); + if (!backend || !failure_variable) { + lardon3d_orb_vulkan_backend_destroy(backend); + return false; + } + std::vector descriptors = make_descriptors(1024, 0x61616161U); + std::vector output(1024); + Lardon3DOrbVulkanRequest request{}; + bool ok = lardon3d_orb_vulkan_internal_top2_begin( + backend, descriptors.data(), 1024, descriptors.data(), 1024, + &request) == LARDON3D_ORB_VULKAN_OK + && setenv(failure_variable, "1", 1) == 0; + if (ok) { + ok = lardon3d_orb_vulkan_internal_top2_finish( + backend, &request, output.data(), output.size()) == + LARDON3D_ORB_VULKAN_FAILED; + } + bool environment_restored = unsetenv(failure_variable) == 0; + if (ok) { + ok = lardon3d_orb_vulkan_internal_top2_begin( + backend, descriptors.data(), 1024, descriptors.data(), 1024, + &request) == LARDON3D_ORB_VULKAN_UNAVAILABLE; + } + Lardon3DOrbVulkanTelemetry telemetry{}; + if (ok) { + ok = lardon3d_orb_vulkan_internal_telemetry(backend, &telemetry) + && telemetry.submits == 1 + && telemetry.completions == expected_completions + && telemetry.failures == 1 && telemetry.pending_slots == 0 + && !telemetry.slot_pending; + } + lardon3d_orb_vulkan_backend_destroy(backend); + return environment_restored && ok; +} #endif } // namespace -int main() { +int main(int argc, char **argv) { + if (argc == 2 && std::strcmp(argv[1], "--unsafe-policy-only") == 0) { + return check_current_driver_policy_rejection() ? 0 : 1; + } + if (argc != 1) return 2; + if (!check_driver_policy_gate()) { + std::fprintf(stderr, "Vulkan driver process-policy gate failed\n"); + return 1; + } Lardon3DOrbVulkanBackend *backend = lardon3d_orb_vulkan_backend_create(); if (!backend) { return 1; @@ -202,6 +668,21 @@ int main() { ok = check_case(backend, 64, 2, 0x88112233U) && ok; ok = check_ties(backend) && ok; ok = check_invalid_inputs(backend) && ok; + const bool private_slot_ok = check_private_slot_contract(backend); + if (!private_slot_ok) { + std::fprintf(stderr, "private Vulkan slot contract failed\n"); + } + ok = private_slot_ok && ok; + const bool two_request_ok = check_two_request_identity(backend); + if (!two_request_ok) { + std::fprintf(stderr, "private Vulkan two-request identity failed\n"); + } + ok = two_request_ok && ok; + const bool telemetry_ok = check_private_telemetry_lifecycle(); + if (!telemetry_ok) { + std::fprintf(stderr, "private Vulkan telemetry lifecycle failed\n"); + } + ok = telemetry_ok && ok; ok = check_serialized_threads(backend) && ok; ok = !lardon3d_orb_vulkan_should_use(256, 256) && ok; ok = !lardon3d_orb_vulkan_should_use(512, 512) && ok; @@ -209,7 +690,9 @@ int main() { ok = lardon3d_orb_vulkan_should_use(1024, 1024) && ok; Lardon3DOrbVulkanInfo info{}; - ok = lardon3d_orb_vulkan_backend_info(backend, &info) && info.available && ok; + ok = lardon3d_orb_vulkan_backend_info(backend, &info) && info.available && + info.permanent_payload_bytes == + LARDON3D_ORB_VULKAN_PER_SLOT_BYTES && ok; if (info.available) { std::printf("device=%s workgroup=%u payload=%llu init_ms=%.3f gpu_ms=%.3f\n", info.device_name, info.workgroup_size, @@ -220,7 +703,34 @@ int main() { lardon3d_orb_vulkan_backend_destroy(backend); ok = check_cached_unavailable() && ok; #ifdef LARDON3D_ORB_VULKAN_TESTING + const bool capacity_ok = check_capacity_lifecycle(); + if (!capacity_ok) { + std::fprintf(stderr, "Vulkan sequence-capacity lifecycle failed\n"); + } + ok = capacity_ok && ok; + const bool generation_ok = check_generation_saturation(); + if (!generation_ok) { + std::fprintf(stderr, "Vulkan request generation saturation failed\n"); + } + ok = generation_ok && ok; ok = check_cached_device_failure() && ok; + const bool discard_wait_ok = check_discard_wait_failure(); + if (!discard_wait_ok) { + std::fprintf(stderr, "Vulkan discard wait-failure contract failed\n"); + } + ok = discard_wait_ok && ok; + const bool finish_wait_ok = check_finish_failure( + "LARDON3D_TEST_VULKAN_FINISH_WAIT_FAILURE", 0); + if (!finish_wait_ok) { + std::fprintf(stderr, "Vulkan finish wait-failure contract failed\n"); + } + ok = finish_wait_ok && ok; + const bool readback_ok = check_finish_failure( + "LARDON3D_TEST_VULKAN_READBACK_FAILURE", 1); + if (!readback_ok) { + std::fprintf(stderr, "Vulkan readback-failure contract failed\n"); + } + ok = readback_ok && ok; #endif return ok ? 0 : 1; } diff --git a/tests/test_photo_quality_task.cpp b/tests/test_photo_quality_task.cpp index 841640f..fe02af9 100644 --- a/tests/test_photo_quality_task.cpp +++ b/tests/test_photo_quality_task.cpp @@ -108,11 +108,13 @@ int main() { size_t request_probe_size = 0; CHECK(lardon3d_photo_quality_request_encode(&request, nullptr, 0, &request_probe_size)); + cv::setNumThreads(3); uint64_t task_id = 0; CHECK(lardon3d_project_enqueue_photo_quality(&state, scanset.scanset_id, &request, &task_id)); Lardon3DTaskSnapshot terminal{}; CHECK(wait_terminal(state.task_queue, task_id, &terminal)); CHECK(terminal.state == TASK_COMPLETED && terminal.progress == 100); + CHECK(cv::getNumThreads() == 3); Lardon3DProjectDbPhotoQualityResult paired{}, raw_only{}; CHECK(lardon3d_project_db_load_photo_quality_result(database, task_id, 1, &paired) == LARDON3D_PROJECT_DB_OK); diff --git a/tests/test_pre_sfm_runner_options.cpp b/tests/test_pre_sfm_runner_options.cpp new file mode 100644 index 0000000..b27bccd --- /dev/null +++ b/tests/test_pre_sfm_runner_options.cpp @@ -0,0 +1,495 @@ +#include +#include +#include +#include +#include + +#define main lardon3d_pre_sfm_real_execution_main +#include "pre_sfm_real_execution.cpp" +#undef main + +#define CHECK(condition) \ + do { \ + if (!(condition)) { \ + std::cerr << "failed line " << __LINE__ << ": " << #condition << '\n'; \ + return EXIT_FAILURE; \ + } \ + } while (false) + +static bool parse_case(std::initializer_list arguments, + Options &options) { + std::vector storage(arguments.begin(), arguments.end()); + std::vector argv; + argv.reserve(storage.size()); + for (std::string &argument : storage) argv.push_back(argument.data()); + return parse_options(static_cast(argv.size()), argv.data(), options); +} + +static int invoke_case(std::initializer_list arguments) { + std::vector storage(arguments.begin(), arguments.end()); + std::vector argv; + argv.reserve(storage.size()); + for (std::string &argument : storage) argv.push_back(argument.data()); + return lardon3d_pre_sfm_real_execution_main( + static_cast(argv.size()), argv.data()); +} + +static bool capture_matcher_evidence(Runtime &runtime, std::string &output) { + FILE *capture = std::tmpfile(); + if (!capture) return false; + const int saved_stdout = dup(STDOUT_FILENO); + if (saved_stdout < 0 || std::fflush(stdout) != 0 || + dup2(fileno(capture), STDOUT_FILENO) < 0) { + if (saved_stdout >= 0) close(saved_stdout); + std::fclose(capture); + return false; + } + (void)end_matcher_evidence(runtime); + const bool flushed = std::fflush(stdout) == 0; + const bool restored = dup2(saved_stdout, STDOUT_FILENO) >= 0; + close(saved_stdout); + bool read_ok = flushed && restored && std::fseek(capture, 0, SEEK_SET) == 0; + char buffer[1024]; + while (read_ok) { + const size_t count = std::fread(buffer, 1, sizeof(buffer), capture); + output.append(buffer, count); + if (count < sizeof(buffer)) { + read_ok = std::feof(capture) != 0 && std::ferror(capture) == 0; + break; + } + } + return std::fclose(capture) == 0 && read_ok; +} + +static bool json_valid_and_submit_cpu(const std::string &line, + bool require_submit_cpu) { + sqlite3 *database = nullptr; + sqlite3_stmt *statement = nullptr; + bool ok = sqlite3_open(":memory:", &database) == SQLITE_OK && + sqlite3_prepare_v2( + database, + "SELECT json_valid(?1), json_extract(?1, " + "'$.vulkan_submit_cpu_ns')", + -1, &statement, nullptr) == SQLITE_OK && + sqlite3_bind_text(statement, 1, line.c_str(), -1, + SQLITE_TRANSIENT) == SQLITE_OK && + sqlite3_step(statement) == SQLITE_ROW && + sqlite3_column_int(statement, 0) == 1; + if (ok && require_submit_cpu) { + ok = sqlite3_column_type(statement, 1) == SQLITE_INTEGER && + sqlite3_column_int64(statement, 1) == 123456789; + } + if (statement) sqlite3_finalize(statement); + if (database && sqlite3_close(database) != SQLITE_OK) ok = false; + return ok; +} + +static bool matcher_evidence_aggregate_case() { + Lardon3DHardwareProfile profile{}; + profile.logical_cpu_count = 16; + profile.page_size_bytes = 4096; + profile.memory_total_bytes = UINT64_C(16) * 1024 * 1024 * 1024; + profile.gpu_available = true; + profile.gpu_uses_shared_memory = true; + std::snprintf(profile.cpu_architecture, sizeof(profile.cpu_architecture), + "%s", "test"); + Lardon3DResourcePolicy policy{}; + if (!lardon3d_resource_policy_default(&profile, &policy)) return false; + policy.gpu_slot_capacity = 1; + Lardon3DResourceGovernor *governor = + lardon3d_resource_governor_create(&profile, &policy); + if (!governor) return false; + + Lardon3DTaskCapabilityEnvelope envelope{}; + envelope.count = 1; + envelope.capabilities[0].estimate.memory_bytes_per_item = 10 * 1024 * 1024; + envelope.capabilities[0].estimate.minimum_batch_size = 4; + envelope.capabilities[0].estimate.maximum_batch_size = 4; + envelope.capabilities[0].estimate.desired_cpu_threads = 1; + envelope.capabilities[0].estimate.desired_gpu_slots = 1; + envelope.capabilities[0].estimate.desired_io_slots = 1; + envelope.capabilities[0].estimate.task_class = LARDON3D_RESOURCE_TASK_GPU; + envelope.capabilities[0].backend = LARDON3D_RESOURCE_BACKEND_ORB_VULKAN; + envelope.capabilities[0].inflight_limit = 2; + envelope.capabilities[0].minimum_inflight_limit = 2; + envelope.capabilities[0].gpu_memory_bytes_per_inflight = 640 * 1024; + Lardon3DResourceSnapshot snapshot{}; + if (clock_gettime(CLOCK_MONOTONIC, &snapshot.captured_at) != 0) { + lardon3d_resource_governor_destroy(governor); + return false; + } + snapshot.memory_available_bytes = UINT64_C(8) * 1024 * 1024 * 1024; + snapshot.swap_activity_known = true; + Lardon3DResourceCapabilitySelection selection{}; + Lardon3DResourceReservation *reservation = nullptr; + const bool admitted = + lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &envelope, &selection, + &reservation) && + reservation; + const bool released = + admitted && lardon3d_resource_governor_release(governor, reservation); + Lardon3DResourceExecutionMetrics metrics{}; + metrics.vulkan_submits = 2; + metrics.vulkan_completions = 2; + metrics.vulkan_submit_cpu_ns = 123456789; + metrics.vulkan_fence_wait_ns = 11; + metrics.vulkan_readback_ns = 12; + metrics.vulkan_gpu_time_known = true; + metrics.vulkan_gpu_ns = 13; + metrics.vulkan_starvation_ns = 14; + metrics.matcher_cpu_ns = 15; + metrics.publication_ns = 16; + metrics.local_ineligible_fallback_items = 1; + const bool recorded = released && + lardon3d_resource_governor_internal_record_fallback_items( + governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &selection, + LARDON3D_RESOURCE_FALLBACK_ITEM_LOCAL_INELIGIBLE, 1) && + lardon3d_resource_governor_internal_record_sequence_execution_metrics( + governor, LARDON3D_MATCHER_TASK_KIND, + LARDON3D_MATCHER_TASK_KIND_VERSION, &selection, 1000, 1, + LARDON3D_RESOURCE_BACKEND_MIXED, + "vulkan-and-ineligible-pair-cpu-fallback", &metrics); + Runtime runtime{}; + runtime.state.resource_governor = governor; + runtime.state.hardware_profile = profile; + runtime.matcher_inflight_override = 2; + runtime.matcher_batch_override = 4; + begin_matcher_evidence(runtime); + std::string output; + const bool captured = recorded && capture_matcher_evidence(runtime, output); + lardon3d_resource_governor_destroy(governor); + if (!captured) { + std::cerr << "unable to construct/capture Matcher evidence fixture: " + << "admitted=" << admitted << " released=" << released + << " recorded=" << recorded << '\n'; + return false; + } + + bool saw_aggregate = false; + size_t offset = 0; + while (offset < output.size()) { + const size_t newline = output.find('\n', offset); + const size_t end = newline == std::string::npos ? output.size() : newline; + const std::string line = output.substr(offset, end - offset); + if (!line.empty()) { + const bool aggregate = line.find( + "\"record\":\"matcher_evidence_aggregate\"") != + std::string::npos; + if (!json_valid_and_submit_cpu(line, aggregate)) { + std::cerr << "invalid Matcher evidence JSON: " << line << '\n'; + return false; + } + saw_aggregate = saw_aggregate || aggregate; + } + if (newline == std::string::npos) break; + offset = newline + 1; + } + const bool exact_field = + output.find("\"vulkan_submit_cpu_ns\":123456789") != + std::string::npos; + const bool affinity_fields = + output.find("\"runtime_thread_policy_active\":true") != + std::string::npos && + output.find("\"mesa_shader_cache_disabled\":true") != + std::string::npos && + output.find("\"runtime_thread_policy_reason\":" + "\"worker-self-affinity-plus-mesa-disk-cache-disabled\"") != + std::string::npos && + output.find("auxiliary_affinity_active") == std::string::npos; + const bool inflight_field = + output.find("\"matcher_inflight_override\":2") != std::string::npos; + const bool batch_field = + output.find("\"matcher_batch_override\":4") != std::string::npos; + const bool experiment_fields = + output.find("\"experiment_valid\":false") != std::string::npos && + output.find("\"experiment_reason\":\"backend-telemetry-unavailable\"") != + std::string::npos && + output.find("\"local_ineligible_fallback_sequences\":1") != + std::string::npos && + output.find("\"local_ineligible_fallback_items\":1") != + std::string::npos && + output.find("\"backend_failure_fallback_items\":0") != + std::string::npos && + output.find("\"backend_other_fallback_items\":0") != + std::string::npos && + output.find( + "\"comparison_requires_equal_local_ineligible_fallback_items\":" + "true") != std::string::npos; + if (!saw_aggregate || !exact_field || !affinity_fields || !inflight_field || + !batch_field || !experiment_fields) + std::cerr << "missing exact Matcher aggregate field: " << output << '\n'; + return saw_aggregate && exact_field && affinity_fields && inflight_field && + batch_field && experiment_fields; +} + +static bool forced_matcher_experiment_validation_case() { + Runtime runtime{}; + runtime.matcher_inflight_override = 1; + Lardon3DResourceSequenceAggregate aggregate{}; + aggregate.admission_count = 2; + aggregate.sequence_count = 2; + aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN] = 2; + aggregate.actual_backend_sequences[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN] = 2; + aggregate.vulkan_submits = 4; + aggregate.vulkan_completions = 4; + Lardon3DResourceSequenceDiagnostic last{}; + last.backend = LARDON3D_RESOURCE_BACKEND_ORB_VULKAN; + last.actual_backend = LARDON3D_RESOURCE_BACKEND_ORB_VULKAN; + last.cpu_threads = 1; + last.gpu_slots = 1; + last.batch_size = 2; + last.inflight_limit = 1; + last.io_slots = 1; + last.memory_bytes = 2 * 10 * 1024 * 1024; + last.gpu_memory_bytes = 640 * 1024; + MatcherExperimentValidation validated = validate_forced_matcher_experiment( + runtime, true, aggregate, true, last, true, 0, 0, false); + if (!validated.valid || validated.local_ineligible_fallback_items != 0) + return false; + + /* The second controlled cohort differs only in admitted inflight/payload; + * selected backend, CPU/GPU/batch/helpers, and scientific output stay fixed. */ + runtime.matcher_inflight_override = 2; + last.inflight_limit = 2; + last.gpu_memory_bytes = 2 * 640 * 1024; + validated = validate_forced_matcher_experiment( + runtime, true, aggregate, true, last, true, 0, 0, false); + if (!validated.valid) return false; + + runtime.matcher_batch_override = 4; + last.batch_size = 4; + last.memory_bytes = 4 * 10 * 1024 * 1024; + validated = validate_forced_matcher_experiment( + runtime, true, aggregate, true, last, true, 0, 0, false); + if (!validated.valid) return false; + last.batch_size = 2; + validated = validate_forced_matcher_experiment( + runtime, true, aggregate, true, last, true, 0, 0, false); + if (validated.valid || + std::string(validated.reason) != "forced-contract-mismatch") + return false; + runtime.matcher_batch_override = 0; + last.batch_size = 2; + last.memory_bytes = 2 * 10 * 1024 * 1024; + + aggregate.actual_backend_sequences[ + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN] = 1; + aggregate.actual_backend_sequences[LARDON3D_RESOURCE_BACKEND_MIXED] = 1; + aggregate.backend_fallback_sequences = 1; + aggregate.backend_ineligible_fallback_sequences = 1; + aggregate.local_ineligible_fallback_items = 3; + aggregate.durable_items = 4; + validated = validate_forced_matcher_experiment( + runtime, true, aggregate, true, last, true, 0, 0, false); + if (!validated.valid || validated.local_ineligible_fallback_items != 3) + return false; + aggregate.local_ineligible_fallback_items = 0; + validated = validate_forced_matcher_experiment( + runtime, true, aggregate, true, last, true, 0, 0, false); + if (validated.valid || + std::string(validated.reason) != + "fallback-item-classification-mismatch") + return false; + + aggregate.backend_ineligible_fallback_sequences = 0; + aggregate.backend_failure_fallback_sequences = 1; + aggregate.local_ineligible_fallback_items = 0; + aggregate.backend_failure_fallback_items = 1; + validated = validate_forced_matcher_experiment( + runtime, true, aggregate, true, last, true, 0, 0, false); + if (validated.valid || std::string(validated.reason) != "backend-failure") + return false; + + aggregate.backend_failure_fallback_sequences = 0; + aggregate.backend_failure_fallback_items = 0; + aggregate.backend_other_fallback_items = 1; + validated = validate_forced_matcher_experiment( + runtime, true, aggregate, true, last, true, 0, 0, false); + if (validated.valid || + std::string(validated.reason) != "unclassified-backend-fallback") + return false; + + aggregate = {}; + aggregate.admission_count = 1; + aggregate.sequence_count = 1; + aggregate.selected_backend_admissions[LARDON3D_RESOURCE_BACKEND_CPU] = 1; + aggregate.actual_backend_sequences[LARDON3D_RESOURCE_BACKEND_CPU] = 1; + validated = validate_forced_matcher_experiment( + runtime, true, aggregate, true, last, true, 0, 0, false); + return !validated.valid && + std::string(validated.reason) == + "selected-contract-not-exclusively-vulkan"; +} + +int main() { + CHECK(unsetenv("MESA_SHADER_CACHE_DISABLE") == 0); + CHECK(lardon3d_resource_governor_internal_configure_driver_policy() == + LARDON3D_RESOURCE_DRIVER_POLICY_DEFAULTED); + CHECK(std::string(std::getenv("MESA_SHADER_CACHE_DISABLE")) == "true"); + CHECK(lardon3d_resource_governor_internal_configure_driver_policy() == + LARDON3D_RESOURCE_DRIVER_POLICY_INHERITED_SAFE); + CHECK(setenv("MESA_SHADER_CACHE_DISABLE", "false", 1) == 0); + CHECK(lardon3d_resource_governor_internal_configure_driver_policy() == + LARDON3D_RESOURCE_DRIVER_POLICY_REJECTED_UNSAFE); + CHECK(std::string(std::getenv("MESA_SHADER_CACHE_DISABLE")) == "false"); + CHECK(setenv("MESA_SHADER_CACHE_DISABLE", "true", 1) == 0); + + Options options; + CHECK(parse_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened"}, options)); + CHECK(options.matcher_mode == MatcherMode::kAuto && + options.matcher_pipeline == MatcherPipeline::kRolling && + !options.has_matcher_mode && !options.has_matcher_pipeline && + !options.has_matcher_inflight_override && + options.matcher_inflight_override == 0 && + !options.has_matcher_batch_override && + options.matcher_batch_override == 0); + + options = {}; + CHECK(parse_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-mode", "auto", + "--matcher-pipeline", "synchronous"}, options)); + CHECK(options.matcher_mode == MatcherMode::kAuto && + options.matcher_pipeline == MatcherPipeline::kSynchronous && + options.has_matcher_mode && options.has_matcher_pipeline); + + options = {}; + CHECK(parse_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-mode", "vulkan", + "--matcher-pipeline", "rolling", "--gpu-budget", "1"}, + options)); + CHECK(options.matcher_mode == MatcherMode::kVulkan && + options.matcher_pipeline == MatcherPipeline::kRolling); + + options = {}; + CHECK(parse_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-mode", "auto", + "--matcher-pipeline", "rolling", "--matcher-inflight", + "1"}, options)); + CHECK(options.matcher_inflight_override == 1 && + options.has_matcher_inflight_override); + options = {}; + CHECK(parse_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-inflight", "2"}, + options)); + CHECK(options.matcher_inflight_override == 2 && + options.has_matcher_inflight_override); + for (const unsigned batch : {2U, 4U, 8U, 12U}) { + options = {}; + const std::string batch_text = std::to_string(batch); + CHECK(parse_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-inflight", "1", + "--matcher-batch", batch_text.c_str()}, options)); + CHECK(options.matcher_batch_override == batch && + options.has_matcher_batch_override); + } + + options = {}; + CHECK(!parse_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-mode", "invalid"}, options)); + options = {}; + CHECK(!parse_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-pipeline", "invalid"}, + options)); + for (const char *invalid_inflight : {"0", "3", "01", "+1", "2x"}) { + options = {}; + CHECK(!parse_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-inflight", + invalid_inflight}, options)); + } + for (const char *invalid_batch : {"1", "3", "6", "10", "16", "02", + "+2", "4x"}) { + options = {}; + CHECK(!parse_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-inflight", "1", + "--matcher-batch", invalid_batch}, options)); + } + options = {}; + CHECK(!parse_case({"runner", "--resume-candidate-existing", "--project-dir", + "/tmp/not-opened", "--matcher-mode", "auto"}, options)); + options = {}; + CHECK(!parse_case({"runner", "--resume-candidate-existing", "--project-dir", + "/tmp/not-opened", "--matcher-inflight", "1"}, + options)); + options = {}; + CHECK(!parse_case({"runner", "--resume-candidate-existing", "--project-dir", + "/tmp/not-opened", "--matcher-inflight", "1", + "--matcher-batch", "2"}, options)); + options = {}; + CHECK(!parse_case({"runner", "--resume-geometry-existing", "--project-dir", + "/tmp/not-opened", "--cpu-budget", "2"}, options)); + options = {}; + CHECK(!parse_case({"runner", "--mode", "s21", "--project-dir", + "/tmp/not-opened", "--root", "/tmp", "--matcher-mode", + "auto"}, options)); + + /* These contradictions are rejected by main before project inspection or a + * durable Task allocation, which is the CLI contract the harness relies on. */ + CHECK(invoke_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-mode", "cpu", + "--matcher-pipeline", "synchronous"}) == 2); + CHECK(invoke_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-mode", "vulkan", + "--gpu-budget", "0"}) == 2); + CHECK(invoke_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-inflight", "2", + "--gpu-budget", "0"}) == 2); + CHECK(invoke_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-mode", "cpu", + "--matcher-inflight", "1"}) == 2); + CHECK(invoke_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-mode", "vulkan", + "--matcher-inflight", "1"}) == 2); + CHECK(invoke_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-pipeline", "synchronous", + "--matcher-inflight", "2"}) == 2); + CHECK(invoke_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-batch", "2"}) == 2); + CHECK(invoke_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-pipeline", "synchronous", + "--matcher-inflight", "1", "--matcher-batch", "2"}) == + 2); + CHECK(invoke_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-inflight", "1", + "--matcher-batch", "4", "--gpu-budget", "0"}) == 2); + + /* The runner owns all three process controls only for one invocation. Even a + * project-validation exit must restore exact inherited values, while absent + * controls remain absent after the same path. */ + CHECK(setenv(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV, + "inherited-pipeline", 1) == 0 && + setenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV, + "inherited-inflight", 1) == 0 && + setenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV, + "inherited-batch", 1) == 0 && + invoke_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-inflight", "2", + "--matcher-batch", "8"}) == 2 && + std::string(std::getenv( + LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV)) == + "inherited-pipeline" && + std::string(std::getenv( + LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV)) == + "inherited-inflight" && + std::string(std::getenv( + LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV)) == + "inherited-batch" && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV) == 0 && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV) == 0 && + unsetenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV) == 0 && + invoke_case({"runner", "--resume-pre-gv-existing", "--project-dir", + "/tmp/not-opened", "--matcher-inflight", "1", + "--matcher-batch", "4"}) == 2 && + std::getenv(LARDON3D_MATCHER_TASK_BENCHMARK_SYNCHRONOUS_ENV) == + nullptr && + std::getenv(LARDON3D_MATCHER_TASK_BENCHMARK_INFLIGHT_ENV) == nullptr && + std::getenv(LARDON3D_MATCHER_TASK_BENCHMARK_BATCH_ENV) == nullptr); + CHECK(matcher_evidence_aggregate_case()); + CHECK(forced_matcher_experiment_validation_case()); + return EXIT_SUCCESS; +} diff --git a/tests/test_resource_governor.c b/tests/test_resource_governor.c index cf8676a..2f87700 100644 --- a/tests/test_resource_governor.c +++ b/tests/test_resource_governor.c @@ -1,10 +1,21 @@ +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif + +#include +#include #include #include +#ifdef __linux__ +#include +#endif #include #include #include #include #include +#include +#include #include @@ -28,6 +39,130 @@ use_fixed_test_clock(Lardon3DResourceGovernor *governor) return lardon3d_resource_governor_internal_set_monotonic_now(governor, &now); } +static bool +cpu_mask_has(const uint64_t mask[LARDON3D_RESOURCE_CPU_MASK_WORDS], + unsigned int cpu) +{ + return cpu < LARDON3D_RESOURCE_CPU_MAX + && (mask[cpu / 64] & (UINT64_C(1) << (cpu % 64))) != 0; +} + +static bool +topology_file_result( + const char *text, + size_t length, + bool expected_result, + unsigned int expected_value +) +{ + char path[] = "/tmp/lardon3d-topology-token-XXXXXX"; + int descriptor = mkstemp(path); + if (descriptor < 0) { + return false; + } + size_t offset = 0; + bool written = true; + while (offset < length) { + ssize_t count = write(descriptor, text + offset, length - offset); + if (count <= 0) { + written = false; + break; + } + offset += (size_t)count; + } + bool closed = close(descriptor) == 0; + unsigned int value = UINT_MAX; + bool result = written && closed + && lardon3d_resource_governor_internal_read_topology_value_file( + path, &value); + bool removed = unlink(path) == 0; + return removed && result == expected_result + && (!expected_result || value == expected_value); +} + +static bool +run_topology_value_reader_test(void) +{ + char maximum[32]; + int maximum_length = snprintf(maximum, sizeof(maximum), "%u\n", UINT_MAX); + char truncated[33]; + truncated[0] = '1'; + memset(truncated + 1, ' ', sizeof(truncated) - 1); + /* The first 32 bytes are independently valid. The 33rd whitespace byte + * proves the reader checks EOF instead of accepting a truncated prefix. */ + CHECK(maximum_length > 0 && (size_t)maximum_length < sizeof(maximum)); + CHECK(topology_file_result("0\n", 2, true, 0)); + CHECK(topology_file_result(maximum, (size_t)maximum_length, true, UINT_MAX)); + CHECK(topology_file_result("12 \t\r\n", 6, true, 12)); + CHECK(topology_file_result("", 0, false, 0)); + CHECK(topology_file_result("-1\n", 3, false, 0)); + CHECK(topology_file_result("+1\n", 3, false, 0)); + CHECK(topology_file_result(" 1\n", 3, false, 0)); + CHECK(topology_file_result("1x\n", 3, false, 0)); + CHECK(topology_file_result("1\n2\n", 4, false, 0)); + CHECK(topology_file_result(truncated, sizeof(truncated), false, 0)); + return true; +} + +static bool +write_text_file(const char *path, const char *text) +{ + int descriptor = open(path, O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, + 0600); + if (descriptor < 0) return false; + size_t length = strlen(text); + size_t offset = 0; + while (offset < length) { + ssize_t written = write(descriptor, text + offset, length - offset); + if (written < 0 && errno == EINTR) continue; + if (written <= 0) { + (void)close(descriptor); + return false; + } + offset += (size_t)written; + } + return close(descriptor) == 0; +} + +static bool +run_gpu_busy_identity_reader_test(void) +{ + char root[] = "/tmp/lardon3d-drm-telemetry-XXXXXX"; + CHECK(mkdtemp(root)); + char card0[PATH_MAX]; + char card37[PATH_MAX]; + char device0[PATH_MAX]; + char device37[PATH_MAX]; + char busy0[PATH_MAX]; + char busy37[PATH_MAX]; + CHECK(snprintf(card0, sizeof(card0), "%s/card0", root) > 0 + && snprintf(card37, sizeof(card37), "%s/card37", root) > 0 + && snprintf(device0, sizeof(device0), "%s/device", card0) > 0 + && snprintf(device37, sizeof(device37), "%s/device", card37) > 0 + && snprintf(busy0, sizeof(busy0), "%s/gpu_busy_percent", device0) > 0 + && snprintf(busy37, sizeof(busy37), "%s/gpu_busy_percent", device37) > 0 + && mkdir(card0, 0700) == 0 && mkdir(card37, 0700) == 0 + && mkdir(device0, 0700) == 0 && mkdir(device37, 0700) == 0 + && write_text_file(busy0, "99\n") + && write_text_file(busy37, "42\n")); + uint32_t basis_points = 0; + CHECK(lardon3d_resource_governor_internal_read_gpu_busy_at_root( + root, 37, &basis_points) + && basis_points == 4200); + /* A malformed retained card must never fall through to readable card0. */ + CHECK(write_text_file(busy37, "42 percent\n") + && !lardon3d_resource_governor_internal_read_gpu_busy_at_root( + root, 37, &basis_points) + && !lardon3d_resource_governor_internal_read_gpu_busy_at_root( + root, 38, &basis_points) + && !lardon3d_resource_governor_internal_read_gpu_busy_at_root( + root, 64, &basis_points)); + CHECK(unlink(busy0) == 0 && unlink(busy37) == 0 + && rmdir(device0) == 0 && rmdir(device37) == 0 + && rmdir(card0) == 0 && rmdir(card37) == 0 && rmdir(root) == 0); + return true; +} + typedef struct { Lardon3DResourceGovernor *governor; Lardon3DResourceSnapshot snapshot; @@ -134,7 +269,10 @@ run_test(void) )); CHECK(decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH); CHECK(decision.batch_size == 8); - CHECK(decision.cpu_threads == 15); + Lardon3DResourceCpuPolicyDiagnostic live_cpu_policy; + CHECK(lardon3d_resource_governor_internal_cpu_policy( + governor, &live_cpu_policy)); + CHECK(decision.cpu_threads == live_cpu_policy.compute_cpu_count); CHECK(decision.reason[0]); snapshot.swap_activity_known = true; @@ -197,6 +335,10 @@ run_test(void) CHECK(lardon3d_resource_governor_decide(governor, &snapshot, &request, &decision)); CHECK(decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH); CHECK(decision.batch_size == 4); + /* Even a fully available reported UMA payload aperture cannot bypass the + * host MemAvailable target/floor; it is observation, not separate RAM. */ + snapshot.gpu_memory_available_known = true; + snapshot.gpu_memory_available_bytes = MEBIBYTES(512); snapshot.memory_available_bytes = GIBIBYTES(3); CHECK(lardon3d_resource_governor_decide(governor, &snapshot, &request, &decision)); CHECK(decision.kind == LARDON3D_RESOURCE_WAIT); @@ -949,11 +1091,1631 @@ run_gate_g_boundary_test(void) return true; } +static Lardon3DTaskCapabilityEnvelope +adaptive_orb_envelope(void) +{ + Lardon3DResourceEstimate cpu = { + .memory_bytes_per_item = MEBIBYTES(10), + .minimum_batch_size = 1, + .maximum_batch_size = 12, + .desired_cpu_threads = 12, + .desired_io_slots = 1, + .task_class = LARDON3D_RESOURCE_TASK_CPU, + }; + Lardon3DResourceEstimate gpu = cpu; + gpu.gpu_memory_fixed_bytes = 640 * 1024; + gpu.desired_cpu_threads = 1; + gpu.desired_gpu_slots = 1; + return (Lardon3DTaskCapabilityEnvelope) { + .count = 2, + .capabilities = { + { + .estimate = gpu, + .backend = LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, + .inflight_limit = 1, + .preferred = true, + .batch_adaptive = true, + .requires_runtime_backend = true, + }, + { + .estimate = cpu, + .backend = LARDON3D_RESOURCE_BACKEND_CPU, + .inflight_limit = 1, + .helper_limit = 0, + .cpu_reducible = true, + .batch_adaptive = true, + }, + }, + }; +} + +static bool +run_cpu_topology_policy_test(void) +{ + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 16, + .page_size_bytes = 4096, + .memory_total_bytes = GIBIBYTES(16), + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy = { + .system_cpu_reserve = 4, + .maximum_cpu_load_ratio = 1.0, + .maximum_io_pressure_avg10 = 100.0, + .io_slot_capacity = 1, + }; + Lardon3DResourceGovernor *governor = + lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor && use_fixed_test_clock(governor)); + + Lardon3DResourceCpuTopologyInput topology = { + .affinity_available = true, + .topology_available = true, + .allowed_cpu_count = 16, + .topology_entry_count = 16, + }; + for (unsigned int cpu = 0; cpu < 16; ++cpu) { + topology.allowed_cpu_ids[cpu] = cpu; + topology.topology_entries[cpu] = + (Lardon3DResourceCpuTopologyEntry) { + .cpu_id = cpu, + .package_id = 0, + .core_id = cpu % 8, + }; + } + CHECK(lardon3d_resource_governor_internal_configure_cpu_topology( + governor, &topology)); + Lardon3DResourceCpuPolicyDiagnostic diagnostic; + CHECK(lardon3d_resource_governor_internal_cpu_policy( + governor, &diagnostic)); + CHECK(diagnostic.affinity_configured && !diagnostic.affinity_active + && !diagnostic.externally_constrained + && diagnostic.compute_cpu_count == 12 + && diagnostic.reserved_cpu_count == 4); + for (unsigned int cpu = 0; cpu < 16; ++cpu) { + bool reserved = cpu == 6 || cpu == 7 || cpu == 14 || cpu == 15; + CHECK(cpu_mask_has(diagnostic.allowed_mask, cpu)); + CHECK(cpu_mask_has(diagnostic.reserved_mask, cpu) == reserved); + CHECK(cpu_mask_has(diagnostic.compute_mask, cpu) == !reserved); + } + + /* Admission consumes the derived compute count, not logical total minus + * a second, unrelated reduction. */ + Lardon3DTaskCapabilityEnvelope envelope = { + .count = 1, + .capabilities = {{ + .estimate = { + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 16, + .task_class = LARDON3D_RESOURCE_TASK_CPU, + }, + .backend = LARDON3D_RESOURCE_BACKEND_CPU, + .inflight_limit = 1, + .cpu_reducible = true, + }}, + }; + Lardon3DResourceSnapshot snapshot = { + .memory_available_bytes = GIBIBYTES(12), + }; + Lardon3DResourceCapabilitySelection selection; + Lardon3DResourceReservation *reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.topology", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + + /* A caller already constrained to the desired twelve CPUs supplies the + * complete compute mask. Governor must not reserve four more. */ + topology = (Lardon3DResourceCpuTopologyInput) { + .affinity_available = true, + .topology_available = false, + .allowed_cpu_count = 12, + }; + size_t allowed = 0; + for (unsigned int cpu = 0; cpu < 16; ++cpu) { + if (cpu < 6 || (cpu >= 8 && cpu < 14)) { + topology.allowed_cpu_ids[allowed++] = cpu; + } + } + CHECK(allowed == 12 + && lardon3d_resource_governor_internal_configure_cpu_topology( + governor, &topology) + && lardon3d_resource_governor_internal_cpu_policy( + governor, &diagnostic)); + CHECK(diagnostic.affinity_configured + && diagnostic.externally_constrained + && diagnostic.compute_cpu_count == 12 + && diagnostic.reserved_cpu_count == 0); + for (unsigned int cpu = 0; cpu < 16; ++cpu) { + bool compute = cpu < 6 || (cpu >= 8 && cpu < 14); + CHECK(cpu_mask_has(diagnostic.compute_mask, cpu) == compute); + CHECK(!cpu_mask_has(diagnostic.reserved_mask, cpu)); + } + + /* With a full allowed mask but no topology, portable count budgeting + * remains available and no arbitrary CPU/sibling mask is fabricated. */ + topology = (Lardon3DResourceCpuTopologyInput) { + .affinity_available = true, + .topology_available = false, + .allowed_cpu_count = 16, + }; + for (unsigned int cpu = 0; cpu < 16; ++cpu) { + topology.allowed_cpu_ids[cpu] = cpu; + } + CHECK(lardon3d_resource_governor_internal_configure_cpu_topology( + governor, &topology) + && lardon3d_resource_governor_internal_cpu_policy( + governor, &diagnostic)); + CHECK(!diagnostic.affinity_configured && !diagnostic.affinity_active + && diagnostic.compute_cpu_count == 12 + && diagnostic.reserved_cpu_count == 4 + && strcmp(diagnostic.reason, + "fallback-portable-topology-unavailable") == 0); + for (size_t word = 0; word < LARDON3D_RESOURCE_CPU_MASK_WORDS; ++word) { + CHECK(diagnostic.compute_mask[word] == 0 + && diagnostic.reserved_mask[word] == 0); + } + lardon3d_resource_governor_destroy(governor); + + /* Read-only current-host assertion: when the caller really sees the + * unrestricted 0..15 topology, the production discovery must reproduce + * the validated compute/reserved masks. Other hosts skip this exact IDs. */ + CHECK(lardon3d_resource_policy_default(&profile, &policy)); + governor = lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor && lardon3d_resource_governor_internal_cpu_policy( + governor, &diagnostic)); + bool exact_host_mask = diagnostic.affinity_configured; + for (unsigned int cpu = 0; cpu < 16 && exact_host_mask; ++cpu) { + exact_host_mask = cpu_mask_has(diagnostic.allowed_mask, cpu); + } + if (exact_host_mask) { + CHECK(diagnostic.compute_cpu_count == 12 + && diagnostic.reserved_cpu_count == 4); + for (unsigned int cpu = 0; cpu < 16; ++cpu) { + bool reserved = cpu == 6 || cpu == 7 || cpu == 14 || cpu == 15; + CHECK(cpu_mask_has(diagnostic.reserved_mask, cpu) == reserved); + CHECK(cpu_mask_has(diagnostic.compute_mask, cpu) == !reserved); + } + } + lardon3d_resource_governor_destroy(governor); + return true; +} + +static bool +run_driver_runtime_policy_test(void) +{ + CHECK(unsetenv("MESA_SHADER_CACHE_DISABLE") == 0 + && lardon3d_resource_governor_internal_configure_driver_policy() + == LARDON3D_RESOURCE_DRIVER_POLICY_DEFAULTED + && getenv("MESA_SHADER_CACHE_DISABLE") + && strcmp(getenv("MESA_SHADER_CACHE_DISABLE"), "true") == 0); + + CHECK(lardon3d_resource_governor_internal_configure_driver_policy() + == LARDON3D_RESOURCE_DRIVER_POLICY_INHERITED_SAFE); + + /* An explicit request for Mesa disk-cache workers is not silently + * overwritten. Startup rejects it before threads or Vulkan can exist. */ + CHECK(setenv("MESA_SHADER_CACHE_DISABLE", "false", 1) == 0 + && lardon3d_resource_governor_internal_configure_driver_policy() + == LARDON3D_RESOURCE_DRIVER_POLICY_REJECTED_UNSAFE + && strcmp(getenv("MESA_SHADER_CACHE_DISABLE"), "false") == 0); + CHECK(setenv("MESA_SHADER_CACHE_DISABLE", "unsafe", 1) == 0 + && lardon3d_resource_governor_internal_configure_driver_policy() + == LARDON3D_RESOURCE_DRIVER_POLICY_REJECTED_UNSAFE + && strcmp(getenv("MESA_SHADER_CACHE_DISABLE"), "unsafe") == 0); + + /* Leave the test process in the production-safe inherited state before + * any later test creates a Queue or another pthread. */ + CHECK(setenv("MESA_SHADER_CACHE_DISABLE", "1", 1) == 0 + && lardon3d_resource_governor_internal_configure_driver_policy() + == LARDON3D_RESOURCE_DRIVER_POLICY_INHERITED_SAFE); + return true; +} + +static bool +run_capability_governor_test(void) +{ + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 16, + .page_size_bytes = 4096, + .memory_total_bytes = GIBIBYTES(16), + .gpu_available = true, + .gpu_memory_known = true, + .gpu_uses_shared_memory = true, + .gpu_memory_total_bytes = MEBIBYTES(512), + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy; + CHECK(lardon3d_resource_policy_default(&profile, &policy)); + policy.maximum_cpu_load_ratio = 1.0; + Lardon3DResourceGovernor *governor = + lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor && use_fixed_test_clock(governor)); + CHECK(lardon3d_resource_governor_internal_set_backend_available( + governor, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, true)); + Lardon3DResourceSnapshot snapshot = { + .captured_at = {0}, + .memory_available_bytes = GIBIBYTES(12), + .swap_activity_known = true, + }; + Lardon3DTaskCapabilityEnvelope envelope = adaptive_orb_envelope(); + Lardon3DResourceCapabilitySelection selection; + Lardon3DResourceReservation *reservation = NULL; + + /* AUTO starts at the validated minimum and prefers the usable GPU. Two + * completed healthy sequences, rather than one throughput sample, ramp + * the next immutable sequence to batch two. */ + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.capability.backend == + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN); + CHECK(selection.decision.batch_size == 1 && + strcmp(selection.reason, "healthy-slow-start") == 0); + Lardon3DResourceAvailability uma_charge; + CHECK(lardon3d_resource_governor_availability( + governor, &snapshot, &uma_charge)); + CHECK(uma_charge.memory_reserved_bytes == MEBIBYTES(10) + 640 * 1024 && + uma_charge.gpu_memory_reserved_bytes == 640 * 1024); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.run", 1, &selection, 1000000000ULL, 1)); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.run", 1, &selection, 1000000000ULL, 1)); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 2); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + + Lardon3DResourceSequenceDiagnostic diagnostic; + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.run", 1, &diagnostic)); + CHECK(diagnostic.backend == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + diagnostic.actual_backend == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN && + diagnostic.helper_limit == 0 && + diagnostic.batch_size == 2 && diagnostic.gpu_slots == 1 && + diagnostic.gpu_memory_bytes == 640 * 1024); + + /* The higher-resource batch-two trial needs two independently material + * durable-throughput gains before the next sequence may grow again. */ + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.run", 1, &selection, 1000000000ULL, 2)); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.run", 1, &diagnostic) && + strcmp(diagnostic.reason, "throughput-trial") == 0); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 2); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.run", 1, &selection, 1000000000ULL, 2)); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.run", 1, &diagnostic) && + strcmp(diagnostic.reason, "throughput-improved") == 0); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 4 && + strcmp(selection.reason, "throughput-trial") == 0); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + + /* Establish batch four as the accepted capability before pressure. The + * next operational trial is eight, so admitting one here would expose a + * stale pre-pressure adaptive maximum. */ + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.run", 1, &selection, 1000000000ULL, 4)); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 4); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.run", 1, &selection, 1000000000ULL, 4)); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.run", 1, &diagnostic) && + diagnostic.batch_size == 4 && + strcmp(diagnostic.reason, "throughput-improved") == 0); + + /* Equal throughput at batch two is not a gain: the Governor returns to + * the last accepted batch and stops growth despite a free GPU slot. */ + for (size_t sample = 0; sample < 2; ++sample) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.no_gain", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.no_gain", 1, &selection, 1000000000ULL, 1)); + } + for (size_t sample = 0; sample < 2; ++sample) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.no_gain", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 2); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.no_gain", 1, &selection, 2000000000ULL, 2)); + } + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.no_gain", 1, &diagnostic) && + strcmp(diagnostic.reason, "throughput-no-gain") == 0); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.no_gain", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 1 && + strcmp(selection.reason, "throughput-no-gain") == 0); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + + /* One high outlier leaves the trial unchanged; a second non-improving + * observation decides conservatively without oscillating 1->2->1 early. */ + for (size_t sample = 0; sample < 2; ++sample) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.noisy", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.noisy", 1, &selection, 1000000000ULL, 1)); + } + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.noisy", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 2); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.noisy", 1, &selection, 100000000ULL, 2)); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.noisy", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 2); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.noisy", 1, &selection, 4000000000ULL, 2)); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.noisy", 1, &diagnostic) && + strcmp(diagnostic.reason, "throughput-no-gain") == 0); + + /* Complete CPU fallback is scientifically valid, but it cannot train a + * pure Vulkan trial. The next GPU admission stays at the accepted batch + * while two fresh pure observations rebuild its baseline. */ + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.fallback_hold", 1, &envelope, + &selection, &reservation)); + CHECK(reservation && selection.decision.batch_size == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence_execution( + governor, "matcher.fallback_hold", 1, &selection, 1000000000ULL, 1, + LARDON3D_RESOURCE_BACKEND_CPU, "whole-pair-cpu-fallback")); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.fallback_hold", 1, &diagnostic) + && strcmp(diagnostic.reason, "backend-fallback-hold") == 0); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.fallback_hold", 1, &envelope, + &selection, &reservation)); + CHECK(reservation && selection.capability.backend + == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN + && selection.decision.batch_size == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + + Lardon3DResourceExecutionMetrics starvation_metrics = { + .vulkan_submits = 1, + .vulkan_completions = 1, + .vulkan_starvation_ns = 1000000, + }; + Lardon3DResourceTelemetryRaw unknown_gpu_raw = {0}; + Lardon3DResourceHostTelemetry unknown_gpu_telemetry; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &unknown_gpu_raw, &unknown_gpu_telemetry) + && !unknown_gpu_telemetry.gpu_busy_known); + for (unsigned int observation = 0; observation < 2; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.starvation", 1, &envelope, + &selection, &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 1 + && selection.decision.batch_size == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence_execution_metrics( + governor, "matcher.starvation", 1, &selection, 1000000000ULL, 1, + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, "vulkan-completed", + &starvation_metrics)); + } + for (unsigned int observation = 0; observation < 2; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.starvation", 1, &envelope, + &selection, &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 1 + && selection.decision.batch_size == 2 + && selection.capability.inflight_limit == 1 + && selection.capability.helper_limit == 0 + && strcmp(selection.reason, + "gpu-starvation-throughput-trial") == 0); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence_execution_metrics( + governor, "matcher.starvation", 1, &selection, 1000000000ULL, 2, + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, "vulkan-completed", + &starvation_metrics)); + } + Lardon3DResourceTelemetryRaw busy_raw = { + .gpu_busy_percent = "95\n", + }; + Lardon3DResourceHostTelemetry busy_telemetry; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &busy_raw, &busy_telemetry)); + for (unsigned int observation = 0; observation < 2; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.starvation", 1, &envelope, + &selection, &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 1 + && selection.decision.batch_size == 4); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence_execution_metrics( + governor, "matcher.starvation", 1, &selection, 2000000000ULL, 4, + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, "vulkan-completed", + &starvation_metrics)); + } + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.starvation", 1, &envelope, + &selection, &reservation)); + CHECK(reservation && selection.decision.batch_size == 2 + && strcmp(selection.reason, "throughput-no-gain") == 0); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.starvation", 1, &diagnostic) + && diagnostic.host.gpu_busy_known + && diagnostic.host.gpu_busy_basis_points == 9500 + && diagnostic.execution.vulkan_starvation_ns == 1000000); + + /* The first swap-I/O-only signal changes GREEN to YELLOW inside admission. + * That same immutable sequence must discard the accepted batch-four + * history, reserve only batch one, and report the pressure reason. */ + snapshot.swap_pages_out = 1; + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.pressure == + LARDON3D_RESOURCE_PRESSURE_YELLOW && + selection.decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH && + selection.decision.batch_size == 1 && + strcmp(selection.reason, "pressure-decrease") == 0); + CHECK(lardon3d_resource_governor_availability( + governor, &snapshot, &uma_charge)); + CHECK(uma_charge.memory_reserved_bytes == MEBIBYTES(10) + 640 * 1024 && + uma_charge.gpu_memory_reserved_bytes == 640 * 1024); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.run", 1, &diagnostic) && + diagnostic.batch_size == 1 && + diagnostic.pressure == LARDON3D_RESOURCE_PRESSURE_YELLOW && + strcmp(diagnostic.reason, "pressure-decrease") == 0); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + + /* Three healthy observations clear YELLOW. Two fresh batch-one samples + * then reopen only batch two. Keeping that limit after the host slow-start + * reaches four proves the pre-pressure accepted batch-four trial reset. */ + for (size_t observation = 0; observation < 3; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + } + CHECK(lardon3d_resource_governor_pressure(governor) == + LARDON3D_RESOURCE_PRESSURE_GREEN); + for (size_t sample = 0; sample < 2; ++sample) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.run", 1, &selection, 1000000000ULL, 1)); + } + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 2 && + strcmp(selection.reason, "throughput-trial") == 0); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + for (size_t observation = 0; observation < 3; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 2); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + } + + /* A single PSI observation immediately throttles admission but does not + * become RED until sustained. Active swap deltas use the same path. */ + snapshot.memory_pressure_known = true; + snapshot.memory_pressure_avg10 = 100.0; + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(!reservation && selection.decision.kind == LARDON3D_RESOURCE_WAIT && + selection.pressure == LARDON3D_RESOURCE_PRESSURE_YELLOW); + snapshot.memory_pressure_known = false; + snapshot.swap_pages_out = 2; + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(!reservation && selection.decision.kind == LARDON3D_RESOURCE_WAIT && + selection.pressure == LARDON3D_RESOURCE_PRESSURE_RED); + + /* RED and YELLOW each require three healthy observations. This bounded + * recovery hysteresis prevents one alternating sample from oscillating. */ + snapshot.swap_pages_out = 2; + for (size_t observation = 0; observation < 6; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + if (reservation) + CHECK(lardon3d_resource_governor_release(governor, reservation)); + } + CHECK(lardon3d_resource_governor_pressure(governor) == + LARDON3D_RESOURCE_PRESSURE_GREEN); + /* Pressure reset discarded matcher.run's batch-four trial. After the + * bounded recovery streak, two new baseline observations reopen only a + * controlled batch-two trial. */ + for (size_t sample = 0; sample < 2; ++sample) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.run", 1, &selection, 1000000000ULL, 1)); + } + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 2 && + strcmp(selection.reason, "throughput-trial") == 0); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + snapshot.memory_available_bytes = GIBIBYTES(3); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(!reservation && selection.pressure == + LARDON3D_RESOURCE_PRESSURE_YELLOW); + snapshot.memory_available_bytes = GIBIBYTES(2); + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.run", 1, &envelope, &selection, + &reservation)); + CHECK(!reservation && selection.pressure == + LARDON3D_RESOURCE_PRESSURE_RED); + lardon3d_resource_governor_destroy(governor); + + /* CPU and batch are independent dimensions: CPU can reduce to two while + * the callback-supported batch remains twelve. */ + profile.gpu_available = false; + profile.gpu_memory_known = false; + profile.gpu_uses_shared_memory = false; + profile.gpu_memory_total_bytes = 0; + policy = (Lardon3DResourcePolicy) { + .system_cpu_reserve = 14, + .maximum_cpu_load_ratio = 1.0, + .maximum_io_pressure_avg10 = 100.0, + .io_slot_capacity = 1, + }; + governor = lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor && use_fixed_test_clock(governor)); + snapshot.memory_available_bytes = GIBIBYTES(12); + Lardon3DTaskCapabilityEnvelope cpu_only = { + .count = 1, + .capabilities = {{ + .estimate = envelope.capabilities[1].estimate, + .backend = LARDON3D_RESOURCE_BACKEND_FIXED, + .inflight_limit = 1, + .cpu_reducible = true, + }}, + }; + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "candidate_pairs.generate", 1, &cpu_only, + &selection, &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 1 && + selection.decision.batch_size == 12 && + strcmp(selection.reason, "healthy-slow-start") == 0); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + lardon3d_resource_governor_destroy(governor); + return true; +} + +static bool +run_inflight_feedback_test(void) +{ + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 16, + .page_size_bytes = 4096, + .memory_total_bytes = GIBIBYTES(16), + .gpu_available = true, + .gpu_memory_known = true, + .gpu_uses_shared_memory = true, + .gpu_memory_total_bytes = MEBIBYTES(512), + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy; + CHECK(lardon3d_resource_policy_default(&profile, &policy)); + policy.maximum_cpu_load_ratio = 1.0; + Lardon3DResourceGovernor *governor = + lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor && use_fixed_test_clock(governor)); + CHECK(lardon3d_resource_governor_internal_set_backend_available( + governor, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, true)); + Lardon3DTaskCapabilityEnvelope envelope = { + .count = 1, + .capabilities = {{ + .estimate = { + .memory_bytes_per_item = MEBIBYTES(10), + .minimum_batch_size = 2, + .maximum_batch_size = 2, + .desired_cpu_threads = 1, + .desired_gpu_slots = 1, + .desired_io_slots = 1, + .task_class = LARDON3D_RESOURCE_TASK_MIXED, + }, + .backend = LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, + .inflight_limit = 2, + .minimum_inflight_limit = 1, + .gpu_memory_bytes_per_inflight = 640 * 1024, + .preferred = true, + .inflight_adaptive = true, + .requires_runtime_backend = true, + }}, + }; + Lardon3DResourceSnapshot snapshot = { + .memory_available_bytes = GIBIBYTES(12), + .swap_activity_known = true, + }; + Lardon3DResourceCapabilitySelection selection; + Lardon3DResourceReservation *reservation = NULL; + + /* Two healthy baseline observations are required before depth two, and + * the immutable reservation charges one 640 KiB mapped payload per slot + * exactly once against shared host memory. */ + for (unsigned int observation = 0; observation < 2; ++observation) { + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.depth", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.inflight_limit == 1); + Lardon3DResourceAvailability availability; + CHECK(lardon3d_resource_governor_availability( + governor, &snapshot, &availability)); + CHECK(availability.memory_reserved_bytes == MEBIBYTES(20) + 640 * 1024 + && availability.gpu_memory_reserved_bytes == 640 * 1024); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.depth", 1, &selection, 2000000000ULL, 2)); + } + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.depth", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.inflight_limit == 2 + && strcmp(selection.reason, "inflight-throughput-trial") == 0); + Lardon3DResourceAvailability availability; + CHECK(lardon3d_resource_governor_availability( + governor, &snapshot, &availability)); + CHECK(availability.memory_reserved_bytes == MEBIBYTES(20) + 1280 * 1024 + && availability.gpu_memory_reserved_bytes == 1280 * 1024); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.depth", 1, &selection, 1600000000ULL, 2)); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.depth", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.inflight_limit == 2); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.depth", 1, &selection, 1600000000ULL, 2)); + Lardon3DResourceSequenceDiagnostic diagnostic; + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.depth", 1, &diagnostic) + && diagnostic.inflight_limit == 2 + && strcmp(diagnostic.reason, "inflight-throughput-improved") == 0); + + for (unsigned int observation = 0; observation < 2; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.depth.no_gain", 1, &envelope, + &selection, &reservation)); + CHECK(reservation); + CHECK(selection.inflight_limit == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.depth.no_gain", 1, &selection, + 2000000000ULL, 2)); + } + for (unsigned int observation = 0; observation < 2; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.depth.no_gain", 1, &envelope, + &selection, &reservation)); + CHECK(reservation && selection.inflight_limit == 2); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.depth.no_gain", 1, &selection, + 2000000000ULL, 2)); + } + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.depth.no_gain", 1, &envelope, + &selection, &reservation)); + CHECK(reservation && selection.inflight_limit == 1 + && strcmp(selection.reason, "inflight-throughput-no-gain") == 0); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + + /* The first active swap-I/O delta clamps the very same admission to depth + * one and its exact reservation, then discards the accepted trial history. */ + snapshot.swap_pages_out = 1; + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.depth", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.pressure == LARDON3D_RESOURCE_PRESSURE_YELLOW + && selection.decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH + && selection.inflight_limit == 1 + && strcmp(selection.reason, "pressure-decrease") == 0); + CHECK(lardon3d_resource_governor_availability( + governor, &snapshot, &availability)); + CHECK(availability.memory_reserved_bytes == MEBIBYTES(20) + 640 * 1024 + && availability.gpu_memory_reserved_bytes == 640 * 1024); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.depth", 1, &diagnostic) + && diagnostic.inflight_limit == 1 + && strcmp(diagnostic.reason, "pressure-decrease") == 0); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + + /* Recovery requires the existing healthy pressure streak and then a fresh + * two-observation baseline before depth two can reopen. */ + for (unsigned int observation = 0; observation < 3; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.depth", 1, &envelope, &selection, + &reservation)); + if (!reservation) { + CHECK(selection.decision.kind == LARDON3D_RESOURCE_WAIT); + } else { + CHECK(selection.inflight_limit == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + } + } + CHECK(lardon3d_resource_governor_pressure(governor) + == LARDON3D_RESOURCE_PRESSURE_GREEN); + + /* Gate G's generic slow-start limit also rebuilds after pressure. This + * fixture has a fixed batch minimum of two so its first two healthy + * attempts remain WAIT; the third reopens that batch at depth one. */ + for (unsigned int observation = 0; observation < 3; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.depth", 1, &envelope, &selection, + &reservation)); + if (observation < 2) { + CHECK(!reservation + && selection.decision.kind == LARDON3D_RESOURCE_WAIT); + } else { + CHECK(reservation && selection.inflight_limit == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + } + } + + for (unsigned int observation = 0; observation < 2; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.depth", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.inflight_limit == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "matcher.depth", 1, &selection, 2000000000ULL, 2)); + } + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.depth", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.inflight_limit == 2 + && strcmp(selection.reason, "inflight-throughput-trial") == 0); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + lardon3d_resource_governor_destroy(governor); + return true; +} + +static bool +run_private_telemetry_test(void) +{ + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 16, + .page_size_bytes = 4096, + .memory_total_bytes = GIBIBYTES(16), + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy; + CHECK(lardon3d_resource_policy_default(&profile, &policy)); + policy.maximum_cpu_load_ratio = 1.0; + Lardon3DResourceGovernor *governor = + lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor && use_fixed_test_clock(governor)); + Lardon3DResourceCpuTopologyInput topology = { + .affinity_available = true, + .allowed_cpu_count = 2, + .allowed_cpu_ids = {0, 2}, + }; + CHECK(lardon3d_resource_governor_internal_configure_cpu_topology( + governor, &topology)); + + const char *stat_first = + "cpu 0 0 0 0 0 0 0 0\n" + "cpu0 100 0 100 800 0 0 0 0\n" + "cpu1 9 0 9 9 0 0 0 0\n" + "cpu2 200 0 100 700 0 0 0 0\n"; + const char *stat_second = + "cpu 0 0 0 0 0 0 0 0\n" + "cpu0 130 0 130 840 0 0 0 0\n" + "cpu1 9 0 9 9 0 0 0 0\n" + "cpu2 220 0 120 760 0 0 0 0\n"; + Lardon3DResourceTelemetryRaw raw = { + .proc_stat = stat_first, + .meminfo = "MemTotal: 16777216 kB\nMemAvailable: 4194304 kB\n", + .memory_psi = + "some avg10=1.25 avg60=0.20 avg300=0.10 total=9\n" + "full avg10=0.50 avg60=0.10 avg300=0.01 total=2\n", + .io_psi = + "some avg10=2.75 avg60=0.20 avg300=0.10 total=9\n" + "full avg10=0.25 avg60=0.10 avg300=0.01 total=2\n", + .vmstat = "pgpgin 1\npswpin 10\npswpout 20\n", + .process_status = "Name:\ttest\nVmRSS:\t1024 kB\nVmHWM:\t2048 kB\n", + .gpu_busy_percent = "42\n", + }; + Lardon3DResourceHostTelemetry telemetry; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &raw, &telemetry)); + CHECK(!telemetry.compute_pool_utilization_known + && !telemetry.swap_delta_known + && telemetry.memory_available_known + && telemetry.memory_available_bytes == GIBIBYTES(4) + && telemetry.memory_psi_some_known + && telemetry.memory_psi_some_basis_points == 125 + && telemetry.memory_psi_full_basis_points == 50 + && telemetry.io_psi_some_basis_points == 275 + && telemetry.io_psi_full_basis_points == 25 + && telemetry.gpu_busy_known + && telemetry.gpu_busy_basis_points == 4200 + && telemetry.process_rss_bytes == MEBIBYTES(1) + && telemetry.process_peak_rss_bytes == MEBIBYTES(2)); + + raw.proc_stat = stat_second; + raw.vmstat = "pswpin 13\npswpout 27\n"; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &raw, &telemetry)); + CHECK(telemetry.compute_pool_utilization_known + && telemetry.compute_pool_utilization_basis_points == 5000 + && telemetry.swap_delta_known + && telemetry.swap_pages_in_delta == 3 + && telemetry.swap_pages_out_delta == 7); + + /* Zero deltas, counter regression/wrap, malformed selected CPU fields, + * signs, and trailing GPU text all degrade to unknown without reusing a + * stale low-pressure value. */ + raw.vmstat = "pswpin 1\npswpout 2\n"; + raw.gpu_busy_percent = "-1\n"; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &raw, &telemetry)); + CHECK(!telemetry.compute_pool_utilization_known + && !telemetry.swap_delta_known && !telemetry.gpu_busy_known); + raw.proc_stat = "cpu0 1 2\ncpu2 1 2 3 4\n"; + raw.gpu_busy_percent = "42 percent\n"; + raw.memory_psi = + "some avg10=-1.00 avg60=0.00 avg300=0.00 total=1\n" + "full avg10=0.10tail avg60=0.00 avg300=0.00 total=1\n"; + raw.process_status = NULL; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &raw, &telemetry)); + CHECK(!telemetry.compute_pool_utilization_known + && !telemetry.gpu_busy_known + && !telemetry.memory_psi_some_known + && !telemetry.memory_psi_full_known + && !telemetry.process_rss_known); + + /* PSI tokens are exact fields, not substring searches or rounded input. */ + raw = (Lardon3DResourceTelemetryRaw) { + .memory_psi = + "some xavg10=1.25 avg60=0.00 total=1\n" + "full avg10=0.10 avg60=0.00 total=1\n", + }; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &raw, &telemetry) + && !telemetry.memory_psi_some_known + && telemetry.memory_psi_full_known); + raw.memory_psi = + "some avg10=1.234 avg60=0.00 total=1\n" + "full avg10=0.10 avg10=0.20 total=1\n"; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &raw, &telemetry) + && !telemetry.memory_psi_some_known + && !telemetry.memory_psi_full_known); + raw.memory_psi = + "some avg10=+1.00 avg60=0.00 total=1\n" + "full avg10=.10 avg60=0.00 total=1\n"; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &raw, &telemetry) + && !telemetry.memory_psi_some_known + && !telemetry.memory_psi_full_known); + + /* Every selected CPU must occur exactly once. A duplicate cpu0 cannot + * substitute for the missing cpu2, and per-CPU regression cannot be + * hidden by growth on its sibling. */ + raw = (Lardon3DResourceTelemetryRaw) { + .proc_stat = stat_first, + }; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &raw, &telemetry) + && !telemetry.compute_pool_utilization_known); + raw.proc_stat = + "cpu0 130 0 130 840 0 0 0 0\n" + "cpu0 140 0 140 850 0 0 0 0\n"; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &raw, &telemetry) + && !telemetry.compute_pool_utilization_known); + raw.proc_stat = stat_first; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &raw, &telemetry)); + raw.proc_stat = + "cpu0 99 0 100 800 0 0 0 0\n" + "cpu2 400 0 200 900 0 0 0 0\n"; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &raw, &telemetry) + && !telemetry.compute_pool_utilization_known); + raw.proc_stat = + "cpu0 18446744073709551615 1 0 0\n" + "cpu2 1 0 0 0\n"; + CHECK(lardon3d_resource_governor_internal_sample_telemetry_raw( + governor, &raw, &telemetry) + && !telemetry.compute_pool_utilization_known); + + Lardon3DTaskCapabilityEnvelope envelope = { + .count = 1, + .capabilities = {{ + .estimate = { + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 1, + }, + .backend = LARDON3D_RESOURCE_BACKEND_FIXED, + .inflight_limit = 1, + }}, + }; + Lardon3DResourceSnapshot snapshot = { + .memory_available_bytes = GIBIBYTES(8), + }; + Lardon3DResourceCapabilitySelection selection; + Lardon3DResourceReservation *reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.telemetry", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && lardon3d_resource_governor_release( + governor, reservation)); + Lardon3DResourceExecutionMetrics metrics = { + .vulkan_submits = 2, + .vulkan_completions = 2, + .vulkan_submit_cpu_ns = 10, + .vulkan_fence_wait_ns = 11, + .vulkan_gpu_ns = 12, + .vulkan_starvation_ns = 13, + .matcher_cpu_ns = 14, + .publication_ns = 15, + }; + CHECK(lardon3d_resource_governor_internal_record_sequence_execution_metrics( + governor, "test.telemetry", 1, &selection, 1000, 1, + LARDON3D_RESOURCE_BACKEND_FIXED, "fixed-completed", &metrics)); + Lardon3DResourceSequenceDiagnostic diagnostic; + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "test.telemetry", 1, &diagnostic) + && diagnostic.serial > 0 + && diagnostic.execution.vulkan_submits == 2 + && diagnostic.execution.publication_ns == 15 + && diagnostic.execution.local_ineligible_fallback_items == 0 + && diagnostic.execution.backend_failure_fallback_items == 0 + && diagnostic.execution.backend_other_fallback_items == 0 + && !lardon3d_resource_governor_internal_diagnostic_since( + governor, "test.telemetry", 1, diagnostic.serial, &diagnostic)); + char formatted[4096]; + CHECK(lardon3d_resource_governor_internal_format_diagnostic( + &diagnostic, formatted, sizeof(formatted)) + && strstr(formatted, "vk_submit=2") + && strstr(formatted, "fallback_items=0/0/0") + && strstr(formatted, "fallback_items_saturated=0") + && strstr(formatted, "reason=fixed-envelope") + && !lardon3d_resource_governor_internal_format_diagnostic( + &diagnostic, formatted, 8)); + + /* Serial saturation intentionally stops since-cursor notification, but + * last_diagnostic must still select the most recently updated backend. */ + Lardon3DResourceCapabilitySelection fixed_selection = selection; + CHECK(lardon3d_resource_governor_internal_set_diagnostic_serial( + governor, UINT64_MAX - 1) + && lardon3d_resource_governor_internal_record_sequence( + governor, "test.telemetry", 1, &fixed_selection, 1000, 1)); + Lardon3DTaskCapabilityEnvelope cpu_envelope = envelope; + cpu_envelope.capabilities[0].backend = LARDON3D_RESOURCE_BACKEND_CPU; + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.telemetry", 1, &cpu_envelope, &selection, + &reservation)); + CHECK(reservation && lardon3d_resource_governor_release( + governor, reservation) + && lardon3d_resource_governor_internal_last_diagnostic( + governor, "test.telemetry", 1, &diagnostic) + && diagnostic.serial == UINT64_MAX + && diagnostic.backend == LARDON3D_RESOURCE_BACKEND_CPU); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "test.telemetry", 1, &fixed_selection, 1000, 1) + && lardon3d_resource_governor_internal_last_diagnostic( + governor, "test.telemetry", 1, &diagnostic) + && diagnostic.serial == UINT64_MAX + && diagnostic.backend == LARDON3D_RESOURCE_BACKEND_FIXED); + /* Polling may coalesce serial changes, but the fixed-size aggregate must + * retain every admission/completed sequence and must not double count the + * CPU admission that has not yet produced durable work. */ + Lardon3DResourceSequenceAggregate aggregate; + CHECK(lardon3d_resource_governor_internal_sequence_aggregate( + governor, "test.telemetry", 1, &aggregate) + && !aggregate.saturated + && aggregate.admission_count == 2 + && aggregate.sequence_count == 3 + && aggregate.durable_items == 3 + && aggregate.total_wall_time_ns == 3000 + && aggregate.contract_change_count == 1 + && aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_FIXED] == 1 + && aggregate.selected_backend_admissions[ + LARDON3D_RESOURCE_BACKEND_CPU] == 1 + && aggregate.actual_backend_sequences[ + LARDON3D_RESOURCE_BACKEND_FIXED] == 3 + && aggregate.vulkan_submits == 2 + && aggregate.vulkan_completions == 2 + && aggregate.vulkan_submit_cpu_ns == 10 + && aggregate.vulkan_fence_wait_ns == 11 + && aggregate.vulkan_gpu_known_sequences == 0 + && aggregate.vulkan_gpu_ns == 0 + && aggregate.vulkan_starvation_ns == 13 + && aggregate.matcher_cpu_ns == 14 + && aggregate.publication_ns == 15 + && aggregate.local_ineligible_fallback_items == 0 + && aggregate.backend_failure_fallback_items == 0 + && aggregate.backend_other_fallback_items == 0); + + /* Private telemetry has a bounded lifetime, but its fixed counters still + * fail closed under injected saturation: they never wrap to a plausible + * small comparator value. */ + Lardon3DResourceCapabilitySelection saturation_selection = selection; + saturation_selection.capability.backend = + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN; + CHECK(lardon3d_resource_governor_internal_record_fallback_items( + governor, "test.fallback-saturation", 1, &saturation_selection, + LARDON3D_RESOURCE_FALLBACK_ITEM_LOCAL_INELIGIBLE, UINT64_MAX)); + CHECK(lardon3d_resource_governor_internal_record_fallback_items( + governor, "test.fallback-saturation", 1, &saturation_selection, + LARDON3D_RESOURCE_FALLBACK_ITEM_LOCAL_INELIGIBLE, 1)); + CHECK(lardon3d_resource_governor_internal_sequence_aggregate( + governor, "test.fallback-saturation", 1, &aggregate) + && aggregate.saturated + && aggregate.local_ineligible_fallback_items == UINT64_MAX + && aggregate.sequence_count == 0 + && aggregate.durable_items == 0); + lardon3d_resource_governor_destroy(governor); + return true; +} + +static bool +record_gpu_feedback_sample( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceSnapshot *snapshot, + const Lardon3DTaskCapabilityEnvelope *envelope, + const char *task_kind, + size_t expected_batch, + uint64_t wall_time_ns, + size_t items_completed, + Lardon3DResourceBackend actual_backend +) +{ + Lardon3DResourceCapabilitySelection selection; + Lardon3DResourceReservation *reservation = NULL; + bool ok = lardon3d_resource_governor_internal_reserve_capability( + governor, snapshot, task_kind, 1, envelope, &selection, + &reservation) + && reservation + && selection.capability.backend + == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN + && selection.decision.batch_size == expected_batch; + if (reservation) { + bool released = lardon3d_resource_governor_release( + governor, reservation); + reservation = NULL; + ok = ok && released; + } + if (!ok) { + return false; + } + return lardon3d_resource_governor_internal_record_sequence_execution( + governor, task_kind, 1, &selection, wall_time_ns, items_completed, + actual_backend, + actual_backend == LARDON3D_RESOURCE_BACKEND_ORB_VULKAN + ? "vulkan-completed" : "whole-pair-cpu-fallback"); +} + +static bool +run_sustained_gpu_batch_feedback_test(void) +{ + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 16, + .page_size_bytes = 4096, + .memory_total_bytes = GIBIBYTES(16), + .gpu_available = true, + .gpu_memory_known = true, + .gpu_uses_shared_memory = true, + .gpu_memory_total_bytes = MEBIBYTES(512), + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy; + CHECK(lardon3d_resource_policy_default(&profile, &policy)); + policy.maximum_cpu_load_ratio = 1.0; + Lardon3DResourceGovernor *governor = + lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor && use_fixed_test_clock(governor)); + CHECK(lardon3d_resource_governor_internal_set_backend_available( + governor, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, true)); + Lardon3DTaskCapabilityEnvelope envelope = adaptive_orb_envelope(); + envelope.count = 1; + envelope.capabilities[0].estimate.maximum_batch_size = 8; + envelope.capabilities[0].sustained_gpu_batch_feedback = true; + Lardon3DResourceSnapshot snapshot = { + .memory_available_bytes = GIBIBYTES(12), + .swap_activity_known = true, + }; + Lardon3DResourceSequenceDiagnostic diagnostic; + Lardon3DResourceCapabilitySelection selection; + Lardon3DResourceReservation *reservation = NULL; + + /* Eight pure observations establish the baseline. The eighth opens the + * batch-two trial; neither one nor two samples can decide a GPU batch. */ + for (unsigned int sample = 0; sample < 8; ++sample) { + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.ramp", 1, + 1000000000ULL, 1, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.gpu.ramp", 1, &diagnostic)); + CHECK(strcmp(diagnostic.reason, + sample == 7 ? "gpu-batch-throughput-trial" + : "gpu-batch-baseline") == 0); + } + const size_t trials[] = {2, 4, 8}; + for (size_t step = 0; step < sizeof(trials) / sizeof(trials[0]); ++step) { + for (unsigned int sample = 0; sample < 8; ++sample) { + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.ramp", + trials[step], 1000000000ULL, trials[step], + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.gpu.ramp", 1, &diagnostic)); + CHECK(strcmp(diagnostic.reason, + sample == 7 ? "gpu-batch-throughput-improved" + : "gpu-batch-throughput-trial") == 0); + } + } + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.gpu.ramp", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 8 + && selection.capability.estimate.maximum_batch_size == 8 + && strcmp(selection.reason, "gpu-first") == 0 + && lardon3d_resource_governor_release(governor, reservation)); + + /* Eight equal-rate trial observations reject growth. One high and one low + * sample in an independent trial remain undecided and cannot oscillate. */ + for (unsigned int sample = 0; sample < 8; ++sample) { + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.no-gain", 1, + 1000000000ULL, 1, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + } + for (unsigned int sample = 0; sample < 8; ++sample) { + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.no-gain", 2, + 2000000000ULL, 2, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + } + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.gpu.no-gain", 1, &diagnostic) + && strcmp(diagnostic.reason, "gpu-batch-throughput-no-gain") == 0); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.gpu.no-gain", 1, &envelope, + &selection, &reservation)); + CHECK(reservation && selection.decision.batch_size == 1 + && strcmp(selection.reason, "gpu-batch-throughput-no-gain") == 0 + && lardon3d_resource_governor_release(governor, reservation)); + + for (unsigned int sample = 0; sample < 8; ++sample) { + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.noisy", 1, + 1000000000ULL, 1, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + } + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.noisy", 2, + 100000000ULL, 2, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.noisy", 2, + 4000000000ULL, 2, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "matcher.gpu.noisy", 1, &diagnostic) + && strcmp(diagnostic.reason, "gpu-batch-throughput-trial") == 0); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.gpu.noisy", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.batch_size == 2 + && strcmp(selection.reason, "gpu-batch-throughput-trial") == 0 + && lardon3d_resource_governor_release(governor, reservation)); + + /* Pressure during a trial clamps the same admission to batch one and + * clears every sample. Recovery, zero-work and fallback each require a + * new consecutive pure eight-observation baseline. */ + for (unsigned int sample = 0; sample < 8; ++sample) { + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.pressure", 1, + 1000000000ULL, 1, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + } + for (unsigned int sample = 0; sample < 3; ++sample) { + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.pressure", 2, + 1000000000ULL, 2, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + } + snapshot.swap_pages_out = 1; + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.gpu.pressure", 1, &envelope, + &selection, &reservation)); + CHECK(reservation && selection.pressure + == LARDON3D_RESOURCE_PRESSURE_YELLOW + && selection.decision.batch_size == 1 + && strcmp(selection.reason, "pressure-decrease") == 0 + && lardon3d_resource_governor_release(governor, reservation)); + for (unsigned int sample = 0; sample < 3; ++sample) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.gpu.pressure", 1, &envelope, + &selection, &reservation)); + CHECK(reservation && selection.decision.batch_size == 1 + && lardon3d_resource_governor_release(governor, reservation)); + } + CHECK(lardon3d_resource_governor_pressure(governor) + == LARDON3D_RESOURCE_PRESSURE_GREEN); + for (unsigned int sample = 0; sample < 2; ++sample) { + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.pressure", 1, + 1000000000ULL, 1, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + } + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.pressure", 1, + 1000000000ULL, 0, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + for (unsigned int sample = 0; sample < 2; ++sample) { + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.pressure", 1, + 1000000000ULL, 1, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + } + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.pressure", 1, + 1000000000ULL, 1, LARDON3D_RESOURCE_BACKEND_CPU)); + for (unsigned int sample = 0; sample < 7; ++sample) { + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.pressure", 1, + 1000000000ULL, 1, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + } + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.gpu.pressure", 1, &envelope, + &selection, &reservation)); + CHECK(reservation && selection.decision.batch_size == 1 + && lardon3d_resource_governor_release(governor, reservation)); + CHECK(record_gpu_feedback_sample( + governor, &snapshot, &envelope, "matcher.gpu.pressure", 1, + 1000000000ULL, 1, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN)); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "matcher.gpu.pressure", 1, &envelope, + &selection, &reservation)); + CHECK(reservation && selection.decision.batch_size == 2 + && strcmp(selection.reason, "gpu-batch-throughput-trial") == 0 + && lardon3d_resource_governor_release(governor, reservation)); + lardon3d_resource_governor_destroy(governor); + return true; +} + +static bool +run_cpu_feedback_progression_test(void) +{ + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 16, + .page_size_bytes = 4096, + .memory_total_bytes = GIBIBYTES(16), + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy; + CHECK(lardon3d_resource_policy_default(&profile, &policy)); + policy.maximum_cpu_load_ratio = 1.0; + Lardon3DResourceGovernor *governor = + lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor && use_fixed_test_clock(governor)); + Lardon3DTaskCapabilityEnvelope envelope = { + .count = 1, + .capabilities = {{ + .estimate = { + .memory_bytes_per_item = MEBIBYTES(1), + .minimum_batch_size = 1, + .maximum_batch_size = 4, + .desired_cpu_threads = 12, + .task_class = LARDON3D_RESOURCE_TASK_CPU, + }, + .backend = LARDON3D_RESOURCE_BACKEND_CPU, + .inflight_limit = 1, + .cpu_reducible = true, + .batch_adaptive = true, + }}, + }; + Lardon3DResourceSnapshot snapshot = { + .memory_available_bytes = GIBIBYTES(12), + .swap_activity_known = true, + }; + Lardon3DResourceCapabilitySelection selection; + Lardon3DResourceReservation *reservation = NULL; + const unsigned int expected_cpu[] = {1, 2, 4, 8, 12}; + for (size_t step = 0; + step < sizeof(expected_cpu) / sizeof(expected_cpu[0]); ++step) { + for (unsigned int observation = 0; observation < 2; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.ramp", 1, &envelope, + &selection, &reservation)); + CHECK(reservation + && selection.decision.cpu_threads == expected_cpu[step] + && selection.decision.batch_size == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "test.cpu.ramp", 1, &selection, 1000000000ULL, + expected_cpu[step])); + } + } + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.ramp", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 12 + && selection.decision.batch_size == 2 + && strcmp(selection.reason, "throughput-trial") == 0); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + Lardon3DResourceSequenceDiagnostic ramp_diagnostic; + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "test.cpu.ramp", 1, &ramp_diagnostic) + && ramp_diagnostic.previous_wall_time_ns == 1000000000ULL + && ramp_diagnostic.items_completed == 12 + && ramp_diagnostic.durable_items_per_second_milli == 12000); + + /* The first active swap-I/O delta is detected after the CPU12/batch2 + * operational trial was copied. This same admission must replace both + * dimensions with CPU1/batch1 before it creates the reservation. */ + snapshot.swap_pages_out = 1; + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.ramp", 1, &envelope, &selection, + &reservation)); + Lardon3DResourceReservationInfo pressure_reservation; + CHECK(reservation); + CHECK(selection.pressure == LARDON3D_RESOURCE_PRESSURE_YELLOW); + CHECK(selection.decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH); + CHECK(selection.decision.cpu_threads == 1); + CHECK(selection.decision.batch_size == 1); + CHECK(strcmp(selection.reason, "pressure-decrease") == 0); + CHECK(lardon3d_resource_reservation_get_active( + governor, reservation, &pressure_reservation)); + CHECK(pressure_reservation.cpu_threads == 1); + CHECK(pressure_reservation.batch_size == 1); + CHECK(pressure_reservation.memory_bytes == MEBIBYTES(1)); + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "test.cpu.ramp", 1, &ramp_diagnostic) + && ramp_diagnostic.cpu_threads == 1 + && ramp_diagnostic.batch_size == 1 + && ramp_diagnostic.pressure == LARDON3D_RESOURCE_PRESSURE_YELLOW + && strcmp(ramp_diagnostic.reason, "pressure-decrease") == 0 + && lardon3d_resource_governor_release(governor, reservation)); + + /* Recovery cannot reuse the old CPU12 baseline. Three healthy admissions + * clear YELLOW, then two fresh CPU1 samples reopen only a CPU2 trial. */ + for (unsigned int observation = 0; observation < 3; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.ramp", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 1 + && selection.decision.batch_size == 1 + && lardon3d_resource_governor_release(governor, reservation)); + } + CHECK(lardon3d_resource_governor_pressure(governor) + == LARDON3D_RESOURCE_PRESSURE_GREEN); + for (unsigned int observation = 0; observation < 2; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.ramp", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 1 + && selection.decision.batch_size == 1 + && lardon3d_resource_governor_release(governor, reservation) + && lardon3d_resource_governor_internal_record_sequence( + governor, "test.cpu.ramp", 1, &selection, 1000000000ULL, 1)); + } + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.ramp", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 2 + && selection.decision.batch_size == 1 + && strcmp(selection.reason, "cpu-throughput-trial") == 0 + && lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "test.cpu.ramp", 1, &selection, 1000000000ULL, 2)); + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.ramp", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 2 + && lardon3d_resource_governor_release(governor, reservation) + && lardon3d_resource_governor_internal_record_sequence( + governor, "test.cpu.ramp", 1, &selection, 1000000000ULL, 2)); + + /* Gate G's PSI threshold still yields WAIT rather than manufacturing a + * current reservation; the adaptive history is nevertheless abandoned. */ + snapshot.memory_pressure_known = true; + snapshot.memory_pressure_avg10 = 100.0; + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.ramp", 1, &envelope, &selection, + &reservation)); + CHECK(!reservation && selection.decision.kind == LARDON3D_RESOURCE_WAIT + && selection.pressure == LARDON3D_RESOURCE_PRESSURE_YELLOW); + snapshot.memory_pressure_known = false; + for (unsigned int observation = 0; observation < 3; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.ramp", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 1 + && selection.decision.batch_size == 1 + && lardon3d_resource_governor_release(governor, reservation)); + } + CHECK(lardon3d_resource_governor_pressure(governor) + == LARDON3D_RESOURCE_PRESSURE_GREEN); + + /* Equal durable rate at CPU2 rejects that dimension after two samples, + * retains accepted CPU1, and only then opens the independent batch trial. */ + for (unsigned int observation = 0; observation < 2; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.no_gain", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 1 + && selection.decision.batch_size == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "test.cpu.no_gain", 1, &selection, 1000000000ULL, 1)); + } + for (unsigned int observation = 0; observation < 2; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.no_gain", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 2 + && selection.decision.batch_size == 1); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "test.cpu.no_gain", 1, &selection, 2000000000ULL, 2)); + } + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.no_gain", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 1 + && selection.decision.batch_size == 2 + && strcmp(selection.reason, "throughput-trial") == 0); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + + Lardon3DResourceSequenceDiagnostic diagnostic; + CHECK(lardon3d_resource_governor_internal_last_diagnostic( + governor, "test.cpu.no_gain", 1, &diagnostic) + && diagnostic.cpu_threads == 1 + && diagnostic.batch_size == 2); + lardon3d_resource_governor_destroy(governor); + + profile.logical_cpu_count = 12; + policy.system_cpu_reserve = 4; + governor = lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor && use_fixed_test_clock(governor)); + envelope.capabilities[0].estimate.maximum_batch_size = 1; + envelope.capabilities[0].batch_adaptive = false; + const unsigned int capped_cpu[] = {1, 2, 4, 8}; + for (size_t step = 0; + step < sizeof(capped_cpu) / sizeof(capped_cpu[0]); ++step) { + for (unsigned int observation = 0; observation < 2; ++observation) { + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.cap", 1, &envelope, &selection, + &reservation)); + CHECK(reservation + && selection.decision.cpu_threads == capped_cpu[step]); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(lardon3d_resource_governor_internal_record_sequence( + governor, "test.cpu.cap", 1, &selection, 1000000000ULL, + capped_cpu[step])); + } + } + reservation = NULL; + CHECK(lardon3d_resource_governor_internal_reserve_capability( + governor, &snapshot, "test.cpu.cap", 1, &envelope, &selection, + &reservation)); + CHECK(reservation && selection.decision.cpu_threads == 8); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + lardon3d_resource_governor_destroy(governor); + return true; +} + int main(void) { - return (run_test() && run_generation_test() && run_adaptive_batch_test() - && run_gate_g_boundary_test()) + return (run_driver_runtime_policy_test() + && run_test() && run_generation_test() && run_adaptive_batch_test() + && run_gate_g_boundary_test() && run_topology_value_reader_test() + && run_gpu_busy_identity_reader_test() + && run_cpu_topology_policy_test() + && run_capability_governor_test() + && run_inflight_feedback_test() + && run_private_telemetry_test() + && run_sustained_gpu_batch_feedback_test() + && run_cpu_feedback_progression_test()) ? EXIT_SUCCESS : EXIT_FAILURE; } diff --git a/tests/test_resource_snapshot.c b/tests/test_resource_snapshot.c index f52aae8..b7b2815 100644 --- a/tests/test_resource_snapshot.c +++ b/tests/test_resource_snapshot.c @@ -1,5 +1,6 @@ #include #include +#include #include #include #include @@ -37,33 +38,53 @@ write_text(const char *path, const char *text) } static bool -create_card(const char *root, unsigned int index, const char *total, const char *used) +card_path(char path[PATH_MAX], const char *root, unsigned int index, + const char *name) { - char card[256]; - char device[256]; - char path[256]; - if (snprintf(card, sizeof(card), "%s/card%u", root, index) < 0 - || snprintf(device, sizeof(device), "%s/device", card) < 0 + int written = snprintf(path, PATH_MAX, "%s/card%u/device/%s", root, + index, name); + return written > 0 && written < PATH_MAX; +} + +static bool +create_card(const char *root, unsigned int index, const char *total, + const char *used, const char *gtt) +{ + char card[PATH_MAX]; + char device[PATH_MAX]; + char path[PATH_MAX]; + int card_written = snprintf(card, sizeof(card), "%s/card%u", root, index); + int device_written = snprintf(device, sizeof(device), "%s/device", card); + if (card_written <= 0 || (size_t)card_written >= sizeof(card) + || device_written <= 0 || (size_t)device_written >= sizeof(device) || mkdir(card, 0700) != 0 || mkdir(device, 0700) != 0) { return false; } - (void)snprintf(path, sizeof(path), "%s/vendor", device); - if (!write_text(path, "0x1002\n")) { + if (!card_path(path, root, index, "vendor") + || !write_text(path, "0x1002\n")) { return false; } - (void)snprintf(path, sizeof(path), "%s/mem_info_vram_total", device); - if (!write_text(path, total)) { + if (!card_path(path, root, index, "mem_info_vram_total") + || !write_text(path, total)) { return false; } - (void)snprintf(path, sizeof(path), "%s/mem_info_vram_used", device); - return write_text(path, used); + if (!card_path(path, root, index, "mem_info_vram_used") + || !write_text(path, used)) { + return false; + } + if (!gtt) return true; + return card_path(path, root, index, "mem_info_gtt_total") + && write_text(path, gtt); } static void remove_card(const char *root, unsigned int index) { char path[256]; - const char *files[] = {"vendor", "mem_info_vram_total", "mem_info_vram_used"}; + const char *files[] = { + "vendor", "mem_info_vram_total", "mem_info_vram_used", + "mem_info_gtt_total", + }; for (size_t i = 0; i < sizeof(files) / sizeof(files[0]); ++i) { (void)snprintf(path, sizeof(path), "%s/card%u/device/%s", root, index, files[i]); (void)unlink(path); @@ -79,13 +100,16 @@ test_selected_gpu_pairing(void) { char root[] = "/tmp/lardon3d-drm-XXXXXX"; CHECK(mkdtemp(root)); - CHECK(create_card(root, 0, "1000\n", "100\n")); - CHECK(create_card(root, 1, "4000\n", "3000\n")); - Lardon3DHardwareProfile profile = {0}; + CHECK(create_card(root, 0, "1000\n", "100\n", "8000\n")); + CHECK(create_card(root, 1, "4000\n", "3000\n", NULL)); + Lardon3DHardwareProfile profile = { + .memory_total_bytes = 16ULL * 1024 * 1024 * 1024, + }; lardon3d_hardware_profile_detect_gpu_at_root(&profile, root); CHECK(profile.gpu_available); CHECK(profile.gpu_drm_card_index == 0); CHECK(profile.gpu_memory_total_bytes == 1000); + CHECK(profile.gpu_uses_shared_memory); Lardon3DResourceSnapshot snapshot = {.memory_available_bytes = 800}; lardon3d_resource_snapshot_capture_gpu_at_root(&profile, &snapshot, root); CHECK(snapshot.gpu_memory_available_known); @@ -98,7 +122,7 @@ test_selected_gpu_pairing(void) "%s/card0/device/mem_info_vram_used", root ); - CHECK(unlink(selected_usage) == 0); + CHECK(write_text(selected_usage, "100junk\n")); snapshot = (Lardon3DResourceSnapshot) {0}; lardon3d_resource_snapshot_capture_gpu_at_root(&profile, &snapshot, root); CHECK(!snapshot.gpu_memory_available_known); diff --git a/tests/test_sift_vulkan_feasibility.cpp b/tests/test_sift_vulkan_feasibility.cpp index 5b43ed5..6315c74 100644 --- a/tests/test_sift_vulkan_feasibility.cpp +++ b/tests/test_sift_vulkan_feasibility.cpp @@ -20,6 +20,8 @@ extern "C" { #include #include +#include "vulkan_process_startup.h" + struct Comparison { uint64_t queries = 0; uint64_t index_divergences = 0; @@ -41,6 +43,25 @@ static bool fp64_requested() { return value && std::strcmp(value, "1") == 0; } +static bool driver_policy_gate_rejects_unsafe() { + if (setenv("MESA_SHADER_CACHE_DISABLE", "false", 1) != 0) return false; + Lardon3DOrbVulkanBackend *backend = lardon3d_orb_vulkan_backend_create(); + if (!backend) return false; + float descriptor[128]{}; + Lardon3DSiftTop2 output{7, 11, 13.0F, 17, 19.0F}; + const Lardon3DSiftTop2 unchanged = output; + Lardon3DOrbVulkanResult result = lardon3d_sift_vulkan_top2( + backend, descriptor, 1, descriptor, 1, &output, 1); + Lardon3DOrbVulkanInfo info{}; + bool ok = result == LARDON3D_ORB_VULKAN_UNAVAILABLE && + std::memcmp(&output, &unchanged, sizeof(output)) == 0 && + lardon3d_orb_vulkan_backend_info(backend, &info) && + info.initialized && !info.available && + std::strcmp(std::getenv("MESA_SHADER_CACHE_DISABLE"), "false") == 0; + lardon3d_orb_vulkan_backend_destroy(backend); + return setenv("MESA_SHADER_CACHE_DISABLE", "true", 1) == 0 && ok; +} + static std::vector make_descriptors(uint32_t count, bool rootsift, uint32_t seed) { std::vector descriptors(static_cast(count) * 128); @@ -339,6 +360,15 @@ static bool run_distribution(bool rootsift) { } int main() { + if (!lardon3d_vulkan_evidence_process_startup()) { + std::fprintf(stderr, + "MESA_SHADER_CACHE_DISABLE must be true for safe CPU affinity\n"); + return EXIT_FAILURE; + } + if (!driver_policy_gate_rejects_unsafe()) { + std::fprintf(stderr, "SIFT Vulkan driver process-policy gate failed\n"); + return EXIT_FAILURE; + } cv::setNumThreads(12); return run_distribution(false) && run_distribution(true) ? EXIT_SUCCESS diff --git a/tests/test_task.c b/tests/test_task.c index 0990b94..182b1a5 100644 --- a/tests/test_task.c +++ b/tests/test_task.c @@ -8,6 +8,7 @@ #include #include "resource_snapshot_test_utils.h" +#include "../src/task_internal.h" #define CHECK(condition) \ do { \ @@ -79,6 +80,49 @@ failure_callback(Lardon3DTask *task, void *userdata) return lardon3d_task_fail(task, "Erreur contrôlée.") && false; } +typedef struct { + Lardon3DResourceGovernor *governor; + bool sequence_rejected; + bool active_reservation_released; +} AssociationMismatchProbe; + +typedef struct { + size_t inflight_limit; + uint64_t gpu_memory_bytes; +} DirectAdaptiveProbe; + +static bool +direct_adaptive_callback(Lardon3DTask *task, void *userdata) +{ + DirectAdaptiveProbe *probe = userdata; + Lardon3DResourceCapabilitySelection selection; + Lardon3DTaskExecutionContract contract; + if (!probe + || !lardon3d_task_internal_execution_selection(task, &selection) + || !lardon3d_task_execution_contract(task, &contract)) { + return false; + } + probe->inflight_limit = selection.inflight_limit; + probe->gpu_memory_bytes = contract.gpu_memory_bytes; + return lardon3d_task_set_progress(task, 100, "Contrat direct observé."); +} + +static bool +association_mismatch_callback(Lardon3DTask *task, void *userdata) +{ + AssociationMismatchProbe *probe = userdata; + Lardon3DResourceReservation *reservation = NULL; + Lardon3DTaskExecutionContract contract; + if (!lardon3d_task_internal_test_force_sequence_association_mismatch(task)) { + return false; + } + probe->sequence_rejected = !lardon3d_task_sequence_break( + task, probe->governor, &reservation, &contract); + probe->active_reservation_released = + lardon3d_resource_governor_reservation_count(probe->governor) == 0; + return false; +} + static void * start_task(void *context) { @@ -118,12 +162,15 @@ run_test(void) .logical_cpu_count = 4, .page_size_bytes = 4096, .memory_total_bytes = UINT64_MAX, + .gpu_available = true, + .gpu_uses_shared_memory = true, .cpu_architecture = "test", }; Lardon3DResourcePolicy policy = { .maximum_cpu_load_ratio = 1.0, .maximum_io_pressure_avg10 = 100.0, .io_slot_capacity = 1, + .gpu_slot_capacity = 1, }; Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create( &profile, @@ -240,6 +287,63 @@ run_test(void) && failed_probe.reservation_released); lardon3d_task_destroy(task); + AssociationMismatchProbe mismatch = {.governor = governor}; + task = lardon3d_task_create( + "Association invalide", &estimate, association_mismatch_callback, + &mismatch); + CHECK(task); + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resource_snapshot)); + CHECK(lardon3d_resource_governor_reserve( + governor, &resource_snapshot, &estimate, &decision, &reservation)); + CHECK(lardon3d_task_start(task, governor, reservation)); + CHECK(mismatch.sequence_rejected && mismatch.active_reservation_released); + CHECK(!lardon3d_task_internal_test_has_reservation_ownership(task)); + CHECK(lardon3d_task_internal_test_association_failure_releases(task) == 1); + CHECK(lardon3d_resource_governor_reservation_count(governor) == 0); + CHECK(lardon3d_task_snapshot(task, &snapshot)); + CHECK(snapshot.state == TASK_FAILED); + lardon3d_task_destroy(task); + + /* A direct public reserve/start call has no private capability decision. + * It must therefore install the durable depth-one minimum, never the + * adaptive maximum that only Queue-owned admission may reserve. */ + const Lardon3DResourceEstimate adaptive_estimate = { + .gpu_memory_fixed_bytes = 640 * 1024, + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 1, + .desired_gpu_slots = 1, + .task_class = LARDON3D_RESOURCE_TASK_MIXED, + }; + Lardon3DTaskCapabilityEnvelope adaptive_envelope = { + .count = 1, + .capabilities = {{ + .estimate = adaptive_estimate, + .backend = LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, + .inflight_limit = 2, + .minimum_inflight_limit = 1, + .gpu_memory_bytes_per_inflight = 640 * 1024, + .inflight_adaptive = true, + }}, + }; + adaptive_envelope.capabilities[0].estimate.gpu_memory_fixed_bytes = 0; + DirectAdaptiveProbe direct_probe = {0}; + task = lardon3d_task_create_typed( + "Contrat adaptatif direct", &adaptive_estimate, + "test.direct.adaptive", 1, direct_adaptive_callback, &direct_probe, + NULL); + CHECK(task + && lardon3d_task_internal_set_capability_envelope( + task, &adaptive_envelope)); + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resource_snapshot)); + CHECK(lardon3d_resource_governor_reserve( + governor, &resource_snapshot, &adaptive_estimate, &decision, + &reservation)); + CHECK(reservation && lardon3d_task_start(task, governor, reservation)); + CHECK(direct_probe.inflight_limit == 1 + && direct_probe.gpu_memory_bytes == 640 * 1024); + lardon3d_task_destroy(task); + task = lardon3d_task_create( "Pause avant départ", &estimate, diff --git a/tests/test_task_kind_registry.c b/tests/test_task_kind_registry.c index cefd033..224d1f5 100644 --- a/tests/test_task_kind_registry.c +++ b/tests/test_task_kind_registry.c @@ -115,6 +115,88 @@ snapshot(void) return result; } +static bool +run_legacy_estimate_validation_test(void) +{ + static const Lardon3DTaskKindDescriptor descriptors[] = { + {.kind = "candidate_pair.generate", .kind_version = 1, + .reconstruct = reconstruct}, + {.kind = "features.extract.sift", .kind_version = 1, + .reconstruct = reconstruct}, + {.kind = "features.extract.rootsift", .kind_version = 1, + .reconstruct = reconstruct}, + }; + Lardon3DTaskKindRegistry registry; + CHECK(lardon3d_task_kind_registry_init(®istry, descriptors, 3)); + int destroyed = 0; + int finished = 0; + ReconstructContext context = { + .destroyed = &destroyed, + .finished = &finished, + }; + const char *kinds[] = { + "candidate_pair.generate", + "features.extract.sift", + "features.extract.rootsift", + }; + for (size_t index = 0; index < 3; ++index) { + Lardon3DTaskDurableSnapshot corrupt = snapshot(); + corrupt.estimate = (Lardon3DResourceEstimate) {0}; + Lardon3DTask *task = NULL; + /* An absent legacy form must never make the all-zero estimate look + * like an exact historical signature. Reconstruction owns and frees + * the binding userdata on this corruption rejection. */ + CHECK(lardon3d_task_kind_registry_restore( + ®istry, kinds[index], 1, &corrupt, &context, &task + ) == LARDON3D_TASK_KIND_RECONSTRUCTION_FAILED); + CHECK(!task && destroyed == (int)index + 1); + } + + const Lardon3DResourceEstimate historical[] = { + { + .memory_fixed_bytes = 128 * 1024, + .memory_bytes_per_item = 64 * 1024, + .minimum_batch_size = 1, + .maximum_batch_size = 64, + .desired_cpu_threads = 1, + .desired_io_slots = 1, + .task_class = LARDON3D_RESOURCE_TASK_CPU, + }, + { + .memory_fixed_bytes = 64ULL * 1024 * 1024, + .memory_bytes_per_item = 1024ULL * 1024 * 1024, + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 1, + .desired_io_slots = 1, + .task_class = LARDON3D_RESOURCE_TASK_CPU, + }, + { + .memory_fixed_bytes = 64ULL * 1024 * 1024, + .memory_bytes_per_item = 1024ULL * 1024 * 1024, + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 1, + .desired_io_slots = 1, + .task_class = LARDON3D_RESOURCE_TASK_CPU, + }, + }; + for (size_t index = 0; index < 3; ++index) { + Lardon3DTaskDurableSnapshot durable = snapshot(); + durable.estimate = historical[index]; + Lardon3DTask *task = NULL; + CHECK(lardon3d_task_kind_registry_restore( + ®istry, kinds[index], 1, &durable, &context, &task + ) == LARDON3D_TASK_KIND_OK); + Lardon3DResourceEstimate effective; + CHECK(task && lardon3d_task_resource_estimate(task, &effective)); + CHECK(effective.desired_cpu_threads == 12); + lardon3d_task_destroy(task); + } + CHECK(destroyed == 6); + return true; +} + static bool run_test(void) { @@ -209,5 +291,6 @@ run_test(void) int main(void) { - return run_test() ? EXIT_SUCCESS : EXIT_FAILURE; + return (run_test() && run_legacy_estimate_validation_test()) + ? EXIT_SUCCESS : EXIT_FAILURE; } diff --git a/tests/test_task_queue.c b/tests/test_task_queue.c index 89c7411..b24108b 100644 --- a/tests/test_task_queue.c +++ b/tests/test_task_queue.c @@ -1,12 +1,20 @@ +#ifndef _GNU_SOURCE +#define _GNU_SOURCE +#endif + +#include #include +#include #include #include #include +#include #include #include #include "../src/resource_governor_internal.h" +#include "../src/task_internal.h" #include "resource_snapshot_test_utils.h" #define CHECK(condition) \ @@ -484,11 +492,26 @@ run_test(void) &reduced ); uint64_t reduced_id; - CHECK(reduced_task); + Lardon3DTaskCapabilityEnvelope reduced_envelope = { + .count = 1, + .capabilities = {{ + .estimate = reduced_estimate, + .backend = LARDON3D_RESOURCE_BACKEND_CPU, + .inflight_limit = 1, + .cpu_reducible = true, + }}, + }; + /* This synthetic callback explicitly consumes the contract; unlike a + * production kind, it has no registry entry from which to reconstruct the + * otherwise private adaptive envelope. */ + CHECK(reduced_task && lardon3d_task_internal_set_capability_envelope( + reduced_task, &reduced_envelope)); CHECK(lardon3d_task_queue_add(queue, reduced_task, &reduced_id)); CHECK(wait_terminal(queue, reduced_id, &snapshot)); CHECK(snapshot.state == TASK_COMPLETED); - CHECK(reduced.contract.cpu_threads == 1024); + /* CPU-reducible capabilities slow-start at one participant. A later + * sequence may grow only after two baseline and two improving samples. */ + CHECK(reduced.contract.cpu_threads == 1); Lardon3DResourceEstimate rejected_estimate = { .memory_fixed_bytes = UINT64_MAX, .memory_bytes_per_item = 1, @@ -926,10 +949,363 @@ test_stress_concurrent(Lardon3DResourceGovernor *governor) return true; } +typedef struct { + Lardon3DResourceGovernor *governor; + Lardon3DTaskExecutionContract first; + Lardon3DTaskExecutionContract unchanged; + Lardon3DTaskExecutionContract second; + Lardon3DResourceCapabilitySelection first_selection; + Lardon3DResourceCapabilitySelection unchanged_selection; +} ImmutableSequenceWork; + +#ifdef __linux__ +typedef struct { + cpu_set_t worker_mask; + cpu_set_t child_mask; + bool child_started; +} AffinityWork; + +static void * +capture_child_affinity(void *userdata) +{ + AffinityWork *work = userdata; + work->child_started = sched_getaffinity( + 0, sizeof(work->child_mask), &work->child_mask) == 0; + return NULL; +} + +static bool +affinity_callback(Lardon3DTask *task, void *userdata) +{ + AffinityWork *work = userdata; + pthread_t child; + if (sched_getaffinity(0, sizeof(work->worker_mask), &work->worker_mask) != 0 + || pthread_create(&child, NULL, capture_child_affinity, work) != 0 + || pthread_join(child, NULL) != 0 || !work->child_started) { + return false; + } + return lardon3d_task_set_progress(task, 100, "Affinité observée."); +} + +static bool +test_worker_only_affinity(void) +{ + cpu_set_t main_before; + CPU_ZERO(&main_before); + if (sched_getaffinity(0, sizeof(main_before), &main_before) != 0) { + return true; + } + Lardon3DResourceCpuTopologyInput topology = { + .affinity_available = true, + .topology_available = true, + }; + for (unsigned int cpu = 0; cpu < CPU_SETSIZE + && cpu < LARDON3D_RESOURCE_CPU_MAX; ++cpu) { + if (!CPU_ISSET((size_t)cpu, &main_before)) { + continue; + } + size_t index = topology.allowed_cpu_count++; + topology.allowed_cpu_ids[index] = cpu; + topology.topology_entries[index] = + (Lardon3DResourceCpuTopologyEntry) { + .cpu_id = cpu, + .package_id = 0, + .core_id = (unsigned int)index, + }; + } + topology.topology_entry_count = topology.allowed_cpu_count; + if (topology.allowed_cpu_count < 2 + || topology.allowed_cpu_count > UINT_MAX) { + return true; + } + Lardon3DHardwareProfile profile = { + .logical_cpu_count = (unsigned int)topology.allowed_cpu_count, + .page_size_bytes = 4096, + .memory_total_bytes = 16ULL * 1024 * 1024 * 1024, + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy = { + .system_cpu_reserve = 1, + .maximum_cpu_load_ratio = 1.0, + .maximum_io_pressure_avg10 = 100.0, + .io_slot_capacity = 1, + }; + Lardon3DResourceEstimate estimate = { + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 1, + .task_class = LARDON3D_RESOURCE_TASK_CPU, + }; + + Lardon3DResourceGovernor *governor = + lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor + && lardon3d_resource_governor_internal_configure_cpu_topology( + governor, &topology)); + Lardon3DTaskQueue *queue = lardon3d_task_queue_create(governor, 1); + CHECK(queue); + Lardon3DResourceCpuPolicyDiagnostic diagnostic; + CHECK(lardon3d_resource_governor_internal_cpu_policy( + governor, &diagnostic)); + CHECK(diagnostic.affinity_attempted && diagnostic.affinity_active + && diagnostic.compute_cpu_count == topology.allowed_cpu_count - 1); + cpu_set_t expected; + CPU_ZERO(&expected); + for (unsigned int cpu = 0; cpu < CPU_SETSIZE + && cpu < LARDON3D_RESOURCE_CPU_MAX; ++cpu) { + if ((diagnostic.compute_mask[cpu / 64] + & (UINT64_C(1) << (cpu % 64))) != 0) { + CPU_SET((size_t)cpu, &expected); + } + } + AffinityWork work = {0}; + Lardon3DTask *task = lardon3d_task_create_typed( + "Affinité worker", &estimate, "test.affinity", 1, + affinity_callback, &work, NULL); + uint64_t id = 0; + CHECK(task && lardon3d_task_queue_add(queue, task, &id)); + Lardon3DTaskSnapshot snapshot; + CHECK(wait_terminal(queue, id, &snapshot) + && snapshot.state == TASK_COMPLETED + && CPU_EQUAL(&work.worker_mask, &expected) + && CPU_EQUAL(&work.child_mask, &expected)); + cpu_set_t main_after; + CPU_ZERO(&main_after); + CHECK(sched_getaffinity(0, sizeof(main_after), &main_after) == 0 + && CPU_EQUAL(&main_before, &main_after)); + lardon3d_task_queue_destroy(queue); + lardon3d_resource_governor_destroy(governor); + + /* An injected application failure must restore/retain the inherited mask, + * remain observable, and still permit Queue-owned Task cleanup. */ + governor = lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor + && lardon3d_resource_governor_internal_configure_cpu_topology( + governor, &topology)); + lardon3d_resource_governor_internal_force_worker_affinity_failure( + governor, true); + queue = lardon3d_task_queue_create(governor, 1); + CHECK(queue && lardon3d_resource_governor_internal_cpu_policy( + governor, &diagnostic)); + CHECK(diagnostic.affinity_attempted && !diagnostic.affinity_active + && strcmp(diagnostic.reason, "worker-affinity-apply-failed") == 0); + work = (AffinityWork) {0}; + task = lardon3d_task_create_typed( + "Échec affinité worker", &estimate, "test.affinity.failure", 1, + affinity_callback, &work, NULL); + id = 0; + CHECK(task && lardon3d_task_queue_add(queue, task, &id) + && wait_terminal(queue, id, &snapshot) + && snapshot.state == TASK_COMPLETED + && CPU_EQUAL(&work.worker_mask, &main_before) + && CPU_EQUAL(&work.child_mask, &main_before)); + lardon3d_task_queue_destroy(queue); + lardon3d_resource_governor_destroy(governor); + return true; +} +#else +static bool test_worker_only_affinity(void) { return true; } +#endif + +static bool +immutable_sequence_callback(Lardon3DTask *task, void *userdata) +{ + ImmutableSequenceWork *work = userdata; + if (!lardon3d_task_execution_contract(task, &work->first) + || !lardon3d_task_internal_execution_selection( + task, &work->first_selection) + || work->first.gpu_slots != 1 + || work->first_selection.inflight_limit != 1 + || !lardon3d_resource_governor_internal_set_backend_available( + work->governor, + LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, + false + ) + || !lardon3d_task_execution_contract(task, &work->unchanged) + || !lardon3d_task_internal_execution_selection( + task, &work->unchanged_selection) + || work->first.batch_size != work->unchanged.batch_size + || work->first.memory_bytes != work->unchanged.memory_bytes + || work->first.gpu_memory_bytes != work->unchanged.gpu_memory_bytes + || work->first.cpu_threads != work->unchanged.cpu_threads + || work->first.gpu_slots != work->unchanged.gpu_slots + || work->first.io_slots != work->unchanged.io_slots + || work->first_selection.inflight_limit + != work->unchanged_selection.inflight_limit) { + return false; + } + Lardon3DResourceReservation *reservation = NULL; + return lardon3d_task_sequence_break( + task, + work->governor, + &reservation, + &work->second + ) + && work->second.gpu_slots == 0 && work->second.cpu_threads == 1; +} + +static bool +test_sequence_contract_immutability(void) +{ + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 8, + .page_size_bytes = 4096, + .memory_total_bytes = 16ULL * 1024 * 1024 * 1024, + .gpu_available = true, + .gpu_uses_shared_memory = true, + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy = { + .maximum_cpu_load_ratio = 1.0, + .maximum_io_pressure_avg10 = 100.0, + .gpu_slot_capacity = 1, + .io_slot_capacity = 1, + }; + Lardon3DResourceGovernor *governor = + lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor); + CHECK(lardon3d_resource_governor_internal_set_backend_available( + governor, LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, true)); + Lardon3DTaskQueue *queue = lardon3d_task_queue_create(governor, 1); + CHECK(queue); + Lardon3DResourceEstimate cpu = { + .memory_bytes_per_item = 1024 * 1024, + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 4, + .task_class = LARDON3D_RESOURCE_TASK_CPU, + }; + Lardon3DResourceEstimate gpu = cpu; + gpu.gpu_memory_fixed_bytes = 0; + gpu.desired_cpu_threads = 1; + gpu.desired_gpu_slots = 1; + Lardon3DTaskCapabilityEnvelope envelope = { + .count = 2, + .capabilities = { + { + .estimate = gpu, + .backend = LARDON3D_RESOURCE_BACKEND_ORB_VULKAN, + .inflight_limit = 2, + .minimum_inflight_limit = 1, + .gpu_memory_bytes_per_inflight = 640 * 1024, + .preferred = true, + .inflight_adaptive = true, + .requires_runtime_backend = true, + }, + { + .estimate = cpu, + .backend = LARDON3D_RESOURCE_BACKEND_CPU, + .inflight_limit = 1, + .cpu_reducible = true, + }, + }, + }; + ImmutableSequenceWork work = {.governor = governor}; + Lardon3DTask *task = lardon3d_task_create_typed( + "Contrat immuable", + &cpu, + "test.sequence", + 1, + immutable_sequence_callback, + &work, + NULL + ); + CHECK(task && lardon3d_task_internal_set_capability_envelope(task, &envelope)); + uint64_t id = 0; + CHECK(lardon3d_task_queue_add(queue, task, &id)); + Lardon3DTaskSnapshot snapshot; + CHECK(wait_terminal(queue, id, &snapshot)); + CHECK(snapshot.state == TASK_COMPLETED && work.first.gpu_slots == 1 + && work.second.gpu_slots == 0); + lardon3d_task_queue_destroy(queue); + lardon3d_resource_governor_destroy(governor); + return true; +} + +typedef struct { + bool called; + unsigned int cpu_threads; +} CpuEnvelopeWork; + +static bool +cpu_envelope_callback(Lardon3DTask *task, void *userdata) +{ + CpuEnvelopeWork *work = userdata; + Lardon3DTaskExecutionContract contract; + if (!work || !lardon3d_task_execution_contract(task, &contract)) { + return false; + } + work->called = true; + work->cpu_threads = contract.cpu_threads; + return true; +} + +static bool +test_fixed_default_and_validated_cpu_range(void) +{ + /* An unknown kind must not acquire CPU adaptation merely because its + * durable estimate asks for several threads. Only registered callbacks + * whose output was validated across counts may consume a reduced count. */ + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 1024, + .page_size_bytes = 4096, + .memory_total_bytes = 16ULL * 1024 * 1024 * 1024, + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy = { + .system_cpu_reserve = 1022, + .maximum_cpu_load_ratio = 1.0, + .maximum_io_pressure_avg10 = 100.0, + .io_slot_capacity = 1, + }; + Lardon3DResourceGovernor *governor = + lardon3d_resource_governor_create(&profile, &policy); + CHECK(governor); + Lardon3DResourceEstimate estimate = { + .memory_bytes_per_item = 1024, + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 4, + .desired_io_slots = 1, + .task_class = LARDON3D_RESOURCE_TASK_CPU, + }; + CpuEnvelopeWork fixed_work = {0}; + Lardon3DTask *fixed = lardon3d_task_create_typed( + "Enveloppe fixe", &estimate, "test.fixed", 1, + cpu_envelope_callback, &fixed_work, NULL); + CHECK(fixed); + Lardon3DResourceDecision decision; + Lardon3DResourceReservation *reservation = NULL; + CHECK(lardon3d_task_internal_reserve_available( + fixed, governor, &decision, &reservation)); + CHECK(decision.kind == LARDON3D_RESOURCE_WAIT && !reservation + && !fixed_work.called); + lardon3d_task_destroy(fixed); + + CpuEnvelopeWork adaptive_work = {0}; + Lardon3DTask *adaptive = lardon3d_task_create_typed( + "Enveloppe validée", &estimate, "features.extract", 1, + cpu_envelope_callback, &adaptive_work, NULL); + CHECK(adaptive); + CHECK(lardon3d_task_internal_reserve_available( + adaptive, governor, &decision, &reservation)); + CHECK((decision.kind == LARDON3D_RESOURCE_START + || decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH) + && reservation); + CHECK(lardon3d_task_start(adaptive, governor, reservation)); + CHECK(adaptive_work.called && adaptive_work.cpu_threads == 1); + (void)lardon3d_resource_governor_release(governor, reservation); + lardon3d_task_destroy(adaptive); + lardon3d_resource_governor_destroy(governor); + return true; +} + int main(void) { - if (!run_test()) { + if (!run_test() || !test_worker_only_affinity() + || !test_sequence_contract_immutability() + || !test_fixed_default_and_validated_cpu_range()) { return EXIT_FAILURE; } Lardon3DHardwareProfile profile = { diff --git a/tests/vulkan_process_startup.h b/tests/vulkan_process_startup.h new file mode 100644 index 0000000..076303c --- /dev/null +++ b/tests/vulkan_process_startup.h @@ -0,0 +1,22 @@ +#ifndef LARDON3D_TESTS_VULKAN_PROCESS_STARTUP_H +#define LARDON3D_TESTS_VULKAN_PROCESS_STARTUP_H + +#include +#include + +/* Standalone Vulkan evidence tools own this process-start action. Call it as + * the first statement of main, before OpenCV or any other library may create a + * pthread. An absent value gets the safe default; an explicit value is never + * overwritten and must already be exact true/1. The production backend itself + * remains a non-mutating late boundary. */ +static inline bool lardon3d_vulkan_evidence_process_startup() { + const char *value = std::getenv("MESA_SHADER_CACHE_DISABLE"); + if (!value) { + if (setenv("MESA_SHADER_CACHE_DISABLE", "true", 0) != 0) return false; + value = std::getenv("MESA_SHADER_CACHE_DISABLE"); + } + return value && + (std::strcmp(value, "true") == 0 || std::strcmp(value, "1") == 0); +} + +#endif