diff --git a/README.md b/README.md index 6b369fe..6e27b0f 100644 --- a/README.md +++ b/README.md @@ -172,8 +172,8 @@ fallback CPU. La feasibility Vulkan SIFT/RootSIFT a été rejetée ; ces deux ma restent sur OpenCV L2. Track Model/Builder, les primitives géométriques Gate C, le noyau Sparse SfM incrémental Gate D et le Bundle Adjustment final Gate E sont implémentés et validés. L'orchestration Sparse SfM Gate F est PASS / FROZEN ; -l'intégration Governor Gate G, le DAG, le viewer et les étapes denses restent des tickets -séparés planifiés. Le Resource Governor ne +l'intégration Governor Gate G est **PASS / FROZEN** ; le DAG, +le viewer et les étapes denses restent des tickets séparés planifiés. Le Resource Governor ne constitue pas un Resource System générique : voir la décision d’architecture. ## Licence diff --git a/docs/architecture/feature_store.md b/docs/architecture/feature_store.md index 91df1ad..66f100a 100644 --- a/docs/architecture/feature_store.md +++ b/docs/architecture/feature_store.md @@ -122,7 +122,8 @@ les Feature Sets persistés. PASS**, the Sparse SfM v16 persistence model is **FROZEN** after Gate B; Gate C geometry and the synchronous in-memory Gate D incremental core are **IMPLEMENTED / PASS**. Final per-component Gate E BA is **PASS / FROZEN**; -Gate F project/task orchestration is **PASS / FROZEN**. Gate G remains planned. +Gate F project/task orchestration is **PASS / FROZEN**. Gate G is +**PASS / FROZEN**. ## Extension v2 multi-descriptor diff --git a/docs/architecture/overview.md b/docs/architecture/overview.md index 5c00018..96cea1f 100644 --- a/docs/architecture/overview.md +++ b/docs/architecture/overview.md @@ -140,7 +140,7 @@ ni persistance, ni Task Runtime, ni Resource Governor. Gate F relie ces noyaux au Task Runtime durable, à la Queue/Governor/Reservation et à la publication Project DB v17 atomique et idempotente. -**Statut :** GATE F — PASS / FROZEN. Gate G reste planifiée. +**Statut :** GATE F — PASS / FROZEN. Gate G est **PASS / FROZEN**. ## Résultats et publication live diff --git a/docs/architecture/resource_aware_pipeline.md b/docs/architecture/resource_aware_pipeline.md index 375b5c0..2773e49 100644 --- a/docs/architecture/resource_aware_pipeline.md +++ b/docs/architecture/resource_aware_pipeline.md @@ -26,6 +26,22 @@ observations saines sont nécessaires à chaque palier `1 → 2 → 4 → 8`. Un nouvelle pression réinitialise cette progression. Cette mémoire est process-local, bornée et protégée par le mutex du Governor. +Gate G gèle le rafraîchissement initial : lorsqu'il existe du travail PENDING +en `WAIT` de ressources, le worker unique de la Task Queue dort au plus 500 ms +avant de rescanner la file et de recapturer les ressources. Un signal explicite +le réveille plus tôt. Cette cadence ne remplace pas les 50 ms existantes d'une +tâche déjà active qui attend sa réadmission à une frontière de séquence. + +**Gate G — PASS / FROZEN.** Cette réévaluation bornée, la +fraîcheur des snapshots et l'identité GPU sélectionnée sont raccordées aux +chemins de production existants et leur validation finale est terminée. + +Les snapshots emploient `CLOCK_MONOTONIC` et leur âge maximal est 1000 ms. Une +capture complète impossible est une erreur opérationnelle, tandis qu'une PSI +ou télémétrie swap optionnelle absente reste inconnue. Le modèle cible un hôte +Linux natif non contraint ; cgroups, limites systemd/RLIMIT, multi-GPU, +monitoring RSS, scratch et stockage externe restent différés. + ## Feature Extraction ORB est déjà une tâche durable par image : source validée, extraction, @@ -106,6 +122,8 @@ une distribution de latence estimator-only. Le profil maximal explicite et les pools multi-workers restent hors périmètre. SIFT/RootSIFT et Feature Extraction Vulkan restent hors de ce contrat. +Swap, zram et disque externe ne sont jamais ajoutés au budget RAM. Aucun chemin +scratch/spill ni aucune action modifiant l'hôte n'appartient à Gate G core. La validation B3 du modèle Sparse SfM v16 a utilisé des processus frais, un fixture synthétique de 100 000 landmarks et 500 000 observations, cinq passes diff --git a/docs/architecture/resource_boundary.md b/docs/architecture/resource_boundary.md index 65c00ed..79e2ea0 100644 --- a/docs/architecture/resource_boundary.md +++ b/docs/architecture/resource_boundary.md @@ -4,8 +4,7 @@ **ACCEPTED** — architecture decision for the post-Gate C documentation freeze. -Current Sparse SfM gates A through F are **PASS / FROZEN**. Gate G remains -open/planned. +Current Sparse SfM gates A through G are **PASS / FROZEN**. Project Database: current schema **v17**; historical v16 remains frozen. This record is normative for the current architecture. It does not introduce @@ -115,8 +114,9 @@ by `lardon3d_task_create_typed()` and submit the task through the normal queue. It does not inspect resource snapshots, decide admission, create reservations, change Governor or queue policy, or bypass the reservation invariant. The estimate is operational metadata and is excluded from the Sparse SfM parameter -fingerprint, candidate identity and scientific determinism. Future Gate G owns -resource-management policy and operational refinement of that estimate. +fingerprint, candidate identity and scientific determinism. Gate G owns +resource-management and admission policy, but Gate G core does not modify the +fields of the frozen Sparse SfM Gate F v1 estimate. Gate F v1 freezes its declarative Sparse SfM RAM request as the checked sum `128 MiB + I*64 KiB + T*2048 + O*512`, rounded upward to one MiB, where `I`, @@ -126,6 +126,146 @@ fields are zero, batch bounds are one, and the task requests one CPU thread and one IO slot in the CPU class. These coefficients are conservative operational policy inputs, not measured dynamically or included in scientific identity. +### Gate G resource-policy implementation + +**PASS / FROZEN.** The G0a contract is implemented and validated. All seven G0 +human decisions remain resolved. + +#### Pending admission and snapshot freshness + +When pending work exists but every candidate receives `WAIT`, the existing Task +Queue worker performs a timed wait of at most 500 milliseconds. An earlier +enqueue, resume, resource-change, cancellation or shutdown signal wakes it +immediately. On timeout it repeats the normal stable queue scan and obtains new +snapshots through the existing admission path. No monitor thread, scheduler or +subsystem is added. The separate polling interval for a running sequential task +waiting at `lardon3d_task_sequence_break()` remains 50 milliseconds. + +Production snapshots use `CLOCK_MONOTONIC`. A directly supplied snapshot is +fresh through an age of exactly 1000 milliseconds. A snapshot older than 1000 +milliseconds or timestamped in the future must produce `WAIT`, must not produce +`START` or `REDUCE_BATCH`, creates no reservation and does not mutate Governor +policy state. Normal production admission captures synchronously through +`lardon3d_resource_governor_reserve_available()` immediately before evaluation. +There is no last-known-good cache, grace cache or telemetry-cache subsystem. + +A mandatory whole-snapshot capture failure is an operational/internal resource +error: the queued task becomes `FAILED`, no callback starts and no reservation +leaks. Unavailable optional CPU, memory or IO PSI and vmstat/swap telemetry is +unknown and asserts no artificial pressure. Actual GPU demand with no selected +GPU is `REJECT`; temporarily unknown required live VRAM for a selected dedicated +GPU is `WAIT`. VRAM availability is never fabricated. + +#### Conservative RAM accounting and platform + +Gate G core preserves the conservative live admission model: + +```text +available_ram = max( + 0, + min(MemAvailable, physical_ram) + - host_ram_reserve + - active_charged_reservations +) +``` + +The implementation retains its existing checked, saturating subtraction order. +The host reserve remains one quarter of physical RAM and the emergency floor +remains one eighth of physical RAM. +Possible overlap between `MemAvailable` and already-materialized memory from an +active reservation is accepted. The deliberate bias is false `WAIT`, not +overcommit. Gate G adds no RSS tracking, materialization state, allocation +measurement, reservation resizing or live per-task monitoring. The current +single-worker topology remains unchanged. + +Gate G core supports a native unconstrained Linux host process. Capacity uses +the existing `_SC_PHYS_PAGES`, `_SC_PAGESIZE` and `/proc/meminfo` mechanisms. +It is not cgroup v2, systemd `MemoryMax`, `RLIMIT_AS` or `RLIMIT_DATA` aware and +does not claim correct host-capacity admission inside a tighter constrained +container or service. Effective constrained-runtime accounting is deferred. No +cgroup write, systemd dependency or new dependency is authorized. + +#### Sparse SfM estimate authority + +Gate G consumes and restores the exact Sparse SfM Gate F estimate above and +does not alter any producer field. Batch adaptation is permitted only for task +contracts whose existing minimum/maximum range allows it; Sparse SfM remains +fixed at batch one. Restart retains the estimate persisted when the task was +created and evaluates it against newly captured machine telemetry. + +Any future coefficient change requires a separate explicit review and an +operational formula/version contract applying only to newly created tasks. It +must not alter F0, candidate identity, Gate D/E parameters or existing persisted +tasks. Estimate-formula refinement is not part of Gate G core. + +#### Selected GPU + +Gate G core supports one selected GPU and no multi-GPU scheduling. Hardware +Profile deterministically selects the lowest numeric `/sys/class/drm/cardN` +accepted by its selection rules and retains that identity internally. Snapshot +capacity and live usage must refer to that same device; Resource Snapshot must +not perform an independent first-usable-GPU selection. Memory is never summed +across devices and reservations are not made per device. + +Dedicated memory uses the selected device's capacity and usage. UMA/shared GPU +demand is charged exactly once against system RAM and never against a second +fictitious VRAM pool. Backend and fallback selection remain task-producer/task +contract responsibilities, not Governor policy. Multi-GPU and complex hybrid +topologies are deferred. + +#### Scratch, persistence and science + +Scratch and external-storage support is excluded from Gate G core. It adds no +scratch estimate fields, demand/availability API, manager, allocator, removable +media monitor, mount logic, capacity policy, spill contract, cleanup contract, +project scratch path or SSD detection. Sparse SfM has no scratch consumer, +spill algorithm or out-of-core path. Swap, zram and external SSD capacity never +enlarge scientific RAM admission. + +No automatic `swapon`, `swapoff`, mount, unmount, formatting, partitioning or +destructive cleanup is authorized. A future feature requires a real task +consumer, an explicit task-specific contract, user opt-in where appropriate and +a separate architecture review. No dormant generic API is introduced. + +Gate G core requires no Project DB v18, resource-history, reservation, +telemetry, policy or scratch table. Reservations and snapshots remain ephemeral; +the generic Task checkpoint already owns estimate durability. Resource policy, +machine data and hardware identity never enter F0, `sfm_version`, candidate +identity or a resource fingerprint. No resource-policy version is required for +Gate G core. + +Gate G never changes scientific thresholds, seeds, iteration limits, image or +track selection, Bundle Adjustment parameters, Gate D, Gate E, F0, candidate +identity, Track Model, Track Builder scientific semantics, Feature Store, +calibration identity or Project DB reconstruction semantics. + +#### Known derivable implementation defects + +These implementation obligations required no further human policy decision and +are implemented: + +- **G-D01:** `UINT64_MAX` is the final valid reservation ID; the following + reservation creation fails without an executable decision, reservation or + accounting charge. +- **G-D02:** pressure, recovery and slow-start streak counters must saturate at + the largest meaningful threshold and never wrap. +- **G-D03:** selected dedicated-GPU capacity and live usage must refer to the + same Hardware Profile-selected DRM device. + +`TOTAL REMAINING GATE G HUMAN DECISIONS: 0`. + +#### Closure validation + +The complete normal suite passes 41/41 and the Gate G resource/task core passes +the targeted ASan/UBSan/LSan validation with leak detection enabled. The three +OpenCL-touching tests `candidate-pair-task`, `feature-task` and +`precision-consolidation` pass functionally and under ASan/UBSan with leak +detection disabled. Their LeakSanitizer-only termination has the identical +external `/opt/cuda/lib64/libOpenCL.so` signature of 3808 bytes in 68 +allocations and is not classified as a Lardon3D Gate G leak. The real-machine +`orb-vulkan-backend` test, `git diff --check`, and the complete human diff review +pass. No Gate G human decision or implementation blocker remains. + ## Project DB boundary The historical Project DB v16 migration and Sparse SfM reconstruction model diff --git a/docs/architecture/resource_governor.md b/docs/architecture/resource_governor.md index 75e127f..e40c271 100644 --- a/docs/architecture/resource_governor.md +++ b/docs/architecture/resource_governor.md @@ -30,6 +30,32 @@ RED vers YELLOW, puis trois autres YELLOW vers GREEN. Le plafond reste 1 pendant ces phases. Une fois GREEN, chaque groupe de trois observations saines double le plafond : 1, 2, 4, 8, puis les paliers supérieurs utiles aux autres kinds. +## Contrat Gate G gelé + +**PASS / FROZEN.** Les constantes existantes ci-dessus +restent inchangées. La RAM disponible conserve le modèle conservateur +`min(MemAvailable, RAM physique) - réserve hôte - réservations actives`, borné à +zéro. Le double comptage conservateur possible d'une allocation déjà visible +dans `MemAvailable` est accepté : un faux `WAIT` est préféré à un overcommit. + +Les snapshots de production emploient `CLOCK_MONOTONIC` et sont valides jusqu'à +un âge exact de 1000 ms inclus. Un snapshot plus ancien ou daté dans le futur +produit `WAIT`, sans réservation ni mutation de l'état de politique du +Governor. La capture synchrone complète impossible reste une erreur +opérationnelle qui fait échouer la tâche avant callback. Une télémétrie PSI ou +vmstat optionnelle absente reste inconnue et ne crée aucune pression fictive. + +Gate G core cible un processus Linux natif non contraint et n'est pas cgroup, +systemd `MemoryMax` ou RLIMIT-aware. Il gouverne un seul GPU : le périphérique +DRM de plus petit numéro retenu par Hardware Profile. La capacité et l'usage +doivent provenir de ce même périphérique. La mémoire UMA est débitée exactement +une fois du budget RAM. Le multi-GPU est différé. + +Le Governor ne garantit aucune allocation et ne transforme ni swap, ni zram, +ni stockage externe en RAM. Il ne modifie aucun paramètre scientifique. Aucun +scratch, cache de télémétrie, suivi RSS, redimensionnement de réservation ou +monitoring live n'appartient à Gate G core. + ## API principale ### Création et destruction @@ -136,4 +162,12 @@ le plafond : 1, 2, 4, 8, puis les paliers supérieurs utiles aux autres kinds. - Pas de persistance des métriques - Pas de communication avec d'autres gouverneurs -## Statut : IMPLEMENTED +Les corrections dérivables G-D01 (`UINT64_MAX` est le dernier ID valide et la +création suivante échoue sans réservation ni charge comptable), +G-D02 (saturation des compteurs de streak) et G-D03 (identité DRM identique +entre capacité et usage) sont implémentées. Les sept décisions G-B01 à G-B07 +sont gelées ; il ne reste aucune décision humaine Gate G. + +## Statut + +**GATE G — PASS / FROZEN.** diff --git a/docs/architecture/scheduler_resource_integration.md b/docs/architecture/scheduler_resource_integration.md index cb31c35..845d046 100644 --- a/docs/architecture/scheduler_resource_integration.md +++ b/docs/architecture/scheduler_resource_integration.md @@ -41,6 +41,11 @@ Documenter l'intégration architecturale entre le scheduler de tâches et le Res - Le scheduler saute la tâche en tête de file - Il évalue la tâche suivante - Pas de blocage de la file +- Si aucune tâche n'est admissible mais qu'un `WAIT` PENDING subsiste, le worker + effectue une attente temporisée d'au plus 500 ms, puis rescane normalement la + file avec de nouveaux snapshots. Tout signal explicite le réveille plus tôt. +- Cette attente appartient au worker existant : aucun thread de monitoring, + nouveau scheduler ou nouveau sous-système n'est créé. ## Gestion des pauses @@ -60,6 +65,9 @@ Documenter l'intégration architecturale entre le scheduler de tâches et le Res - Capturer un nouvel instantané de ressources - Obtenir un contrat actualisé - Reprendre le callback en conservant la progression +- L'attente d'admission à cette frontière conserve son polling existant de + 50 ms ; elle est distincte des 500 ms de réévaluation d'une tâche initialement + PENDING. ### Avantages - Adaptation dynamique des lots en cours d'exécution @@ -80,4 +88,4 @@ Documenter l'intégration architecturale entre le scheduler de tâches et le Res - Pas de priorités - Pas de notification automatique de libération externe -## Statut : DOCUMENTATION DE L'IMPLÉMENTATION ACTUELLE +## Statut : GATE G — PASS / FROZEN diff --git a/docs/architecture/sparse_sfm.md b/docs/architecture/sparse_sfm.md index 430638b..5f4bfbe 100644 --- a/docs/architecture/sparse_sfm.md +++ b/docs/architecture/sparse_sfm.md @@ -198,7 +198,7 @@ as a whole, so Track identity and observation ownership remain simple. Bundle Adjustment implementation, E01--E35 matrix, normal suite, targeted ASan/UBSan with LeakSanitizer, full sequential ASan/UBSan suite and at least 20 fresh-process E27 comparisons are validated. Gate F project orchestration -is now **PASS / FROZEN**; Gate G resource integration remains a later gate. +is now **PASS / FROZEN**; Gate G resource integration is **PASS / FROZEN**. **DECISION: Gate E v1 is a synchronous, independent final per-component Bundle Adjustment applied as post-processing to a copy of the immutable final Gate D @@ -823,13 +823,21 @@ it never depends on current RAM, swap, PSI, load, queue depth, task attempt or Governor state. Gate F submits through the existing queue and does not perform admission or reservation itself. -The existing Governor owns admission and reservation policy. Future Gate G owns -telemetry, adaptive tuning, pressure and swap/scratch policy, scheduling policy -and estimate refinement. The estimate never enters the 372-byte parameter -record, candidate identity, `sfm_version` or scientific decisions, and cannot +The existing Governor owns admission and reservation policy. Gate G owns +telemetry, pressure and scheduling policy, but Gate G core neither changes the +frozen Sparse SfM producer estimate nor adds scratch support. The estimate never +enters the 372-byte parameter record, candidate identity, `sfm_version` or scientific decisions, and cannot change Gate D or Gate E parameters. This clarification preserves `NO_NEW_SUBSYSTEM` and the mandatory reservation invariant. +Gate G G0a freezes consumption of the exact Gate F v1 estimate. A restored task +keeps its persisted estimate and is evaluated with newly captured machine +telemetry; it is never recomputed with later coefficients. A future formula +change requires a separate operational formula/version review for newly created +tasks and cannot affect F0, candidate identity, Gate D/E parameters or existing +tasks. Sparse SfM remains batch one and has no scratch, spill or out-of-core +path. Swap, zram and external storage do not enlarge its RAM capacity. + ### Gate F durable task payload **FROZEN.** Gate F advances the current Project Database schema head from v16 @@ -916,7 +924,7 @@ component keys nor changes candidate identity. No persistent diagnostic table, sidecar, metadata blob or schema beyond v17 `sparse_sfm_tasks` is introduced. Gate F v1 is **PASS / FROZEN**. Gate D and Gate E remain **PASS / FROZEN**; -Gate G remains open and planned. +Gate G architecture decisions and implementation are **PASS / FROZEN**. ### Gate F validation closure @@ -933,7 +941,7 @@ Gate C — PASS / FROZEN Gate D — PASS / FROZEN Gate E — PASS / FROZEN Gate F — PASS / FROZEN -Gate G — OPEN / PLANNED +Gate G — PASS / FROZEN ``` The five candidate-identity dimensions remain separate: @@ -985,11 +993,11 @@ bounded storage. Triangulation/registration are light CPU units and can be batched. Gate E v1 uses local scientific limits for its final per-component BA and does not query -the Governor. Future Gate G admission may use `C`, `P`, `O`, solver mode and -calibration-variable count without changing scientific results. The existing -Resource Governor owns RAM/PSI/swap policy; Sparse SfM adds no system-pressure -thresholds. Swap is never normal working memory, and UMA RAM must preserve -several GiB of desktop/iGPU headroom. +the Governor. Gate G consumes the frozen Gate F estimate derived from immutable +workload shape without changing scientific results. The existing Resource +Governor owns RAM/PSI/swap policy; Sparse SfM adds no system-pressure thresholds. +Swap is never normal working memory, and UMA RAM must preserve several GiB of +desktop/iGPU headroom. ## Hardware and probe study @@ -1024,8 +1032,9 @@ production Sparse SfM code is created by this gate. contract frozen here; interleaved local BA is deferred. - **Gate F — PASS / FROZEN — Project orchestration:** explicit Track Set/calibration input, atomic publication and durable runtime integration. -- **Gate G — OPEN / PLANNED — Resource/freeze:** Governor admission, sustained hardware safety, - recovery, full validation and final freeze. +- **Gate G — PASS / FROZEN — Resource/freeze:** Governor + admission, sustained hardware safety and recovery are implemented and fully + validated. ## Algorithm comparison and Gate A evidence diff --git a/docs/architecture/task_queue.md b/docs/architecture/task_queue.md index 1a717a2..1e0d75c 100644 --- a/docs/architecture/task_queue.md +++ b/docs/architecture/task_queue.md @@ -44,6 +44,13 @@ Le type public réel est l'opaque `Lardon3DTaskQueue`. 4. Si aucune tâche n'est admissible, le worker attend un changement. 5. L'ordre de soumission est toujours respecté entre tâches de même priorité. +Lorsqu'au moins une tâche PENDING reste en `WAIT` de ressources, cette attente +est temporisée à 500 ms maximum. À l'expiration, le worker reprend le scan stable +depuis la tête et le chemin d'admission capture de nouveaux snapshots. Enqueue, +resume, `resources_changed`, annulation et arrêt continuent de réveiller +immédiatement le worker ; le timeout n'impose donc jamais 500 ms après un signal. +Cette règle utilise le worker unique existant et n'ajoute aucun thread. + ## Sélection adaptative Le sélecteur saute les tâches pour lesquelles le governor répond `WAIT` et @@ -77,8 +84,9 @@ le blocage par la tête de file lorsqu'une tâche ne peut pas démarrer. ## Statut -**IMPLÉMENTÉ** — file FIFO avec worker unique, sélection adaptative, pause, -annulation coopérative et accueil des tâches restaurées avec ID préassigné. +**GATE G — PASS / FROZEN** — file FIFO avec worker unique, +sélection adaptative, réévaluation autonome des `WAIT`, pause, annulation +coopérative et accueil des tâches restaurées avec ID préassigné. La reprise projet utilise `try_add_ex()` et ne bloque jamais `project_open()`. À saturation, elle arrête sa fenêtre : les tâches non transférées restent @@ -94,3 +102,5 @@ pas un second scheduler. une place se libère pendant la session courante. - Pas de pool de workers CPU/IO/GPU. - La backpressure borne les producteurs à la capacité configurée. +- Les 500 ms concernent uniquement l'admission initiale PENDING. Le polling + existant à une rupture de séquence reste 50 ms. diff --git a/docs/concepts/geometric_constraints.md b/docs/concepts/geometric_constraints.md index 8751adc..2f62aae 100644 --- a/docs/concepts/geometric_constraints.md +++ b/docs/concepts/geometric_constraints.md @@ -144,7 +144,7 @@ La qualité des triangles de triangulation影响 la précision de la reconstruct | **Matching & Tracks** | Les contraintes filtrent les matches bruts pour ne garder que les géométries cohérentes. | | **Visual Index** | Les candidats du Visual Index sont les entrées des contraintes géométriques. | | **Reconstruction Layers** | Chaque couche applique les contraintes appropriées pour valider ses résultats. | -| **Resource Governor** | Le gouverneur peut ajuster les seuils de contrainte en fonction des ressources disponibles. | +| **Resource Governor** | Admission et lots neutres uniquement ; aucun paramètre scientifique modifié. | | **Hardware Profile** | Le calcul des matrices fondamentales/essentielles est CPU-bound. | ## Contraintes de conception diff --git a/docs/roadmap/roadmap.md b/docs/roadmap/roadmap.md index 51945d4..96d924f 100644 --- a/docs/roadmap/roadmap.md +++ b/docs/roadmap/roadmap.md @@ -54,6 +54,7 @@ Lardon3D suit une feuille de route ordonnée qui privilégie la stabilité et la implémentés - ✅ Sparse SfM Gate E : Bundle Adjustment final par composante PASS / FROZEN - ✅ Sparse SfM Gate F : orchestration durable et publication atomique PASS / FROZEN +- ✅ Sparse SfM Gate G : politique et cœur **PASS / FROZEN** ### Phase 5 : Reconstruction (PLANNED) - 📋 Orchestration de reconstruction incrémentale diff --git a/include/lardon3d/hardware_profile.h b/include/lardon3d/hardware_profile.h index 7036ddd..19c9b6b 100644 --- a/include/lardon3d/hardware_profile.h +++ b/include/lardon3d/hardware_profile.h @@ -14,6 +14,7 @@ typedef struct { uint64_t page_size_bytes; uint64_t memory_total_bytes; bool gpu_available; + unsigned int gpu_drm_card_index; bool gpu_memory_known; bool gpu_uses_shared_memory; uint64_t gpu_memory_total_bytes; diff --git a/include/lardon3d/resource_snapshot.h b/include/lardon3d/resource_snapshot.h index 1b74abb..5c4fed2 100644 --- a/include/lardon3d/resource_snapshot.h +++ b/include/lardon3d/resource_snapshot.h @@ -8,6 +8,10 @@ #include +enum { + LARDON3D_RESOURCE_SNAPSHOT_MAX_AGE_MILLISECONDS = 1000, +}; + typedef struct { struct timespec captured_at; uint64_t memory_available_bytes; diff --git a/src/hardware_profile.c b/src/hardware_profile.c index 96a9705..ad8765d 100644 --- a/src/hardware_profile.c +++ b/src/hardware_profile.c @@ -10,6 +10,8 @@ #include +#include "hardware_profile_internal.h" + static void set_error(char *message, size_t size, const char *text) { @@ -81,22 +83,30 @@ vendor_name(const char *vendor) return vendor; } -static void -detect_gpu(Lardon3DHardwareProfile *profile) +void +lardon3d_hardware_profile_detect_gpu_at_root( + Lardon3DHardwareProfile *profile, + const char *drm_root +) { + if (!profile || !drm_root) { + return; + } for (unsigned int index = 0; index < 64; ++index) { char vendor_path[PATH_MAX]; char memory_path[PATH_MAX]; int vendor_written = snprintf( vendor_path, sizeof(vendor_path), - "/sys/class/drm/card%u/device/vendor", + "%s/card%u/device/vendor", + drm_root, index ); int memory_written = snprintf( memory_path, sizeof(memory_path), - "/sys/class/drm/card%u/device/mem_info_vram_total", + "%s/card%u/device/mem_info_vram_total", + drm_root, index ); if (vendor_written < 0 || (size_t)vendor_written >= sizeof(vendor_path) @@ -109,6 +119,7 @@ detect_gpu(Lardon3DHardwareProfile *profile) continue; } profile->gpu_available = true; + profile->gpu_drm_card_index = index; (void)snprintf( profile->gpu_name, sizeof(profile->gpu_name), @@ -177,6 +188,6 @@ lardon3d_hardware_profile_detect( "Linux" ); } - detect_gpu(profile); + lardon3d_hardware_profile_detect_gpu_at_root(profile, "/sys/class/drm"); return true; } diff --git a/src/hardware_profile_internal.h b/src/hardware_profile_internal.h new file mode 100644 index 0000000..2d55417 --- /dev/null +++ b/src/hardware_profile_internal.h @@ -0,0 +1,11 @@ +#ifndef LARDON3D_HARDWARE_PROFILE_INTERNAL_H +#define LARDON3D_HARDWARE_PROFILE_INTERNAL_H + +#include + +void lardon3d_hardware_profile_detect_gpu_at_root( + Lardon3DHardwareProfile *profile, + const char *drm_root +); + +#endif diff --git a/src/resource_governor.c b/src/resource_governor.c index f09fecd..3aad74e 100644 --- a/src/resource_governor.c +++ b/src/resource_governor.c @@ -6,6 +6,8 @@ #include +#include "resource_governor_internal.h" + struct Lardon3DResourceReservation { Lardon3DResourceReservationInfo information; uint64_t charged_memory_bytes; @@ -47,6 +49,9 @@ struct Lardon3DResourceGovernor { size_t slow_start_limit; bool slow_start_active; Lardon3DResourcePressure pressure; + bool internal_now_known; + struct timespec internal_now; + bool internal_force_capture_failure; Lardon3DResourceReservation *active; Lardon3DResourceReservation *released; }; @@ -56,6 +61,7 @@ valid_profile(const Lardon3DHardwareProfile *profile) { return profile && profile->logical_cpu_count > 0 && profile->memory_total_bytes > 0 + && (!profile->gpu_available || profile->gpu_drm_card_index < 64) && (!profile->gpu_memory_known || (profile->gpu_available && profile->gpu_memory_total_bytes > 0)); @@ -93,7 +99,10 @@ valid_snapshot( const Lardon3DResourceSnapshot *snapshot ) { - return snapshot && snapshot->cpu_load_1m >= 0.0 + return snapshot && snapshot->captured_at.tv_sec >= 0 + && snapshot->captured_at.tv_nsec >= 0 + && snapshot->captured_at.tv_nsec < 1000000000L + && snapshot->cpu_load_1m >= 0.0 && (!snapshot->cpu_pressure_known || (snapshot->cpu_pressure_avg10 >= 0.0 && snapshot->cpu_pressure_avg10 <= 100.0)) @@ -110,6 +119,53 @@ valid_snapshot( <= profile->gpu_memory_total_bytes); } +static unsigned int +increment_to_limit(unsigned int value, unsigned int limit) +{ + return value < limit ? value + 1 : limit; +} + +static bool +snapshot_is_fresh_locked( + const Lardon3DResourceGovernor *governor, + const Lardon3DResourceSnapshot *snapshot, + bool *fresh +) +{ + struct timespec now; + if (governor->internal_now_known) { + now = governor->internal_now; + } else if (clock_gettime(CLOCK_MONOTONIC, &now) != 0) { + return false; + } + if (snapshot->captured_at.tv_sec > now.tv_sec + || (snapshot->captured_at.tv_sec == now.tv_sec + && snapshot->captured_at.tv_nsec > now.tv_nsec)) { + *fresh = false; + return true; + } + if (snapshot->captured_at.tv_sec == now.tv_sec + && snapshot->captured_at.tv_nsec == now.tv_nsec) { + *fresh = true; + return true; + } + time_t seconds = now.tv_sec - snapshot->captured_at.tv_sec; + const uint64_t maximum_seconds = + LARDON3D_RESOURCE_SNAPSHOT_MAX_AGE_MILLISECONDS / 1000U; + const long maximum_nanoseconds = (long)( + LARDON3D_RESOURCE_SNAPSHOT_MAX_AGE_MILLISECONDS % 1000U + ) * 1000000L; + if ((uint64_t)seconds < maximum_seconds) { + *fresh = true; + } else if ((uint64_t)seconds > maximum_seconds) { + *fresh = false; + } else { + long elapsed_nanoseconds = now.tv_nsec - snapshot->captured_at.tv_nsec; + *fresh = elapsed_nanoseconds <= maximum_nanoseconds; + } + return true; +} + static bool valid_estimate(const Lardon3DResourceEstimate *estimate) { @@ -560,7 +616,7 @@ lardon3d_resource_governor_wait_for_change( return changed; } -static void +static bool evaluate_locked( Lardon3DResourceGovernor *governor, const Lardon3DResourceSnapshot *snapshot, @@ -570,7 +626,16 @@ evaluate_locked( { if (!valid_estimate(estimate)) { set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Estimation de ressources invalide."); - return; + return true; + } + bool fresh; + if (!snapshot_is_fresh_locked(governor, snapshot, &fresh)) { + return false; + } + if (!fresh) { + set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, + "Instantané de ressources périmé."); + return true; } bool swap_changed = false; if (snapshot->swap_activity_known) { @@ -612,7 +677,10 @@ evaluate_locked( } else if (pressure_signal) { governor->recovery_streak = 0; governor->slow_start_streak = 0; - ++governor->pressure_streak; + governor->pressure_streak = increment_to_limit( + governor->pressure_streak, + 2 + ); if (governor->pressure == LARDON3D_RESOURCE_PRESSURE_RED || governor->pressure_streak >= 2) { governor->pressure = LARDON3D_RESOURCE_PRESSURE_RED; @@ -624,7 +692,10 @@ evaluate_locked( } else { governor->pressure_streak = 0; if (governor->pressure == LARDON3D_RESOURCE_PRESSURE_RED) { - ++governor->recovery_streak; + governor->recovery_streak = increment_to_limit( + governor->recovery_streak, + 3 + ); if (governor->recovery_streak >= 3) { governor->pressure = LARDON3D_RESOURCE_PRESSURE_YELLOW; governor->recovery_streak = 0; @@ -635,14 +706,20 @@ evaluate_locked( governor->slow_start_limit = 1; governor->slow_start_streak = 0; } - ++governor->recovery_streak; + governor->recovery_streak = increment_to_limit( + governor->recovery_streak, + 3 + ); if (governor->recovery_streak >= 3) { governor->pressure = LARDON3D_RESOURCE_PRESSURE_GREEN; governor->recovery_streak = 0; governor->slow_start_streak = 0; } } else if (governor->slow_start_active) { - ++governor->slow_start_streak; + governor->slow_start_streak = increment_to_limit( + governor->slow_start_streak, + 3 + ); if (governor->slow_start_streak >= 3) { governor->slow_start_streak = 0; if (governor->slow_start_limit > SIZE_MAX / 2) { @@ -657,14 +734,14 @@ evaluate_locked( if (governor->pressure == LARDON3D_RESOURCE_PRESSURE_RED) { set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Pression mémoire ou swap persistante."); - return; + return true; } if ((estimate->desired_gpu_slots > 0 || estimate->gpu_memory_fixed_bytes > 0 || estimate->gpu_memory_bytes_per_item > 0) && !governor->profile.gpu_available) { set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Aucun GPU disponible."); - return; + return true; } uint64_t memory_fixed = estimate->memory_fixed_bytes; @@ -674,7 +751,7 @@ evaluate_locked( || memory_per_item > UINT64_MAX - estimate->gpu_memory_bytes_per_item) { set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Estimation mémoire trop grande."); - return; + return true; } memory_fixed += estimate->gpu_memory_fixed_bytes; memory_per_item += estimate->gpu_memory_bytes_per_item; @@ -702,14 +779,14 @@ evaluate_locked( } if (theoretical_batch < estimate->minimum_batch_size) { set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Tâche impossible sur cette machine."); - return; + return true; } double load_limit = (double)governor->profile.logical_cpu_count * governor->policy.maximum_cpu_load_ratio; if (snapshot->cpu_load_1m >= load_limit) { set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Charge CPU trop élevée."); - return; + return true; } if (governor->policy.maximum_cpu_pressure_avg10 > 0.0 && snapshot->cpu_pressure_known @@ -717,7 +794,7 @@ evaluate_locked( >= governor->policy.maximum_cpu_pressure_avg10) { set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Pression CPU trop élevée."); - return; + return true; } if (governor->policy.maximum_memory_pressure_avg10 > 0.0 && snapshot->memory_pressure_known @@ -725,19 +802,19 @@ evaluate_locked( >= governor->policy.maximum_memory_pressure_avg10) { set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Pression mémoire trop élevée."); - return; + return true; } if (estimate->desired_io_slots > 0 && snapshot->io_pressure_known && snapshot->io_pressure_avg10 >= governor->policy.maximum_io_pressure_avg10) { set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Pression d'entrée-sortie trop élevée."); - return; + return true; } Lardon3DResourceAvailability available; if (!availability_locked(governor, snapshot, &available)) { set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Instantané de ressources invalide."); - return; + return true; } size_t batch = batch_capacity( available.memory_available_bytes, @@ -749,7 +826,7 @@ evaluate_locked( || estimate->gpu_memory_bytes_per_item > 0)) { if (!available.gpu_memory_known) { set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Mémoire GPU disponible inconnue."); - return; + return true; } batch = minimum_size( batch, @@ -794,7 +871,7 @@ evaluate_locked( 0, "Ressources déjà réservées ou temporairement insuffisantes." ); - return; + return true; } if (available.cpu_available == 0 || (estimate->desired_gpu_slots > 0 @@ -802,7 +879,7 @@ evaluate_locked( || (estimate->desired_io_slots > 0 && available.io_slots_available == 0)) { set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Slots de calcul déjà réservés."); - return; + return true; } unsigned int cpu = estimate->desired_cpu_threads < available.cpu_available ? estimate->desired_cpu_threads @@ -828,6 +905,7 @@ evaluate_locked( reduced ? "Contrat réduit aux ressources disponibles." : "Ressources disponibles." ); + return true; } bool @@ -853,14 +931,24 @@ lardon3d_resource_governor_reserve( free(created); return false; } - evaluate_locked(governor, snapshot, estimate, decision); - if ((decision->kind != LARDON3D_RESOURCE_START - && decision->kind != LARDON3D_RESOURCE_REDUCE_BATCH) - || governor->next_reservation_id == 0) { + if (!evaluate_locked(governor, snapshot, estimate, decision)) { + (void)pthread_mutex_unlock(&governor->mutex); + free(created); + return false; + } + if (decision->kind != LARDON3D_RESOURCE_START + && decision->kind != LARDON3D_RESOURCE_REDUCE_BATCH) { (void)pthread_mutex_unlock(&governor->mutex); free(created); return true; } + if (governor->next_reservation_id == 0) { + set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, + "Identités de réservation épuisées."); + (void)pthread_mutex_unlock(&governor->mutex); + free(created); + return false; + } uint64_t memory_bytes; uint64_t gpu_memory_bytes; if (!resource_size( @@ -920,7 +1008,11 @@ lardon3d_resource_governor_reserve_available( } (void)pthread_mutex_lock(&governor->mutex); Lardon3DHardwareProfile profile = governor->profile; + bool force_capture_failure = governor->internal_force_capture_failure; (void)pthread_mutex_unlock(&governor->mutex); + if (force_capture_failure) { + return false; + } Lardon3DResourceSnapshot snapshot; if (!lardon3d_resource_snapshot_capture(&profile, &snapshot, NULL, 0)) { return false; @@ -1147,11 +1239,95 @@ lardon3d_resource_governor_decide( (void)pthread_mutex_unlock(&governor->mutex); return false; } - evaluate_locked(governor, snapshot, &estimate, decision); + bool success = evaluate_locked(governor, snapshot, &estimate, decision); + (void)pthread_mutex_unlock(&governor->mutex); + return success; +} + +bool +lardon3d_resource_governor_internal_set_next_reservation_id( + Lardon3DResourceGovernor *governor, + uint64_t next_reservation_id +) +{ + if (!governor || next_reservation_id == 0) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + governor->next_reservation_id = next_reservation_id; (void)pthread_mutex_unlock(&governor->mutex); return true; } +bool +lardon3d_resource_governor_internal_set_counters( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceGovernorInternalCounters *counters +) +{ + if (!governor || !counters) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + governor->pressure_streak = counters->pressure_streak; + governor->recovery_streak = counters->recovery_streak; + governor->slow_start_streak = counters->slow_start_streak; + (void)pthread_mutex_unlock(&governor->mutex); + return true; +} + +bool +lardon3d_resource_governor_internal_get_counters( + Lardon3DResourceGovernor *governor, + Lardon3DResourceGovernorInternalCounters *counters +) +{ + if (!governor || !counters) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + *counters = (Lardon3DResourceGovernorInternalCounters) { + .pressure_streak = governor->pressure_streak, + .recovery_streak = governor->recovery_streak, + .slow_start_streak = governor->slow_start_streak, + }; + (void)pthread_mutex_unlock(&governor->mutex); + return true; +} + +bool +lardon3d_resource_governor_internal_set_monotonic_now( + Lardon3DResourceGovernor *governor, + const struct timespec *now +) +{ + if (!governor || (now && (now->tv_sec < 0 || now->tv_nsec < 0 + || now->tv_nsec >= 1000000000L))) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + governor->internal_now_known = now != NULL; + if (now) { + governor->internal_now = *now; + } + (void)pthread_mutex_unlock(&governor->mutex); + return true; +} + +void +lardon3d_resource_governor_internal_force_capture_failure( + Lardon3DResourceGovernor *governor, + bool force_failure +) +{ + if (!governor) { + return; + } + (void)pthread_mutex_lock(&governor->mutex); + governor->internal_force_capture_failure = force_failure; + (void)pthread_mutex_unlock(&governor->mutex); +} + const char * lardon3d_resource_decision_name(Lardon3DResourceDecisionKind kind) { diff --git a/src/resource_governor_internal.h b/src/resource_governor_internal.h new file mode 100644 index 0000000..b2277e6 --- /dev/null +++ b/src/resource_governor_internal.h @@ -0,0 +1,37 @@ +#ifndef LARDON3D_RESOURCE_GOVERNOR_INTERNAL_H +#define LARDON3D_RESOURCE_GOVERNOR_INTERNAL_H + +#include +#include +#include + +#include + +typedef struct { + unsigned int pressure_streak; + unsigned int recovery_streak; + unsigned int slow_start_streak; +} Lardon3DResourceGovernorInternalCounters; + +bool lardon3d_resource_governor_internal_set_next_reservation_id( + Lardon3DResourceGovernor *governor, + uint64_t next_reservation_id +); +bool lardon3d_resource_governor_internal_set_counters( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceGovernorInternalCounters *counters +); +bool lardon3d_resource_governor_internal_get_counters( + Lardon3DResourceGovernor *governor, + Lardon3DResourceGovernorInternalCounters *counters +); +bool lardon3d_resource_governor_internal_set_monotonic_now( + Lardon3DResourceGovernor *governor, + const struct timespec *now +); +void lardon3d_resource_governor_internal_force_capture_failure( + Lardon3DResourceGovernor *governor, + bool force_failure +); + +#endif diff --git a/src/resource_snapshot.c b/src/resource_snapshot.c index 760f604..296ff72 100644 --- a/src/resource_snapshot.c +++ b/src/resource_snapshot.c @@ -8,6 +8,8 @@ #include +#include "resource_snapshot_internal.h" + enum { PROC_BUFFER_CAPACITY = 32768, }; @@ -132,13 +134,14 @@ vmstat_counter(const char *buffer, const char *key, uint64_t *value) return false; } -static void -capture_gpu_memory( +void +lardon3d_resource_snapshot_capture_gpu_at_root( const Lardon3DHardwareProfile *profile, - Lardon3DResourceSnapshot *snapshot + Lardon3DResourceSnapshot *snapshot, + const char *drm_root ) { - if (!profile->gpu_available) { + if (!profile || !snapshot || !drm_root || !profile->gpu_available) { return; } if (profile->gpu_uses_shared_memory && !profile->gpu_memory_known) { @@ -149,31 +152,29 @@ capture_gpu_memory( if (!profile->gpu_memory_known) { return; } - for (unsigned int index = 0; index < 64; ++index) { - char path[256]; - int written = snprintf( - path, - sizeof(path), - "/sys/class/drm/card%u/device/mem_info_vram_used", - index - ); - if (written < 0 || (size_t)written >= sizeof(path)) { - continue; - } - char buffer[64]; - if (!read_file(path, buffer, sizeof(buffer))) { - continue; - } - errno = 0; - char *end; - unsigned long long used = strtoull(buffer, &end, 10); - if (errno == 0 && end != buffer - && (uint64_t)used <= profile->gpu_memory_total_bytes) { - snapshot->gpu_memory_available_known = true; - snapshot->gpu_memory_available_bytes = - profile->gpu_memory_total_bytes - (uint64_t)used; - return; - } + char path[256]; + int written = snprintf( + path, + sizeof(path), + "%s/card%u/device/mem_info_vram_used", + drm_root, + profile->gpu_drm_card_index + ); + if (written < 0 || (size_t)written >= sizeof(path)) { + return; + } + char buffer[64]; + if (!read_file(path, buffer, sizeof(buffer))) { + return; + } + errno = 0; + char *end; + unsigned long long used = strtoull(buffer, &end, 10); + if (errno == 0 && end != buffer + && (uint64_t)used <= profile->gpu_memory_total_bytes) { + snapshot->gpu_memory_available_known = true; + snapshot->gpu_memory_available_bytes = + profile->gpu_memory_total_bytes - (uint64_t)used; } } @@ -202,7 +203,7 @@ lardon3d_resource_snapshot_capture( || !meminfo_bytes(buffer, "MemFree", &snapshot->memory_free_bytes) || !meminfo_bytes(buffer, "SwapFree", &snapshot->swap_available_bytes) || !capture_load(snapshot) - || clock_gettime(CLOCK_REALTIME, &snapshot->captured_at) != 0) { + || clock_gettime(CLOCK_MONOTONIC, &snapshot->captured_at) != 0) { set_error(error_message, error_message_size, "Instantané système impossible."); return false; } @@ -217,6 +218,10 @@ lardon3d_resource_snapshot_capture( buffer, "pswpin", &snapshot->swap_pages_in) && vmstat_counter(buffer, "pswpout", &snapshot->swap_pages_out); } - capture_gpu_memory(profile, snapshot); + lardon3d_resource_snapshot_capture_gpu_at_root( + profile, + snapshot, + "/sys/class/drm" + ); return true; } diff --git a/src/resource_snapshot_internal.h b/src/resource_snapshot_internal.h new file mode 100644 index 0000000..da753f7 --- /dev/null +++ b/src/resource_snapshot_internal.h @@ -0,0 +1,12 @@ +#ifndef LARDON3D_RESOURCE_SNAPSHOT_INTERNAL_H +#define LARDON3D_RESOURCE_SNAPSHOT_INTERNAL_H + +#include + +void lardon3d_resource_snapshot_capture_gpu_at_root( + const Lardon3DHardwareProfile *profile, + Lardon3DResourceSnapshot *snapshot, + const char *drm_root +); + +#endif diff --git a/src/task_queue.c b/src/task_queue.c index 5ad03bf..7e16980 100644 --- a/src/task_queue.c +++ b/src/task_queue.c @@ -2,6 +2,7 @@ #include #include #include +#include #include @@ -11,6 +12,10 @@ typedef struct TaskNode { struct TaskNode *next_pending; } TaskNode; +enum { + LARDON3D_PENDING_RESOURCE_WAIT_MILLISECONDS = 500, +}; + struct Lardon3DTaskQueue { pthread_mutex_t mutex; pthread_cond_t not_empty; @@ -63,9 +68,11 @@ unlink_pending(Lardon3DTaskQueue *queue, TaskNode *previous, TaskNode *node) static Lardon3DTask * select_admissible( Lardon3DTaskQueue *queue, - Lardon3DResourceReservation **reservation + Lardon3DResourceReservation **reservation, + bool *resource_wait_pending ) { + *resource_wait_pending = false; TaskNode *previous = NULL; TaskNode *node = queue->pending_head; while (node) { @@ -101,6 +108,7 @@ select_admissible( continue; } if (decision.kind == LARDON3D_RESOURCE_WAIT) { + *resource_wait_pending = true; previous = node; node = next; continue; @@ -118,6 +126,30 @@ select_admissible( return NULL; } +static void +wait_for_pending_change(Lardon3DTaskQueue *queue, bool resource_wait_pending) +{ + if (!resource_wait_pending) { + (void)pthread_cond_wait(&queue->not_empty, &queue->mutex); + return; + } + struct timespec deadline; + if (clock_gettime(CLOCK_MONOTONIC, &deadline) != 0) { + return; + } + deadline.tv_nsec += + LARDON3D_PENDING_RESOURCE_WAIT_MILLISECONDS * 1000000L; + if (deadline.tv_nsec >= 1000000000L) { + ++deadline.tv_sec; + deadline.tv_nsec -= 1000000000L; + } + (void)pthread_cond_timedwait( + &queue->not_empty, + &queue->mutex, + &deadline + ); +} + static void * queue_worker(void *context) { @@ -132,9 +164,14 @@ queue_worker(void *context) return NULL; } Lardon3DResourceReservation *reservation = NULL; - Lardon3DTask *selected = select_admissible(queue, &reservation); + bool resource_wait_pending; + Lardon3DTask *selected = select_admissible( + queue, + &reservation, + &resource_wait_pending + ); if (!selected) { - (void)pthread_cond_wait(&queue->not_empty, &queue->mutex); + wait_for_pending_change(queue, resource_wait_pending); (void)pthread_mutex_unlock(&queue->mutex); continue; } @@ -177,11 +214,26 @@ lardon3d_task_queue_create(Lardon3DResourceGovernor *governor, size_t capacity) free(queue); return NULL; } - if (pthread_cond_init(&queue->not_empty, NULL) != 0) { + pthread_condattr_t not_empty_attributes; + if (pthread_condattr_init(¬_empty_attributes) != 0) { (void)pthread_mutex_destroy(&queue->mutex); free(queue); return NULL; } + if (pthread_condattr_setclock( + ¬_empty_attributes, + CLOCK_MONOTONIC + ) != 0 + || pthread_cond_init( + &queue->not_empty, + ¬_empty_attributes + ) != 0) { + (void)pthread_condattr_destroy(¬_empty_attributes); + (void)pthread_mutex_destroy(&queue->mutex); + free(queue); + return NULL; + } + (void)pthread_condattr_destroy(¬_empty_attributes); if (pthread_cond_init(&queue->not_full, NULL) != 0) { (void)pthread_cond_destroy(&queue->not_empty); (void)pthread_mutex_destroy(&queue->mutex); diff --git a/tests/resource_snapshot_test_utils.h b/tests/resource_snapshot_test_utils.h new file mode 100644 index 0000000..c46e2d6 --- /dev/null +++ b/tests/resource_snapshot_test_utils.h @@ -0,0 +1,18 @@ +#ifndef LARDON3D_RESOURCE_SNAPSHOT_TEST_UTILS_H +#define LARDON3D_RESOURCE_SNAPSHOT_TEST_UTILS_H + +#include +#include + +#include + +static inline bool +lardon3d_test_resource_snapshot_make_fresh( + Lardon3DResourceSnapshot *snapshot +) +{ + return snapshot + && clock_gettime(CLOCK_MONOTONIC, &snapshot->captured_at) == 0; +} + +#endif diff --git a/tests/test_candidate_pair_task.c b/tests/test_candidate_pair_task.c index ffdc231..012bd19 100644 --- a/tests/test_candidate_pair_task.c +++ b/tests/test_candidate_pair_task.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include @@ -95,14 +96,23 @@ static bool runtime(Lardon3DAppState *s) { static bool wait_state(Lardon3DTaskQueue *q, uint64_t id, Lardon3DTaskState wanted, Lardon3DTaskSnapshot *out) { - for (size_t i = 0; i < 2000000; i++) { + struct timespec deadline; + if (clock_gettime(CLOCK_MONOTONIC, &deadline) != 0) return false; + deadline.tv_sec += 5; + for (;;) { if (lardon3d_task_queue_get(q, id, out) && out->state == wanted) { return true; } + struct timespec now; + if (clock_gettime(CLOCK_MONOTONIC, &now) != 0 || + now.tv_sec > deadline.tv_sec || + (now.tv_sec == deadline.tv_sec && + now.tv_nsec >= deadline.tv_nsec)) { + return false; + } sched_yield(); } - return false; } static bool run_test(void) { diff --git a/tests/test_feature_task.c b/tests/test_feature_task.c index ad49998..eea277c 100644 --- a/tests/test_feature_task.c +++ b/tests/test_feature_task.c @@ -10,6 +10,7 @@ #include #include #include +#include #include #include @@ -106,13 +107,20 @@ static bool runtime(Lardon3DAppState *s) { } static bool wait_state(Lardon3DTaskQueue *q, uint64_t id, Lardon3DTaskState wanted, Lardon3DTaskSnapshot *out) { - for (size_t i = 0; i < 2000000; i++) { + struct timespec deadline; + if (clock_gettime(CLOCK_MONOTONIC, &deadline) != 0) return false; + deadline.tv_sec += 5; + for (;;) { if (lardon3d_task_queue_get(q, id, out) && out->state == wanted) { return true; } + struct timespec now; + if (clock_gettime(CLOCK_MONOTONIC, &now) != 0 || now.tv_sec > deadline.tv_sec || + (now.tv_sec == deadline.tv_sec && now.tv_nsec >= deadline.tv_nsec)) { + return false; + } sched_yield(); } - return false; } typedef struct { diff --git a/tests/test_hardware_profile.c b/tests/test_hardware_profile.c index 8956b34..c1a5dfa 100644 --- a/tests/test_hardware_profile.c +++ b/tests/test_hardware_profile.c @@ -32,6 +32,7 @@ run_test(void) } if (profile.gpu_available) { CHECK(profile.gpu_name[0]); + CHECK(profile.gpu_drm_card_index < 64); } Lardon3DHardwareProfile second; CHECK(lardon3d_hardware_profile_detect(&second, NULL, 0)); diff --git a/tests/test_import_task.c b/tests/test_import_task.c index 7fc7c17..b017017 100644 --- a/tests/test_import_task.c +++ b/tests/test_import_task.c @@ -15,6 +15,8 @@ #include #include +#include "resource_snapshot_test_utils.h" + #define CHECK(condition) do { if (!(condition)) { \ (void)fprintf(stderr, "Échec ligne %d : %s\n", __LINE__, #condition); return false; \ } } while (0) @@ -253,8 +255,10 @@ test_selective_capacity_one(void) }; Lardon3DResourceDecision decision; Lardon3DResourceReservation *held = NULL; + CHECK(lardon3d_test_resource_snapshot_make_fresh(&available)); CHECK(lardon3d_resource_governor_reserve(state.resource_governor, &available, &held_estimate, &decision, &held)); + CHECK(held); state.task_queue = lardon3d_task_queue_create(state.resource_governor, 1); CHECK(state.task_queue && lardon3d_project_open(&state, "Selective Recovery")); diff --git a/tests/test_precision_consolidation.c b/tests/test_precision_consolidation.c index 771ab40..1b819cf 100644 --- a/tests/test_precision_consolidation.c +++ b/tests/test_precision_consolidation.c @@ -158,12 +158,18 @@ static bool runtime(Lardon3DAppState *s) { static bool wait_state(Lardon3DTaskQueue *q, uint64_t id, Lardon3DTaskState wanted, Lardon3DTaskSnapshot *out) { - for (size_t i = 0; i < 2000000; i++) { + struct timespec deadline; + if (clock_gettime(CLOCK_MONOTONIC, &deadline) != 0) return false; + deadline.tv_sec += 5; + for (;;) { if (lardon3d_task_queue_get(q, id, out) && out->state == wanted) return true; + struct timespec now; + if (clock_gettime(CLOCK_MONOTONIC, &now) != 0 || now.tv_sec > deadline.tv_sec || + (now.tv_sec == deadline.tv_sec && now.tv_nsec >= deadline.tv_nsec)) + return false; sched_yield(); } - return false; } /* -- Concurrency types -- */ @@ -1616,4 +1622,4 @@ int main(void) { "===\n", pass, fail, total); return fail ? 1 : 0; -} \ No newline at end of file +} diff --git a/tests/test_resource_governor.c b/tests/test_resource_governor.c index fa671b2..cf8676a 100644 --- a/tests/test_resource_governor.c +++ b/tests/test_resource_governor.c @@ -1,4 +1,5 @@ #include +#include #include #include #include @@ -7,6 +8,8 @@ #include +#include "../src/resource_governor_internal.h" + #define CHECK(condition) \ do { \ if (!(condition)) { \ @@ -18,6 +21,13 @@ #define GIBIBYTES(value) ((uint64_t)(value) * 1024 * 1024 * 1024) #define MEBIBYTES(value) ((uint64_t)(value) * 1024 * 1024) +static bool +use_fixed_test_clock(Lardon3DResourceGovernor *governor) +{ + const struct timespec now = {0}; + return lardon3d_resource_governor_internal_set_monotonic_now(governor, &now); +} + typedef struct { Lardon3DResourceGovernor *governor; Lardon3DResourceSnapshot snapshot; @@ -104,7 +114,7 @@ run_test(void) &profile, &policy ); - CHECK(governor); + CHECK(governor && use_fixed_test_clock(governor)); Lardon3DResourceSnapshot snapshot = { .memory_available_bytes = GIBIBYTES(10), .cpu_load_1m = 2.0, @@ -214,7 +224,7 @@ run_test(void) lardon3d_resource_governor_destroy(governor); governor = lardon3d_resource_governor_create(&profile, &policy); - CHECK(governor); + CHECK(governor && use_fixed_test_clock(governor)); snapshot.cpu_pressure_known = false; snapshot.memory_pressure_known = false; snapshot.io_pressure_known = false; @@ -230,7 +240,7 @@ run_test(void) policy.emergency_memory_floor_bytes = GIBIBYTES(1); governor = lardon3d_resource_governor_create(&profile, &policy); - CHECK(governor); + CHECK(governor && use_fixed_test_clock(governor)); request = (Lardon3DResourceRequest) { .minimum_batch_size = 1, .preferred_batch_size = 8, @@ -255,7 +265,7 @@ run_test(void) lardon3d_resource_governor_destroy(governor); governor = lardon3d_resource_governor_create(&profile, &policy); - CHECK(governor); + CHECK(governor && use_fixed_test_clock(governor)); snapshot.memory_available_bytes = GIBIBYTES(10); snapshot.swap_activity_known = true; snapshot.swap_pages_in = 50; @@ -281,7 +291,7 @@ run_test(void) policy.gpu_memory_reserve_bytes = 0; policy.gpu_slot_capacity = 2; governor = lardon3d_resource_governor_create(&profile, &policy); - CHECK(governor); + CHECK(governor && use_fixed_test_clock(governor)); request = (Lardon3DResourceRequest) { .memory_bytes_per_item = GIBIBYTES(1), .gpu_memory_bytes_per_item = GIBIBYTES(1), @@ -302,7 +312,7 @@ run_test(void) profile.gpu_memory_total_bytes = GIBIBYTES(4); policy.gpu_memory_reserve_bytes = MEBIBYTES(512); governor = lardon3d_resource_governor_create(&profile, &policy); - CHECK(governor); + CHECK(governor && use_fixed_test_clock(governor)); request.minimum_batch_size = 1; request.preferred_batch_size = 3; request.gpu_memory_bytes_per_item = GIBIBYTES(1); @@ -365,7 +375,7 @@ run_generation_test(void) &profile, &policy ); - CHECK(governor); + CHECK(governor && use_fixed_test_clock(governor)); /* Test 1 : génération initiale = 0. */ CHECK(lardon3d_resource_governor_generation(governor) == 0); @@ -529,7 +539,7 @@ run_adaptive_batch_test(void) &profile, &policy ); - CHECK(governor); + CHECK(governor && use_fixed_test_clock(governor)); Lardon3DResourceSnapshot snapshot = { .memory_available_bytes = GIBIBYTES(10), @@ -734,10 +744,216 @@ run_adaptive_batch_test(void) return true; } +static bool +run_gate_g_boundary_test(void) +{ + CHECK(LARDON3D_RESOURCE_SNAPSHOT_MAX_AGE_MILLISECONDS == 1000); + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 8, + .page_size_bytes = 4096, + .memory_total_bytes = GIBIBYTES(8), + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy = { + .system_memory_reserve_bytes = GIBIBYTES(2), + .emergency_memory_floor_bytes = GIBIBYTES(1), + .system_cpu_reserve = 2, + .maximum_cpu_load_ratio = 1.0, + .maximum_cpu_pressure_avg10 = 20.0, + .maximum_memory_pressure_avg10 = 1.0, + .maximum_io_pressure_avg10 = 80.0, + .io_slot_capacity = 1, + }; + Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create( + &profile, + &policy + ); + CHECK(governor); + const struct timespec now = {.tv_sec = 10, .tv_nsec = 500}; + CHECK(lardon3d_resource_governor_internal_set_monotonic_now(governor, &now)); + Lardon3DResourceSnapshot snapshot = { + .captured_at = {.tv_sec = 9, .tv_nsec = 500}, + .memory_available_bytes = GIBIBYTES(8), + .cpu_load_1m = 0.0, + }; + Lardon3DResourceEstimate estimate = { + .memory_fixed_bytes = MEBIBYTES(1), + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 1, + .task_class = LARDON3D_RESOURCE_TASK_CPU, + }; + Lardon3DResourceDecision decision; + Lardon3DResourceReservation *reservation = NULL; + CHECK(lardon3d_resource_governor_reserve( + governor, &snapshot, &estimate, &decision, &reservation)); + CHECK(reservation); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + + Lardon3DResourceGovernorInternalCounters counters = { + .pressure_streak = 1, + .recovery_streak = 2, + .slow_start_streak = 2, + }; + CHECK(lardon3d_resource_governor_internal_set_counters(governor, &counters)); + snapshot.captured_at = (struct timespec) {.tv_sec = 9, .tv_nsec = 499}; + snapshot.memory_pressure_known = true; + snapshot.memory_pressure_avg10 = 100.0; + reservation = NULL; + CHECK(lardon3d_resource_governor_reserve( + governor, &snapshot, &estimate, &decision, &reservation)); + CHECK(decision.kind == LARDON3D_RESOURCE_WAIT); + CHECK(!reservation); + Lardon3DResourceGovernorInternalCounters after; + CHECK(lardon3d_resource_governor_internal_get_counters(governor, &after)); + CHECK(after.pressure_streak == 1); + CHECK(after.recovery_streak == 2); + CHECK(after.slow_start_streak == 2); + + const struct timespec future_snapshots[] = { + {.tv_sec = 10, .tv_nsec = 501}, + {.tv_sec = 11, .tv_nsec = 0}, + }; + for (size_t i = 0; i < sizeof(future_snapshots) / sizeof(future_snapshots[0]); + ++i) { + snapshot.captured_at = future_snapshots[i]; + reservation = NULL; + CHECK(lardon3d_resource_governor_reserve( + governor, &snapshot, &estimate, &decision, &reservation)); + CHECK(decision.kind == LARDON3D_RESOURCE_WAIT); + CHECK(!reservation); + CHECK(lardon3d_resource_governor_internal_get_counters(governor, &after)); + CHECK(after.pressure_streak == 1); + CHECK(after.recovery_streak == 2); + CHECK(after.slow_start_streak == 2); + } + + snapshot.captured_at = now; + CHECK(lardon3d_resource_governor_internal_set_next_reservation_id( + governor, + UINT64_MAX + )); + snapshot.memory_pressure_known = false; + CHECK(lardon3d_resource_governor_reserve( + governor, &snapshot, &estimate, &decision, &reservation)); + CHECK(reservation); + Lardon3DResourceReservationInfo information; + CHECK(lardon3d_resource_reservation_get_active( + governor, reservation, &information)); + CHECK(information.id == UINT64_MAX); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + Lardon3DResourceAvailability availability_before; + Lardon3DResourceAvailability availability_after; + CHECK(lardon3d_resource_governor_availability( + governor, + &snapshot, + &availability_before + )); + reservation = NULL; + CHECK(!lardon3d_resource_governor_reserve( + governor, &snapshot, &estimate, &decision, &reservation)); + CHECK(!reservation); + CHECK(decision.kind != LARDON3D_RESOURCE_START); + CHECK(decision.kind != LARDON3D_RESOURCE_REDUCE_BATCH); + CHECK(lardon3d_resource_governor_reservation_count(governor) == 0); + CHECK(lardon3d_resource_governor_availability( + governor, + &snapshot, + &availability_after + )); + CHECK(availability_after.memory_reserved_bytes + == availability_before.memory_reserved_bytes); + CHECK(availability_after.gpu_memory_reserved_bytes + == availability_before.gpu_memory_reserved_bytes); + CHECK(availability_after.cpu_reserved == availability_before.cpu_reserved); + CHECK(availability_after.gpu_slots_reserved + == availability_before.gpu_slots_reserved); + CHECK(availability_after.io_slots_reserved + == availability_before.io_slots_reserved); + + counters = (Lardon3DResourceGovernorInternalCounters) { + .pressure_streak = UINT_MAX, + }; + CHECK(lardon3d_resource_governor_internal_set_counters(governor, &counters)); + snapshot.memory_pressure_known = true; + snapshot.memory_pressure_avg10 = 100.0; + CHECK(lardon3d_resource_governor_decide( + governor, + &snapshot, + &(Lardon3DResourceRequest) { + .minimum_batch_size = 1, + .preferred_batch_size = 1, + .requested_cpu_threads = 1, + }, + &decision + )); + CHECK(lardon3d_resource_governor_internal_get_counters(governor, &after)); + CHECK(after.pressure_streak == 2); + CHECK(lardon3d_resource_governor_pressure(governor) + == LARDON3D_RESOURCE_PRESSURE_RED); + + counters = (Lardon3DResourceGovernorInternalCounters) { + .recovery_streak = UINT_MAX, + }; + CHECK(lardon3d_resource_governor_internal_set_counters(governor, &counters)); + snapshot.memory_pressure_known = false; + CHECK(lardon3d_resource_governor_decide( + governor, + &snapshot, + &(Lardon3DResourceRequest) { + .minimum_batch_size = 1, + .preferred_batch_size = 1, + .requested_cpu_threads = 1, + }, + &decision + )); + CHECK(lardon3d_resource_governor_internal_get_counters(governor, &after)); + CHECK(after.recovery_streak == 0); + CHECK(lardon3d_resource_governor_pressure(governor) + == LARDON3D_RESOURCE_PRESSURE_YELLOW); + + counters = (Lardon3DResourceGovernorInternalCounters) { + .recovery_streak = UINT_MAX, + }; + CHECK(lardon3d_resource_governor_internal_set_counters(governor, &counters)); + CHECK(lardon3d_resource_governor_decide( + governor, + &snapshot, + &(Lardon3DResourceRequest) { + .minimum_batch_size = 1, + .preferred_batch_size = 1, + .requested_cpu_threads = 1, + }, + &decision + )); + CHECK(lardon3d_resource_governor_pressure(governor) + == LARDON3D_RESOURCE_PRESSURE_GREEN); + counters = (Lardon3DResourceGovernorInternalCounters) { + .slow_start_streak = UINT_MAX, + }; + CHECK(lardon3d_resource_governor_internal_set_counters(governor, &counters)); + CHECK(lardon3d_resource_governor_decide( + governor, + &snapshot, + &(Lardon3DResourceRequest) { + .minimum_batch_size = 1, + .preferred_batch_size = 1, + .requested_cpu_threads = 1, + }, + &decision + )); + CHECK(lardon3d_resource_governor_internal_get_counters(governor, &after)); + CHECK(after.slow_start_streak == 0); + + lardon3d_resource_governor_destroy(governor); + return true; +} + int main(void) { - return (run_test() && run_generation_test() && run_adaptive_batch_test()) + return (run_test() && run_generation_test() && run_adaptive_batch_test() + && run_gate_g_boundary_test()) ? EXIT_SUCCESS : EXIT_FAILURE; } diff --git a/tests/test_resource_reservation.c b/tests/test_resource_reservation.c index 911f8bf..15337c2 100644 --- a/tests/test_resource_reservation.c +++ b/tests/test_resource_reservation.c @@ -6,6 +6,8 @@ #include +#include "../src/resource_governor_internal.h" + #define CHECK(condition) \ do { \ if (!(condition)) { \ @@ -17,6 +19,13 @@ #define GIBIBYTES(value) ((uint64_t)(value) * 1024 * 1024 * 1024) #define MEBIBYTES(value) ((uint64_t)(value) * 1024 * 1024) +static bool +use_fixed_test_clock(Lardon3DResourceGovernor *governor) +{ + const struct timespec now = {0}; + return lardon3d_resource_governor_internal_set_monotonic_now(governor, &now); +} + enum { THREAD_COUNT = 16, RESERVATIONS_PER_THREAD = 25, @@ -101,7 +110,7 @@ test_adaptive_batches(void) &profile, &policy ); - CHECK(governor); + CHECK(governor && use_fixed_test_clock(governor)); Lardon3DResourceSnapshot snapshot = { .memory_available_bytes = GIBIBYTES(37), .cpu_load_1m = 0.0, @@ -231,7 +240,7 @@ test_multiple_reservations(void) &profile, &policy ); - CHECK(governor); + CHECK(governor && use_fixed_test_clock(governor)); Lardon3DResourceSnapshot snapshot = { .memory_available_bytes = GIBIBYTES(8), .cpu_load_1m = 0.0, @@ -310,6 +319,8 @@ test_gpu_reservation(void) &policy ); CHECK(governor && other); + CHECK(use_fixed_test_clock(governor)); + CHECK(use_fixed_test_clock(other)); Lardon3DResourceSnapshot snapshot = { .memory_available_bytes = GIBIBYTES(16), .gpu_memory_available_known = true, @@ -378,7 +389,7 @@ test_concurrency(void) &profile, &policy ); - CHECK(governor); + CHECK(governor && use_fixed_test_clock(governor)); Lardon3DResourceReservation *reservations[RESERVATION_COUNT] = {0}; pthread_barrier_t created; pthread_barrier_t release; @@ -455,7 +466,7 @@ test_destroy_with_active_reservations(void) &profile, &policy ); - CHECK(governor); + CHECK(governor && use_fixed_test_clock(governor)); Lardon3DResourceSnapshot snapshot = { .memory_available_bytes = GIBIBYTES(4), .cpu_load_1m = 0.0, diff --git a/tests/test_resource_snapshot.c b/tests/test_resource_snapshot.c index 80a46dd..f52aae8 100644 --- a/tests/test_resource_snapshot.c +++ b/tests/test_resource_snapshot.c @@ -1,10 +1,17 @@ #include +#include #include #include +#include +#include +#include #include #include +#include "../src/hardware_profile_internal.h" +#include "../src/resource_snapshot_internal.h" + #define CHECK(condition) \ do { \ if (!(condition)) { \ @@ -13,6 +20,94 @@ } \ } while (0) +static bool +write_text(const char *path, const char *text) +{ + int descriptor = open(path, O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, 0600); + if (descriptor < 0) { + return false; + } + size_t length = strlen(text); + ssize_t written = write(descriptor, text, length); + bool success = written == (ssize_t)length; + if (close(descriptor) != 0) { + success = false; + } + return success; +} + +static bool +create_card(const char *root, unsigned int index, const char *total, const char *used) +{ + char card[256]; + char device[256]; + char path[256]; + if (snprintf(card, sizeof(card), "%s/card%u", root, index) < 0 + || snprintf(device, sizeof(device), "%s/device", card) < 0 + || mkdir(card, 0700) != 0 || mkdir(device, 0700) != 0) { + return false; + } + (void)snprintf(path, sizeof(path), "%s/vendor", device); + if (!write_text(path, "0x1002\n")) { + return false; + } + (void)snprintf(path, sizeof(path), "%s/mem_info_vram_total", device); + if (!write_text(path, total)) { + return false; + } + (void)snprintf(path, sizeof(path), "%s/mem_info_vram_used", device); + return write_text(path, used); +} + +static void +remove_card(const char *root, unsigned int index) +{ + char path[256]; + const char *files[] = {"vendor", "mem_info_vram_total", "mem_info_vram_used"}; + for (size_t i = 0; i < sizeof(files) / sizeof(files[0]); ++i) { + (void)snprintf(path, sizeof(path), "%s/card%u/device/%s", root, index, files[i]); + (void)unlink(path); + } + (void)snprintf(path, sizeof(path), "%s/card%u/device", root, index); + (void)rmdir(path); + (void)snprintf(path, sizeof(path), "%s/card%u", root, index); + (void)rmdir(path); +} + +static bool +test_selected_gpu_pairing(void) +{ + char root[] = "/tmp/lardon3d-drm-XXXXXX"; + CHECK(mkdtemp(root)); + CHECK(create_card(root, 0, "1000\n", "100\n")); + CHECK(create_card(root, 1, "4000\n", "3000\n")); + Lardon3DHardwareProfile profile = {0}; + lardon3d_hardware_profile_detect_gpu_at_root(&profile, root); + CHECK(profile.gpu_available); + CHECK(profile.gpu_drm_card_index == 0); + CHECK(profile.gpu_memory_total_bytes == 1000); + Lardon3DResourceSnapshot snapshot = {.memory_available_bytes = 800}; + lardon3d_resource_snapshot_capture_gpu_at_root(&profile, &snapshot, root); + CHECK(snapshot.gpu_memory_available_known); + CHECK(snapshot.gpu_memory_available_bytes == 900); + + char selected_usage[256]; + (void)snprintf( + selected_usage, + sizeof(selected_usage), + "%s/card0/device/mem_info_vram_used", + root + ); + CHECK(unlink(selected_usage) == 0); + snapshot = (Lardon3DResourceSnapshot) {0}; + lardon3d_resource_snapshot_capture_gpu_at_root(&profile, &snapshot, root); + CHECK(!snapshot.gpu_memory_available_known); + remove_card(root, 0); + remove_card(root, 1); + CHECK(rmdir(root) == 0); + return true; +} + static bool run_test(void) { @@ -30,14 +125,24 @@ run_test(void) Lardon3DHardwareProfile profile; CHECK(lardon3d_hardware_profile_detect(&profile, error, sizeof(error))); + struct timespec before; + struct timespec after; + CHECK(clock_gettime(CLOCK_MONOTONIC, &before) == 0); CHECK(lardon3d_resource_snapshot_capture( &profile, &snapshot, error, sizeof(error) )); + CHECK(clock_gettime(CLOCK_MONOTONIC, &after) == 0); CHECK(error[0] == '\0'); CHECK(snapshot.captured_at.tv_sec > 0); + CHECK(snapshot.captured_at.tv_sec > before.tv_sec + || (snapshot.captured_at.tv_sec == before.tv_sec + && snapshot.captured_at.tv_nsec >= before.tv_nsec)); + CHECK(snapshot.captured_at.tv_sec < after.tv_sec + || (snapshot.captured_at.tv_sec == after.tv_sec + && snapshot.captured_at.tv_nsec <= after.tv_nsec)); CHECK(snapshot.memory_available_bytes > 0); CHECK(snapshot.memory_free_bytes <= profile.memory_total_bytes); CHECK(snapshot.cpu_load_1m >= 0.0); @@ -58,6 +163,7 @@ run_test(void) CHECK(snapshot.memory_pressure_avg10 >= 0.0); CHECK(snapshot.memory_pressure_avg10 <= 100.0); } + CHECK(test_selected_gpu_pairing()); return true; } diff --git a/tests/test_sequential_task.c b/tests/test_sequential_task.c index f0dcbe8..564bbe9 100644 --- a/tests/test_sequential_task.c +++ b/tests/test_sequential_task.c @@ -8,6 +8,8 @@ #include +#include "resource_snapshot_test_utils.h" + #define CHECK(condition) \ do { \ if (!(condition)) { \ @@ -222,6 +224,7 @@ run_sequential_test(void) }; Lardon3DResourceDecision decision; Lardon3DResourceReservation *reservation; + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resource_snapshot)); CHECK(lardon3d_resource_governor_reserve( governor, &resource_snapshot, @@ -302,6 +305,7 @@ run_wait_retry_test(void) }; Lardon3DResourceDecision decision; Lardon3DResourceReservation *reservation; + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resource_snapshot)); CHECK(lardon3d_resource_governor_reserve( governor, &resource_snapshot, @@ -387,6 +391,7 @@ run_cancel_during_wait_test(void) }; Lardon3DResourceDecision decision; Lardon3DResourceReservation *reservation; + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resource_snapshot)); CHECK(lardon3d_resource_governor_reserve( governor, &resource_snapshot, @@ -459,6 +464,7 @@ run_pause_during_wait_test(void) }; Lardon3DResourceDecision decision; Lardon3DResourceReservation *reservation; + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resource_snapshot)); CHECK(lardon3d_resource_governor_reserve( governor, &resource_snapshot, @@ -545,6 +551,7 @@ run_reject_test(void) }; Lardon3DResourceDecision decision; Lardon3DResourceReservation *reservation; + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resource_snapshot)); CHECK(lardon3d_resource_governor_reserve( governor, &resource_snapshot, @@ -647,6 +654,7 @@ run_cancel_test(void) }; Lardon3DResourceDecision decision; Lardon3DResourceReservation *reservation; + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resource_snapshot)); CHECK(lardon3d_resource_governor_reserve( governor, &resource_snapshot, @@ -715,4 +723,4 @@ main(void) return EXIT_FAILURE; } return EXIT_SUCCESS; -} \ No newline at end of file +} diff --git a/tests/test_task.c b/tests/test_task.c index 8414ad6..0990b94 100644 --- a/tests/test_task.c +++ b/tests/test_task.c @@ -7,6 +7,8 @@ #include +#include "resource_snapshot_test_utils.h" + #define CHECK(condition) \ do { \ if (!(condition)) { \ @@ -161,6 +163,7 @@ run_test(void) CHECK(strcmp(snapshot.name, "Tâche de test") == 0); pthread_t thread; + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resource_snapshot)); CHECK(lardon3d_resource_governor_reserve( governor, &resource_snapshot, &estimate, &decision, &reservation )); @@ -199,6 +202,7 @@ run_test(void) FinishProbe cancelled_probe = {.governor = governor}; CHECK(lardon3d_task_set_finished_callback(task, finished_callback, &cancelled_probe)); + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resource_snapshot)); CHECK(lardon3d_resource_governor_reserve( governor, &resource_snapshot, &estimate, &decision, &reservation )); @@ -222,6 +226,7 @@ run_test(void) FinishProbe failed_probe = {.governor = governor}; CHECK(lardon3d_task_set_finished_callback(task, finished_callback, &failed_probe)); + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resource_snapshot)); CHECK(lardon3d_resource_governor_reserve( governor, &resource_snapshot, &estimate, &decision, &reservation )); diff --git a/tests/test_task_checkpoint.c b/tests/test_task_checkpoint.c index 7d9f029..d9e1eb7 100644 --- a/tests/test_task_checkpoint.c +++ b/tests/test_task_checkpoint.c @@ -10,6 +10,8 @@ #include +#include "resource_snapshot_test_utils.h" + #define CHECK(condition) \ do { \ if (!(condition)) { \ @@ -355,6 +357,7 @@ run_test(void) &running ); CHECK(governor && running_task && lardon3d_task_assign_id(running_task, 51)); + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resources)); CHECK(lardon3d_resource_governor_reserve( governor, &resources, &estimate, &decision, &reservation )); @@ -374,6 +377,7 @@ run_test(void) ); CHECK(restarted_task); reservation = NULL; + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resources)); CHECK(lardon3d_resource_governor_reserve( governor, &resources, &estimate, &decision, &reservation )); @@ -393,6 +397,7 @@ run_test(void) ); CHECK(sequence_task && lardon3d_task_assign_id(sequence_task, 52)); reservation = NULL; + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resources)); CHECK(lardon3d_resource_governor_reserve( governor, &resources, &estimate, &decision, &reservation )); diff --git a/tests/test_task_kind_registry.c b/tests/test_task_kind_registry.c index 68c88fd..d6e5319 100644 --- a/tests/test_task_kind_registry.c +++ b/tests/test_task_kind_registry.c @@ -6,6 +6,8 @@ #include +#include "resource_snapshot_test_utils.h" + #define CHECK(condition) do { if (!(condition)) { \ (void)fprintf(stderr, "Échec ligne %d : %s\n", __LINE__, #condition); return false; \ } } while (0) @@ -162,6 +164,7 @@ run_test(void) }; Lardon3DResourceDecision decision; Lardon3DResourceReservation *reservation = NULL; + CHECK(lardon3d_test_resource_snapshot_make_fresh(&resources)); CHECK(governor && lardon3d_resource_governor_reserve( governor, &resources, &durable.estimate, &decision, &reservation)); CHECK(lardon3d_task_start(task, governor, reservation)); diff --git a/tests/test_task_queue.c b/tests/test_task_queue.c index 1d72c65..89c7411 100644 --- a/tests/test_task_queue.c +++ b/tests/test_task_queue.c @@ -6,6 +6,9 @@ #include +#include "../src/resource_governor_internal.h" +#include "resource_snapshot_test_utils.h" + #define CHECK(condition) \ do { \ if (!(condition)) { \ @@ -122,7 +125,7 @@ hold_resources( Lardon3DResourceReservation **reservation ) { - const Lardon3DResourceSnapshot blocking_snapshot = { + Lardon3DResourceSnapshot blocking_snapshot = { .memory_available_bytes = UINT64_MAX, .cpu_load_1m = 0.0, }; @@ -132,7 +135,8 @@ hold_resources( .desired_cpu_threads = 1024, }; Lardon3DResourceDecision decision; - return lardon3d_resource_governor_reserve( + return lardon3d_test_resource_snapshot_make_fresh(&blocking_snapshot) + && lardon3d_resource_governor_reserve( governor, &blocking_snapshot, &blocking_estimate, @@ -322,6 +326,7 @@ run_test(void) }; Lardon3DResourceDecision decision; Lardon3DResourceReservation *blocking_reservation; + CHECK(lardon3d_test_resource_snapshot_make_fresh(&blocking_snapshot)); CHECK(lardon3d_resource_governor_reserve( governor, &blocking_snapshot, @@ -365,11 +370,31 @@ run_test(void) short_pause(); CHECK(!awakened.contract_seen); CHECK(lardon3d_resource_governor_release(governor, blocking_reservation)); - lardon3d_task_queue_resources_changed(queue); CHECK(wait_terminal(queue, awakened_id, &snapshot)); CHECK(snapshot.state == TASK_COMPLETED); CHECK(awakened.contract_seen); CHECK(lardon3d_resource_governor_reservation_count(governor) == 0); + + QueueWork capture_failed = {.log = &wait_log, .value = 3, .steps = 1}; + Lardon3DTask *capture_failed_task = lardon3d_task_create( + "Échec capture", + &estimate, + queue_callback, + &capture_failed + ); + uint64_t capture_failed_id; + CHECK(capture_failed_task); + lardon3d_resource_governor_internal_force_capture_failure(governor, true); + CHECK(lardon3d_task_queue_add( + queue, + capture_failed_task, + &capture_failed_id + )); + CHECK(wait_terminal(queue, capture_failed_id, &snapshot)); + CHECK(snapshot.state == TASK_FAILED); + CHECK(!capture_failed.contract_seen); + CHECK(lardon3d_resource_governor_reservation_count(governor) == 0); + lardon3d_resource_governor_internal_force_capture_failure(governor, false); lardon3d_task_queue_destroy(queue); CHECK(pthread_mutex_destroy(&wait_log.mutex) == 0); @@ -387,6 +412,7 @@ run_test(void) }; Lardon3DResourceDecision io_decision; Lardon3DResourceReservation *io_blocking_reservation; + CHECK(lardon3d_test_resource_snapshot_make_fresh(&io_blocking_snapshot)); CHECK(lardon3d_resource_governor_reserve( governor, &io_blocking_snapshot, @@ -432,7 +458,6 @@ run_test(void) CHECK(snapshot.state == TASK_PENDING); CHECK(!blocked.contract_seen); CHECK(lardon3d_resource_governor_release(governor, io_blocking_reservation)); - lardon3d_task_queue_resources_changed(queue); CHECK(wait_terminal(queue, blocked_id, &snapshot)); CHECK(snapshot.state == TASK_COMPLETED); CHECK(blocked.contract_seen);