diff --git a/README.md b/README.md index cb13c27..f819292 100644 --- a/README.md +++ b/README.md @@ -52,4 +52,6 @@ la file et de ses tâches. Les fondations du moteur de ressources séparent la détection matérielle, les instantanés de disponibilité et les décisions du gouverneur. Le gouverneur conserve des marges système et GPU, puis autorise, diffère, réduit ou refuse un -lot sans dépendre du scheduler ni d'un traitement métier. +lot sans dépendre du scheduler ni d'un traitement métier. Les admissions +acceptées créent désormais une réservation atomique ; l'écran `F6` affiche les +budgets engagés et restants. diff --git a/docs/architecture/resource_governor.md b/docs/architecture/resource_governor.md new file mode 100644 index 0000000..95dfcc9 --- /dev/null +++ b/docs/architecture/resource_governor.md @@ -0,0 +1,128 @@ +# Gouverneur de ressources + +## Rôle + +Le gouverneur protège la réactivité de Lardon3D et la stabilité du système. Les +étapes de photogrammétrie peuvent durer plusieurs heures et manipuler des jeux +de données plus grands que la mémoire disponible. Autoriser chaque tâche à +choisir seule ses ressources conduirait à des pointes de RAM, à la saturation de +la mémoire GPU partagée et à une concurrence incontrôlée sur le stockage. + +Le scheduler et le gouverneur ont donc des responsabilités distinctes. Le +gouverneur décide quelles ressources peuvent être promises. Le scheduler décide +quand et sur quel worker exécuter le travail autorisé. Le scheduler ne doit pas +interpréter lui-même la RAM disponible, la charge CPU ou la pression IO. + +## Contrat d'exécution + +Le chemin d'admission est le suivant : + +```text +Estimate + ↓ +Governor + ↓ +Reservation + ↓ +Scheduler + ↓ +Worker +``` + +Une décision seule est une observation périssable. Deux threads pourraient +observer le même budget et tous deux démarrer, alors que leur consommation +cumulée dépasse la capacité. Une réservation résout cette course : le calcul du +lot et l'inscription dans les budgets sont effectués atomiquement sous le mutex +du gouverneur. Seule une réservation active constitue une autorisation +d'exécution. + +`Lardon3DResourceEstimate` décrit la demande sans être modifié par le +gouverneur : coûts fixes, coûts par élément, bornes du lot, threads et slots +souhaités, et classe de tâche. `Lardon3DResourceReservation` est opaque. Son +instantané public décrit exactement le contrat retenu, y compris le lot réduit. + +## Cycle de vie + +1. Le producteur construit une estimation pour une unité de pipeline. +2. Un instantané des ressources dynamiques est capturé. +3. Le gouverneur compare la demande aux marges, aux ressources disponibles et + aux réservations actives. +4. Une demande impossible est refusée. Une pénurie temporaire demande + d'attendre. Une demande admissible produit une réservation, éventuellement + avec un lot ou des slots réduits. +5. Le scheduler vérifie que la réservation est toujours active avant de confier + la tâche à un worker. +6. Le worker respecte strictement le lot, les threads et les slots réservés. +7. À la fin, après annulation ou après échec, la réservation est libérée une + seule fois. Les budgets redeviennent immédiatement disponibles. +8. Le gouverneur détruit les objets de réservation restants lors de son propre + arrêt. Les pointeurs de réservation ne doivent plus être utilisés ensuite. + +Une seconde libération est refusée sans modifier les compteurs. Les +réservations libérées sont conservées comme tombstones jusqu'à la destruction +du gouverneur afin de détecter cette erreur de cycle de vie sans accès mémoire +invalide. + +## Budgets + +### RAM + +Le budget part de la mémoire actuellement disponible, bornée par la mémoire +physique détectée, puis retire la marge système et toutes les réservations +actives. Les coûts fixes et par élément sont comptabilisés avec contrôle des +débordements. Pour un GPU à mémoire partagée, les besoins RAM et GPU sont +additionnés dans cette même enveloppe. + +### GPU + +Lorsque la VRAM dédiée est mesurable, sa marge et ses réservations ont un budget +distinct. Si sa disponibilité est inconnue, une tâche qui en dépend attend au +lieu de supposer une capacité. Les slots GPU limitent aussi le nombre +d'opérations concurrentes, indépendamment du nombre d'octets annoncé. + +### CPU + +Une partie des CPU logiques reste réservée au système et à la TUI. Le gouverneur +borne les threads souhaités par les threads encore disponibles. La charge +moyenne peut différer une nouvelle admission sans interrompre les travaux déjà +réservés. + +### IO + +Les slots IO limitent les opérations lourdes concurrentes. La pression PSI +Linux peut différer une tâche afin de conserver un terminal réactif et d'éviter +une file d'attente disque excessive. + +### Swap et zram + +L'instantané mesure le swap disponible, y compris la zram exposée comme swap +par Linux. Ce volume est un signal de sécurité et non une extension normale du +budget RAM : le gouverneur ne doit pas dimensionner un lot en comptant sur le +swap. Une future politique pourra utiliser sa baisse pour suspendre les +admissions ou réduire davantage les lots. + +## Lots adaptatifs + +Le gouverneur calcule le plus grand lot compris entre les bornes de +l'estimation et compatible avec tous les budgets restants. Une demande de 128 +éléments peut ainsi produire un contrat de 37 éléments. Le scheduler répétera +le travail sur plusieurs lots, avec écriture des résultats et libération de la +mémoire entre chaque lot. Cette règle privilégie une progression régulière à +une allocation monolithique. + +Les coûts fixes ne sont payés qu'une fois par lot ; les coûts par élément +déterminent sa capacité maximale. Les slots et threads recommandés font partie +du même contrat et ne doivent pas être augmentés par le worker. + +## Utilisation future + +Le même protocole s'applique aux imports, miniatures et extractions EXIF, puis +aux features, au matching, au SfM, aux depth maps, au mesh et aux textures. Les +estimations pourront différer par classe sans déplacer les décisions dans le +scheduler. Le viewer Vulkan utilisera également une réservation GPU afin de ne +pas concurrencer silencieusement une reconstruction sur une machine à mémoire +partagée. + +Cette séparation permettra ultérieurement plusieurs workers : chacun recevra +un contrat déjà arbitré, tandis que le gouverneur restera l'unique propriétaire +des budgets globaux. diff --git a/include/lardon3d/app_state.h b/include/lardon3d/app_state.h index 740d483..305bc93 100644 --- a/include/lardon3d/app_state.h +++ b/include/lardon3d/app_state.h @@ -3,9 +3,11 @@ #include #include +#include typedef struct Lardon3DImageCatalog Lardon3DImageCatalog; typedef struct Lardon3DImageView Lardon3DImageView; typedef struct Lardon3DTaskQueue Lardon3DTaskQueue; +typedef struct Lardon3DResourceGovernor Lardon3DResourceGovernor; typedef enum { LARDON3D_SCREEN_HOME = 0, @@ -13,7 +15,8 @@ typedef enum { LARDON3D_SCREEN_IMPORT, LARDON3D_SCREEN_VIEWER, LARDON3D_SCREEN_HELP, - LARDON3D_SCREEN_TASKS + LARDON3D_SCREEN_TASKS, + LARDON3D_SCREEN_RESOURCES } Lardon3DScreen; typedef struct { @@ -26,6 +29,8 @@ typedef struct { Lardon3DImageCatalog *image_catalog; Lardon3DImageView *image_view; Lardon3DTaskQueue *task_queue; + Lardon3DHardwareProfile hardware_profile; + Lardon3DResourceGovernor *resource_governor; } Lardon3DAppState; void lardon3d_app_state_init(Lardon3DAppState *state); diff --git a/include/lardon3d/layout.h b/include/lardon3d/layout.h index 358e55d..ff5f241 100644 --- a/include/lardon3d/layout.h +++ b/include/lardon3d/layout.h @@ -4,6 +4,7 @@ #include #include #include +#include void lardon3d_layout_draw( const Lardon3DAppState *state, @@ -13,6 +14,7 @@ void lardon3d_layout_draw( const Lardon3DTaskSnapshot *task_snapshots, size_t task_count, const Lardon3DTaskQueueSummary *task_summary, + const Lardon3DResourceAvailability *resource_availability, int rows, int cols ); diff --git a/include/lardon3d/resource_governor.h b/include/lardon3d/resource_governor.h index 9482f88..0b51d72 100644 --- a/include/lardon3d/resource_governor.h +++ b/include/lardon3d/resource_governor.h @@ -4,6 +4,7 @@ #include #include #include +#include #include #include @@ -13,6 +14,7 @@ enum { }; typedef struct Lardon3DResourceGovernor Lardon3DResourceGovernor; +typedef struct Lardon3DResourceReservation Lardon3DResourceReservation; typedef struct { uint64_t system_memory_reserve_bytes; @@ -20,8 +22,32 @@ typedef struct { unsigned int system_cpu_reserve; double maximum_cpu_load_ratio; double maximum_io_pressure_avg10; + unsigned int gpu_slot_capacity; + unsigned int io_slot_capacity; } Lardon3DResourcePolicy; +typedef enum { + LARDON3D_RESOURCE_TASK_GENERAL = 0, + LARDON3D_RESOURCE_TASK_IMPORT, + LARDON3D_RESOURCE_TASK_CPU, + LARDON3D_RESOURCE_TASK_GPU, + LARDON3D_RESOURCE_TASK_IO, + LARDON3D_RESOURCE_TASK_MIXED +} Lardon3DResourceTaskClass; + +typedef struct { + uint64_t memory_fixed_bytes; + uint64_t gpu_memory_fixed_bytes; + uint64_t memory_bytes_per_item; + uint64_t gpu_memory_bytes_per_item; + size_t minimum_batch_size; + size_t maximum_batch_size; + unsigned int desired_cpu_threads; + unsigned int desired_gpu_slots; + unsigned int desired_io_slots; + Lardon3DResourceTaskClass task_class; +} Lardon3DResourceEstimate; + typedef struct { uint64_t memory_bytes_per_item; uint64_t gpu_memory_bytes_per_item; @@ -42,9 +68,49 @@ typedef struct { Lardon3DResourceDecisionKind kind; size_t batch_size; unsigned int cpu_threads; + unsigned int gpu_slots; + unsigned int io_slots; char reason[LARDON3D_RESOURCE_REASON_CAPACITY]; } Lardon3DResourceDecision; +typedef enum { + LARDON3D_RESERVATION_ACTIVE = 0, + LARDON3D_RESERVATION_RELEASED +} Lardon3DResourceReservationState; + +typedef struct { + uint64_t id; + uint64_t memory_bytes; + uint64_t gpu_memory_bytes; + unsigned int cpu_threads; + unsigned int gpu_slots; + unsigned int io_slots; + size_t batch_size; + Lardon3DResourceTaskClass task_class; + Lardon3DResourceReservationState state; + struct timespec created_at; +} Lardon3DResourceReservationInfo; + +typedef struct { + uint64_t memory_budget_bytes; + uint64_t memory_reserved_bytes; + uint64_t memory_available_bytes; + bool gpu_memory_known; + uint64_t gpu_memory_budget_bytes; + uint64_t gpu_memory_reserved_bytes; + uint64_t gpu_memory_available_bytes; + unsigned int cpu_budget; + unsigned int cpu_reserved; + unsigned int cpu_available; + unsigned int gpu_slot_budget; + unsigned int gpu_slots_reserved; + unsigned int gpu_slots_available; + unsigned int io_slot_budget; + unsigned int io_slots_reserved; + unsigned int io_slots_available; + size_t active_reservations; +} Lardon3DResourceAvailability; + bool lardon3d_resource_policy_default( const Lardon3DHardwareProfile *profile, Lardon3DResourcePolicy *policy @@ -66,6 +132,39 @@ bool lardon3d_resource_governor_decide( const Lardon3DResourceRequest *request, Lardon3DResourceDecision *decision ); +bool lardon3d_resource_governor_reserve( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceSnapshot *snapshot, + const Lardon3DResourceEstimate *estimate, + Lardon3DResourceDecision *decision, + Lardon3DResourceReservation **reservation +); +bool lardon3d_resource_governor_release( + Lardon3DResourceGovernor *governor, + Lardon3DResourceReservation *reservation +); +bool lardon3d_resource_governor_reservation_is_valid( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceReservation *reservation +); +bool lardon3d_resource_reservation_get( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceReservation *reservation, + Lardon3DResourceReservationInfo *information +); +size_t lardon3d_resource_governor_list_reservations( + Lardon3DResourceGovernor *governor, + Lardon3DResourceReservationInfo *reservations, + size_t capacity +); +size_t lardon3d_resource_governor_reservation_count( + Lardon3DResourceGovernor *governor +); +bool lardon3d_resource_governor_availability( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceSnapshot *snapshot, + Lardon3DResourceAvailability *availability +); const char *lardon3d_resource_decision_name( Lardon3DResourceDecisionKind kind ); diff --git a/meson.build b/meson.build index 73aa5b1..50b4ca6 100644 --- a/meson.build +++ b/meson.build @@ -154,3 +154,15 @@ resource_governor_test = executable( ) test('resource-governor', resource_governor_test, timeout: 30) + +resource_reservation_test = executable( + 'test-resource-reservation', + sources: [ + 'tests/test_resource_reservation.c', + 'src/resource_governor.c', + ], + include_directories: include_directories('include'), + dependencies: [threads], +) + +test('resource-reservation', resource_reservation_test, timeout: 60) diff --git a/src/app.c b/src/app.c index e013383..3db2a1e 100644 --- a/src/app.c +++ b/src/app.c @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -17,13 +18,35 @@ lardon3d_app_run(void) if (!setlocale(LC_ALL, "")) { return EXIT_FAILURE; } + char error[256]; + Lardon3DResourcePolicy resource_policy; + if (!lardon3d_hardware_profile_detect( + &state.hardware_profile, + error, + sizeof(error) + ) + || !lardon3d_resource_policy_default( + &state.hardware_profile, + &resource_policy + )) { + return EXIT_FAILURE; + } + state.resource_governor = lardon3d_resource_governor_create( + &state.hardware_profile, + &resource_policy + ); + if (!state.resource_governor) { + return EXIT_FAILURE; + } state.task_queue = lardon3d_task_queue_create(); if (!state.task_queue) { + lardon3d_resource_governor_destroy(state.resource_governor); return EXIT_FAILURE; } if (!lardon3d_tui_init()) { lardon3d_task_queue_destroy(state.task_queue); + lardon3d_resource_governor_destroy(state.resource_governor); return EXIT_FAILURE; } @@ -32,6 +55,7 @@ lardon3d_app_run(void) lardon3d_image_catalog_destroy(state.image_catalog); lardon3d_tui_shutdown(); lardon3d_task_queue_destroy(state.task_queue); + lardon3d_resource_governor_destroy(state.resource_governor); return success ? EXIT_SUCCESS : EXIT_FAILURE; } diff --git a/src/layout.c b/src/layout.c index 82345ad..9c644cd 100644 --- a/src/layout.c +++ b/src/layout.c @@ -61,7 +61,7 @@ screen_texts( const char **footer ) { - *footer = "ESC Accueil F1 Aide F2 Projets F3 Import F4 Viewer F5 Tâches Q"; + *footer = "ESC Accueil F1 Aide F2 Projets F3 Import F4 Viewer F5 Tâches F6 Ressources Q"; switch (screen) { case LARDON3D_SCREEN_PROJECTS: @@ -87,11 +87,16 @@ screen_texts( *content = "Tâches en arrière-plan"; *footer = "F5 Tâches ESC Accueil Q Quit"; break; + case LARDON3D_SCREEN_RESOURCES: + *title = "Ressources"; + *content = "Gouverneur de ressources"; + *footer = "F6 Ressources ESC Accueil Q Quit"; + break; case LARDON3D_SCREEN_HOME: default: *title = "Accueil"; *content = "Bienvenue dans Lardon3D"; - *footer = "F1 Aide F2 Projets F3 Import F4 Viewer F5 Tâches Q Quit"; + *footer = "F1 Aide F2 Projets F3 Import F4 Viewer F5 Tâches F6 Ressources Q"; break; } } @@ -344,6 +349,104 @@ draw_tasks( } } +static void +draw_resource_line( + int row, + int columns, + const char *label, + uint64_t reserved, + uint64_t available +) +{ + char reserved_text[64]; + char available_text[64]; + char line[256]; + lardon3d_image_catalog_format_size( + reserved, + reserved_text, + sizeof(reserved_text) + ); + lardon3d_image_catalog_format_size( + available, + available_text, + sizeof(available_text) + ); + (void)snprintf( + line, + sizeof(line), + "%s réservée : %s Restante : %s", + label, + reserved_text, + available_text + ); + draw_text(row, 4, columns - 6, line); +} + +static void +draw_resources( + const Lardon3DResourceAvailability *availability, + int columns +) +{ + if (!availability) { + draw_text(7, 4, columns - 6, "Ressources système indisponibles."); + return; + } + char line[256]; + (void)snprintf( + line, + sizeof(line), + "Réservations actives : %zu", + availability->active_reservations + ); + draw_text(6, 4, columns - 6, line); + draw_resource_line( + 8, + columns, + "RAM", + availability->memory_reserved_bytes, + availability->memory_available_bytes + ); + if (availability->gpu_memory_known) { + draw_resource_line( + 9, + columns, + "GPU", + availability->gpu_memory_reserved_bytes, + availability->gpu_memory_available_bytes + ); + } else { + draw_text(9, 4, columns - 6, "Mémoire GPU disponible : inconnue"); + } + (void)snprintf( + line, + sizeof(line), + "CPU réservés : %u Restants : %u / %u", + availability->cpu_reserved, + availability->cpu_available, + availability->cpu_budget + ); + draw_text(10, 4, columns - 6, line); + (void)snprintf( + line, + sizeof(line), + "Slots GPU réservés : %u Restants : %u / %u", + availability->gpu_slots_reserved, + availability->gpu_slots_available, + availability->gpu_slot_budget + ); + draw_text(11, 4, columns - 6, line); + (void)snprintf( + line, + sizeof(line), + "Slots IO réservés : %u Restants : %u / %u", + availability->io_slots_reserved, + availability->io_slots_available, + availability->io_slot_budget + ); + draw_text(12, 4, columns - 6, line); +} + static void draw_content( const Lardon3DAppState *state, @@ -353,6 +456,7 @@ draw_content( const Lardon3DTaskSnapshot *task_snapshots, size_t task_count, const Lardon3DTaskQueueSummary *task_summary, + const Lardon3DResourceAvailability *resource_availability, int rows, int columns ) @@ -402,6 +506,8 @@ draw_content( rows, columns ); + } else if (state->screen == LARDON3D_SCREEN_RESOURCES) { + draw_resources(resource_availability, columns); } else { draw_text( (3 + journal_row) / 2, @@ -429,6 +535,7 @@ lardon3d_layout_draw( const Lardon3DTaskSnapshot *task_snapshots, size_t task_count, const Lardon3DTaskQueueSummary *task_summary, + const Lardon3DResourceAvailability *resource_availability, int rows, int columns ) @@ -447,6 +554,7 @@ lardon3d_layout_draw( task_snapshots, task_count, task_summary, + resource_availability, rows, columns ); diff --git a/src/resource_governor.c b/src/resource_governor.c index 59f970e..e556448 100644 --- a/src/resource_governor.c +++ b/src/resource_governor.c @@ -5,10 +5,25 @@ #include +struct Lardon3DResourceReservation { + Lardon3DResourceReservationInfo information; + uint64_t charged_memory_bytes; + struct Lardon3DResourceReservation *next; +}; + struct Lardon3DResourceGovernor { pthread_mutex_t mutex; Lardon3DHardwareProfile profile; Lardon3DResourcePolicy policy; + uint64_t next_reservation_id; + uint64_t memory_reserved_bytes; + uint64_t gpu_memory_reserved_bytes; + unsigned int cpu_reserved; + unsigned int gpu_slots_reserved; + unsigned int io_slots_reserved; + size_t active_count; + Lardon3DResourceReservation *active; + Lardon3DResourceReservation *released; }; static bool @@ -34,17 +49,107 @@ valid_policy( && policy->maximum_cpu_load_ratio <= 1.0 && policy->maximum_io_pressure_avg10 >= 0.0 && policy->maximum_io_pressure_avg10 <= 100.0 + && policy->io_slot_capacity > 0 + && (!profile->gpu_available || policy->gpu_slot_capacity > 0) && (!profile->gpu_memory_known || policy->gpu_memory_reserve_bytes < profile->gpu_memory_total_bytes); } +static bool +valid_snapshot( + const Lardon3DHardwareProfile *profile, + const Lardon3DResourceSnapshot *snapshot +) +{ + return snapshot && snapshot->cpu_load_1m >= 0.0 + && (!snapshot->io_pressure_known + || (snapshot->io_pressure_avg10 >= 0.0 + && snapshot->io_pressure_avg10 <= 100.0)) + && (!snapshot->gpu_memory_available_known + || !profile->gpu_memory_known + || profile->gpu_uses_shared_memory + || snapshot->gpu_memory_available_bytes + <= profile->gpu_memory_total_bytes); +} + +static bool +valid_estimate(const Lardon3DResourceEstimate *estimate) +{ + return estimate && estimate->minimum_batch_size > 0 + && estimate->maximum_batch_size >= estimate->minimum_batch_size + && estimate->desired_cpu_threads > 0 + && estimate->task_class >= LARDON3D_RESOURCE_TASK_GENERAL + && estimate->task_class <= LARDON3D_RESOURCE_TASK_MIXED + && ((estimate->gpu_memory_fixed_bytes == 0 + && estimate->gpu_memory_bytes_per_item == 0) + || estimate->desired_gpu_slots > 0); +} + +static uint64_t +subtract_floor(uint64_t value, uint64_t amount) +{ + return value > amount ? value - amount : 0; +} + +static unsigned int +subtract_unsigned(unsigned int value, unsigned int amount) +{ + return value > amount ? value - amount : 0; +} + +static uint64_t +minimum_uint64(uint64_t left, uint64_t right) +{ + return left < right ? left : right; +} + +static size_t +minimum_size(size_t left, size_t right) +{ + return left < right ? left : right; +} + +static bool +resource_size( + uint64_t fixed, + uint64_t per_item, + size_t batch, + uint64_t *total +) +{ + if (per_item > 0 && batch > UINT64_MAX / per_item) { + return false; + } + uint64_t variable = per_item * (uint64_t)batch; + if (fixed > UINT64_MAX - variable) { + return false; + } + *total = fixed + variable; + return true; +} + +static size_t +batch_capacity(uint64_t available, uint64_t fixed, uint64_t per_item) +{ + if (available < fixed) { + return 0; + } + if (per_item == 0) { + return SIZE_MAX; + } + uint64_t capacity = (available - fixed) / per_item; + return capacity > SIZE_MAX ? SIZE_MAX : (size_t)capacity; +} + static void set_decision( Lardon3DResourceDecision *decision, Lardon3DResourceDecisionKind kind, size_t batch_size, unsigned int cpu_threads, + unsigned int gpu_slots, + unsigned int io_slots, const char *reason ) { @@ -52,6 +157,8 @@ set_decision( .kind = kind, .batch_size = batch_size, .cpu_threads = cpu_threads, + .gpu_slots = gpu_slots, + .io_slots = io_slots, }; (void)snprintf(decision->reason, sizeof(decision->reason), "%s", reason); } @@ -73,6 +180,8 @@ lardon3d_resource_policy_default( .system_cpu_reserve = profile->logical_cpu_count > 2 ? 1 : 0, .maximum_cpu_load_ratio = 0.90, .maximum_io_pressure_avg10 = 80.0, + .gpu_slot_capacity = profile->gpu_available ? 1 : 0, + .io_slot_capacity = 1, }; return valid_policy(profile, policy); } @@ -96,6 +205,7 @@ lardon3d_resource_governor_create( } governor->profile = *profile; governor->policy = *policy; + governor->next_reservation_id = 1; return governor; } @@ -105,6 +215,18 @@ lardon3d_resource_governor_destroy(Lardon3DResourceGovernor *governor) if (!governor) { return; } + Lardon3DResourceReservation *reservation = governor->active; + while (reservation) { + Lardon3DResourceReservation *next = reservation->next; + free(reservation); + reservation = next; + } + reservation = governor->released; + while (reservation) { + Lardon3DResourceReservation *next = reservation->next; + free(reservation); + reservation = next; + } (void)pthread_mutex_destroy(&governor->mutex); free(governor); } @@ -119,7 +241,23 @@ lardon3d_resource_governor_set_policy( return false; } (void)pthread_mutex_lock(&governor->mutex); - bool accepted = valid_policy(&governor->profile, policy); + bool policy_valid = valid_policy(&governor->profile, policy); + uint64_t memory_budget = policy_valid + ? governor->profile.memory_total_bytes + - policy->system_memory_reserve_bytes + : 0; + uint64_t gpu_budget = policy_valid && governor->profile.gpu_memory_known + && !governor->profile.gpu_uses_shared_memory + ? governor->profile.gpu_memory_total_bytes + - policy->gpu_memory_reserve_bytes + : UINT64_MAX; + bool accepted = policy_valid + && memory_budget >= governor->memory_reserved_bytes + && gpu_budget >= governor->gpu_memory_reserved_bytes + && governor->profile.logical_cpu_count - policy->system_cpu_reserve + >= governor->cpu_reserved + && policy->gpu_slot_capacity >= governor->gpu_slots_reserved + && policy->io_slot_capacity >= governor->io_slots_reserved; if (accepted) { governor->policy = *policy; } @@ -127,23 +265,426 @@ lardon3d_resource_governor_set_policy( return accepted; } -static size_t -capacity_for(uint64_t available, uint64_t reserve, uint64_t per_item) +static bool +availability_locked( + const Lardon3DResourceGovernor *governor, + const Lardon3DResourceSnapshot *snapshot, + Lardon3DResourceAvailability *availability +) { - if (per_item == 0) { - return SIZE_MAX; + if (!valid_snapshot(&governor->profile, snapshot)) { + return false; } - if (available <= reserve) { - return 0; + uint64_t detected_memory = minimum_uint64( + snapshot->memory_available_bytes, + governor->profile.memory_total_bytes + ); + uint64_t memory_budget = subtract_floor( + detected_memory, + governor->policy.system_memory_reserve_bytes + ); + uint64_t gpu_budget = 0; + bool gpu_known = false; + if (governor->profile.gpu_uses_shared_memory) { + gpu_known = true; + gpu_budget = memory_budget; + } else if (governor->profile.gpu_memory_known + && snapshot->gpu_memory_available_known) { + gpu_known = true; + gpu_budget = subtract_floor( + snapshot->gpu_memory_available_bytes, + governor->policy.gpu_memory_reserve_bytes + ); } - uint64_t capacity = (available - reserve) / per_item; - return capacity > SIZE_MAX ? SIZE_MAX : (size_t)capacity; + unsigned int cpu_budget = governor->profile.logical_cpu_count + - governor->policy.system_cpu_reserve; + *availability = (Lardon3DResourceAvailability) { + .memory_budget_bytes = memory_budget, + .memory_reserved_bytes = governor->memory_reserved_bytes, + .memory_available_bytes = subtract_floor( + memory_budget, + governor->memory_reserved_bytes + ), + .gpu_memory_known = gpu_known, + .gpu_memory_budget_bytes = gpu_budget, + .gpu_memory_reserved_bytes = governor->gpu_memory_reserved_bytes, + .gpu_memory_available_bytes = governor->profile.gpu_uses_shared_memory + ? subtract_floor(memory_budget, governor->memory_reserved_bytes) + : subtract_floor(gpu_budget, governor->gpu_memory_reserved_bytes), + .cpu_budget = cpu_budget, + .cpu_reserved = governor->cpu_reserved, + .cpu_available = subtract_unsigned(cpu_budget, governor->cpu_reserved), + .gpu_slot_budget = governor->policy.gpu_slot_capacity, + .gpu_slots_reserved = governor->gpu_slots_reserved, + .gpu_slots_available = subtract_unsigned( + governor->policy.gpu_slot_capacity, + governor->gpu_slots_reserved + ), + .io_slot_budget = governor->policy.io_slot_capacity, + .io_slots_reserved = governor->io_slots_reserved, + .io_slots_available = subtract_unsigned( + governor->policy.io_slot_capacity, + governor->io_slots_reserved + ), + .active_reservations = governor->active_count, + }; + return true; } -static size_t -minimum_size(size_t left, size_t right) +bool +lardon3d_resource_governor_availability( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceSnapshot *snapshot, + Lardon3DResourceAvailability *availability +) { - return left < right ? left : right; + if (!governor || !snapshot || !availability) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + bool success = availability_locked(governor, snapshot, availability); + (void)pthread_mutex_unlock(&governor->mutex); + return success; +} + +static void +evaluate_locked( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceSnapshot *snapshot, + const Lardon3DResourceEstimate *estimate, + Lardon3DResourceDecision *decision +) +{ + if (!valid_estimate(estimate)) { + set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Estimation de ressources invalide."); + return; + } + if ((estimate->desired_gpu_slots > 0 + || estimate->gpu_memory_fixed_bytes > 0 + || estimate->gpu_memory_bytes_per_item > 0) + && !governor->profile.gpu_available) { + set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Aucun GPU disponible."); + return; + } + + uint64_t memory_fixed = estimate->memory_fixed_bytes; + uint64_t memory_per_item = estimate->memory_bytes_per_item; + if (governor->profile.gpu_uses_shared_memory) { + if (memory_fixed > UINT64_MAX - estimate->gpu_memory_fixed_bytes + || memory_per_item + > UINT64_MAX - estimate->gpu_memory_bytes_per_item) { + set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Estimation mémoire trop grande."); + return; + } + memory_fixed += estimate->gpu_memory_fixed_bytes; + memory_per_item += estimate->gpu_memory_bytes_per_item; + } + + size_t theoretical_batch = batch_capacity( + governor->profile.memory_total_bytes + - governor->policy.system_memory_reserve_bytes, + memory_fixed, + memory_per_item + ); + if (!governor->profile.gpu_uses_shared_memory + && (estimate->gpu_memory_fixed_bytes > 0 + || estimate->gpu_memory_bytes_per_item > 0) + && governor->profile.gpu_memory_known) { + theoretical_batch = minimum_size( + theoretical_batch, + batch_capacity( + governor->profile.gpu_memory_total_bytes + - governor->policy.gpu_memory_reserve_bytes, + estimate->gpu_memory_fixed_bytes, + estimate->gpu_memory_bytes_per_item + ) + ); + } + if (theoretical_batch < estimate->minimum_batch_size) { + set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Tâche impossible sur cette machine."); + return; + } + + double load_limit = (double)governor->profile.logical_cpu_count + * governor->policy.maximum_cpu_load_ratio; + if (snapshot->cpu_load_1m >= load_limit) { + set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Charge CPU trop élevée."); + return; + } + if (estimate->desired_io_slots > 0 && snapshot->io_pressure_known + && snapshot->io_pressure_avg10 + >= governor->policy.maximum_io_pressure_avg10) { + set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Pression d'entrée-sortie trop élevée."); + return; + } + + Lardon3DResourceAvailability available; + if (!availability_locked(governor, snapshot, &available)) { + set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Instantané de ressources invalide."); + return; + } + size_t batch = batch_capacity( + available.memory_available_bytes, + memory_fixed, + memory_per_item + ); + if (!governor->profile.gpu_uses_shared_memory + && (estimate->gpu_memory_fixed_bytes > 0 + || estimate->gpu_memory_bytes_per_item > 0)) { + if (!available.gpu_memory_known) { + set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Mémoire GPU disponible inconnue."); + return; + } + batch = minimum_size( + batch, + batch_capacity( + available.gpu_memory_available_bytes, + estimate->gpu_memory_fixed_bytes, + estimate->gpu_memory_bytes_per_item + ) + ); + } + batch = minimum_size(batch, estimate->maximum_batch_size); + if (batch < estimate->minimum_batch_size) { + set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Ressources déjà réservées ou temporairement insuffisantes."); + return; + } + if (available.cpu_available == 0 + || (estimate->desired_gpu_slots > 0 + && available.gpu_slots_available == 0) + || (estimate->desired_io_slots > 0 + && available.io_slots_available == 0)) { + set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Slots de calcul déjà réservés."); + return; + } + unsigned int cpu = estimate->desired_cpu_threads < available.cpu_available + ? estimate->desired_cpu_threads + : available.cpu_available; + unsigned int gpu = estimate->desired_gpu_slots + < available.gpu_slots_available + ? estimate->desired_gpu_slots + : available.gpu_slots_available; + unsigned int io = estimate->desired_io_slots < available.io_slots_available + ? estimate->desired_io_slots + : available.io_slots_available; + bool reduced = batch < estimate->maximum_batch_size + || cpu < estimate->desired_cpu_threads + || gpu < estimate->desired_gpu_slots + || io < estimate->desired_io_slots; + set_decision( + decision, + reduced ? LARDON3D_RESOURCE_REDUCE_BATCH : LARDON3D_RESOURCE_START, + batch, + cpu, + gpu, + io, + reduced ? "Contrat réduit aux ressources disponibles." + : "Ressources disponibles." + ); +} + +bool +lardon3d_resource_governor_reserve( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceSnapshot *snapshot, + const Lardon3DResourceEstimate *estimate, + Lardon3DResourceDecision *decision, + Lardon3DResourceReservation **reservation +) +{ + if (!governor || !snapshot || !estimate || !decision || !reservation) { + return false; + } + *reservation = NULL; + Lardon3DResourceReservation *created = calloc(1, sizeof(*created)); + if (!created) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + if (!valid_snapshot(&governor->profile, snapshot)) { + (void)pthread_mutex_unlock(&governor->mutex); + free(created); + return false; + } + evaluate_locked(governor, snapshot, estimate, decision); + if ((decision->kind != LARDON3D_RESOURCE_START + && decision->kind != LARDON3D_RESOURCE_REDUCE_BATCH) + || governor->next_reservation_id == 0) { + (void)pthread_mutex_unlock(&governor->mutex); + free(created); + return true; + } + uint64_t memory_bytes; + uint64_t gpu_memory_bytes; + if (!resource_size( + estimate->memory_fixed_bytes, + estimate->memory_bytes_per_item, + decision->batch_size, + &memory_bytes + ) + || !resource_size( + estimate->gpu_memory_fixed_bytes, + estimate->gpu_memory_bytes_per_item, + decision->batch_size, + &gpu_memory_bytes + ) + || clock_gettime(CLOCK_REALTIME, &created->information.created_at) != 0) { + (void)pthread_mutex_unlock(&governor->mutex); + free(created); + return false; + } + created->information.id = governor->next_reservation_id++; + created->information.memory_bytes = memory_bytes; + created->information.gpu_memory_bytes = gpu_memory_bytes; + created->information.cpu_threads = decision->cpu_threads; + created->information.gpu_slots = decision->gpu_slots; + created->information.io_slots = decision->io_slots; + created->information.batch_size = decision->batch_size; + created->information.task_class = estimate->task_class; + created->information.state = LARDON3D_RESERVATION_ACTIVE; + created->charged_memory_bytes = governor->profile.gpu_uses_shared_memory + ? memory_bytes + gpu_memory_bytes + : memory_bytes; + created->next = governor->active; + governor->active = created; + governor->memory_reserved_bytes += created->charged_memory_bytes; + governor->gpu_memory_reserved_bytes += gpu_memory_bytes; + governor->cpu_reserved += decision->cpu_threads; + governor->gpu_slots_reserved += decision->gpu_slots; + governor->io_slots_reserved += decision->io_slots; + ++governor->active_count; + *reservation = created; + (void)pthread_mutex_unlock(&governor->mutex); + return true; +} + +static Lardon3DResourceReservation * +find_reservation( + Lardon3DResourceReservation *head, + const Lardon3DResourceReservation *wanted +) +{ + for (Lardon3DResourceReservation *current = head; + current; + current = current->next) { + if (current == wanted) { + return current; + } + } + return NULL; +} + +bool +lardon3d_resource_governor_release( + Lardon3DResourceGovernor *governor, + Lardon3DResourceReservation *reservation +) +{ + if (!governor || !reservation) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + Lardon3DResourceReservation *previous = NULL; + Lardon3DResourceReservation *current = governor->active; + while (current && current != reservation) { + previous = current; + current = current->next; + } + if (!current) { + (void)pthread_mutex_unlock(&governor->mutex); + return false; + } + if (previous) { + previous->next = current->next; + } else { + governor->active = current->next; + } + governor->memory_reserved_bytes -= current->charged_memory_bytes; + governor->gpu_memory_reserved_bytes -= current->information.gpu_memory_bytes; + governor->cpu_reserved -= current->information.cpu_threads; + governor->gpu_slots_reserved -= current->information.gpu_slots; + governor->io_slots_reserved -= current->information.io_slots; + --governor->active_count; + current->information.state = LARDON3D_RESERVATION_RELEASED; + current->next = governor->released; + governor->released = current; + (void)pthread_mutex_unlock(&governor->mutex); + return true; +} + +bool +lardon3d_resource_governor_reservation_is_valid( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceReservation *reservation +) +{ + if (!governor || !reservation) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + bool valid = find_reservation(governor->active, reservation) != NULL; + (void)pthread_mutex_unlock(&governor->mutex); + return valid; +} + +bool +lardon3d_resource_reservation_get( + Lardon3DResourceGovernor *governor, + const Lardon3DResourceReservation *reservation, + Lardon3DResourceReservationInfo *information +) +{ + if (!governor || !reservation || !information) { + return false; + } + (void)pthread_mutex_lock(&governor->mutex); + Lardon3DResourceReservation *found = find_reservation( + governor->active, + reservation + ); + if (!found) { + found = find_reservation(governor->released, reservation); + } + if (found) { + *information = found->information; + } + (void)pthread_mutex_unlock(&governor->mutex); + return found != NULL; +} + +size_t +lardon3d_resource_governor_list_reservations( + Lardon3DResourceGovernor *governor, + Lardon3DResourceReservationInfo *reservations, + size_t capacity +) +{ + if (!governor || (!reservations && capacity > 0)) { + return 0; + } + (void)pthread_mutex_lock(&governor->mutex); + size_t count = 0; + for (Lardon3DResourceReservation *current = governor->active; + current && count < capacity; + current = current->next) { + reservations[count++] = current->information; + } + (void)pthread_mutex_unlock(&governor->mutex); + return count; +} + +size_t +lardon3d_resource_governor_reservation_count( + Lardon3DResourceGovernor *governor +) +{ + if (!governor) { + return 0; + } + (void)pthread_mutex_lock(&governor->mutex); + size_t count = governor->active_count; + (void)pthread_mutex_unlock(&governor->mutex); + return count; } bool @@ -157,116 +698,25 @@ lardon3d_resource_governor_decide( if (!governor || !snapshot || !request || !decision) { return false; } + Lardon3DResourceEstimate estimate = { + .memory_bytes_per_item = request->memory_bytes_per_item, + .gpu_memory_bytes_per_item = request->gpu_memory_bytes_per_item, + .minimum_batch_size = request->minimum_batch_size, + .maximum_batch_size = request->preferred_batch_size, + .desired_cpu_threads = request->requested_cpu_threads, + .desired_gpu_slots = request->gpu_memory_bytes_per_item > 0 ? 1 : 0, + .desired_io_slots = request->io_intensive ? 1 : 0, + .task_class = request->io_intensive + ? LARDON3D_RESOURCE_TASK_IO + : LARDON3D_RESOURCE_TASK_GENERAL, + }; (void)pthread_mutex_lock(&governor->mutex); - const Lardon3DHardwareProfile profile = governor->profile; - const Lardon3DResourcePolicy policy = governor->policy; - (void)pthread_mutex_unlock(&governor->mutex); - - if (!(snapshot->cpu_load_1m >= 0.0) - || (snapshot->io_pressure_known - && (!(snapshot->io_pressure_avg10 >= 0.0) - || snapshot->io_pressure_avg10 > 100.0)) - || (snapshot->gpu_memory_available_known - && profile.gpu_memory_known - && !profile.gpu_uses_shared_memory - && snapshot->gpu_memory_available_bytes - > profile.gpu_memory_total_bytes)) { + if (!valid_snapshot(&governor->profile, snapshot)) { + (void)pthread_mutex_unlock(&governor->mutex); return false; } - - if (request->minimum_batch_size == 0 - || request->preferred_batch_size < request->minimum_batch_size - || request->requested_cpu_threads == 0) { - set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, "Demande de ressources invalide."); - return true; - } - unsigned int available_threads = profile.logical_cpu_count - - policy.system_cpu_reserve; - unsigned int cpu_threads = request->requested_cpu_threads < available_threads - ? request->requested_cpu_threads - : available_threads; - - uint64_t system_memory_per_item = request->memory_bytes_per_item; - if (request->gpu_memory_bytes_per_item > 0 - && profile.gpu_uses_shared_memory) { - if (system_memory_per_item - > UINT64_MAX - request->gpu_memory_bytes_per_item) { - set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, cpu_threads, "Besoin mémoire trop grand."); - return true; - } - system_memory_per_item += request->gpu_memory_bytes_per_item; - } - size_t theoretical_batch = capacity_for( - profile.memory_total_bytes, - policy.system_memory_reserve_bytes, - system_memory_per_item - ); - if (request->gpu_memory_bytes_per_item > 0) { - if (!profile.gpu_available) { - set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, cpu_threads, "Aucun GPU disponible."); - return true; - } - if (profile.gpu_memory_known && !profile.gpu_uses_shared_memory) { - theoretical_batch = minimum_size( - theoretical_batch, - capacity_for( - profile.gpu_memory_total_bytes, - policy.gpu_memory_reserve_bytes, - request->gpu_memory_bytes_per_item - ) - ); - } - } - if (theoretical_batch < request->minimum_batch_size) { - set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, cpu_threads, "Tâche impossible sur cette machine."); - return true; - } - - double load_limit = (double)profile.logical_cpu_count - * policy.maximum_cpu_load_ratio; - if (snapshot->cpu_load_1m >= load_limit) { - set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, cpu_threads, "Charge CPU trop élevée."); - return true; - } - if (request->io_intensive && snapshot->io_pressure_known - && snapshot->io_pressure_avg10 >= policy.maximum_io_pressure_avg10) { - set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, cpu_threads, "Pression d'entrée-sortie trop élevée."); - return true; - } - - size_t current_batch = capacity_for( - snapshot->memory_available_bytes, - policy.system_memory_reserve_bytes, - system_memory_per_item - ); - if (request->gpu_memory_bytes_per_item > 0 - && !profile.gpu_uses_shared_memory) { - if (!snapshot->gpu_memory_available_known) { - set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, cpu_threads, "Mémoire GPU disponible inconnue."); - return true; - } - current_batch = minimum_size( - current_batch, - capacity_for( - snapshot->gpu_memory_available_bytes, - policy.gpu_memory_reserve_bytes, - request->gpu_memory_bytes_per_item - ) - ); - } - if (current_batch < request->minimum_batch_size) { - set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, cpu_threads, "Ressources temporairement insuffisantes."); - return true; - } - size_t batch_size = minimum_size( - current_batch, - request->preferred_batch_size - ); - if (batch_size < request->preferred_batch_size) { - set_decision(decision, LARDON3D_RESOURCE_REDUCE_BATCH, batch_size, cpu_threads, "Taille de lot réduite."); - return true; - } - set_decision(decision, LARDON3D_RESOURCE_START, batch_size, cpu_threads, "Ressources disponibles."); + evaluate_locked(governor, snapshot, &estimate, decision); + (void)pthread_mutex_unlock(&governor->mutex); return true; } diff --git a/src/tui.c b/src/tui.c index 46a83a1..9d44ecd 100644 --- a/src/tui.c +++ b/src/tui.c @@ -10,6 +10,8 @@ #include #include #include +#include +#include #include #include @@ -180,6 +182,23 @@ redraw( DISPLAYED_TASK_CAPACITY, &task_summary ); + Lardon3DResourceSnapshot resource_snapshot; + Lardon3DResourceAvailability resource_availability; + const Lardon3DResourceAvailability *displayed_resources = NULL; + if (state->screen == LARDON3D_SCREEN_RESOURCES + && lardon3d_resource_snapshot_capture( + &state->hardware_profile, + &resource_snapshot, + NULL, + 0 + ) + && lardon3d_resource_governor_availability( + state->resource_governor, + &resource_snapshot, + &resource_availability + )) { + displayed_resources = &resource_availability; + } lardon3d_layout_draw( state, text, @@ -188,6 +207,7 @@ redraw( task_snapshots, task_count, &task_summary, + displayed_resources, rows, columns ); @@ -412,6 +432,9 @@ handle_normal_input( case KEY_F(5): state->screen = LARDON3D_SCREEN_TASKS; return true; + case KEY_F(6): + state->screen = LARDON3D_SCREEN_RESOURCES; + return true; case 27: state->screen = LARDON3D_SCREEN_HOME; return true; @@ -651,7 +674,8 @@ lardon3d_tui_run(Lardon3DAppState *state) int key = getch(); bool should_redraw = task != NULL - || state->screen == LARDON3D_SCREEN_TASKS; + || state->screen == LARDON3D_SCREEN_TASKS + || state->screen == LARDON3D_SCREEN_RESOURCES; if (task && lardon3d_import_task_is_finished(task)) { Lardon3DImportTaskSnapshot snapshot; if (!lardon3d_import_task_join(task) diff --git a/tests/test_resource_governor.c b/tests/test_resource_governor.c index b70da5d..705c1cd 100644 --- a/tests/test_resource_governor.c +++ b/tests/test_resource_governor.c @@ -39,6 +39,13 @@ decide_repeatedly(void *argument) ) || decision.kind != LARDON3D_RESOURCE_START || decision.batch_size != context->request.preferred_batch_size) { + (void)fprintf( + stderr, + "Décision concurrente : kind=%d batch=%zu cpu=%u\n", + (int)decision.kind, + decision.batch_size, + decision.cpu_threads + ); atomic_store(context->failed, true); break; } @@ -65,6 +72,7 @@ run_test(void) .system_cpu_reserve = 1, .maximum_cpu_load_ratio = 0.90, .maximum_io_pressure_avg10 = 80.0, + .io_slot_capacity = 8, }; CHECK(!lardon3d_resource_governor_create(NULL, &policy)); Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create( @@ -89,7 +97,7 @@ run_test(void) &request, &decision )); - CHECK(decision.kind == LARDON3D_RESOURCE_START); + CHECK(decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH); CHECK(decision.batch_size == 8); CHECK(decision.cpu_threads == 15); CHECK(decision.reason[0]); @@ -134,6 +142,7 @@ run_test(void) profile.gpu_available = true; profile.gpu_uses_shared_memory = true; policy.gpu_memory_reserve_bytes = 0; + policy.gpu_slot_capacity = 2; governor = lardon3d_resource_governor_create(&profile, &policy); CHECK(governor); request = (Lardon3DResourceRequest) { diff --git a/tests/test_resource_reservation.c b/tests/test_resource_reservation.c new file mode 100644 index 0000000..911f8bf --- /dev/null +++ b/tests/test_resource_reservation.c @@ -0,0 +1,499 @@ +#include +#include +#include +#include +#include + +#include + +#define CHECK(condition) \ + do { \ + if (!(condition)) { \ + (void)fprintf(stderr, "Échec ligne %d : %s\n", __LINE__, #condition); \ + return false; \ + } \ + } while (0) + +#define GIBIBYTES(value) ((uint64_t)(value) * 1024 * 1024 * 1024) +#define MEBIBYTES(value) ((uint64_t)(value) * 1024 * 1024) + +enum { + THREAD_COUNT = 16, + RESERVATIONS_PER_THREAD = 25, + RESERVATION_COUNT = THREAD_COUNT * RESERVATIONS_PER_THREAD, +}; + +typedef struct { + Lardon3DResourceGovernor *governor; + Lardon3DResourceSnapshot snapshot; + Lardon3DResourceEstimate estimate; + Lardon3DResourceReservation **reservations; + size_t first; + pthread_barrier_t *created; + pthread_barrier_t *release; + atomic_bool *failed; +} ReservationThread; + +static Lardon3DResourcePolicy +test_policy(unsigned int io_slots) +{ + return (Lardon3DResourcePolicy) { + .system_memory_reserve_bytes = 0, + .gpu_memory_reserve_bytes = 0, + .system_cpu_reserve = 0, + .maximum_cpu_load_ratio = 1.0, + .maximum_io_pressure_avg10 = 100.0, + .gpu_slot_capacity = 0, + .io_slot_capacity = io_slots, + }; +} + +static void * +reserve_concurrently(void *argument) +{ + ReservationThread *thread = argument; + for (size_t offset = 0; offset < RESERVATIONS_PER_THREAD; ++offset) { + Lardon3DResourceDecision decision; + size_t index = thread->first + offset; + if (!lardon3d_resource_governor_reserve( + thread->governor, + &thread->snapshot, + &thread->estimate, + &decision, + &thread->reservations[index] + ) + || !thread->reservations[index] + || decision.kind != LARDON3D_RESOURCE_START) { + atomic_store(thread->failed, true); + } + } + (void)pthread_barrier_wait(thread->created); + (void)pthread_barrier_wait(thread->release); + for (size_t offset = 0; offset < RESERVATIONS_PER_THREAD; ++offset) { + Lardon3DResourceReservation *reservation = + thread->reservations[thread->first + offset]; + if (!reservation + || !lardon3d_resource_governor_release( + thread->governor, + reservation + ) + || lardon3d_resource_governor_release( + thread->governor, + reservation + )) { + atomic_store(thread->failed, true); + } + } + return NULL; +} + +static bool +test_adaptive_batches(void) +{ + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 64, + .page_size_bytes = 4096, + .memory_total_bytes = GIBIBYTES(64), + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy = test_policy(64); + Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create( + &profile, + &policy + ); + CHECK(governor); + Lardon3DResourceSnapshot snapshot = { + .memory_available_bytes = GIBIBYTES(37), + .cpu_load_1m = 0.0, + }; + Lardon3DResourceEstimate estimate = { + .memory_bytes_per_item = GIBIBYTES(1), + .minimum_batch_size = 1, + .maximum_batch_size = 128, + .desired_cpu_threads = 8, + .desired_io_slots = 2, + .task_class = LARDON3D_RESOURCE_TASK_MIXED, + }; + Lardon3DResourceDecision decision; + Lardon3DResourceReservation *reservation; + CHECK(lardon3d_resource_governor_reserve( + governor, + &snapshot, + &estimate, + &decision, + &reservation + )); + CHECK(reservation); + CHECK(decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH); + CHECK(decision.batch_size == 37); + CHECK(lardon3d_resource_governor_reservation_is_valid( + governor, + reservation + )); + Lardon3DResourceReservationInfo information; + CHECK(lardon3d_resource_reservation_get( + governor, + reservation, + &information + )); + CHECK(information.id == 1); + CHECK(information.memory_bytes == GIBIBYTES(37)); + CHECK(information.batch_size == 37); + CHECK(information.cpu_threads == 8); + CHECK(information.io_slots == 2); + CHECK(information.state == LARDON3D_RESERVATION_ACTIVE); + CHECK(information.created_at.tv_sec > 0); + CHECK(lardon3d_resource_governor_reservation_count(governor) == 1); + + Lardon3DResourceAvailability availability; + CHECK(lardon3d_resource_governor_availability( + governor, + &snapshot, + &availability + )); + CHECK(availability.memory_reserved_bytes == GIBIBYTES(37)); + CHECK(availability.memory_available_bytes == 0); + CHECK(availability.cpu_reserved == 8); + CHECK(availability.io_slots_reserved == 2); + CHECK(availability.active_reservations == 1); + Lardon3DResourcePolicy reduced_policy = policy; + reduced_policy.system_memory_reserve_bytes = GIBIBYTES(28); + CHECK(!lardon3d_resource_governor_set_policy( + governor, + &reduced_policy + )); + + Lardon3DResourceReservation *second = NULL; + CHECK(lardon3d_resource_governor_reserve( + governor, + &snapshot, + &estimate, + &decision, + &second + )); + CHECK(!second); + CHECK(decision.kind == LARDON3D_RESOURCE_WAIT); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + CHECK(!lardon3d_resource_governor_release(governor, reservation)); + CHECK(!lardon3d_resource_governor_reservation_is_valid( + governor, + reservation + )); + CHECK(lardon3d_resource_reservation_get( + governor, + reservation, + &information + )); + CHECK(information.state == LARDON3D_RESERVATION_RELEASED); + CHECK(lardon3d_resource_governor_reservation_count(governor) == 0); + CHECK(lardon3d_resource_governor_availability( + governor, + &snapshot, + &availability + )); + CHECK(availability.memory_available_bytes == GIBIBYTES(37)); + CHECK(lardon3d_resource_governor_set_policy( + governor, + &reduced_policy + )); + + estimate = (Lardon3DResourceEstimate) { + .memory_fixed_bytes = GIBIBYTES(65), + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 1, + .task_class = LARDON3D_RESOURCE_TASK_GENERAL, + }; + CHECK(lardon3d_resource_governor_reserve( + governor, + &snapshot, + &estimate, + &decision, + &second + )); + CHECK(!second); + CHECK(decision.kind == LARDON3D_RESOURCE_REJECT); + lardon3d_resource_governor_destroy(governor); + return true; +} + +static bool +test_multiple_reservations(void) +{ + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 16, + .page_size_bytes = 4096, + .memory_total_bytes = GIBIBYTES(16), + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy = test_policy(16); + Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create( + &profile, + &policy + ); + CHECK(governor); + Lardon3DResourceSnapshot snapshot = { + .memory_available_bytes = GIBIBYTES(8), + .cpu_load_1m = 0.0, + }; + Lardon3DResourceEstimate fixed = { + .memory_fixed_bytes = GIBIBYTES(3), + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 2, + .desired_io_slots = 1, + .task_class = LARDON3D_RESOURCE_TASK_IO, + }; + Lardon3DResourceEstimate variable = { + .memory_bytes_per_item = GIBIBYTES(1), + .minimum_batch_size = 1, + .maximum_batch_size = 6, + .desired_cpu_threads = 2, + .desired_io_slots = 1, + .task_class = LARDON3D_RESOURCE_TASK_IO, + }; + Lardon3DResourceDecision decision; + Lardon3DResourceReservation *first; + Lardon3DResourceReservation *second; + CHECK(lardon3d_resource_governor_reserve( + governor, + &snapshot, + &fixed, + &decision, + &first + )); + CHECK(first && decision.kind == LARDON3D_RESOURCE_START); + CHECK(lardon3d_resource_governor_reserve( + governor, + &snapshot, + &variable, + &decision, + &second + )); + CHECK(second && decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH); + CHECK(decision.batch_size == 5); + CHECK(lardon3d_resource_governor_reservation_count(governor) == 2); + Lardon3DResourceReservationInfo listed[2]; + CHECK(lardon3d_resource_governor_list_reservations( + governor, + listed, + 2 + ) == 2); + CHECK(listed[0].id != listed[1].id); + CHECK(lardon3d_resource_governor_release(governor, first)); + CHECK(lardon3d_resource_governor_release(governor, second)); + lardon3d_resource_governor_destroy(governor); + return true; +} + +static bool +test_gpu_reservation(void) +{ + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 16, + .page_size_bytes = 4096, + .memory_total_bytes = GIBIBYTES(16), + .gpu_available = true, + .gpu_memory_known = true, + .gpu_memory_total_bytes = GIBIBYTES(4), + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy = test_policy(4); + policy.gpu_memory_reserve_bytes = MEBIBYTES(512); + policy.gpu_slot_capacity = 2; + Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create( + &profile, + &policy + ); + Lardon3DResourceGovernor *other = lardon3d_resource_governor_create( + &profile, + &policy + ); + CHECK(governor && other); + Lardon3DResourceSnapshot snapshot = { + .memory_available_bytes = GIBIBYTES(16), + .gpu_memory_available_known = true, + .gpu_memory_available_bytes = GIBIBYTES(3) + MEBIBYTES(512), + .cpu_load_1m = 0.0, + }; + Lardon3DResourceEstimate estimate = { + .gpu_memory_fixed_bytes = MEBIBYTES(512), + .gpu_memory_bytes_per_item = MEBIBYTES(512), + .minimum_batch_size = 1, + .maximum_batch_size = 6, + .desired_cpu_threads = 2, + .desired_gpu_slots = 1, + .task_class = LARDON3D_RESOURCE_TASK_GPU, + }; + Lardon3DResourceDecision decision; + Lardon3DResourceReservation *reservation; + CHECK(lardon3d_resource_governor_reserve( + governor, + &snapshot, + &estimate, + &decision, + &reservation + )); + CHECK(reservation && decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH); + CHECK(decision.batch_size == 5); + Lardon3DResourceReservationInfo information; + CHECK(lardon3d_resource_reservation_get( + governor, + reservation, + &information + )); + CHECK(information.gpu_memory_bytes == GIBIBYTES(3)); + CHECK(information.gpu_slots == 1); + Lardon3DResourceAvailability availability; + CHECK(lardon3d_resource_governor_availability( + governor, + &snapshot, + &availability + )); + CHECK(availability.gpu_memory_reserved_bytes == GIBIBYTES(3)); + CHECK(availability.gpu_memory_available_bytes == 0); + CHECK(availability.gpu_slots_available == 1); + CHECK(!lardon3d_resource_governor_reservation_is_valid( + other, + reservation + )); + CHECK(!lardon3d_resource_governor_release(other, reservation)); + CHECK(lardon3d_resource_governor_release(governor, reservation)); + lardon3d_resource_governor_destroy(other); + lardon3d_resource_governor_destroy(governor); + return true; +} + +static bool +test_concurrency(void) +{ + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 512, + .page_size_bytes = 4096, + .memory_total_bytes = GIBIBYTES(4), + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy = test_policy(512); + Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create( + &profile, + &policy + ); + CHECK(governor); + Lardon3DResourceReservation *reservations[RESERVATION_COUNT] = {0}; + pthread_barrier_t created; + pthread_barrier_t release; + CHECK(pthread_barrier_init(&created, NULL, THREAD_COUNT + 1) == 0); + CHECK(pthread_barrier_init(&release, NULL, THREAD_COUNT + 1) == 0); + atomic_bool failed = false; + ReservationThread contexts[THREAD_COUNT]; + pthread_t threads[THREAD_COUNT]; + for (size_t index = 0; index < THREAD_COUNT; ++index) { + contexts[index] = (ReservationThread) { + .governor = governor, + .snapshot = { + .memory_available_bytes = GIBIBYTES(4), + .cpu_load_1m = 0.0, + }, + .estimate = { + .memory_fixed_bytes = MEBIBYTES(1), + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 1, + .desired_io_slots = 1, + .task_class = LARDON3D_RESOURCE_TASK_GENERAL, + }, + .reservations = reservations, + .first = index * RESERVATIONS_PER_THREAD, + .created = &created, + .release = &release, + .failed = &failed, + }; + CHECK(pthread_create( + &threads[index], + NULL, + reserve_concurrently, + &contexts[index] + ) == 0); + } + (void)pthread_barrier_wait(&created); + CHECK(!atomic_load(&failed)); + CHECK(lardon3d_resource_governor_reservation_count(governor) + == RESERVATION_COUNT); + Lardon3DResourceAvailability availability; + CHECK(lardon3d_resource_governor_availability( + governor, + &contexts[0].snapshot, + &availability + )); + CHECK(availability.memory_reserved_bytes + == MEBIBYTES(RESERVATION_COUNT)); + CHECK(availability.cpu_reserved == RESERVATION_COUNT); + CHECK(availability.io_slots_reserved == RESERVATION_COUNT); + (void)pthread_barrier_wait(&release); + for (size_t index = 0; index < THREAD_COUNT; ++index) { + CHECK(pthread_join(threads[index], NULL) == 0); + } + CHECK(!atomic_load(&failed)); + CHECK(lardon3d_resource_governor_reservation_count(governor) == 0); + CHECK(pthread_barrier_destroy(&created) == 0); + CHECK(pthread_barrier_destroy(&release) == 0); + lardon3d_resource_governor_destroy(governor); + return true; +} + +static bool +test_destroy_with_active_reservations(void) +{ + Lardon3DHardwareProfile profile = { + .logical_cpu_count = 4, + .page_size_bytes = 4096, + .memory_total_bytes = GIBIBYTES(4), + .cpu_architecture = "test", + }; + Lardon3DResourcePolicy policy = test_policy(4); + Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create( + &profile, + &policy + ); + CHECK(governor); + Lardon3DResourceSnapshot snapshot = { + .memory_available_bytes = GIBIBYTES(4), + .cpu_load_1m = 0.0, + }; + Lardon3DResourceEstimate estimate = { + .memory_fixed_bytes = MEBIBYTES(1), + .minimum_batch_size = 1, + .maximum_batch_size = 1, + .desired_cpu_threads = 1, + .task_class = LARDON3D_RESOURCE_TASK_GENERAL, + }; + Lardon3DResourceDecision decision; + Lardon3DResourceReservation *reservation; + CHECK(lardon3d_resource_governor_reserve( + governor, + &snapshot, + &estimate, + &decision, + &reservation + )); + CHECK(reservation); + lardon3d_resource_governor_destroy(governor); + return true; +} + +static bool +run_test(void) +{ + CHECK(test_adaptive_batches()); + CHECK(test_multiple_reservations()); + CHECK(test_gpu_reservation()); + CHECK(test_concurrency()); + CHECK(test_destroy_with_active_reservations()); + return true; +} + +int +main(void) +{ + return run_test() ? EXIT_SUCCESS : EXIT_FAILURE; +}