fy59/lardon3d
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: add resource reservation engine

Browse source
This commit is contained in:
fy59 2026-08-06 21:05:02 +02:00
parent 0290331c60
commit bbb1808898
12 changed files with 1487 additions and 125 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

4
README.md
View file

@ -52,4 +52,6 @@ la file et de ses tâches.
Les fondations du moteur de ressources séparent la détection matérielle, les Les fondations du moteur de ressources séparent la détection matérielle, les
instantanés de disponibilité et les décisions du gouverneur. Le gouverneur instantanés de disponibilité et les décisions du gouverneur. Le gouverneur
conserve des marges système et GPU, puis autorise, diffère, réduit ou refuse un conserve des marges système et GPU, puis autorise, diffère, réduit ou refuse un
lot sans dépendre du scheduler ni d'un traitement métier. lot sans dépendre du scheduler ni d'un traitement métier. Les admissions
acceptées créent désormais une réservation atomique ; l'écran `F6` affiche les
budgets engagés et restants.

128
docs/architecture/resource_governor.md Normal file
View file

@ -0,0 +1,128 @@
# Gouverneur de ressources
## Rôle
Le gouverneur protège la réactivité de Lardon3D et la stabilité du système. Les
étapes de photogrammétrie peuvent durer plusieurs heures et manipuler des jeux
de données plus grands que la mémoire disponible. Autoriser chaque tâche à
choisir seule ses ressources conduirait à des pointes de RAM, à la saturation de
la mémoire GPU partagée et à une concurrence incontrôlée sur le stockage.
Le scheduler et le gouverneur ont donc des responsabilités distinctes. Le
gouverneur décide quelles ressources peuvent être promises. Le scheduler décide
quand et sur quel worker exécuter le travail autorisé. Le scheduler ne doit pas
interpréter lui-même la RAM disponible, la charge CPU ou la pression IO.
## Contrat d'exécution
Le chemin d'admission est le suivant :
```text
Estimate
↓
Governor
↓
Reservation
↓
Scheduler
↓
Worker
```
Une décision seule est une observation périssable. Deux threads pourraient
observer le même budget et tous deux démarrer, alors que leur consommation
cumulée dépasse la capacité. Une réservation résout cette course : le calcul du
lot et l'inscription dans les budgets sont effectués atomiquement sous le mutex
du gouverneur. Seule une réservation active constitue une autorisation
d'exécution.
`Lardon3DResourceEstimate` décrit la demande sans être modifié par le
gouverneur : coûts fixes, coûts par élément, bornes du lot, threads et slots
souhaités, et classe de tâche. `Lardon3DResourceReservation` est opaque. Son
instantané public décrit exactement le contrat retenu, y compris le lot réduit.
## Cycle de vie
1. Le producteur construit une estimation pour une unité de pipeline.
2. Un instantané des ressources dynamiques est capturé.
3. Le gouverneur compare la demande aux marges, aux ressources disponibles et
aux réservations actives.
4. Une demande impossible est refusée. Une pénurie temporaire demande
d'attendre. Une demande admissible produit une réservation, éventuellement
avec un lot ou des slots réduits.
5. Le scheduler vérifie que la réservation est toujours active avant de confier
la tâche à un worker.
6. Le worker respecte strictement le lot, les threads et les slots réservés.
7. À la fin, après annulation ou après échec, la réservation est libérée une
seule fois. Les budgets redeviennent immédiatement disponibles.
8. Le gouverneur détruit les objets de réservation restants lors de son propre
arrêt. Les pointeurs de réservation ne doivent plus être utilisés ensuite.
Une seconde libération est refusée sans modifier les compteurs. Les
réservations libérées sont conservées comme tombstones jusqu'à la destruction
du gouverneur afin de détecter cette erreur de cycle de vie sans accès mémoire
invalide.
## Budgets
### RAM
Le budget part de la mémoire actuellement disponible, bornée par la mémoire
physique détectée, puis retire la marge système et toutes les réservations
actives. Les coûts fixes et par élément sont comptabilisés avec contrôle des
débordements. Pour un GPU à mémoire partagée, les besoins RAM et GPU sont
additionnés dans cette même enveloppe.
### GPU
Lorsque la VRAM dédiée est mesurable, sa marge et ses réservations ont un budget
distinct. Si sa disponibilité est inconnue, une tâche qui en dépend attend au
lieu de supposer une capacité. Les slots GPU limitent aussi le nombre
d'opérations concurrentes, indépendamment du nombre d'octets annoncé.
### CPU
Une partie des CPU logiques reste réservée au système et à la TUI. Le gouverneur
borne les threads souhaités par les threads encore disponibles. La charge
moyenne peut différer une nouvelle admission sans interrompre les travaux déjà
réservés.
### IO
Les slots IO limitent les opérations lourdes concurrentes. La pression PSI
Linux peut différer une tâche afin de conserver un terminal réactif et d'éviter
une file d'attente disque excessive.
### Swap et zram
L'instantané mesure le swap disponible, y compris la zram exposée comme swap
par Linux. Ce volume est un signal de sécurité et non une extension normale du
budget RAM : le gouverneur ne doit pas dimensionner un lot en comptant sur le
swap. Une future politique pourra utiliser sa baisse pour suspendre les
admissions ou réduire davantage les lots.
## Lots adaptatifs
Le gouverneur calcule le plus grand lot compris entre les bornes de
l'estimation et compatible avec tous les budgets restants. Une demande de 128
éléments peut ainsi produire un contrat de 37 éléments. Le scheduler répétera
le travail sur plusieurs lots, avec écriture des résultats et libération de la
mémoire entre chaque lot. Cette règle privilégie une progression régulière à
une allocation monolithique.
Les coûts fixes ne sont payés qu'une fois par lot ; les coûts par élément
déterminent sa capacité maximale. Les slots et threads recommandés font partie
du même contrat et ne doivent pas être augmentés par le worker.
## Utilisation future
Le même protocole s'applique aux imports, miniatures et extractions EXIF, puis
aux features, au matching, au SfM, aux depth maps, au mesh et aux textures. Les
estimations pourront différer par classe sans déplacer les décisions dans le
scheduler. Le viewer Vulkan utilisera également une réservation GPU afin de ne
pas concurrencer silencieusement une reconstruction sur une machine à mémoire
partagée.
Cette séparation permettra ultérieurement plusieurs workers : chacun recevra
un contrat déjà arbitré, tandis que le gouverneur restera l'unique propriétaire
des budgets globaux.

7
include/lardon3d/app_state.h
View file

@ -3,9 +3,11 @@
#include <stdbool.h> #include <stdbool.h>
#include <limits.h> #include <limits.h>
#include <lardon3d/hardware_profile.h>
typedef struct Lardon3DImageCatalog Lardon3DImageCatalog; typedef struct Lardon3DImageCatalog Lardon3DImageCatalog;
typedef struct Lardon3DImageView Lardon3DImageView; typedef struct Lardon3DImageView Lardon3DImageView;
typedef struct Lardon3DTaskQueue Lardon3DTaskQueue; typedef struct Lardon3DTaskQueue Lardon3DTaskQueue;
typedef struct Lardon3DResourceGovernor Lardon3DResourceGovernor;
typedef enum { typedef enum {
LARDON3D_SCREEN_HOME = 0, LARDON3D_SCREEN_HOME = 0,
@ -13,7 +15,8 @@ typedef enum {
LARDON3D_SCREEN_IMPORT, LARDON3D_SCREEN_IMPORT,
LARDON3D_SCREEN_VIEWER, LARDON3D_SCREEN_VIEWER,
LARDON3D_SCREEN_HELP, LARDON3D_SCREEN_HELP,
LARDON3D_SCREEN_TASKS LARDON3D_SCREEN_TASKS,
LARDON3D_SCREEN_RESOURCES
} Lardon3DScreen; } Lardon3DScreen;
typedef struct { typedef struct {
@ -26,6 +29,8 @@ typedef struct {
Lardon3DImageCatalog *image_catalog; Lardon3DImageCatalog *image_catalog;
Lardon3DImageView *image_view; Lardon3DImageView *image_view;
Lardon3DTaskQueue *task_queue; Lardon3DTaskQueue *task_queue;
Lardon3DHardwareProfile hardware_profile;
Lardon3DResourceGovernor *resource_governor;
} Lardon3DAppState; } Lardon3DAppState;
void lardon3d_app_state_init(Lardon3DAppState *state); void lardon3d_app_state_init(Lardon3DAppState *state);

2
include/lardon3d/layout.h
View file

@ -4,6 +4,7 @@
#include <lardon3d/app_state.h> #include <lardon3d/app_state.h>
#include <lardon3d/import_task.h> #include <lardon3d/import_task.h>
#include <lardon3d/task_queue.h> #include <lardon3d/task_queue.h>
#include <lardon3d/resource_governor.h>
void lardon3d_layout_draw( void lardon3d_layout_draw(
const Lardon3DAppState *state, const Lardon3DAppState *state,
@ -13,6 +14,7 @@ void lardon3d_layout_draw(
const Lardon3DTaskSnapshot *task_snapshots, const Lardon3DTaskSnapshot *task_snapshots,
size_t task_count, size_t task_count,
const Lardon3DTaskQueueSummary *task_summary, const Lardon3DTaskQueueSummary *task_summary,
const Lardon3DResourceAvailability *resource_availability,
int rows, int rows,
int cols int cols
); );

99
include/lardon3d/resource_governor.h
View file

@ -4,6 +4,7 @@
#include <stdbool.h> #include <stdbool.h>
#include <stddef.h> #include <stddef.h>
#include <stdint.h> #include <stdint.h>
#include <time.h>
#include <lardon3d/hardware_profile.h> #include <lardon3d/hardware_profile.h>
#include <lardon3d/resource_snapshot.h> #include <lardon3d/resource_snapshot.h>
@ -13,6 +14,7 @@ enum {
}; };
typedef struct Lardon3DResourceGovernor Lardon3DResourceGovernor; typedef struct Lardon3DResourceGovernor Lardon3DResourceGovernor;
typedef struct Lardon3DResourceReservation Lardon3DResourceReservation;
typedef struct { typedef struct {
uint64_t system_memory_reserve_bytes; uint64_t system_memory_reserve_bytes;
@ -20,8 +22,32 @@ typedef struct {
unsigned int system_cpu_reserve; unsigned int system_cpu_reserve;
double maximum_cpu_load_ratio; double maximum_cpu_load_ratio;
double maximum_io_pressure_avg10; double maximum_io_pressure_avg10;
unsigned int gpu_slot_capacity;
unsigned int io_slot_capacity;
} Lardon3DResourcePolicy; } Lardon3DResourcePolicy;
typedef enum {
LARDON3D_RESOURCE_TASK_GENERAL = 0,
LARDON3D_RESOURCE_TASK_IMPORT,
LARDON3D_RESOURCE_TASK_CPU,
LARDON3D_RESOURCE_TASK_GPU,
LARDON3D_RESOURCE_TASK_IO,
LARDON3D_RESOURCE_TASK_MIXED
} Lardon3DResourceTaskClass;
typedef struct {
uint64_t memory_fixed_bytes;
uint64_t gpu_memory_fixed_bytes;
uint64_t memory_bytes_per_item;
uint64_t gpu_memory_bytes_per_item;
size_t minimum_batch_size;
size_t maximum_batch_size;
unsigned int desired_cpu_threads;
unsigned int desired_gpu_slots;
unsigned int desired_io_slots;
Lardon3DResourceTaskClass task_class;
} Lardon3DResourceEstimate;
typedef struct { typedef struct {
uint64_t memory_bytes_per_item; uint64_t memory_bytes_per_item;
uint64_t gpu_memory_bytes_per_item; uint64_t gpu_memory_bytes_per_item;
@ -42,9 +68,49 @@ typedef struct {
Lardon3DResourceDecisionKind kind; Lardon3DResourceDecisionKind kind;
size_t batch_size; size_t batch_size;
unsigned int cpu_threads; unsigned int cpu_threads;
unsigned int gpu_slots;
unsigned int io_slots;
char reason[LARDON3D_RESOURCE_REASON_CAPACITY]; char reason[LARDON3D_RESOURCE_REASON_CAPACITY];
} Lardon3DResourceDecision; } Lardon3DResourceDecision;
typedef enum {
LARDON3D_RESERVATION_ACTIVE = 0,
LARDON3D_RESERVATION_RELEASED
} Lardon3DResourceReservationState;
typedef struct {
uint64_t id;
uint64_t memory_bytes;
uint64_t gpu_memory_bytes;
unsigned int cpu_threads;
unsigned int gpu_slots;
unsigned int io_slots;
size_t batch_size;
Lardon3DResourceTaskClass task_class;
Lardon3DResourceReservationState state;
struct timespec created_at;
} Lardon3DResourceReservationInfo;
typedef struct {
uint64_t memory_budget_bytes;
uint64_t memory_reserved_bytes;
uint64_t memory_available_bytes;
bool gpu_memory_known;
uint64_t gpu_memory_budget_bytes;
uint64_t gpu_memory_reserved_bytes;
uint64_t gpu_memory_available_bytes;
unsigned int cpu_budget;
unsigned int cpu_reserved;
unsigned int cpu_available;
unsigned int gpu_slot_budget;
unsigned int gpu_slots_reserved;
unsigned int gpu_slots_available;
unsigned int io_slot_budget;
unsigned int io_slots_reserved;
unsigned int io_slots_available;
size_t active_reservations;
} Lardon3DResourceAvailability;
bool lardon3d_resource_policy_default( bool lardon3d_resource_policy_default(
const Lardon3DHardwareProfile *profile, const Lardon3DHardwareProfile *profile,
Lardon3DResourcePolicy *policy Lardon3DResourcePolicy *policy
@ -66,6 +132,39 @@ bool lardon3d_resource_governor_decide(
const Lardon3DResourceRequest *request, const Lardon3DResourceRequest *request,
Lardon3DResourceDecision *decision Lardon3DResourceDecision *decision
); );
bool lardon3d_resource_governor_reserve(
Lardon3DResourceGovernor *governor,
const Lardon3DResourceSnapshot *snapshot,
const Lardon3DResourceEstimate *estimate,
Lardon3DResourceDecision *decision,
Lardon3DResourceReservation **reservation
);
bool lardon3d_resource_governor_release(
Lardon3DResourceGovernor *governor,
Lardon3DResourceReservation *reservation
);
bool lardon3d_resource_governor_reservation_is_valid(
Lardon3DResourceGovernor *governor,
const Lardon3DResourceReservation *reservation
);
bool lardon3d_resource_reservation_get(
Lardon3DResourceGovernor *governor,
const Lardon3DResourceReservation *reservation,
Lardon3DResourceReservationInfo *information
);
size_t lardon3d_resource_governor_list_reservations(
Lardon3DResourceGovernor *governor,
Lardon3DResourceReservationInfo *reservations,
size_t capacity
);
size_t lardon3d_resource_governor_reservation_count(
Lardon3DResourceGovernor *governor
);
bool lardon3d_resource_governor_availability(
Lardon3DResourceGovernor *governor,
const Lardon3DResourceSnapshot *snapshot,
Lardon3DResourceAvailability *availability
);
const char *lardon3d_resource_decision_name( const char *lardon3d_resource_decision_name(
Lardon3DResourceDecisionKind kind Lardon3DResourceDecisionKind kind
); );

12
meson.build
View file

@ -154,3 +154,15 @@ resource_governor_test = executable(
) )
test('resource-governor', resource_governor_test, timeout: 30) test('resource-governor', resource_governor_test, timeout: 30)
resource_reservation_test = executable(
'test-resource-reservation',
sources: [
'tests/test_resource_reservation.c',
'src/resource_governor.c',
],
include_directories: include_directories('include'),
dependencies: [threads],
)
test('resource-reservation', resource_reservation_test, timeout: 60)

24
src/app.c
View file

@ -5,6 +5,7 @@
#include <lardon3d/app_state.h> #include <lardon3d/app_state.h>
#include <lardon3d/image_catalog.h> #include <lardon3d/image_catalog.h>
#include <lardon3d/image_view.h> #include <lardon3d/image_view.h>
#include <lardon3d/resource_governor.h>
#include <lardon3d/task_queue.h> #include <lardon3d/task_queue.h>
#include <lardon3d/tui.h> #include <lardon3d/tui.h>
@ -17,13 +18,35 @@ lardon3d_app_run(void)
if (!setlocale(LC_ALL, "")) { if (!setlocale(LC_ALL, "")) {
return EXIT_FAILURE; return EXIT_FAILURE;
} }
char error[256];
Lardon3DResourcePolicy resource_policy;
if (!lardon3d_hardware_profile_detect(
&state.hardware_profile,
error,
sizeof(error)
)
|| !lardon3d_resource_policy_default(
&state.hardware_profile,
&resource_policy
)) {
return EXIT_FAILURE;
}
state.resource_governor = lardon3d_resource_governor_create(
&state.hardware_profile,
&resource_policy
);
if (!state.resource_governor) {
return EXIT_FAILURE;
}
state.task_queue = lardon3d_task_queue_create(); state.task_queue = lardon3d_task_queue_create();
if (!state.task_queue) { if (!state.task_queue) {
lardon3d_resource_governor_destroy(state.resource_governor);
return EXIT_FAILURE; return EXIT_FAILURE;
} }
if (!lardon3d_tui_init()) { if (!lardon3d_tui_init()) {
lardon3d_task_queue_destroy(state.task_queue); lardon3d_task_queue_destroy(state.task_queue);
lardon3d_resource_governor_destroy(state.resource_governor);
return EXIT_FAILURE; return EXIT_FAILURE;
} }
@ -32,6 +55,7 @@ lardon3d_app_run(void)
lardon3d_image_catalog_destroy(state.image_catalog); lardon3d_image_catalog_destroy(state.image_catalog);
lardon3d_tui_shutdown(); lardon3d_tui_shutdown();
lardon3d_task_queue_destroy(state.task_queue); lardon3d_task_queue_destroy(state.task_queue);
lardon3d_resource_governor_destroy(state.resource_governor);
return success ? EXIT_SUCCESS : EXIT_FAILURE; return success ? EXIT_SUCCESS : EXIT_FAILURE;
} }

112
src/layout.c
View file

@ -61,7 +61,7 @@ screen_texts(
const char **footer const char **footer
) )
{ {
*footer = "ESC Accueil F1 Aide F2 Projets F3 Import F4 Viewer F5 Tâches Q"; *footer = "ESC Accueil F1 Aide F2 Projets F3 Import F4 Viewer F5 Tâches F6 Ressources Q";
switch (screen) { switch (screen) {
case LARDON3D_SCREEN_PROJECTS: case LARDON3D_SCREEN_PROJECTS:
@ -87,11 +87,16 @@ screen_texts(
*content = "Tâches en arrière-plan"; *content = "Tâches en arrière-plan";
*footer = "F5 Tâches ESC Accueil Q Quit"; *footer = "F5 Tâches ESC Accueil Q Quit";
break; break;
case LARDON3D_SCREEN_RESOURCES:
*title = "Ressources";
*content = "Gouverneur de ressources";
*footer = "F6 Ressources ESC Accueil Q Quit";
break;
case LARDON3D_SCREEN_HOME: case LARDON3D_SCREEN_HOME:
default: default:
*title = "Accueil"; *title = "Accueil";
*content = "Bienvenue dans Lardon3D"; *content = "Bienvenue dans Lardon3D";
*footer = "F1 Aide F2 Projets F3 Import F4 Viewer F5 Tâches Q Quit"; *footer = "F1 Aide F2 Projets F3 Import F4 Viewer F5 Tâches F6 Ressources Q";
break; break;
} }
} }
@ -344,6 +349,104 @@ draw_tasks(
} }
} }
static void
draw_resource_line(
int row,
int columns,
const char *label,
uint64_t reserved,
uint64_t available
)
{
char reserved_text[64];
char available_text[64];
char line[256];
lardon3d_image_catalog_format_size(
reserved,
reserved_text,
sizeof(reserved_text)
);
lardon3d_image_catalog_format_size(
available,
available_text,
sizeof(available_text)
);
(void)snprintf(
line,
sizeof(line),
"%s réservée : %s Restante : %s",
label,
reserved_text,
available_text
);
draw_text(row, 4, columns - 6, line);
}
static void
draw_resources(
const Lardon3DResourceAvailability *availability,
int columns
)
{
if (!availability) {
draw_text(7, 4, columns - 6, "Ressources système indisponibles.");
return;
}
char line[256];
(void)snprintf(
line,
sizeof(line),
"Réservations actives : %zu",
availability->active_reservations
);
draw_text(6, 4, columns - 6, line);
draw_resource_line(
8,
columns,
"RAM",
availability->memory_reserved_bytes,
availability->memory_available_bytes
);
if (availability->gpu_memory_known) {
draw_resource_line(
9,
columns,
"GPU",
availability->gpu_memory_reserved_bytes,
availability->gpu_memory_available_bytes
);
} else {
draw_text(9, 4, columns - 6, "Mémoire GPU disponible : inconnue");
}
(void)snprintf(
line,
sizeof(line),
"CPU réservés : %u Restants : %u / %u",
availability->cpu_reserved,
availability->cpu_available,
availability->cpu_budget
);
draw_text(10, 4, columns - 6, line);
(void)snprintf(
line,
sizeof(line),
"Slots GPU réservés : %u Restants : %u / %u",
availability->gpu_slots_reserved,
availability->gpu_slots_available,
availability->gpu_slot_budget
);
draw_text(11, 4, columns - 6, line);
(void)snprintf(
line,
sizeof(line),
"Slots IO réservés : %u Restants : %u / %u",
availability->io_slots_reserved,
availability->io_slots_available,
availability->io_slot_budget
);
draw_text(12, 4, columns - 6, line);
}
static void static void
draw_content( draw_content(
const Lardon3DAppState *state, const Lardon3DAppState *state,
@ -353,6 +456,7 @@ draw_content(
const Lardon3DTaskSnapshot *task_snapshots, const Lardon3DTaskSnapshot *task_snapshots,
size_t task_count, size_t task_count,
const Lardon3DTaskQueueSummary *task_summary, const Lardon3DTaskQueueSummary *task_summary,
const Lardon3DResourceAvailability *resource_availability,
int rows, int rows,
int columns int columns
) )
@ -402,6 +506,8 @@ draw_content(
rows, rows,
columns columns
); );
} else if (state->screen == LARDON3D_SCREEN_RESOURCES) {
draw_resources(resource_availability, columns);
} else { } else {
draw_text( draw_text(
(3 + journal_row) / 2, (3 + journal_row) / 2,
@ -429,6 +535,7 @@ lardon3d_layout_draw(
const Lardon3DTaskSnapshot *task_snapshots, const Lardon3DTaskSnapshot *task_snapshots,
size_t task_count, size_t task_count,
const Lardon3DTaskQueueSummary *task_summary, const Lardon3DTaskQueueSummary *task_summary,
const Lardon3DResourceAvailability *resource_availability,
int rows, int rows,
int columns int columns
) )
@ -447,6 +554,7 @@ lardon3d_layout_draw(
task_snapshots, task_snapshots,
task_count, task_count,
task_summary, task_summary,
resource_availability,
rows, rows,
columns columns
); );

688
src/resource_governor.c
View file

@ -5,10 +5,25 @@
#include <lardon3d/resource_governor.h> #include <lardon3d/resource_governor.h>
struct Lardon3DResourceReservation {
Lardon3DResourceReservationInfo information;
uint64_t charged_memory_bytes;
struct Lardon3DResourceReservation *next;
};
struct Lardon3DResourceGovernor { struct Lardon3DResourceGovernor {
pthread_mutex_t mutex; pthread_mutex_t mutex;
Lardon3DHardwareProfile profile; Lardon3DHardwareProfile profile;
Lardon3DResourcePolicy policy; Lardon3DResourcePolicy policy;
uint64_t next_reservation_id;
uint64_t memory_reserved_bytes;
uint64_t gpu_memory_reserved_bytes;
unsigned int cpu_reserved;
unsigned int gpu_slots_reserved;
unsigned int io_slots_reserved;
size_t active_count;
Lardon3DResourceReservation *active;
Lardon3DResourceReservation *released;
}; };
static bool static bool
@ -34,17 +49,107 @@ valid_policy(
&& policy->maximum_cpu_load_ratio <= 1.0 && policy->maximum_cpu_load_ratio <= 1.0
&& policy->maximum_io_pressure_avg10 >= 0.0 && policy->maximum_io_pressure_avg10 >= 0.0
&& policy->maximum_io_pressure_avg10 <= 100.0 && policy->maximum_io_pressure_avg10 <= 100.0
&& policy->io_slot_capacity > 0
&& (!profile->gpu_available || policy->gpu_slot_capacity > 0)
&& (!profile->gpu_memory_known && (!profile->gpu_memory_known
|| policy->gpu_memory_reserve_bytes || policy->gpu_memory_reserve_bytes
< profile->gpu_memory_total_bytes); < profile->gpu_memory_total_bytes);
} }
static bool
valid_snapshot(
const Lardon3DHardwareProfile *profile,
const Lardon3DResourceSnapshot *snapshot
)
{
return snapshot && snapshot->cpu_load_1m >= 0.0
&& (!snapshot->io_pressure_known
|| (snapshot->io_pressure_avg10 >= 0.0
&& snapshot->io_pressure_avg10 <= 100.0))
&& (!snapshot->gpu_memory_available_known
|| !profile->gpu_memory_known
|| profile->gpu_uses_shared_memory
|| snapshot->gpu_memory_available_bytes
<= profile->gpu_memory_total_bytes);
}
static bool
valid_estimate(const Lardon3DResourceEstimate *estimate)
{
return estimate && estimate->minimum_batch_size > 0
&& estimate->maximum_batch_size >= estimate->minimum_batch_size
&& estimate->desired_cpu_threads > 0
&& estimate->task_class >= LARDON3D_RESOURCE_TASK_GENERAL
&& estimate->task_class <= LARDON3D_RESOURCE_TASK_MIXED
&& ((estimate->gpu_memory_fixed_bytes == 0
&& estimate->gpu_memory_bytes_per_item == 0)
|| estimate->desired_gpu_slots > 0);
}
static uint64_t
subtract_floor(uint64_t value, uint64_t amount)
{
return value > amount ? value - amount : 0;
}
static unsigned int
subtract_unsigned(unsigned int value, unsigned int amount)
{
return value > amount ? value - amount : 0;
}
static uint64_t
minimum_uint64(uint64_t left, uint64_t right)
{
return left < right ? left : right;
}
static size_t
minimum_size(size_t left, size_t right)
{
return left < right ? left : right;
}
static bool
resource_size(
uint64_t fixed,
uint64_t per_item,
size_t batch,
uint64_t *total
)
{
if (per_item > 0 && batch > UINT64_MAX / per_item) {
return false;
}
uint64_t variable = per_item * (uint64_t)batch;
if (fixed > UINT64_MAX - variable) {
return false;
}
*total = fixed + variable;
return true;
}
static size_t
batch_capacity(uint64_t available, uint64_t fixed, uint64_t per_item)
{
if (available < fixed) {
return 0;
}
if (per_item == 0) {
return SIZE_MAX;
}
uint64_t capacity = (available - fixed) / per_item;
return capacity > SIZE_MAX ? SIZE_MAX : (size_t)capacity;
}
static void static void
set_decision( set_decision(
Lardon3DResourceDecision *decision, Lardon3DResourceDecision *decision,
Lardon3DResourceDecisionKind kind, Lardon3DResourceDecisionKind kind,
size_t batch_size, size_t batch_size,
unsigned int cpu_threads, unsigned int cpu_threads,
unsigned int gpu_slots,
unsigned int io_slots,
const char *reason const char *reason
) )
{ {
@ -52,6 +157,8 @@ set_decision(
.kind = kind, .kind = kind,
.batch_size = batch_size, .batch_size = batch_size,
.cpu_threads = cpu_threads, .cpu_threads = cpu_threads,
.gpu_slots = gpu_slots,
.io_slots = io_slots,
}; };
(void)snprintf(decision->reason, sizeof(decision->reason), "%s", reason); (void)snprintf(decision->reason, sizeof(decision->reason), "%s", reason);
} }
@ -73,6 +180,8 @@ lardon3d_resource_policy_default(
.system_cpu_reserve = profile->logical_cpu_count > 2 ? 1 : 0, .system_cpu_reserve = profile->logical_cpu_count > 2 ? 1 : 0,
.maximum_cpu_load_ratio = 0.90, .maximum_cpu_load_ratio = 0.90,
.maximum_io_pressure_avg10 = 80.0, .maximum_io_pressure_avg10 = 80.0,
.gpu_slot_capacity = profile->gpu_available ? 1 : 0,
.io_slot_capacity = 1,
}; };
return valid_policy(profile, policy); return valid_policy(profile, policy);
} }
@ -96,6 +205,7 @@ lardon3d_resource_governor_create(
} }
governor->profile = *profile; governor->profile = *profile;
governor->policy = *policy; governor->policy = *policy;
governor->next_reservation_id = 1;
return governor; return governor;
} }
@ -105,6 +215,18 @@ lardon3d_resource_governor_destroy(Lardon3DResourceGovernor *governor)
if (!governor) { if (!governor) {
return; return;
} }
Lardon3DResourceReservation *reservation = governor->active;
while (reservation) {
Lardon3DResourceReservation *next = reservation->next;
free(reservation);
reservation = next;
}
reservation = governor->released;
while (reservation) {
Lardon3DResourceReservation *next = reservation->next;
free(reservation);
reservation = next;
}
(void)pthread_mutex_destroy(&governor->mutex); (void)pthread_mutex_destroy(&governor->mutex);
free(governor); free(governor);
} }
@ -119,7 +241,23 @@ lardon3d_resource_governor_set_policy(
return false; return false;
} }
(void)pthread_mutex_lock(&governor->mutex); (void)pthread_mutex_lock(&governor->mutex);
bool accepted = valid_policy(&governor->profile, policy); bool policy_valid = valid_policy(&governor->profile, policy);
uint64_t memory_budget = policy_valid
? governor->profile.memory_total_bytes
- policy->system_memory_reserve_bytes
: 0;
uint64_t gpu_budget = policy_valid && governor->profile.gpu_memory_known
&& !governor->profile.gpu_uses_shared_memory
? governor->profile.gpu_memory_total_bytes
- policy->gpu_memory_reserve_bytes
: UINT64_MAX;
bool accepted = policy_valid
&& memory_budget >= governor->memory_reserved_bytes
&& gpu_budget >= governor->gpu_memory_reserved_bytes
&& governor->profile.logical_cpu_count - policy->system_cpu_reserve
>= governor->cpu_reserved
&& policy->gpu_slot_capacity >= governor->gpu_slots_reserved
&& policy->io_slot_capacity >= governor->io_slots_reserved;
if (accepted) { if (accepted) {
governor->policy = *policy; governor->policy = *policy;
} }
@ -127,23 +265,426 @@ lardon3d_resource_governor_set_policy(
return accepted; return accepted;
} }
static size_t static bool
capacity_for(uint64_t available, uint64_t reserve, uint64_t per_item) availability_locked(
const Lardon3DResourceGovernor *governor,
const Lardon3DResourceSnapshot *snapshot,
Lardon3DResourceAvailability *availability
)
{ {
if (per_item == 0) { if (!valid_snapshot(&governor->profile, snapshot)) {
return SIZE_MAX; return false;
} }
if (available <= reserve) { uint64_t detected_memory = minimum_uint64(
return 0; snapshot->memory_available_bytes,
governor->profile.memory_total_bytes
);
uint64_t memory_budget = subtract_floor(
detected_memory,
governor->policy.system_memory_reserve_bytes
);
uint64_t gpu_budget = 0;
bool gpu_known = false;
if (governor->profile.gpu_uses_shared_memory) {
gpu_known = true;
gpu_budget = memory_budget;
} else if (governor->profile.gpu_memory_known
&& snapshot->gpu_memory_available_known) {
gpu_known = true;
gpu_budget = subtract_floor(
snapshot->gpu_memory_available_bytes,
governor->policy.gpu_memory_reserve_bytes
);
} }
uint64_t capacity = (available - reserve) / per_item; unsigned int cpu_budget = governor->profile.logical_cpu_count
return capacity > SIZE_MAX ? SIZE_MAX : (size_t)capacity; - governor->policy.system_cpu_reserve;
*availability = (Lardon3DResourceAvailability) {
.memory_budget_bytes = memory_budget,
.memory_reserved_bytes = governor->memory_reserved_bytes,
.memory_available_bytes = subtract_floor(
memory_budget,
governor->memory_reserved_bytes
),
.gpu_memory_known = gpu_known,
.gpu_memory_budget_bytes = gpu_budget,
.gpu_memory_reserved_bytes = governor->gpu_memory_reserved_bytes,
.gpu_memory_available_bytes = governor->profile.gpu_uses_shared_memory
? subtract_floor(memory_budget, governor->memory_reserved_bytes)
: subtract_floor(gpu_budget, governor->gpu_memory_reserved_bytes),
.cpu_budget = cpu_budget,
.cpu_reserved = governor->cpu_reserved,
.cpu_available = subtract_unsigned(cpu_budget, governor->cpu_reserved),
.gpu_slot_budget = governor->policy.gpu_slot_capacity,
.gpu_slots_reserved = governor->gpu_slots_reserved,
.gpu_slots_available = subtract_unsigned(
governor->policy.gpu_slot_capacity,
governor->gpu_slots_reserved
),
.io_slot_budget = governor->policy.io_slot_capacity,
.io_slots_reserved = governor->io_slots_reserved,
.io_slots_available = subtract_unsigned(
governor->policy.io_slot_capacity,
governor->io_slots_reserved
),
.active_reservations = governor->active_count,
};
return true;
} }
static size_t bool
minimum_size(size_t left, size_t right) lardon3d_resource_governor_availability(
Lardon3DResourceGovernor *governor,
const Lardon3DResourceSnapshot *snapshot,
Lardon3DResourceAvailability *availability
)
{ {
return left < right ? left : right; if (!governor || !snapshot || !availability) {
return false;
}
(void)pthread_mutex_lock(&governor->mutex);
bool success = availability_locked(governor, snapshot, availability);
(void)pthread_mutex_unlock(&governor->mutex);
return success;
}
static void
evaluate_locked(
Lardon3DResourceGovernor *governor,
const Lardon3DResourceSnapshot *snapshot,
const Lardon3DResourceEstimate *estimate,
Lardon3DResourceDecision *decision
)
{
if (!valid_estimate(estimate)) {
set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Estimation de ressources invalide.");
return;
}
if ((estimate->desired_gpu_slots > 0
|| estimate->gpu_memory_fixed_bytes > 0
|| estimate->gpu_memory_bytes_per_item > 0)
&& !governor->profile.gpu_available) {
set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Aucun GPU disponible.");
return;
}
uint64_t memory_fixed = estimate->memory_fixed_bytes;
uint64_t memory_per_item = estimate->memory_bytes_per_item;
if (governor->profile.gpu_uses_shared_memory) {
if (memory_fixed > UINT64_MAX - estimate->gpu_memory_fixed_bytes
|| memory_per_item
> UINT64_MAX - estimate->gpu_memory_bytes_per_item) {
set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Estimation mémoire trop grande.");
return;
}
memory_fixed += estimate->gpu_memory_fixed_bytes;
memory_per_item += estimate->gpu_memory_bytes_per_item;
}
size_t theoretical_batch = batch_capacity(
governor->profile.memory_total_bytes
- governor->policy.system_memory_reserve_bytes,
memory_fixed,
memory_per_item
);
if (!governor->profile.gpu_uses_shared_memory
&& (estimate->gpu_memory_fixed_bytes > 0
|| estimate->gpu_memory_bytes_per_item > 0)
&& governor->profile.gpu_memory_known) {
theoretical_batch = minimum_size(
theoretical_batch,
batch_capacity(
governor->profile.gpu_memory_total_bytes
- governor->policy.gpu_memory_reserve_bytes,
estimate->gpu_memory_fixed_bytes,
estimate->gpu_memory_bytes_per_item
)
);
}
if (theoretical_batch < estimate->minimum_batch_size) {
set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Tâche impossible sur cette machine.");
return;
}
double load_limit = (double)governor->profile.logical_cpu_count
* governor->policy.maximum_cpu_load_ratio;
if (snapshot->cpu_load_1m >= load_limit) {
set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Charge CPU trop élevée.");
return;
}
if (estimate->desired_io_slots > 0 && snapshot->io_pressure_known
&& snapshot->io_pressure_avg10
>= governor->policy.maximum_io_pressure_avg10) {
set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Pression d'entrée-sortie trop élevée.");
return;
}
Lardon3DResourceAvailability available;
if (!availability_locked(governor, snapshot, &available)) {
set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, 0, 0, "Instantané de ressources invalide.");
return;
}
size_t batch = batch_capacity(
available.memory_available_bytes,
memory_fixed,
memory_per_item
);
if (!governor->profile.gpu_uses_shared_memory
&& (estimate->gpu_memory_fixed_bytes > 0
|| estimate->gpu_memory_bytes_per_item > 0)) {
if (!available.gpu_memory_known) {
set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Mémoire GPU disponible inconnue.");
return;
}
batch = minimum_size(
batch,
batch_capacity(
available.gpu_memory_available_bytes,
estimate->gpu_memory_fixed_bytes,
estimate->gpu_memory_bytes_per_item
)
);
}
batch = minimum_size(batch, estimate->maximum_batch_size);
if (batch < estimate->minimum_batch_size) {
set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Ressources déjà réservées ou temporairement insuffisantes.");
return;
}
if (available.cpu_available == 0
|| (estimate->desired_gpu_slots > 0
&& available.gpu_slots_available == 0)
|| (estimate->desired_io_slots > 0
&& available.io_slots_available == 0)) {
set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, 0, 0, 0, "Slots de calcul déjà réservés.");
return;
}
unsigned int cpu = estimate->desired_cpu_threads < available.cpu_available
? estimate->desired_cpu_threads
: available.cpu_available;
unsigned int gpu = estimate->desired_gpu_slots
< available.gpu_slots_available
? estimate->desired_gpu_slots
: available.gpu_slots_available;
unsigned int io = estimate->desired_io_slots < available.io_slots_available
? estimate->desired_io_slots
: available.io_slots_available;
bool reduced = batch < estimate->maximum_batch_size
|| cpu < estimate->desired_cpu_threads
|| gpu < estimate->desired_gpu_slots
|| io < estimate->desired_io_slots;
set_decision(
decision,
reduced ? LARDON3D_RESOURCE_REDUCE_BATCH : LARDON3D_RESOURCE_START,
batch,
cpu,
gpu,
io,
reduced ? "Contrat réduit aux ressources disponibles."
: "Ressources disponibles."
);
}
bool
lardon3d_resource_governor_reserve(
Lardon3DResourceGovernor *governor,
const Lardon3DResourceSnapshot *snapshot,
const Lardon3DResourceEstimate *estimate,
Lardon3DResourceDecision *decision,
Lardon3DResourceReservation **reservation
)
{
if (!governor || !snapshot || !estimate || !decision || !reservation) {
return false;
}
*reservation = NULL;
Lardon3DResourceReservation *created = calloc(1, sizeof(*created));
if (!created) {
return false;
}
(void)pthread_mutex_lock(&governor->mutex);
if (!valid_snapshot(&governor->profile, snapshot)) {
(void)pthread_mutex_unlock(&governor->mutex);
free(created);
return false;
}
evaluate_locked(governor, snapshot, estimate, decision);
if ((decision->kind != LARDON3D_RESOURCE_START
&& decision->kind != LARDON3D_RESOURCE_REDUCE_BATCH)
|| governor->next_reservation_id == 0) {
(void)pthread_mutex_unlock(&governor->mutex);
free(created);
return true;
}
uint64_t memory_bytes;
uint64_t gpu_memory_bytes;
if (!resource_size(
estimate->memory_fixed_bytes,
estimate->memory_bytes_per_item,
decision->batch_size,
&memory_bytes
)
|| !resource_size(
estimate->gpu_memory_fixed_bytes,
estimate->gpu_memory_bytes_per_item,
decision->batch_size,
&gpu_memory_bytes
)
|| clock_gettime(CLOCK_REALTIME, &created->information.created_at) != 0) {
(void)pthread_mutex_unlock(&governor->mutex);
free(created);
return false;
}
created->information.id = governor->next_reservation_id++;
created->information.memory_bytes = memory_bytes;
created->information.gpu_memory_bytes = gpu_memory_bytes;
created->information.cpu_threads = decision->cpu_threads;
created->information.gpu_slots = decision->gpu_slots;
created->information.io_slots = decision->io_slots;
created->information.batch_size = decision->batch_size;
created->information.task_class = estimate->task_class;
created->information.state = LARDON3D_RESERVATION_ACTIVE;
created->charged_memory_bytes = governor->profile.gpu_uses_shared_memory
? memory_bytes + gpu_memory_bytes
: memory_bytes;
created->next = governor->active;
governor->active = created;
governor->memory_reserved_bytes += created->charged_memory_bytes;
governor->gpu_memory_reserved_bytes += gpu_memory_bytes;
governor->cpu_reserved += decision->cpu_threads;
governor->gpu_slots_reserved += decision->gpu_slots;
governor->io_slots_reserved += decision->io_slots;
++governor->active_count;
*reservation = created;
(void)pthread_mutex_unlock(&governor->mutex);
return true;
}
static Lardon3DResourceReservation *
find_reservation(
Lardon3DResourceReservation *head,
const Lardon3DResourceReservation *wanted
)
{
for (Lardon3DResourceReservation *current = head;
current;
current = current->next) {
if (current == wanted) {
return current;
}
}
return NULL;
}
bool
lardon3d_resource_governor_release(
Lardon3DResourceGovernor *governor,
Lardon3DResourceReservation *reservation
)
{
if (!governor || !reservation) {
return false;
}
(void)pthread_mutex_lock(&governor->mutex);
Lardon3DResourceReservation *previous = NULL;
Lardon3DResourceReservation *current = governor->active;
while (current && current != reservation) {
previous = current;
current = current->next;
}
if (!current) {
(void)pthread_mutex_unlock(&governor->mutex);
return false;
}
if (previous) {
previous->next = current->next;
} else {
governor->active = current->next;
}
governor->memory_reserved_bytes -= current->charged_memory_bytes;
governor->gpu_memory_reserved_bytes -= current->information.gpu_memory_bytes;
governor->cpu_reserved -= current->information.cpu_threads;
governor->gpu_slots_reserved -= current->information.gpu_slots;
governor->io_slots_reserved -= current->information.io_slots;
--governor->active_count;
current->information.state = LARDON3D_RESERVATION_RELEASED;
current->next = governor->released;
governor->released = current;
(void)pthread_mutex_unlock(&governor->mutex);
return true;
}
bool
lardon3d_resource_governor_reservation_is_valid(
Lardon3DResourceGovernor *governor,
const Lardon3DResourceReservation *reservation
)
{
if (!governor || !reservation) {
return false;
}
(void)pthread_mutex_lock(&governor->mutex);
bool valid = find_reservation(governor->active, reservation) != NULL;
(void)pthread_mutex_unlock(&governor->mutex);
return valid;
}
bool
lardon3d_resource_reservation_get(
Lardon3DResourceGovernor *governor,
const Lardon3DResourceReservation *reservation,
Lardon3DResourceReservationInfo *information
)
{
if (!governor || !reservation || !information) {
return false;
}
(void)pthread_mutex_lock(&governor->mutex);
Lardon3DResourceReservation *found = find_reservation(
governor->active,
reservation
);
if (!found) {
found = find_reservation(governor->released, reservation);
}
if (found) {
*information = found->information;
}
(void)pthread_mutex_unlock(&governor->mutex);
return found != NULL;
}
size_t
lardon3d_resource_governor_list_reservations(
Lardon3DResourceGovernor *governor,
Lardon3DResourceReservationInfo *reservations,
size_t capacity
)
{
if (!governor || (!reservations && capacity > 0)) {
return 0;
}
(void)pthread_mutex_lock(&governor->mutex);
size_t count = 0;
for (Lardon3DResourceReservation *current = governor->active;
current && count < capacity;
current = current->next) {
reservations[count++] = current->information;
}
(void)pthread_mutex_unlock(&governor->mutex);
return count;
}
size_t
lardon3d_resource_governor_reservation_count(
Lardon3DResourceGovernor *governor
)
{
if (!governor) {
return 0;
}
(void)pthread_mutex_lock(&governor->mutex);
size_t count = governor->active_count;
(void)pthread_mutex_unlock(&governor->mutex);
return count;
} }
bool bool
@ -157,116 +698,25 @@ lardon3d_resource_governor_decide(
if (!governor || !snapshot || !request || !decision) { if (!governor || !snapshot || !request || !decision) {
return false; return false;
} }
Lardon3DResourceEstimate estimate = {
.memory_bytes_per_item = request->memory_bytes_per_item,
.gpu_memory_bytes_per_item = request->gpu_memory_bytes_per_item,
.minimum_batch_size = request->minimum_batch_size,
.maximum_batch_size = request->preferred_batch_size,
.desired_cpu_threads = request->requested_cpu_threads,
.desired_gpu_slots = request->gpu_memory_bytes_per_item > 0 ? 1 : 0,
.desired_io_slots = request->io_intensive ? 1 : 0,
.task_class = request->io_intensive
? LARDON3D_RESOURCE_TASK_IO
: LARDON3D_RESOURCE_TASK_GENERAL,
};
(void)pthread_mutex_lock(&governor->mutex); (void)pthread_mutex_lock(&governor->mutex);
const Lardon3DHardwareProfile profile = governor->profile; if (!valid_snapshot(&governor->profile, snapshot)) {
const Lardon3DResourcePolicy policy = governor->policy; (void)pthread_mutex_unlock(&governor->mutex);
(void)pthread_mutex_unlock(&governor->mutex);
if (!(snapshot->cpu_load_1m >= 0.0)
|| (snapshot->io_pressure_known
&& (!(snapshot->io_pressure_avg10 >= 0.0)
|| snapshot->io_pressure_avg10 > 100.0))
|| (snapshot->gpu_memory_available_known
&& profile.gpu_memory_known
&& !profile.gpu_uses_shared_memory
&& snapshot->gpu_memory_available_bytes
> profile.gpu_memory_total_bytes)) {
return false; return false;
} }
evaluate_locked(governor, snapshot, &estimate, decision);
if (request->minimum_batch_size == 0 (void)pthread_mutex_unlock(&governor->mutex);
|| request->preferred_batch_size < request->minimum_batch_size
|| request->requested_cpu_threads == 0) {
set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, 0, "Demande de ressources invalide.");
return true;
}
unsigned int available_threads = profile.logical_cpu_count
- policy.system_cpu_reserve;
unsigned int cpu_threads = request->requested_cpu_threads < available_threads
? request->requested_cpu_threads
: available_threads;
uint64_t system_memory_per_item = request->memory_bytes_per_item;
if (request->gpu_memory_bytes_per_item > 0
&& profile.gpu_uses_shared_memory) {
if (system_memory_per_item
> UINT64_MAX - request->gpu_memory_bytes_per_item) {
set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, cpu_threads, "Besoin mémoire trop grand.");
return true;
}
system_memory_per_item += request->gpu_memory_bytes_per_item;
}
size_t theoretical_batch = capacity_for(
profile.memory_total_bytes,
policy.system_memory_reserve_bytes,
system_memory_per_item
);
if (request->gpu_memory_bytes_per_item > 0) {
if (!profile.gpu_available) {
set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, cpu_threads, "Aucun GPU disponible.");
return true;
}
if (profile.gpu_memory_known && !profile.gpu_uses_shared_memory) {
theoretical_batch = minimum_size(
theoretical_batch,
capacity_for(
profile.gpu_memory_total_bytes,
policy.gpu_memory_reserve_bytes,
request->gpu_memory_bytes_per_item
)
);
}
}
if (theoretical_batch < request->minimum_batch_size) {
set_decision(decision, LARDON3D_RESOURCE_REJECT, 0, cpu_threads, "Tâche impossible sur cette machine.");
return true;
}
double load_limit = (double)profile.logical_cpu_count
* policy.maximum_cpu_load_ratio;
if (snapshot->cpu_load_1m >= load_limit) {
set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, cpu_threads, "Charge CPU trop élevée.");
return true;
}
if (request->io_intensive && snapshot->io_pressure_known
&& snapshot->io_pressure_avg10 >= policy.maximum_io_pressure_avg10) {
set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, cpu_threads, "Pression d'entrée-sortie trop élevée.");
return true;
}
size_t current_batch = capacity_for(
snapshot->memory_available_bytes,
policy.system_memory_reserve_bytes,
system_memory_per_item
);
if (request->gpu_memory_bytes_per_item > 0
&& !profile.gpu_uses_shared_memory) {
if (!snapshot->gpu_memory_available_known) {
set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, cpu_threads, "Mémoire GPU disponible inconnue.");
return true;
}
current_batch = minimum_size(
current_batch,
capacity_for(
snapshot->gpu_memory_available_bytes,
policy.gpu_memory_reserve_bytes,
request->gpu_memory_bytes_per_item
)
);
}
if (current_batch < request->minimum_batch_size) {
set_decision(decision, LARDON3D_RESOURCE_WAIT, 0, cpu_threads, "Ressources temporairement insuffisantes.");
return true;
}
size_t batch_size = minimum_size(
current_batch,
request->preferred_batch_size
);
if (batch_size < request->preferred_batch_size) {
set_decision(decision, LARDON3D_RESOURCE_REDUCE_BATCH, batch_size, cpu_threads, "Taille de lot réduite.");
return true;
}
set_decision(decision, LARDON3D_RESOURCE_START, batch_size, cpu_threads, "Ressources disponibles.");
return true; return true;
} }

26
src/tui.c
View file

@ -10,6 +10,8 @@
#include <lardon3d/image_view.h> #include <lardon3d/image_view.h>
#include <lardon3d/layout.h> #include <lardon3d/layout.h>
#include <lardon3d/project.h> #include <lardon3d/project.h>
#include <lardon3d/resource_governor.h>
#include <lardon3d/resource_snapshot.h>
#include <lardon3d/task_queue.h> #include <lardon3d/task_queue.h>
#include <lardon3d/tui.h> #include <lardon3d/tui.h>
@ -180,6 +182,23 @@ redraw(
DISPLAYED_TASK_CAPACITY, DISPLAYED_TASK_CAPACITY,
&task_summary &task_summary
); );
Lardon3DResourceSnapshot resource_snapshot;
Lardon3DResourceAvailability resource_availability;
const Lardon3DResourceAvailability *displayed_resources = NULL;
if (state->screen == LARDON3D_SCREEN_RESOURCES
&& lardon3d_resource_snapshot_capture(
&state->hardware_profile,
&resource_snapshot,
NULL,
0
)
&& lardon3d_resource_governor_availability(
state->resource_governor,
&resource_snapshot,
&resource_availability
)) {
displayed_resources = &resource_availability;
}
lardon3d_layout_draw( lardon3d_layout_draw(
state, state,
text, text,
@ -188,6 +207,7 @@ redraw(
task_snapshots, task_snapshots,
task_count, task_count,
&task_summary, &task_summary,
displayed_resources,
rows, rows,
columns columns
); );
@ -412,6 +432,9 @@ handle_normal_input(
case KEY_F(5): case KEY_F(5):
state->screen = LARDON3D_SCREEN_TASKS; state->screen = LARDON3D_SCREEN_TASKS;
return true; return true;
case KEY_F(6):
state->screen = LARDON3D_SCREEN_RESOURCES;
return true;
case 27: case 27:
state->screen = LARDON3D_SCREEN_HOME; state->screen = LARDON3D_SCREEN_HOME;
return true; return true;
@ -651,7 +674,8 @@ lardon3d_tui_run(Lardon3DAppState *state)
int key = getch(); int key = getch();
bool should_redraw = task != NULL bool should_redraw = task != NULL
|| state->screen == LARDON3D_SCREEN_TASKS; || state->screen == LARDON3D_SCREEN_TASKS
|| state->screen == LARDON3D_SCREEN_RESOURCES;
if (task && lardon3d_import_task_is_finished(task)) { if (task && lardon3d_import_task_is_finished(task)) {
Lardon3DImportTaskSnapshot snapshot; Lardon3DImportTaskSnapshot snapshot;
if (!lardon3d_import_task_join(task) if (!lardon3d_import_task_join(task)

11
tests/test_resource_governor.c
View file

@ -39,6 +39,13 @@ decide_repeatedly(void *argument)
) )
|| decision.kind != LARDON3D_RESOURCE_START || decision.kind != LARDON3D_RESOURCE_START
|| decision.batch_size != context->request.preferred_batch_size) { || decision.batch_size != context->request.preferred_batch_size) {
(void)fprintf(
stderr,
"Décision concurrente : kind=%d batch=%zu cpu=%u\n",
(int)decision.kind,
decision.batch_size,
decision.cpu_threads
);
atomic_store(context->failed, true); atomic_store(context->failed, true);
break; break;
} }
@ -65,6 +72,7 @@ run_test(void)
.system_cpu_reserve = 1, .system_cpu_reserve = 1,
.maximum_cpu_load_ratio = 0.90, .maximum_cpu_load_ratio = 0.90,
.maximum_io_pressure_avg10 = 80.0, .maximum_io_pressure_avg10 = 80.0,
.io_slot_capacity = 8,
}; };
CHECK(!lardon3d_resource_governor_create(NULL, &policy)); CHECK(!lardon3d_resource_governor_create(NULL, &policy));
Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create( Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create(
@ -89,7 +97,7 @@ run_test(void)
&request, &request,
&decision &decision
)); ));
CHECK(decision.kind == LARDON3D_RESOURCE_START); CHECK(decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH);
CHECK(decision.batch_size == 8); CHECK(decision.batch_size == 8);
CHECK(decision.cpu_threads == 15); CHECK(decision.cpu_threads == 15);
CHECK(decision.reason[0]); CHECK(decision.reason[0]);
@ -134,6 +142,7 @@ run_test(void)
profile.gpu_available = true; profile.gpu_available = true;
profile.gpu_uses_shared_memory = true; profile.gpu_uses_shared_memory = true;
policy.gpu_memory_reserve_bytes = 0; policy.gpu_memory_reserve_bytes = 0;
policy.gpu_slot_capacity = 2;
governor = lardon3d_resource_governor_create(&profile, &policy); governor = lardon3d_resource_governor_create(&profile, &policy);
CHECK(governor); CHECK(governor);
request = (Lardon3DResourceRequest) { request = (Lardon3DResourceRequest) {

499
tests/test_resource_reservation.c Normal file
View file

@ -0,0 +1,499 @@
#include <pthread.h>
#include <stdatomic.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <lardon3d/resource_governor.h>
#define CHECK(condition) \
do { \
if (!(condition)) { \
(void)fprintf(stderr, "Échec ligne %d : %s\n", __LINE__, #condition); \
return false; \
} \
} while (0)
#define GIBIBYTES(value) ((uint64_t)(value) * 1024 * 1024 * 1024)
#define MEBIBYTES(value) ((uint64_t)(value) * 1024 * 1024)
enum {
THREAD_COUNT = 16,
RESERVATIONS_PER_THREAD = 25,
RESERVATION_COUNT = THREAD_COUNT * RESERVATIONS_PER_THREAD,
};
typedef struct {
Lardon3DResourceGovernor *governor;
Lardon3DResourceSnapshot snapshot;
Lardon3DResourceEstimate estimate;
Lardon3DResourceReservation **reservations;
size_t first;
pthread_barrier_t *created;
pthread_barrier_t *release;
atomic_bool *failed;
} ReservationThread;
static Lardon3DResourcePolicy
test_policy(unsigned int io_slots)
{
return (Lardon3DResourcePolicy) {
.system_memory_reserve_bytes = 0,
.gpu_memory_reserve_bytes = 0,
.system_cpu_reserve = 0,
.maximum_cpu_load_ratio = 1.0,
.maximum_io_pressure_avg10 = 100.0,
.gpu_slot_capacity = 0,
.io_slot_capacity = io_slots,
};
}
static void *
reserve_concurrently(void *argument)
{
ReservationThread *thread = argument;
for (size_t offset = 0; offset < RESERVATIONS_PER_THREAD; ++offset) {
Lardon3DResourceDecision decision;
size_t index = thread->first + offset;
if (!lardon3d_resource_governor_reserve(
thread->governor,
&thread->snapshot,
&thread->estimate,
&decision,
&thread->reservations[index]
)
|| !thread->reservations[index]
|| decision.kind != LARDON3D_RESOURCE_START) {
atomic_store(thread->failed, true);
}
}
(void)pthread_barrier_wait(thread->created);
(void)pthread_barrier_wait(thread->release);
for (size_t offset = 0; offset < RESERVATIONS_PER_THREAD; ++offset) {
Lardon3DResourceReservation *reservation =
thread->reservations[thread->first + offset];
if (!reservation
|| !lardon3d_resource_governor_release(
thread->governor,
reservation
)
|| lardon3d_resource_governor_release(
thread->governor,
reservation
)) {
atomic_store(thread->failed, true);
}
}
return NULL;
}
static bool
test_adaptive_batches(void)
{
Lardon3DHardwareProfile profile = {
.logical_cpu_count = 64,
.page_size_bytes = 4096,
.memory_total_bytes = GIBIBYTES(64),
.cpu_architecture = "test",
};
Lardon3DResourcePolicy policy = test_policy(64);
Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create(
&profile,
&policy
);
CHECK(governor);
Lardon3DResourceSnapshot snapshot = {
.memory_available_bytes = GIBIBYTES(37),
.cpu_load_1m = 0.0,
};
Lardon3DResourceEstimate estimate = {
.memory_bytes_per_item = GIBIBYTES(1),
.minimum_batch_size = 1,
.maximum_batch_size = 128,
.desired_cpu_threads = 8,
.desired_io_slots = 2,
.task_class = LARDON3D_RESOURCE_TASK_MIXED,
};
Lardon3DResourceDecision decision;
Lardon3DResourceReservation *reservation;
CHECK(lardon3d_resource_governor_reserve(
governor,
&snapshot,
&estimate,
&decision,
&reservation
));
CHECK(reservation);
CHECK(decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH);
CHECK(decision.batch_size == 37);
CHECK(lardon3d_resource_governor_reservation_is_valid(
governor,
reservation
));
Lardon3DResourceReservationInfo information;
CHECK(lardon3d_resource_reservation_get(
governor,
reservation,
&information
));
CHECK(information.id == 1);
CHECK(information.memory_bytes == GIBIBYTES(37));
CHECK(information.batch_size == 37);
CHECK(information.cpu_threads == 8);
CHECK(information.io_slots == 2);
CHECK(information.state == LARDON3D_RESERVATION_ACTIVE);
CHECK(information.created_at.tv_sec > 0);
CHECK(lardon3d_resource_governor_reservation_count(governor) == 1);
Lardon3DResourceAvailability availability;
CHECK(lardon3d_resource_governor_availability(
governor,
&snapshot,
&availability
));
CHECK(availability.memory_reserved_bytes == GIBIBYTES(37));
CHECK(availability.memory_available_bytes == 0);
CHECK(availability.cpu_reserved == 8);
CHECK(availability.io_slots_reserved == 2);
CHECK(availability.active_reservations == 1);
Lardon3DResourcePolicy reduced_policy = policy;
reduced_policy.system_memory_reserve_bytes = GIBIBYTES(28);
CHECK(!lardon3d_resource_governor_set_policy(
governor,
&reduced_policy
));
Lardon3DResourceReservation *second = NULL;
CHECK(lardon3d_resource_governor_reserve(
governor,
&snapshot,
&estimate,
&decision,
&second
));
CHECK(!second);
CHECK(decision.kind == LARDON3D_RESOURCE_WAIT);
CHECK(lardon3d_resource_governor_release(governor, reservation));
CHECK(!lardon3d_resource_governor_release(governor, reservation));
CHECK(!lardon3d_resource_governor_reservation_is_valid(
governor,
reservation
));
CHECK(lardon3d_resource_reservation_get(
governor,
reservation,
&information
));
CHECK(information.state == LARDON3D_RESERVATION_RELEASED);
CHECK(lardon3d_resource_governor_reservation_count(governor) == 0);
CHECK(lardon3d_resource_governor_availability(
governor,
&snapshot,
&availability
));
CHECK(availability.memory_available_bytes == GIBIBYTES(37));
CHECK(lardon3d_resource_governor_set_policy(
governor,
&reduced_policy
));
estimate = (Lardon3DResourceEstimate) {
.memory_fixed_bytes = GIBIBYTES(65),
.minimum_batch_size = 1,
.maximum_batch_size = 1,
.desired_cpu_threads = 1,
.task_class = LARDON3D_RESOURCE_TASK_GENERAL,
};
CHECK(lardon3d_resource_governor_reserve(
governor,
&snapshot,
&estimate,
&decision,
&second
));
CHECK(!second);
CHECK(decision.kind == LARDON3D_RESOURCE_REJECT);
lardon3d_resource_governor_destroy(governor);
return true;
}
static bool
test_multiple_reservations(void)
{
Lardon3DHardwareProfile profile = {
.logical_cpu_count = 16,
.page_size_bytes = 4096,
.memory_total_bytes = GIBIBYTES(16),
.cpu_architecture = "test",
};
Lardon3DResourcePolicy policy = test_policy(16);
Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create(
&profile,
&policy
);
CHECK(governor);
Lardon3DResourceSnapshot snapshot = {
.memory_available_bytes = GIBIBYTES(8),
.cpu_load_1m = 0.0,
};
Lardon3DResourceEstimate fixed = {
.memory_fixed_bytes = GIBIBYTES(3),
.minimum_batch_size = 1,
.maximum_batch_size = 1,
.desired_cpu_threads = 2,
.desired_io_slots = 1,
.task_class = LARDON3D_RESOURCE_TASK_IO,
};
Lardon3DResourceEstimate variable = {
.memory_bytes_per_item = GIBIBYTES(1),
.minimum_batch_size = 1,
.maximum_batch_size = 6,
.desired_cpu_threads = 2,
.desired_io_slots = 1,
.task_class = LARDON3D_RESOURCE_TASK_IO,
};
Lardon3DResourceDecision decision;
Lardon3DResourceReservation *first;
Lardon3DResourceReservation *second;
CHECK(lardon3d_resource_governor_reserve(
governor,
&snapshot,
&fixed,
&decision,
&first
));
CHECK(first && decision.kind == LARDON3D_RESOURCE_START);
CHECK(lardon3d_resource_governor_reserve(
governor,
&snapshot,
&variable,
&decision,
&second
));
CHECK(second && decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH);
CHECK(decision.batch_size == 5);
CHECK(lardon3d_resource_governor_reservation_count(governor) == 2);
Lardon3DResourceReservationInfo listed[2];
CHECK(lardon3d_resource_governor_list_reservations(
governor,
listed,
2
) == 2);
CHECK(listed[0].id != listed[1].id);
CHECK(lardon3d_resource_governor_release(governor, first));
CHECK(lardon3d_resource_governor_release(governor, second));
lardon3d_resource_governor_destroy(governor);
return true;
}
static bool
test_gpu_reservation(void)
{
Lardon3DHardwareProfile profile = {
.logical_cpu_count = 16,
.page_size_bytes = 4096,
.memory_total_bytes = GIBIBYTES(16),
.gpu_available = true,
.gpu_memory_known = true,
.gpu_memory_total_bytes = GIBIBYTES(4),
.cpu_architecture = "test",
};
Lardon3DResourcePolicy policy = test_policy(4);
policy.gpu_memory_reserve_bytes = MEBIBYTES(512);
policy.gpu_slot_capacity = 2;
Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create(
&profile,
&policy
);
Lardon3DResourceGovernor *other = lardon3d_resource_governor_create(
&profile,
&policy
);
CHECK(governor && other);
Lardon3DResourceSnapshot snapshot = {
.memory_available_bytes = GIBIBYTES(16),
.gpu_memory_available_known = true,
.gpu_memory_available_bytes = GIBIBYTES(3) + MEBIBYTES(512),
.cpu_load_1m = 0.0,
};
Lardon3DResourceEstimate estimate = {
.gpu_memory_fixed_bytes = MEBIBYTES(512),
.gpu_memory_bytes_per_item = MEBIBYTES(512),
.minimum_batch_size = 1,
.maximum_batch_size = 6,
.desired_cpu_threads = 2,
.desired_gpu_slots = 1,
.task_class = LARDON3D_RESOURCE_TASK_GPU,
};
Lardon3DResourceDecision decision;
Lardon3DResourceReservation *reservation;
CHECK(lardon3d_resource_governor_reserve(
governor,
&snapshot,
&estimate,
&decision,
&reservation
));
CHECK(reservation && decision.kind == LARDON3D_RESOURCE_REDUCE_BATCH);
CHECK(decision.batch_size == 5);
Lardon3DResourceReservationInfo information;
CHECK(lardon3d_resource_reservation_get(
governor,
reservation,
&information
));
CHECK(information.gpu_memory_bytes == GIBIBYTES(3));
CHECK(information.gpu_slots == 1);
Lardon3DResourceAvailability availability;
CHECK(lardon3d_resource_governor_availability(
governor,
&snapshot,
&availability
));
CHECK(availability.gpu_memory_reserved_bytes == GIBIBYTES(3));
CHECK(availability.gpu_memory_available_bytes == 0);
CHECK(availability.gpu_slots_available == 1);
CHECK(!lardon3d_resource_governor_reservation_is_valid(
other,
reservation
));
CHECK(!lardon3d_resource_governor_release(other, reservation));
CHECK(lardon3d_resource_governor_release(governor, reservation));
lardon3d_resource_governor_destroy(other);
lardon3d_resource_governor_destroy(governor);
return true;
}
static bool
test_concurrency(void)
{
Lardon3DHardwareProfile profile = {
.logical_cpu_count = 512,
.page_size_bytes = 4096,
.memory_total_bytes = GIBIBYTES(4),
.cpu_architecture = "test",
};
Lardon3DResourcePolicy policy = test_policy(512);
Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create(
&profile,
&policy
);
CHECK(governor);
Lardon3DResourceReservation *reservations[RESERVATION_COUNT] = {0};
pthread_barrier_t created;
pthread_barrier_t release;
CHECK(pthread_barrier_init(&created, NULL, THREAD_COUNT + 1) == 0);
CHECK(pthread_barrier_init(&release, NULL, THREAD_COUNT + 1) == 0);
atomic_bool failed = false;
ReservationThread contexts[THREAD_COUNT];
pthread_t threads[THREAD_COUNT];
for (size_t index = 0; index < THREAD_COUNT; ++index) {
contexts[index] = (ReservationThread) {
.governor = governor,
.snapshot = {
.memory_available_bytes = GIBIBYTES(4),
.cpu_load_1m = 0.0,
},
.estimate = {
.memory_fixed_bytes = MEBIBYTES(1),
.minimum_batch_size = 1,
.maximum_batch_size = 1,
.desired_cpu_threads = 1,
.desired_io_slots = 1,
.task_class = LARDON3D_RESOURCE_TASK_GENERAL,
},
.reservations = reservations,
.first = index * RESERVATIONS_PER_THREAD,
.created = &created,
.release = &release,
.failed = &failed,
};
CHECK(pthread_create(
&threads[index],
NULL,
reserve_concurrently,
&contexts[index]
) == 0);
}
(void)pthread_barrier_wait(&created);
CHECK(!atomic_load(&failed));
CHECK(lardon3d_resource_governor_reservation_count(governor)
== RESERVATION_COUNT);
Lardon3DResourceAvailability availability;
CHECK(lardon3d_resource_governor_availability(
governor,
&contexts[0].snapshot,
&availability
));
CHECK(availability.memory_reserved_bytes
== MEBIBYTES(RESERVATION_COUNT));
CHECK(availability.cpu_reserved == RESERVATION_COUNT);
CHECK(availability.io_slots_reserved == RESERVATION_COUNT);
(void)pthread_barrier_wait(&release);
for (size_t index = 0; index < THREAD_COUNT; ++index) {
CHECK(pthread_join(threads[index], NULL) == 0);
}
CHECK(!atomic_load(&failed));
CHECK(lardon3d_resource_governor_reservation_count(governor) == 0);
CHECK(pthread_barrier_destroy(&created) == 0);
CHECK(pthread_barrier_destroy(&release) == 0);
lardon3d_resource_governor_destroy(governor);
return true;
}
static bool
test_destroy_with_active_reservations(void)
{
Lardon3DHardwareProfile profile = {
.logical_cpu_count = 4,
.page_size_bytes = 4096,
.memory_total_bytes = GIBIBYTES(4),
.cpu_architecture = "test",
};
Lardon3DResourcePolicy policy = test_policy(4);
Lardon3DResourceGovernor *governor = lardon3d_resource_governor_create(
&profile,
&policy
);
CHECK(governor);
Lardon3DResourceSnapshot snapshot = {
.memory_available_bytes = GIBIBYTES(4),
.cpu_load_1m = 0.0,
};
Lardon3DResourceEstimate estimate = {
.memory_fixed_bytes = MEBIBYTES(1),
.minimum_batch_size = 1,
.maximum_batch_size = 1,
.desired_cpu_threads = 1,
.task_class = LARDON3D_RESOURCE_TASK_GENERAL,
};
Lardon3DResourceDecision decision;
Lardon3DResourceReservation *reservation;
CHECK(lardon3d_resource_governor_reserve(
governor,
&snapshot,
&estimate,
&decision,
&reservation
));
CHECK(reservation);
lardon3d_resource_governor_destroy(governor);
return true;
}
static bool
run_test(void)
{
CHECK(test_adaptive_batches());
CHECK(test_multiple_reservations());
CHECK(test_gpu_reservation());
CHECK(test_concurrency());
CHECK(test_destroy_with_active_reservations());
return true;
}
int
main(void)
{
return run_test() ? EXIT_SUCCESS : EXIT_FAILURE;
}