lardon3d/docs/architecture/visual_index.md

Visual Index v1

Status

VISUAL_INDEX_V1=IMPLEMENTED
VISUAL_INDEX_KIND=orb-lsh
VISUAL_INDEX_VERSION=1

CANDIDATE_PAIR=IMPLEMENTED
MATCHER=IMPLEMENTED

VISUAL_INDEX_GPU=REJECTED_WITH_MEASURED_REASON
CURRENT_PROJECT_DB_SCHEMA=v25
REAL_A6000_PRE_SFM=PASS/FROZEN

Visual Index turns a homogeneous collection of immutable Feature Sets into bounded image-retrieval candidates.

It is a retrieval stage, not a Matcher and not a geometric verifier.

Current downstream consumers are implemented:

Feature Store
-> Visual Index
-> Candidate Pair Generator
-> Matcher
-> Geometric Verification
-> Tracks

Older text describing Candidate Pair or Matcher as future consumers is historical design context and is not current status.

Algorithm

Visual Index v1 uses deterministic binary LSH over ORB descriptors.

It uses six tables.

Each table selects 24 distinct positions from the 256 ORB bits.

The frozen v1 position rule is:

position = (41 * table + 11 * bit) mod 256

A posting key is:

(table_id, key24)

Changing this bit-selection policy requires a new Visual Index scientific version.

Identity and configuration

Current kind/version:

orb-lsh / 1

One index contains Feature Sets with homogeneous:

• descriptor type;
• descriptor dimension;
• extractor kind;
• extractor version;
• extractor parameter fingerprint.

Frozen v1 configuration contains:

table_count = 6
key_bits = 24
max_features_per_set = 1..1024, default 512
max_bucket_postings = 1..4096, default 256
max_segments = 256
max_feature_sets_per_segment = 16

The canonical parameter fingerprint uses domain:

L3DVICF1

with explicit little-endian fields.

No C struct padding, locale or host endianness enters the fingerprint.

Sampling

At most max_features_per_set Feature entries are indexed.

V1 selects the increasing feature_index prefix.

Postings retain the original Feature index.

An empty Feature Set is a valid member and contributes no posting.

Segment persistence

Project DB v6 introduced Visual Index persistence. v7 retained the model.

Later schema versions through v25 do not reinterpret Visual Index v1.

A logical index owns immutable READY segments.

One update publishes one segment containing between one and sixteen new Feature Sets.

Membership uniqueness is enforced on:

(visual_index_id, feature_set_id)

A query snapshots the bounded READY segment list before asset reads.

A segment committed after that snapshot is visible to the next query, not retroactively injected into the running query.

Capacity

V1 currently allows:

max_segments = 256
max_feature_sets_per_segment = 16

Therefore one v1 index can contain exactly up to:

4096 Feature Sets

before another update returns the Visual Index limit.

This is an index-v1 capacity bound, not a project-wide image-count limit.

Compaction/base-delta redesign remains deferred.

Segment File v1

Segment File v1 is explicitly little-endian and does not serialize C structs.

Magic:

L3DVIDX\0

A posting contains:

table_id:u32
key24:u32
feature_set_id:u64
feature_index:u32
reserved_zero:u32

Canonical persistent ordering is:

table_id
key24
feature_set_id
feature_index

The complete Segment File is content-addressed by SHA-256 under the Visual Index asset tree.

Publication

Publication follows the normal immutable-asset pattern:

local temp
-> write
-> fsync
-> hash
-> no-overwrite publication/adoption validation
-> fsync directory
-> short Project DB transaction

A physical file may remain orphaned if DB publication fails after the file is published.

No partially committed READY segment is invented.

Durability distinguishes:

DURABLE
PUBLISHED_NOT_DURABLE

Reader validation

The reader validates the asset SHA before trusting the format.

It rejects:

• invalid counts;
• invalid offsets;
• overflow;
• malformed reserved fields;
• inconsistent DB metadata;
• unsupported future version.

A coherent future format version is UNSUPPORTED_VERSION, not generic corruption.

Query

Query identity is centered on:

(visual_index_id, query_feature_set_id)

Options include:

• ScanSet filter;
• same/other ScanSet policy;
• source-asset exclusion;
• minimum evidence count;
• top_k in 1..256.

The source Feature Set/image is never returned as its own candidate.

Evidence and score

One evidence unit is one distinct query feature_index that collides with the candidate.

Multiple tables or candidate postings do not multiply the same query-feature evidence.

Score:

evidence_count / sampled_query_feature_count

Range:

0..1

Canonical result order:

score descending
evidence_count descending
image_id ascending
feature_set_id ascending

Visual Index score is retrieval evidence only.

It is not descriptor-match evidence and not geometric evidence.

Burstiness bound

Bucket frequency is bounded across the retained query snapshot.

A bucket above max_bucket_postings is ignored.

This prevents common patterns from dominating score or creating unbounded posting accumulation.

The query accumulator is bounded to 4096 candidates and 256 returned results.

No global query cache is required.

Durable Task

Task Kind:

visual_index.update/1

Durable cursor:

after_feature_set_id

One sequence handles a bounded admitted set of new Feature Sets, publishes a complete segment, commits memberships, advances the cursor/checkpoint and returns through sequence_break() if more work remains.

Restart resumes from the durable cursor and membership uniqueness makes replay idempotent.

Internal parallelism

The Queue owns one active heavy callback.

Visual Index may use bounded internal CPU participants inside that callback.

Current validated shape:

CPU up to 16
batch/window 1..16
GPU 0
fixed RAM approximately 8 MiB
per-item RAM approximately 2 MiB

Each participant reads immutable Feature data into private work.

Participants do not publish the segment.

After join, the owner performs canonical total ordering, serialization, asset publication and Project DB commit.

Thread-creation failure may fall back to owner computation of that slice without changing output.

Determinism

The following must match the serial scientific result:

• posting set;
• posting order;
• Segment File bytes;
• SHA-256;
• membership set;
• generation ordering;
• query results.

Operational CPU width does not enter scientific identity.

GPU policy

Current GPU classification:

VISUAL_INDEX_GPU=REJECTED_WITH_MEASURED_REASON

The stage is dominated by posting construction, total ordering, hashing and deterministic publication, and there is no validated production GPU seam that preserves the full contract with useful measured benefit.

This does not authorize avoidable CPU serialism.

RESOURCE_UTILIZATION_POLICY=MAXIMUM_SAFE_USEFUL_THROUGHPUT
SERIALISM_REQUIRES_PROOF=CANONICAL

Current downstream relationship

Candidate Pair Generator is implemented and consumes Visual Index queries.

Matcher is implemented and consumes persisted Candidate Pairs.

Therefore current relationship is:

Visual Index
-> Candidate Pair Generator
-> Candidate Pair persistence
-> Matcher

Visual Index does not pass raw feature_set_id + feature_index pairs directly into a hypothetical future Matcher.

The persisted Candidate Pair boundary remains explicit.

Real A6000 evidence

The retained A6000 proof contains:

Feature Sets     689
Candidate Pairs  38,420
Match Results    38,420

Final continuation replayed no new Visual Index work.

Checkpoint:

real-a6000-pre-sfm-2026-09-02
REAL_A6000_PRE_SFM=PASS/FROZEN

This confirms the current Visual Index path was already durably reusable before downstream GV/Tracks continuation.

Limits

Current v1 limits/non-goals include:

• no segment compaction;
• one index limited to 4096 Feature Sets;
• no GPU backend;
• no geometric meaning assigned to retrieval score;
• no dense project-wide pair matrix.

A future index version may change capacity or data structure only through an explicit versioned scientific/persistence decision.

Summary

VISUAL_INDEX_V1=IMPLEMENTED
VISUAL_INDEX_KIND=orb-lsh
VISUAL_INDEX_VERSION=1
VISUAL_INDEX_CAPACITY=4096_FEATURE_SETS
VISUAL_INDEX_SEGMENT_MEMBERS=16
VISUAL_INDEX_TOP_K_MAX=256

VISUAL_INDEX_TASK=visual_index.update/1
VISUAL_INDEX_GPU=REJECTED_WITH_MEASURED_REASON

CANDIDATE_PAIR=IMPLEMENTED
MATCHER=IMPLEMENTED

CURRENT_PROJECT_DB_SCHEMA=v25
REAL_A6000_PRE_SFM=PASS/FROZEN