Compare commits
4 commits
835d96f8c4
...
af7ecdf466
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
af7ecdf466 | ||
|
|
ca3170d7fe | ||
|
|
a0de28092e | ||
|
|
1150a61ccc |
5 changed files with 1351 additions and 50 deletions
|
|
@ -125,8 +125,8 @@ are inferred from focal pixels, image resolution or baseline normalization.
|
|||
|
||||
## Reconstruction strategy
|
||||
|
||||
**DECISION: incremental SfM with bounded local refinement and optional final
|
||||
global refinement.** It matches the expected sequential vehicle/phone capture,
|
||||
**DECISION: incremental SfM followed by final per-component refinement.** It
|
||||
matches the expected sequential vehicle/phone capture,
|
||||
allows unregistered images to remain visible as a scientific result, and keeps
|
||||
the active problem bounded. Global-only rotation/translation averaging would
|
||||
add a larger initialization and robustness surface without a current project
|
||||
|
|
@ -192,27 +192,382 @@ only on geometry; multi-view Tracks use all valid observations rather than a
|
|||
random pair. Robust observation dropping is deferred: v1 rejects the landmark
|
||||
as a whole, so Track identity and observation ownership remain simple.
|
||||
|
||||
## Bundle Adjustment decision
|
||||
## Gate E v1 — Final Bundle Adjustment decision
|
||||
|
||||
**DECISION: BA is required for a useful final reconstruction but is not part of
|
||||
the first pure-geometry gate.** The later BA gate will use a block-sparse
|
||||
camera/landmark problem with binary64 poses and points, fixed or explicitly
|
||||
fingerprinted calibration variables, and a robust loss whose kind/scale belong
|
||||
to scientific identity. Dense camera×landmark Jacobians are forbidden.
|
||||
**DECISION: Gate E v1 is a synchronous, independent final per-component Bundle
|
||||
Adjustment applied as post-processing to a copy of the immutable final Gate D
|
||||
result.** It consumes two caller-owned immutable views that must remain coherent
|
||||
for the complete call: that final Gate D result, and the same resolved
|
||||
observation/calibration view used to construct the scientific Gate D input. It
|
||||
never mutates either view, never creates constraints between disconnected
|
||||
components, preserves each component's independent gauge, and produces a
|
||||
distinct in-memory BA result.
|
||||
|
||||
Ceres is not available in the current host pkg-config environment and is not a
|
||||
Lardon3D production dependency. It is a **NEW_CANDIDATE**, not silently added.
|
||||
Its sparse Schur solvers and block structure make it the leading BA study
|
||||
candidate; Eigen is host-available, while SuiteSparse is not detected. A later
|
||||
gate must prove license, reproducibility, thread behavior, memory scaling and
|
||||
fallback before adding Ceres. OpenCV remains appropriate for small relative
|
||||
pose/PnP/triangulation primitives, not as an implicit BA architecture.
|
||||
The Gate D result alone is authoritative for final components, registered
|
||||
cameras, initial poses, landmarks and the observations associated with each
|
||||
landmark. The second view only resolves an observation already published by
|
||||
Gate D. Its canonical key is `(feature_set_id, feature_index)`; resolution must
|
||||
return the matching `image_id`, source keypoint `x,y` and immutable calibration,
|
||||
and must also agree with the published Track and image identities. Missing,
|
||||
ambiguous, duplicate or inconsistent resolution is a Gate E input error. Gate E
|
||||
must not use array position, proximity or another heuristic fallback, add an
|
||||
observation, restore a rejected association or camera, or rerun incremental SfM.
|
||||
|
||||
The first BA implementation should be local-window BA after registration,
|
||||
followed by at most one explicitly admitted global BA at finalization. No two
|
||||
heavy global BAs run concurrently. Global BA is allowed to be deferred by the
|
||||
Resource Governor. The trigger, window selection, robust loss, convergence and
|
||||
thread policy are configuration fields, not runtime identity.
|
||||
Source keypoint coordinates are the Feature File binary32 `x,y` in decoded-image
|
||||
pixels, with top-left origin, +x right and +y down. Gate E converts them to
|
||||
binary64 for computation; it does not treat them as already undistorted or
|
||||
normalized. Given `Xc = R_cw * Xw + t_cw`, define `xn = Xc.x / Xc.z`,
|
||||
`yn = Xc.y / Xc.z`, and `r2 = xn*xn + yn*yn`. The canonical OpenCV-compatible
|
||||
forward model is:
|
||||
|
||||
```text
|
||||
radial = 1 + k1*r2 + k2*r2*r2
|
||||
xd = xn*radial + 2*p1*xn*yn + p2*(r2 + 2*xn*xn)
|
||||
yd = yn*radial + p1*(r2 + 2*yn*yn) + 2*p2*xn*yn
|
||||
u = fx*xd + cx
|
||||
v = fy*yd + cy
|
||||
residual = [u - observed_x, v - observed_y]
|
||||
```
|
||||
|
||||
The residual is therefore binary64 in source pixels and uses the complete
|
||||
canonical calibration model. A private Gate D validation helper that omits
|
||||
distortion does not redefine this contract and is not a precedent for Gate E.
|
||||
The second immutable view is an explicit scientific input, not persistence,
|
||||
Project DB integration, a loader, resolver subsystem, cache, handle or Resource
|
||||
System.
|
||||
|
||||
### Scientific and numerical contract
|
||||
|
||||
Gate E processes every reconstructed Gate D component independently. It
|
||||
resolves the selected observations, copies the component poses and landmarks
|
||||
into a private working set, builds and solves one BA problem, validates the
|
||||
complete candidate, then either publishes that candidate in the distinct Gate
|
||||
E result or preserves the original Gate D component. No component constrains or
|
||||
influences another component.
|
||||
|
||||
Gate E v1 optimizes only camera extrinsic rotations, camera centers and
|
||||
landmark positions. The known `fx`, `fy`, `cx`, `cy`, `k1`, `k2`, `p1`, `p2`,
|
||||
observations, Track membership, identities and observation/landmark
|
||||
associations are fixed and immutable. Future intrinsic optimization requires a
|
||||
separate scientific and identity decision.
|
||||
|
||||
The public boundary remains solver-independent and world-to-camera. The private
|
||||
C++ adapter uses a unit quaternion for `R_cw`, with an appropriate quaternion
|
||||
manifold, and world camera center `Cw`:
|
||||
|
||||
```text
|
||||
Xc = R_cw * (Xw - Cw)
|
||||
t_cw = -R_cw * Cw
|
||||
```
|
||||
|
||||
Conversion to or from public rotation matrices canonicalizes quaternion sign,
|
||||
so `q` and `-q` cannot produce distinct observable representations. No Ceres
|
||||
type crosses the future C17 ABI.
|
||||
|
||||
### Gate E gauge
|
||||
|
||||
Gate E derives deterministic BA anchors from the final Gate D result and does
|
||||
not depend on historical seed IDs. In each component, the registered camera
|
||||
with the lowest `image_id` is the pose anchor; its complete initial Gate D
|
||||
rotation and camera center are fixed.
|
||||
|
||||
Among the other registered cameras, the scale anchor is the camera whose
|
||||
binary64 Euclidean distance from the pose anchor is greatest. An exact distance
|
||||
tie selects the lowest `image_id`; no hidden tolerance participates. For
|
||||
`delta = C_scale - C_anchor`, the coordinate with greatest absolute value is
|
||||
the scale axis, with exact ties resolved X, then Y, then Z. That one initial
|
||||
Gate D coordinate of `C_scale` is fixed. Its other two center coordinates and
|
||||
its rotation remain variable. The fixed pose removes the six rigid degrees of
|
||||
freedom and the fixed nonzero scale coordinate removes the scale degree of
|
||||
freedom without fixing a second pose.
|
||||
|
||||
The scale anchor is degenerate when:
|
||||
|
||||
```text
|
||||
max(abs(delta.x), abs(delta.y), abs(delta.z)) <= 1e-9
|
||||
```
|
||||
|
||||
Gate D fixes each valid component to a unit seed baseline, so `1e-9` world
|
||||
units is a numerically negligible separation in that scientific gauge. A
|
||||
component with no second valid camera or a degenerate scale anchor is not
|
||||
optimized; its Gate D data is retained with a gauge/degenerate diagnostic.
|
||||
|
||||
### Objective and solver
|
||||
|
||||
Every valid observation contributes one two-dimensional source-pixel residual
|
||||
block `f_i = [dx, dy]`, where `dx = predicted_x - observed_x` and
|
||||
`dy = predicted_y - observed_y`. Gate E uses binary64 throughout. Define:
|
||||
|
||||
```text
|
||||
s_i = dx*dx + dy*dy
|
||||
delta = 2.0
|
||||
delta2 = 4.0
|
||||
|
||||
rho_delta(s) = s if s <= delta2
|
||||
rho_delta(s) = 2*delta*sqrt(s) - delta2 if s > delta2
|
||||
|
||||
robust_cost = 0.5 * sum_i(rho_delta(s_i))
|
||||
```
|
||||
|
||||
Thus, with `delta = 2.0` source pixels, the second branch is
|
||||
`4.0*sqrt(s) - 4.0`. The Huber loss applies once to the norm squared of the
|
||||
complete 2D observation, never independently to `dx` and `dy`; the factor
|
||||
`0.5` is contractual. Each observation must likewise be one 2D Ceres residual
|
||||
block, not two scalar blocks.
|
||||
|
||||
Lardon3D computes initial and final robust costs independently of the solver
|
||||
using exactly this formula, and those values govern acceptance. Ceres summary
|
||||
costs may only diagnose or cross-check them. With the identical problem, a
|
||||
disagreement beyond the applicable numerical tolerance stops implementation
|
||||
for contract review; neither value silently replaces the other. Non-finite
|
||||
`dx`, `dy`, `s_i`, `rho_delta(s_i)`, accumulation, pose, landmark or projection,
|
||||
and camera-frame depth invalid under the frozen camera invariants, reject the
|
||||
candidate. No clamp or fallback is permitted.
|
||||
|
||||
The Huber kind and scale are Gate E scientific policy, not Governor parameters,
|
||||
Resource parameters or a Gate E fingerprint.
|
||||
|
||||
Gate E v1 selects the Ceres Solver 2.2.x API, CPU-only, with these explicit
|
||||
options:
|
||||
|
||||
```text
|
||||
minimizer_type = TRUST_REGION
|
||||
trust_region_strategy_type = LEVENBERG_MARQUARDT
|
||||
linear_solver_type = ITERATIVE_SCHUR
|
||||
preconditioner_type = SCHUR_JACOBI
|
||||
num_threads = 1
|
||||
max_num_iterations = 50
|
||||
function_tolerance = 1e-6
|
||||
gradient_tolerance = 1e-10
|
||||
parameter_tolerance = 1e-8
|
||||
```
|
||||
|
||||
Landmark parameter blocks form elimination group 0 in increasing canonical
|
||||
Track/landmark identity; camera blocks form group 1 in increasing `image_id`.
|
||||
Components, cameras, landmarks, observations and residual blocks are all built
|
||||
in canonical identity order. Automatic Ceres ordering is not used when the API
|
||||
accepts an explicit ordering.
|
||||
|
||||
There is exactly one solver attempt per eligible component, with no automatic
|
||||
retry, wall-clock timeout or `max_solver_time`. Environment variables, hardware
|
||||
profiles, Tasks, schedulers and the Governor cannot change the single-thread
|
||||
reference. `ITERATIVE_SCHUR` with `SCHUR_JACOBI` provides the required
|
||||
block-sparse path without a functional SuiteSparse dependency; `SPARSE_SCHUR`,
|
||||
CUDA and GPU execution are not Gate E v1.
|
||||
|
||||
### Eligibility, bounds and acceptance
|
||||
|
||||
An eligible component has at least two registered cameras, at least one valid
|
||||
BA landmark, exactly resolved observations and calibrations, finite inputs, a
|
||||
valid gauge, overflow-safe dimensions and no manifest underconstraint after
|
||||
the anchors. Gate D already guarantees multi-view support for every published
|
||||
landmark, so Gate E introduces no separate support threshold.
|
||||
|
||||
For a component with a valid non-degenerate Gate E gauge, let `C` be its
|
||||
registered camera count, `P` its optimized landmark count and `O` its retained,
|
||||
resolved observation count. Camera intrinsics and distortion are fixed. The
|
||||
free tangent dimension is therefore:
|
||||
|
||||
```text
|
||||
free_dof = 6*C + 3*P - 7
|
||||
scalar_residual_count = 2*O
|
||||
```
|
||||
|
||||
The completely fixed pose anchor removes six degrees of freedom, and the fixed
|
||||
scale-anchor center coordinate removes one. Gate E v1 defines **manifest
|
||||
underconstraint** as at least one of these exact structural conditions:
|
||||
|
||||
- **UC1:** `2*O < 6*C + 3*P - 7`, using overflow-checked integer arithmetic;
|
||||
- **UC2:** an optimized landmark is observed by fewer than two distinct
|
||||
registered cameras;
|
||||
- **UC3:** an optimizable camera, including the scale anchor but excluding the
|
||||
completely fixed pose anchor, observes fewer than three distinct landmarks;
|
||||
- **UC4:** the bipartite camera-landmark optimization graph is not one connected
|
||||
component containing the pose anchor.
|
||||
|
||||
E19 evaluates UC1--UC4 only after the existing structural validation and valid
|
||||
anchor selection. E18 remains the existing insufficient-camera case and is not
|
||||
redefined by E19. These conditions are necessary structural checks, not proof
|
||||
of full numerical rank. Gate E v1 performs no numerical rank estimate, SVD,
|
||||
singular-value or condition-number threshold, Jacobian/Hessian rank epsilon, or
|
||||
Ceres covariance/rank heuristic for E19. Geometry that passes UC1--UC4 can
|
||||
still be rejected by the existing projection, solver termination, finite-value,
|
||||
cost non-regression and atomic-publication contracts.
|
||||
|
||||
UC1--UC4 are deterministic and solver-independent. Their implementation uses
|
||||
the existing canonical flat Gate E working set and temporary storage bounded by
|
||||
`O(C + P + O)` or better. It uses no hash-order dependency, dense `C * P`
|
||||
storage, materialized rank matrix or new Resource subsystem.
|
||||
|
||||
Gate E retains the identically-scoped Gate D bounds of at most 4096 registered
|
||||
cameras, 250,000 Tracks/landmarks and 1,000,000 observations. The Gate D
|
||||
landmarks-per-growth-round bound is not a Gate E bound. All allocation and
|
||||
dimension arithmetic is overflow-checked. The architecture is block-sparse;
|
||||
no dense camera-count × landmark-count allocation or Jacobian is permitted.
|
||||
|
||||
Ceres `NO_CONVERGENCE` is rejection even if an intermediate candidate has
|
||||
lower cost. Only a termination classified as successful convergence by the
|
||||
private Ceres adapter is acceptable. The robust-cost comparison uses exactly:
|
||||
|
||||
```text
|
||||
cost_tolerance = 1e-12 * max(1.0, abs(initial_robust_cost))
|
||||
final_robust_cost <= initial_robust_cost + cost_tolerance
|
||||
```
|
||||
|
||||
This comparison tolerance absorbs insignificant binary64 noise and is not a
|
||||
Ceres convergence tolerance. A component is published only when its inputs are
|
||||
coherent and eligible, termination is accepted, all candidate poses,
|
||||
landmarks, required projections and robust costs are finite, both gauge anchors
|
||||
are strictly preserved in their contract representations, the cost condition
|
||||
holds, and no consumed frozen invariant is violated. Otherwise the original
|
||||
Gate D component is preserved exactly and accompanied by a rejection
|
||||
diagnostic. All optimization occurs on a private copy, so publication is atomic
|
||||
per component and requires no in-place rollback.
|
||||
|
||||
### Result and diagnostics
|
||||
|
||||
The future solver-independent Gate E result has these conceptual states:
|
||||
|
||||
- `COMPLETE`: at least one component is eligible and every eligible component
|
||||
is optimized and accepted;
|
||||
- `PARTIAL`: at least one component is accepted and at least one other eligible
|
||||
component is rejected or fails;
|
||||
- `FAILED`: no eligible component produces an accepted BA result, including an
|
||||
input with no eligible component.
|
||||
|
||||
`Lardon3DSparseBundleAdjustmentStatus` contains only these three scientific
|
||||
result states. In particular, `FAILED` is not an invalid-argument,
|
||||
out-of-memory or internal execution error.
|
||||
|
||||
The future synchronous execution function returns the separate,
|
||||
solver-independent `Lardon3DSparseBundleAdjustmentExecutionStatus`:
|
||||
|
||||
```text
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_EXECUTION_OK
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_EXECUTION_INVALID_ARGUMENT
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_EXECUTION_OUT_OF_MEMORY
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_EXECUTION_INTERNAL_ERROR
|
||||
```
|
||||
|
||||
`EXECUTION_OK` means the public input was structurally valid, Gate E reached a
|
||||
complete scientific decision and produced the owned result. Its scientific
|
||||
status may be `COMPLETE`, `PARTIAL` or `FAILED`; `EXECUTION_OK` with scientific
|
||||
`FAILED` is valid and means that no eligible component was accepted.
|
||||
|
||||
`EXECUTION_INVALID_ARGUMENT` covers a violated public input contract, including
|
||||
pointer/count, bounds, identity, finiteness, observation-resolution or
|
||||
Gate-D/result-view coherence failures. `EXECUTION_OUT_OF_MEMORY` covers an
|
||||
allocation failure, including `std::bad_alloc` caught at the C/C++ boundary,
|
||||
that prevents production of a complete scientific result.
|
||||
`EXECUTION_INTERNAL_ERROR` is reserved for an unexpected internal failure that
|
||||
prevents safe completion; it is not a component-rejection fallback. Normal
|
||||
component rejection for insufficient cameras, gauge degeneracy, manifest
|
||||
underconstraint, invalid candidate projection, solver `NO_CONVERGENCE` or
|
||||
`FAILURE`, a non-finite candidate or robust-cost regression contributes only to
|
||||
the scientific `COMPLETE`/`PARTIAL`/`FAILED` result.
|
||||
|
||||
On every execution status other than `EXECUTION_OK`, the public result remains
|
||||
in its canonical zero state: all counts are zero, all owned array and diagnostic
|
||||
pointers are null, and destruction is safe. The execution function never
|
||||
publishes a partial owned result and then returns an execution error.
|
||||
|
||||
Ineligible and rejected components retain their Gate D data. Each component
|
||||
diagnostic contains at least component key, camera/landmark/observation counts,
|
||||
pose-anchor and scale-anchor `image_id`, scale axis X/Y/Z, initial and final
|
||||
robust cost, initial and final reprojection RMSE, iteration count, solver
|
||||
termination class, accepted/rejected state and rejection reason. It exposes no
|
||||
Ceres pointer or type.
|
||||
|
||||
Diagnostic reprojection RMSE is non-robust:
|
||||
|
||||
```text
|
||||
sqrt(sum(dx*dx + dy*dy) / observation_count)
|
||||
```
|
||||
|
||||
Acceptance remains based on robust cost and all contract invariants. Raw RMSE
|
||||
is not required to improve universally in the presence of outliers.
|
||||
|
||||
### Reproducibility
|
||||
|
||||
For identical input, executable, build, dependency versions and machine with
|
||||
one solver thread, component order, anchors, parameter/residual ordering,
|
||||
states, accept/reject decisions and structural diagnostics are deterministic.
|
||||
Comparable binary64 geometric scalars satisfy:
|
||||
|
||||
```text
|
||||
abs(a - b) <= 1e-12 * max(1.0, abs(a), abs(b))
|
||||
```
|
||||
|
||||
Rotations are compared geometrically rather than by raw quaternion sign. If
|
||||
fresh-process tests in an identical environment cannot meet this tolerance,
|
||||
implementation stops for contract review; tests must not widen it silently.
|
||||
|
||||
### Canonical Gate E validation matrix
|
||||
|
||||
| Case | Contract evidence |
|
||||
|---|---|
|
||||
| E01 Null/invalid input | Safe rejection; no exception crosses C |
|
||||
| E02 Empty/non-eligible result | Deterministic `FAILED` with diagnostics |
|
||||
| E03 Clean synthetic component | Finite accepted result, gauge held, cost non-regression |
|
||||
| E04 Perturbed poses | Fixture-defined measurable improvement |
|
||||
| E05 Perturbed landmarks | Fixture-defined measurable improvement |
|
||||
| E06 Perturbed poses and landmarks | Convergence and fixture-defined improvement |
|
||||
| E07 Noise 0.5 px | Finite accepted result or contractually justified rejection |
|
||||
| E08 Noise 1.0 px | Finite accepted result or contractually justified rejection |
|
||||
| E09 Noise 2.0 px | Finite accepted result or contractually justified rejection |
|
||||
| E10 Outliers 10% | Huber active; finite result or clean rejection; no invariant violation |
|
||||
| E11 Outliers 20% | Huber active; finite result or clean rejection; no invariant violation |
|
||||
| E12 Outliers 40% | Huber active; finite result or clean rejection; no invariant violation |
|
||||
| E13 Disconnected components | Independent optimization and gauges |
|
||||
| E14 One success, one failure | Global `PARTIAL` |
|
||||
| E15 Pose anchor | Initial rotation and center strictly preserved |
|
||||
| E16 Scale anchor | Selected center coordinate strictly preserved |
|
||||
| E17 Deterministic anchors | Exact distance/ID and X/Y/Z ties; `1e-9` degeneracy boundary |
|
||||
| E18 Insufficient cameras | No solve; Gate D data retained |
|
||||
| E19 Underconstrained geometry | No solve; Gate D data retained |
|
||||
| E20 Non-finite input | Input rejection |
|
||||
| E21 Non-finite projection candidate | Atomic candidate rejection |
|
||||
| E22 Forced non-convergence | Private summary interpreter rejects `NO_CONVERGENCE` |
|
||||
| E23 Candidate regression | Cost condition prevents publication |
|
||||
| E24 Atomic rejection | Original component preserved exactly |
|
||||
| E25 Canonical ordering | Explicit groups and parameter/residual order |
|
||||
| E26 Same-process repeats | Structural equality and numeric tolerance |
|
||||
| E27 Fresh-process repeats | At least 20 processes in one identical environment |
|
||||
| E28 Ownership/destruction | Caller inputs retained; owned result safely destroyed |
|
||||
| E29 Null/repeated destroy | Required only if E1 adopts the existing null-safe convention |
|
||||
| E30 Allocation/overflow | Checked rejection before allocation |
|
||||
| E31 Maximum boundary guards | Exact documented limits without a giant solve where isolatable |
|
||||
| E32 Sparse architecture | No dense camera-count × landmark-count allocation |
|
||||
| E33 Calibration immutability | Before/after identical |
|
||||
| E34 Track/observation immutability | Before/after identical |
|
||||
| E35 Gate D immutability | Input unchanged after success and every failure path |
|
||||
|
||||
E22 tests the private solver-summary-to-decision interpreter directly. It does
|
||||
not expose an iteration override, add a production behavior for testing or
|
||||
change `max_num_iterations = 50`. Synthetic ground-truth fixtures measure
|
||||
pre/post geometric error and robust cost. Fixtures intended to improve define
|
||||
their own scientifically measurable improvement; no universal pose or landmark
|
||||
threshold is invented.
|
||||
|
||||
Local BA after registration is deferred. Gate D exposes no intermediate
|
||||
scientific seam or complete registration history, and an interleaved BA could
|
||||
change its subsequent growth. Introducing that policy requires a future
|
||||
explicit scientific seam/version architecture decision; Gate E v1 does not
|
||||
create or name such a version.
|
||||
|
||||
Gate E v1 remains independent of Project DB, Task Runtime, the Resource
|
||||
Governor and any Resource System. It neither computes nor carries a parameter
|
||||
fingerprint, defines no persistent identity, and publishes nothing. Gate F
|
||||
retains project/task orchestration and persistence; Gate G retains Resource
|
||||
Governor integration and final resource validation.
|
||||
|
||||
Ceres availability on a host must be distinguished from Lardon3D dependency
|
||||
declaration. Gate E selects the Ceres Solver 2.2.x API scientifically, but
|
||||
Lardon3D currently declares no production Ceres dependency in Meson. A future
|
||||
dependency slice must verify the used API, licensing, CPU-only construction and
|
||||
a build without required SuiteSparse or CUDA. Package discovery may use CMake;
|
||||
a pkg-config miss alone does not prove host unavailability, and an installed
|
||||
host package is not a declared Lardon3D dependency.
|
||||
|
||||
## Determinism and scientific identity
|
||||
|
||||
|
|
@ -267,13 +622,13 @@ working set. It forbids a dense `C×P`, `C×C` or co-visibility matrix. Track
|
|||
length has no arbitrary 256 cap; long Tracks are iterated through checked
|
||||
bounded storage.
|
||||
|
||||
Triangulation/registration are light CPU units and can be batched. Local BA is
|
||||
bounded by an active camera/landmark window and needs a Governor reservation.
|
||||
Global BA is one heavy job at a time, with explicit admission and a conservative
|
||||
thread cap. Future reservation inputs are `C`, `P`, `O`, active window size,
|
||||
solver mode and calibration-variable count. The existing Resource Governor
|
||||
owns RAM/PSI/swap policy; Sparse SfM adds no thresholds. Swap is never normal
|
||||
working memory, and UMA RAM must preserve several GiB of desktop/iGPU headroom.
|
||||
Triangulation/registration are light CPU units and can be batched. Gate E v1
|
||||
uses local scientific limits for its final per-component BA and does not query
|
||||
the Governor. Future Gate G admission may use `C`, `P`, `O`, solver mode and
|
||||
calibration-variable count without changing scientific results. The existing
|
||||
Resource Governor owns RAM/PSI/swap policy; Sparse SfM adds no system-pressure
|
||||
thresholds. Swap is never normal working memory, and UMA RAM must preserve
|
||||
several GiB of desktop/iGPU headroom.
|
||||
|
||||
## Hardware and probe study
|
||||
|
||||
|
|
@ -282,11 +637,10 @@ Gate A preflight measured 16 logical CPUs, `MemTotal=15597716 KiB`,
|
|||
zram device, and zero current memory/IO PSI average. The host is the Ryzen 7
|
||||
8845HS/Radeon 780M UMA target described by the performance document.
|
||||
|
||||
The project already links OpenCV 5.0.0. Eigen 5.0.1 and Ceres 3.12.0 are
|
||||
available through host pkg-config but are not current production dependencies;
|
||||
SuiteSparse/BLAS/LAPACK availability is host capability only. TBB 2023.1 is
|
||||
present through the existing OpenCV stack. No package, system setting, swap
|
||||
device or GPU mode was changed.
|
||||
The project already links OpenCV 5.0.0. Host-installed libraries and their
|
||||
pkg-config or CMake discovery metadata are capabilities, not Lardon3D
|
||||
production dependencies. Lardon3D currently declares no Ceres dependency in
|
||||
Meson. No package, system setting, swap device or GPU mode was changed.
|
||||
|
||||
Gate A probes use deterministic synthetic camera arcs, controlled noise and
|
||||
degenerate planar/pure-rotation cases. Every RSS probe is a separate normal
|
||||
|
|
@ -304,8 +658,9 @@ production Sparse SfM code is created by this gate.
|
|||
deterministic seed, triangulation and PnP with synthetic ground truth.
|
||||
- **Gate D — Incremental core:** registration ordering, components,
|
||||
unregistered-image policy and deterministic reconstruction output.
|
||||
- **Gate E — BA integration:** sparse BA candidate, robust loss, local/global
|
||||
policy, numerical reproducibility and solver dependency decision.
|
||||
- **Gate E — Final Bundle Adjustment:** synchronous final per-component BA on a
|
||||
copy of the immutable Gate D result, with its scientific and numerical
|
||||
contract frozen here; interleaved local BA is deferred.
|
||||
- **Gate F — Project orchestration:** explicit Track Set/calibration input,
|
||||
atomic publication and durable runtime integration.
|
||||
- **Gate G — Resource/freeze:** Governor admission, sustained hardware safety,
|
||||
|
|
@ -317,7 +672,7 @@ production Sparse SfM code is created by this gate.
|
|||
|
||||
Seed/order risk is controlled by deterministic policy. It is robust for
|
||||
sequential capture, has canonical queues and seeds, moderate complexity, and
|
||||
sparse `C,T,O` scaling with local BA. **SELECTED v1.**
|
||||
sparse `C,T,O` scaling followed by final per-component BA. **SELECTED v1.**
|
||||
|
||||
### Global SfM
|
||||
|
||||
|
|
@ -342,7 +697,7 @@ Triangulation candidates:
|
|||
|---|---|---|---|
|
||||
| Dense normal equations | Prohibited for serious `C×P` problems | No | Rejected |
|
||||
| OpenCV generic optimization | Not a sparse BA contract | Present, wrong abstraction | Rejected |
|
||||
| Ceres sparse Schur | Appropriate block structure | New candidate dependency | **Later-gate candidate** |
|
||||
| Ceres 2.2.x iterative Schur | Block-sparse | Scientific selection; not in Meson | **Selected Gate E v1** |
|
||||
|
||||
### Synthetic geometry probe
|
||||
|
||||
|
|
@ -365,23 +720,23 @@ checks before accepting a component.
|
|||
|
||||
The project already links OpenCV 5.0.0. Host probes found Eigen 5.0.1, BLAS
|
||||
3.12.0, LAPACK 3.12.0 and TBB 2023.1 as host capabilities or transitive
|
||||
facilities rather than current Lardon3D production dependencies. Ceres and
|
||||
SuiteSparse are not available through the current host pkg-config environment;
|
||||
Ceres remains a new dependency candidate, not an installed fact. No new
|
||||
dependency is added by Gate A. The measured machine has 16 logical CPUs,
|
||||
facilities rather than current Lardon3D production dependencies. Ceres may use
|
||||
CMake discovery, so pkg-config alone does not establish host availability.
|
||||
Ceres 2.2.x is the selected Gate E scientific API, but Lardon3D declares no
|
||||
production Ceres dependency yet. No new dependency is added by this contract
|
||||
slice. The measured machine has 16 logical CPUs,
|
||||
`MemTotal=15597716 KiB`, `MemAvailable=8245288 KiB` at preflight, an 8 GiB
|
||||
swapfile, 6 GiB zram and zero memory/IO PSI averages at the probe start. A
|
||||
single heavy BA and a future solver thread cap of 4 are the conservative
|
||||
resource candidates; these are not yet Governor settings.
|
||||
swapfile, 6 GiB zram and zero memory/IO PSI averages at the probe start. Gate E
|
||||
uses one solver thread; future Gate G resource admission cannot change that
|
||||
scientific setting.
|
||||
|
||||
## Gate A unresolved boundaries
|
||||
|
||||
The following remain deliberately deferred to later gates rather than hidden:
|
||||
exact numeric parallax/reprojection thresholds, Ceres licensing/dependency
|
||||
adoption, robust-loss scale, local-window selection, BA convergence criteria,
|
||||
metric alignment, persistent reconstruction schema and durable SfM checkpoints.
|
||||
Their semantic ownership is decided here; their final numeric values require
|
||||
the synthetic ground-truth and sparse-solver gates.
|
||||
The following remain deliberately deferred rather than hidden: Ceres
|
||||
licensing/dependency integration, metric alignment, persistent orchestration
|
||||
and durable SfM checkpoints. Gate E freezes its own robust loss, convergence,
|
||||
ordering and acceptance policy here without introducing persistence or a
|
||||
fingerprint.
|
||||
|
||||
## Gate C — pure calibrated geometry
|
||||
|
||||
|
|
@ -439,7 +794,7 @@ parameters and do not alter Project DB identity.
|
|||
|
||||
## Gate D — incremental Sparse SfM core
|
||||
|
||||
**GATE D — PASS.** Gate D is the first executable link
|
||||
**GATE D — PASS / FROZEN.** Gate D is the first executable link
|
||||
between the immutable Track/Calibration contracts and the Gate C primitives.
|
||||
The reference implementation is synchronous, deterministic, CPU-only,
|
||||
in-memory, bounded and independent of Project DB, Task Runtime, Resource
|
||||
|
|
|
|||
113
include/lardon3d/sparse_sfm_bundle_adjustment.h
Normal file
113
include/lardon3d/sparse_sfm_bundle_adjustment.h
Normal file
|
|
@ -0,0 +1,113 @@
|
|||
#ifndef LARDON3D_SPARSE_SFM_BUNDLE_ADJUSTMENT_H
|
||||
#define LARDON3D_SPARSE_SFM_BUNDLE_ADJUSTMENT_H
|
||||
|
||||
#include <stdbool.h>
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
#include <lardon3d/sparse_sfm_incremental.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
/* COMPLETE means every eligible component was accepted, PARTIAL means accepted
|
||||
* and rejected eligible components coexist, and FAILED means none was
|
||||
* accepted. E1 defines the ABI but produces no final status before E2. */
|
||||
typedef enum {
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_COMPLETE = 0,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_PARTIAL,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_FAILED
|
||||
} Lardon3DSparseBundleAdjustmentStatus;
|
||||
|
||||
typedef enum {
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_AXIS_NONE = 0,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_AXIS_X,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_AXIS_Y,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_AXIS_Z
|
||||
} Lardon3DSparseBundleAdjustmentScaleAxis;
|
||||
|
||||
typedef enum {
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_TERMINATION_NOT_RUN = 0,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_TERMINATION_CONVERGED,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_TERMINATION_NO_CONVERGENCE,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_TERMINATION_FAILURE
|
||||
} Lardon3DSparseBundleAdjustmentTermination;
|
||||
|
||||
typedef enum {
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_REJECTION_NONE = 0,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_REJECTION_INELIGIBLE,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_REJECTION_GAUGE_DEGENERATE,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_REJECTION_INPUT,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_REJECTION_NONFINITE,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_REJECTION_NO_CONVERGENCE,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_REJECTION_SOLVER_FAILURE,
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_REJECTION_COST_REGRESSION
|
||||
} Lardon3DSparseBundleAdjustmentRejectionReason;
|
||||
|
||||
/* Both views are caller-owned and immutable for the future synchronous Gate E
|
||||
* call. incremental_result is authoritative for components, registered
|
||||
* image_id values, world-to-camera poses, landmarks and final associations.
|
||||
* images/observations are the same resolved scientific view used for Gate D.
|
||||
* An observation is identified by (feature_set_id, feature_index); its image_id
|
||||
* must agree with the final association. Source binary32 x/y become binary64
|
||||
* source-image pixels, origin top-left, +x right and +y down. Calibrations are
|
||||
* known, fixed and immutable. Each pointer is paired with its size_t count. */
|
||||
typedef struct {
|
||||
const Lardon3DSparseIncrementalResult *incremental_result;
|
||||
const Lardon3DSparseIncrementalImage *images;
|
||||
size_t image_count;
|
||||
const Lardon3DSparseIncrementalObservation *observations;
|
||||
size_t observation_count;
|
||||
} Lardon3DSparseBundleAdjustmentInput;
|
||||
|
||||
/* Counts describe one component. pose_anchor_image_id fixes the complete pose;
|
||||
* scale_anchor_image_id and scale_axis identify the fixed camera-center
|
||||
* coordinate. Costs are 0.5*sum(Huber(dx*dx+dy*dy)) with delta 2 source pixels. RMSE is
|
||||
* sqrt(sum(dx*dx+dy*dy)/observation_count) in source pixels. has_costs,
|
||||
* has_rmse and has_anchors determine whether the corresponding fields are
|
||||
* available; unavailable doubles are zero, never NaN sentinels. */
|
||||
typedef struct {
|
||||
uint64_t component_key;
|
||||
uint64_t camera_count;
|
||||
uint64_t landmark_count;
|
||||
uint64_t observation_count;
|
||||
bool eligible;
|
||||
bool has_anchors;
|
||||
uint64_t pose_anchor_image_id;
|
||||
uint64_t scale_anchor_image_id;
|
||||
Lardon3DSparseBundleAdjustmentScaleAxis scale_axis;
|
||||
bool has_costs;
|
||||
double initial_robust_cost;
|
||||
double final_robust_cost;
|
||||
bool has_rmse;
|
||||
double initial_reprojection_rmse_px;
|
||||
double final_reprojection_rmse_px;
|
||||
uint32_t iteration_count;
|
||||
Lardon3DSparseBundleAdjustmentTermination termination;
|
||||
bool accepted;
|
||||
Lardon3DSparseBundleAdjustmentRejectionReason rejection_reason;
|
||||
} Lardon3DSparseBundleAdjustmentComponentDiagnostic;
|
||||
|
||||
/* Future E2 output. All arrays are owned by the result and ordered by the
|
||||
* canonical component/image/Track/observation identities. Poses remain
|
||||
* world-to-camera binary64. A destruction function is added with the E2 run;
|
||||
* E1 intentionally exposes no fake execution function. */
|
||||
typedef struct {
|
||||
Lardon3DSparseBundleAdjustmentStatus status;
|
||||
Lardon3DSparseIncrementalComponent *components;
|
||||
size_t component_count;
|
||||
Lardon3DSparseIncrementalCamera *cameras;
|
||||
size_t camera_count;
|
||||
Lardon3DSparseIncrementalLandmark *landmarks;
|
||||
size_t landmark_count;
|
||||
Lardon3DSparseIncrementalLandmarkObservation *observations;
|
||||
size_t observation_count;
|
||||
Lardon3DSparseBundleAdjustmentComponentDiagnostic *diagnostics;
|
||||
} Lardon3DSparseBundleAdjustmentResult;
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
|
||||
#endif
|
||||
14
meson.build
14
meson.build
|
|
@ -146,6 +146,7 @@ executable(
|
|||
'src/project_db.c', 'src/project_db_sparse_sfm.c',
|
||||
'src/sparse_sfm_geometry.cpp',
|
||||
'src/sparse_sfm_incremental.cpp',
|
||||
'src/sparse_sfm_bundle_adjustment.cpp',
|
||||
'src/task.c',
|
||||
'src/task_checkpoint.c',
|
||||
'src/task_kind_registry.c',
|
||||
|
|
@ -546,6 +547,19 @@ sparse_sfm_incremental_test = executable(
|
|||
|
||||
test('sparse-sfm-incremental', sparse_sfm_incremental_test, timeout: 60)
|
||||
|
||||
sparse_sfm_bundle_adjustment_test = executable(
|
||||
'test-sparse-sfm-bundle-adjustment',
|
||||
sources: [
|
||||
'tests/test_sparse_sfm_bundle_adjustment.cpp',
|
||||
'src/sparse_sfm_bundle_adjustment.cpp',
|
||||
],
|
||||
cpp_args: ['-DLARDON3D_SPARSE_BUNDLE_ADJUSTMENT_TESTING'],
|
||||
include_directories: include_directories('include'),
|
||||
)
|
||||
|
||||
test('sparse-sfm-bundle-adjustment', sparse_sfm_bundle_adjustment_test,
|
||||
timeout: 30)
|
||||
|
||||
executable(
|
||||
'benchmark-sparse-sfm-incremental',
|
||||
sources: [
|
||||
|
|
|
|||
463
src/sparse_sfm_bundle_adjustment.cpp
Normal file
463
src/sparse_sfm_bundle_adjustment.cpp
Normal file
|
|
@ -0,0 +1,463 @@
|
|||
#include <lardon3d/sparse_sfm_bundle_adjustment.h>
|
||||
|
||||
#include <algorithm>
|
||||
#include <cmath>
|
||||
#include <new>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
namespace lardon3d::sparse_bundle_adjustment {
|
||||
|
||||
constexpr size_t maximum_images = 4096;
|
||||
constexpr size_t maximum_landmarks = 250000;
|
||||
constexpr size_t maximum_observations = 1000000;
|
||||
constexpr double depth_epsilon = 1e-9;
|
||||
constexpr double gauge_epsilon = 1e-9;
|
||||
|
||||
enum class PreparationStatus { prepared, invalid_argument, out_of_memory };
|
||||
enum class PrivateTermination { converged, no_convergence, failure };
|
||||
|
||||
struct ResolvedObservation {
|
||||
Lardon3DSparseIncrementalObservation source;
|
||||
size_t image_index;
|
||||
};
|
||||
|
||||
struct Preparation {
|
||||
std::vector<Lardon3DSparseIncrementalImage> images;
|
||||
std::vector<Lardon3DSparseIncrementalComponent> components;
|
||||
std::vector<Lardon3DSparseIncrementalCamera> cameras;
|
||||
std::vector<Lardon3DSparseIncrementalLandmark> landmarks;
|
||||
std::vector<Lardon3DSparseIncrementalLandmarkObservation> observations;
|
||||
std::vector<ResolvedObservation> resolved_observations;
|
||||
std::vector<Lardon3DSparseBundleAdjustmentComponentDiagnostic> diagnostics;
|
||||
};
|
||||
|
||||
struct ImageIndex {
|
||||
uint64_t image_id;
|
||||
size_t index;
|
||||
};
|
||||
|
||||
struct ObservationIndex {
|
||||
uint64_t feature_set_id;
|
||||
uint32_t feature_index;
|
||||
size_t index;
|
||||
};
|
||||
|
||||
bool finite_calibration(const Lardon3DSparseGeometryCalibration &value) {
|
||||
return value.width > 0 && value.height > 0 && std::isfinite(value.fx) &&
|
||||
std::isfinite(value.fy) && value.fx > 0.0 && value.fy > 0.0 &&
|
||||
std::isfinite(value.cx) && std::isfinite(value.cy) && value.cx >= 0.0 &&
|
||||
value.cy >= 0.0 && value.cx < value.width && value.cy < value.height &&
|
||||
std::isfinite(value.k1) && std::isfinite(value.k2) &&
|
||||
std::isfinite(value.p1) && std::isfinite(value.p2);
|
||||
}
|
||||
|
||||
bool finite_pose(const Lardon3DSparseGeometryPose &value) {
|
||||
for (double item : value.rotation_cw)
|
||||
if (!std::isfinite(item)) return false;
|
||||
for (double item : value.translation_cw)
|
||||
if (!std::isfinite(item)) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool valid_rotation(const Lardon3DSparseGeometryPose &value) {
|
||||
const double *r = value.rotation_cw;
|
||||
for (size_t row = 0; row < 3; ++row) {
|
||||
for (size_t column = 0; column < 3; ++column) {
|
||||
double dot = 0.0;
|
||||
for (size_t item = 0; item < 3; ++item)
|
||||
dot += r[row * 3 + item] * r[column * 3 + item];
|
||||
if (!std::isfinite(dot) ||
|
||||
std::abs(dot - (row == column ? 1.0 : 0.0)) >= 1e-6)
|
||||
return false;
|
||||
}
|
||||
}
|
||||
const double determinant =
|
||||
r[0] * (r[4] * r[8] - r[5] * r[7]) -
|
||||
r[1] * (r[3] * r[8] - r[5] * r[6]) +
|
||||
r[2] * (r[3] * r[7] - r[4] * r[6]);
|
||||
return std::isfinite(determinant) && std::abs(determinant - 1.0) < 1e-6;
|
||||
}
|
||||
|
||||
bool finite_point(const Lardon3DSparseGeometryPoint3 &value) {
|
||||
return std::isfinite(value.x) && std::isfinite(value.y) && std::isfinite(value.z);
|
||||
}
|
||||
|
||||
bool camera_center(const Lardon3DSparseGeometryPose &pose, double center[3]) {
|
||||
if (!finite_pose(pose) || !valid_rotation(pose)) return false;
|
||||
for (size_t column = 0; column < 3; ++column) {
|
||||
center[column] = -(pose.rotation_cw[column] * pose.translation_cw[0] +
|
||||
pose.rotation_cw[3 + column] * pose.translation_cw[1] +
|
||||
pose.rotation_cw[6 + column] * pose.translation_cw[2]);
|
||||
if (!std::isfinite(center[column])) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
bool select_anchors(const std::vector<Lardon3DSparseIncrementalCamera> &cameras,
|
||||
uint64_t component_key,
|
||||
Lardon3DSparseBundleAdjustmentComponentDiagnostic *diagnostic) {
|
||||
const Lardon3DSparseIncrementalCamera *pose_anchor = nullptr;
|
||||
for (const auto &camera : cameras) {
|
||||
if (camera.component_key == component_key &&
|
||||
(!pose_anchor || camera.image_id < pose_anchor->image_id))
|
||||
pose_anchor = &camera;
|
||||
}
|
||||
if (!pose_anchor) return false;
|
||||
double anchor_center[3];
|
||||
if (!camera_center(pose_anchor->pose_cw, anchor_center)) return false;
|
||||
const Lardon3DSparseIncrementalCamera *scale_anchor = nullptr;
|
||||
double best_distance = -1.0;
|
||||
double best_delta[3] = {};
|
||||
for (const auto &camera : cameras) {
|
||||
if (camera.component_key != component_key || camera.image_id == pose_anchor->image_id)
|
||||
continue;
|
||||
double center[3];
|
||||
if (!camera_center(camera.pose_cw, center)) return false;
|
||||
double delta[3] = {center[0] - anchor_center[0], center[1] - anchor_center[1],
|
||||
center[2] - anchor_center[2]};
|
||||
const double distance = std::hypot(delta[0], delta[1], delta[2]);
|
||||
if (!std::isfinite(distance)) return false;
|
||||
if (!scale_anchor || distance > best_distance ||
|
||||
(distance == best_distance && camera.image_id < scale_anchor->image_id)) {
|
||||
scale_anchor = &camera;
|
||||
best_distance = distance;
|
||||
std::copy(delta, delta + 3, best_delta);
|
||||
}
|
||||
}
|
||||
if (!scale_anchor) return false;
|
||||
size_t axis = 0;
|
||||
if (std::abs(best_delta[1]) > std::abs(best_delta[axis])) axis = 1;
|
||||
if (std::abs(best_delta[2]) > std::abs(best_delta[axis])) axis = 2;
|
||||
diagnostic->pose_anchor_image_id = pose_anchor->image_id;
|
||||
diagnostic->scale_anchor_image_id = scale_anchor->image_id;
|
||||
diagnostic->scale_axis =
|
||||
static_cast<Lardon3DSparseBundleAdjustmentScaleAxis>(axis + 1);
|
||||
diagnostic->has_anchors = std::abs(best_delta[axis]) > gauge_epsilon;
|
||||
return diagnostic->has_anchors;
|
||||
}
|
||||
|
||||
bool termination_accepted(PrivateTermination value) {
|
||||
return value == PrivateTermination::converged;
|
||||
}
|
||||
|
||||
bool project(const Lardon3DSparseGeometryCalibration &calibration,
|
||||
const Lardon3DSparseGeometryPose &pose,
|
||||
const Lardon3DSparseGeometryPoint3 &point,
|
||||
Lardon3DSparseGeometryPoint2 *pixel) {
|
||||
if (!pixel || !finite_calibration(calibration) || !finite_pose(pose) ||
|
||||
!valid_rotation(pose) || !finite_point(point))
|
||||
return false;
|
||||
const double *r = pose.rotation_cw;
|
||||
const double *t = pose.translation_cw;
|
||||
const double x = r[0] * point.x + r[1] * point.y + r[2] * point.z + t[0];
|
||||
const double y = r[3] * point.x + r[4] * point.y + r[5] * point.z + t[1];
|
||||
const double z = r[6] * point.x + r[7] * point.y + r[8] * point.z + t[2];
|
||||
if (!std::isfinite(x) || !std::isfinite(y) || !std::isfinite(z) ||
|
||||
z <= depth_epsilon)
|
||||
return false;
|
||||
const double xn = x / z;
|
||||
const double yn = y / z;
|
||||
const double r2 = xn * xn + yn * yn;
|
||||
const double radial = 1.0 + calibration.k1 * r2 + calibration.k2 * r2 * r2;
|
||||
const double xd = xn * radial + 2.0 * calibration.p1 * xn * yn +
|
||||
calibration.p2 * (r2 + 2.0 * xn * xn);
|
||||
const double yd = yn * radial + calibration.p1 * (r2 + 2.0 * yn * yn) +
|
||||
2.0 * calibration.p2 * xn * yn;
|
||||
pixel->x = calibration.fx * xd + calibration.cx;
|
||||
pixel->y = calibration.fy * yd + calibration.cy;
|
||||
return std::isfinite(pixel->x) && std::isfinite(pixel->y);
|
||||
}
|
||||
|
||||
bool residual_metrics(const double *residuals, size_t count, double *rmse,
|
||||
double *huber_cost) {
|
||||
if (!residuals || count == 0 || !rmse || !huber_cost || count > SIZE_MAX / 2)
|
||||
return false;
|
||||
double squared_sum = 0.0;
|
||||
double rho_sum = 0.0;
|
||||
for (size_t index = 0; index < count; ++index) {
|
||||
const double dx = residuals[index * 2];
|
||||
const double dy = residuals[index * 2 + 1];
|
||||
const double squared = dx * dx + dy * dy;
|
||||
if (!std::isfinite(dx) || !std::isfinite(dy) || !std::isfinite(squared))
|
||||
return false;
|
||||
const double rho = squared <= 4.0 ? squared : 4.0 * std::sqrt(squared) - 4.0;
|
||||
squared_sum += squared;
|
||||
rho_sum += rho;
|
||||
if (!std::isfinite(rho) || !std::isfinite(squared_sum) || !std::isfinite(rho_sum))
|
||||
return false;
|
||||
}
|
||||
*rmse = std::sqrt(squared_sum / static_cast<double>(count));
|
||||
*huber_cost = 0.5 * rho_sum;
|
||||
return std::isfinite(*rmse) && std::isfinite(*huber_cost);
|
||||
}
|
||||
|
||||
bool cost_acceptable(double initial_cost, double final_cost) {
|
||||
if (!std::isfinite(initial_cost) || !std::isfinite(final_cost)) return false;
|
||||
const double tolerance = 1e-12 * std::max(1.0, std::abs(initial_cost));
|
||||
return std::isfinite(tolerance) && final_cost <= initial_cost + tolerance;
|
||||
}
|
||||
|
||||
template <typename T>
|
||||
void copy_view(std::vector<T> *destination, const T *source, size_t count) {
|
||||
destination->clear();
|
||||
if (count != 0) destination->assign(source, source + count);
|
||||
}
|
||||
|
||||
PreparationStatus prepare(const Lardon3DSparseBundleAdjustmentInput &input,
|
||||
Preparation *preparation) {
|
||||
if (!preparation || !input.incremental_result)
|
||||
return PreparationStatus::invalid_argument;
|
||||
const auto &result = *input.incremental_result;
|
||||
if (result.status < LARDON3D_SPARSE_INCREMENTAL_COMPLETE ||
|
||||
result.status > LARDON3D_SPARSE_INCREMENTAL_FAILED ||
|
||||
input.image_count > maximum_images || result.camera_count > maximum_images ||
|
||||
result.landmark_count > maximum_landmarks ||
|
||||
result.observation_count > maximum_observations ||
|
||||
input.observation_count > maximum_observations ||
|
||||
(input.image_count && !input.images) ||
|
||||
(input.observation_count && !input.observations) ||
|
||||
(result.component_count && !result.components) ||
|
||||
(result.camera_count && !result.cameras) ||
|
||||
(result.landmark_count && !result.landmarks) ||
|
||||
(result.observation_count && !result.observations))
|
||||
return PreparationStatus::invalid_argument;
|
||||
try {
|
||||
Preparation candidate;
|
||||
copy_view(&candidate.images, input.images, input.image_count);
|
||||
copy_view(&candidate.components, result.components, result.component_count);
|
||||
copy_view(&candidate.cameras, result.cameras, result.camera_count);
|
||||
copy_view(&candidate.landmarks, result.landmarks, result.landmark_count);
|
||||
copy_view(&candidate.observations, result.observations, result.observation_count);
|
||||
std::sort(candidate.images.begin(), candidate.images.end(),
|
||||
[](const auto &a, const auto &b) { return a.image_id < b.image_id; });
|
||||
|
||||
std::vector<ImageIndex> images;
|
||||
images.reserve(input.image_count);
|
||||
for (size_t index = 0; index < input.image_count; ++index) {
|
||||
const auto &image = candidate.images[index];
|
||||
if (image.image_id == 0 || !finite_calibration(image.calibration) ||
|
||||
(index && candidate.images[index - 1].image_id == image.image_id))
|
||||
return PreparationStatus::invalid_argument;
|
||||
images.push_back({image.image_id, index});
|
||||
}
|
||||
|
||||
std::vector<ObservationIndex> observation_index;
|
||||
observation_index.reserve(input.observation_count);
|
||||
for (size_t index = 0; index < input.observation_count; ++index) {
|
||||
const auto &observation = input.observations[index];
|
||||
if (observation.track_id == 0 || observation.image_id == 0 ||
|
||||
observation.feature_set_id == 0 || observation.feature_count == 0 ||
|
||||
observation.feature_index >= observation.feature_count ||
|
||||
!std::isfinite(observation.x) || !std::isfinite(observation.y))
|
||||
return PreparationStatus::invalid_argument;
|
||||
auto image = std::lower_bound(
|
||||
images.begin(), images.end(), observation.image_id,
|
||||
[](const auto &item, uint64_t key) { return item.image_id < key; });
|
||||
if (image == images.end() || image->image_id != observation.image_id)
|
||||
return PreparationStatus::invalid_argument;
|
||||
observation_index.push_back(
|
||||
{observation.feature_set_id, observation.feature_index, index});
|
||||
}
|
||||
std::sort(observation_index.begin(), observation_index.end(),
|
||||
[](const auto &a, const auto &b) {
|
||||
return std::pair{a.feature_set_id, a.feature_index} <
|
||||
std::pair{b.feature_set_id, b.feature_index};
|
||||
});
|
||||
for (size_t index = 1; index < observation_index.size(); ++index) {
|
||||
if (observation_index[index - 1].feature_set_id ==
|
||||
observation_index[index].feature_set_id &&
|
||||
observation_index[index - 1].feature_index ==
|
||||
observation_index[index].feature_index)
|
||||
return PreparationStatus::invalid_argument;
|
||||
}
|
||||
|
||||
std::vector<uint64_t> component_camera_counts(result.component_count, 0);
|
||||
std::vector<uint64_t> component_landmark_counts(result.component_count, 0);
|
||||
for (size_t index = 0; index < result.component_count; ++index) {
|
||||
const auto &component = candidate.components[index];
|
||||
if (component.component_key == 0 ||
|
||||
(index && candidate.components[index - 1].component_key >=
|
||||
component.component_key) ||
|
||||
component.registered_image_count > component.image_count)
|
||||
return PreparationStatus::invalid_argument;
|
||||
}
|
||||
for (size_t index = 0; index < result.camera_count; ++index) {
|
||||
const auto &camera = candidate.cameras[index];
|
||||
if (camera.image_id == 0 ||
|
||||
(index && candidate.cameras[index - 1].image_id >= camera.image_id) ||
|
||||
!finite_pose(camera.pose_cw) || !valid_rotation(camera.pose_cw))
|
||||
return PreparationStatus::invalid_argument;
|
||||
auto component = std::lower_bound(
|
||||
candidate.components.begin(), candidate.components.end(), camera.component_key,
|
||||
[](const auto &item, uint64_t key) { return item.component_key < key; });
|
||||
auto image = std::lower_bound(
|
||||
images.begin(), images.end(), camera.image_id,
|
||||
[](const auto &item, uint64_t key) { return item.image_id < key; });
|
||||
if (component == candidate.components.end() ||
|
||||
component->component_key != camera.component_key || image == images.end() ||
|
||||
image->image_id != camera.image_id)
|
||||
return PreparationStatus::invalid_argument;
|
||||
++component_camera_counts[static_cast<size_t>(component - candidate.components.begin())];
|
||||
}
|
||||
for (size_t index = 0; index < result.landmark_count; ++index) {
|
||||
const auto &landmark = candidate.landmarks[index];
|
||||
if (landmark.landmark_id == 0 || landmark.track_id == 0 ||
|
||||
landmark.observation_count < 2 || !finite_point(landmark.point) ||
|
||||
(index && candidate.landmarks[index - 1].track_id >= landmark.track_id))
|
||||
return PreparationStatus::invalid_argument;
|
||||
auto component = std::lower_bound(
|
||||
candidate.components.begin(), candidate.components.end(), landmark.component_key,
|
||||
[](const auto &item, uint64_t key) { return item.component_key < key; });
|
||||
if (component == candidate.components.end() ||
|
||||
component->component_key != landmark.component_key)
|
||||
return PreparationStatus::invalid_argument;
|
||||
++component_landmark_counts[static_cast<size_t>(component - candidate.components.begin())];
|
||||
}
|
||||
for (size_t index = 0; index < result.component_count; ++index) {
|
||||
if (candidate.components[index].registered_image_count !=
|
||||
component_camera_counts[index] ||
|
||||
candidate.components[index].landmark_count != component_landmark_counts[index])
|
||||
return PreparationStatus::invalid_argument;
|
||||
}
|
||||
|
||||
candidate.diagnostics.resize(result.component_count);
|
||||
std::vector<uint64_t> landmark_observation_counts(result.landmark_count, 0);
|
||||
std::vector<uint8_t> published_observations(input.observation_count, 0);
|
||||
candidate.resolved_observations.reserve(result.observation_count);
|
||||
uint64_t previous_landmark = 0;
|
||||
uint32_t previous_position = 0;
|
||||
for (size_t index = 0; index < result.observation_count; ++index) {
|
||||
const auto &published = candidate.observations[index];
|
||||
const bool ordered = index == 0 || published.landmark_id > previous_landmark ||
|
||||
(published.landmark_id == previous_landmark &&
|
||||
published.position_in_track > previous_position);
|
||||
auto landmark = std::lower_bound(
|
||||
candidate.landmarks.begin(), candidate.landmarks.end(), published.track_id,
|
||||
[](const auto &item, uint64_t key) { return item.track_id < key; });
|
||||
auto observation = std::lower_bound(
|
||||
observation_index.begin(), observation_index.end(),
|
||||
std::pair{published.feature_set_id, published.feature_index},
|
||||
[](const auto &item, const auto &key) {
|
||||
return std::pair{item.feature_set_id, item.feature_index} < key;
|
||||
});
|
||||
auto image = std::lower_bound(
|
||||
images.begin(), images.end(), published.image_id,
|
||||
[](const auto &item, uint64_t key) { return item.image_id < key; });
|
||||
if (!ordered || landmark == candidate.landmarks.end() ||
|
||||
landmark->landmark_id != published.landmark_id ||
|
||||
landmark->track_id != published.track_id || observation == observation_index.end() ||
|
||||
observation->feature_set_id != published.feature_set_id ||
|
||||
observation->feature_index != published.feature_index || image == images.end() ||
|
||||
image->image_id != published.image_id)
|
||||
return PreparationStatus::invalid_argument;
|
||||
const auto &source = input.observations[observation->index];
|
||||
if (published_observations[observation->index])
|
||||
return PreparationStatus::invalid_argument;
|
||||
published_observations[observation->index] = 1;
|
||||
auto camera = std::lower_bound(
|
||||
candidate.cameras.begin(), candidate.cameras.end(), published.image_id,
|
||||
[](const auto &item, uint64_t key) { return item.image_id < key; });
|
||||
if (source.track_id != published.track_id || source.image_id != published.image_id ||
|
||||
camera == candidate.cameras.end() || camera->image_id != published.image_id ||
|
||||
camera->component_key != landmark->component_key)
|
||||
return PreparationStatus::invalid_argument;
|
||||
const size_t landmark_index = static_cast<size_t>(landmark - candidate.landmarks.begin());
|
||||
auto component = std::lower_bound(
|
||||
candidate.components.begin(), candidate.components.end(), landmark->component_key,
|
||||
[](const auto &item, uint64_t key) { return item.component_key < key; });
|
||||
const size_t component_index =
|
||||
static_cast<size_t>(component - candidate.components.begin());
|
||||
++landmark_observation_counts[landmark_index];
|
||||
++candidate.diagnostics[component_index].observation_count;
|
||||
candidate.resolved_observations.push_back({source, image->index});
|
||||
previous_landmark = published.landmark_id;
|
||||
previous_position = published.position_in_track;
|
||||
}
|
||||
for (size_t index = 0; index < candidate.landmarks.size(); ++index) {
|
||||
if (candidate.landmarks[index].observation_count !=
|
||||
landmark_observation_counts[index])
|
||||
return PreparationStatus::invalid_argument;
|
||||
}
|
||||
for (size_t index = 0; index < candidate.components.size(); ++index) {
|
||||
auto &diagnostic = candidate.diagnostics[index];
|
||||
const auto &component = candidate.components[index];
|
||||
diagnostic.component_key = component.component_key;
|
||||
diagnostic.camera_count = component.registered_image_count;
|
||||
diagnostic.landmark_count = component.landmark_count;
|
||||
diagnostic.termination = LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_TERMINATION_NOT_RUN;
|
||||
diagnostic.rejection_reason =
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_REJECTION_INELIGIBLE;
|
||||
diagnostic.eligible = component.registered_image_count >= 2 &&
|
||||
component.landmark_count >= 1 &&
|
||||
select_anchors(candidate.cameras, component.component_key,
|
||||
&diagnostic);
|
||||
if (diagnostic.eligible)
|
||||
diagnostic.rejection_reason = LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_REJECTION_NONE;
|
||||
else if (!diagnostic.has_anchors && component.registered_image_count >= 2)
|
||||
diagnostic.rejection_reason =
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_REJECTION_GAUGE_DEGENERATE;
|
||||
}
|
||||
*preparation = std::move(candidate);
|
||||
return PreparationStatus::prepared;
|
||||
} catch (const std::bad_alloc &) {
|
||||
return PreparationStatus::out_of_memory;
|
||||
} catch (...) {
|
||||
return PreparationStatus::invalid_argument;
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace lardon3d::sparse_bundle_adjustment
|
||||
|
||||
#ifdef LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_TESTING
|
||||
extern "C" int lardon3d_sparse_bundle_adjustment_test_prepare(
|
||||
const Lardon3DSparseBundleAdjustmentInput *input,
|
||||
Lardon3DSparseBundleAdjustmentComponentDiagnostic *diagnostics,
|
||||
size_t diagnostic_capacity, Lardon3DSparseIncrementalObservation *resolved,
|
||||
size_t resolved_capacity, size_t *diagnostic_count, size_t *resolved_count) {
|
||||
using namespace lardon3d::sparse_bundle_adjustment;
|
||||
if (!input || !diagnostic_count || !resolved_count) return 1;
|
||||
Preparation preparation;
|
||||
const auto status = prepare(*input, &preparation);
|
||||
if (status != PreparationStatus::prepared)
|
||||
return status == PreparationStatus::out_of_memory ? 2 : 1;
|
||||
*diagnostic_count = preparation.diagnostics.size();
|
||||
*resolved_count = preparation.resolved_observations.size();
|
||||
if ((preparation.diagnostics.size() > diagnostic_capacity) ||
|
||||
(preparation.resolved_observations.size() > resolved_capacity) ||
|
||||
(!preparation.diagnostics.empty() && !diagnostics) ||
|
||||
(!preparation.resolved_observations.empty() && !resolved))
|
||||
return 1;
|
||||
std::copy(preparation.diagnostics.begin(), preparation.diagnostics.end(), diagnostics);
|
||||
for (size_t index = 0; index < preparation.resolved_observations.size(); ++index)
|
||||
resolved[index] = preparation.resolved_observations[index].source;
|
||||
return 0;
|
||||
}
|
||||
|
||||
extern "C" bool lardon3d_sparse_bundle_adjustment_test_project(
|
||||
const Lardon3DSparseGeometryCalibration *calibration,
|
||||
const Lardon3DSparseGeometryPose *pose,
|
||||
const Lardon3DSparseGeometryPoint3 *point,
|
||||
Lardon3DSparseGeometryPoint2 *pixel) {
|
||||
return calibration && pose && point &&
|
||||
lardon3d::sparse_bundle_adjustment::project(*calibration, *pose, *point, pixel);
|
||||
}
|
||||
|
||||
extern "C" bool lardon3d_sparse_bundle_adjustment_test_metrics(
|
||||
const double *residuals, size_t count, double *rmse, double *huber_cost) {
|
||||
return lardon3d::sparse_bundle_adjustment::residual_metrics(
|
||||
residuals, count, rmse, huber_cost);
|
||||
}
|
||||
|
||||
extern "C" bool lardon3d_sparse_bundle_adjustment_test_cost_acceptable(
|
||||
double initial_cost, double final_cost) {
|
||||
return lardon3d::sparse_bundle_adjustment::cost_acceptable(initial_cost, final_cost);
|
||||
}
|
||||
|
||||
extern "C" bool lardon3d_sparse_bundle_adjustment_test_termination_accepted(
|
||||
int value) {
|
||||
using namespace lardon3d::sparse_bundle_adjustment;
|
||||
if (value < 0 || value > 2) return false;
|
||||
return termination_accepted(static_cast<PrivateTermination>(value));
|
||||
}
|
||||
#endif
|
||||
356
tests/test_sparse_sfm_bundle_adjustment.cpp
Normal file
356
tests/test_sparse_sfm_bundle_adjustment.cpp
Normal file
|
|
@ -0,0 +1,356 @@
|
|||
#include <lardon3d/sparse_sfm_bundle_adjustment.h>
|
||||
|
||||
#include <cmath>
|
||||
#include <cstdio>
|
||||
#include <cstring>
|
||||
#include <limits>
|
||||
|
||||
#define CHECK(value) \
|
||||
do { \
|
||||
if (!(value)) { \
|
||||
std::fprintf(stderr, "bundle adjustment check failed at line %d: %s\n", \
|
||||
__LINE__, #value); \
|
||||
return 1; \
|
||||
} \
|
||||
} while (0)
|
||||
|
||||
extern "C" bool lardon3d_sparse_bundle_adjustment_test_termination_accepted(
|
||||
int value);
|
||||
extern "C" int lardon3d_sparse_bundle_adjustment_test_prepare(
|
||||
const Lardon3DSparseBundleAdjustmentInput *input,
|
||||
Lardon3DSparseBundleAdjustmentComponentDiagnostic *diagnostics,
|
||||
size_t diagnostic_capacity, Lardon3DSparseIncrementalObservation *resolved,
|
||||
size_t resolved_capacity, size_t *diagnostic_count, size_t *resolved_count);
|
||||
extern "C" bool lardon3d_sparse_bundle_adjustment_test_project(
|
||||
const Lardon3DSparseGeometryCalibration *calibration,
|
||||
const Lardon3DSparseGeometryPose *pose,
|
||||
const Lardon3DSparseGeometryPoint3 *point,
|
||||
Lardon3DSparseGeometryPoint2 *pixel);
|
||||
extern "C" bool lardon3d_sparse_bundle_adjustment_test_metrics(
|
||||
const double *residuals, size_t count, double *rmse, double *huber_cost);
|
||||
extern "C" bool lardon3d_sparse_bundle_adjustment_test_cost_acceptable(
|
||||
double initial_cost, double final_cost);
|
||||
|
||||
static Lardon3DSparseGeometryCalibration calibration() {
|
||||
return {1280, 960, 800.0, 810.0, 640.0, 480.0, 0.0, 0.0, 0.0, 0.0};
|
||||
}
|
||||
|
||||
static int test_helpers() {
|
||||
const Lardon3DSparseGeometryPose pose = {
|
||||
{1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0}, {0.0, 0.0, 0.0}};
|
||||
const Lardon3DSparseGeometryPoint3 point = {0.2, -0.1, 2.0};
|
||||
auto camera = calibration();
|
||||
Lardon3DSparseGeometryPoint2 pixel = {};
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_project(&camera, &pose, &point, &pixel));
|
||||
CHECK(std::abs(pixel.x - 720.0) < 1e-12);
|
||||
CHECK(std::abs(pixel.y - 439.5) < 1e-12);
|
||||
|
||||
camera.k1 = 0.1;
|
||||
camera.k2 = -0.02;
|
||||
camera.p1 = 0.003;
|
||||
camera.p2 = -0.004;
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_project(&camera, &pose, &point, &pixel));
|
||||
const double xn = 0.1;
|
||||
const double yn = -0.05;
|
||||
const double r2 = xn * xn + yn * yn;
|
||||
const double radial = 1.0 + camera.k1 * r2 + camera.k2 * r2 * r2;
|
||||
const double xd = xn * radial + 2.0 * camera.p1 * xn * yn +
|
||||
camera.p2 * (r2 + 2.0 * xn * xn);
|
||||
const double yd = yn * radial + camera.p1 * (r2 + 2.0 * yn * yn) +
|
||||
2.0 * camera.p2 * xn * yn;
|
||||
CHECK(std::abs(pixel.x - (camera.fx * xd + camera.cx)) < 1e-12);
|
||||
CHECK(std::abs(pixel.y - (camera.fy * yd + camera.cy)) < 1e-12);
|
||||
|
||||
double residuals[4] = {3.0, 4.0, 0.0, 0.0};
|
||||
double rmse = 0.0;
|
||||
double cost = 0.0;
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_metrics(residuals, 2, &rmse, &cost));
|
||||
CHECK(std::abs(rmse - std::sqrt(12.5)) < 1e-12);
|
||||
CHECK(cost == 8.0);
|
||||
double boundary[2] = {2.0, 0.0};
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_metrics(boundary, 1, &rmse, &cost));
|
||||
CHECK(rmse == 2.0 && cost == 2.0);
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_cost_acceptable(10.0, 10.0 + 1e-11));
|
||||
CHECK(!lardon3d_sparse_bundle_adjustment_test_cost_acceptable(10.0,
|
||||
10.0 + 2e-11));
|
||||
CHECK(!lardon3d_sparse_bundle_adjustment_test_cost_acceptable(NAN, 0.0));
|
||||
|
||||
residuals[0] = std::numeric_limits<double>::infinity();
|
||||
CHECK(!lardon3d_sparse_bundle_adjustment_test_metrics(residuals, 2, &rmse, &cost));
|
||||
CHECK(!lardon3d_sparse_bundle_adjustment_test_project(nullptr, &pose, &point, &pixel));
|
||||
Lardon3DSparseGeometryPoint3 behind = {0.0, 0.0, -1.0};
|
||||
CHECK(!lardon3d_sparse_bundle_adjustment_test_project(&camera, &pose, &behind,
|
||||
&pixel));
|
||||
Lardon3DSparseGeometryPoint3 depth_boundary = {0.0, 0.0, 1e-9};
|
||||
CHECK(!lardon3d_sparse_bundle_adjustment_test_project(
|
||||
&camera, &pose, &depth_boundary, &pixel));
|
||||
depth_boundary.z = std::nextafter(1e-9, 1.0);
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_project(
|
||||
&camera, &pose, &depth_boundary, &pixel));
|
||||
auto invalid_pose = pose;
|
||||
invalid_pose.rotation_cw[0] += 1e-6;
|
||||
CHECK(!lardon3d_sparse_bundle_adjustment_test_project(
|
||||
&camera, &invalid_pose, &point, &pixel));
|
||||
auto invalid_calibration = camera;
|
||||
invalid_calibration.cx = -1.0;
|
||||
CHECK(!lardon3d_sparse_bundle_adjustment_test_project(
|
||||
&invalid_calibration, &pose, &point, &pixel));
|
||||
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_termination_accepted(0));
|
||||
CHECK(!lardon3d_sparse_bundle_adjustment_test_termination_accepted(1));
|
||||
CHECK(!lardon3d_sparse_bundle_adjustment_test_termination_accepted(2));
|
||||
CHECK(!lardon3d_sparse_bundle_adjustment_test_termination_accepted(3));
|
||||
return 0;
|
||||
}
|
||||
|
||||
struct Fixture {
|
||||
Lardon3DSparseIncrementalImage images[3];
|
||||
Lardon3DSparseIncrementalObservation input_observations[3];
|
||||
Lardon3DSparseIncrementalComponent components[1];
|
||||
Lardon3DSparseIncrementalCamera cameras[3];
|
||||
Lardon3DSparseIncrementalLandmark landmarks[1];
|
||||
Lardon3DSparseIncrementalLandmarkObservation result_observations[3];
|
||||
Lardon3DSparseIncrementalUnregisteredImage unregistered_images[1];
|
||||
Lardon3DSparseIncrementalResult result;
|
||||
Lardon3DSparseBundleAdjustmentInput input;
|
||||
};
|
||||
|
||||
static Fixture fixture() {
|
||||
Fixture value = {};
|
||||
const auto intrinsic = calibration();
|
||||
value.images[0] = {10, intrinsic};
|
||||
value.images[1] = {20, intrinsic};
|
||||
value.images[2] = {30, intrinsic};
|
||||
value.input_observations[0] = {7, 10, 100, 1, 2, 650.0, 480.0};
|
||||
value.input_observations[1] = {7, 20, 200, 1, 2, 500.0, 480.0};
|
||||
value.input_observations[2] = {7, 30, 300, 1, 2, 700.0, 480.0};
|
||||
value.components[0] = {10, 3, 3, 1};
|
||||
const double identity[9] = {1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0};
|
||||
std::memcpy(value.cameras[0].pose_cw.rotation_cw, identity, sizeof(identity));
|
||||
std::memcpy(value.cameras[1].pose_cw.rotation_cw, identity, sizeof(identity));
|
||||
std::memcpy(value.cameras[2].pose_cw.rotation_cw, identity, sizeof(identity));
|
||||
value.cameras[0].image_id = 10;
|
||||
value.cameras[1].image_id = 20;
|
||||
value.cameras[2].image_id = 30;
|
||||
for (auto &camera : value.cameras) camera.component_key = 10;
|
||||
value.cameras[1].pose_cw.translation_cw[0] = -2.0;
|
||||
value.cameras[1].pose_cw.translation_cw[1] = -2.0;
|
||||
value.cameras[2].pose_cw.translation_cw[0] = 2.0;
|
||||
value.cameras[2].pose_cw.translation_cw[1] = 2.0;
|
||||
value.landmarks[0] = {1, 7, 10, {0.0, 0.0, 5.0}, 0.5, 0.4, 3};
|
||||
for (size_t index = 0; index < 3; ++index) {
|
||||
value.result_observations[index] = {
|
||||
1, 7, value.images[index].image_id,
|
||||
value.input_observations[index].feature_set_id,
|
||||
value.input_observations[index].feature_index,
|
||||
static_cast<uint32_t>(index)};
|
||||
}
|
||||
value.result.status = LARDON3D_SPARSE_INCREMENTAL_COMPLETE;
|
||||
value.result.track_set_id = 77;
|
||||
value.result.calibration_scope_id = 88;
|
||||
value.result.component_count = 1;
|
||||
value.result.camera_count = 3;
|
||||
value.result.landmark_count = 1;
|
||||
value.result.observation_count = 3;
|
||||
value.unregistered_images[0] = {99, 10};
|
||||
value.result.unregistered_image_count = 1;
|
||||
value.result.seed_candidates_considered = 3;
|
||||
value.result.seed_candidates_available = 4;
|
||||
value.result.seed_image_a = 10;
|
||||
value.result.seed_image_b = 20;
|
||||
value.result.last_seed_geometry_status = 2;
|
||||
value.result.last_seed_parallax_rad = 0.25;
|
||||
value.result.registration_rounds = 5;
|
||||
value.result.registration_attempts = 6;
|
||||
value.result.registration_successes = 3;
|
||||
value.result.registration_failures = 3;
|
||||
value.result.last_pnp_inlier_count = 8;
|
||||
value.result.triangulation_attempts = 9;
|
||||
value.result.triangulation_failures = 1;
|
||||
value.result.rejected_behind_camera = 2;
|
||||
value.result.rejected_reprojection = 3;
|
||||
value.result.rejected_landmarks = 4;
|
||||
value.result.last_triangulation_status = 5;
|
||||
value.result.landmark_update_attempts = 6;
|
||||
value.result.landmark_update_successes = 7;
|
||||
value.result.landmark_update_failures = 8;
|
||||
value.result.no_growth_terminations = 9;
|
||||
value.result.round_limit_terminations = 10;
|
||||
value.result.point_refinement_attempts = 11;
|
||||
value.result.point_refinement_successes = 12;
|
||||
return value;
|
||||
}
|
||||
|
||||
static void bind_fixture(Fixture *value) {
|
||||
value->result.components = value->components;
|
||||
value->result.cameras = value->cameras;
|
||||
value->result.landmarks = value->landmarks;
|
||||
value->result.observations = value->result_observations;
|
||||
value->result.unregistered_images = value->unregistered_images;
|
||||
value->input = {&value->result, value->images, 3, value->input_observations, 3};
|
||||
}
|
||||
|
||||
static int test_preparation() {
|
||||
Fixture value = fixture();
|
||||
bind_fixture(&value);
|
||||
const Fixture original = value;
|
||||
Lardon3DSparseBundleAdjustmentComponentDiagnostic diagnostics[1] = {};
|
||||
Lardon3DSparseIncrementalObservation resolved[3] = {};
|
||||
size_t diagnostic_count = 0;
|
||||
size_t resolved_count = 0;
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_prepare(
|
||||
&value.input, diagnostics, 1, resolved, 3, &diagnostic_count,
|
||||
&resolved_count) == 0);
|
||||
CHECK(diagnostic_count == 1 && resolved_count == 3);
|
||||
CHECK(resolved[1].image_id == 20 && resolved[1].x == 500.0);
|
||||
CHECK(diagnostics[0].eligible);
|
||||
CHECK(diagnostics[0].pose_anchor_image_id == 10);
|
||||
CHECK(diagnostics[0].scale_anchor_image_id == 20);
|
||||
CHECK(diagnostics[0].scale_axis ==
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_AXIS_X);
|
||||
CHECK(diagnostics[0].termination ==
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_TERMINATION_NOT_RUN);
|
||||
CHECK(!diagnostics[0].has_costs && !diagnostics[0].has_rmse);
|
||||
CHECK(!diagnostics[0].accepted);
|
||||
CHECK(std::memcmp(&value.result, &original.result, sizeof(value.result)) == 0);
|
||||
CHECK(std::memcmp(value.images, original.images, sizeof(value.images)) == 0);
|
||||
CHECK(std::memcmp(value.input_observations, original.input_observations,
|
||||
sizeof(value.input_observations)) == 0);
|
||||
CHECK(std::memcmp(value.cameras, original.cameras, sizeof(value.cameras)) == 0);
|
||||
CHECK(std::memcmp(value.landmarks, original.landmarks, sizeof(value.landmarks)) == 0);
|
||||
CHECK(std::memcmp(value.components, original.components, sizeof(value.components)) == 0);
|
||||
CHECK(std::memcmp(value.result_observations, original.result_observations,
|
||||
sizeof(value.result_observations)) == 0);
|
||||
CHECK(std::memcmp(value.unregistered_images, original.unregistered_images,
|
||||
sizeof(value.unregistered_images)) == 0);
|
||||
|
||||
value = fixture();
|
||||
bind_fixture(&value);
|
||||
value.cameras[1].pose_cw.translation_cw[0] = -1e-9;
|
||||
value.cameras[1].pose_cw.translation_cw[1] = 0.0;
|
||||
value.cameras[2].pose_cw.translation_cw[0] = 1e-9;
|
||||
value.cameras[2].pose_cw.translation_cw[1] = 0.0;
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_prepare(
|
||||
&value.input, diagnostics, 1, resolved, 3, &diagnostic_count,
|
||||
&resolved_count) == 0);
|
||||
CHECK(!diagnostics[0].eligible);
|
||||
CHECK(diagnostics[0].rejection_reason ==
|
||||
LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_REJECTION_GAUGE_DEGENERATE);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int test_invalid() {
|
||||
Lardon3DSparseBundleAdjustmentComponentDiagnostic diagnostics[1] = {};
|
||||
Lardon3DSparseIncrementalObservation resolved[3] = {};
|
||||
size_t diagnostic_count = 0;
|
||||
size_t resolved_count = 0;
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_prepare(
|
||||
nullptr, diagnostics, 1, resolved, 3, &diagnostic_count,
|
||||
&resolved_count) == 1);
|
||||
Fixture value = fixture();
|
||||
bind_fixture(&value);
|
||||
value.input_observations[1].feature_set_id = 999;
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_prepare(
|
||||
&value.input, diagnostics, 1, resolved, 3, &diagnostic_count,
|
||||
&resolved_count) == 1);
|
||||
value = fixture();
|
||||
bind_fixture(&value);
|
||||
value.input_observations[1].image_id = 30;
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_prepare(
|
||||
&value.input, diagnostics, 1, resolved, 3, &diagnostic_count,
|
||||
&resolved_count) == 1);
|
||||
value = fixture();
|
||||
bind_fixture(&value);
|
||||
value.input_observations[1].x = NAN;
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_prepare(
|
||||
&value.input, diagnostics, 1, resolved, 3, &diagnostic_count,
|
||||
&resolved_count) == 1);
|
||||
value = fixture();
|
||||
bind_fixture(&value);
|
||||
value.input_observations[1].feature_set_id = value.input_observations[0].feature_set_id;
|
||||
value.input_observations[1].feature_index = value.input_observations[0].feature_index;
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_prepare(
|
||||
&value.input, diagnostics, 1, resolved, 3, &diagnostic_count,
|
||||
&resolved_count) == 1);
|
||||
value = fixture();
|
||||
bind_fixture(&value);
|
||||
std::swap(value.cameras[0], value.cameras[1]);
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_prepare(
|
||||
&value.input, diagnostics, 1, resolved, 3, &diagnostic_count,
|
||||
&resolved_count) == 1);
|
||||
value = fixture();
|
||||
bind_fixture(&value);
|
||||
value.input.image_count = 4097;
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_prepare(
|
||||
&value.input, diagnostics, 1, resolved, 3, &diagnostic_count,
|
||||
&resolved_count) == 1);
|
||||
|
||||
Lardon3DSparseIncrementalResult empty = {};
|
||||
empty.status = LARDON3D_SPARSE_INCREMENTAL_FAILED;
|
||||
Lardon3DSparseBundleAdjustmentInput empty_input = {&empty, nullptr, 0, nullptr, 0};
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_prepare(
|
||||
&empty_input, nullptr, 0, nullptr, 0, &diagnostic_count,
|
||||
&resolved_count) == 0);
|
||||
CHECK(diagnostic_count == 0 && resolved_count == 0);
|
||||
return 0;
|
||||
}
|
||||
|
||||
static int test_multiple_components() {
|
||||
const auto intrinsic = calibration();
|
||||
Lardon3DSparseIncrementalImage images[4] = {
|
||||
{10, intrinsic}, {20, intrinsic}, {30, intrinsic}, {40, intrinsic}};
|
||||
Lardon3DSparseIncrementalObservation source_observations[4] = {
|
||||
{7, 10, 100, 1, 2, 640.0, 480.0},
|
||||
{7, 20, 200, 1, 2, 600.0, 480.0},
|
||||
{8, 30, 300, 1, 2, 640.0, 480.0},
|
||||
{8, 40, 400, 1, 2, 600.0, 480.0}};
|
||||
Lardon3DSparseIncrementalComponent components[2] = {
|
||||
{10, 2, 2, 1}, {30, 2, 2, 1}};
|
||||
Lardon3DSparseIncrementalCamera cameras[4] = {};
|
||||
const double identity[9] = {1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0};
|
||||
for (size_t index = 0; index < 4; ++index) {
|
||||
cameras[index].image_id = images[index].image_id;
|
||||
cameras[index].component_key = index < 2 ? 10 : 30;
|
||||
std::memcpy(cameras[index].pose_cw.rotation_cw, identity, sizeof(identity));
|
||||
}
|
||||
cameras[1].pose_cw.translation_cw[0] = -1.0;
|
||||
cameras[3].pose_cw.translation_cw[1] = -1.0;
|
||||
Lardon3DSparseIncrementalLandmark landmarks[2] = {
|
||||
{1, 7, 10, {0.0, 0.0, 5.0}, 0.5, 0.4, 2},
|
||||
{2, 8, 30, {0.0, 0.0, 6.0}, 0.5, 0.4, 2}};
|
||||
Lardon3DSparseIncrementalLandmarkObservation observations[4] = {
|
||||
{1, 7, 10, 100, 1, 0}, {1, 7, 20, 200, 1, 1},
|
||||
{2, 8, 30, 300, 1, 0}, {2, 8, 40, 400, 1, 1}};
|
||||
Lardon3DSparseIncrementalResult result = {};
|
||||
result.status = LARDON3D_SPARSE_INCREMENTAL_COMPLETE;
|
||||
result.components = components;
|
||||
result.component_count = 2;
|
||||
result.cameras = cameras;
|
||||
result.camera_count = 4;
|
||||
result.landmarks = landmarks;
|
||||
result.landmark_count = 2;
|
||||
result.observations = observations;
|
||||
result.observation_count = 4;
|
||||
Lardon3DSparseBundleAdjustmentInput input = {
|
||||
&result, images, 4, source_observations, 4};
|
||||
Lardon3DSparseBundleAdjustmentComponentDiagnostic diagnostics[2] = {};
|
||||
Lardon3DSparseIncrementalObservation resolved[4] = {};
|
||||
size_t diagnostic_count = 0;
|
||||
size_t resolved_count = 0;
|
||||
CHECK(lardon3d_sparse_bundle_adjustment_test_prepare(
|
||||
&input, diagnostics, 2, resolved, 4, &diagnostic_count,
|
||||
&resolved_count) == 0);
|
||||
CHECK(diagnostic_count == 2 && resolved_count == 4);
|
||||
CHECK(diagnostics[0].component_key == 10 && diagnostics[0].observation_count == 2);
|
||||
CHECK(diagnostics[1].component_key == 30 && diagnostics[1].observation_count == 2);
|
||||
CHECK(diagnostics[0].scale_axis == LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_AXIS_X);
|
||||
CHECK(diagnostics[1].scale_axis == LARDON3D_SPARSE_BUNDLE_ADJUSTMENT_AXIS_Y);
|
||||
return 0;
|
||||
}
|
||||
|
||||
int main() {
|
||||
if (test_helpers() != 0) return 1;
|
||||
if (test_preparation() != 0) return 1;
|
||||
if (test_invalid() != 0) return 1;
|
||||
return test_multiple_components();
|
||||
}
|
||||
Loading…
Reference in a new issue