docs(sync): close web end-to-end validation
This commit is contained in:
parent
ae4e493253
commit
aa8ab7ae69
11 changed files with 173 additions and 5 deletions
|
|
@ -17,6 +17,10 @@ Trainlog consists of:
|
||||||
|
|
||||||
The desktop SQLite database is the canonical long-term history.
|
The desktop SQLite database is the canonical long-term history.
|
||||||
|
|
||||||
|
The full-generation orchestrator and Web sync surface are implemented behind a
|
||||||
|
trusted local opt-in. Default deployed automatic exchange remains V3 until the
|
||||||
|
separate controlled-rollout decision.
|
||||||
|
|
||||||
Android has an independent local SQLite store for offline capture. SQLite
|
Android has an independent local SQLite store for offline capture. SQLite
|
||||||
database files are never synchronized directly.
|
database files are never synchronized directly.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -2,6 +2,12 @@
|
||||||
|
|
||||||
## Unreleased — 0.1.2
|
## Unreleased — 0.1.2
|
||||||
|
|
||||||
|
- Completed the grouped `TRAINLOG_SYNC_WEB_END_TO_END_V1` mission: the opt-in
|
||||||
|
full-generation orchestrator, protected asynchronous Web API, global sync
|
||||||
|
control, durable reconnect reporting, Dashboard refresh and read-only draft
|
||||||
|
summary pass isolated C, Python, Android, frontend and Firefox validation.
|
||||||
|
Default V3, physical MTP, Drive and deployment remain unchanged.
|
||||||
|
|
||||||
- Completed `TRAINLOG_SYNC_GENERATION_ACK_V1` with desktop schema v21 and
|
- Completed `TRAINLOG_SYNC_GENERATION_ACK_V1` with desktop schema v21 and
|
||||||
Android schema v20. Added coherent per-platform capture, strict bounded
|
Android schema v20. Added coherent per-platform capture, strict bounded
|
||||||
manifest V1, immutable manifest-last publication, whole-generation SQLite
|
manifest V1, immutable manifest-last publication, whole-generation SQLite
|
||||||
|
|
|
||||||
|
|
@ -323,8 +323,9 @@ As of 2026-09-17:
|
||||||
`WEB_DASHBOARD_V1=PASS/FROZEN`; the other four Web routes remain placeholders;
|
`WEB_DASHBOARD_V1=PASS/FROZEN`; the other four Web routes remain placeholders;
|
||||||
- `TRAINLOG_WEB_V1=CONTRACT_FROZEN / IMPLEMENTATION_STARTED`;
|
- `TRAINLOG_WEB_V1=CONTRACT_FROZEN / IMPLEMENTATION_STARTED`;
|
||||||
- `TRAINLOG_SYNC_GENERATION_ACK_V1=PASS/FROZEN` through explicit staged
|
- `TRAINLOG_SYNC_GENERATION_ACK_V1=PASS/FROZEN` through explicit staged
|
||||||
services; automatic synchronization still selects V3. The next cursor is
|
services; automatic synchronization still selects V3. The opt-in grouped Web
|
||||||
`TRAINLOG_SYNC_ORCHESTRATOR_REPORT_V1`.
|
path is validated in isolation; the next boundary is
|
||||||
|
`TRAINLOG_SYNC_DEPLOYMENT_READINESS_V1`, not an authorized deployment.
|
||||||
|
|
||||||
The latest executable result belongs in
|
The latest executable result belongs in
|
||||||
[current state](docs/current_state.md), not in multiple README narratives.
|
[current state](docs/current_state.md), not in multiple README narratives.
|
||||||
|
|
|
||||||
|
|
@ -414,6 +414,15 @@ carrier.
|
||||||
|
|
||||||
`trainlog-syncd` is a small user-session agent.
|
`trainlog-syncd` is a small user-session agent.
|
||||||
|
|
||||||
|
### Full-generation application orchestrator
|
||||||
|
|
||||||
|
The opt-in full-generation path is owned by `sync_orchestrator.py`, launched as
|
||||||
|
an owned process by the C Web adapter. It uses the same XDG `sync.lock` as the
|
||||||
|
legacy engine, never borrows the Web process's SQLite handle, and accepts peer
|
||||||
|
transport only from a trusted local configuration. The HTTP event loop only
|
||||||
|
admits work and reads a bounded durable report. V3 remains the default when the
|
||||||
|
trusted generation configuration is absent.
|
||||||
|
|
||||||
It polls for a new Android request and invokes the same shared C synchronization
|
It polls for a new Android request and invokes the same shared C synchronization
|
||||||
engine used by the TUI.
|
engine used by the TUI.
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -47,6 +47,10 @@ parallel implementation of its rules.
|
||||||
| Web Dashboard tiles | `WEB_DASHBOARD_TILES_V1=PASS/FROZEN` |
|
| Web Dashboard tiles | `WEB_DASHBOARD_TILES_V1=PASS/FROZEN` |
|
||||||
| Web Dashboard visualizations | `WEB_DASHBOARD_VISUALIZATIONS_V1=PASS/FROZEN` |
|
| Web Dashboard visualizations | `WEB_DASHBOARD_VISUALIZATIONS_V1=PASS/FROZEN` |
|
||||||
| Web Dashboard V1 | `WEB_DASHBOARD_V1=PASS/FROZEN` |
|
| Web Dashboard V1 | `WEB_DASHBOARD_V1=PASS/FROZEN` |
|
||||||
|
| Sync orchestrator/report V1 | `TRAINLOG_SYNC_ORCHESTRATOR_REPORT_V1=PASS/FROZEN` |
|
||||||
|
| Web sync API V1 | `TRAINLOG_WEB_SYNC_API_V1=PASS/FROZEN` |
|
||||||
|
| Web sync button V1 | `TRAINLOG_WEB_SYNC_BUTTON_V1=PASS/FROZEN` |
|
||||||
|
| Sync Web end-to-end wrapper | `TRAINLOG_SYNC_WEB_END_TO_END_V1=PASS/FROZEN` (isolated) |
|
||||||
| Local Web | `TRAINLOG_WEB_V1=CONTRACT_FROZEN / IMPLEMENTATION_STARTED` |
|
| Local Web | `TRAINLOG_WEB_V1=CONTRACT_FROZEN / IMPLEMENTATION_STARTED` |
|
||||||
| Complete synchronization gap contract | `TRAINLOG_SYNC_GAP_CONTRACT_V1=CONTRACT_FROZEN / IMPLEMENTATION_IN_PROGRESS` |
|
| Complete synchronization gap contract | `TRAINLOG_SYNC_GAP_CONTRACT_V1=CONTRACT_FROZEN / IMPLEMENTATION_IN_PROGRESS` |
|
||||||
| Isolated synchronization test environment | `TRAINLOG_SYNC_TEST_ENV_V1=PASS/FROZEN` |
|
| Isolated synchronization test environment | `TRAINLOG_SYNC_TEST_ENV_V1=PASS/FROZEN` |
|
||||||
|
|
|
||||||
75
docs/design/sync_web_end_to_end_v1_evidence.md
Normal file
75
docs/design/sync_web_end_to_end_v1_evidence.md
Normal file
|
|
@ -0,0 +1,75 @@
|
||||||
|
# Sync Web end-to-end V1 evidence
|
||||||
|
|
||||||
|
Status: `TRAINLOG_SYNC_WEB_END_TO_END_V1=PASS/FROZEN` on the development
|
||||||
|
branch. This status means implemented and validated with isolated stores and a
|
||||||
|
directory transport adapter. It does not mean physical MTP, Drive, deployment,
|
||||||
|
or a merge to `main`.
|
||||||
|
|
||||||
|
## Production call graph
|
||||||
|
|
||||||
|
```text
|
||||||
|
React SyncControl
|
||||||
|
-> POST /api/v1/sync (Host + Origin + ephemeral CSRF)
|
||||||
|
-> C Web admission (trusted opt-in config, one request identity)
|
||||||
|
-> owned sync_orchestrator.py process
|
||||||
|
-> common XDG sync.lock
|
||||||
|
-> trusted peer adapter
|
||||||
|
-> Android SyncGenerationService / desktop sync_generation_exchange.py
|
||||||
|
-> durable generation and ACK evidence
|
||||||
|
-> bounded run report
|
||||||
|
-> GET /api/v1/sync/status polling
|
||||||
|
-> Dashboard Core snapshot refresh after local commit
|
||||||
|
```
|
||||||
|
|
||||||
|
The deployed/default path remains V3. Full generation mode is unavailable
|
||||||
|
unless `TRAINLOG_SYNC_GENERATION_CONFIG` names an absolute, trusted local
|
||||||
|
configuration. HTTP input cannot select a database, executable, transport,
|
||||||
|
capability, path, timeout, or causal bypass.
|
||||||
|
|
||||||
|
## Requirement-to-proof matrix
|
||||||
|
|
||||||
|
| Requirement | Production entry point | Exact proof | Observed result |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Web admission and passive status | `web_server.c` `/api/v1/sync`, `/api/v1/sync/status` | `web_server`, `sync API` | 202/409/403/405, passive GET and CSRF header pass |
|
||||||
|
| Async owned worker | `sync_orchestrator.py` | `sync_orchestrator`, Firefox scenario | HTTP remains responsive and child is reaped |
|
||||||
|
| Durable typed phases | atomic `.sync-run.json` replacement | `test_sync_orchestrator.py` | correlated completion survives reopen |
|
||||||
|
| Common contention | XDG `sync.lock` | orchestrator conflict test and legacy lock contract | second owner fails without a second import |
|
||||||
|
| Strict compatibility | orchestrator capability set | missing-capability test | `explicitly_degraded`, missing capabilities retained |
|
||||||
|
| Generation semantics | desktop and Android generation services | `sync_generation_exchange`, `SyncGenerationServiceTest` | coherent capture, whole transaction, ACK and replay pass |
|
||||||
|
| Independent producers | public C fixture and Android repository fixture | `androidGenerationCapturePublishConsumeAndAckSurviveReopen` | both direction chains pass after reopen |
|
||||||
|
| All mandatory domains | generation descriptors and apply order | generation suites and causal seven-kind test | required empty artifacts included and validated |
|
||||||
|
| Causal immutability | `sync_causal_publications` | causal multi-generation tests | operation bytes/digests unchanged |
|
||||||
|
| Late failure | transaction-neutral apply helpers | desktop and Android late-domain tests | prior writes roll back; no consumed ACK |
|
||||||
|
| Lost/late ACK | durable ACK tables | generation restart tests | exact ACK reconstructed; replay unchanged |
|
||||||
|
| UI click/reload | `SyncControl` | `test_web_sync_browser.py` | real Firefox click, completion, draft details and reload pass |
|
||||||
|
| Draft visibility | status `drafts` projection | `SyncControl.test.tsx`, Firefox scenario | stable ID/state/type/count shown read-only |
|
||||||
|
| Dashboard freshness | `onCommitted` callback | component test and browser flow | Core snapshot refetched after committed phase |
|
||||||
|
| Mutation security | existing Host/Origin/CSRF plus strict body | `web_server`, `sync API` | proxy origin retained; no CORS or client paths |
|
||||||
|
| Optional AI | separate `ai_midpoint` and `ai_post_sync` | orchestrator report validation | not-configured remains explicit, never main success |
|
||||||
|
| Default isolation | missing trusted config | C HTTP test | button disabled and POST returns explicit conflict |
|
||||||
|
|
||||||
|
The Firefox peer command in the browser scenario is the bounded directory
|
||||||
|
transport fixture. Cross-implementation Android/desktop behavior is proven by
|
||||||
|
the production generation suites in the same final validation run. The test
|
||||||
|
does not claim physical-device timing, MTP object behavior, or Drive access.
|
||||||
|
|
||||||
|
## Bounds
|
||||||
|
|
||||||
|
- request body: existing 4 KiB HTTP limit;
|
||||||
|
- headers: existing 8 KiB limit;
|
||||||
|
- durable report and each worker output stream: 64 KiB;
|
||||||
|
- trusted configuration: 16 KiB;
|
||||||
|
- diagnostic: 1024 UTF-8 bytes;
|
||||||
|
- draft summaries: 32;
|
||||||
|
- peer command: 16 fixed arguments, each at most 4096 bytes;
|
||||||
|
- trusted timeout: 1 to 900 seconds;
|
||||||
|
- frontend polling: one request at a time, 750 ms between completions;
|
||||||
|
- generation retention: existing eight outgoing generations per consumer.
|
||||||
|
|
||||||
|
## Real limitations
|
||||||
|
|
||||||
|
Android must participate through its foreground synchronization flow to create
|
||||||
|
a publication correlated to the accepted run. No always-on Android background
|
||||||
|
service was added. A waiting phase is therefore truthful and may require user
|
||||||
|
action. Physical MTP and Drive remain rollout gates. A browser disconnect does
|
||||||
|
not cancel the worker; server shutdown terminates only its owned process group.
|
||||||
|
|
@ -701,3 +701,13 @@ Generation membership never changes a causal V1 operation. The immutable
|
||||||
operation retains null `publication_context`, its original ID and digest;
|
operation retains null `publication_context`, its original ID and digest;
|
||||||
`sync_causal_publications` records first emission and retransmission by
|
`sync_causal_publications` records first emission and retransmission by
|
||||||
operation/generation identity.
|
operation/generation identity.
|
||||||
|
## Operational full-generation run report
|
||||||
|
|
||||||
|
The Web/application boundary uses a separate bounded operational report; it
|
||||||
|
does not extend any frozen business artifact. It carries the stable run and
|
||||||
|
request IDs, trigger, requested/effective mode, peer and generation IDs,
|
||||||
|
canonical phase, monotonic progress revision, real or unavailable counters,
|
||||||
|
per-domain coverage, bounded draft summaries, diagnostics and separate AI
|
||||||
|
results. `completed` is accepted only after the worker supplies the correlated
|
||||||
|
manifest/ACK evidence. The report is at most 64 KiB and is replaced atomically
|
||||||
|
with private permissions.
|
||||||
|
|
|
||||||
|
|
@ -33,7 +33,7 @@ visualizations are implemented; the other four Web routes remain placeholders.
|
||||||
## Current cursor
|
## Current cursor
|
||||||
|
|
||||||
```text
|
```text
|
||||||
CURRENT_OPERATIONAL_CURSOR=TRAINLOG_SYNC_ORCHESTRATOR_REPORT_V1
|
CURRENT_OPERATIONAL_CURSOR=TRAINLOG_SYNC_DEPLOYMENT_READINESS_V1
|
||||||
```
|
```
|
||||||
|
|
||||||
The focused `TRAINLOG_SYNC_CAUSAL_DELETE_V1_CLOSEOUT` is complete before this
|
The focused `TRAINLOG_SYNC_CAUSAL_DELETE_V1_CLOSEOUT` is complete before this
|
||||||
|
|
@ -42,8 +42,9 @@ cross-platform draft revision identity and resource-bound admission are closed.
|
||||||
`TRAINLOG_SYNC_GENERATION_ACK_V1=PASS/FROZEN`: both platforms expose explicit
|
`TRAINLOG_SYNC_GENERATION_ACK_V1=PASS/FROZEN`: both platforms expose explicit
|
||||||
coherent capture, immutable manifest publication, whole-generation
|
coherent capture, immutable manifest publication, whole-generation
|
||||||
transactional consumption, durable correlated ACK and restart/replay entry
|
transactional consumption, durable correlated ACK and restart/replay entry
|
||||||
points. Automatic V3 selection remains unchanged. The next cursor is only
|
points. The orchestrator, Web API and Web control are now complete behind an
|
||||||
`TRAINLOG_SYNC_ORCHESTRATOR_REPORT_V1`; no orchestrator work starts here.
|
explicit trusted opt-in. Automatic V3 selection remains unchanged. The next
|
||||||
|
boundary is deployment readiness; no rollout starts here.
|
||||||
|
|
||||||
`TRAINLOG_SYNC_GAP_CONTRACT_V1=CONTRACT_FROZEN / IMPLEMENTATION_IN_PROGRESS`
|
`TRAINLOG_SYNC_GAP_CONTRACT_V1=CONTRACT_FROZEN / IMPLEMENTATION_IN_PROGRESS`
|
||||||
defines the complete-sync target and its bounded dependency order.
|
defines the complete-sync target and its bounded dependency order.
|
||||||
|
|
|
||||||
|
|
@ -777,6 +777,18 @@ and retransmission because generation membership lives only in
|
||||||
|
|
||||||
## 19. Legacy active-protocol limitations
|
## 19. Legacy active-protocol limitations
|
||||||
|
|
||||||
|
The Web end-to-end tranche adds an explicit opt-in application orchestrator.
|
||||||
|
`POST /api/v1/sync` admits one correlated run and `GET /api/v1/sync/status` is
|
||||||
|
passive. The worker owns blocking peer/transport work, shares `sync.lock` with
|
||||||
|
the TUI and daemon, and publishes bounded durable phases. Android foreground
|
||||||
|
participation is required for a fresh run-correlated publication. No filesystem
|
||||||
|
mtime or old generation can satisfy that freshness boundary.
|
||||||
|
|
||||||
|
The opt-in is `TRAINLOG_SYNC_GENERATION_CONFIG`, an absolute trusted local
|
||||||
|
configuration path. It is launcher-owned and cannot be supplied through HTTP.
|
||||||
|
Without it, the staged path reports disabled and the deployed V3 path remains
|
||||||
|
unchanged.
|
||||||
|
|
||||||
The complete-sync semantics remain frozen separately in
|
The complete-sync semantics remain frozen separately in
|
||||||
[`TRAINLOG_SYNC_GAP_CONTRACT_V1`](design/sync_gap_contract_v1.md). The staged
|
[`TRAINLOG_SYNC_GAP_CONTRACT_V1`](design/sync_gap_contract_v1.md). The staged
|
||||||
generation slice implements its batch boundary but does not activate it.
|
generation slice implements its batch boundary but does not activate it.
|
||||||
|
|
|
||||||
38
docs/sync_rollout_readiness.md
Normal file
38
docs/sync_rollout_readiness.md
Normal file
|
|
@ -0,0 +1,38 @@
|
||||||
|
# Full-generation sync rollout readiness
|
||||||
|
|
||||||
|
This is a non-executed rollout checklist. The development branch implements the
|
||||||
|
path; the daily installation still uses V3.
|
||||||
|
|
||||||
|
## Compatible set
|
||||||
|
|
||||||
|
- desktop binary containing schema v21, the Web API and the generation worker;
|
||||||
|
- `sync_generation_exchange.py` and all matching transaction-neutral codecs;
|
||||||
|
- Android schema v20 with `SyncGenerationService`;
|
||||||
|
- peers advertising manifest V1, ACK V1, causal deletion V1, history V4 and
|
||||||
|
execution draft V1 under their stable `peer_<uuid-v4>` identities;
|
||||||
|
- a trusted local orchestrator configuration referencing the packaged peer/MTP
|
||||||
|
helper with fixed arguments.
|
||||||
|
|
||||||
|
## Controlled sequence
|
||||||
|
|
||||||
|
1. Back up both SQLite stores independently and record their schema versions.
|
||||||
|
2. Stop the user services only during the separately authorized deployment.
|
||||||
|
3. Install the mutually compatible desktop helper, binary and Android APK.
|
||||||
|
4. Migrate synthetic copies first; migration rollback means restoring a backup,
|
||||||
|
not merely replacing the binary.
|
||||||
|
5. Verify peer identities and advertised capabilities without pairing by name.
|
||||||
|
6. Perform one non-destructive foreground Android publication and desktop run.
|
||||||
|
7. Verify both manifest digests, both durable ACK records, lineage and readers.
|
||||||
|
8. Keep V3 selectable until the controlled full-generation smoke is accepted.
|
||||||
|
|
||||||
|
## Recovery limits
|
||||||
|
|
||||||
|
Do not delete an unacknowledged generation, the last acknowledged generation,
|
||||||
|
its previous valid generation, or causal evidence. An ACK timeout is unresolved,
|
||||||
|
not rollback. Eight retained outgoing generations exhaust admission and require
|
||||||
|
operator diagnosis; this tranche adds no tombstone or evidence garbage
|
||||||
|
collection. A migrated database is recovered from its verified backup, never by
|
||||||
|
assuming an older binary reverses its migration.
|
||||||
|
|
||||||
|
Physical MTP, actual Drive configuration, service coordination, installed APK,
|
||||||
|
main merge and deployment are intentionally not performed here.
|
||||||
|
|
@ -1216,6 +1216,14 @@ does not author expected business JSON or repair artifacts.
|
||||||
|
|
||||||
After these additions the direct normal suites report 78/78 Meson tests and
|
After these additions the direct normal suites report 78/78 Meson tests and
|
||||||
216 Android tests: 212 passed, four historical-fixture skips, zero failures.
|
216 Android tests: 212 passed, four historical-fixture skips, zero failures.
|
||||||
|
|
||||||
|
The grouped Web end-to-end tranche raises the direct normal and sanitizer
|
||||||
|
inventories to 79/79 and adds `sync_orchestrator`, protected sync API
|
||||||
|
coverage, frontend sync assertions and a real headless Firefox scenario in
|
||||||
|
`tests/test_web_sync_browser.py`. Its detailed matrix is
|
||||||
|
[`sync_web_end_to_end_v1_evidence.md`](design/sync_web_end_to_end_v1_evidence.md).
|
||||||
|
The browser uses the production C server and worker with an isolated directory
|
||||||
|
transport fixture; physical MTP and Drive are not claimed.
|
||||||
The publication adapter is an isolated directory/object-store double. No
|
The publication adapter is an isolated directory/object-store double. No
|
||||||
physical MTP, phone, Drive, active service, or user database participates.
|
physical MTP, phone, Drive, active service, or user database participates.
|
||||||
## Causal deletion V1
|
## Causal deletion V1
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue