[ { "id": "P00", "title": "Foundation: workspace, frozen API stubs for all crates, scion-core, scion.v1 proto, contract docs, CI scripts", "wave": 0, "spec_sections": "3.2, 3.4, 4.1-4.3 (type shapes), 4.2 (ids and text form), 5.2 (row types), 6.5 (Limits), 7.1-7.8 (complete scion.v1 proto), 7.2 (error table), 8.3 (line-diff3-v1 contract), 9.2 (commit_graph formula), 18.5, 19.0 (fault registry, checkpoint names), 20.1", "owns": [ "Cargo.toml", "Cargo.lock", "rust-toolchain.toml", "rustfmt.toml", "clippy.toml", ".cargo/config.toml", ".gitignore", "crates/*/Cargo.toml (all 23 crates)", "crates/*/src/lib.rs (all 23 crates)", "crates/scion-proto/build.rs", "proto/scion/v1/scion.proto", "crates/scion-core/**", "crates/scion-history/src/{hooks.rs,config.rs,types.rs,graph_compute.rs}", "crates/scion-derived/src/types.rs", "crates/scion-server/src/{grpc.rs,handlers/mod.rs,authz/mod.rs}", "crates/scion-wc/src/points.rs", "initial stub content of every other module file listed in PACKAGES.json (ownership of each transfers to the listed package)", "scripts/{check.sh,check-layers.sh,layers.toml,test-external.sh}", "docker/compose.test.yml", ".github/workflows/ci.yml", "docs/contracts/{line-diff3-v1.md,storage-semantics.md,amendments.md}", "docs/implementation-defined.md", "docs/testing.md", "testdata/README.md" ], "depends_on": [], "deliverables": "Create the whole Cargo workspace so that it compiles with every public API of docs/PLAN.md section 2 present. Concretely: (1) Root Cargo.toml: resolver = 3, edition 2024, an explicit members list of the 23 crates in PLAN 1.2. It has [workspace.package] and [workspace.lints] (rust unsafe_code = deny; clippy unwrap_used/expect_used/panic = deny, with clippy.toml allow-unwrap-in-tests/allow-expect-in-tests = true). It has [workspace.dependencies] pinning EXACT versions, resolved at scaffold time, of every third-party crate in PLAN 5.3, plus the profiles in PLAN 3.5. Generate and commit Cargo.lock. rust-toolchain.toml pins the installed 1.98 toolchain with rustfmt and clippy. (2) Every crate Cargo.toml declares all internal deps from PLAN 1.2 and all third-party deps it will need (PLAN 5.3 column 'used by'), dev-deps included, so later packages never edit manifests. It also declares features: scion-server/testing, scion-daemon/testing, scion-wc/testing, scion-import/testing, scion-auth/test-hooks. Bins: scion-server, scion-daemon, scion (crate scion-cli), scion-import; each src/main.rs is a stub. (3) Fully implement scion-core exactly as PLAN 2.1, with unit tests: error table, ids and text form, empty_tree_id, Mode, conflict types, Principal, Tier, Clock/ManualClock, Limits defaults, FaultRegistry semantics, points, trace, test_support. (4) Write proto/scion/v1/scion.proto completely per PLAN 2.13 (all six services, every message). crates/scion-proto/build.rs uses tonic-prost-build (protoc is on PATH) and emits a file descriptor set. The v1 module and FILE_DESCRIPTOR_SET are real; the other scion-proto modules are stubs. (5) Every other crate gets lib.rs and the module files named in PACKAGES.json owns lists. Each holds every PLAN section 2 item with rustdoc that cites the SPEC section. Bodies follow PLAN 2.0: todo!(\"\") for behaviour; real assembly constructors and accessors (History::new, ObjectRepo::new, Storage::new, Storage::in_memory, MemMetaStore::new, MemBlobStore::new, DerivedEngine::new, StoreDeriveCtx::new, Registry::empty, Redactor::new, ObjectAuthorizer::new, RedactionSet::new); real trivial impls of NoEager and AllowAll; real compute_commit_graph (9.2: generation = 1 + max parent generation, root 1; skip[k] = first-parent ancestor at distance 2^k); no-op handlers::admin::spawn_background. Handler stubs return Err(Status::unimplemented(\"Pxx\")). scion-server/src/grpc.rs is real: it implements the six generated tonic server traits, and each method delegates in one line to the handlers:::: function listed in PLAN 2.14. (6) docs/contracts/line-diff3-v1.md (normative, PLAN 6.3): line splitting. Myers forward greedy O(ND) with the exact k-selection rule: move down (insert) when k == -d or (k != d and V[k-1] < V[k+1]), else move right (delete). Snake extension, storage of V per d, backtracking with the same rule, no heuristics, no cost cap. Edit-script normalization (within a changed run, deletions are listed before insertions). Hunk extraction. diff3 region building (transitive closure of b->o and b->t hunks whose base ranges overlap or touch, where an empty insertion range touching or inside another range counts as touching). A one-sided region applies. A two-sided region applies once if both sides produce identical text, otherwise it conflicts. Result assembly. At least 12 hand-worked examples, including tie-sensitive ones (swap AB/BA, repeated lines, empty inputs, unterminated last line, CRLF). (7) docs/contracts/storage-semantics.md: for every MetaStore method, the transaction boundary, reads, writes, uniqueness/conflict behaviour, outcome mapping, the CockroachDB DDL/SQL shape (all SPEC 5.2 tables, plus audit_log and secondary indexes: changeset id prefix, derived source_object, redactions epoch, redactions purge_after) and notes on the Mem implementation. Also ObjectRepo write ordering and the key layout. (8) docs/contracts/amendments.md (protocol plus an empty log). docs/implementation-defined.md: transcribe PLAN section 6 verbatim as the documented implementation-defined behaviours. docs/testing.md: summary of PLAN section 3 with env vars and commands. testdata/README.md. (9) Scripts. scripts/check.sh runs fmt check, build, test, clippy -D warnings and check-layers; --full sets SCION_FULL=1 and runs cargo test --release -- --include-ignored. scripts/check-layers.sh plus scripts/layers.toml implement PLAN 1.3 from cargo metadata, with a --self-test mode proving an injected upward edge is rejected. scripts/test-external.sh. docker/compose.test.yml: cockroachdb single-node insecure on 26257 and MinIO on 9000, with bucket creation. .github/workflows/ci.yml: check.sh on push; nightly --full and external jobs. (10) Before finishing, self-review against SPEC.md and PLAN section 2. cargo doc --workspace --no-deps must be warning-free. Gate G0 (a fresh-agent contract review) follows this package. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "scion-core unit tests: ErrorCode::ALL maps to the exact SPEC 7.2 gRPC status and retryable flag (19.7). ObjectId text form round-trips for every kind; ids with equal digests and different kinds are unequal and print differently (19.1). parse_text rejects bad input. ChangeId hex round-trips. empty_tree_id equals BLAKE3 of [0x01,0x00,0x00] with kind Tree. Principal parse/display. FaultRegistry: arm with times, hits counted when unarmed, Crash latch until reset, Delay. compute_commit_graph on hand-built rows (root generation 1; skip pointers at distances 1, 2, 4, 8 on a 10-chain; merge generation = 1 + max). Limits::default matches SPEC 6.5. scripts/check-layers.sh passes, and --self-test proves an upward edge is rejected. cargo build, clippy and doc are clean.", "size": "XL" }, { "id": "P01", "title": "scion-object: canonical codec, entities, path/ref/revision grammar, golden vectors", "wave": 1, "spec_sections": "4.1, 4.2 (ids inside objects), 4.3.1-4.3.4, 4.3.6 (ref name grammar), 4.4, 9.2 (HistoryNode encoding, derived kind 0x10), 11.8 (revision grammar parsing), 17 (decoded-form validation)", "owns": [ "crates/scion-object/src/{enc.rs,canonical.rs,extensions.rs,path.rs,tree.rs,meta.rs,changeset.rs,history_node.rs,refname.rs,revision.rs,flat.rs}", "crates/scion-object/tests/**", "testdata/golden/objects/**" ], "depends_on": [ "P00" ], "deliverables": "Implement PLAN 2.2 completely. Encoder/Decoder: uvarint is minimal LEB128 of at most 10 bytes, with overflow rejected; svarint is zigzag; bytes/list lengths may not exceed the remaining input; option tag must be 0 or 1; finish() rejects trailing bytes. All errors are INVALID_ARGUMENT with the scion-core decoding subcodes. Extensions: strictly ascending unique tags. Unknown odd tags are rejected (unknown_critical_extension). Even tags are preserved byte-for-byte; v1 has no known odd tags. Tree: format_version must be 1 (else unknown_format_version). Entries are strictly ascending by bytewise name and unique; names obey 4.4. Modes 1-3 require a Blob id, mode 4 requires a Tree id with size 0. validate_non_root rejects an empty tree (empty_tree). Meta: fields in SPEC order; Timestamp is a pair of svarints. Changeset: parents of kind Changeset, unique; root_tree of kind Tree; meta of kind Meta; change_id is 16 raw bytes; delta strictly ascending by path, op 1 = Put (mode 1..=3, id of kind Blob, size) and 2 = Delete. validate_shape(limits) checks parent count/uniqueness, delta sort/uniqueness, max_delta_entries and path rules (TOO_LARGE for limits, with a Limit payload). HistoryNode: format_version 1, kind u8 (1 File, 2 Dir), content, changeset, parents list, children list<(name, id)> ascending (must be empty for File), extensions. KIND = Derived(0x10). Name/RepoPath: 1-255 bytes, no NUL or '/', not '.' or '..', not '.scion' under ASCII case-insensitive comparison; paths are at most 4096 bytes with no leading/trailing '/' and no empty component; the root is the empty path; ancestors() and is_ancestor_of are component-wise. RefName: ^[a-z0-9][a-z0-9._/-]{0,199}$, with no '..', no '//' and no trailing '/'. parse_revision covers: full id text; 'c1' plus 8..63 hex (IdPrefix); a bare name, treated as a ref name or a bare hex prefix of at least 8 digits, decided later by the server; '@'; 'git:<40 hex>'; and chains of '~n' / '^k' suffixes (a bare '~' or '^' means 1; ^0 is invalid). flat::file_diff implements SPEC 4.3.4 FileDiff over FlatTree. Golden vectors: testdata/golden/objects/*.json with at least 14 cases: empty tree; a tree with all four modes; a tree with an even extension; a Meta with and without committer; a Meta with extension tag 2; the root changeset; a changeset with 2 parents and mixed Put/Delete; a changeset with an even extension; HistoryNode File and Dir; a blob. Each file is {name, kind, description, hex, id_text}. A test re-encodes every case from its constructor in Rust and compares bytes and id. A second test asserts a BLAKE3 digest constant over the sorted golden file contents, so any edit to the files fails CI (19.1 freeze). Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.1: golden vectors frozen (encode equality plus digest constant). Round trip decode(encode(x)) == x and encode(decode(b)) == b with proptest over random valid Tree/Meta/Changeset/HistoryNode. The default count is proptest_cases(10_000); a #[ignore] full_ test runs 10^6. One rejection test per class: non-minimal varint, 11-byte varint, length overrun, trailing bytes, unsorted entries, duplicate entries, unknown format_version, unknown odd extension tag, invalid mode, mode/kind mismatch. An unknown even tag is preserved byte-for-byte and changes the id. Equal digests with different kinds are never confused (a Tree id is rejected where a Blob id is required). 19.2 path-rule violations at parse level: '.', '..', '/', NUL, '.scion'/'.SCION'/'.Scion', 256-byte name, 4097-byte path, leading/trailing '/'. Ref-name grammar table. Revision grammar parse table (at least 25 cases, valid and invalid). file_diff unit cases (file<->dir transitions produce Delete plus Put).", "size": "M" }, { "id": "P02", "title": "scion-merge: normative Myers diff and line-diff3-v1 engine", "wave": 1, "spec_sections": "8.3 (content merge, binary detection, decision table), 9.2 (blame line mapping), 7.4 (Diff hunks), 11.7 (marker format), 19.4", "owns": [ "crates/scion-merge/src/{split.rs,myers.rs,hunks.rs,merge3.rs,binary.rs,classify.rs,markers.rs,origins.rs,unified.rs}", "crates/scion-merge/tests/**", "testdata/golden/diff/**" ], "depends_on": [ "P00" ], "deliverables": "Implement PLAN 2.5 exactly according to docs/contracts/line-diff3-v1.md. That document is normative; if it is ambiguous, follow its worked examples and file an amendment. Do not use any third-party diff crate in src (similar is allowed only as a dev-dependency to cross-check edit-script minimality). split_lines keeps terminators. myers is the forward greedy O(ND) algorithm with the contract's k-selection rule and backtracking. diff_lines groups non-equal runs into hunks. is_binary: a NUL within the first 8000 bytes. merge3: region building, one-sided apply, identical two-sided apply-once, conflicts reported as ConflictHunk line ranges (base/ours/theirs, u32), and byte-exact output (no CRLF normalization, unterminated final lines preserved). classify implements 8.3 rows 1-6 in order: o==b take t; t==b take o; o==t take o; b absent with o and t present and different gives AddAdd; exactly one side absent and the other differing from b gives ModifyDelete; any side a symlink with sides differing gives Symlink; otherwise NeedContent. merge_mode applies the first three rows to modes. render_markers writes the exact 11.7 marker lines. line_origins maps child lines to parent lines via diff_lines (for blame). unified_hunks produces unified-diff hunks with context. Produce at least 60 golden vectors in testdata/golden/diff/*.json: diff cases {a_hex, b_hex, edits} and merge cases {base_hex, ours_hex, theirs_hex, result_hex | conflicts}. Include all worked examples from the contract doc and tie-break-sensitive inputs. Freeze them with a digest-constant test. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.4 content level: o==b => t; t==b => o; o==t => o; non-overlapping, non-adjacent hunks from both sides both apply; overlapping hunks conflict; adjacent hunks (zero unchanged lines between) conflict; identical hunks apply once. Each property runs with proptest_cases(10_000), with #[ignore] full_ variants at 10^6. Binary boundary: NUL at byte 7999 is binary, at byte 8000 is text. CRLF and unterminated-final-line merges are byte-exact. classify has one test per 8.3 table row. Myers optimality: edit length equals an O(N*M) LCS-based minimum on small random inputs. Determinism: the same input always gives the same output. Golden vectors are frozen. RECOMMENDED: a differential against `git merge-file -p` on clean text merges (skipped when git is absent), with disagreements recorded in crates/scion-merge/tests/GIT_MERGE_FILE_DIFFERENCES.md.", "size": "M" }, { "id": "P03", "title": "scion-store: in-memory MetaStore/BlobStore, ObjectRepo, counters and fault decorators, fsck, backend conformance suites", "wave": 1, "spec_sections": "5.1, 5.2, 5.3 step 1, 5.4, 4.5 (fsck), 9.1 (leases), 13.2 (redaction transactions and byte moves), 16.1 classes 2/3/7, 17 (audit log, decoded-form validation), 19.0 (counters, fault hooks), 19.3", "owns": [ "crates/scion-store/src/{types.rs,meta.rs,blob.rs,storage.rs,keys.rs,mem_meta.rs,mem_blob.rs,repo.rs,instrument.rs,fsck.rs,conformance.rs}", "crates/scion-store/tests/**" ], "depends_on": [ "P00" ], "deliverables": "Implement PLAN 2.3 and docs/contracts/storage-semantics.md. MemMetaStore: all state is im persistent maps behind one parking_lot::Mutex, and each trait method is one atomic step (strictly serializable). Required behaviour: - ON CONFLICT DO NOTHING for objects, changesets, children, object_paths and derived; commit_changeset inserts every row independently, so a re-commit fills in missing rows. - cas_ref: RefExpect::Absent or Version(v); on success version becomes v+1 (1 on create); ref_log row, landed rows and audit row in the same step; a landed (ref, change_id) unique violation returns LandedConflict and applies nothing; a version mismatch returns the current row. - redact: increments counters.redaction_epoch, inserts rows stamped with the new epoch, sets objects.state = quarantined. unredact: Quarantined only (else INVALID_ARGUMENT/not_quarantined), epoch+1, state live. mark_purged: clears inline_data, state purged. - Leases are TTL-based against the provided now. - A ring of the last K committed snapshots serves Consistency::Stale (stale_lag_commits, set_stale_lag, advance_stale_view). - fail_txn_every: every Nth transaction fails with UNAVAILABLE/txn_retry and no effect. MemBlobStore supports corrupt(), set_down() and keys(). keys.rs implements the SPEC 5.1 layout. InstrumentedMeta/InstrumentedBlob count meta_txns, meta_rows_read, object_paths_rows_read, lease_calls and blob ops/bytes, and check fault points meta..before / meta..after and blob..before / blob..after (FailAfter means the effect is applied and the call then fails). Leases emit a tracing span with target scion::lock. ObjectRepo: - put_blob/put_encoded/put_verified validate size (TOO_LARGE) and the decoded form of known kinds (Tree/Meta/Changeset/HistoryNode), and reject redacted ids (REDACTED). - Blobs always go to object storage. Tree/Meta/Derived objects at or below inline_threshold_bytes are stored inline, others as zstd frames at zstd_level. - Write order: bytes first (checkpoint upload.after_bytes), then the objects row (checkpoint upload.after_row); a live row therefore always implies durable bytes. - get/get_frame verify BLAKE3 of the decompressed content; on mismatch they record an integrity failure (Counters plus an ERROR log event 'integrity_alert') and return INTEGRITY. - Every read records object reads by kind and bytes. - stat/get use a stale read first and retry a miss with a strong read before NOT_FOUND. - LRU caches of decoded trees, metas and changesets serve immutable reads even while the metadata store is down. - quarantine_bytes copies to the quarantine key and then deletes the live key (checkpoint quarantine.after_copy). restore_bytes reverses it. purge_bytes deletes both keys. fsck walks every changeset and checks SPEC 4.5: digest of every live object, presence of root tree, meta, parents and delta blobs, delta == FileDiff(parent tree, root tree), generation and skip consistency, required eager rows, and ref targets exist. Quarantined or purged bytes are not violations. conformance::meta_store_suite and blob_store_suite are public async fns (always compiled, a test-support API). They cover every method contract, including the concurrency cases below, and run against MemMetaStore/MemBlobStore in this package's tests. P15 reuses them for CRDB/S3. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "Tests in crates/scion-store/tests: - meta_store_suite and blob_store_suite on Mem. - CAS: 1000 concurrent cas_ref calls at the same expected version give exactly one Committed; versions advance by exactly 1; ref_log has no gaps (19.5, store level). ABA: A->B->A, then a CAS expecting the first version fails. - LandedConflict rolls back the whole cas_ref. - 19.3: a blob-store fault before put leaves no objects row; FailAfter on put still leaves no row unless the row insert ran; a row never exists without bytes. Inline threshold at threshold-1, threshold and threshold+1 stores in the expected place and reads back identically. A stale-read miss is retried strongly (stale_lag) before NOT_FOUND. Orphan bytes (a put with no row) are invisible to stat/get. - 19.1: a blob id is independent of zstd level (1, 3, 19) and framing (a multi-frame encoding of the same content). - Corrupt stored bytes give INTEGRITY and integrity_failures +1 (16.1 class 7). - fail_txn_every leaves no partial state. - Redaction epoch increments atomically with the state changes. - Counters count exactly (tree reads include inline trees). - fsck detects each Violation kind on hand-corrupted state and passes on valid state.", "size": "L" }, { "id": "P04", "title": "scion-config and scion-obs: TOML configs, structured logging, metrics, health", "wave": 1, "spec_sections": "6.1, 6.3, 6.4, 6.5, 15.1, 15.2, 15.3, 16.1 class 7 (integrity alert metric), 19.15, 19.17", "owns": [ "crates/scion-config/src/**", "crates/scion-config/tests/**", "crates/scion-obs/src/**", "crates/scion-obs/tests/**" ], "depends_on": [ "P00" ], "deliverables": "Implement PLAN 2.11 and 2.12; lib.rs and Cargo.toml stay P00's. Config: - serde with #[serde(deny_unknown_fields)] everywhere, wrapped in serde_path_to_error so ConfigError::UnknownKey names the full dotted key (for example 'limits.max_blobs'). - '$NAME' string values are resolved from Env in a pre-pass over toml::Value, keeping the key path. An unresolved variable is ConfigError::UnresolvedVar naming the variable and the key. - Every default from SPEC 6.5 and PLAN 2.11 (including the implementation-defined (+) keys) applies when a key is omitted. - Durations use humantime ('12h', '7d'); millisecond keys are integers. - resolve_endpoint precedence: flag > SCION_ENDPOINT > working-copy config > user config. - for_tests constructors. Obs: - init_logging emits JSON (one event per line) or text. - RpcLogEvent::emit logs the REQUIRED 15.1 fields: request_id, rpc, tier, principal, repo, status, duration_ms, and when applicable changeset_id, ref, ref_version, land_attempt. - path_hash returns 16 hex characters of BLAKE3. Raw repository paths are never logged (PLAN 6.9 log access policy). - Metrics registers every SPEC 15.2 metric with the exact names and labels (scion_rpc_duration_seconds{rpc,tier,code}, scion_cache_requests_total{tier,kind,result}, scion_cache_bytes{tier}, scion_land_attempts{result}, scion_ref_cas_failures_total{ref}, scion_land_duration_seconds, scion_derive_duration_seconds{type,mode}, scion_derive_backlog{type}, scion_redaction_epoch{tier}, scion_redaction_lag_seconds{tier}, scion_policy_epoch_info{epoch}, scion_object_bytes_served_total{tier,encoding}, scion_upstream_requests_total{tier}), plus scion_integrity_failures_total{tier}. NAMES lists them all. - health_router serves /metrics (text exposition), /healthz (200 while alive) and /readyz (200 or 503 from Readiness::check). serve_admin binds it. - capture::LogCapture installs a process-global in-memory JSON layer for tests, with find_needles for log scans. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.15: an unknown key is rejected with the key name, at top level and in a nested section, for server, daemon and client configs. $VAR resolves. An unresolved $VAR errors, naming the variable and key. Every SPEC 6.5 default is applied when its key is omitted (one assertion per row of the 6.5 table, plus the (+) keys). The SPEC 6.1 example file parses with env vars supplied. 19.17 unit level: every 15.2 metric name appears in render() after one observation; /healthz returns 200; /readyz returns 503 then 200 as a toggled Readiness changes; RpcLogEvent produces all required fields as JSON keys; path_hash is stable and never equals the input; LogCapture.find_needles finds a planted secret.", "size": "M" }, { "id": "P05", "title": "scion-policy: acl.toml parsing, evaluation, epoch, hot reload", "wave": 1, "spec_sections": "6.2, 12.4, 4.3.8, 16.1 class 9, 19.12 (policy semantics and reload)", "owns": [ "crates/scion-policy/src/**", "crates/scion-policy/tests/**" ], "depends_on": [ "P00" ], "deliverables": "Implement PLAN 2.9. Parse with toml and serde (deny_unknown_fields; UnknownKey names the key). version must be 1. A '/' rule is REQUIRED (MissingRootRule). Groups: nested groups are resolved; cycles are a GroupCycle error listing the cycle; unknown groups are rejected. Principal forms: 'github:' (compared case-insensitively), 'github-id:' (matches Principal::GitHub{id: Some(n)}), 'service:', 'group:'. Prefixes are written with a leading '/' and normalized (no trailing '/'). Matching is component-wise, so '/secrets' matches secrets and secrets/x but not secrets2. For each path, the longest matching rule governs, independently for read and for write; rules never merge. read_governed_by_root reports whether the governing read rule is '/'. Refs: an exact name beats any pattern; among patterns (a single trailing '*' segment) the longest wins. Admin permissions: redact, purge, refs. Default deny. epoch = first 8 bytes, big-endian, of BLAKE3 over a canonical normalized serialization (sorted groups, rules and principals), so that equivalent files agree. PolicyHandle: arc-swap of Arc; load() fails on an invalid or missing file. reload_now swaps atomically only when the new policy is valid. An invalid file keeps the last good policy, records last_error and logs an operator-visible ERROR. A watch channel publishes epoch changes. spawn_watcher uses notify plus a 1 s polling fallback. Tests use reload_now for determinism. allow_all_for_tests() grants everything to everyone. impl From for ScionError (INVALID_ARGUMENT). Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.12 policy semantics: longest prefix wins; component-wise matching ('/secrets' vs '/secrets2'); independent read and write resolution; nested groups; cycle detection; the github-id: form; case-insensitive github logins; ref pattern precedence (exact > longest pattern) and default deny; admin permissions. Reload: a valid change is applied atomically and the epoch changes; an equivalent reformatted file gives the same epoch; an invalid change keeps the last good policy, surfaces last_error and leaves the epoch unchanged; an invalid or missing file at startup fails. A concurrency test: readers never observe a torn policy during repeated reloads.", "size": "M" }, { "id": "P06", "title": "scion-auth: Ed25519 PASETO tokens, keyring rotation, constant-time secrets, GitHub client, device flow, FakeGitHub", "wave": 1, "spec_sections": "12.1, 12.2, 12.3, 17 (secrets never logged), 16.1 class 10, 19.11", "owns": [ "crates/scion-auth/src/**", "crates/scion-auth/tests/**", "crates/scion-testkit/src/fake_github.rs", "crates/scion-testkit/tests/fake_github_*.rs" ], "depends_on": [ "P00" ], "deliverables": "Implement PLAN 2.10 and the FakeGitHub part of PLAN 2.21. Tokens: PASETO v4.public via pasetors (Ed25519). Claims sub, gid, iat, exp. kid goes in the footer as JSON {\"kid\":...}; kid = first 16 hex characters of BLAKE3(public key). Keyring::from_files reads PKCS#8 PEM Ed25519 keys. The first key signs and all keys verify. generate_for_tests and write_pem_files support harnesses. Verifier::verify returns UNAUTHENTICATED for malformed, unknown-kid, bad-signature, tampered and expired tokens (exp <= now). ct_eq wraps subtle::ConstantTimeEq and is the ONLY comparison primitive for secrets; under feature test-hooks it counts calls. ServicePrincipals::verify always computes SHA-256(secret) and calls ct_eq, comparing against a dummy digest for unknown names. Secret has redacting Debug/Display. GitHubClient::user calls GET {api_url}/user with 'Authorization: Bearer'. 401/403 maps to UNAUTHENTICATED; transport errors and 5xx map to UNAVAILABLE. device_flow::start/poll implement the GitHub device flow (POST {oauth_base}/login/device/code, POST {oauth_base}/login/oauth/access_token), handling authorization_pending, slow_down and expired_token. FakeGitHub (axum on 127.0.0.1:0) serves GET /user (tokens registered with add_user), the device-code endpoints (approve_device_code completes the flow) and a set_down switch returning 503. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.11 token items: a valid token verifies with the right sub and gid; expired, malformed, wrong-key, tampered-payload and tampered-footer tokens are rejected. Rotation: a token signed by key 2 verifies while key 2 is listed and fails once it is removed; new tokens are signed by the first key. Service secret: right succeeds, wrong fails, unknown name fails; ct_eq_calls increases on BOTH the right and the wrong path (the constant-time primitive is used on both). GitHubClient against FakeGitHub: valid token maps login and id; invalid token gives UNAUTHENTICATED; FakeGitHub down gives UNAVAILABLE. A device-flow round trip against FakeGitHub, including authorization_pending. Secret Debug output never contains the secret.", "size": "M" }, { "id": "P07", "title": "scion-refmodel: independent reference model (19.0 oracle)", "wave": 1, "spec_sections": "4.1-4.4, 4.3.4 validity, 8.1 (validation outcomes), 8.2, 8.3, 8.4 (sequential land semantics), 9.2 (commit_graph, history TREESAME, blame), 11.2-11.3 (materialization, sparse scope), 13.2 (--all-versions), 13.5 (redacted content as empty)", "owns": [ "crates/scion-refmodel/src/**", "crates/scion-refmodel/tests/**" ], "depends_on": [ "P00" ], "deliverables": "Implement PLAN 2.20 as a deliberately naive, single-threaded oracle. INDEPENDENCE RULES: the only third-party dependency is blake3 (and std). Do NOT read or copy any production crate under crates/ other than this one; work only from SPEC.md and docs/contracts/line-diff3-v1.md. check-layers.sh verifies that there are no scion dependencies. Contents: - Its own canonical encoder (enc) for trees (built recursively from the flat MTree), Meta and Changeset, plus id text. - apply_delta, which flattens, applies and rebuilds, returning the 8.1/18.2 subcode strings on invalid deltas (delete_absent, delete_not_file, noop_put, file_directory). - file_diff. - Its own Myers and diff3 written from the contract doc, using the simplest correct code, with no optimizations. - merge_trees implementing the full 8.3 table, both structural file_directory rules, the binary and merge_max_text_bytes rules, and redacted conflicts. - MRepo: commit graph (generation, skip pointers by naive walk), is_ancestor (full BFS), merge_bases (full ancestor-set intersection minus dominated), rebase per 8.2 (fast path, EMPTY), log_path with TREESAME semantics equivalent to the 9.2 history node-reuse rule (follow parents[0] when the content equals parents[0]'s, otherwise all parents where the path exists), blame per 9.2 (first parent, then subsequent parents, remaining lines to the commit; redacted content empty), all_versions (distinct contents ever held). - land_sequential. - materialize(sparse) per 11.2. O(n^2) algorithms are acceptable. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "Self-tests on hand examples taken from the SPEC tables: every 8.3 row and both structural rules; the 18.2 apply_delta subcodes; root changeset rules; generation and skip on a hand DAG including an octopus merge; a criss-cross DAG giving two merge bases; the TREESAME log on a merge that keeps the first parent's version; blame on a 3-commit example; all worked examples of docs/contracts/line-diff3-v1.md reproduced exactly; materialize with nested sparse prefixes. Property: file_diff(t, apply_delta(t, d)) == d for random t and random valid d (proptest_cases(5_000)). Cross-checks against production code are done later by P17 and other packages.", "size": "L" }, { "id": "P08", "title": "scion-proto: conversions, error mapping, epoch trailer layer, ScionClient, RawCodec; testkit FakeUpstream and TestTls", "wave": 1, "spec_sections": "7.1, 7.2, 7.3-7.7 (message mapping), 7.8 (verbatim forwarding codec), 3.3, 17 (TLS), 19.7 (error mapping, forward compatibility)", "owns": [ "crates/scion-proto/src/{convert.rs,status.rs,trailers.rs,client.rs,endpoint.rs,raw.rs}", "crates/scion-proto/tests/**", "crates/scion-testkit/src/{fake_upstream.rs,tls.rs}", "crates/scion-testkit/tests/upstream_*.rs" ], "depends_on": [ "P00" ], "deliverables": "Implement the Rust surface of PLAN 2.13 (the proto file and codegen are P00's and frozen), plus FakeUpstream and TestTls from PLAN 2.21. convert: From/TryFrom between v1 messages and scion-core/scion-object/scion-store types (ObjectId bytes, EntryInfo, DeltaEntry, MetaFields, ConflictReport, LandedEntry, RefRow/RefLogRow, Limits, ServerInfo pieces). A missing required field or a malformed id gives INVALID_ARGUMENT/missing_field or bad_id_text. Unknown fields are ignored (prost default). status: status_from_error sets the tonic code from ErrorCode::grpc_code and the message, and puts the ErrorDetail encoding (code string, retryable, message, payload oneof) into Status details. error_from_status inverts it; a Status without details maps from its gRPC code, defaulting to UNAVAILABLE. trailers: TRAILER_* constants, read_epochs/write_epochs. EpochTrailerLayer is a tower Layer over http Request/Response bodies. It appends scion-policy-epoch, scion-redaction-epoch and scion-tier to the HTTP/2 trailers frame of EVERY response: unary, server-streaming, client-streaming, error, and trailers-only responses (where tonic puts grpc-status in the headers, add the three keys to those headers). Epoch values are decimal u64. endpoint: Endpoint::parse accepts 'unix:///abs/path' and 'https://host:port'. client: ScionClient::connect uses tonic Channel over a UnixStream connector (hyper-util) or https with rustls (CA from TlsOptions, otherwise native roots). It installs a bearer-token interceptor and a response layer capturing the last epochs from trailers (and trailers-only headers), and keeps an atomic call counter (calls/reset_calls). raw: RawCodec is a tonic Codec whose Encode and Decode are both bytes::Bytes, for byte-level forwarding by the daemon. TestTls::generate creates an rcgen CA plus a server certificate for localhost and 127.0.0.1. FakeUpstream: an in-memory tonic server on 127.0.0.1:0 (TLS optional, via TestTls) implementing: ObjectService (GetServerInfo with configured public keys; GetObjects with per-item ok/not_found/denied/redacted, zstd or raw frames, offset/length ranges, access_class, kill_stream_after_bytes; HasObjects; ResolveRef); QueryService GetChangesets and Resolve; AuthService CheckAccess and WhoAmI; AdminService ListRedactions. Every other method returns UNIMPLEMENTED. The principal comes from the bearer token via the configured principal_of closure. The server is wrapped in EpochTrailerLayer using set_epochs values (tier server). It has set_down, corrupt, redact and deny knobs, and records every request (method, ids, principal). scion-proto tests must not dev-depend on scion-testkit (dependency rule PLAN 1.2); proto-level transport tests use rcgen directly. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.7: every ErrorCode round-trips through status_from_error/error_from_status with the correct tonic code and retryable flag, and every payload variant round-trips. A request with unknown fields (hand-encoded extra tag) decodes. Missing required fields in CreateChangesetRequest/MergeRequest give INVALID_ARGUMENT/missing_field. Property round trip for conversions. Transport tests (crates/scion-proto/tests, rcgen certificates, a minimal ObjectService implementation defined in the test): over TLS and over a Unix socket, ScionClient sees all three trailers on a unary success, a unary error, a server-streaming response, a client-streaming response and a trailers-only error, and last_epochs is updated; the call counter counts. RawCodec forwards a unary and a streaming call byte-identically through a test proxy. FakeUpstream tests (crates/scion-testkit/tests/upstream_*.rs): batch mixing ok/not_found/denied/redacted; range read; stream kill and resume by offset; requests are recorded.", "size": "L" }, { "id": "P09", "title": "scion-testkit corpus: generators, shared-subtree repos, edge cases, vendored git bundles, linearizability checker", "wave": 1, "spec_sections": "19.0 (Corpus, History Recorder), 19.5 (linearizability checker), 19.14 (real git histories), 19.18 (equal-depth generated repos)", "owns": [ "crates/scion-testkit/src/{corpus.rs,shared.rs,edge.rs,encode.rs,git.rs,lincheck.rs}", "crates/scion-testkit/tests/corpus_*.rs", "crates/scion-testkit/tests/lincheck_*.rs", "corpus/git/**" ], "depends_on": [ "P00" ], "deliverables": "Implement PLAN 2.21 except fake_upstream.rs and tls.rs (P08) and fake_github.rs (P06). corpus::generate(CorpusSpec): a seeded (rand_chacha) generator of GenRepo histories of structural MTree snapshots, with configurable file count, depth, fanout, history length, merge ratio, octopus ratio, binary ratio and symlink ratio, plus criss-cross shapes. Text edits are line-level, so merges are meaningful; output is deterministic per seed. shared::shared_subtree_repo(files, depth): a balanced tree of identical subtrees reaching 10^3, 10^5 or 10^7 logical files at the SAME depth (fanout chosen per size, depth fixed), perturbable along one path. encoded_objects() returns only the O(depth x fanout) distinct objects. edge::edge_cases(): empty root, deep path at the 4096-byte limit, 255-byte names, case-only-differing names, file<->dir flips, CRLF, NUL content (binary), executable flips, symlink retargets, delete-and-readd, unterminated last lines. encode::encode_tree uses scion-object to produce production-encoded trees and blobs from an MTree. It is exercised first by P17; this package must not run scion-object code in its own tests, because P01 is in the same wave. git: vendor at least two REAL open-source git histories as bundles under corpus/git/ (candidates: rust-lang/cfg-if and dtolnay/itoa, both MIT OR Apache-2.0 and small with merge commits). Verify license and size (each at most 5 MB, full history via `git bundle create --all`), and add corpus/git/README.md plus LICENSE- files recording source URL, commit and license. Also build and commit synthetic edge bundles with the git CLI: an octopus merge, multiple roots, empty commits, mode 100664, symlinks, a gitlink, a merge keeping the first parent's version, and rename-free line edits for blame. Tests must then not require the git CLI; bundles(), unpack() (via gix) and GitFixture (which needs git and skips when it is absent) are provided. If network access is impossible, stop and report instead of substituting synthetic history for the real-repository requirement. lincheck: HistoryRecorder (thread-safe, monotonic timestamps) and check_ref_registers, a WGL linearizability search with memoization over a versioned-register model per ref. Read returns (version, target); Cas succeeds iff its expect equals the current version; Land succeeds with version+1. An Unknown completion may or may not have taken effect (crash or timeout). Histories are partitioned per ref; document the cap of about 200 operations per history. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "Generators are deterministic per seed and realize the requested shapes (octopus, multiple roots, criss-cross detected structurally). The shared-subtree repo has the same depth at 10^3, 10^5 and 10^7 files, and its distinct structural subtree count is O(depth x fanout) (computed structurally). Edge-case catalog sanity. lincheck accepts known-linearizable histories (sequential; concurrent with overlap; with Unknown completions) and rejects known violations (stale read after a completed CAS, two successful CAS at the same version, a version gap) on hand-built cases. Bundles exist, their README and licenses are present, and the size limits hold (no git CLI required).", "size": "L" }, { "id": "P10", "title": "scion-cache: verified disk cache, coalescer, upstream limiter, ref/redaction/access caches", "wave": 1, "spec_sections": "10.2, 10.3, 10.4 (decision caches), 13.3 (daemon redaction set and staleness), 16.1 classes 6/7, 19.9", "owns": [ "crates/scion-cache/src/**", "crates/scion-cache/tests/**" ], "depends_on": [ "P00" ], "deliverables": "Implement PLAN 2.15. DiskCache layout: /objects///, holding the payload as received (a zstd frame or raw bytes, with a 1-byte header distinguishing them). Insert writes a temp file in the same directory, verifies BLAKE3 of the decompressed content against the id (INTEGRITY, nothing cached, on mismatch), fsyncs and renames. open() deletes leftover temp files and rebuilds the size and access index by scanning. get verifies on read when verify_on_read is set; a mismatch deletes the entry and returns Ok(None) so the caller refetches. Approximate LRU by last access time: after an insert that pushes usage over max_bytes, evict down to low_watermark x max_bytes. usage() is exact. Coalescer is single-flight per key: concurrent callers share one future, errors are shared, and the entry is removed after completion. UpstreamLimiter: semaphore(max_concurrency) plus a bounded waiter count; overflow gives RESOURCE_EXHAUSTED immediately. RefCache: TTL-bound, and never returns a lower version than previously returned for a ref, even after TTL expiry or out-of-order upstream answers. LocalRedactions: id set, epoch, last refresh and is_stale. AccessCache: entries keyed by (principal, object) with TTL, plus a per-principal root-reader flag with TTL. Decisions never leak across principals, and on_policy_epoch(epoch) clears everything when the epoch differs. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.9 unit level: a corrupt insert is rejected and not cached; a bit flip in a cached file is detected on read (verify_on_read), deleted and get returns None; temp files are removed on open; a simulated crash after the temp write and before rename leaves no visible entry; 10000 concurrent get_or_fetch calls for one key run the fetch exactly once; a randomized insert/get workload keeps usage <= max_bytes at all times and evicted ids read back as None; ref TTL is honored and versions stay monotonic across TTL expiry and reordered observations; queue overflow gives RESOURCE_EXHAUSTED. AccessCache isolates principals and clears on an epoch change.", "size": "M" }, { "id": "P11", "title": "scion-wc local layer: safe filesystem, stat-cache index, status engine, ignore, state, lock, credentials", "wave": 1, "spec_sections": "4.4 (path rules, case collisions), 6.4 (credentials), 11.1, 11.4, 11.5, 17 (checkout path safety, credentials 0600), 18.7, 19.10 (status, ignore, path safety), 19.11 (credentials)", "owns": [ "crates/scion-wc/src/{fs_safe.rs,index.rs,status.rs,ignore.rs,state.rs,lock.rs,credentials.rs}", "crates/scion-wc/tests/local_*.rs" ], "depends_on": [ "P00" ], "deliverables": "Implement the corresponding modules of PLAN 2.17. SafeRoot uses rustix openat with O_NOFOLLOW for every path component (a pre-existing symlinked parent is refused, never followed). Writes go to a temp file plus renameat in the target directory, with the exec bit set for Mode::Executable; symlink() creates symlinks; nothing is ever written outside the root. is_case_insensitive probes the filesystem once. validate_entry_path enforces 4.4 and rejects '.scion' in any case. case_collisions uses ASCII case folding, so it is deterministic on any filesystem; checkout uses it whenever the filesystem is case-insensitive or WcOptions.force_case_insensitive is set. StatCache: (size, mtime_ns, ctime_ns, inode, mode) tuples plus written_at_ns, stored as a binary file .scion/index (versioned header) and saved atomically. is_racy means mtime >= written_at. status follows 18.7 over the TreeLookup trait: walk in-scope files, skip .scion and ignored files, skip inaccessible prefixes; hash files (symlinks hash their target) only when the stat tuple changed or the entry is racy; report modified/added/deleted/mode_changed; refresh clean entries; an unmodified redacted placeholder (index redacted flag plus placeholder bytes) is clean, and an edited placeholder is reported in redacted_dirty and modified; count hashed files. IgnoreMatcher uses ignore::gitignore (nested .gitignore files, negation, directory patterns, anchoring) with git semantics. state: WcState and PendingOperation as canonical TOML (sorted keys, deterministic), saved atomically under .scion/state. lock: fs4 advisory non-blocking lock on .scion/lock. credentials: write creates the file with mode 0600 (and its parent with 0700); read refuses a group- or world-readable file with PERMISSION_DENIED and a clear message. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.10 status: randomized edit sequences (create, modify, delete, chmod +x/-x, symlink swap, rename, truncate-and-write, atomic-rename save, edits within the same second as the index write, i.e. racy-clean) match a full-rehash oracle after every step (proptest_cases(500); #[ignore] full_ at 10^4). Clean status hashes only racy or stat-changed files. .gitignore semantics match `git check-ignore` on a corpus of at least 40 patterns (skipped when git is absent), and a pure table of expected results runs always. Path safety: '..', '.scion', '.SCION' and NUL paths are refused; writes through a pre-existing symlinked parent directory are refused and nothing is created outside the root. case_collisions reports collisions (deterministic on any fs; plus a real collision test when the tmp fs is case-insensitive, as on macOS APFS). State and index round-trip and are atomic (a crash simulated mid-save leaves the old file). The lock excludes a second holder. 19.11: the credentials file is created 0600; read refuses 0640 and 0604.", "size": "L" }, { "id": "P12", "title": "scion-history: tree reading, apply_delta, FileDiff, CreateChangeset, bootstrap", "wave": 2, "spec_sections": "4.3.2, 4.3.4, 4.5, 5.3 step 2, 8.1, 9.2 (commit_graph columns, path_index rows), 13.6, 14 (parentless import_mode), 18.1, 18.2, 16.1 class 4, 19.2, 19.3, 19.18 (CreateChangeset gate)", "owns": [ "crates/scion-history/src/{history.rs,tree.rs,apply.rs,diff.rs,create.rs,bootstrap.rs}", "crates/scion-history/tests/create_*.rs" ], "depends_on": [ "P01", "P03", "P07" ], "deliverables": "Implement the P12 part of PLAN 2.6 (History::new and accessors already exist). tree/lookup/changeset go through ObjectRepo caches. apply_delta follows 18.2 exactly: walk only the ancestors of delta paths, deepest first, applying deletes, then rebuilt children, then puts. Subcodes: delete_absent, delete_not_file, file_directory (ancestor file not deleted, or a directory not fully deleted), noop_put. An empty non-root directory is pruned; the root may become empty. Return the new root plus the new non-empty trees with their canonical bytes. file_diff prunes identical subtree ids and supports a scope filter. create_changeset validates in the exact 8.1 order with the first failure returned: - parents exist, are unique and within max_parents (TOO_LARGE); empty parents only when has_root_changeset() is false, or when import_mode is set and authz.can_import() (else parentless_not_allowed or PERMISSION_DENIED); - delta sorted/unique/limit/paths; - Put blobs live with matching size (missing_blob, size_mismatch); - REDACTED for quarantined or purged Put ids, while entries carried over unchanged need no check; - delete/put versus T1 (delete_absent, delete_not_file, noop_put, file_directory via apply_delta); - authz.can_write on every delta path (PERMISSION_DENIED); - Meta fields within max_message_bytes (TOO_LARGE). Then apply_delta, the expected_root_tree check (root_tree_mismatch), and Meta with creator = who.id_string() (a client never supplies creator). Persist per 5.3, passing checkpoints create.after_validate/after_trees/after_meta/after_derive/after_commit: 1. store new trees and the meta via ObjectRepo.put_encoded; 2. call the EagerHook with EagerInput; 3. compute generation/skip with compute_commit_graph; 4. build object_paths rows for every Put blob at its path and every new tree at its directory path (path_index); 5. one commit_changeset bundle. Identical requests return the same id and never error, both sequentially and concurrently. create_internal (used by rebase and merge) skips client-facing authz and accepts an existing meta id. bootstrap is idempotent: ensure the root changeset (empty root tree; Meta author 'scion ', time 0/0, creator = config.bootstrap_principal, message 'root'; change_id = BLAKE3('scion-root-v1')[0..16]) and ref main at version 1 via cas_ref(Absent); an existing ref is fine. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.2: property test, for random trees T1 and random valid deltas: apply_delta's resulting tree, flattened, equals scion-refmodel apply_delta, and file_diff(T1, result) == delta (proptest_cases(2_000); #[ignore] full_ 10^5). Each subcode has a failing test: missing blob, size mismatch, noop put, delete absent, delete of a directory, file/dir without deletes (both directions), each path rule, each limit. Deleting the last file in a directory removes the directory; the root may become empty. File-to-directory and directory-to-file transitions with the required deletes succeed. A merge changeset's delta is validated against the first parent only. Identical CreateChangeset requests, sequential and 50-way concurrent, return one id and leave one row. expected_root_tree mismatch. creator is the server-set principal. 19.3: arm Crash at each create.* checkpoint and at meta.commit_changeset.before; afterwards fsck finds no violation and no visible changeset exists, and a retry returns the same id. 19.13: a Put of a redacted (quarantined) blob fails REDACTED; a changeset not touching a redacted file in the parent tree succeeds. 19.18 unit gate: on shared_subtree_repo at 10^3 and 10^5 (10^7 in #[ignore] full_), a 1-file CreateChangeset reads at most depth+2 trees and writes at most depth+1 trees (Counters deltas), independent of size. bootstrap is idempotent.", "size": "L" }, { "id": "P13", "title": "scion-history: commit-graph queries (IsAncestor, MergeBase, first-parent, log iteration)", "wave": 2, "spec_sections": "9.2 commit_graph, 18.5, 8.4 (stack detection support), 11.8 (~n, ^k), 7.4 (Log without path), 19.6 (commit_graph)", "owns": [ "crates/scion-history/src/graph.rs", "crates/scion-history/tests/graph_*.rs" ], "depends_on": [ "P01", "P03", "P07" ], "deliverables": "Implement the P13 part of PLAN 2.6 in graph.rs, as impl History blocks. Read ChangesetRows (generation, skip, parents decoded from the encoded changeset) through ObjectRepo/MetaStore; this package does not depend on create_changeset. is_ancestor follows 18.5 (generation-pruned DFS; a is an ancestor of itself). is_first_parent_ancestor uses skip pointers in O(log n). merge_bases returns ALL best common ancestors (common ancestors not dominated by another common ancestor), sorted by id, using generation-ordered search. nth_first_parent and nth_parent implement '~n' and '^k'. log_page walks newest-first by generation then id (deterministic), optionally first-parent only, with an opaque page token (an encoded frontier) giving stable paging. Tests build DAGs by inserting ChangesetRows directly into MemMetaStore, using P00's compute_commit_graph, and compare with scion-refmodel MRepo. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.6 commit_graph: generation and skip pointers equal the refmodel for random DAGs (including octopus merges, multiple roots and criss-cross). IsAncestor, is_first_parent_ancestor and MergeBase agree with the refmodel on random queries (10^4 by default; #[ignore] full_ 10^5). A criss-cross gives 2 bases. log_page paging is stable and complete (concatenated pages == the full walk). ~n and ^k edge cases (beyond root gives None).", "size": "M" }, { "id": "P14", "title": "scion-derived framework: registry, engine, lazy iterative derivation, leases, generic query path", "wave": 2, "spec_sections": "9.1, 5.2 (derived, derive_leases), 5.3 (eager rows), 13.5 (purge of content-bearing records), 19.6 (framework items)", "owns": [ "crates/scion-derived/src/{registry.rs,engine.rs,ctx.rs,lease.rs}", "crates/scion-derived/tests/framework_*.rs" ], "depends_on": [ "P01", "P03" ], "deliverables": "Implement the P14 part of PLAN 2.7. Registry: register and look up by name and version; v1_default registers the commit_graph and path_index descriptors (computed by History), history and git_mapping as eager types (constructing scion_derived::history::HistoryType and git_mapping::GitMappingType, whose behaviour is implemented by P21), and blame as lazy (blame::BlameType, P24). latest_version; content_bearing_types. StoreDeriveCtx implements DeriveCtx over ObjectRepo: blob_or_empty returns empty for quarantined or purged blobs; meta returns None when redacted; put_object uses ObjectRepo.put_encoded. DerivedEngine: - impl EagerHook runs every registered eager type and concatenates their rows. required_eager lists eager (name, version) pairs for fsck. - get(ty, version, key) returns the stored value or derives lazily. The lazy algorithm uses an explicit work stack (NEVER recursion): find the nearest keys whose values are stored by walking deps(), then derive forward in dependency order, writing back every intermediate value with put_derived (ON CONFLICT DO NOTHING). - Leases (try_lease/release_lease) only deduplicate concurrent work. A derivation proceeds even if the lease is not acquired, after a short wait-and-recheck. - Version selection: None means the latest registered version; old versions remain readable. - get_stored. purge_content_for deletes records of content-bearing types whose source_object is the id. - Reports derive duration and backlog through the DeriveMetrics hook set via set_metrics (no scion-obs dependency). Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.6: two workers deriving the same key concurrently produce byte-identical values and exactly one row (test-only lazy type). A lazy derivation over a chain of 10^5 dependencies (default 10^4; #[ignore] full_ 10^5) completes on a thread with a 256 KiB stack (no recursion growth) and writes back every intermediate value. A version bump re-derives on query while the old version's values stay readable. A test-only type registered at runtime is queryable through DerivedEngine::get (the generic path) with no other change. Lease loss does not affect correctness. purge_content_for deletes only content-bearing records keyed by the id. The EagerHook output contains the rows of all registered test eager types.", "size": "M" }, { "id": "P15", "title": "Gated real backends: scion-store-crdb (CockroachDB) and scion-store-s3 (S3/MinIO)", "wave": 2, "spec_sections": "3.5, 5.1, 5.2, 5.3, 5.4, 16.1 classes 1/2, 19.3 (on real backends)", "owns": [ "crates/scion-store-crdb/src/**", "crates/scion-store-crdb/tests/**", "crates/scion-store-s3/src/**", "crates/scion-store-s3/tests/**" ], "depends_on": [ "P01", "P03" ], "deliverables": "Implement PLAN 2.4 following docs/contracts/storage-semantics.md exactly. CRDB: embedded MIGRATIONS create every SPEC 5.2 table keyed by repo_id first, plus audit_log and the secondary indexes listed in the contract; migrate is idempotent. Each MetaStore method runs in one SERIALIZABLE transaction via deadpool-postgres and tokio-postgres, with a client-side retry loop on SQLSTATE 40001 (bounded, with jitter; when exhausted, return UNAVAILABLE/txn_retry). Inserts use ON CONFLICT DO NOTHING. cas_ref runs UPDATE ... WHERE name=$1 AND version=$2 (or INSERT for Absent and DELETE for delete), inserts the ref_log, inserts landed rows (a unique violation 23505 rolls back and returns LandedConflict with the existing row) and inserts the audit row, all in the same transaction. Consistency::Stale uses AS OF SYSTEM TIME follower_read_timestamp(). changesets_by_prefix uses a range scan on the id column. Connection errors map to UNAVAILABLE/store_unavailable. connect_fresh_for_tests creates a unique database. S3: object_store AmazonS3 with endpoint, path-style and allow_http for MinIO; get on a missing key returns None; copy and delete are idempotent; list uses prefix listing. Keys are prefixed with key_prefix (unique per test run). Every test is gated: when SCION_TEST_CRDB_URL or SCION_TEST_S3_ENDPOINT is unset it prints 'SKIP: ...' and passes. Test names start with external_. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "When the env vars are set: scion_store::conformance::meta_store_suite against CrdbMetaStore and blob_store_suite against S3BlobStore. Also: migration idempotency; 40001 retry exercised by two conflicting concurrent transactions; Stale reads may miss a just-committed row while Strong reads see it; 1000-way concurrent cas_ref yields exactly one winner. 19.3 on real backends: ObjectRepo over CRDB plus S3 with InstrumentedBlob faults (put fails) leaves no objects row; FailAfter on meta.insert_object leaves a durable row and bytes. When unset, every test prints SKIP and passes.", "size": "L" }, { "id": "P16", "title": "scion-daemon: Unix-socket caching tier, local authorization, redaction tracking, verbatim forwarding", "wave": 2, "spec_sections": "3.3, 6.3, 7.3 (daemon semantics), 7.8, 10.1-10.5, 13.3 (daemon), 16.1 classes 1/6/7, 17 (socket permissions), 18.8 (daemon loop), 19.8, 19.9, 19.11 (daemon), 19.12 (daemon), 19.13 (daemon)", "owns": [ "crates/scion-daemon/src/**", "crates/scion-daemon/tests/**" ], "depends_on": [ "P01", "P04", "P06", "P08", "P10" ], "deliverables": "Implement PLAN 2.16. Daemon::start binds a Unix socket (mode 0666; authorization rests only on tokens, never on socket identity) and serves a tonic Router. (a) ObjectService is implemented locally: - GetObjects: verify the bearer token locally with keys from upstream GetServerInfo (refreshed on unknown kid). Check LocalRedactions before serving any cached object (status redacted). Authorization per 10.4: root-class objects are served to principals confirmed as root readers via a positive CheckAccess on a root-class object, cached for access_cache_ttl_ms; restricted objects need a positive per-(principal, object) CheckAccess decision cached up to access_cache_ttl_ms; one principal's decision is never used for another. On a miss: Coalescer, then UpstreamLimiter, then upstream GetObjects (at most 3 attempts with jittered exponential backoff); verify the digest before caching and before replying. Items denied, not_found or redacted upstream are passed through unchanged. Payloads are byte-identical after decompression, and per-item statuses equal the server's. Range reads serve raw bytes. - HasObjects is forwarded, because it depends on upload records. - ResolveRef goes through the RefCache (TTL plus monotonic versions). - GetServerInfo is cached and refreshed periodically. (b) Every other gRPC path is forwarded verbatim through a RawCodec passthrough selected by path prefix. Request metadata (including authorization) and response trailers are passed through unmodified, so forwarded responses keep the server's scion-tier=server (PLAN 6.13). Epochs are observed from those trailers. (c) Locally served responses carry trailers from EpochTrailerLayer with tier=daemon and the daemon's latest known epochs. (d) A policy epoch observed in any upstream trailer that differs from the last one clears the AccessCache. (e) The redaction loop polls ListRedactions(since=local epoch) every redaction_poll_interval_ms and immediately on a higher trailer epoch. It applies the entries, deletes cached copies and marks refreshed. If is_stale(redaction_max_staleness_ms), it enters fail-closed mode: no cache serving, every read forwarded upstream; normal mode resumes after a successful refresh. (f) Upstream outage: cached objects keep serving; misses fail UNAVAILABLE; recovery needs no restart. (g) Metrics: cache_requests, cache_bytes, upstream_requests, redaction_epoch, redaction_lag, integrity_failures. RPC log events use tier daemon. (h) DirectUpstream records destinations for the network-policy test. (i) testing::InProcessDaemon. main.rs: load_daemon config, then start. Tests use scion_testkit::FakeUpstream (with principal_of built from a test Keyring Verifier) as upstream; real-server tier tests are P30's. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.9 daemon level: 10000 concurrent GetObjects for one missing id give exactly 1 upstream request; a corrupted upstream frame is rejected (INTEGRITY to the client, nothing cached); a bit flip in the cache is detected, deleted and refetched; temp files are cleaned on restart; a kill during insert leaves no partial entry and the cache stays valid (16.1 class 6); eviction keeps usage <= max and evicted objects are refetched transparently; ref TTL is honored and versions are monotonic across expiry and reordered upstream answers; upstream down gives cached hits served, misses UNAVAILABLE, and recovery without restart; queue overflow gives RESOURCE_EXHAUSTED. 19.8 basics: a batch mixing ok/not_found/denied/redacted is identical through the daemon and direct to FakeUpstream; QueryService/AuthService calls forwarded via RawCodec are byte-identical; only the configured upstream is contacted (DirectUpstream destinations). 19.11: the daemon rejects expired, malformed, wrong-key and tampered tokens. 19.12: a restricted object cached for principal A is not served to principal B without B's own positive decision; a policy epoch change in a trailer drops decisions. 19.13: a redaction is learned within one poll interval (ManualClock or short intervals) plus one request, and the cached copy is deleted; with polling blocked beyond max staleness the daemon stops serving from cache and forwards reads (fail closed). Trailers on daemon-served responses show tier=daemon.", "size": "L" }, { "id": "P17", "title": "Oracle cross-validation: refmodel vs production encoding, Myers/diff3 vectors, FileDiff; testkit encoders", "wave": 2, "spec_sections": "19.0 (Reference Model independence), 19.1 (golden vectors, id agreement), 19.4 (merge properties against the reference model), 4.3.4 (FileDiff)", "owns": [ "crates/scion-conformance/tests/oracle_crosscheck.rs", "crates/scion-conformance/tests/oracle_support/mod.rs" ], "depends_on": [ "P01", "P02", "P07", "P09" ], "deliverables": "Write a cross-validation suite in scion-conformance that detects divergence between the independent oracle and production code early, before engines depend on it. (1) Every testdata/golden/objects vector: the refmodel enc produces identical bytes and id for trees, metas and changesets (build the refmodel value from the vector description). (2) Random valid trees (from MTree), metas and changesets: the scion-object id equals the refmodel id. (3) Every testdata/golden/diff vector is reproduced exactly by scion-refmodel::diff (myers edits and merge3 results/conflict ranges). (4) Differential: scion_merge::merge3 vs scion_refmodel::diff::merge3 on generated text triples (line-level random edits from a common base, including CRLF, unterminated last lines and identical edits); clean/conflict outcome and clean bytes must match. (5) scion_object::file_diff vs scion_refmodel::file_diff on random flat trees. (6) scion_testkit::encode_tree root ids equal the refmodel tree ids (first exercise of the testkit encoders). When a disagreement is found, do NOT edit P01/P02/P07 files. Minimize the failing case, record it, and report which side violates docs/contracts/line-diff3-v1.md or SPEC 4.1; the orchestrator routes the fix as an amendment. The package is done only when all checks pass, so fixes may be needed upstream; report them as amendment requests with the minimized case. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.1: golden vectors agree across both encoders; production id == refmodel id for proptest_cases(10_000) random objects (#[ignore] full_ 10^6). 19.4: merge3 differential at proptest_cases(10_000) (#[ignore] full_ 10^6); every golden diff vector is reproduced by the refmodel. FileDiff differential. Testkit encode_tree agrees with the refmodel.", "size": "M" }, { "id": "P18", "title": "scion-wc remote layer: fetch walk with resume, checkout, goto, sparse, placeholders, redaction refresh, recovery dispatch, status integration", "wave": 2, "spec_sections": "10.1, 11.2, 11.3, 11.5 (integration, zero server calls), 12.6 (inaccessible subtrees), 13.7 (placeholders, replacement of newly redacted clean files), 4.4 (case collisions), 17 (checkout path safety), 19.10, 19.13 (working copy), 19.18 (sparse checkout and status gates)", "owns": [ "crates/scion-wc/src/{wc.rs,remote.rs,fetch.rs,tree_cache.rs,checkout.rs,goto.rs,sparse.rs,placeholder.rs,redaction_refresh.rs,recover.rs,diff.rs,testing.rs}", "crates/scion-wc/tests/checkout_*.rs", "crates/scion-wc/tests/goto_*.rs", "crates/scion-wc/tests/sparse_*.rs", "crates/scion-wc/tests/fetch_*.rs" ], "depends_on": [ "P01", "P07", "P08", "P09", "P10", "P11" ], "deliverables": "Implement the P18 part of PLAN 2.17. remote: helpers over ScionClient (ResolveRef/Resolve, GetChangesets for root_tree, batched GetObjects with per-item status handling). fetch: breadth-first by depth, requesting only missing trees within the sparse scope in batches; skip out-of-scope subtrees; record denied subtrees as inaccessible; then stream missing in-scope blobs in batches. Every item is verified (BLAKE3 after decompression) before it is written to the local tree cache (.scion/cache via scion-cache DiskCache) or to the working copy. A broken stream is resumed by re-requesting incomplete objects with offset. checkout: validate every entry path (4.4, '.scion') and case collisions BEFORE any write. Write files atomically through SafeRoot (exec bits, symlinks); write redacted files as placeholder_bytes and mark the index entry redacted; initialize the index and WcState (base, tracking ref, sparse, inaccessible). Directories on the way to a scoped prefix are created, but their direct files are only materialized when in scope (11.2). goto: pruned diff of old and new trees within scope, applying only the differences. Dirty files the diff would touch abort unless force; other dirty files are preserved. sparse_add/sparse_remove: materialize or remove exactly the affected files; removing scope that contains dirty files fails unless force. Every mutating command first calls recover(), then refresh_redactions() (replace clean materialized copies of newly redacted blobs with placeholders, detected via GetObjects status redacted on the base tree's in-scope blobs; dirty files are reported and untouched), and journals a PendingOperation before touching files (fault points wc.goto.*, wc.sparse.*, wc.checkout.*). recover() completes or rolls back Checkout/Goto/SparseChange itself and dispatches Commit/Rebase to crate::commit::recover_commit and crate::rebase::recover_rebase (P28). status integration: a TreeLookup backed by the local tree cache, so status needs zero RPCs when trees are cached. diff(): working-copy changes. testing.rs (feature testing): reusable scenario functions taking a ScionClient, so P30 and P33 can re-run these suites against the real server and daemon. Tests use scion_testkit::FakeUpstream seeded via scion_testkit::encode_tree from generated corpora, including a malicious-tree mode. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.10 against FakeUpstream: a sparse checkout of directory D requests exactly the trees on the path to D plus the objects under D (asserted from FakeUpstream.requests()). Resumption: kill streams at random offsets; the resumed checkout re-fetches only incomplete objects and the working copy is byte-identical. The checkout result equals scion-refmodel materialize for random trees and sparse profiles (contents, exec bits, symlinks). Malicious trees ('..', '.scion', '.SCION', symlink traversal) are refused before any write, and pre-existing symlinked parents are not followed. Case collisions are reported, not overwritten (force_case_insensitive plus a real APFS test when available). goto touches only differing paths (mtime unchanged elsewhere), preserves unrelated dirty files and aborts on dirty files it would overwrite (force overrides). status makes zero server calls with a warm cache (ScionClient.calls()). sparse add/remove changes exactly the affected files and refuses dirty removal. Inaccessible (denied) subtrees are recorded and excluded from status. Crash injection at every wc.goto.*, wc.sparse.* and wc.checkout.* point leaves a pending_operation that the next command completes or rolls back. 19.13 working copy: checkout writes placeholders, status treats them as clean, newly redacted clean files are replaced on the next mutating command, dirty files are untouched and reported. 19.18: sparse checkout of a fixed directory fetches the same object count on shared_subtree_repo at 10^3 and 10^5 (10^7 #[ignore] full_); clean status reads zero objects upstream and hashes only stat-changed or racy files.", "size": "L" }, { "id": "P19", "title": "scion-history: deterministic rebase and Merge(a,b) with the 8.3 tree-level rules", "wave": 3, "spec_sections": "8.2, 8.3 (tree-level table, structural conflicts, binary/size/redacted rules), 13.6, 18.3, Appendix A (criss-cross), 19.4, 19.13 (merge with redacted content), 19.18 (Rebase gate)", "owns": [ "crates/scion-history/src/{rebase.rs,merge.rs,conflict.rs}", "crates/scion-history/tests/merge_*.rs", "crates/scion-history/tests/rebase_*.rs" ], "depends_on": [ "P02", "P07", "P12", "P13" ], "deliverables": "Implement History::rebase, History::merge and impl Rebaser for History (PLAN 2.6), following 18.3. rebase(C, X): - requires len(C.parents) == 1 (INVALID_ARGUMENT/multi_parent). - Fast path: X == P returns C itself. - B = tree(P), O = tree(X), T = tree(C). Examine exactly the paths in C.delta plus their ancestor directories in O (O(|delta| x depth)). - structural_conflict: T puts a file at p where O has a directory at p or a file at an ancestor of p, or T deletes p where O has a directory at p, gives file_directory. - Otherwise use scion_merge::classify. NeedContent: fetch b/o/t content via ObjectRepo. A redacted or purged side gives conflict redacted; is_binary on any side, or a size above limits.merge_max_text_bytes, gives binary; otherwise merge3, then Clean stores the merged blob via put_blob with mode from merge_mode, and Conflict becomes a content conflict with hunks. - Collect every conflict. When any exists, return MERGE_CONFLICT with ConflictReport{clean_entries for all non-conflicting delta paths, conflicts with base/ours/theirs EntryInfo and hunks}. - Otherwise delta = resolved entries differing from O; an empty delta gives EMPTY_CHANGE. Return create_internal(parents=[X], delta, MetaSource::Existing(C.meta), C.change_id). - No wall-clock value enters the result. Conflict markers are NEVER written to any stored object. merge(a, b): - merge_bases must be exactly one (else AMBIGUOUS_MERGE_BASE); zero bases is treated as the empty tree. - Compute FileDiff(base, a) and FileDiff(base, b) with identical-subtree pruning, and apply the same per-path rules with ours = a, theirs = b. - Write authorization: authz.can_write on every resulting delta path versus a. - Result: a new changeset with parents [a, b], new Meta from MetaFields, and the given change_id, via create_changeset's persistence path. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.4: a dedicated test for every 8.3 table row (o==b, t==b, o==t, add_add, modify_delete, symlink, binary, redacted, content clean, content conflict, mode merge) and both structural rules. merge_max_text_bytes boundary (equal passes, +1 gives binary). Rebase determinism: the same (C, X) gives the same id across runs, across two History instances over the same storage, and after recreating Storage handles. Fast path X == P returns C. EMPTY_CHANGE when the change is already in X. Zero blob reads when X changed only unrelated paths, and O(|delta| x depth) tree reads (Counters). Conflict payloads include clean_entries and per-conflict base/ours/theirs ids and hunks. A scan of every object created during the suite finds no '<<<<<<<' marker produced by the engine. Merge(a,b) with a unique base matches scion-refmodel merge_trees on generated corpora; criss-cross histories give AMBIGUOUS_MERGE_BASE. 19.13: a merge or rebase needing redacted content gives a redacted conflict. 19.18 unit: the Rebase gate on shared_subtree_repo at 10^3 and 10^5 (10^7 #[ignore] full_).", "size": "L" }, { "id": "P20", "title": "scion-history: refs, UpdateRef, ref log, lock-free Land (with a fake-Rebaser seam)", "wave": 3, "spec_sections": "4.3.6, 4.3.7, 5.3 step 3, 8.4, 8.5, 18.4, 18.5 (stack detection), 16.1 class 5, 19.5 (library level), 19.18 (Land txn gate)", "owns": [ "crates/scion-history/src/{refs.rs,land.rs}", "crates/scion-history/tests/refs_*.rs", "crates/scion-history/tests/land_*.rs" ], "depends_on": [ "P07", "P12", "P13" ], "deliverables": "Implement the P20 part of PLAN 2.6. resolve_ref uses a strong read; at_version is served from the ref_log entry with that version (NOT_FOUND when absent). update_ref: the authz.can_update check; a version-checked cas_ref with a ref_log row in the same transaction; REF_VERSION_MISMATCH with a RefMismatch payload (current version and target). A non-fast-forward update (new target not a descendant of the old one, via is_ancestor) requires allow_non_fast_forward AND can_force, else NOT_FAST_FORWARD; forced updates write an audit row. create_ref/delete_ref use cas_ref(Absent) and Version(v) with an audit row. The caller (server) checks admin.refs. land/land_with implement the 8.4 state machine. Per attempt: 1. Resolving: a strong read of (tip, v), then authz = source.current() (policy re-read on EVERY attempt) and the can_update check. 2. Stack: the first-parent chain from head back to, and excluding, the first changeset that is an ancestor of tip. Use is_first_parent_ancestor when the tip's first-parent history is linear, otherwise fall back to is_ancestor. Every member must have exactly one parent (INVALID_ARGUMENT/merge_in_stack). 3. Checking: get_landed per member. Same source: skip and record. Different source: ALREADY_LANDED with payload. All skipped: return the recorded result (idempotent). 4. Rebasing: can_write on every delta path of each remaining member, then rebaser.rebase chained onto the previous result (checkpoint land.after_rebase). MERGE_CONFLICT fails the land; EMPTY_CHANGE fails unless allow_empty, in which case the member is skipped. 5. Committing: one cas_ref(Version(v)) with ref_log and landed rows (checkpoint land.after_commit). Every attempt reports to the configured LandObserver (Committed, Idempotent, CasMismatch, LandedConflict, Failed(code)) with its elapsed time. VersionMismatch or LandedConflict leads to full-jitter backoff between backoff_base and backoff_max (seedable rng; tokio::time::sleep) and a new attempt. After max_attempts: UNAVAILABLE/land_contention (retryable) with the ref unchanged. No lock or lease is ever taken (nothing emits a scion::lock span); a constant number of metadata transactions per attempt. land uses self as the Rebaser (impl from P19). Tests use land_with plus a fake Rebaser that re-applies C.delta onto `onto` via create_internal (conflict when apply_delta fails), so this package runs in parallel with P19. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.5 library level (fake Rebaser): ABA safety (A->B->A, then a CAS expecting the first version fails); fast-forward enforcement, with non-fast-forward requiring the flag and force; ref_log written atomically with gap-free versions; idempotency (arm FailAfter at meta.cas_ref.after, so land returns an error, then a retry returns the recorded result and no second ref version exists); an amended change id after landing gives ALREADY_LANDED; a partially landed stack skips lower members and lands upper ones; a stack containing a merge changeset is rejected; authorization is re-checked every attempt (a LandAuthorizerSource that revokes write after attempt 1, combined with a forced CAS mismatch, makes the land fail PERMISSION_DENIED); max_attempts exhaustion returns retryable UNAVAILABLE and leaves the ref unchanged; 50 concurrent non-overlapping lands all succeed with each change id landed once. No-locks: the Counters lease_calls delta is 0 and no scion::lock span is recorded on the land path. 19.18: a 1-file Land performs a constant number of meta_txns per attempt (equal at repository sizes 10^3 and 10^5).", "size": "L" }, { "id": "P21", "title": "scion-derived types: history nodes (kind 0x10), path_index/commit_graph descriptors, git_mapping", "wave": 3, "spec_sections": "9.2 (history, path_index, git_mapping), 4.2 (derived kind 0x10), 14 (git_mapping via import_mode and Meta extension tag 2), 19.6 (history items), 19.18 (history node gate)", "owns": [ "crates/scion-derived/src/{history_type.rs,path_index.rs,git_mapping.rs}", "crates/scion-derived/tests/history_*.rs", "crates/scion-derived/tests/git_mapping_*.rs" ], "depends_on": [ "P07", "P12", "P14" ], "deliverables": "Implement the P21 part of PLAN 2.7. HistoryType (eager, not content-bearing): for each changeset, create history nodes only for paths whose content changed versus parents[0], meaning the delta paths and their ancestor directories (O(|delta| x depth) new nodes). Node reuse: when a path's content (blob id, or tree id for directories) equals its content in parents[0], reuse that parent's node. Otherwise create HistoryNode{kind, content, changeset, parents = nodes for that path in each parent where it exists (deduplicated, parent order), children (Dir: (name, node id) for every entry, reusing child nodes)}. Store each node via cx.put_object(Derived(0x10)). The per-changeset derived row is key = changeset id bytes, value = root node id. For the root changeset with an empty tree, the root node is a Dir with no children. node_for_path walks children from the root node; get_node decodes. path_index and commit_graph are descriptors only (rows and columns are produced by History::create_changeset). GitMappingType (eager, not content-bearing): only when EagerInput.import_mode is set and the Meta carries extension tag 2 (a 20-byte git commit id). It emits two rows: key 'g:'+40hex gives the changeset id, and key 'c:'+changeset id bytes gives the 40hex. Lookups changeset_for_git and git_for_changeset read rows via DerivedEngine::get_stored. Test the derive functions both through a mock DeriveCtx (an in-memory map) and end to end through History::create_changeset with DerivedEngine (Registry::v1_default) as the EagerHook. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.6 history: for every path in generated corpora (scion-testkit generate), walking node parents reproduces scion-refmodel MRepo::log_path's TREESAME history. The node reuse rule is verified on merges (a merge keeping the first parent's version reuses its node; a merge taking the second parent's version creates a node with both parents). Eager values exist for every visible changeset when Crash is armed at create.after_derive and at meta.commit_changeset.before (fsck with required_eager reports no violation). 19.18: a 1-file change creates at most depth+1 new history nodes (counted via objects_written of the derived kind) at 10^3 and 10^5. git_mapping: rows exist in both directions only for import_mode creations; a non-import changeset with tag 2 creates no mapping; a re-commit in import_mode fills in missing mapping rows.", "size": "M" }, { "id": "P22", "title": "scion-redaction: Redact/Unredact/Purge lifecycle, quarantine moves, scheduled purge, backup-retention documentation", "wave": 3, "spec_sections": "4.3.9, 13.1, 13.2, 13.5, 13.8, 17 (audit), 18.8 (server side)", "owns": [ "crates/scion-redaction/src/**", "crates/scion-redaction/tests/**", "docs/operations/backup-retention.md" ], "depends_on": [ "P03", "P14" ], "deliverables": "Implement PLAN 2.8 (authorization is checked by the server, not here). redact: every id must exist with kind Blob or Meta (else INVALID_ARGUMENT/not_redactable; unknown ids give NOT_FOUND). One MetaStore.redact transaction (epoch+1, rows stamped, objects quarantined, plus append_audit with actor and reason). Then move bytes to the quarantine key asynchronously (move_pending_bytes is idempotent and resumable after a crash at quarantine.after_copy; inline data stays in its row but is never served). Return {epoch, effective_by = now + propagation_bound}. unredact: only from Quarantined (else INVALID_ARGUMENT/not_quarantined); restore the bytes, then MetaStore.unredact (epoch+1) with audit. purge_now and run_due_purges (due_purges(now)): delete the live and quarantine keys and inline data, delete content-bearing derived records via DerivedEngine::purge_content_for, then mark_purged with audit. Purge is irreversible (checkpoint purge.after_bytes; a re-run completes it). spawn_background runs the mover and the purge scheduler every purge_interval. Write docs/operations/backup-retention.md: the backup retention policy (CockroachDB backups and object-store versioning/lifecycle at most 30 days; purged content persists in backups until they expire), the redaction runbook (redact, verify propagation, purge, verify backups expire) and verification steps (SPEC 13.8, 20.3). Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.13 lifecycle (library level over Storage::in_memory with ManualClock): redacting a blob changes no tree, changeset or ref id (ids captured before and after); the state goes Quarantined, then Purged after purge_after elapses and run_due_purges runs; unredact from quarantine restores reads byte-identically and bumps the epoch; unredact after purge fails not_quarantined; purge deletes the live and quarantine keys and inline data, deletes content-bearing derived records of a test content-bearing type keyed by the id, and keeps non-content-bearing records; a Meta object can be redacted; a Tree cannot (not_redactable); every action writes an audit row with actor and reason; a crash after the redact transaction but before the byte move is completed by move_pending_bytes; effective_by = commit time + propagation bound.", "size": "M" }, { "id": "P23", "title": "scion-server core: app wiring, TLS, auth interceptor, trailers, error mapping, rate limit and load shedding, RPC logs/metrics, policy reload, RedactionSet, Login/WhoAmI/GetServerInfo, binary, InProcessServer", "wave": 3, "spec_sections": "3.1, 6.1, 6.2 (reload), 7.1, 7.2, 7.3 (GetServerInfo), 7.7 (Login, WhoAmI), 10.5 (rate limits, load shedding), 12.1-12.3, 13.3 (server propagation), 15.1-15.3, 16.1 classes 9/10, 17 (TLS), 19.7, 19.11 (server), 19.12 (reload), 19.16 (GitHub down)", "owns": [ "crates/scion-server/src/{main.rs,app.rs,context.rs,interceptor.rs,errors.rs,ratelimit.rs,logging.rs,reload.rs,tls.rs,redaction_set.rs,testing.rs}", "crates/scion-server/src/handlers/{server_info.rs,auth.rs}", "crates/scion-server/tests/core_*.rs" ], "depends_on": [ "P03", "P04", "P05", "P06", "P08", "P12", "P14", "P15" ], "deliverables": "Implement the P23 part of PLAN 2.14. ServerApp::build wires the components in PLAN 1.4: ObjectRepo from config.objects, History (EagerHook = DerivedEngine with registry = deps.registry or Registry::v1_default; with_land_observer(MetricsLandObserver), defined in app.rs, which maps outcomes to scion_land_attempts{result}, scion_ref_cas_failures_total{ref} and scion_land_duration_seconds), Redactor, RedactionSet with spawn_refresher(propagation_bound) and observe() on epoch increments, ObjectAuthorizer, Verifier from the Keyring, ServicePrincipals and the policy watcher (when policy.reload is set). It also wires DerivedEngine::set_metrics with a Prometheus DeriveMetrics adapter (scion_derive_duration_seconds, scion_derive_backlog), and RedactionSet refreshes update scion_redaction_epoch{tier=server} and scion_redaction_lag_seconds. It calls History::bootstrap(), then handlers::admin::spawn_background(&ctx). ServerDeps.registry and ServerTestOpts.registry (frozen in PLAN 2.14) let core tests pass a registry without the history and git_mapping eager types, because those are implemented in parallel by P21. Layers, in order: request id, then a bearer interceptor verifying tokens with the clock (all RPCs except GetServerInfo and Login; inserts CallerCtx into request extensions; failure gives UNAUTHENTICATED), then load shedding (tower ConcurrencyLimit at rate_limit.max_inflight plus load_shed, giving RESOURCE_EXHAUSTED), then the governor per-principal limiter (RESOURCE_EXHAUSTED), then the RPC log and metrics layer (RpcLogEvent with tier server; scion_rpc_duration_seconds{rpc,tier,code}), then EpochTrailerLayer with AppContext as EpochSource. errors.rs maps ScionError to Status via scion-proto. Handlers: get_server_info (protocol_versions ['scion.v1'], repo, epochs, token public keys, limits, github_client_id, github_oauth_base_url); login (GitHub: GitHubClient.user, then Principal::GitHub{login, id} with gid in the token, invalid giving UNAUTHENTICATED and GitHub down giving UNAVAILABLE; service: ServicePrincipals.verify; returns token, principal, expires_at); who_am_i (principal, groups from policy, expires_at). reload.rs: on reload update the policy_epoch_info metric; an invalid file logs an ERROR and keeps the old policy. Readiness: metadata ping, policy loaded. The admin listener serves health_router. tls.rs: rustls server config from PEM. main.rs: load_server(config), build CrdbMetaStore and S3BlobStore, serve. testing::InProcessServer (feature testing): Storage::in_memory unless provided; temp dir holding acl.toml, Keyring PEM files, and TLS from TestTls when tls is set; ServerApp on 127.0.0.1:0; token_for issues tokens directly from the Keyring; set_policy writes the file and calls reload_now; kill() aborts the serving tasks; restart() builds a new ServerApp over the same Storage on the same address. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.11 server side: GitHub login against FakeGitHub (valid token gives principal github: and the numeric id in the token's gid; invalid gives UNAUTHENTICATED); service login with right and wrong secret; the server rejects expired, malformed, wrong-key and tampered tokens on a protected RPC (WhoAmI). Key rotation through config (an old key still verifies while listed). 19.7: GetServerInfo and Login work without a token; every protected RPC without a token gives UNAUTHENTICATED (iterate the FILE_DESCRIPTOR_SET method list; stub handlers still sit behind the interceptor); all three trailers are present on success, error and streaming responses (WhoAmI, an UNAUTHENTICATED failure, GetObjects stub stream). 19.12: set_policy with a valid change changes the scion-policy-epoch trailer; an invalid file keeps the old epoch and the server keeps serving. 19.16 class 10: with FakeGitHub down, existing tokens still work and new GitHub logins fail UNAVAILABLE. Per-principal rate limit and load shedding return RESOURCE_EXHAUSTED (tiny limits). /readyz turns 503 when the MemMetaStore ping fails. Restart keeps data. RpcLogEvent fields are present for every RPC (LogCapture).", "size": "L" }, { "id": "P24", "title": "scion-derived blame (lazy), Log(path), all-versions; Log and Blame RPC handlers", "wave": 4, "spec_sections": "9.2 (blame, history TREESAME), 7.4 (Log, Blame), 12.6 (Log/Blame filtering), 13.2 (--all-versions), 13.4 (redacted Meta in Log), 13.5, 19.6 (blame, Log(path), version bump), 19.12 (Log/Blame filtering), 19.13 (all-versions, Meta rendering)", "owns": [ "crates/scion-derived/src/{blame.rs,log.rs,versions.rs}", "crates/scion-derived/tests/blame_*.rs", "crates/scion-derived/tests/log_*.rs", "crates/scion-server/src/handlers/history_query.rs", "crates/scion-server/tests/history_query_*.rs" ], "depends_on": [ "P02", "P07", "P13", "P21", "P23" ], "deliverables": "Implement the P24 part of PLAN 2.7 and the Log/Blame handlers of PLAN 2.14. BlameType (lazy, not content-bearing, version configurable through with_version). The key is a file history node id. deps() are the blame keys of the node's parent nodes. derive follows 9.2: diff the node content against parents[0]'s content with scion_merge::line_origins and attribute matching lines from that parent's blame, then do the same with subsequent parents for the remaining lines; remaining lines belong to node.changeset. Redacted content is treated as empty (cx.blob_or_empty). Values are encoded with encode_ranges as a compact canonical list of ranges, adjacent equal-origin lines merged. blame(e, cs, path, version) resolves node_for_path (a missing or non-file path gives NOT_FOUND) and then calls e.get('blame', version, node id). log_path walks history node parents (first_parent_only follows parents[0]), returning node.changeset in deterministic generation-descending order, with a page token. versions::all_versions walks every reachable node for the path and returns distinct content blob ids. handlers/history_query.rs: - log: no path means History::log_page; with a path, log_path. - Read checks: Log with a path requires read on the path (PERMISSION_DENIED); Log without a path returns each changeset's GetChangesets-style item, where a delta path the caller cannot read produces a filtered view with those entries removed. - MetaView is redacted=true with only meta_id when the Meta is quarantined or purged (clients render ''). - blame requires read on the path and returns ranges. - Errors map via scion-proto; the CallerCtx comes from context::caller. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.6: blame equals scion-refmodel MRepo::blame for every file at several changesets in a generated corpus (with merges). Log(path) via log_path equals the refmodel's TREESAME history for every path in the corpus. Lazy blame over a 10^4-commit single-file history completes iteratively (#[ignore] full_ 10^5). Version bump: registering BlameType::with_version(2) makes Blame re-derive on query through the unchanged RPC while version-1 values stay readable via DerivedEngine::get. Blame of a file with redacted history content is deterministic (redacted treated as empty). 19.13: all_versions finds every distinct blob a path ever held (compared with refmodel all_versions); Log renders redacted Meta as redacted=true with ids unchanged. RPC tests over InProcessServer: Log and Blame success; NOT_FOUND; PERMISSION_DENIED for a restricted path; Log without a path omits unreadable delta entries (19.12); unauthenticated calls fail; trailers are present.", "size": "M" }, { "id": "P25", "title": "scion-server object, upload, access and admin services: retroactive object authorization, redaction RPCs, audit, background jobs", "wave": 4, "spec_sections": "7.3 (GetObjects, HasObjects, ResolveRef), 7.5 (UploadObjects), 7.7 (CheckAccess, Redact, Unredact, PurgeNow, ListRedactions), 10.4 (CheckAccess semantics for daemons), 12.5, 13.3, 13.4, 13.6, 16.1 class 7, 17 (audit), 18.6, 18.8, 19.7, 19.12, 19.13, 19.18 (object authorization gate)", "owns": [ "crates/scion-server/src/handlers/{object.rs,upload.rs,access.rs,admin.rs}", "crates/scion-server/src/authz/object.rs", "crates/scion-server/src/jobs.rs", "crates/scion-server/tests/object_*.rs", "crates/scion-server/tests/admin_*.rs" ], "depends_on": [ "P12", "P21", "P22", "P23" ], "deliverables": "Implement the P25 handlers of PLAN 2.14 and ObjectAuthorizer::can_read per 18.6. can_read: first RedactionSet.contains gives Redacted. Then check the (principal, id, policy_epoch) decision cache (bounded LRU). Then read object_paths(id) rows for THIS id only, stopping at the first path the policy lets the principal read: Allow(Root) when that path's read rule is '/', otherwise Allow(Restricted) (if any readable row is root-governed the class is Root). No rows, or none readable, gives Deny. Trees: a tree's object_paths rows come from path_index (new trees at their directory path), so root and subdirectory trees authorize the same way. The root tree is recorded at the empty path, governed by the '/' rule. get_objects streams per item: ok (StoredFrame::Zstd passed through unmodified as ZSTD, inline data as RAW; total_size; access_class; final), not_found, denied, redacted. Range reads (offset/length) return RAW decompressed slices. A missing or denied item never fails the batch. Server-side corruption gives an INTEGRITY item failure with metric and alert. Bytes served increment scion_object_bytes_served_total{tier=server,encoding}. has_objects: true only when the object exists and is readable, or the caller uploaded it (has_upload); otherwise false. resolve_ref: History::resolve_ref with at_version (strong). upload_objects (client stream): per item, verify the kind and the digest of the data (raw content), validate the decoded form for structured kinds, reject redacted ids (REDACTED), put via ObjectRepo, record_upload(principal). A redundant upload of an existing object is accepted and deduplicated with no leak (status ok without revealing prior existence). check_access: verify subject_token with the Verifier, then can_read for each id, returning allowed and access_class (used by daemons). Admin RPCs check Policy::can_admin (redact and unredact need Redact; purge_now needs Purge) and call the Redactor with actor = the caller's principal. list_redactions needs a valid token only (daemons poll it). Every admin action is audited by the Redactor. jobs.rs plus admin::spawn_background start Redactor::spawn_background (quarantine mover and purge scheduler). Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.7 for GetObjects, HasObjects, ResolveRef, UploadObjects, CheckAccess, Redact, Unredact, PurgeNow and ListRedactions: success, each error code each can return, and authentication failure. A batch mixing ok/not_found/denied/redacted returns each status correctly. 19.12: retroactivity (after set_policy adds a restriction, objects appearing only under the new prefix become unreadable immediately with no reindex, while objects also present at an unrestricted path remain readable); the HasObjects existence probe (false for an existing object the caller cannot read and did not upload; true after the caller uploads it; a duplicate upload succeeds without leaking); unreferenced uploaded objects are not readable via GetObjects by anyone; admin RPCs require their permissions. 19.13 server: after Redact, GetObjects returns redacted no later than effective_by (ManualClock plus the RedactionSet refresher bound); re-upload of a redacted id fails REDACTED; ListRedactions returns entries and current_epoch; the audit log records actor and reason; unredact restores reads byte-identically. 16.1 class 7: corrupting stored bytes yields an INTEGRITY item and scion_integrity_failures_total increments. 19.18: object authorization reads object_paths rows only for the requested object (Counters object_paths_rows_read equals that object's row count, independent of repository size at 10^3 and 10^5).", "size": "L" }, { "id": "P26", "title": "scion-server write and ref services: CreateChangeset, Rebase, Merge, Land, UpdateRef, CreateRef, DeleteRef, GetRefLog, policy authorizer adapters", "wave": 4, "spec_sections": "7.5 (CreateChangeset, Rebase, Merge, Land), 7.6, 8.1, 8.4, 8.5, 12.4, 14 (import_mode authorization), 15.2 (land metrics), 17 (audit of ref admin and forced updates), 19.2 (RPC items), 19.4 (RPC determinism), 19.5 (RPC items), 19.7, 19.12 (write checks)", "owns": [ "crates/scion-server/src/handlers/{write.rs,refs.rs}", "crates/scion-server/src/authz/policy_adapters.rs", "crates/scion-server/tests/write_*.rs", "crates/scion-server/tests/refs_*.rs" ], "depends_on": [ "P19", "P20", "P21", "P23" ], "deliverables": "Implement the P26 handlers of PLAN 2.14 and the policy adapters. PolicyWriteAuthz(Arc, who) implements WriteAuthorizer: can_write = Policy::can_write; can_import = can_admin(Refs) && can_write('/'). PolicyLandAuthzSource(handle, who) implements LandAuthorizerSource: current() snapshots PolicyHandle::current() on every call, so policy is re-read on each land attempt. create_changeset converts the request (a missing meta_fields or change_id gives INVALID_ARGUMENT/missing_field; any client-supplied creator is impossible because MetaFields has no creator) and calls History::create_changeset with PolicyWriteAuthz. rebase calls History::rebase. Rebasing someone else's changeset is allowed for any repository reader: rebase needs read on '/' only, because results are unnamed changesets and landing checks write. merge calls History::merge with PolicyWriteAuthz. land calls History::land with PolicyLandAuthzSource. The land metrics come from the LandObserver that P23 wires into History. This handler logs land_attempt, ref, ref_version and changeset_id on the RPC event. update_ref calls History::update_ref with the current policy snapshot as LandAuthorizer. create_ref and delete_ref require can_admin(Refs) and pass the reason into the audit. get_ref_log returns entries. MERGE_CONFLICT, REF_VERSION_MISMATCH and ALREADY_LANDED payloads are encoded through scion-proto. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.2 via RPC: creator is server-set (a principal X creates and Meta.creator == X; MetaFields has no creator field); expected_root_tree mismatch is rejected; import_mode without admin.refs gives PERMISSION_DENIED; import_mode allows a second parentless changeset. 19.5 via RPC: 1000 concurrent UpdateRef calls at the same expected version give exactly one success, versions increase by exactly 1, and the ref log has no gaps; ABA; NOT_FAST_FORWARD without the flag or without force, success with both; Land happy path; Land idempotent retry after FailAfter at meta.cas_ref.after returns the recorded result. 19.4: Rebase via RPC gives the same id across InProcessServer.restart(). 19.12 write checks: CreateChangeset and Land reject unwritable paths; ref update and force are enforced per ref pattern; CreateRef and DeleteRef require admin.refs. 19.7: every write and ref RPC has success, each listed error (MERGE_CONFLICT payload with clean_entries, EMPTY_CHANGE, AMBIGUOUS_MERGE_BASE, REF_VERSION_MISMATCH payload, NOT_FAST_FORWARD, ALREADY_LANDED payload, INVALID_ARGUMENT subcodes, TOO_LARGE, REDACTED, PERMISSION_DENIED) and authentication failure. The land metrics move.", "size": "L" }, { "id": "P27", "title": "scion-server query service: GetChangesets, ReadPath, ListTree, Diff, IsAncestor, MergeBase, Resolve, GetDerived; history filtering", "wave": 4, "spec_sections": "7.4 (except Log/Blame), 9.1 (generic derived query path), 11.8 (revision resolution), 12.6, 13.4, 4.2 (prefix resolution with candidates), 19.1 (ambiguous prefix), 19.6 (generic path), 19.7, 19.12 (history filtering), 19.13 (ReadPath REDACTED)", "owns": [ "crates/scion-server/src/handlers/{query.rs,derived.rs}", "crates/scion-server/src/revision.rs", "crates/scion-server/tests/query_*.rs" ], "depends_on": [ "P12", "P13", "P14", "P21", "P23" ], "deliverables": "Implement the P27 handlers of PLAN 2.14. Every query requires repository read (read on '/') unless noted. get_changesets: per item ok/not_found. Return the full canonical encoding only when every delta path is readable; otherwise a filtered_view with the unreadable delta entries removed and filtered = true (12.6). Include the generation. read_path: read on the path required; resolve the entry through History::lookup; stream the content for files; REDACTED for a redacted blob; NOT_FOUND for a missing path; a directory returns the entry with no content. list_tree: read on the directory; entries include restricted subdirectory names and ids (12.6 limitation); the tree id is returned. diff: History::file_diff between a and b with optional path scope; entries the caller cannot read are omitted; include_content_hunks uses scion_merge::unified_hunks for text, with binary flagged; streamed. is_ancestor and merge_base map to History. resolve (revision.rs): parse_revision, then the base: - Id: must exist. - IdPrefix or a bare hex Name (8 or more hex digits): changesets_by_prefix, with zero giving NOT_FOUND and more than one giving INVALID_ARGUMENT/ambiguous_prefix with a Candidates payload. For a bare Name, try an exact ref name first. - Ref and RefAt: History::resolve_ref. - Git: git_mapping::changeset_for_git. Then apply the ~n/^k suffixes through History. get_derived: DerivedEngine::get(type, version, key) for registered types; an unknown type gives NOT_FOUND. Content-bearing types require read on '/', and so do non-content types. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.12 history filtering: GetChangesets returns a filtered view for a principal lacking read on one delta path, and the full encoding when every path is readable; Diff omits restricted entries; ReadPath and ListTree on restricted paths give PERMISSION_DENIED; restricted subdirectory names are visible in a readable parent listing. 19.13: ReadPath on a redacted blob gives REDACTED. 19.1: an ambiguous short prefix (two changesets crafted to share an 8-hex prefix by brute-force search over change_ids) is rejected with both candidates; full ids, c1-prefixes, refs, ref@version, git: (via an import_mode changeset) and ~n/^k resolve correctly. 19.6: a test-only derived type registered through ServerTestOpts.registry is queryable through GetDerived with no client change; a version parameter selects old values. 19.7: every query RPC has success, NOT_FOUND, INVALID_ARGUMENT, PERMISSION_DENIED and UNAUTHENTICATED cases; trailers are present on the streaming Diff and ReadPath.", "size": "L" }, { "id": "P28", "title": "scion-wc write layer: commit, amend, rebase with conflict materialization, resolve, abort, land, commit/rebase recovery", "wave": 5, "spec_sections": "11.6, 11.7, 11.3 (pending-operation recovery), 13.7 (editing placeholders), 16.1 class 8, 19.10 (commit, amend, conflict flow, crash recovery, inaccessible paths)", "owns": [ "crates/scion-wc/src/{commit.rs,rebase.rs,resolve.rs,land.rs,testing_write.rs}", "crates/scion-wc/tests/commit_*.rs", "crates/scion-wc/tests/rebase_*.rs", "crates/scion-wc/tests/land_*.rs", "crates/scion-wc/tests/recover_*.rs" ], "depends_on": [ "P18", "P25", "P26", "P27" ], "deliverables": "Implement the P28 methods of WorkingCopy (PLAN 2.17) plus crate::commit::recover_commit and crate::rebase::recover_rebase, which P18's recover() dispatches to. commit: run status within scope (paths restricts it); refuse paths outside scope or inaccessible. Compute the delta versus base (Put with mode, blob id and size; Delete), sorted. HasObjects, then UploadObjects only for missing blobs. Journal PendingOperation::Commit (fault points wc.commit.after_journal, after_upload, after_create), then CreateChangeset(parents = [base], new ChangeId::random(), MetaFields with author and timestamp from the clock and the local tz). Move base to the result, refresh the index and clear pending. amend: CreateChangeset(parents = base.parents, same change_id), reusing the original Meta fields unless a new message is given. rebase(onto): resolve onto, compute the local stack (the first-parent chain from base back to the first ancestor of onto, via IsAncestor), journal PendingOperation::Rebase, and call server Rebase per member, chaining. On MERGE_CONFLICT: goto onto's merged state (the previous rebased member), write clean_entries, then for each conflict: - text (content kind): write the file with render_markers(base, ours, theirs) using the 11.7 markers; - other kinds: write the ours version (or nothing when absent) and save every side under .scion/conflicts/.{base,ours,theirs}. Record conflicts in pending. resolve(path, force) refuses while marker lines remain, unless force. commit while rebasing requires every conflict resolved, creates the changeset with the original member's change_id, author and message (parents = [current onto]), then continues with the remaining stack. rebase_abort returns to the original changeset (goto original) and clears pending. land: server Land(ref, base); on success base becomes the landed result (resolved via the returned target, the stack member's result), and tracking is updated. recover_commit: if the changeset was created (detectable by re-issuing the deterministic CreateChangeset, which is idempotent), finish by moving base; otherwise roll back by clearing pending, leaving the files untouched. recover_rebase resumes or rolls back to a consistent state. testing_write.rs (feature testing) exports scenario functions for P30 and P33. Tests run against InProcessServer (real server) directly. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.10: commit uploads only missing blobs (UploadObjects item count equals the new blobs) and sends a delta equal to scion-refmodel file_diff(base tree, working tree). amend preserves change_id. Commits cannot include out-of-scope or inaccessible paths. Full conflict flow: markers written for text conflicts, side files for binary/symlink/modify_delete, resolve refuses remaining markers (force overrides), continuing the commit preserves change_id, author and message, and --abort restores the original exactly. Crash injection at every wc.commit.* and wc.rebase.* fault point leaves a recoverable pending_operation that the next command completes or rolls back, after which status is consistent and the server state is valid. Land moves base to the landed result. Editing a redacted placeholder and committing creates new content normally (13.7).", "size": "L" }, { "id": "P29", "title": "scion-import: one-way git importer with fidelity and incremental re-import", "wave": 5, "spec_sections": "14, 9.2 (git_mapping), 4.3.3 (extension tag 2), 4.3.4 (importer parentless changesets), 19.14, 19.6 (blame vs git blame)", "owns": [ "crates/scion-import/src/**", "crates/scion-import/tests/**" ], "depends_on": [ "P09", "P24", "P25", "P26", "P27" ], "deliverables": "Implement PLAN 2.19 using gix (pure Rust; no libgit2). CLI: `scion-import --git --ref --target-ref main [--skip-submodules] [--endpoint ..]`. The token comes from SCION_TOKEN or service login using SCION_SERVICE_NAME/SCION_SERVICE_SECRET. Walk commits reachable from the git ref in topological order, parents first. For each commit: - Skip it if Resolve('git:') already resolves (incremental and idempotent). - Upload blob contents not yet present (HasObjects, then UploadObjects in batches up to batch_bytes). - delta = FileDiff(first parent's git tree, commit tree) using map_mode: 100644/100664 map to File, 100755 to Executable, 120000 to Symlink. Empty git trees are dropped naturally because Scion has no empty directories. Gitlinks (160000) fail the import unless skip_submodules is set; when it is, they are omitted and recorded in Meta extension tag 4 (list<(path, 20-byte sha)>). - Meta: author and committer bytes 'Name ', author_time/committer_time seconds plus tz offset in minutes, message bytes verbatim, extension tag 2 = the 20-byte git commit id. - change_id = BLAKE3('scion-import-v1' || 20-byte id)[0..16]. - CreateChangeset with import_mode = true and parents = the mapped Scion ids of all git parents (octopus merges keep every parent; parentless roots are allowed). The server derives git_mapping eagerly. After the walk: CreateRef(target, tip) when absent, else UpdateRef fast-forward with expected_version. ImportReport. testing.rs (feature testing) exposes fidelity helpers. Tests use InProcessServer with a service principal that has write on '/' and admin.refs, plus the vendored corpus/git bundles (P09). Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.14: differential fidelity over every commit of both real bundles and every synthetic edge bundle. The Scion tree's file set, modes and contents (ListTree/ReadPath, or GetObjects plus tree walks) equal the git tree read via gix, and via `git ls-tree -r` too when the git CLI is present; skipped gitlinks and empty trees are the documented exclusions. Meta preserves author, committer, both timestamps and zones, and message byte-for-byte. git_mapping resolves in both directions for every commit (Resolve('git:') and GetDerived git_mapping 'c:' key). Re-import is idempotent (MemMetaStore table_counts unchanged). Incremental import after adding commits imports only the new ones and fast-forwards the ref. Octopus merges, root commits, multiple roots, empty commits, mode 100664 and symlinks import correctly. A gitlink fails without --skip-submodules and is recorded in Meta extension 4 with it. 19.6: Blame RPC output equals `git blame --porcelain` line origins (mapped through git_mapping) for every file without renames in the imported real corpora (skipped when git is absent); exceptions are documented in tests/BLAME_EXCEPTIONS.md.", "size": "L" }, { "id": "P30", "title": "scion-conformance harness and suites A: TestCluster, RPC contract matrix, tier conformance, auth end to end, ACL matrix, observability", "wave": 5, "spec_sections": "7.1-7.8, 10.4, 12.1-12.6, 15.1-15.3, 19.0, 19.7, 19.8, 19.11 (end to end), 19.12 (matrix), 19.17", "owns": [ "crates/scion-conformance/src/{cluster.rs,invariants.rs,seed.rs,support.rs}", "crates/scion-conformance/tests/{rpc_contracts.rs,tier.rs,auth_e2e.rs,access_matrix.rs,observability.rs}" ], "depends_on": [ "P16", "P24", "P25", "P26", "P27" ], "deliverables": "Implement PLAN 2.22 (lib.rs is P00's and declares the modules). TestCluster::start: - Backend::Memory gives Storage::in_memory. Backend::External gives CrdbMetaStore::connect_fresh_for_tests plus S3BlobStore::test_config wrapped by Storage::new. - N InProcessServers share that Storage (servers > 1 for the multi-server suites). - An optional InProcessDaemon in front of servers[0]. - A FakeGitHub. - A LogCapture installed. - A default ACL with groups eng/admins/security, '/' readable by eng, and a /secrets restriction. client(via, who) returns a ScionClient authenticated as who, directly or through the daemon. seed(GenRepo) uploads blobs and creates changesets through RPCs, returning ids. counters() snapshots. invariants::check_16_2 runs fsck, checks ref-log provenance (every ref version equals a recorded successful update) and verifies digests of every object served in the run. Suites run with Backend::from_env(): Memory always, External when the env vars are set. rpc_contracts.rs: a table-driven matrix that enumerates every RPC from v1::FILE_DESCRIPTOR_SET, so an RPC without a case fails the test. For each RPC: success, each error code listed for it in SPEC 7.2-7.7, and authentication failure. ErrorDetail.retryable matches 7.2 for every code observed. All three trailers are on every response from both tiers. Unknown request fields are ignored; missing required fields give INVALID_ARGUMENT. tier.rs: ONE read-RPC suite (GetServerInfo, GetObjects including batches, ranges and mixed statuses, HasObjects, ResolveRef) run unchanged with Via::Server and Via::Daemon; per-item results and decompressed payloads are byte-identical for the same principal and epochs. Write, query, ref, auth and admin RPCs through the daemon behave identically to direct calls. Network-policy test: DirectUpstream destinations contain only the configured upstream. Re-run the P18 scion_wc::testing checkout scenarios against the real stack via the daemon. auth_e2e.rs: GitHub and service login through the daemon; token rejection at both tiers; rotation. access_matrix.rs: every read and query RPC x {allowed, denied at '/', denied at a subdirectory} x {server, daemon} returns the specified result. observability.rs: after running a representative workload, every RPC log event has the required fields; LogCapture.find_needles finds no token, service secret, GitHub token, file content or raw restricted path (PLAN 6.9); every 15.2 metric is exported and moves in its scenario; /healthz and /readyz reflect dependency state (MemBlobStore.set_down and the MemMetaStore fault point make readyz 503). Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.7 (complete matrix, descriptor-driven), 19.8 (tier conformance including the network policy), 19.11 end to end through the daemon, 19.12 matrix, 19.17 (log fields, log scan, metrics move, health), all on the Memory backend and additionally on External when SCION_TEST_CRDB_URL and SCION_TEST_S3_ENDPOINT are set (test names prefixed external_ for the External variant).", "size": "L" }, { "id": "P31", "title": "scion-cli: all core commands, --json with versioned published schemas, device-flow login, redact tooling", "wave": 6, "spec_sections": "11.8, 6.4, 12.2, 13.2 (redact tooling), 13.4 ( rendering), 17 (credentials), 19.10 (--json schemas), 19.11 (login, credentials), 19.13 (--all-versions, rendering)", "owns": [ "crates/scion-cli/src/**", "crates/scion-cli/tests/**", "schemas/v1/**" ], "depends_on": [ "P28", "P30" ], "deliverables": "Implement PLAN 2.18 for scion-cli with clap derive. Commands: - login: GetServerInfo for github_client_id and github_oauth_base_url, the device flow (scion_auth::device_flow, prompting with the user code and URL), Login, then the credentials file at 0600 with the GitHub token stored for silent re-exchange; on UNAUTHENTICATED from an expired token, re-exchange automatically. - whoami. - checkout [--sparse p...]; sparse add|remove [--force]; status; diff; commit -m [--path]; amend [-m]; goto [--force]; rebase --onto | --abort; resolve [--force]; land [--ref main]. - log [path] [--first-parent] [--limit]; blame ; show ; cat :. - redact --path

--at [--all-versions] | --object | --meta , with [--reason] [--purge-now]. --all-versions resolves every distinct blob via Log(path) plus ListTree/ReadPath entries per version. Revision display uses short ids (c1 + 12 hex) for humans and full ids in --json. Endpoint precedence: flag > SCION_ENDPOINT > .scion/config.toml > ~/.config/scion/config.toml. SCION_TOKEN overrides credentials. Redacted Meta renders as '' in log and show. Every command supports --json, emitting Envelope{schema: 'scion.cli..v1', data}. Schemas are generated with schemars into schemas/v1/.schema.json and committed. A test regenerates them and fails on any diff. Exit codes: 0 success, 1 error, 2 usage, 3 conflicts, 4 dirty-tree refusal (documented in the --help epilogue). Tests use assert_cmd plus run() against a TestCluster. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.10: the --json output of EVERY command validates against its published schema (jsonschema crate) in end-to-end runs against TestCluster, and the committed schemas match the generated ones. 19.11: device-flow login against FakeGitHub yields principal github:; an invalid token gives an UNAUTHENTICATED exit; the credentials file is written 0600 and a group-readable file is refused. 19.13: redact --path --all-versions finds every distinct blob the path ever held (compared with scion-refmodel all_versions on a generated history); log and show render after a Meta redaction; redact actions produce audit entries. Smoke: every command's human output on a scripted session.", "size": "L" }, { "id": "P32", "title": "Conformance suites B1: land concurrency, linearizability, no-locks, whole-system differential against the reference model", "wave": 6, "spec_sections": "8.4, 8.5, 16.1 class 5, 19.0 (History Recorder), 19.4 (determinism across server processes), 19.5 (concurrency and linearizability)", "owns": [ "crates/scion-conformance/tests/{land_concurrency.rs,linearizability.rs,differential.rs}" ], "depends_on": [ "P09", "P26", "P30" ], "deliverables": "Cross-cutting suites over TestCluster (servers = 3 sharing one Storage, for multi-process semantics). land_concurrency.rs: N = 10 and 100 concurrent clients by default (N = 1000 in an #[ignore] full_ test) landing non-overlapping changes onto main via the Land RPC across the servers; overlapping text changes that merge cleanly; true conflicts. linearizability.rs: concurrent ResolveRef/UpdateRef/Land operations recorded with scion_testkit::lincheck::HistoryRecorder, under injected server kills (kill_server/restart_server mid-run; in-flight operations complete as Unknown) and network delays (a tower delay layer configured through ServerTestOpts.tweak or the fault registry's Delay at meta points). check_ref_registers must report zero violations over many seeds, with about 200 operations per history. differential.rs: a seeded random operation driver (create, rebase, merge, land, update_ref, redact) applied both to the real stack via RPC and to scion-refmodel; it compares trees (flattened), conflict path sets and kinds, ref-log order, error codes and landed sets, with at least 200 seeds by default (more under full_). Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.5: with N concurrent clients landing non-overlapping changes, all land, the final tree contains every delta exactly once, and each change id appears exactly once in landed. Overlapping clean merges: all land, and the final content equals scion-refmodel land_sequential applied in the landed order recorded in the ref log. True conflicts: exactly the conflicting lands fail MERGE_CONFLICT and the others land. Idempotency after a crash between CAS commit and response (FailAfter at meta.cas_ref.after): the retry returns the recorded result with no second version. No locks: the lease_calls delta is 0 and no scion::lock span appears during the concurrent land suite (LogCapture/tracing). Linearizability: zero violations with kills and delays. 19.4: Rebase gives the same id across different server processes and after restart. Whole-system differential: zero divergences. Every suite also runs on External when the env vars are set.", "size": "L" }, { "id": "P33", "title": "Conformance suites B2: failure classes, complexity gates, write ordering at RPC level, redaction end to end, storage end to end, client end to end", "wave": 6, "spec_sections": "4.5, 5.3, 13.3, 13.7, 16.1, 16.2, 19.3, 19.6 (eager values under faults), 19.10 (real stack), 19.13 (end to end), 19.16, 19.18", "owns": [ "crates/scion-conformance/tests/{failures.rs,complexity.rs,write_ordering.rs,redaction_e2e.rs,storage_e2e.rs,client_e2e.rs}" ], "depends_on": [ "P22", "P28", "P29", "P30" ], "deliverables": "Cross-cutting suites over TestCluster. failures.rs: one test per SPEC 16.1 class, each followed by invariants::check_16_2. 1. Metadata down (MemMetaStore fault Error on all meta points): writes give UNAVAILABLE; immutable reads continue from server caches (decoded LRU) and daemon cache; ref reads fail at the server while the daemon serves its TTL ref cache. 2. Object storage down (MemBlobStore.set_down): cache-miss reads fail; writes give UNAVAILABLE. 3. Crash mid-upload leaves harmless orphans. 4. Crash mid-CreateChangeset (Crash at each create.* point, then restart_server): no visible row; a retry yields the same id. 5. Crash after the CAS commit, before the response: Land retry returns the recorded result. 6. Daemon crash (kill/restart InProcessDaemon): temp files deleted, cache valid. 7. Corrupt object at each tier: discarded and refetched; the server reports INTEGRITY and scion_integrity_failures_total. 8. Client crash mid-operation (wc fault points): recovered by the next command. 9. Invalid policy reload: the last good policy is retained and the error surfaced. 10. GitHub down: existing tokens work and new logins fail. complexity.rs: every 19.18 gate end to end via RPC on shared_subtree_repo at 10^3 and 10^5 files (10^7 in an #[ignore] full_ test) with equal depth, using Counters deltas only: - CreateChangeset 1-file: reads at most depth+c trees, writes at most depth+1. - Rebase over an unrelated tip: 0 blobs, O(|delta| x depth) trees. - Land 1-file: constant meta_txns per attempt. - Sparse checkout of a fixed directory: same object count at every size. - Clean status: 0 upstream reads (ScionClient.calls) and only racy or stat-changed files hashed. - Eager derivation: at most depth+1 history nodes. - Object authorization: object_paths rows read only for the requested object. write_ordering.rs: crash injection at every Checkpoint and every meta/blob decorator point during upload, CreateChangeset and Land via RPC. No visible changeset violates 4.5 (fsck with required_eager); eager values exist for every visible changeset (19.6). redaction_e2e.rs: the full runbook through the real stack and the daemon: redact, the server returns redacted by effective_by, the daemon by one poll interval plus one request with its cached copy deleted, daemon fail-closed, purge, Meta redaction rendered in Log, re-introduction blocked, carry-forward allowed, and working-copy placeholder replacement. storage_e2e.rs: the 19.3 items at RPC level (a row never exists without bytes, inline threshold +-1 via UploadObjects/CreateChangeset, the stale-read retry, orphans harmless). client_e2e.rs: re-run the scion_wc::testing and testing_write scenario functions (P18/P28) against the real stack, directly and via the daemon. All suites run on External when the env vars are set. Definition of done: cargo fmt --check, cargo build --workspace --all-targets, cargo test --workspace, cargo clippy --workspace --all-targets -- -D warnings and scripts/check-layers.sh all pass (scripts/check.sh), only owned files are touched, and every frozen signature in docs/PLAN.md section 2 is unchanged. Additive pub items are allowed only inside owned files; any other contract change goes into the final report as an amendment request.", "tests": "19.16: each of the 10 failure classes with the 16.2 invariants checked afterwards. 19.18: all seven gates at 10^3 and 10^5 (10^7 under full_). 19.3: write ordering across every crash point and storage items at RPC level. 19.6: eager values exist under faults between derive and insert. 19.13: end-to-end propagation bounds (server effective_by; daemon one poll interval plus one request; fail-closed), runbook flow, working-copy replacement. 19.10: client scenarios pass on the real stack directly and via the daemon.", "size": "L" } ]