Skip to content

State Model

Location

The primary state database is local to the machine, not placed automatically on a scanned removable or network volume.

  • macOS: ~/Library/Application Support/optiflow/state.sqlite3
  • Linux: $XDG_STATE_HOME/optiflow/state.sqlite3
  • Linux fallback: ~/.local/state/optiflow/state.sqlite3

SQLite policy

The database uses the rollback journal and FULL synchronization. This avoids assuming SQLite WAL shared-memory behavior is safe on arbitrary scanned filesystems. A busy timeout handles short contention without hiding persistent ownership conflicts.

Tables

Table Purpose
schema_migrations Applied database schema versions
scan_runs Run lifecycle and final manifest/report JSON
observations Queryable path, size, hash, and complete observation JSON
duplicate_groups Derived group JSON for each completed run
file_cache Reusable unchanged-path analysis

Cache identity and invalidation

The cache key is:

native path + filesystem id + file id + size_bytes + modified_unix_ns + changed_unix_ns

Any mismatch causes re-analysis. Missing filesystem identity or metadata-change time disables cache lookup and insertion for that observation. Cache rows from older migrations have null handle-signature fields and are therefore not reused. A cache hit is accepted only inside a fresh, stable handle-bound observation window.

If a cached file previously had no full hash but becomes a size-based duplicate candidate, OptiFlow calculates and stores its full hash during the new scan.

Probe-enabled cache lookup also binds the resolved provider's version, canonical path, binary digest, and invocation fingerprint. A cached missing-media result from an available provider is not reused; later scans retry the provider. The probe-disabled path does not reuse a provider-bearing cache row, so unrequested media facts never lose their provider provenance. The derived media-profile document is recomputed from accepted observations and the current effective evidence-policy fingerprint, then stored only inside the immutable report. No database migration or profile-specific cache table is introduced.

The cache recognizes unchanged filesystem objects at unchanged native paths. Content-addressed recognition after arbitrary moves is a later enhancement because non-candidate files are not all hashed in the MVP.

Run lifecycle

  1. Insert a running scan row.
  2. Perform discovery and analysis.
  3. Validate and stage effective-policy.json, run.json, report.json, and their digest-bearing commit marker.
  4. Flush the members and marker, then publish the directory with one rename.
  5. Store observations and groups in one database transaction.
  6. Mark the run completed and attach final JSON.

Handled interruptions mark rows interrupted; internal failures mark rows failed; a hard process failure may leave recoverable running state. None is returned as a completed report. A still-running row with a committed v5 run and v6 report artifact set is recovered idempotently when state is next opened. Stale staging directories are removed; incomplete or incompatible final sets are never promoted.

The effective-policy sidecar is referenced through the existing immutable run artifact directory, so migrations 0001–0003 and v1-v4 run/report schemas are unchanged. Historical runs without a sidecar have unknown policy evidence; the current resolver never fabricates a snapshot for them.

See the artifact-set commit protocol for marker, reader-state, plan-handshake, crash-recovery, and durability semantics.

Execution evidence (migration 0006)

execution_plans and execution_approvals hold immutable fingerprinted documents. execution_runs stores versioned dry-run, attempt, validation, commit and recovery evidence. Migration is additive, transactional and idempotent; it does not derive approval from historical scan/review state.

The execution dry-run protocol defines the process lock, validating/validated/rejected/interrupted lifecycle, terminal immutability and abandoned-attempt recovery. Source commit is impossible in these v1 records. Execution opens do not perform scan artifact cleanup. Unknown future execution state versions are refused before writable initialization.

Migration 0007 keeps Linux quarantine execution_mutation_runs in separate v2 evidence. Migration 0008 appends immutable v3 recovery transitions under execution_recovery_events, with an authority binding before new mutation. Historical v2 rows without that binding remain inspection-only. The recovery protocol defines status, restore, resume, empty namespace cleanup and ambiguous interruption handling.

Migration 0009 adds immutable execution_finalization_events with v4 removal_pending and removed evidence. It leaves v1-v3 documents intact; the finalization protocol requires a separate preview and authorization, and a pending removal never becomes committed by status inspection.