System Journal

The system journal records what an instance DID, one record per operational event: a store opened or closed, a checkpoint written, a backup taken or failed, damage found, a rollback, a permission denied, the system clock stepping back. A person reads it later ("what happened to this store last night?"); a host forwards it to its own logging.

It sits beside three other streams and replaces none of them:

The journal decides nothing: no part of Graphersal reads it to decide anything, and a failed journal write never fails the operation it describes (it is a tracing warning and a counted lost record, SystemJournal::lost_records). Records hold names, counts, durations and ids: never data values, query text or secrets.

Records

Every record has the same envelope plus typed fields:

A reader of an older version shows a record of a kind it does not know by its message and fields.

A recommendation field says how to fix what the record reports, in the layout of every Graphersal note: a short headline, then each command on its own line under a # comment saying what it does. The text form of a record (JournalRecord::to_line, graphersal store journal, the playground's journal panel) keeps the one-line fields in parentheses on the record's line and puts a field of several lines below it, indented:

2026-10-09T08:53:20.000000Z #12 error  model.unavailable the model acme/tiny is missing: queries with it fail (repo=acme/tiny, identity=model2vec/acme/tiny@0123456789ab, cause=missing)
  recommendation: download the model again and import it into the store, or restore the store from a backup or replica
    # download the model's files again
    graphersal embed-model download acme/tiny
    # import them into the store in place of the damaged copy
    graphersal embed-model import acme/tiny --into <store> --replace

The JSON forms (--json, GET /api/journal, MCP, Python) keep the field as one string with its line breaks.

Repeats and states

The journal is append-only, so a repeat is collected BEFORE it is written, never by rewriting a record. Every kind declares a collapse key (its kind, its source and some of its fields; never the time). The first occurrence is written at once. A record whose key equals one of the last five written records is counted in memory instead; the group is closed by ONE journal.repeated record (with the original's origin_seq, the count and the time range) when the original leaves the last five records, when an hour has passed since the original (a continuing problem shows again), or when the journal is flushed or closed. A crash loses only the count of an open group.

A state (store.damage_found, store.read_only) is written once, its repeats are counted while it holds, a journal.state_reminder repeats its current values once per hour, and its ..._cleared kind ends it.

Where it is kept

• A graph without a store keeps its journal in memory: the newest 10 000 records (JournalPage::truncated says when older ones are gone). The playground's graphs too.
• A Store keeps it in its directory (format spec 4.7): segments journal/<20 digits>.jnl (the name is the first seq the segment may hold), append-only, framed like the WAL (sync marker, length, header and body CRC: a torn tail is cut by the next writer, a damaged record is skipped by resyncing on the next frame). A new segment starts past 4 MiB (StoreOptions::with_journal_segment_bytes), at an open by another instance, and after a failed write. There is no retention limit.
• error records are written and synced at once. The others are batched in memory and written as described in When records appear, so the journal never writes in the middle of another operation of the store; a crash loses at most about the last second of records.
• A store opened read-only (a backup, maintenance mode, a store holding something critical of a newer version) reads its stored journal; its new records stay in memory. Once damage is found while a store is open (either damage policy), its journal writes nothing more to the store's files: the records stay in memory and in the tracing mirror.
• The instance identity lives in the instance-local file INSTANCE at the store root (a random id and a name: StoreOptions::with_instance_name, else the host name). No copy carries it: a restored backup or a converted store opened elsewhere is another instance, which writes into a new segment.
• The journal is never store damage: verify lists its problems as notes (VerifyReport::journal_notes), and its files are left out of the damage found while open.

When records appear

• In the process that holds the graph or the store a record is visible at once: every read (Graph::system_journal(), Store::journal(), the session, GET /api/journal, the MCP tool journal, the playground, Python's graph.journal/store.journal, the REPL) answers from the records in memory, written or not, and reads only older records from the segments, so a page never has a gap or a duplicate, before or after the batch is written.
• On disk, and so for another process (graphersal store journal or Store::journal_dir on a store a server holds open), a record appears within about a second: a store on a file system writes the batch half a second after its first record, from a thread of its own, independent of the statistics job stats_flush. The thread writes only while no other operation of the store runs (a checkpoint, prune, compaction, rollback, attic change, verify, backup, fork, export or close; commits are not waited for, they write other files); during a long checkpoint the batch waits for its end.
• Without threads (wasm32), and for a store in memory or in a host's own StoreDir, the host calls Store::flush_journal(): the playground does on its every-second tick (Session::tick_jobs). The system job stats_flush, SystemJournal::flush (which also closes the groups of repeats), the close and the next error record write the batch too, as does a batch of 1000 records or 1 MiB.

Copies

Reading it

• Rust: Graph::system_journal() (every graph), Store::journal() (an open store), Store::journal_dir(dir, &query) (a closed store, or one open elsewhere: no lock, nothing written). SystemJournal::query(&JournalQuery) filters by time range (since/until), level (min_level), source (source_prefix: store matches store.backup), kind, op and origin, from a cursor (after(seq)), at most limit records (default 1000): JournalPage::next_after is the next cursor, more says that more follow. JournalQuery::newest_first() answers the NEWEST matching records instead, newest first (a stored journal is read from its last segment backwards, one segment in memory at a time, and no older segment than needed is read): next_before is the cursor of the next older page (before(seq)), more says that older ones exist, and next_after is the newest record covered, so a reader shows the end of a long journal at once and then follows it. SystemJournal::subscribe(after) follows the journal (wait(timeout) blocks natively, WebAssembly polls); add_sink forwards every written record to a host's JournalSink; open_operations() is the last record of each operation without a _done/_failed record. Facade: graphersal::system_journal.
• tracing: every written record is mirrored under the target graphersal::journal at its level (notice as info); the CLI shows them on stderr with GRAPHERSAL_LOG=info.
• CLI: graphersal store journal <DIR> [--level L] [--since TIME] [--source P] [--kind K] [--after SEQ | --last N] [--limit N] [--follow] [--json]; --last N prints only the newest N records (The graphersal store Command).
• Dev server: GET /api/journal?after=N&level=..&source=..&kind=..&limit=.. (newest_first=true&before=N: newest first, paging backwards), and the read-only MCP tool journal (the same keys) (Dev Server and MCP).
• Python: graph.journal(...) (every graph; without a store its in-memory journal), store.journal(...) and Store.journal_dir(path, ...) (a closed store, or one open elsewhere), with the keyword arguments after, before, newest_first, since, until, level, source, kind, op, origin, limit and policy. A page is a dict {"records", "next_after", "truncated", "more", "notes"}; each record has the keys of store journal --json (Python).
• Playground: Catalog ▸ System journal… shows the served graph's journal, newest first, with a level and a source filter, in the WebAssembly playground (the graph's store in memory) and over the dev server. It opens on the newest 300 records (a newest-first page), then reads only the records written since, every five seconds; "Show more" shows 300 more rows, reading older pages backwards (Web Playground).
• Permissions: reading is the resource Journal (Resource::Journal, Read); the session, the dev server and MCP ask the authorizer before they answer. AccessPolicy::read_only() allows it. The Rust API itself asks nothing (it is the host's).

The journal has no DSL access yet.

Event catalogue

Every kind the journal writes, generated from the declarations in the code (GRAPHERSAL_UPDATE_BOOK=1 cargo test -p graphersal --lib catalogue):