Memory¶
The narrative memory subsystem. In every mode it provides hybrid search
(axiom.memory + axiom.retrieval); in living mode it additionally distils
the narrative into structured facts (axiom.facts + axiom.factextract)
and, with beliefs enabled, into evolving observations (axiom.observations +
axiom.consolidate) shaped by per-character missions (axiom.missions) and,
one layer up, curated mental models (axiom.mental_models + axiom.reflect).
See the Memory guide for the concepts.
axiom.memory¶
llm_engine/vector_memory.py
Local vector-database memory for Axiom AI narrative chunks.
Every piece of narrative embedded here carries a turn_id metadata tag. This enables the surgical rollback required by the Checkpoint system: when the player rewinds to turn N, all chunks with turn_id > N are permanently deleted so they cannot bleed into the rebuilt timeline.
Backend: ChromaDB (persistent, local) Embedding model: sentence-transformers all-MiniLM-L6-v2 (fully offline)
Collection layout¶
Collection name : “narrative_memory” Document : the text chunk Metadata fields : save_id (str), turn_id (int), chunk_type (str) ID : UUID string, generated per chunk
- axiom.memory.preload_embedding_runtime()[source]¶
Force torch’s native runtime to load on the calling (main) thread.
The sentence-transformers embedding model is loaded and used on worker threads (VectorInitWorker / NarrativeWorker). The first encode lazily pulls in
torch._dynamo→triton, whichdlopen()``s ``libtriton.so. Doing thatdlopenfrom a secondary thread while Qt is running segfaults (native crash, no Python traceback). Importing it once here, on the main thread at startup, makes the later cross-thread use safe.Call this from the GUI/CLI entry point before any worker thread touches VectorMemory. Idempotent, never raises. Returns True if the runtime was pre-loaded, False if torch is unavailable (e.g. headless test stubs).
- Return type:
- class axiom.memory.VectorMemory(persist_dir, reranker=None)[source]¶
Local semantic memory store backed by ChromaDB.
- Parameters:
- embed_chunk(save_id, turn_id, text, chunk_type='narrative', metadata_extra=None)[source]¶
Embed a text chunk and store it with turn_id metadata.
- query(save_id, query_text, k=5, current_turn_id=None, max_turn_id=None, focus_terms=None, chunk_type=None, exclude_chunk_type=None)[source]¶
Retrieve the top-k most relevant chunks using hybrid scored search.
- Parameters:
focus_terms (list[str] | None) – Optional “here-and-now” terms (current location name, on-scene character names). A chunk whose text mentions any of them gets a small additive boost so memories about the current scene surface more readily. No effect when omitted.
chunk_type (str | None) – Restrict the search to chunks of this type (e.g. “lore”).
exclude_chunk_type (str | None) – Exclude chunks of this type (e.g. the narrative query passes “lore” so lore entries never eat its k budget).
save_id (str)
query_text (str)
k (int)
current_turn_id (int | None)
max_turn_id (int | None)
- Return type:
Each returned candidate carries
entry_id(the chunk’s source id when one was stored, e.g. a lore entry’s id; otherwiseNone).
- sync_lore(save_id, entries)[source]¶
Re-embed this save’s Lore Book entries (idempotent).
Deletes any existing lore chunks for the save, then embeds each entry tagged
chunk_type="lore"andturn_id=0(timeless → survives any rewind) with itsentry_idin metadata, so a retrieval hit maps back to the structured row for link expansion. Call once per session (and after a hot reload) to keep lore embeddings in sync with the definition. No-op when the embedding runtime is unavailable.
axiom.retrieval.fusion¶
Rank fusion for hybrid search.
Algorithm adapted from Hindsight (MIT, engine/search/fusion.py), kept
dependency-free and deterministic. We fuse ranked lists of document ids (one
per retrieval arm — semantic, lexical, …) into a single ranking using
Reciprocal Rank Fusion (RRF).
RRF is rank-based, not score-based, on purpose: each arm produces scores on its own incomparable scale (cosine distance vs BM25), so combining raw scores would let one arm’s scale dominate. RRF only looks at where a document ranks within each arm, which makes the arms commensurable.
- axiom.retrieval.fusion.cap_per_source(ranked_ids, cap)[source]¶
Truncate a single arm’s ranked ids to its top
cap.Applied per arm before fusion so one over-expanding backend cannot crowd out the others.
cap <= 0disables the cap. The caller is responsible for orderingranked_idsbest-first; this only slices.
- axiom.retrieval.fusion.reciprocal_rank_fusion(ranked_lists, k=60)[source]¶
Merge several ranked id lists into one via Reciprocal Rank Fusion.
RRF formula:
score(d) = sum over arms of 1 / (k + rank(d))whererankis 1-based within each arm. A document absent from an arm contributes nothing for that arm.- Parameters:
- Returns:
(doc_id, rrf_score)pairs sorted by score descending. Ties are broken deterministically by first appearance order across the input lists, so the result is stable for identical inputs.- Return type:
axiom.retrieval.lexical¶
Lexical (BM25) retrieval arm.
The semantic arm (sentence-transformer embeddings) is great at meaning but can
miss an exact token the player typed — a proper noun (“Kael”), an item name,
a coined word — because such tokens carry little distributed meaning. BM25 is
the classic lexical complement: it rewards rare exact-term overlap. We fuse the
two arms with Reciprocal Rank Fusion (see fusion.py).
Graceful degradation: if rank_bm25 is unavailable the lexical arm is simply
absent and search falls back to the semantic arm alone (no crash), mirroring the
VectorMemory._disabled philosophy for the embedding runtime.
Algorithm reference: Hindsight (MIT) runs BM25 in Postgres (tsvector); that
SQL is not portable, so we reimplement the idea with the pure-Python
rank_bm25 library over the in-memory corpus of a save’s chunks.
- axiom.retrieval.lexical.is_available()[source]¶
True when the BM25 backend can be used.
- Return type:
- axiom.retrieval.lexical.tokenize(text)[source]¶
Deterministic, Unicode-aware word tokenizer (lowercased).
Splits on anything that is not a word character, so punctuation and underscores act as separators. Keeps accented letters and non-Latin scripts (the game ships 10 languages), so a name like
"Kaël"stays one token.
- axiom.retrieval.lexical.build_bm25(corpus_texts)[source]¶
Build a BM25 index over
corpus_texts(orNoneif unavailable).Split out from
rank_by_bm25()so callers can cache the index when the corpus is unchanged (building it tokenises the whole corpus and computes IDF — the expensive part), then score many queries against it cheaply.
- axiom.retrieval.lexical.rank_with_bm25(bm25, query_text, corpus_ids)[source]¶
Rank
corpus_idsagainst a pre-builtbm25index, best first.corpus_idsmust be aligned 1:1 with the texts the index was built from. Returns an empty list when the index isNoneor the query has no usable tokens; drops zero-score docs (no lexical signal). Ties keep input order.
- axiom.retrieval.lexical.rank_by_bm25(query_text, corpus_ids, corpus_texts)[source]¶
Rank
corpus_idsby BM25 relevance toquery_text, best first.Convenience wrapper that builds a one-shot index then scores. Hot paths that reuse a stable corpus should cache
build_bm25()and callrank_with_bm25()instead.- Parameters:
- Returns:
corpus_idsreordered by descending BM25 score, dropping documents whose score is zero (no query-term overlap → no lexical signal). Returns an empty list when BM25 is unavailable or the query has no usable tokens. Ties keep the input order (deterministic).- Return type:
axiom.retrieval.reranker¶
Optional cross-encoder reranking.
The retrieval arms (semantic embeddings + BM25) score a document independently of the query’s other terms. A cross-encoder instead reads the (query, document) pair jointly and judges their relevance directly — the most accurate ranking signal available, used as the final re-sort after fusion.
It is OFF by default and degrades to a no-op when unavailable:
needs a torch model (~90 MB,
cross-encoder/ms-marco-MiniLM-L-6-v2), the same native runtime that fails on Windows without VC++ (TICKET-070);the heavy import / model load is lazy and guarded, so a missing or broken runtime simply means “no reranking”, never a crash.
Algorithm reference: Hindsight (MIT, engine/search/reranking.py). Local
sentence-transformers cross-encoders return logits → squashed to [0, 1] with a
sigmoid; already-calibrated [0, 1] scores are passed through.
- class axiom.retrieval.reranker.CrossEncoderReranker(model_name='cross-encoder/ms-marco-MiniLM-L-6-v2', model=None)[source]¶
Lazy wrapper around a sentence-transformers
CrossEncoder.- Parameters:
axiom.facts¶
Structured-fact storage for “living” memory mode.
In living mode the engine distils each turn’s narrative into atomic facts
(a who/what/when/where/why model adapted from Hindsight) and stores them here,
tagged with the turn_id that produced them. The turn tag makes rollback
trivial — rewinding to turn N simply drops every fact from a later turn — and
keeps the facts in lockstep with the event log they were derived from.
This module is the deterministic storage layer: no LLM, no network. The LLM
extraction that produces the facts lives in axiom.factextract; the background
job that calls it lives in the app’s worker layer. Facts live in the same SQLite
database as Event_Log / State_Cache (keyed by save_id + turn_id).
- class axiom.facts.Fact(statement, fact_type='world', who='', what='', when='', where='', why='', entities=<factory>, turn_id=None, fact_id=None)[source]¶
One atomic fact extracted from the narrative.
statementis the canonical one-sentence form used for recall/embedding; the who/what/when/where/why fields are the structured decomposition.
- axiom.facts.insert_facts(db_path, save_id, turn_id, facts)[source]¶
Persist a turn’s extracted facts. Returns the new
fact_idvalues.Empty statements are skipped (an extractor that found nothing is normal and must not write blank rows). Idempotency is the caller’s concern: re-extracting a turn should first
rollback_factsto that turn.Side effect: each
Factthat is actually inserted has itsfact_idandturn_idset in place, so the caller can use the objects directly without re-aligning a separate id list (skipped/blank facts keepfact_id=None).
- axiom.facts.get_facts(db_path, save_id, *, max_turn_id=None, entity=None, limit=None)[source]¶
Fetch a save’s facts, most recent first.
- Parameters:
max_turn_id (int | None) – Only facts from turns
<= max_turn_id(honours the history window / rewind horizon).None= no upper bound.entity (str | None) – Keep only facts whose
entitieslist contains this name (case-insensitive exact match).None= no filter.limit (int | None) – Cap the number of rows returned.
db_path (str)
save_id (str)
- Return type:
- axiom.facts.rollback_facts(db_path, save_id, target_turn_id)[source]¶
Delete a save’s facts from turns after
target_turn_id. Returns the count.Standalone helper (opens its own connection). The in-session rewind path deletes facts inside
CheckpointManager.rewind’s own transaction instead, so events and facts roll back atomically.
axiom.factextract¶
LLM fact extraction for living memory mode.
Turns a slice of narrative prose into a list of atomic, structured
Fact objects via an LLMBackend.
The who/what/when/where/why schema and the “atomic, verifiable, no speculation”
discipline are adapted from Hindsight’s retain/fact_extraction.py prompt,
reimplemented for our backend (causal links deliberately left out — see Phase 2
DOC). Causal relations / consolidation come later.
Design rules:
Background only: callers run this off the turn loop; it must never block play.
Graceful: any failure (LLM down, bad JSON, cancellation) yields
[]— the game keeps running, the turn simply produced no facts.No persistence here: returns
Factobjects; the worker stores them viaaxiom.facts.insert_facts(tagged with the turn id).
- axiom.factextract.extract_facts(llm, narrative_text, *, known_entities=None, when_hint=None, max_facts=8)[source]¶
Extract structured facts from
narrative_textusingllm.Returns
[]for empty input or on any backend/parse failure (never raises), so a living-mode background job can call it fire-and-forget.
axiom.observations¶
Consolidated-belief storage for “living” memory mode (Phase 3).
Where axiom.facts stores the atomic, immutable facts extracted each turn,
this module stores observations — synthetic beliefs that evolve as facts
accumulate (a Hindsight-inspired idea: an NPC remembers a betrayal hundreds of
turns later and revises its opinion). A belief carries:
statement— the canonical belief, one sentence;subject— the entity it is about / who holds it (”” = the world);sources— the supporting facts as[{"fact_id", "turn_id"}];proof_count— cachedlen(sources);history— JSON trail of CREATE/UPDATE/DELETE changes.
This is the deterministic storage + rollback layer (no LLM, no network). The
LLM consolidation that decides CREATE/UPDATE/DELETE lives in
axiom.consolidate; the background job that calls it lives in the app layer.
Rollback (the hard part, solved). Beliefs derive from several turns, so a
plain turn_id column is not enough. The sources turn ids are the rollback
key: rewinding to turn N drops every belief created after N and, for the
survivors, keeps only the sources at turns <= N, recomputing proof_count
and flagging stale so the next consolidation pass re-examines them. Beliefs
thus roll back atomically with the facts/events they were built from.
- axiom.observations.compute_trend(source_turns, now_turn, *, recent_turns=15, old_turns=45)[source]¶
Classify a belief’s trend from the turn ids of its supporting sources.
Returns one of the
TREND_*constants:NEW— every source falls in the recent window;STRENGTHENING— denser recent evidence than older (ratio > 1.5);WEAKENING— sparser recent evidence than older (ratio < 0.5);STALE— no source in the recent window (may be outdated);STABLE— steady, or trend unknown (no sources / no current turn).
Deterministic and side-effect free.
now_turnis the current turn (the rewind horizon during replay), so the trend is always read at the right “now”.
- class axiom.observations.Observation(statement, subject='', proof_count=1, sources=<factory>, history=<factory>, created_turn_id=0, updated_turn_id=0, stale=False, observation_id=None)[source]¶
One consolidated belief.
sourcesis a list of{"fact_id": int, "turn_id": int}dicts — the facts backing the belief and the turns they came from (the rollback key).- Parameters:
- trend(now_turn)[source]¶
This belief’s trend at
now_turn(seecompute_trend()).
- axiom.observations.insert_observation(db_path, save_id, obs)[source]¶
Persist a single new belief. Returns its
observation_id(orNone).Blank statements are skipped (an empty belief is never written).
proof_countis derived fromsourcesso it can never disagree with them.- Parameters:
db_path (str)
save_id (str)
obs (Observation)
- Return type:
int | None
- axiom.observations.get_observations(db_path, save_id, *, max_turn_id=None, subject=None, limit=None)[source]¶
Fetch a save’s beliefs, most recently updated first.
- Parameters:
max_turn_id (int | None) – Only beliefs created at turns
<= max_turn_id(honours the history window / rewind horizon).None= no bound.subject (str | None) – Keep only beliefs whose
subjectmatches (case-insensitive).None= no filter;""keeps the world-level beliefs.limit (int | None) – Cap the number of rows returned.
db_path (str)
save_id (str)
- Return type:
- axiom.observations.count_observations(db_path, save_id)[source]¶
Number of stored beliefs for a save (cheap COUNT).
- axiom.observations.rollback_observations(conn, save_id, target_turn_id)[source]¶
Roll a save’s beliefs back to their state at
target_turn_id.Operates on an already-open connection so
CheckpointManager.rewindcan run it inside the same transaction as the Event_Log / Facts deletes (atomic).- Rule (see module docstring):
belief
created_turn_id > target→ it did not exist yet → DELETE;else keep its sources at turns
<= target; if any were dropped (an UPDATE absorbed a now-rewound fact) recomputeproof_count, clampupdated_turn_idto<= targetand flagstalefor the next consolidation pass.
Returns
{"deleted": n, "updated": m}.
- axiom.observations.apply_consolidation(db_path, save_id, turn_id, actions, fact_turn_map)[source]¶
Apply consolidator actions (CREATE/UPDATE/DELETE) to the beliefs store.
Deterministic: it just executes the decisions the LLM already made (the LLM call lives in
axiom.consolidate).fact_turn_mapmaps each citedfact_idto itsturn_idso new sources carry the turn (the rollback key). Unknown belief ids / empty statements are skipped. Returns counts.
- axiom.observations.rollback_observations_standalone(db_path, save_id, target_turn_id)[source]¶
Standalone variant of
rollback_observations()(opens its own conn).The in-session rewind path uses the connection-based variant so events, facts and beliefs roll back in one transaction; this helper is for tests and out-of-band cleanup.
axiom.consolidate¶
LLM belief consolidation for living memory mode (Phase 3).
Turns a batch of freshly-extracted Fact objects, together
with the beliefs already held, into a list of consolidation actions
(CREATE / UPDATE / DELETE Observation). The decision
rules — prefer UPDATE over duplicate CREATE, one facet per belief, be
conservative about DELETE, never do arithmetic, preserve what changed — are
adapted from Hindsight’s consolidation/prompts.py, reimplemented on our
LLMBackend.
Design rules (same discipline as axiom.factextract):
Background only: callers run this off the turn loop; it must never block.
Graceful: any failure (LLM down, bad JSON) yields
[]— no belief changes this pass, the game keeps running.No persistence here: returns action objects; the deterministic application to the DB lives in
axiom.observations.apply_consolidation.
- class axiom.consolidate.ConsolidationAction(kind, statement='', subject='', observation_id=None, source_fact_ids=<factory>)[source]¶
One belief change proposed by the consolidator LLM.
- axiom.consolidate.consolidate(llm, new_facts, existing, *, mission=None, missions=None, max_existing=24)[source]¶
Ask
llmhow the beliefs should change givennew_facts.- Parameters:
mission (str | None) – The universe-wide default mission (what this world remembers).
missions (dict[str, str] | None) – Per-character memory styles
{entity_name: mission}(B-3) — only those whose character appears in this batch are shown.max_existing (int) – Cap on existing beliefs shown to the LLM (TICKET-077); scoped to the batch’s characters + most recent.
<= 0disables.llm (LLMBackend)
existing (list[Observation])
- Return type:
Returns
[]for empty input or on any backend/parse failure (never raises), so a living-mode background job can call it fire-and-forget.
axiom.missions¶
Per-character belief missions (Phase 4, B-3).
A belief mission is a memory style for the consolidator: what a character
tends to remember and dwell on (a rancorous NPC remembers betrayals, a merchant
remembers transactions, a loyal guard remembers favours). It biases which
beliefs axiom.consolidate forms about each subject, so NPCs “remember
differently” according to their nature.
Stored in Universe_Meta (no schema change; it round-trips losslessly through
the Universe-as-Code [extra] mechanism, is copied into saves, and is
packaged):
belief_mission— the universe-wide default mission (one string);belief_missions— JSON{entity_name: mission}per-character overrides.
Keyed by entity name because a belief’s subject is a name and authoring
“Name: mission” is natural. Reading degrades gracefully (missing/blank/malformed
→ sensible empties) so the consolidator always has something usable.
- axiom.missions.get_universe_mission(db_path)[source]¶
The universe-wide default belief mission (”” when unset).
- axiom.missions.get_belief_missions_from_value(raw)[source]¶
Parse a
belief_missionsJSON value into{entity_name: mission}.Tolerates blank/malformed JSON or non-string values — anything unusable is dropped rather than raised. Used when the meta value is already in hand (e.g. the Studio’s loaded meta dict).
- axiom.missions.get_belief_missions(db_path)[source]¶
Per-character missions as
{entity_name: mission}read from the DB.
- axiom.missions.parse_missions_text(text)[source]¶
Parse a ‘Name: mission’ per-line block into
{name: mission}.The GUI Metadata field stores missions this way (one entity per line). Lines without a colon, or with an empty name/mission, are skipped. The first colon splits — a mission may itself contain colons.
axiom.mental_models¶
Mental-model storage for “living” memory mode (Hindsight §7.8).
A mental model is a curated, synthetic profile — one per subject (a character,
or "" for the world) — that sits above the beliefs in the recall hierarchy:
mental model → beliefs (observations) → facts → raw narrative chunks (most synthetic) (most raw)
Where a belief is one durable statement, a mental model is a short paragraph that distils all of a subject’s beliefs into “who this is now”: their relationships, goals, grudges and how they have changed. The narrator reads it first, so a long campaign’s accumulated memory lands as a coherent character note rather than a pile of disjointed facts.
Idea adapted from Hindsight (MIT, reflect/): their reflect agent maintains
curated mental models above the raw memories. We keep the principle of a
synthetic top layer but drop the heavy tool-calling agent — for a single-player
game one regenerated paragraph per subject is enough.
This module is the deterministic storage + rollback layer (no LLM, no network).
The LLM that writes the summary lives in axiom.reflect; the background job
that calls it lives in the app’s worker layer.
Rollback. A mental model is fully reconstructible from the beliefs it was
built from (which themselves roll back correctly), so rewinding to turn N only has
to: drop every model created after N, and flag the survivors stale (clamping
updated_turn_id) so the next refresh regenerates them from the rewound beliefs.
There is at most one model per (save_id, subject) — refresh is an UPSERT.
- class axiom.mental_models.MentalModel(subject, summary, sources=<factory>, created_turn_id=0, updated_turn_id=0, stale=False, model_id=None)[source]¶
One curated profile for a subject (a character, or “” for the world).
sourcesis a list of theobservation_idvalues the summary was built from (kept for traceability / future link expansion; the rollback key is the turn ids, which the beliefs themselves carry).
- axiom.mental_models.upsert_mental_model(db_path, save_id, subject, summary, turn_id, sources=None)[source]¶
Create or refresh the mental model for
subject. Returns itsmodel_id.There is at most one model per
(save_id, subject): an existing one is updated in place (created_turn_idpreserved), otherwise a new one is inserted. Blank summaries are skipped (never overwrite a profile with nothing).
- axiom.mental_models.get_mental_models(db_path, save_id, *, max_turn_id=None, subject=None, limit=None)[source]¶
Fetch a save’s mental models, most recently updated first.
- Parameters:
max_turn_id (int | None) – Only models created at turns
<= max_turn_id(honours the rewind horizon).None= no bound.subject (str | None) – Keep only the model for this subject (case-insensitive).
""keeps the world-level model.limit (int | None) – Cap the number of rows returned.
db_path (str)
save_id (str)
- Return type:
- axiom.mental_models.count_mental_models(db_path, save_id)[source]¶
Number of stored mental models for a save (cheap COUNT).
- axiom.mental_models.stale_subjects(db_path, save_id, *, max_turn_id=None, limit=5)[source]¶
Subjects whose model is flagged
stale(oldest update first).A rewind flags survivors stale; this lets the refresh job find and regenerate them even when their beliefs do not change again. Capped so a long backlog never floods one consolidation pass.
- axiom.mental_models.rollback_mental_models(conn, save_id, target_turn_id)[source]¶
Roll a save’s mental models back to their state at
target_turn_id.Operates on an already-open connection so
CheckpointManager.rewindcan run it inside the same transaction as the Event_Log / Facts / Observations deletes.- Rule (see module docstring):
model
created_turn_id > target→ did not exist yet → DELETE;else, if it was last refreshed after the target, clamp
updated_turn_idtotargetand flagstaleso the next refresh regenerates it from the rewound beliefs (the old summary stays as a graceful fallback meanwhile).
Returns
{"deleted": n, "updated": m}.
axiom.reflect¶
LLM generation of mental models for living memory mode (Hindsight §7.8).
Turns the beliefs held about one subject (a character, or the world) into a short
mental model: a 2-4 sentence profile of who that subject is now. The decision
discipline — base it only on the beliefs, no invention, no arithmetic, capture
change and relationships — is adapted from Hindsight’s reflect/ prompts,
reimplemented on our LLMBackend.
Design rules (same as axiom.factextract / axiom.consolidate):
Background only: callers run this off the turn loop; it must never block play.
Graceful: any failure (LLM down, empty answer) yields
""— the profile is simply not refreshed this pass, the game keeps running.No persistence here: returns the summary string; the deterministic UPSERT lives in
axiom.mental_models.upsert_mental_model().
- axiom.reflect.affected_subjects(actions)[source]¶
Subjects whose beliefs changed in a consolidation batch (order-preserving).
Reads the
subjectof each create/update action (delete actions drop a belief and don’t name a subject worth re-modelling here). De-duplicated case-insensitively, keeping first-seen spelling. World beliefs (subject="") are included (the world model).
- axiom.reflect.reflect(llm, subject, beliefs, *, mission=None)[source]¶
Write a mental-model summary for
subjectfrom itsbeliefs.Returns
""when there are too few beliefs to be worth modelling, on empty input, or on any backend failure (never raises) — so a living-mode background job can call it fire-and-forget.- Parameters:
llm (LLMBackend)
subject (str)
beliefs (list[Observation])
mission (str | None)
- Return type: