Observations and Formation

Observations preserve what arrived

An observation is the record of a captured report. Its original UTF-8 content is stored byte for byte and is never rewritten during later processing. Capture also records the authenticated caller and receipt time. A caller may supply labels such as a speaker name or conversational context, but those labels are kept as declarations: they do not establish a person’s identity, authority, or independent evidence source.

This makes the observation the durable answer to “what did the system receive?” Later extraction can be corrected or repeated without rewriting that answer. If an event identifier is supplied, retrying the same event for the same caller returns the original observation; reusing that identifier with conflicting content or metadata is rejected.

Evidence identity and independence

Evidence groups help the system avoid treating repeated copies as independent confirmation. A report captured under a stored user belongs to that user’s group. Reports without a user share an unattributed group. A declared external persistent origin can define an origin group, and a forwarded or copied report can retain a lineage link to its origin. Multiple origins declared on one report still form one joined group; they do not multiply the report’s weight.

Names and roles in message text do not create evidence groups. “The doctor said…” is a statement about a source, not authenticated proof that the doctor made it. A copy of another report also does not become a new independent source just because it arrived through another conversation.

The design segments observations deterministically using a versioned segmenter. The same bytes and segmenter version produce the same episode segments. These segments can support episodic lookup, while the original observation remains the source record.

Formation creates candidates, not verdicts

Formation is the work that interprets observations and proposes focused memories or operations on existing ones. The proposer receives exact input bytes, known candidate versions, and bounded context. It can return descriptions of claims and operations; it cannot set the final support grade, evidence identity, or belief status.

The acceptance boundary resolves those descriptions against canonical records and fixed rules. It checks that references point to versions the proposer was allowed to see, that citations identify exact spans in supplied evidence, and that the operation is valid for the target. An invalid operation is refused without preventing independent valid operations in the same response from being recorded. Each formation decision is durable and identifies its inputs, processing generation, proposer, accepted operations, and refusals.

Exact citations improve traceability, but they do not prove that the cited words logically entail a model’s description. The model may still misread a span. The system therefore treats extraction as a proposal and applies independent acceptance rules to the proposal’s meaning and structure. Model inference without a registered rule remains a hypothesis until supported by eligible evidence.

Structured statements

Some inputs can already be represented as explicit structured statements. In that case formation can preserve the supplied claim, scope, time, and cited span exactly instead of asking a model to paraphrase it. Structure does not grant the statement authority: it still passes through the same acceptance rules. A question, unresolved or hedged statement, or unregistered inference does not become a supported claim merely because it arrived in a structured field.

A statement that confirms or refines an existing memory must identify the exact version it targets. A missing target refuses that statement; it does not silently attach to whichever version happens to be current later. Structured formation cannot retract evidence. Retraction is an explicit operation with its own provenance and validation rules.

Durable background processing

Capture records the work it owes in the same transaction as the observation, so a process crash cannot leave a saved observation with its formation request forgotten. Work is coalesced into a per-tenant drain for each formation type. Drains are bounded by item, byte, and time limits and commit at most one decision for a logical work identity. Retrying after a crash therefore does not create a second conflicting decision.

Status keeps separate milestones separate:

• Captured: the observation exists.
• Formation outcome: processing is pending, running, complete, or failed; completion reports accepted and refused counts, including a valid result with zero accepted statements.
• Index readiness: the material is available to particular search paths.
• Revision completion: labels and current projections have caught up.

One milestone does not imply another. In particular, “processed” does not mean that a claim was accepted, supported, or published as current. Formation status reports identifiers and outcomes rather than echoing observation content.

Retention boundary

Observations and focused memories have different lifetimes. Raw observation retention is configurable at deployment and may be shortened by tenant policy; an observation’s deadline cannot be extended by a later retry or policy change. Focused memories and the evidence needed to justify them have no age limit by default. The design can retain exact excerpts after raw observation bytes expire, with each excerpt linked to its source and checked against the original span while that source is available.

See Revision and Forgetting for how raw expiry differs from support withdrawal, archiving, and erasure.