From a question to a checked explanation

1. Overview

Questions and answers are processed on your device. The website still downloads page assets and a quantized MiniLM sentence-embedding model of about 22 MB, which loads in the background when a chat opens and is cached afterwards. No language-generation API is used.

1. Start: load the selected knowledge base and build its keyword index, word statistics and spelling index.
2. Understand: split the message into questions, recognise what each asks for, and resolve references to earlier turns.
3. Retrieve and ground: combine name matches, BM25 scores and available vector similarities, then check how much of the question a topic really explains.
4. Decide: answer, give the closest match, quote a passage, send you to another assistant, or say the notes do not cover it; the 13,079-parameter policy advises intent and depth in verified WebAssembly or JavaScript.
5. Explain: plan the answer from curated fragments and render it once, with mathematical notation and available source links.

The optional embedding upgrade prepares every vector before switching the active backend. Cached query vectors are cleared at the switch. Public knowledge embeddings are cached in IndexedDB under a hash of the model, knowledge content, and intent examples. Conversations remain in memory unless you save a text copy.

Browse all six chatbots · Inspect the measured policy results

2. Understanding questions and the knowledge boundary

Before the policy runs, a deterministic inference layer reads each message. It is plain JavaScript shared by all six assistants (core/responder.js and the modules it imports), needs no download, and answers in a few milliseconds. It combines techniques that usually live outside chatbots:

1. Clauses and question types. A finite-state cascade splits a message into up to three questions (“what is X and how does it relate to Y?”) while keeping “the difference between X and Y” whole. Each clause is labelled as a definition, example, formula, use, comparison, relation, why, how, who, when, where, how-many, yes/no or condition question. Requests for live data, calculations, code, opinions or actions are recognised separately.
2. Conversation state. A discrete Bayes filter, the forward step of a hidden Markov model used for robot localisation, keeps a probability for every topic. Topics persist between turns, drift to related topics, fade with idle time, and are updated by what you name and by what the previous answer presented. “He” resolves only to people, “it” only to concepts, “the second one” to the order in which topics were offered, and “its memory” to the related topic named “memory”. A scalar Kalman filter remembers whether you asked for short or detailed answers and lets that preference fade.
3. Grounding. Topic names and aliases are matched over the word sequence, tolerating plurals, compounds written apart or together, and one-letter typos found with a symmetric-delete spelling index. Each word is weighted by how informative it is in that knowledge base. The layer measures how much of the question each candidate topic explains, uses the conversation’s topic belief as a prior, and grounds every coordinated phrase (“X, and also Y”) separately.
4. The knowledge boundary. A cascade of checks chooses between answering, answering as the closest match, quoting where an uncovered term is mentioned, sending you to the assistant that covers it, and saying plainly that the notes do not cover it. “I don’t have notes on X” requires specific evidence: an unknown word in a short concept question, a two-word term whose words never occur together in the notes, or a modifier that turns a known topic into another concept, such as “quantum game theory”. Topics asked for by name are never refused. Once the neural model has loaded, a description that words alone cannot place, but whose meaning is clearly closest to one topic, is answered as the closest match.
5. Answer planning. Specific questions are answered first with the sentence that carries the requested information, such as a date for “when” or an attribution for “who”. If no such sentence exists, the reply says so instead of guessing. Otherwise a beam search over the topic’s curated fragments chooses an order that fits the question, reads coherently (definition, intuition, example) and avoids repeating what this conversation already showed.

The trained policy network still advises intent and depth. The boundary thresholds were checked with split-conformal calibration on 1,489 questions generated from the knowledge bases themselves, including one-letter typos and descriptions without the topic’s name; none was refused. On a final question set written by a separate model after the layer was frozen, and scored once, the previous pipeline refused 29.2% of questions about covered topics and the new layer 4.2%. All of those refusals came from short follow-ups such as “Convince me.” being judged as separate questions, which has since been fixed. The same blind run measured 86.1% correct topics (previously 52.8%), 87.5% honest handling of uncovered concepts (52.1%), 91.7% routing to the right assistant (0%) and 91.7% complete multi-part answers (16.7%). Read the dialogue evaluation.

3. Signal Layer

After embedding, a lightweight signal layer prepares a structured DecisionPacket for the policy network. It combines multiple retrieval and classification signals into a coherent pre-policy feature bundle.

1. BM25 Sparse Retrieval — Field-weighted term-frequency scoring (k1=1.5, b=0.75). Entry names are repeated 3× and aliases 2× during indexing, so name-matched terms naturally score higher via TF boosting. Bigram phrase matching catches multi-word queries like "Nash equilibrium" as a single token (nash_equilibrium), with an IDF-weighted bonus that complements unigram scoring.
2. Entity Extraction — Three-pass extraction: (1) exact alias regex matching, (2) fuzzy word-overlap scoring using Levenshtein distance and substring containment for typo tolerance, (3) notation pattern matching for game-theory expressions like (D,D) and (C,C). Session context enriches entities from recent turns with decay weighting.
3. Intent Classification — Cosine similarity against 19 intent prototypes per category (definition, example, formal, application, comparison), scored with a 70/30 blend of best-match and weighted-average to prevent one lucky prototype from dominating. Calibrated with temperature=1.5 softmax for reliable confidence estimates.
4. Ensemble Ranking — Dense cosine similarity (0.7 weight) and BM25 scores (0.3 weight) are fused into a combined ranking. A neural reranking pass applies a token-overlap bonus to refine the top results. Follow-up queries get a 0.35 boost (scaled by conversation depth) to the previous topic, preventing topic drift.
5. Feature Extraction — The ensemble ranking, calibrated intent scores, entity data, and session context are compiled into the 25-feature vector that feeds the policy network.

Topic Correction — The signal layer detects explicit topic corrections (e.g., "I meant X" or "no, just X") via regex pattern matching against the current query and session history. When a correction is detected, the corrected topic embedding is forced to the top of the dense-sparse ranking, overriding the ensemble score. This prevents topic drift when the user redirects the conversation mid-turn.

The signal layer is stateless and runs entirely in the browser — no server calls, no external inference APIs. The resulting DecisionPacket contains the query embedding, entity list, calibrated intent distribution, dense and sparse rankings, confidence metrics, and session context.

4. Feature Extraction

Every query produces a 25-feature vector that feeds the policy network. These features capture similarity, entity presence, intent distribution, session history, and fragment metadata.

25-Feature Layout

5. Policy Network

The policy uses 25 runtime features, two linear layers with ReLU activations, and six output heads. There is no LayerNorm in either training or inference. Training-time feature scaling is folded into the first layer on export.

Input: 25 raw features
Hidden: 25 → 128 ReLU → 64 ReLU
Heads: mode (5), intent (5), topic count (4), fragment count (4), creativity (1), tone (4)
Total: 13,079 parameters
WASM: 3,685 bytes; fixed 128 KiB memory; no host imports
Release: 2.0.0 — September 9, 2026

JavaScript inference is ready when weights load. The runtime verifies the optional WASM hash before attaching it to the same MLP. The WASM kernel produces logits; JavaScript validates the final answer plan. Failed downloads or incompatible schemas keep the basic heuristic available. Explicit greeting and help commands do not get converted into unrelated knowledge topics.

6. Training and evaluation

The CPU-only NumPy trainer uses authored examples and the actual JavaScript feature extractor. One question is one episode: this is a contextual-bandit routing task, not long-horizon conversation training. A supervised warm start is followed by on-policy REINFORCE with an action-independent reward baseline.

The September run used 1,830 training, 420 validation, and 438 held-out test cases. Topics are separated across all splits; training templates differ from the validation/test templates. Joint mode-and-intent accuracy was 93.6% for the old model, 98.6% for the supervised checkpoint, and 98.6% after REINFORCE. The complete trainer run, including export and evaluation, took 3.10 seconds on the development laptop.

These measurements describe routing on authored cases. They do not establish factual accuracy, learning outcomes, or performance on every device. A wall-clock deadline, release gate, and numerical parity checks bound the training and export process. Read the reproducible training guide.

7. Fragment-Based Response Composition

Each knowledge-base entry contains categorized fragments (def, int, ex, form, app) with metadata fields: truth_confidence, source_confidence, difficulty, style, avoid_with.

The policy produces an AnswerPlan specifying:

• Mode — normal, comparison, greeting, help, off_topic
• Intent — definition, example, formal, application, comparison
• Topics — which KB entries to include
• Fragment Plan — which categories and indices per topic
• Template — opener/closer indices, comparison opener key, connector keys
• Creativity — scalar [0,1] controlling response variation
• Tone — neutral, formal, intuitive, playful

composeV2() in core/nlp.js reads the AnswerPlan and assembles the final text by selecting fragments, applying linguistic connectors ("For instance,", "More formally,", etc.), prefixed by openers and suffixed by closers — all indexed from the plan with modulo-safety.

Immediate answer rendering

Responses are composed locally and inserted once. There is no artificial typing delay or repeated partial-HTML update. The chat log announces completed messages, and KaTeX renders mathematical notation after insertion.

Session Memory

The SessionMemory class (core/session.js) tracks up to 30 turns with importance-based eviction (recent 5 turns are always protected). Old responses are compressed to 120-char summaries after 5 turns. An EMA summary vector (exponential moving average, α=0.75) of query embeddings provides dense multi-turn context to the policy without additional feature extraction. Entity mentions decay with a half-life of 5 turns, and fragment diversity penalties prevent repetitive responses across long conversations.

Comparison Mode

When mode === 'comparison', the policy selects a comparisonOpenerKey (both, contrast, or similarity) from the template. The renderer uses patterned openers like "Both A and B are important concepts here." and distributes categories across multiple topics.

8. Action Schema & Validation

Every AnswerPlan passes through validatePlan() (policy/action-schema.js) which enforces:

• Type checking on all 10 top-level fields
• Enum membership (mode, intent, tone)
• Range validation (creativity ∈ [0,1], topics ≤ maxTopics)
• Cross-field consistency (fragmentPlan length matches topics; comparison with <2 topics falls back to normal)
• Sanitization with schema defaults for missing/invalid fields

9. Feature Serialization

For the WASM boundary, features are packed into a 107-byte buffer:

packFeatures(features) → {
  float32: Float32Array(25),     // offset 0,  100 bytes
  uint8:   Uint8Array(7),       // offset 100, 7 bytes
  buffer:  ArrayBuffer(107)      // total
}

Uint8Array layout:
  [0] = entityCount         (u8, 0-3)
  [1] = packed booleans     (bits: entityBoostHit|hasComparisonCue|hasFormalCue|hasExampleCue|wasAmbiguous)
  [2] = lastTopicAge        (u8, 0-8)
  [3] = queryLenTokens      (u8, 1-32)
  [4] = followUpType        (u8, 0-22)
  [5] = minDifficulty       (u8, 0-4)
  [6] = fragDiversity       (u8, 0-5)

10. Heuristic Fallback

When the MLP policy engine is unavailable (e.g., during cold start or weight load failure), planAnswerHeuristic() generates the same AnswerPlan structure using 15 parameterized decision thresholds covering greeting detection, off-topic handling, comparison fallback, entity boost, and creativity defaults. This keeps rule-based response planning available when policy weights fail to load; other required site assets still need to be available.

11. Open Source

The full codebase is available at github.com/yunusemrejr/relu-chat under the MIT license. This includes:

• core/ — NLP engine, chatbot engine, session memory, BM25 scorer, signal layer
• policy/ — Feature extractor, MLP inference, action schema, policy runtime
• dev/scripts/ — PyTorch training, weight export, prompt augmentation

Premium · From the makers of ReLU.chat

Build on the methods in this reference.

The On-Device Chatbot Builder Toolkit turns the architecture above into a buildable system for your own domain: frameworks, templates, and evaluation guides for every stage — signals, retrieval, evidence, decision policy, composition, validation. Ship a privacy-first chatbot that runs entirely on your users' devices.

Input → Signals → Retrieval → Evidence → Decision → Composition → Validation → Response View toolkit on Gumroad →