EEG/fMRI as a Human-Grounding Reward Signal

v2026.07.1 reviewed by Torsten 2026-07-18, as good as this can be at this stage. Includes the "beyond EEG" speculative survey (originally parked as sail-4nwc). Not yet sent to Teon. Tracked as sail-zplm.

Honest framing, up front

This angle wasn’t part of the July synthesis run that produced everything else in this compendium. Hummel’s own KPI-realism review dropped EEG from the mini-study’s KPI line, and the bootstrap plan put passive BCI off the critical path — not because the idea is wrong, but because it’s hardware- and IRB-heavy in a way nothing else here currently is, and this whole thread is deliberately staying zero-cost and fast-moving. Worth saying plainly rather than overstating current involvement: nothing in this chapter is running. It’s a candidate direction, grounded in real, already-built work — this collaborator’s own — not a live component of sail-judge.

What already exists: BrainWaves

BrainWaves is a real, already-designed K-12 classroom EEG tool (design deck, November 2018, built around the EMOTIV EPOC-family headset — the device picker in the actual designs shows an EPOCPLUS-2378 pairing over USB) — not a sketch, a full four-stage application:

1. Review Design — pick a premade paradigm (a “Faces/Houses” experiment is the shipped example: two image categories, framed around why face perception matters for quickly reading trust and emotion) or build a Custom one. The custom builder is genuinely hypothesis-driven at the UI level: it has explicit Research Question, Hypothesis, and Methods fields before it lets you define stimuli, not just an image uploader. Two named stimulus groups, each with its own image folder and a mapped “correct” key response; trial timing set via ITI-duration and image-duration sliders (0.5–2s in the shown example) — a classic, discrete-trial two-alternative design, not a continuous-monitoring one.
2. Collect — headset pairing, then a live per-electrode scalp map color-coded strong/mediocre/weak/no-signal, next to the live raw trace. A built-in tutorial teaches signal-quality practice directly to whoever is running the session (saturate the sensors in saline, re-seat the headset with tension, sweep hair aside) and teaches noise recognition the same way (try closing your eyes, try clenching your jaw — does the signal change?) before a single real trial runs.
3. Clean — a dedicated editor to reject segments contaminated by exactly those noise sources, with its own short tutorial (“before you collected data… you learned how clenching one’s jaw or blinking can affect quality… now you need to reject segments that might be the result of one of these actions, not the stimulus”).
4. Analyze — event-related potential (ERP) output: per-condition averaged waveforms at each electrode, compared side by side (the shown example: Faces vs. Houses, 78 trials each, at a topographic electrode map).

This is real methods pedagogy embedded directly in a classroom tool, not just a wrapper around a headset SDK — worth naming as a serious, mature piece of design work in its own right, independent of whether it connects to anything SAIL is doing.

The honest gap: discrete trials vs. a live conversation

The concrete point that actually matters for SAITO: BrainWaves is built around stimulus-locked, repeated, discrete trials — a brief image shown for a fixed short duration, repeated dozens of times per condition, averaged afterward to pull a clean ERP out of noisy single-trial data (78 trials per condition in the shown example, and that’s for a simple two-category visual task with decades of established ERP literature behind it). It is not built for, and doesn’t obviously generalize to, continuous monitoring of an open, free-running, multi-turn Socratic conversation, where an asker or discourse-driver arm fires once, at an unpredictable moment, with no fixed short duration and no repeated identical stimulus to average against.

That gap is the actual research question here, not “can EEG detect something interesting” (it demonstrably can, per BrainWaves’ own ERP output) — it’s whether SAITO’s live interventions can be recast as something ERP-shaped at all. A candidate framing, not yet attempted: treat each moment a given arm fires (say, every generation-effect prompt across many conversations) as a stimulus onset, capture a fixed short window of EEG around it, and average across many firings of that same arm — literally reusing BrainWaves’ own Collect → Clean → Analyze pipeline, just swapping “Faces vs. Houses” for “generation-effect vs. retrieval-practice vs. …”. Whether that produces a clean, distinguishable ERP the way two visual categories do is a genuinely open empirical question, not a given — SAITO’s arms don’t share anything like the tight timing and stimulus-repetition structure a visual ERP paradigm relies on.

Worth being precise about what a positive result there would and wouldn’t give: even a clean, distinguishable per-arm ERP would only show that different arms produce distinguishable neural responses on average, not automatically hand back a usable live reward value for Thompson sampling. Averaging across dozens of trials happens offline, after the fact — this is exactly the same slow-signal/fast-runtime split already named for role-mining (sail-i9j3’s “we are building synthetic agent roles from real data,” and its SlowTheory/FastTheory/Distill morphism from the bandit-arms-gat chapter). If this is ever pursued, it’s worth framing from the start as another instance of that same pattern — a slow, real, offline-computed signal, distilled into whatever runs live — rather than assuming EEG could feed the bandit’s reward directly and in real time.

Why the fact-store is already shaped for this

sail-judge’s internal state was deliberately designed as a normalized fact store (subject/predicate/object/confidence/disputed/source_event) rather than a flat transcript, specifically so a future signal — EEG or otherwise — could be added later as another source-event type without redesigning anything. That’s a real, already-shipped design decision, not a promise. The door was left open on purpose; it’s just not open this month.

Nearby lab context

This collaborator’s own K-12 work at Gotham Data Clinic (BrainWaves, above) is a nearby, complementary lab context — a place already doing real EEG-in-education work, not itself SAIL’s infrastructure, and not something SAIL has any claim on.

The stretch idea: IOSP workshops

IOSP workshops (iosp.science) may already be silently applying Socratic ideas — multi-disciplinary researchers in the room is exactly the setting that could seize this kind of cross-collaborator connection in person, rather than it staying a compendium chapter. sail-i55b (the ATProto Bots hacking station, bandit.mjs+fact-store.mjs extracted to github.com/ATProto-Science/iosp-hacking-stations) is the concrete foothold already in place there.

Speculative: beyond EEG, what could actually work live?

Everything above establishes what EEG/ERP is good at (a clean, validated signal, per condition, averaged offline) and where it doesn’t fit (anything continuous and live). That still leaves the question genuinely open rather than closed: what, if anything, could work as a live, continuous human-grounding signal for a real Socratic conversation? None of the following is designed, built, or validated — offered as candidates for a real conversation, the same spirit as the three role hypotheses in the bandit-arms-gat chapter, not a plan.

Webcam-based, no new hardware beyond a camera:

• Eye-tracking — WebGazer.js (Brown University): pure JavaScript, runs entirely client-side, infers gaze point from an ordinary webcam, no video leaves the machine. Mature, actively maintained, privacy-friendly by construction.
• Pupillometry specifically — pupil dilation, distinct from gaze location, is a real, targeted marker of cognitive load/arousal in the literature; Jeeliz Pupillometry does this live, browser-only, off a webcam feed. Worth flagging as unusually well-aimed at a specific, already-named gap: sail-wqs3’s desirable-difficulty arm is the one construct with no honest live proxy at all, and pupillometry-as- cognitive-load is a plausible, specific candidate answer to that exact gap, not just generic extra signal.
• Facial affect/engagement recognition — an active research area literally aimed at detecting learner engagement/disengagement from webcam video in real time (OpenFace is the standard open-source toolkit; recent deep-learning classifiers target online-learning settings specifically). SAITO’s own question, being worked on under a different name elsewhere.
• rPPG (remote heart-rate/HRV from webcam skin-color changes) — real and published, but genuinely fragile: the literature itself flags unresolved questions about whether it captures moment-to-moment HRV reliably outside controlled lighting/motion conditions.

No new modality at all:

• Text-behavioral signals — sail-judge already runs over text chat. Response latency, typing duration, and message-length/complexity shifts are already implicit in the existing transcript. No camera, no new consent flow, no change to Chatto’s protocol.
• Multi-session accumulated trace data (STM) — the most already-real candidate of all: sail-3scs (deferred to Phase 3), already using the same facts-table substrate sail-judge runs on today, and literally the “trace-only” fallback the bootstrap plan named when EEG got parked in the first place. It tracks the same modality as the outcome itself (what a learner says, asks, and gets right over many sessions) rather than an indirect physiological correlate — a real advantage. But that closeness cuts both ways: it’s precisely what makes it tempting to treat “the trace pattern moved” as ground truth without checking, which is the same regressional-Goodhart risk flagged throughout this compendium, just harder to spot here since it doesn’t look like a proxy. It still needs the same external check — does the inferred trajectory actually correlate with held-out Zerbe-item performance, per the mini-study design (sail-1uq0) — before it’s trusted as a reward signal.

None of these dodge the validation crux this whole chapter turns on: checking any candidate against real learning gain is the actual work, and that’s learning science’s specialty (the testing-effect/retrieval- practice/desirable-difficulty literature’s whole methodology — pre- registered designs, delayed retention testing days-to-weeks later, real effect sizes — exactly what sail-1uq0’s mini-study already adopts). It’s not that neuroscience has no answer to the same question — there’s real research linking physiological signals to durable memory specifically (pupil response during encoding predicting trial-by-trial long-term memory formation is a genuine, published finding) — but that literature itself reports historically conflicting results and unresolved long-term predictive validity, nowhere near the unified, replicated status the testing-effect literature has, and nearly all of it is done in the same controlled, discrete-trial paradigm already flagged above — never yet bridged to anything like an open dialogue.

Open questions

• Which of the candidates above, if any, is worth prototyping first — and is any of it worth building before there’s a concrete reason (one of these collaborators wanting to try something, or Chatto growing video capability for an unrelated reason)?
• For the STM/trace-data candidate specifically: what would actually catch “the trace pattern moved but real learning didn’t” before it ever reached a live reward — is a held-out Zerbe-item check sufficient, or does it need something more?
• Does a discrete-trial ERP framing fit a live Socratic dialogue’s actual timing at all, given BrainWaves’ own trials run 0.5–2s and a real conversational intervention doesn’t have a fixed short duration?
• How many repeated firings of the same arm would a clean averaged ERP actually need, and does SAIL’s current live conversation volume get anywhere close to that (78 trials per condition, in BrainWaves’ own simpler two-category example)?
• What regulatory/consent machinery would this need beyond what a K-12 program already has — adult participants, a different institution, a different IRB context entirely?
• If a distinguishable per-arm ERP were found, what’s the actual path from “distinguishable on average, offline” to “usable as a live reward” — is that even the right target, or does this only ever make sense as a slow validation check against the existing cheap proxies (sail-wqs3’s Goodhart table), rather than a reward source in its own right?

Sources

BrainWaves App Designs (design deck, November 2018) — references/brainwaves-app-designs-2018.pdf. Tools named in the speculative section (WebGazer.js, Jeeliz Pupillometry, OpenFace) and the rPPG/pupillometry-and-memory research cited there are public, external work — not SAIL’s own, linked by name rather than vendored.