Epistemic
Decisions Lab

Self presentation · Xuanyou Liu

Embodied AI in wearable form factors

Three projects: sense the body, decide how to help, intervene through touch.

EITWatch: the electrode ring on the back of a smartwatch and a person making a gesture

Seeing with the Hands in use: reaching into a cabinet, walking down stairs, and reaching for a kettle

Research agenda · Embodied AI in wearable form factors

A wearable that senses the body, decides how to help, and intervenes through touch.

Sense

Can a wearable read what the body is doing?

EITWatchUIST 2026 · Short Paper

Complementary signalsInformation your prior knowledge misses

ComplLLM figure: three Venn diagrams contrasting identifying concepts, signals, and complementary signals

Decide

Can AI complement a person's judgment under uncertainty?

ComplLLMarXiv 2026 · lab project

Intervene

Can it deliver assistance the person can feel?

Seeing with the HandsCHI 2025

UIST 2026 · Short Paper & Interactive Demo

EITWatch

Smartwatch-integrated planar electrical impedance tomography for hand gesture recognition

Xuanyou Liu* · Novel Alam* · Karan Ahuja *equal contribution

Can the back of a smartwatch recognise hand gestures?

EITWatch hardware, its dimensions relative to a standard smartwatch, and the worn prototype during a pinch gesture — Eight flat electrodes · 31 mm ring · 48 Hz

Electrical impedance tomography

EIT: Electrical signals can reveal what is inside an object.

Inject current, measure voltage at the boundary, and reconstruct the interior.

01 · Geometry

Surround the object with electrodes

Electrodes typically surround a roughly circular cross‑section.

02 · Probe

Inject current and measure voltage

Inject about 1 mA of alternating current, then measure the voltages at the boundary.

03 · Reconstruct

Build a picture of the inside

Reconstruct those voltages into an image of the interior.

Different tissues resist current differently, so those voltages change.

From tomography to interaction

How can we use EIT to sense the body?

The standard approach is a wristband: place a ring of electrodes around the wrist, and measure how muscles and tendons change as the hand moves.

Natural adaptation A circumferential electrode band wired around the wrist and connected to a laptop

Turn the electrode ring into a wristband

Extra form factor, a separate sensing board, and a wearing position that is not fixed.

Our question A person wearing the EITWatch prototype and performing a subtle hand gesture

Must EIT encircle the whole wrist?

A smartwatch contacts only a small patch of skin on one side of the wrist.

Prototype

The sensor is integrated into the watch back

0:00 / 0:00

31 mm eight electrodes in a ring, inside a 40 mm case

0.15 W 35 mA at 4.3 V while sensing and running the model

48 Hz sufficient for held poses and brief gestures

ESP32‑S3 Dual-core prototype: one core for EIT sensing, one for inference

Why several depths?

Planar EIT cannot surround the wrist

Without surrounding the wrist or forearm, we have less information about how muscle changes for different hand poses. To compensate, we developed a multi-depth scanning protocol.

An ablation study shows multi-depth injection ahead of adjacent injection by 15.1 macro and 10.4 micro points.

Evaluation · 12 participants

Within one wearing, recognition remains above 90%

Six static macro-gestures and six dynamic micro classes with their impedance maps

Main result · same wearing

Accuracy of each short window

Held-out round from the same wearing session

Macro: 91.4%
Micro: 90.1%

01 35 channels 48 readings a second

02 Filter, then window 0.83 s macro · 0.06 s micro

03 175 features 5 statistics on each channel

04 ExtraTrees six gestures, each weighted equally

Further tests Macro / micro

Worn again after 48 h 73.2% / 70.4%

New wearer 63.1% / 55.3%

Epistemic
Decisions Lab

Decide · arXiv 2026

ComplLLM

Fine-tuning LLMs to discover complementary signals for decision-making

Ziyang GuoYifan WuJason HartlineKenneth HolsteinJessica Hullman

You already know something. What extra information should you look at?

Three Venn diagrams. Identifying concepts targets the text only. Identifying signals targets the overlap of text and state. Identifying complementary signals targets the overlap of text and state that lies outside the recommendation.

An example

The report repeats most of what you already saw. What is new?

1 Read the image Z · prior knowledge y · real situation: heart problem or not?

2 Get a report T · extra information

3 Mostly overlap hard to spot the new part

4 Only the new part complementary signals

Decision theory

A signal is worth what it adds to the best decision.

1 · Prior knowledge Z alone

Û(Z) = max_d E_{y ~ π(y|Z)}[ U(d, y) ]

Pick the decision d with the best expected payoff U, knowing only Z. Here y is the real situation: cardiac dysfunction or not.

2 · Prior knowledge Z plus signals S

Û(Z, S) = max_d E_{y ~ π(y|Z, S)}[ U(d, y) ]

Same rule, but you also read the signals S taken from the extra information T.

3 · Complementary value

V = E[ Û(Z, S) ] − E[ Û(Z) ]

The hatched area: how much better the best decision gets once you read S.

Goal: train an LLM to read the extra information and output signals S with high V.

Method

Three steps teach an LLM to surface complementary signals.

1
EstimateAsk an LLM which findings appear in each report, then estimate how those findings relate to the true outcome.
2
Fine-tuneTeach by example: for each case, the findings that add value beyond what you already know.
3
ReinforceReward the LLM when its signals raise the best attainable payoff and really appear in the text.

Reward in step 3

R(s) = [ Û(s, z) − Û(z) ] / α

if every signal in s appears in the text, and 0 otherwise. α is the gain from the known complementary signals, used to normalize.

Result · chest X-ray task

A few signals get close to reading the whole report.

83.9 %with a few readable signals, vs 81.9% for the X-ray alone

SetupPredict cardiac dysfunction from the X-ray model's output plus the report, or only the signals.

Human Computer
Integration Lab

Intervene · CHI 2025 · Yokohama, Japan

Seeing with the Hands

A sensory substitution that supports manual interactions

Shan-Yuan Teng*Gene S-H Kim*Xuanyou Liu*Pedro Lopes*equal contribution

What if a blind person could see with their hand?

Three scenes of Seeing with the Hands in use: reaching into a cabinet while cooking, walking down stairs with a hand over the handrail, and reaching toward a kettle with an inset of the kettle as a grid of red tactile dots.

The idea

Sensory substitution usually stands in for the eyes. We put the camera in the palm.

A person uses their eyes to perceive their surroundings, with a blue light showing their gaze. The same person also uses their hands to see, reaching into a cabinet with the device, with an inset of a tactile image on the back of the hand shown as a grid of red and green dots.

Typical

Substitute the eyes, to perceive the surroundings.

Ours

Aim the camera at the palmar side, the side of the hand facing what it is about to grasp.

Participants imaginedfinding the handrail on stairs, grabbing an ingredient from a shelf, picking up something that fell.

Implementation

A palm camera and a 0.1 mm electrotactile grid turn what the hand sees into touch.

A hand wearing the device, with a black strap on the wrist and golden strips on the back of the hand. A callout on the palmar side of the strap is labeled camera, and a callout on the strips is labeled electrotactile display.

1
SeeComputer vision turns the image under the hand into a tactile pattern, in real time.
2
FeelA 5×6 display, 0.1 mm thin, on the back of the hand, so the palm stays free to grasp.
3
PixelEach pixel is a pair of electrodes. Pulses of 3 to 5 mA stimulate the touch receptors: a sense of slight touch.

User study · blind and low-vision participants

After about 15 minutes of practice, participants grasped everyday objects.

A collage of study photos, each showing a different person reaching a hand toward a different task in front of a table with objects.

~15 minof practice, without extensive training

ResultMost participants completed the tasks without touching unintended objects.

ErgonomicsCompared with the eyes' perspective, the hands' perspective was a more ergonomic way of seeing.

Handshake task · participants chose

Given the choice, all participants used both perspectives.

Two people shaking hands. Blue rays extend from the head and from the hand, with call-outs of close-up photos overlaid with a grid of red and white dots.

Eyes' perspectiveOverview

Hands' perspectiveDetails

Not a replacement.
A complement.

Research pillars · Embodied AI through wearable interactive systems

Three pillars, and the papers behind them.

Sense

Wearable multimodal sensing

First page of the paper EITWatch — **EITWatch**UIST 2026 · to appear

First page of the paper MARIO — **MARIO**CVPR 2026

Decide

Human-AI complementarity

First page of the paper Learning Checklists — **Learning Checklists**ICLR 2027 · under review

First page of the paper ComplLLM — **ComplLLM**arXiv 2026 · lab project

Learning Checklists: Workshop Best Paper Honorable Mention

Intervene

On-body haptics

First page of the paper Seeing with the Hands — **Seeing with the Hands**CHI 2025

First page of the paper TacTex — **TacTex**CHI 2024

Xuanyou LiuNorthwestern University · Epistemic Decisions Lab

xuanyouliu.com
[email protected]

Backup · The setting

The supervisor combines its own information with the recommendation.

Diagram of the two-agent setting. A recommending agent turns information X into a recommendation Z. A supervisor agent combines its own information T with Z. ComplLLM sits inside the supervisor and extracts complementary signals S from T. The decision D and the state Y determine the utility.

Recommending agentReads its own information X and outputs a recommendation Z.

SupervisorCombines its own information T with Z to make the final decision D.

ComplLLMExtracts signals S from T that improve the best attainable decision beyond Z.

Backup · Electrotactile stimulation

Each tactile pixel is a pair of electrodes passing tiny current pulses.

Cross section of the skin showing a tactile pixel: two electrodes labeled plus and minus, with a pulse current arrow passing through a tactile receptor.

Traditional EIT

One pair injects current. The others measure voltage.

A circumferential EIT band wrapped around a cross-section of tissue with bone and fat labelled, wired to stacked circuit boards, beside two circular impedance maps reconstructed from the band's measurements. — Electrodes around the cross-section, and two reconstructed images.

The injection appears first. Each later click measures the next pair. The injection steps after the round.

Backup · Methods

Methods at a glance

Sensing 35 channels · 48 Hz

A 50 kHz current of about 1 mA peak‑to‑peak

Held‑pose windows 40 frames · 0.83 s

Held poses. The window steps forward 2 frames. No rest class in this task.

Quick‑gesture windows 3 frames · 0.06 s

Five quick gestures plus rest. The window steps 1 frame. Rest is 51.7% of windows.

Features + model 175 · ExtraTrees

Five statistics on each of 35 channels. Each gesture is weighted equally.

Main number Accuracy of each window

Each short window is scored on its own. Scoring a whole gesture assumes its start and end are known.

Participants 12 · two sessions

Right‑handed; 6 women, 6 men; ages 20–31; second session 48 hours later

Backup · Timing and power

The model is fast. The window and the battery set the practical limits.

Held‑pose compute 3.07 ms

1.01 ms to prepare the signal + 2.06 ms to run the model

Quick‑gesture compute 5.25 ms

0.75 ms to prepare the signal + 4.50 ms to run the model

Movement to answer 847.1 / 72.9 ms

Held poses / quick gestures. For held poses, the 0.83 s window dominates.

Power while running 35 mA · 0.15 W

Measured at 4.3 V while sensing and classifying

A 300 mAh battery at this power use 8.6 hours

A prototype estimate, not an all-day claim for a finished watch

Open schematics, Gerbers, BOM, and ESP‑IDF firmware: github.com/XuanyouLiu/EITWatch_Hardware