04 · Multimodal AI Research

Reading the room through AR

An AR assistant that combines voice, language, and body cues to understand emotional context.

Role

Researcher · multimodal learning

Year

2024

Diagram of a multimodal emotional context analysis system for AR glasses

In one sentence

The prototype demonstrated how wearable interfaces could support more context-aware, personalized communication.

01 · The problem

Why this mattered

Words carry only part of a conversation. Tone, expression, and body language often carry the rest—and most assistive systems ignore them.

02 · My work

From idea to system

I developed a multimodal pipeline with Arpit Kumar and advisors Dr. Abhinav Dhall and Dr. Sudeepta Mishra, trained using IEMOCAP to infer context and suggest responses.

03 · The result

What moved

The prototype demonstrated how wearable interfaces could support more context-aware, personalized communication.

Next projectFlight, without a pilot