By UChicago CS News
Imagine traveling in a foreign country, reaching for a window you’ve never seen before, and instead of struggling to open it, you feel your own muscles gently guide you through the motion, as if an invisible teacher were there, lending their know-how. Now picture that same sensation helping you twist open a child-proof pill bottle, operate a camera, or perform tasks you’ve never practiced before.
This is not science fiction. It’s the vision realized by PhD students Yun Ho and Romain Nith, under the supervision of associate professor Pedro Lopes in the Department of Computer Science at the University of Chicago. Their work, recently honored with the Best Paper Award at the ACM CHI 2026 conference, is turning heads across the human-computer interaction community.
From niche gadgets to general-purpose physical assistance
Electrical muscle stimulation (EMS) isn’t new; for years, researchers have been strapping electrodes to bodies, delivering controlled currents to teach piano sequences, demonstrate sign language, or support stroke rehabilitation. The catch? These systems have always been more like training wheels, only useful for a narrow set of programmed tasks, incapable of adapting on the fly to the messy, unpredictable real world.
As the research team puts it, EMS assistance to date has been “highly-specialized… fixed, and non-contextual.” In other words, your muscle “instructions” only fit the situations a designer anticipated. Ask the EMS to help shake a can of spray paint, and it springs to life. Hold up a spray can of cooking oil, and the device is clueless—because it can’t understand that you don’t need to shake it, and why.
This new system, which the authors nickname as “embodied AI”, however, marks a shift. By tapping into the power of modern multimodal artificial intelligence (think vision models like CLIP and GPT-4-level reasoning with computer vision), it merges what you see, where you are, and even your body’s pose to generate movement guidance tailored to the moment. EMS no longer follows a recipe; it improvises alongside you.
“I am curious about how people understand and build relationships with devices that communicate with them through body movements (rather than audio/visual),” said Ho. “In ‘embodied AI’, I got to explore this question in the realm of physical assistance. It was especially insightful to have participants “think aloud” as they used our system and learn how they interpret machine-induced movements.”
An AI that “knows how,” not just “knows what”
The magic here is procedural knowledge—the embodied, hard-to-describe sense of how to do something: gripping a jar lid just right to twist it open, or combining wrist and shoulder movements to unlatch a European window. For decades, researchers focused on giving people factual information; this approach transmits “know-how” to the muscles directly.
What changes with context-aware, generative EMS? For the first time, users get physically guided through unfamiliar, complex physical tasks, even when they aren’t able to explain what they need. The paper recounts a user study where participants succeeded at opening pill bottles with locking mechanisms, snapping pictures with a disposable camera, or using an avocado tool guided by dynamically generated muscle cues. And in cases where the AI made mistakes (on purpose, for the sake of testing), people noticed, adapted, and figured out solutions by re-prompting the system or correcting for its errors.
This iterative, back-and-forth approach, where the body’s intuition and the AI’s proposals meet, is significant. As one participant described: “the body’s intuitions help notice errors right away,” offering an edge over reading step-by-step instructions or watching a video.
“This could be a game-changer, not only for tasks that are highly physical (such as learning physical skills required for working with manufacturing and materials, or learning musical instruments) but also in situations where users might be situationally impaired (e.g., multitasking and performing several gestures at once, or cannot see in the dark, and so forth),” said Lopes.
Click here to read the full story on the UChicago Computer Science News website.