Exploring Human Information Processing in Virtual Reality during Physical Inverse Design

March 12, 2026

We devise a naturalistic inverse design paradigm in virtual reality (VR) where participants hand sculpt a clay-like surface to guide a stream of water into target wells. The platform combines realistic physics with fine-grained behavior capture (hands, eyes, surface state, symmetry toggles, fluid metrics) to probe how environmental dynamics and non-physical affordances shape planning. Two environments were used: online, with continuous flow during sculpting, and offline, with no flow until evaluation, to contrast feedback-driven control with internally guided, representation-heavy planning. The voluntary use of a user interface to activate symmetries exposes optional non-physical affordances, assessing how bodily attributes shape planning. Baseline abilities are measured via a mental rotation tests as a proxy for physical intuition, and a dexterity "match the shape" task, both in VR. We hypothesize that (i) online dynamics elicit more adaptive control, (ii) optional symmetries alter optimization trajectories by reducing effective dimensionality, and (iii) global and local hand synergies reveal latent intent and skill. This work links high-level reasoning with dexterous manipulation in a continuous, high-dimensional setting.

Authors in the community:

• 

  S

  Sara Maria Silva Monteiro

  ist1108402

Supervisors of this institution:

• 

  A

  Athanasios Vourvopoulos

  ist428516

electrical-engineering-electronic-engineering-information-engineering - Electrical engineering, electronic engineering, information engineering

eng - English

Instituto Superior Técnico