Shape Displays: Spatial Interaction with Dynamic Physical Form
Abstract
Shape displays are a new class of I/O devices
that dynamically render physical shape and
geometry. They allow multiple users to experience
information through touch and deformation
of their surface topology. The rendered shapes
can react to user input or continuously update
their properties based on an underlying simulation.
Shape displays can be used by industrial
designers to quickly render physical CAD models
before 3D printing, urban planners to physically
visualize a site, medical experts to tactually explore
volumetric data sets, or students to learn and
understand parametric equations.
Previous work on shape displays has mostly focused
on physical rendering of digital content to
overcome the limitations of single-point haptic
interfaces—examples include the Feelex and Lumen projects. In our research, we emphasize the
use of shape displays for designing new interaction
techniques that leverage tactile spatial qualities to
guide users. For this purpose, we designed, developed,
and engineered three shape display systems
that integrate physical rendering, synchronized
visual display, shape sensing, spatial tracking, and
object manipulation. This enabling technology has
allowed us to contribute numerous interaction
techniques for virtual, physical, and augmented
reality, in collocated settings as well as for remote
collaboration.
Our systems are based on arrays of motorized
pins, which extend from a tabletop to form 2.5D
shapes: Relief consists of 120 pins in a circular
tabletop, a platform later augmented with spatial
graphics for the Sublimate system. Our next-generation
platform, inFORM renders higher resolution
shapes through 900 pins (see Figure 1).
The Transform system consists of 1,152 pins embedded
into the surface of domestic furniture. To capture objects and gestures and to control visual
appearance, we augment the shape displays with
overhead depth-sensing cameras and projectors.
In this article, we wish to introduce readers
to some of the exciting interaction possibilities
that shape displays enable beyond those found
in traditional 3D displays or haptic interfaces.
We describe new means for physically displaying
3D graphics, interaction techniques that leverage
physical touch, enhanced collaboration through
physical telepresence and unique applications of
shape displays. Our current shape displays are
based on prototype hardware that enabled us to
design, develop, and explore a range of novel interaction
techniques. Although the general applicability
of these prototypes are limited by
resolution, mechanical complexity, and cost, we
believe that many of the techniques we introduce
can be transferred to a range of special-purpose
scenarios that have different sensing and actuation
needs, potentially even using a completely different
technical approach. We thus hope that our
work will inspire future researchers to start considering
dynamic physical form as an interesting
approach to enable new capabilities and expressiveness
beyond today’s flat displays.
that dynamically render physical shape and
geometry. They allow multiple users to experience
information through touch and deformation
of their surface topology. The rendered shapes
can react to user input or continuously update
their properties based on an underlying simulation.
Shape displays can be used by industrial
designers to quickly render physical CAD models
before 3D printing, urban planners to physically
visualize a site, medical experts to tactually explore
volumetric data sets, or students to learn and
understand parametric equations.
Previous work on shape displays has mostly focused
on physical rendering of digital content to
overcome the limitations of single-point haptic
interfaces—examples include the Feelex and Lumen projects. In our research, we emphasize the
use of shape displays for designing new interaction
techniques that leverage tactile spatial qualities to
guide users. For this purpose, we designed, developed,
and engineered three shape display systems
that integrate physical rendering, synchronized
visual display, shape sensing, spatial tracking, and
object manipulation. This enabling technology has
allowed us to contribute numerous interaction
techniques for virtual, physical, and augmented
reality, in collocated settings as well as for remote
collaboration.
Our systems are based on arrays of motorized
pins, which extend from a tabletop to form 2.5D
shapes: Relief consists of 120 pins in a circular
tabletop, a platform later augmented with spatial
graphics for the Sublimate system. Our next-generation
platform, inFORM renders higher resolution
shapes through 900 pins (see Figure 1).
The Transform system consists of 1,152 pins embedded
into the surface of domestic furniture. To capture objects and gestures and to control visual
appearance, we augment the shape displays with
overhead depth-sensing cameras and projectors.
In this article, we wish to introduce readers
to some of the exciting interaction possibilities
that shape displays enable beyond those found
in traditional 3D displays or haptic interfaces.
We describe new means for physically displaying
3D graphics, interaction techniques that leverage
physical touch, enhanced collaboration through
physical telepresence and unique applications of
shape displays. Our current shape displays are
based on prototype hardware that enabled us to
design, develop, and explore a range of novel interaction
techniques. Although the general applicability
of these prototypes are limited by
resolution, mechanical complexity, and cost, we
believe that many of the techniques we introduce
can be transferred to a range of special-purpose
scenarios that have different sensing and actuation
needs, potentially even using a completely different
technical approach. We thus hope that our
work will inspire future researchers to start considering
dynamic physical form as an interesting
approach to enable new capabilities and expressiveness
beyond today’s flat displays.
Research Areas
