Home › English › Research › Holography and Light-field Display › Holographic light-field 3D touch sensing display

Japanese / English

Holographic 3D-touch display

 If the intensity and the direction of light emerged from objects are completely reproduced by a 3D display as if the objects were there, it provides perfect 3D perception. Such 3D display is called “light-field” display (LFD) [1]. [See fig.1] One of the promising applications of such 3D displays is a 3D human interface. A user can directly interact with the reproduced 3D images, which provides rich and appealing experience, and that is promising as a next-generation interface scheme (fig.2).

Fig.1 A light-field display

Fig.2 Concept of “3D touch” user interface

3D interface with bare hands or fingers is beneficial because a user does not need to ware any device, even though haptic feedback is difficult. Gesture recognition systems have been developed and applied to the interface for human-computer interaction or gaming. In addition, it is possible to combine 3D gesture interface technology with 2D and 3D displays. The integration of 3D displays and 3D interface will facilitates intuitive and direct interaction with the 3D images. Moreover, the interface without touching the screen is preferable in various applications to avoid contaminating the screen and fingers.
 

Fig.3 A light field display using a holographic screen and projector

 Automatic calibration between the projector and the holographic screen

We have been developing a 3D user interface by detecting the light scattered from the finger that touches the real image reproduced by the 3D display. The 3D display reproduces light-rays from objects by using a holographic screen and a projector, as shown in fig. 3 [2].
The holographic screen [3] consists of a 2D array of small elementary holograms that reproduce diverging light. The holographic screen is illuminated by the light from a projector, where the projected image contains the light-ray information similar to the one used in integral imaging. The holographic screen reconstructs a 2D array of diverging light-rays, which are modulated by the projected image, and it works as a full-parallax light-field 3D display.
In this optical system, the alignment between the holographic screen and the projected image is critical. We have developed a method for automatic geometric calibration with using a camera placed in front of the holographic screen. By projecting a sequence of images for calibration, the geometric relationship between the projected image and the holographic screen is derived from the image set captured by the camera. Then the images provided to the projector are pre-distorted such that the complete calibration is achieved [2].

Fig.4 “3D touch” display using a holographic screen

The system for 3D touch sensing display is shown in fig. 4 [4-6]. A 3D real image is reproduced by the display presented in the previous section, and a user can touch the floating image. When a user touches the real image by a finger, the fingertip scatters the light that forms the real image by the light-field display. The scattered light is captured by a color camera placed behind the screen. Since the touch to the 3D image is detected directly, it does not cause positional error even without alignment of 3D spaces. Moreover, by utilizing the color information, it is possible to identify various operations with a relatively small amount of calculation. For this reason, comfortable and intuitive 3D interface can be realized without using an additional device in the proposed method.

Fig. 5 Experimental system using monochromatic (red-only) images

3D touch demonstration

Fig. 6 Experimental system using color images

3D touch demonstration (color)

 Fig. 5 shows an example using a holographic screen for red image reproduction. In the left picture, a character ‘T’ is reproduced in front of the screen. The right pictures of fig. 5 show the side view of the display when a fingertip touches the aerial image of the character ‘T’. After touching, a text ‘OK’ is displayed at the bottom left of the screen as shown in fig. 5 (b). In the left picture of fig. 5 (b) the fingertip becomes red because the fingertip is irradiated by the red light of real image. The example of the interface using a color image is shown in fig.6. When the user’s finger touches the character ‘Y’ or ‘N’, the fingertip becomes green or yellow, which is detected by a camera placed behind the screen. Then, the text ‘YES’ is displayed if green is detected, or ‘NO’ is displayed on the screen if blue is detected by the camera, as shown in fig. 6 (a) and (b).

This work is supported by Japan Society for the Promotion of Science (JSPS) (KAKENHI 15K04691).

[1] M. Yamaguchi, “Light-field and holographic three-dimensional displays [Invited],” Journal of the Optical Society of America A, Vol. 33, No. 12, 2348-2364, (2016).
[2] M. Yamaguchi, T. Koyama, N. Ohyama, and T. Honda, “A stereographic display using reflection holographic screen,” Optical Review, 1, 2, 191-194 (1994).
[3] R. Higashida and M. Yamaguchi, “Automatic geometric calibration in full-parallax 3 display using holographic screen,” Proc. IDW’14, 3Dp1-3 (2014)
[4] M. Yamaguchi, “Holographic 3D Touch Sensing Display,” in Digital Holography and 3-D Imaging Meeting, OSA Technical Digest, paper DM3A.1 (2015).
[5] M. Yamaguchi, Ryo Higashida. 3D touchable holographic light-field display, Applied Optics, Vol. 55, No. 3, A178-A183, (2016).
[6] M. Yamaguchi, “Full-Parallax Holographic Light-Field 3-D Displays and Interactive 3-D Touch,” Proceedings of the IEEE, Vol. 105, No. 5, 947-959 (2017).

Publications