1. INTRODUCTION

Floating images in a crystal ball give observers artistic and mysterious impressions. A 2D image can be displayed in a crystal ball by placing a screen in the ball and projecting an image onto it [1]. Though it is amazing and mysterious, it cannot support motion parallax and observers cannot sufficiently feel the existence of objects in it. In order to realize motion parallax in floating images, a viewing-angle-enhanced integral imaging system using a curved lens array was proposed [2]. But the effective viewing angle of the system is only $\pm 16.5\mathrm{deg}$. Moreover, as trapezoidal distortion, which is generated from tilt imaging by a curved lens array, is not considered, the resolution in the integrated image is very low. To realize a 360-deg 3D image, a 360-deg tabletop electronic holographic display was proposed [3]. But the vertical size of the viewing window is only 4 mm, which is too small to be used practically. Consequently, vertical motion parallax cannot be realized. Similar systems have been proposed. By using the rotation of a screen with sharp directivity [4] or multiple projectors [5], floating images can be produced in free space. However, it requires a large number of directional images and vertical motion parallax cannot be reproduced. Therefore, we tried to develop a ball-type display in which highly realistic 3D images could be produced with smooth motion parallax in horizontal and vertical directions, which is also important to give the feel of existence, using small numbers of image sources. To develop such a display, we need optics to produce floating images in the ball with low distortion from highly oblique viewing positions.

2. OUR PREVIOUS STUDIES

We have studied a 3D display method, which can produce smooth motion parallax for horizontal and vertical directions, and a viewing zone control method for projection displays. We applied these methods to a crystal ball display.

A. Depth-Fused 3D Display

We have developed a display method, which consists of a stack of display screens, as shown in Fig. 1(a) [6]. Since observers perceive the layered images fused for depth direction, it is called depth-fused 3D (DFD).

 

Fig. 1. Principle of smooth motion parallax using a multiview DFD display. (a) Ordinary DFD display, (b) ordinary DFD with an inclined stacking axis, and (c) multiview (two-view) DFD display.

Download Full Size | PDF

In this display, the same shape of an image from a viewpoint is displayed on the layer, but luminance is shared depending on the image depth. The perceived image is generated between two stacked display screens, which are the front image and rear image. The algorithm of a DFD is shown below. We use computer graphics (CG) the depth information of which is known. The depth position of a 3D perceived image is defined as $Z(x,y)$ and the luminance to the front direction of this point is defined as $Y(x,y)$. And the depth positions and displayed luminance of the 2D front image and 2D rear image are defined as $Z_{\mathrm{front}}$, $Y_{\mathrm{front}}(x,y)$ and $Z_{\mathrm{rear}}$, $Y_{\mathrm{rear}}(x,y)$, respectively. $Y_{\mathrm{front}}(x,y)$ and $Y_{\mathrm{rear}}(x,y)$ can be expressed, respectively, as follows:

 

Fig. 2. Ordinary DFD algorithm.

Download Full Size | PDF

We use this ordinary DFD algorithm in this 360-deg display.

This display produces disparity using shift of layered images introduced by the difference between the stacking axis of the layered images and the observation axis. Therefore, disparity changes continuously depending on the position of an observer’s viewpoint. It can satisfy not only binocular parallax and convergence, but also motion parallax for a small movement of the observer’s head due to continuous change of disparity. Since the disparity is not restricted to the horizontal direction, motion parallax for vertical or other oblique directions can also be reproduced. However, when the difference between the stacking and observation axis becomes big, the layered images cannot be fused and a corrupt image is observed. That is, the comfortable observation area zone is very narrow.

B. Multiview DFD Display

To expand the viewing zone of DFD displays, we proposed a multiview DFD display [7]. It consists of a stack of two or more multiview displays. Continuous connection of adjacent viewing zones and binocular observation of different viewing zones of the multiview display were confirmed natural by calculation of perceived depth and a tiny prototype of a two-view, two-layer DFD display.

Figure 1(b) and 1(c) shows the principle of multiview DFD displays. As shown in Fig. 1(b), the observation area of an ordinary single-view DFD display is controlled by changing the overlapping axis shifting the front and rear images. A multiview DFD display is constructed by arranging the observation areas of DFD displays to cover a wider observation area as shown in Fig. 1(c). Since a DFD display produces correct disparity at a viewpoint, motion parallax is well connected between neighboring observation areas, which enables a wider viewing zone with smooth motion parallax.

C. Spatially Imaged Iris Method

We have developed a directional display method using a spatially imaged iris [8–11]. It consists of a display engine, a projection lens, an observation lens, and a vertical diffuser. The projection lens produces an image of the display at the observation lens. The observation lens images the aperture of the projection lens at the viewing plane as an iris image. Since all rays from the display engine reach the iris image, the light efficient is high. We placed a vertical diffuser to get enough viewing area for the vertical direction.

In a further study, we proposed using a larger aperture size and to eliminate the vertical diffuser keeping enough viewing zone [12]. This optics is a screen-free configuration. It allows providing higher resolution than conventional displays using vertical diffusers, because a vertical diffuser enhances the effects of image plane distortion as blur. This method can not only limit the viewing area, but also improve optical efficiency.

3. PROBLEMS AND SOLUTION

A. Three Problems of a Wide-Viewing Multiview DFD Display

A multiview (two-view) DFD display is shown in Fig. 1(c). For a wider viewing angle, many oblique multiview DFD displays are needed. But a wide-viewing multiview DFD display has the following three problems:

First, we explain the first problem. Figure 3 shows an ordinary DFD display. In this case, the stacking axis is not inclined. As shown in Fig. 3, the distance between the front and rear images is $L_1$ and the viewing angle of observation area 1 is ${\theta }_1$. Figure 4 shows observation area 2 of neighbors on both sides and an oblique multiview DFD display.

 

Fig. 3. $L_1$ and ${\theta }_1$ in an ordinary DFD display.

Download Full Size | PDF

 

Fig. 4. Observation area 2 of neighbors on both sides and an oblique multiview DFD display.

Download Full Size | PDF

As shown in Fig. 4, the stacking axes are inclined. Consequently, an inclined distance $L_2$ between the displayed rear image and the displayed front image is larger than $L_1$ in Fig. 3. In the principle of a DFD display, a larger distance between the front and rear image requires a smaller angle of the observation area. Approximately a distance $L$ is inversely proportional to an angle $\theta$. That is to say, the equation ${\theta }_2=(L_1/L_2){\theta }_1$ is true. In Fig. 4, the inclined angle of the stacking axis is $({\theta }_1+{\theta }_2)/2$. Therefore, $L_1/L_2$ is described as $\cos \{({\theta }_1+{\theta }_2)/2\}$. This means that the larger the inclined angle of the stacking axis becomes, the smaller is the angle of observation area. When the inclined angle of the stacking axis is close to 90 deg, $L_2$ becomes infinity and the angle of the observation area is nearly 0 deg. This phenomenon is shown in Fig. 5(a) to 5(c).

 

Fig. 5. First problem: narrow viewing zone of inclined observation by the fusion condition of the DFD display. The first problem causes the phenomenon which is that the larger the inclined angle of the stacking axis becomes, the smaller the angle of observation area is. (a) A large inclined angle. (b) A very large inclined angle. (c) A nearly 90 deg inclined angle.

Download Full Size | PDF

Synthesizing Figs. 3,4,5(a),5(b), and 5(c), a wide-viewing multiview DFD display is shown in Fig. 6.

 

Fig. 6. First problem of a wide-viewing multiview DFD display.

Download Full Size | PDF

As shown in Fig. 6, when the inclined angle of the stacking axis becomes large, the angle of the observation area is very small. This means that a large number of image sources and a high-resolution display are needed to slightly widen the viewing angle when the inclined angle of the stacking axis becomes large.

Second, we explain the second problem. In an oblique multiview DFD display, the displayed positions of the rear and front images shift to the opposite direction in the display panel. In Fig. 7, the inclined angle of the stacking axis is $({\theta }_1+{\theta }_5)/2+{\theta }_2+{\theta }_3+{\theta }_4$. Therefore, the shifting distance $S_5$ is expressed as $L_1$ $\tan \{({\theta }_1+{\theta }_5)/2+{\theta }_2+{\theta }_3+{\theta }_4\}$. This means that the larger the inclined angle of the stacking axis becomes, the larger is this shifting distance. In case of the width of the displayed image is $W$, the width of the panel is $S_5+W$. When the inclined angle of the stacking axis is close to 90 deg, $S_5$ becomes infinity and the panel size becomes infinity too.

 

Fig. 7. Second problem: need of a large wide size display of which width is $S_5+W$ by the shifting distance between the rear and front images of inclined observation. The positions of the rear and front images and the shifting distance $S_5$ are displayed.

Download Full Size | PDF

Third, we explain the third problem. In a large-oblique-angle multiview DFD display, a large horizontal size of the display panel compared with the displayed image size $W$ is needed. From the observer’s viewing direction, the width of the viewing image is very small compared with the displayed image size $W$. In Fig. 8, the inclined angle of the stacking axis is $({\theta }_1+{\theta }_5)/2+{\theta }_2+{\theta }_3+{\theta }_4$. Therefore, the width of the viewing image $W_5$ is expressed as $W$ $\cos \{({\theta }_1+{\theta }_5)/2+{\theta }_2+{\theta }_3+{\theta }_4\}$. This means that the larger the inclined angle of the stacking axis becomes, the smaller is the width of viewing image. When the inclined angle of the stacking axis is close to 90 deg, $W_5$ becomes 0.

 

Fig. 8. Third problem: need of large wide size display of which horizontal size is big by the width of the viewing image of inclined observation. The width of the viewing image, $W_5$, from the observer’s viewing direction is displayed.

Download Full Size | PDF

To solve the three problems mentioned above, we develop a rotational 360-deg multiview DFD display.

B. Solution by a 360-deg Rotational Multiview DFD Display

The cause of problems is an oblique multiview DFD display. A rotational 360-deg multiview DFD display is composed of rotational ordinary DFD displays as shown in Fig. 3. The structure of the display is shown in Fig. 9. An oblique multiview DFD display is not used. Many rotational ordinary DFD displays, whose rotational axes are set at the center of the perceived images, are superimposed by varying the angle of the stacking axis at intervals of ${\theta }_1$. Therefore, the observation areas do not overlap and have no gap. The advantage of a DFD display is smooth motion parallax in not only the horizontal but also the vertical direction using a small number of image sources. To realize the smallest number of image sources, the largest angle of observation area of an ordinary DFD display must be selected. Since an oblique multiview DFD display is not used in a rotational 360-deg multiview DFD display, the first problem is solved.

 

Fig. 9. Rotational 360-deg multiview DFD display.

Download Full Size | PDF

As shown in Fig. 9, ordinary DFD displays are aligned radially in all directions. Consequently, we use only the algorithm of an ordinary DFD as shown in Fig. 2. In an ordinary DFD display, the disparity between the front and rear image becomes bigger when viewing from an oblique direction. But by using a spatially imaged iris method, the observer can see only a neighboring DFD display. Therefore, disparity is limited. When ordinary DFD displays are aligned in high angular density, motion parallax is well connected between neighboring observation areas naturally as shown in Ref. [7]. The second and third problems are solved in the same way. All panel sizes are equal to the displayed image sizes $W$. This is the smallest horizontal size and so the second problem is solved. From all 360-deg viewing directions of the observer, the width of the viewing image is nearly equal to the displayed image size $W$ and so the third problem is solved. However, a new problem occurs. The fourth problem is that a real device panel, for example, an LCD, cannot be used for superimposition. To solve this problem, floating images with no screen or no display panel are suitable for rotational superimposition.

4. PRINCIPLE

A. Optics for a 360-deg Rotational Multiview DFD Display

To realize 360-deg rotational superimposition by floating images as shown in Fig. 9, we selected a ball-type display. First, we constructed a ball-lens-type spatially imaged iris method. A spatially imaged iris method consists of a display engine, a projection lens, and an observation lens. The projection lens produces an image of the display at the observation lens. The observation lens images the aperture of the projection lens at the viewing plane as an iris image. Since all rays from the display engine reach any points on the iris image, the observer can see the whole displayed image when his/her eye is on the iris image. This optics is a screen-free configuration and is suitable for superimposition by floating images.

To realize 360-deg rotational superimposition by floating images, we construct the observation lens with a ball lens. A ball lens enables centrosymmetric optics. By setting both the rotational axis of the optical system and the imaging position of the perceived image at the center of a ball lens, 360-deg rotational superimposition by floating images can be realized as shown in Fig. 9. To realize this method, we analyze an imaging at the center of a ball lens and an imaging of an iris image. The function of the observation lens in a spatially imaged iris method is to image the aperture of the projection lens at the viewing plane as an iris image. Therefore, a ball lens must realize the same function as the observation lens. Moreover, we must obtain a displayed floating image at the center position of the ball lens. To realize this function, a parallel beam imaging at the center of the ball lens is suitable because when parallel beams are rotated in relation with the center of a ball lens, the imaging plane exists near the center of the ball lens. To make parallel beams in the ball lens, we analyze the spherical shape side forward focal length of a semispherical ball lens. In Fig. 10, a pinpoint light source A is set in the spherical shape side forward focal plane of a hemispherical ball lens of radius $r$.

 

Fig. 10. Spherical shape side forward focal length $f$ of a hemispherical ball lens.

Download Full Size | PDF

A point O is the center of a ball lens. The spherical shape side forward focal length $f$ of a semispherical ball lens is the distance between the surface of the ball lens and a light source A. The refractive indices of air and the ball lens are $n_1$ and $n_2$, respectively. A beam AO is not refracted by the surface of the ball lens and goes through straight. A beam AB is refracted at point B and this refracted beam BC in the ball lens is parallel to beam AO. At point B, Snell’s laws of refraction can be described as follows using incident angle ${\theta }_1$ and output angle ${\theta }_2$:

 

Fig. 11. Imaging of a projection lens as an iris image by a ball lens and the imaging plane at the center of a ball lens.

Download Full Size | PDF

Therefore, the imaging plane by parallel beams is located at the center of a ball lens. By adding a display panel on the incident side of the projection lens as shown in Fig. 11, a ball-lens-type spatially imaged iris method is accomplished. The optical system is shown in Fig. 12. The distance between the display panel and the projection lens is $a$. The focal length $f_{\mathrm{PRJ}}$ of the projection lens is determined by the following relationship:

 

Fig. 12. Optical system of a ball-lens-type spatially imaged iris method.

Download Full Size | PDF

The size of a perceived image is $(f+r)/a$ times larger than that of the displayed image in the display panel. The size of an image of the projection lens as an iris image is equal to that of the projection lens. When the size of an iris image is larger than that of a perceived image, the shape of the observation area becomes a pentagon and extends to infinity as shown in Fig. 12 [13]. Only in this observation area can an observer see a perceived image.

Second, to realize a floating DFD display at the center of a ball lens, the display panel shown in Fig. 12 is shifted forward and backward according to the optical axis. When the distance between the displayed image and the projection lens is set at $a+\mathrm{\Delta }{a}_1$, the distance between the projection lens and the rear imaging plane of a rear displayed image becomes $f+r-\mathrm{\Delta }{b}_1$ as shown in Fig. 13. The relationship between $a+\mathrm{\Delta }{a}_1$ and $f+r-\mathrm{\Delta }{b}_1$ can be described approximately as follows:

 

Fig. 13. Rear imaging plane of a rear displayed image in a ball lens by a $\mathrm{\Delta }{a}_1$ backward shifting display panel.

Download Full Size | PDF

In Fig. 13, since a projection lens is set in the spherical shape side forward focal plane of a hemispherical ball lens, beams from each point of the projection lens are parallel in the ball lens. And the size of the iris image is equal to that of the projection lens.

In the same way, the relationship between $a-\mathrm{\Delta }{a}_2$ and $f+r+\mathrm{\Delta }{b}_2$ can also be described approximately as follows (as shown in Fig. 14):

 

Fig. 14. Front imaging plane of the front displayed image in a ball lens by a $\mathrm{\Delta }{a}_2$ forward shifting display panel.

Download Full Size | PDF

When $\mathrm{\Delta }{b}_1=\mathrm{\Delta }{b}_2$ is true, the center of the depth range of the DFD display is located at the center of a ball lens. Under the condition $\mathrm{\Delta }{b}_1=\mathrm{\Delta }{b}_2$, $\mathrm{\Delta }{a}_1$ and $\mathrm{\Delta }{a}_2$ are determined by Eqs. (9) and (10), respectively. By superimposing the optical systems shown in Figs. 13 and 14, a DFD optical system of a ball-lens-type spatially imaged iris method is obtained as shown in Fig. 15.

 

Fig. 15. DFD optical system of a ball-lens-type spatially imaged iris method.

Download Full Size | PDF

The depth range of the DFD display is $\mathrm{\Delta }{b}_1+\mathrm{\Delta }{b}_2$. A half-mirror is used for coupling two display panels. Many DFD optical systems of a ball-lens-type spatially imaged iris method, as shown in Fig. 15, are rotated in relation with the center of the ball lens and superimposing at the same position of the ball lens. A ball lens is common to each optical system. Therefore, we obtain rotational superimposition of the rotated rear and front imaging planes of the displayed images for the DFD display at the center of the ball lens. The rotational angle of each optical system is adjusted for no overlap and no gap of iris images as shown in Fig. 16.

 

Fig. 16. Rotational superimposition of the rotated rear and front imaging planes of displayed images for a DFD display at the center of a ball lens.

Download Full Size | PDF

The optical systems shown in Fig. 16 are rotated to seven directions every 24 deg and the total rotation angle is 144 deg. The total rotational angle of eight directionally rotated optical system is 192 deg which is over 180 deg. Consequently, moreover 144 deg rotation makes the collision of observation area and a set of projection lens and display panel which is input side optical system of a ball lens. To avoid this collision, the optical system shown in Fig. 15 is slightly rotated to the vertical direction as shown in Fig. 17. This method enables 360-deg rotation without collision.

 

Fig. 17. Vertical cross section of a 360-deg rotational multiview DFD display.

Download Full Size | PDF

By this method, a 360-deg multiview DFD display is obtained as shown in Fig. 18.

 

Fig. 18. Upper view of the 360-deg multiview DFD display.

Download Full Size | PDF

B. Optics for a 360-deg Rotational Multiview DFD Display by a Magnified Spatially Imaged Iris Method

The 360-deg multiview DFD display mentioned above has centrosymmetric optics in relation with the center of a ball lens. Therefore, the positions of the projection lenses and the images of the projection lenses, which are iris images, are the same from the upper view as shown in Fig. 18. The projection lenses are located beneath the iris images and the display panels and half-mirrors are set underneath the pentagonal observation areas. Consequently, observers cannot look at the iris images and must look away from them.

No gap and no overlap of iris images are observed only in the iris imaging plane. Smooth motion parallax between neighboring iris images is obtained only in the iris imaging plane. And so the best viewing position is the iris imaging planes. In order to achieve viewing in the iris image, we developed a magnified spatially iris method by a ball lens. In Fig. 15, when the distance $f+a+\mathrm{\Delta }{a}_1$ between the display panel and the ball lens is smaller than the distance $f$ between an image of the projection lens and the ball lens, an observer can see the DFD display in an iris image. To make this condition, we shifted a set of display panels, a half-mirror, and a projection lens close to the ball lens on the optical axis. By this method, the distance $f+a+\mathrm{\Delta }{a}_1$ between the display panel and the ball lens is changed to $b+a+\mathrm{\Delta }{a}_1$ and the distance $f$ between an image of the projection lens and the ball lens is changed to $c$ under the condition $b+a+\mathrm{\Delta }{a}_1<c$. In the first step, we analyze an imaging of the projection lens by the ball lens and an imaging plane of the displayed image at the center of the ball lens. In Fig. 19, a pinpoint light source A is set at a distance $b$ from a point A to the ball lens whose radius is $r$.

 

Fig. 19. Optical path analysis of a magnified spatially imaged iris method.

Download Full Size | PDF

A point O is the center of the ball lens. The refractive indices of air and the ball lens are $n_1$ and $n_2$, respectively. A beam AOF is not refracted by the surface of the ball lens and goes through straight. A beam AB is refracted at point B and this refracted beam BC in the ball lens is not parallel to a beam AOF. The inclined angle of beam BC is ${\theta }_3$ as shown in Fig. 19. At point B, Snell’s laws of refraction can be described by Eq. (5).

The law of sines applied to a triangle OAB can be described as follows:

The law of sines applied to a triangle OFC can be described as follows:

 

Fig. 20. Magnified imaging of a projection lens as an iris image by a ball lens and the imaging plane at the center of a ball lens.

Download Full Size | PDF

Points ${\mathrm{A}}^{\prime }$ and ${\mathrm{A}}^{\prime \prime }$ are located at the edge of the projection lens. Consequently, the aperture of the projection lens is imaged at an iris imaging plane $F^{\prime }{F}^{\prime \prime }$. Although this optics is not centrosymmetric in relation with the center of the ball lens, the center of the imaging plane of the displayed image is located at the center of the ball lens. Without this point O, the imaging plane is located slightly forward and has a spherical surface. But near the optical axis, the imaging plane is nearly a flat plane. Therefore, the imaging plane is located at the center of the ball lens.

By adding a display panel on the incident side of the projection lens as shown in Fig. 20, a ball-lens-type magnified spatially imaged iris method is accomplished. The optical system is shown in Fig. 21.

 

Fig. 21. Optical system of a ball-lens-type magnified spatially imaged iris method.

Download Full Size | PDF

The distance between the display panel and the projection lens is $a$. The focal length $f_{\mathrm{PRJ}}$ of the projection lens is determined by the following relationship:

To realize a floating DFD display at the center of the ball lens, the display panel shown in Fig. 21 is shifted forward and backward according to the optical axis. When the distance between the displayed image and the projection lens is set at $a+\mathrm{\Delta }{a}_1$, the distance from the projection lens to the rear imaging plane of the rear displayed image becomes $b+r-\mathrm{\Delta }{b}_1$ as shown in Fig. 22. The relationship between $a+\mathrm{\Delta }{a}_1$ and $b+r-\mathrm{\Delta }{b}_1$ can be described approximately as follows:

 

Fig. 22. Rear imaging plane of the rear displayed image in a ball lens by a $\mathrm{\Delta }{a}_1$ backward shifting display panel in a magnified spatially imaged iris method.

Download Full Size | PDF

In the same way, the relationship between $a-\mathrm{\Delta }{a}_2$ and $b+r+\mathrm{\Delta }{b}_2$ can also be described approximately as follows (as shown in Fig. 23):

 

Fig. 23. Front imaging plane of the front displayed image in a ball lens by a $\mathrm{\Delta }{a}_2$ forward shifting display panel in a magnified spatially imaged iris method.

Download Full Size | PDF

When $\mathrm{\Delta }{b}_1=\mathrm{\Delta }{b}_2$ is true, the center of the depth range of the DFD display is located at the center of the ball lens. Under the condition $\mathrm{\Delta }{b}_1=\mathrm{\Delta }{b}_2$, $\mathrm{\Delta }{a}_1$ and $\mathrm{\Delta }{a}_2$ are determined by Eqs. (18) and (19), respectively.

By superimposing the optical systems shown in Figs. 22 and 23, a DFD optical system of a ball-lens-type magnified spatially imaged iris method is obtained as shown in Fig. 24.

 

Fig. 24. DFD optical system of a ball-lens-type magnified spatially imaged iris method.

Download Full Size | PDF

The depth range of the DFD display is $\mathrm{\Delta }{b}_1+\mathrm{\Delta }{b}_2$. A half-mirror is used for coupling two display panels. Many DFD optical systems of a ball-lens-type magnified spatially imaged iris method, shown in Fig. 24, are rotated in relation with the center of the ball lens and superimposing at the same position of the ball lens. A ball lens is common to each optical system. Therefore, we obtain rotational superimposition of the rotated rear and front imaging planes of the displayed images for the DFD display at the center of the ball lens. The rotational angle of each optical system is adjusted for no overlap and no gap of iris images as shown in Fig. 25.

 

Fig. 25. Rotational superimposition of the rotated rear and front imaging planes of displayed images for a DFD display at the center of a ball lens by a magnified spatially imaged iris method.

Download Full Size | PDF

The optical systems shown in Fig. 25 are rotated in seven directions every 22.5 deg and the total rotation angle is 135 deg. The total rotational angle of eight directionally rotated optical system is 180 deg. Consequently, moreover 135 deg rotation makes the collision of observation area and a set of projection lens, half mirror and display panels which are input side optical system of a ball lens. To avoid this collision, the optical system shown in Fig. 24 is slightly rotated to the vertical direction as shown in Fig. 26. This method enables 360-deg rotation without collision. By this method, a 360-deg multiview DFD display is obtained as shown in Fig. 27.

 

Fig. 26. Vertical cross section of a 360-deg multiview DFD display by a magnified spatially imaged iris method.

Download Full Size | PDF

 

Fig. 27. Upper view of the 360-deg multiview DFD display by a magnified spatially imaged iris method.

Download Full Size | PDF

In Fig. 27, the rotational optical system, which consists of a ball lens, projection lenses, half-mirrors, and display panels, is confined within the inner space surrounded by the magnified images of the projection lens. As shown in Fig. 24, the length between the center of ball lens and display panel is r + b + a + Δa₁ and the length between the center of ball lens and iris image is r + c. When b + a + Δa₁ < c is true, there are no optical system in the outer space of the encircled iris images as shown in Fig. 27. Consequently, observers can see rotational DFD displays in iris images, which have no gaps and no overlaps between neighboring iris images. And so 360-deg smooth motion parallax is obtained by moving along in surrounded iris imaging planes which are the best viewing positions. Figure 27 shows in detail many optical beam lines and arrows to explain imaging. These many optical beam lines and arrows obstruct the optical system and displays from view. In order to show the optical system clearly at a glance, we omit many optical beam lines and arrows in Fig. 27, as shown in Fig. 28, which is simple and makes the optical system easy to understand.

 

Fig. 28. Upper view of the 360-deg multiview DFD display by a magnified spatially imaged iris method without optical beam lines and arrows.

Download Full Size | PDF

5. EXPERIMENTS AND RESULTS

Our final target is a 360-deg rotational multiview DFD display by a magnified spatially imaged iris method as shown in Fig. 27.

For preparing this display, we constructed a two-view rotational multiview DFD display by a magnified spatially imaged iris method as shown in Fig. 29.

 

Fig. 29. Two-view rotational multiview DFD display by a magnified spatially imaged iris method, which is the structure of the experimental setup.

Download Full Size | PDF

We used display optics, which have rotational symmetricity, to avoid distortion for oblique angle observation and to make possible a 360-deg display. Figure 30 shows the optical setup of our ball-type display.

 

Fig. 30. Experimental setup of the two-view rotational multiview DFD display.

Download Full Size | PDF

It consists of an acrylic ball, projection lenses, half-mirrors, and displays. The focal length of the ball ($f_{\mathrm{BALL}}$) is expressed by Eq. (16). In this experiment, we used an acrylic ball $(n_2\approx 1.5)$ with $f_{\mathrm{BALL}}=1.5r$. Since the ball has radial symmetricity, it is suitable for 360-deg view applications. We used small iPod Touch displays as image sources and projection lenses to transmit the rays from the display to the center of the ball and to produce a floating 3D image. The ball lens delivers the rays from the projection lenses to the observers’ viewpoints with angular selectivity. The projection lenses are placed without spacing by cutting the side edges linearly, so that the lens aperture images generated by the crystal ball lens also have no spacing between them. The relationship between the distance of the projection lens $(b+r)$ and the observing position $(c+r)$ from the center of the ball can be described by Eq. (15). To position the floating image at the center of the ball, the distance $(a)$ between the display screen and the projection lens, the distance $(b+r)$ between the projection lens and the center of the ball, and the focal length $(f_{\mathrm{PRJ}})$ of the projection lens should satisfy the relationship described in Eq. (17). The size of the floating image is $(b+r)/a$ times larger than that of the displayed image. With larger magnification, the projection optics become smaller and the system can be made smaller. In this experiment, $f_{\mathrm{PRJ}}$ was 100 mm and distance $(a)$ was set at 200 mm. By Eq. (17), $b+r$ became 200 mm and the magnification ratio $[(b+r)/a]$ was unity. The radius $r$ of the acrylic ball lens was 100 mm. And so the focal length $f_{\mathrm{BALL}}$ became 150 mm using Eq. (16). Under the conditions $b+r=200\mathrm{mm}$ and $f_{\mathrm{BALL}}=150\mathrm{mm}$, $c+r$ became 600 mm using Eq. (15). The magnification ratio $(c+r)/(b+r)$ of the size of the iris image to that of the projection lens is 3. The spacing $\mathrm{\Delta }{a}_1+\mathrm{\Delta }{a}_2$ between the front and rear screens was 10 mm. In order to locate the center of the depth range $\mathrm{\Delta }{b}_1+\mathrm{\Delta }{b}_2$ of the DFD display at the center of the ball lens, the condition $\mathrm{\Delta }{b}_1=\mathrm{\Delta }{b}_2$ must be satisfied. Under the conditions $\mathrm{\Delta }{b}_1=\mathrm{\Delta }{b}_2$ and $\mathrm{\Delta }{a}_1+\mathrm{\Delta }{a}_2=10\mathrm{mm}$, by solving Eqs. (18) and (19), $\mathrm{\Delta }{b}_1=\mathrm{\Delta }{b}_2=4.988\mathrm{mm}$, $\mathrm{\Delta }{a}_1=5.25\mathrm{mm}$, and $\mathrm{\Delta }{a}_2=4.75\mathrm{mm}$ were obtained. Consequently, the condition $b+a+\mathrm{\Delta }{a}_1=305.25\mathrm{mm}<c=500\mathrm{mm}$ was satisfied. When b + a + Δa₁ < c is true, there are no optical system in the outer space of the encircled iris images as shown in Fig. 27.

The optical paths for a viewing zone are shown in Fig. 29. By using a magnified spatially imaged iris method, the viewing area is restricted. In this experiment, the irises of the projection lenses were the same as the apertures of the lenses. All the rays from the image source pass through the iris uniformly and reach the iris image of the projection lens at the observer’s position. Therefore, the observer can only see the displayed image when he/she is in the iris image and directional-selective display can be achieved.

However, this method can connect the viewing zones without spacing and with sharp changes. When the viewpoint of an observer moves horizontally, he/she will find discrete changes in the image at the boundary between two lens images (i.e., viewing zones). To connect two adjacent viewing zones smoothly, we used the visual effects of rotational multiview DFD displays. To make a stacked screen image, we used two screens and a half-mirror for each viewing zone. We used CG images [14] to evaluate our display. We rendered them at the centers of the iris images for each DFD display. Using depth maps, we distributed the luminance to the front and rear screens to induce a DFD visual effect.

Figure 31 shows the displayed images taken at different viewing points.

 

Fig. 31. Displayed Images from different viewing positions. (a) Horizontal motion parallax. Left face of a yellow dice shows significant change. (b) Vertical motion parallax. Top face of the yellow dice shows significant change.

Download Full Size | PDF

The use of a screen-less configuration produced clear images. As can be seen in Fig. 31, the direction of the images changes smoothly, so that smooth motion parallax can be provided in horizontal [Fig. 31(a)] and vertical [Fig. 31(b)] directions. The cross angle of the overlapping axes was 14 deg. This means that we need only 25 projectors to cover a 360-deg viewing area, and vertical motion parallax can be also produced. In order to realize our final target of a 360-deg rotational multiview DFD display by a magnified spatially imaged iris method, as shown in Fig. 27, we extended the two-view rotational multiview DFD display as shown in Fig. 30 to a 360-deg display.

Figure 32 shows the experimental setup without display panels.

 

Fig. 32. Experimental setup without display panels of a 24 directional 360-deg rotational multiview DFD display.

Download Full Size | PDF

This experimental setup has a 24 directional 360-deg rotational multiview DFD display. The cross angle of the overlapping axes was 15 deg.

Figure 33 shows the accomplished 360-deg rotational multiview DFD display with 48 display panels.

 

Fig. 33. 24 directional 360-deg rotational multiview DFD display by a magnified spatially imaging iris method, which shows a swimming goldfish in a crystal ball using 48 synchronized moving images of the goldfish at different angles (see Visualization 1).

Download Full Size | PDF

We used 48 smartphone displays as image sources. This smartphone—Google NEXUS 5X—is produced by LG Company. The LCD mode is IPS (in-plane switching). The display size is 5.2 inch diagonally, with pixel number of $1920\times 1080$ full HD, pixel pitch of 60.0 μm, and resolution of 423 p.p.i. (pixels per inch). The luminance of the device is $490\mathrm{cd}/{\mathrm{m}}^2$. In the ball lens, 24 rotational multiview DFD floating moving images of a goldfish are superimposed every 15 deg. All 48 displayed moving images of the DFD are synchronized. By moving along in surrounded iris imaging planes, which are the best viewing position, as shown in Fig. 27, 360-deg smooth motion parallax of the moving gold fish is obtained. Observers could see as if a real gold fish were swimming in a spherical gold fish bowl.

Figure 34 shows a front 3D image assigned 0 deg of a goldfish in a crystal ball lens and 3D images rotated left by $-15\mathrm{deg}$, left by $-30\mathrm{deg}$, right by $+15\mathrm{deg}$, and right by $+30\mathrm{deg}$ of the goldfish in the crystal ball lens. Clear rotated 3D images of the rotated goldfish at the center of the crystal ball lens were obtained.

 

Fig. 34. 3D goldfish images in five different directions in a crystal ball lens. (a) Front 0 deg, (b) left $-15\mathrm{deg}$, (c) left $-30\mathrm{deg}$, (d) right $+15\mathrm{deg}$, and (e) right $+30\mathrm{deg}$ (see Visualization 1).

Download Full Size | PDF

From 0 deg to 360 deg, 3D gold fish images every 15 deg in 24 different directions are shown in Fig. 35. 360-deg smooth motion parallax in horizontal direction was achieved successfully.

 

Fig. 35. 360-deg smooth motion parallax of 3D goldfish images from 0 deg to 360 deg (see Visualization 1).

Download Full Size | PDF

6. CONCLUSIONS

In conclusion, we have developed a method for displaying clear floating images in a crystal ball. Its symmetric optics can provide clear and natural 360-deg images with smooth motion parallax in horizontal and vertical directions.

Since the optical system is symmetric for rotation, it can maintain high quality for 360-deg observation. This display does not require a screen at the place where the images are displayed. Therefore, the images are clear and the feeling of distortion can be reduced. It also allows the use of 3D display engines as image sources to improve 3D naturalness. Using a rotational multiview DFD display configuration, it can provide smooth motion parallax in not only the 360-deg horizontal direction, but also the vertical direction using a small number of image sources. Rays from the display engines are well controlled, and optical efficiency is very so high that ordinary-view-type displays can be used as display engines.