CN110784693A - Projector calibration method and projection system using this method - Google Patents

Abstract

The invention provides a projector correction method, which is used for imaging a spherical target image at a viewing point. The projector correction method comprises the following steps: projecting a first preset image to a dome screen by a first projector, and shooting the dome screen by a first camera to obtain a first deformed image; calculating a spatial relationship among the first projector, the first camera and the dome screen according to the first preset image and the first deformation image; generating a first front deformation image according to the target image, the spatial relationship between the first projector and the spherical screen and the spatial relationship between the spherical screen and the viewing point; and projecting by the first projector according to the first pre-distorted image. In addition, a projection system using the method is also provided. The invention can prevent the image projected on the spherical screen from being distorted, and ensure that a viewer has good viewing quality.

Description

Projector calibration method and projection system using this method

technical field

The present invention relates to a projection method, and more particularly, to a projector calibration method for spherical projection and a projection system using the method.

Background technique

The projection splicing and fusion technology of multiple projectors can adapt to the needs of large-screen display, especially for projection screens with different geometric shapes such as arcs, spheres, curved surfaces, columns or large planes, so as to achieve special projection effects. Among the projection screens of various shapes, the spherical projection screen is the most wrapping, and the viewer can see images with high resolution and a wide range of viewing angles at a specific position, with an immersive virtual reality experience . However, when the image is projected onto the spherical projection screen, complex distortion will occur, so how to correct it in advance has become a subject that needs to be studied.

Most of the current technologies use hardware devices such as fisheye lenses, convex mirrors, aspherical mirrors or spherical mirrors to pre-distort the image so that the viewer can see the undeformed image.

However, the premise of using a hardware device to achieve the above purpose is that the relative arrangement of the projector, the projection screen and the hardware device must be fixed, so it is difficult to expand the number of projectors in practical applications, and the projector cannot be arbitrarily changed. location, resulting in limited space available. In addition, the light projected from the projector will be reflected or refracted by the hardware device before being projected onto the projection screen, resulting in the final brightness attenuation.

The "Background Art" paragraph is only used to help understand the content of the present invention, so the content disclosed in the "Background Art" paragraph may contain some that do not constitute the conventional technology known to those skilled in the art. The content disclosed in the "Background Art" paragraph does not represent the content or the problem to be solved by one or more embodiments of the present invention, and has been known or recognized by those skilled in the art before the application of the present invention.

SUMMARY OF THE INVENTION

The present invention provides a projector calibration method and a projection system using the method, which can prevent the image projected on the spherical screen from being distorted, and provide the viewer with good viewing quality, convenience and expandability.

Other objects and advantages of the present invention can be further understood from the technical features disclosed in the present invention.

To achieve one or part or all of the above objectives or other objectives, an embodiment of the present invention provides a projection system including a first projector, a spherical screen and a processor. The first projector projects a spherical target image. The dome receives spherical target images. a processor, coupled to the first projector, and configured to generate a first front deformed image according to the target image, the spatial relationship between the first projector and the dome, and the spatial relationship between the dome and the viewing point , wherein the first projector projects according to the first pre-distorted image, so that the spherical target image is imaged at the viewing point, wherein the distance between the position of the viewing point and the spherical center of the spherical screen is at least greater than or equal to the spherical radius of the spherical screen /tan(fov/2) distance, fov is the viewing angle.

In order to achieve one or part or all of the above objectives or other objectives, an embodiment of the present invention provides a projector calibration method for imaging a spherical target image at a viewing point. The projector calibration method includes: projecting a first preset image to a dome by a first projector, and using a first camera to photograph the dome to obtain a first deformed image; according to the first preset image and the first deformed image , calculate the spatial relationship between the first projector, the first camera and the dome; generate the first a front distorted image; and projecting by the first projector according to the first front distorted image.

Based on the above, the embodiments of the present invention have at least one of the following advantages or effects. Regardless of the number of projectors, the projector calibration method proposed in the embodiments of the present invention and the projection system using the above method do not need to know the displacement and angle of the projector relative to the dome in advance, and the projector can be calibrated with the assistance of the camera, so that the The viewer can see the target image without deformation, which is highly convenient and expandable. In addition, in a projection system with multiple projectors, the embodiment of the present invention also proposes a method for correcting the brightness of each projector, so that even if the projection ranges of multiple projectors overlap, the brightness of the target image can be maintained, thereby providing good viewing experience.

In order to make the above-mentioned features and advantages of the present invention more obvious and easy to understand, the following embodiments are given and described in detail with the accompanying drawings as follows.

Description of drawings

FIG. 1A is a schematic diagram of a projection system according to an embodiment of the present invention.

FIG. 1B is a schematic diagram of a projection system according to another embodiment of the present invention.

FIG. 1C is a schematic diagram of a projection system according to another embodiment of the present invention.

FIG. 2 is a flowchart of a projector calibration method according to an embodiment of the present invention.

FIG. 3A is a schematic diagram of a preset image according to an embodiment of the present invention.

FIG. 3B is a schematic diagram of a preset image of the first dome screen according to an embodiment of the present invention.

FIG. 3C is a schematic diagram of a first deformed image according to an embodiment of the present invention.

FIG. 3D is a schematic diagram illustrating a preset image of the second dome screen according to an embodiment of the present invention.

FIG. 3E is a schematic diagram of a second deformed image according to an embodiment of the present invention.

FIG. 3F shows the fundamental matrix between the projector and the camera in an embodiment of the present invention.

FIG. 4 is a schematic structural diagram of a projection system of a plurality of projectors according to an embodiment of the present invention.

FIG. 5 is a schematic diagram of a target image and a spherical target image according to an embodiment of the present invention.

FIG. 6 is a schematic diagram of a first front deformed image according to an embodiment of the present invention.

FIG. 7A is a schematic diagram of a first partial image according to an embodiment of the present invention.

FIG. 7B is a schematic diagram of a second partial image in an embodiment of the present invention.

FIG. 7C is a schematic diagram of a first pre-distorted image according to an embodiment of the present invention.

FIG. 7D is a schematic diagram of a second front distorted image according to an embodiment of the present invention.

FIG. 8 is a schematic diagram illustrating a brightness calibration method according to an embodiment of the present invention.

FIG. 9 is a schematic diagram of adjusting the brightness of the pre-distorted image according to an embodiment of the present invention.

Detailed ways

The foregoing and other technical contents, features and effects of the present invention will be clearly presented in the following detailed description of a preferred embodiment with reference to the drawings. The directional terms mentioned in the following embodiments, such as: up, down, left, right, front or rear, etc., are only for referring to the directions of the attached drawings. Accordingly, the directional terms used are illustrative and not limiting of the present invention. In addition, terms such as connection and coupling used in the present invention describe the transmission of signals between two devices, and are not limited to the relationship of direct contact between the devices.

In the embodiment of the present invention, the spherical target image to be seen by the viewer is prewarped into a prewarped image, so that the projector projects the prewarped image onto a spherical projection screen (also called a spherical screen). After viewing, the viewer can see the spherical target image without deformation at the viewing point. In order to achieve the above purpose, spatial information such as the relative angle and displacement between the projector and the dome must be obtained. Therefore, the embodiment of the present invention uses a camera as an auxiliary device, the projector projects the preset image behind the dome screen, and then uses the camera to capture the preset image imaged on the dome screen to obtain the corresponding deformed image. Through the preset image and the deformed image, the spatial information between the projector and the dome can be calculated. According to this, even if the position or the number of projectors is changed, it is only necessary to calculate the spatial information again.

In some embodiments, the projection system uses a projector to project the image. As shown in FIG. 1A , the projection system PS1 includes a first projector PJT1 , a first camera CAM1 and a processor PSR, wherein the first projector PJT1 and the first camera CAM1 are both coupled to the processor PSR. In addition, the image output device S is coupled to the first projector PJT1 and the first camera CAM1, wherein the processor PSR may be disposed in the image output device S, but not limited thereto, in other embodiments, the processor PSR may also be Set in the first projector PJT1. In other embodiments, the first projector PJT1 and the first camera CAM1 may be integrated into a projector with a camera, and it is not limited that the first projector PJT1 and the first camera CAM1 are two physically separate devices. The image output device S is used to provide an image to the first projector PJT1. The first projector PJT1 and the first camera CAM1 are both set corresponding to the dome screen D, that is, when the first projector PJT1 projects an image toward the dome screen D, the first camera CAM1 can capture the deformation of the dome screen D. image. In addition, the observer can be located at a viewing point v to watch the image projected by the first projector PJT1 to the dome screen D, and the present invention does not limit the position of the viewing point v here.

In some embodiments, the projection system uses multiple projectors to project images. As shown in FIG. 1B, the projection system PS2 includes a first projector PJT1, a second projector PJT2, a first camera CAM1 and a processor PSR, wherein the first projector PJT1, the second projector PJT2 and the first camera CAM1 are all coupled Connected to the processor PSR. In addition, the image output device S is coupled to the first projector PJT1, the second projector PJT2 and the first camera CAM1, wherein the processor PSR may be disposed in the image output device S, but not limited thereto, in other embodiments , the processor PSR may also be disposed in the first projector PJT1 or the second projector PJT2. In other embodiments, the first projector PJT1 and the first camera CAM1 may be integrated into a projector with a camera, and it is not limited that the first projector PJT1 and the first camera CAM1 are two physically separate devices. The image output device S is used for providing images to the first projector PJT1 and the second projector PJT2. The first projector PJT1, the second projector PJT2 and the first camera CAM1 are all set corresponding to the dome screen D. When the first projector PJT1 and the second projector PJT2 jointly project an image to the dome screen D, the first camera CAM1 can shoot images. to the deformed image presented on Dome D. In addition, the observer can be located at a viewing point v to watch the spliced image projected by the first projector PJT1 and the second projector PJT2 to the dome D, and the present invention does not limit the position of the viewing point v here.

In some embodiments, when the projection system uses multiple projectors to project images, the field of view of one camera may not cover the total projection range of the multiple projectors, so one camera is provided for each projector. As shown in FIG. 1C, the projection system PS3 includes a first projector PJT1, a second projector PJT2, a first camera CAM1, a second camera CAM2 and a processor PSR, wherein the first projector PJT1, the second projector PJT2, the first Both a camera CAM1 and a second camera CAM2 are coupled to the processor PSR. In addition, the image output device S is coupled to the first projector PJT1, the second projector PJT2, the first camera CAM1 and the second camera CAM2, wherein the processor PSR may be disposed in the image output device S, but not limited thereto, In other embodiments, the processor PSR may also be disposed in the first projector PJT1 or the second projector PJT2. In other embodiments, the first projector PJT1 and the first camera CAM1 can be integrated into a projector with a camera, and the second projector PJT2 and the second camera CAM2 can be integrated into a projector with a camera. A projector PJT1 and a first camera CAM1 are two physically separate devices and a second projector PJT2 and a second camera CAM2 are two physically separate devices. The image output device S is used for providing images to the first projector PJT1 and the second projector PJT2. The first projector PJT1, the second projector PJT2, the first camera CAM1 and the second camera CAM2 are all set corresponding to the dome screen D. When the first projector PJT1 projects the first image to the dome screen D, the first camera CAM1 can shoot to the deformed first image presented on the dome D, and when the second projector PJT2 projects the second image to the dome D, the second camera CAM2 can capture the deformed second image presented on the dome D . In addition, the observer can be located at a viewing point v to watch the spliced image projected by the first projector PJT1 and the second projector PJT2 to the dome D, and the present invention does not limit the position of the viewing point v here.

Although the projection system is described with the combination of the above three projectors and cameras, the present invention does not limit the numbers of projectors and cameras in the projection system, and the numbers of the two can be the same or different.

In some embodiments, the projector and the camera in the projection system may be implemented by, for example, being integrated and packaged into one projection device, or implemented as separate devices, which are not limited in the present invention.

It is worth mentioning that in the following description, the imaging principle of the pinhole camera model is applied to the projector and the camera in the embodiment of the present invention, that is, the projector and the camera are assumed to be individual pinholes. camera model. However, the present invention is not limited to this. In other words, both projectors and cameras have intrinsic parameters, including intrinsic projection matrices and lens distortion coefficients. In detail, the internal projection matrix includes information about the focal length of the lens and the coordinates of the center of the screen. For example, the projection matrix K is in the form of [fx, 0, cx; 0fy cy; 00 1], the focal lengths of fx and fy lenses Coefficients, cx and cy are the screen center coefficients. Those skilled in the art can clearly know the above calculation method, which is not repeated here. The lens distortion coefficient includes the radiation distortion parameter (radial distortion) and the tangential distortion parameter (tangential distortion). The lens distortion coefficient is the parameter that the lens produces image distortion. The image distortion is, for example, a pincushion distortion image or barrel. Image of Barrel Distortion. After fixing the focal length and the scale of the projected image, the intrinsic parameters do not change. Those skilled in the art can obtain the relevant introduction of the intrinsic parameters from the pinhole camera model, so they will not be repeated here.

In some embodiments, the intrinsic parameters of the projector can be obtained, for example, by using a projector-camera calibration tool to obtain coordinate transformation when the zoom size and focal length of the projected image of the projector are fixed, while the intrinsic parameters of the camera can be determined, for example, in the fixed camera. The zoom size and focal length of the captured image are obtained by using a camera calibration tool, but the present invention is not limited to this. Among them, the projector-camera calibration tool can refer to the document "Projector-Camera Calibration Toolbox", G.Falcao, N.Hurtos, J.Massich, and D.Fofi, 2009 and the document "Simple, Accurate, and RobustProjector-Camera Calibration" , Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission, D. Moreno and G. Taubin. In addition, camera correction tools can refer to the services provided by Open CV, Open Source Computer Vision Library (http://opencv.org/), 2017.

The processor PSR is used for performing operations in the projector calibration method. The processor PSR can be, for example, a micro-controller, an embedded controller, a central processing unit (CPU) or the like. components, or digital signal processors (DSPs), programmable controllers, application specific integrated circuits (ASICs), programmable logic devices (PLDs) or other similar devices or these combination of devices. The present invention is not limited to this. In some embodiments, the processor PSR can be implemented by a personal computer; in other embodiments, the processor PSR can be implemented by a mobile electronic device such as a tablet computer or a smart phone, but the present invention is not limited thereto. It is worth mentioning that the image output device S may be the above-mentioned electronic device such as a personal computer, a tablet computer, or a smart phone, as an image source for providing images.

FIG. 2 is a flowchart of a projector calibration method according to an embodiment of the present invention.

Referring to FIG. 2 , in order to correct the target image projected by the projector so that the viewer can see the spherical target image without deformation on the dome screen, the camera is first used in steps S110 and S120 to construct the relationship between the projector and the dome screen. The spatial relationship (coordinate relationship) between them, such as the relative angle and displacement position between the projector and the dome screen, etc., that is, the external parameters. The so-called spatial relationship is the relationship of coordinate transformation between the two devices, and the following contents all belong to the relationship of coordinate transformation.

In step S110, the projector is used to project the preset image to the dome screen, and the camera is used to shoot the dome screen to obtain the deformed image. Specifically, the projector needs to make it project a preset image to the dome screen, and use a camera to shoot the dome screen to obtain a deformed image corresponding to the preset image. Accordingly, the preset image and the deformed image can be used to calculate the spatial relationship (coordinate relationship) between the projector, the camera and the dome.

In some embodiments, the predetermined image is, for example, a two-dimensional image, which includes a plurality of control points with known two-dimensional coordinates, wherein each control point is different, and each control point has a different pattern. In detail, the control points are feature points that can be identified by the processor PSR. On the other hand, the deformed image corresponding to the preset image includes a plurality of corresponding control points that can be recognized by the processor PSR. In other words, the corresponding control point in each deformed image corresponds to one of the control points in the preset image. In addition, the preset image can be stored in a storage device (not shown), such as a removable random access memory (RAM), a read-only memory (ROM), and a flash memory. (flash memory) or similar elements or a combination of the above elements. The storage device is connected to the processor PSR, so the storage device and the processor PSR are arranged together, so that the processor PSR can control the storage device to provide a preset image to at least one projector.

Referring to FIGS. 3A to 3C , when the first projector PJT1 projects the first predetermined image I1 to the dome D, a first predetermined spherical image I1D is formed on the dome D. At this time, the first camera CAM1 shoots the dome D and obtains the first deformed image C1. As shown in FIG. 3A , the first preset image I1 provided by the processor PSR to the first projector PJT1 is, for example, a rectangle, and includes a plurality of control points pi, wherein each control point pi can be obtained from the first preset image by the processor PSR. Let it be recognized in image I1. After the first preset image I1 is deformed by the intrinsic parameters of the first projector PJT1, the reflection angle of the dome D, and the intrinsic parameters of the first camera CAM1, the first deformed image C1 captured by the first camera CAM1 is not. It is a rectangle again, but the processor PSR can still identify a plurality of corresponding control points qi from the first deformed image C1 provided by the first camera CAM1. The first deformed image C1 and the first preset image I1 can be used to subsequently calculate the spatial relationship (coordinate relationship) between the first projector PJT1 , the camera CAM1 and the dome D.

Referring to FIGS. 3A , 3D and 3E, when the second projector PJT2 projects the second preset image I2 to the dome D, the second preset spherical image I2D will be formed on the dome D. At this time, the first camera CAM1 shoots the dome D and obtains the second deformed image C2. Similarly, the second deformed image C2 and the second preset image I2 can be used to subsequently calculate the spatial relationship between the second projector PJT2 , the first camera CAM1 and the dome D. In particular, the second preset image I2 and the first preset image I1 may be the same or different, and the invention is not limited thereto.

In step S120, the processor calculates the spatial relationship between the projector, the camera and the dome screen according to the preset image and the deformed image. The above-mentioned spatial relationship includes, for example, the relative angle and displacement between the projector and the camera, the relative angle and displacement between the projector and the spherical center of the dome, and the relative angle and displacement between the camera and the spherical center of the dome, etc., That is, the coordinate relationship between them.

In some embodiments, the calibration of one or more projectors may be accomplished by the same camera, which will be described below with reference to FIGS. 3A to 3E .

First, the processor PSR will correct the intrinsic parameters. Specifically, the processor PSR finds out a plurality of control points pi of the first preset image I1, and converts the control points pi by using the intrinsic parameters of the first projector PJT1. In addition, the processor PSR finds out a plurality of corresponding control points qi of the first deformed image C1, and uses the intrinsic parameters of the first camera CAM1 to restore the corresponding control points qi.

Referring to FIG. 3F , with the first camera CAM1 as the world coordinate center, the processor PSR can calculate the distance between the first projector PJT1 and the first camera CAM1 according to the converted control point pi and the restored multiple corresponding control points qi The fundamental matrix F (fundamental matrix), and then through the equation E=KiTFKc can calculate the essential matrix E (essential matrix), where Ki is the intrinsic projection matrix of the first projector PJT1, Kc is the intrinsic projection matrix of the first camera CAM1 Projection matrix. The relative angle and displacement direction between the first projector PJT1 and the first camera CAM1 can be obtained by decomposing the essential matrix E between the two, so calculating the essential matrix E is equivalent to obtaining the first projector PJT1 and the first The relative angle and displacement direction between cameras CAM1. It is worth mentioning that those skilled in the art can obtain the description of the fundamental matrix F and the essential matrix E from the relevant literature, so they will not be repeated here.

In order to obtain the absolute displacement between the first projector PJT1 and the first camera CAM1, the processor PSR, for example, according to the above-mentioned relative angle and displacement direction, uses triangulation and other methods to try to transform the plurality of deformed The control point pi and the corresponding restored multiple corresponding control points qi are restored to multiple three-dimensional coordinate points in the three-dimensional space. According to the 3D coordinate points restored at this time, the 3D coordinate points should fit the dome D. The processor PSR can use methods such as the least squares method to make these 3D coordinate points approximate a spherical equation, and then match the known radius of the dome D to get The spherical center coordinates of the spherical screen D can further calculate the scale of the displacement between the first projector PJT1 and the first camera CAM1 to obtain the absolute displacement. That is, the extrinsic parameters between the first projector PJT1 and the first camera CAM1 are obtained, wherein the extrinsic parameters include the absolute displacement.

According to the method in the preceding paragraph, after obtaining the first preset image I1 and the first deformed image C1, the processor PSR can obtain the distance between the first projector PJT1, the first camera CAM1 and the dome D through calculation. Spatial Relations.

In addition, the spatial relationship between the second projector PJT2, the first camera CAM1, and the dome screen D in FIG. 3D can also be obtained in a manner similar to that described above. In this way, the processor PSR can obtain the spatial relationship between the first projector PJT1 , the second projector PJT2 and the dome screen D in pairs.

In some embodiments, the calibration of multiple projectors can also be accomplished through multiple corresponding cameras. The following description will be given with the architecture of FIG. 4 , which is similar to the projection system PS3 in FIG. 1C .

First, the processor PSR will use the method in the preceding paragraph to obtain the spatial relationship between the first projector PJT1, the first camera CAM1 and the dome D, and the second projector PJT2, the second camera CAM2 and the dome screen, respectively. D Spatial relationship between pairs. Specifically, the first projector PJT1 will project the first preset image to the dome screen D, and the first camera CAM1 will shoot the dome screen D to obtain a first deformed image corresponding to the first preset image, and the processor PSR will The spatial relationship between the first projector PJT1 , the first camera CAM1 and the dome D is calculated according to the first preset image and the first deformed image. The second projector PJT2 will project a second preset image to the dome D, and the second camera CAM2 will shoot the dome D to obtain a second deformed image corresponding to the second preset image, and the processor PSR will according to the second preset image The spatial relationship between the second projector PJT2 , the second camera CAM2 and the dome D is calculated by setting the image and the second deformed image.

Next, the processor PSR calculates the spatial relationship between the first camera CAM1 and the second camera CAM2. For example, the first camera CAM1 and the second camera CAM2 can be made to shoot the third preset image displayed on the dome D at the same time, so as to obtain the third deformed image and the fourth deformed image respectively. Then the processor PSR can calculate the spatial relationship between the first camera, the second camera and the dome D according to the third deformed image and the fourth deformed image in a manner similar to the previous paragraph, and the first camera is obtained. Spatial relationship with the second camera. In some embodiments, the above-mentioned third preset image may be generated by, for example, the first projector PJT1 or the second projector PJT2, or generated on the dome D in other ways, or the dome D itself. patterns, etc., the present invention does not limit the way of generating the third preset image.

In this way, the processor PSR obtains the spatial relationship between the first projector PJT1, the first camera CAM1 and the dome screen D, and the relationship between the second projector PJT2, the second camera CAM2 and the dome screen D. After obtaining the spatial relationship between the first camera and the second camera, the spatial relationship between the first projector PJT1, the second projector PJT2 and the dome D can be obtained.

After obtaining the spatial relationship between the projector and the dome, the processor PSR can calculate the image at any viewing point after a specific image is input to the projector and projected by the projector. Likewise, the processor PSR can also calculate the image that must be input to the projector in order to image the target image at a specific viewing point.

In step S130, the processor generates a pre-distorted image according to the target image, the spatial relationship between the projector and the dome, and the spatial relationship between the dome and the viewing point, and in step S140, the projector generates a pre-distorted image according to the Projection is performed by pre-warping the image so that the spherical target image is imaged at the viewing point.

In some embodiments, the spatial relationship between the dome screen and the viewing point is fixed and preset in the processor, that is, the projection system is provided with an optimal viewing position as the viewing point. In some embodiments, the spatial relationship between the dome screen and the viewing point may be pre-set by the viewer according to its position. In some embodiments, the spatial relationship between the dome screen and the viewing point can also be obtained by using more than two cameras using triangulation. In other words, as long as the processor can obtain the spatial relationship between the dome screen and the viewing point before performing step S130, the present invention does not limit the actual spatial relationship between the dome screen and the viewing point and its obtaining method.

It is worth mentioning that in some embodiments, it is assumed that the viewer at the viewing point also applies the imaging principle of the pinhole camera model. Therefore, the processor will set the intrinsic parameters corresponding to the viewing point to calculate the imaging of the viewing point. For example, the focal length information in the internal projection matrix can be determined according to the viewing angle of the human eye, the image center information can be determined according to the size of the target image, and the lens distortion coefficient is set to 0, for example. In some embodiments, the intrinsic parameters of the corresponding viewing point may be preset by the processor or set by the viewer according to their needs at the moment of viewing (for example, through an input device such as a remote control), and the present invention does not this limit.

In detail, after the spatial relationship between the dome screen and the projector is determined, the viewer's spatial relationship (coordinate relationship) can be determined by themselves. It can be a self-defined virtual coordinate, or it can be calculated by using the spatial position of the dome screen. The optimal position of a viewing point can be set at a distance L translated from the spherical center position DD (not shown) of the spherical screen D to the outside of the spherical screen D, wherein the distance L is at least greater than or equal to the spherical radius of the spherical screen D / tan(fov/2), the viewing range of the viewer includes the entire spherical screen D, where fov represents the viewing angle of the viewer, for example, the viewing angle fov=150 degrees, but not limited thereto. If the radius of the dome is, for example, 1200 millimeters (mm), the distance L from the optimum position of the viewing point is at least 322 millimeters (mm).

Referring to FIG. 5 , in some embodiments, when the processor PSR wants to image the spherical target image TD at the viewing point, it first restores the target image T by using the intrinsic parameters of the corresponding viewing point, and then restores the restored target image TD. The image T calculates the spherical spherical image TD on the dome D according to the spatial relationship between the dome D and the viewing point. In other words, if the image projected by the projector can image the spherical target image TD on the spherical screen D, the spherical target image TD can be imaged at the viewing point. In addition, the target image T is provided by the image output device S to the processor PSR.

First, take the projection system PS1 of a projector as an example, the processor PSR will calculate the value based on the spherical target image TD, the spatial relationship between the first projector PJT1 and the spherical screen D, and the intrinsic parameters of the first projector PJT1. The first front deformed image W1 shown in FIG. 6 . After the processor PSR inputs the calculated first pre-distorted image W1 to the first projector PJT1 for projection, a first spherical image that is the same as the spherical target image TD will be imaged on the spherical screen D. Therefore, A viewer at the viewing point can see the spherical target image TD.

Next, taking the projection system PS2 or PS3 of multiple projectors as an example, as shown in FIGS. 7A and 7B , the processor PSR first determines the space between the first projector PJT1 and the dome D according to the specifications of the first projector PJT1 and the space between the first projector PJT1 and the dome D. relationship, calculate the first projection range that the first projector PJT1 can project on the dome screen, according to the specifications of the second projector PJT2 and the spatial relationship between the second projector PJT2 and the dome screen D, calculate The second projection range that the projector PJT2 can project on the dome screen. Next, the processor PSR selects the part of the spherical target image TD located in the first projection range as the first partial image TD1, and selects the spherical target image TD located in the first partial image TD1 according to the second projection range of the second projector PJT2 on the spherical screen. The part of the two projection ranges is used as the second partial image TD2.

Then, the processor PSR will calculate the first front deformation as shown in FIG. 7C according to the first partial image TD1, the spatial relationship between the first projector PJT1 and the dome D, and the intrinsic parameters of the first projector PJT1 Image W1. In addition, the processor PSR will calculate the second front image as shown in FIG. 7D according to the second partial image TD2, the spatial relationship between the second projector PJT2 and the dome D, and the intrinsic parameters of the second projector PJT2 Deformed image W2.

After the processor PSR inputs the calculated first pre-distorted image W1 to the first projector PJT1 for projection, a first spherical image identical to the first partial image TD1 will be imaged on the spherical screen D. After the processor PSR inputs the calculated second pre-distorted image W2 to the second projector PJT2 for projection, a second spherical image identical to the second partial image TD2 will be imaged on the spherical screen D.

In some embodiments, the first projection range of the first projector PJT1 on the dome screen D and the second projection range of the second projector PJT2 on the dome screen D have an overlapping area, so the processor PSR can, for example, first A front anamorphic image W1 and a second front anamorphic image W2 are subjected to brightness adjustment, and then the adjusted first front anamorphic image W1' is input to the first projector PJT1 for projection, and the adjusted first front anamorphic image W1' is then projected. The two front anamorphic images W2' are input to the second projector PJT2 for projection. In this way, the same synthetic spherical image as the spherical target image TD can be imaged on the dome screen D, wherein the brightness of the synthetic spherical image corresponding to the overlapping area is the same as the brightness of the spherical target image TD corresponding to the overlapping area.

The method of adjusting the brightness of the first pre-distorted image W1 and the second pre-distorted image W2 is exemplified below with reference to FIG. 8 and FIG. 9 . It is worth mentioning that although the described method for adjusting the brightness uses two projectors as an example, a similar method can also be applied to an embodiment of a projection system with more projectors.

Referring to FIG. 8 , taking the projection system PS2 of the first projector PJT1 and the second projector PJT2 as an example, the processor PSR will first determine the first partial image TD1 , the spatial relationship between the viewing point and the dome, and the corresponding viewing point according to the first partial image TD1 . After the intrinsic parameters, the processor PSR establishes the first two-dimensional mask M1, and the part of the first two-dimensional mask M1 with a set value of 1 (white) represents the image area of the first part of the image TD1 at the viewing point. The part of the two-dimensional mask M1 with the set value of 0 (black) represents the non-image area when the first partial image TD1 is imaged at the viewing point. Similarly, the processor PSR also establishes a second two-dimensional mask M2 according to the second partial image TD2 and the spatial relationship between the viewing point and the dome.

Then, the processor PSR performs distance transform on the first two-dimensional mask M1 and the second two-dimensional mask M2 respectively, so as to produce a gradient effect on the boundary of the image area, so as to obtain the first luminance mask D1 and The second luminance mask D2. The method of distance transform is that the position of the pixel (pixel) with the set value of 1 (white) in the first two-dimensional mask M1 is distanced from the set value of 0 (black) in the first two-dimensional mask M1 The distance of the pixel (pixel) position makes the effect of brightness gradient. That is, the position of the pixel (pixel) with the set value of 1 (white) in the first two-dimensional mask M1 is farther from the position of the pixel (pixel) with the set value of 0 (black) in the first two-dimensional mask M1. The brighter the brightness, and vice versa, the closer the brightness is, the darker the brightness.

After obtaining the first brightness mask D1 and the second brightness mask D2, the processor PSR normalizes/normalizes the first brightness mask D1 and the second brightness mask D2 according to the following equation, for example:

where n is the number of luminance masks to be normalized. Accordingly, the processor PSR can obtain the normalized first luminance mask D1' (j=1, n=2) and the second luminance mask D2' (j=2, n=2).

Then, by converting the target image into a pre-distorted image in the projection system PS1 similar to the aforementioned one projector, the processor PSR will mask the first brightness D1 ′ according to the first projector PJT1 , viewing The spatial relationship between the points and the dome D is converted into the first pre-distorted luminance mask A1. On the other hand, the processor PSR converts the second brightness mask D2' into a second pre-distorted brightness mask A2 according to the spatial relationship between the second projector PJT2, the viewing point and the dome D.

Please refer to FIG. 7C , FIG. 8 and FIG. 9 , the processor PSR uses the first pre-distorted brightness mask A1 to adjust the brightness of the first pre-distorted image W1 to obtain the first pre-distorted image W1 after the brightness is adjusted '. For example, the processor PSR may multiply the first pre-distorted image W1 by each corresponding pixel value of the first pre-distorted luminance mask A1 to obtain the adjusted first pre-distorted image W1'. In this way, after the first projector PJT1 projects the first pre-distorted image W1', the first spherical image W1D' after the brightness is adjusted will be imaged on the dome screen D.

Referring to FIG. 7D , FIG. 8 and FIG. 9 , the processor PSR uses the second pre-distorted brightness mask A2 to adjust the brightness of the second pre-distorted image W2 to obtain a second pre-distorted image W2 with the adjusted brightness '. For example, the processor PSR may multiply the second pre-distorted image W2 by each corresponding pixel value of the second pre-distorted luminance mask A2 to obtain the adjusted second pre-distorted image W2'. In this way, after the second projector PJT2 projects the second front anamorphic image W2', the second spherical image W2D' after adjusting the brightness will be imaged on the dome screen D.

When the first projector PJT1 and the second projector PJT2 project the first front anamorphic image W1 ′ and the second front anamorphic image W2 ′ respectively at the same time, the first front anamorphic image W1 ′ and the second front anamorphic image W2 'Partially overlapping, a synthetic spherical image identical to the spherical target image TD will be imaged on the dome screen D, wherein the spherical target image TD has a uniform brightness distribution.

It is worth mentioning that although two projectors are used as an example in the projector calibration method described in the foregoing embodiments, a similar method can also be applied to an embodiment of a projection system with more projectors. In other words, the present invention does not limit the number of projectors in the projection system as long as there are corresponding cameras to assist in the correction.

To sum up, the embodiments of the present invention have at least one of the following advantages or effects. Regardless of the number of projectors, the projector calibration method proposed in the embodiments of the present invention and the projection system using the above method do not need to know the displacement and angle of the projector relative to the dome in advance, and the projector can be calibrated with the assistance of the camera, so that the The viewer can see the spherical target image without deformation, which is highly convenient and expandable. In addition, in a projection system with multiple projectors, the embodiment of the present invention also proposes a method for correcting the brightness of each projector, so that even if the projection ranges of the multiple projectors overlap, the brightness of the spherical target image can be maintained, thereby providing good viewing experience.

The above are only preferred embodiments of the present invention, and should not limit the scope of the present invention, that is, any simple equivalent changes and modifications made according to the claims of the present invention and the contents of the description of the invention are still applicable. It belongs to the scope covered by the patent of the present invention. Furthermore, any embodiment or claim of the present invention is not required to achieve all of the objects or advantages or features disclosed herein. In addition, the abstract part and the title of the invention are only used to assist the retrieval of patent documents, not to limit the scope of rights of the present invention. In addition, terms such as "first" and "second" mentioned in this specification or the claims are only used to name the elements, and are not used to limit the upper or lower limit of the number of elements.

reference number

A1: 1st front deformation luminance mask

A2: 2nd front deformed luminance mask

C1: First deformed image

C2: Second anamorphic image

CAM1: first camera

CAM2: Second camera

D: Dome screen

DD: the center of the dome

D1: first luminance mask

D1': Normalized first luminance mask

D2: Second luminance mask

D2': normalized second luminance mask

I1: The first preset image

I2: Second preset image

I1D: The first preset spherical image

I2D: Second preset spherical image

L: distance

M1: first 2D mask

M2: Second 2D mask

pi: control point

PJT1: The first projector

PJT2: Second projector

PS1, PS2, PS3: Projection system

PSR: Processor

qi: corresponding to the control point

S: Video output device

S110, S120, S130, S140: steps of the projector calibration method

T: target image

TD: spherical target image

TD1: The first part of the video

TD2: The second part of the video

v: viewing point

W1: First front anamorphic image

W1’: The first front anamorphic image after adjusting the brightness

W1D’: The first spherical image after adjusting the brightness

W2: Second front anamorphic image

W2’: The second front anamorphic image after adjusting the brightness

W2D’: The second spherical image after adjusting the brightness.

Claims (22)

1. A projection system, wherein the projection system comprises: the first projector, which projects a spherical target image; a dome screen that receives the spherical target image; and a processor, coupled to the first projector, and configured to, according to the target image, the spatial relationship between the first projector and the spherical screen, and the spatial relationship between the spherical screen and the viewing point, Generating a first pre-distorted image, wherein the first projector projects according to the first pre-distorted image, so that the spherical target image is imaged at the viewing point, wherein the position of the viewing point is the same as the The distance of the spherical center position of the spherical screen is at least greater than or equal to the distance of the spherical radius of the spherical screen/tan(fov/2), where fov is the viewing angle. 2. The projection system of claim 1, further comprising a first camera coupled to the processor, wherein the first projector projects a first preset image to the dome, the first A camera shoots the spherical screen to obtain a first deformed image. 3. The projection system of claim 2, wherein the processor is configured to calculate the first projector, the first camera and the Spatial relationship between domes. 4. The projection system of claim 3, wherein the processor is configured to calculate the first front deformation according to the target image and the spatial relationship between the first projector and the dome screen image, The first projector projects the first pre-distorted image to the dome to form a first spherical image on the dome, and the spherical target image is the same as the first spherical image. 5. The projection system of claim 3, wherein the processor is configured to: The spherical target image is calculated according to the target image, the spatial relationship between the spherical screen and the viewing point, and the intrinsic parameters corresponding to the viewing point. 6. The projection system of claim 1, further comprising: a second projector, coupled to the processor, wherein the processor is configured to: calculate the spherical target image according to the target image and the spatial relationship between the spherical screen and the viewing point; selecting a first partial image of the spherical target image according to the spatial relationship between the first projector and the dome and a first projection range of the first projector; and The first front deformation image is generated according to the first partial image and the spatial relationship between the first projector and the dome. 7. The projection system of claim 6, wherein the processor is further configured to: selecting a second partial image of the spherical target image according to the spatial relationship between the second projector and the dome and a second projection range of the second projector; and The second front distorted image is generated according to the second partial image and the spatial relationship between the second projector and the dome, wherein the second projector is distorted according to the second front image is projected. 8. The projection system of claim 7, wherein the second projector projects a second preset image to the dome screen, and the first camera captures the dome screen to obtain a second deformed image, wherein The processor is more used for: According to the second preset image and the second deformed image, the spatial relationship between the second projector, the first camera and the dome is calculated. 9. The projection system of claim 7, wherein the first projection range and the second projection range have an overlapping area on the spherical screen, and the processor is further configured to: adjusting the brightness of the first front anamorphic image and the second front anamorphic image, wherein the first projector projects according to the adjusted first front anamorphic image, and the second projector Projecting according to the adjusted second pre-distorted image to form a synthetic spherical image on the spherical screen, The brightness of the synthetic spherical image corresponding to the overlapping area is the same as the brightness of the spherical target image corresponding to the overlapping area. 10. The projection system of claim 9, wherein the processor is configured to: A first luminance mask is established according to the first partial image and the spatial relationship between the viewing point and the dome, and a first luminance mask is established according to the second partial image and the space between the viewing point and the dome The relationship establishes a second luminance mask; normalizing the first brightness mask and the second brightness mask; generating a first front deformation brightness mask according to the normalized first brightness mask and the spatial relationship between the first projector, the viewing point and the dome; generating a second pre-distorted brightness mask according to the normalized second brightness mask and the spatial relationship between the second projector, the viewing point, and the dome; and The brightness of the first pre-distorted image and the second pre-distorted image are adjusted by using the first pre-distorted luminance mask and the second pre-distorted luminance mask, respectively. 11. The projection system of claim 6, further comprising: a second camera, coupled to the processor, wherein the second projector projects a second preset image to the dome screen, and the second camera shoots the dome screen to obtain a second deformed image, The first camera captures a third preset image displayed on the dome to obtain a third anamorphic image, and the second camera captures the third preset image displayed on the dome to obtain a third anamorphic image. A fourth deformed image is obtained, wherein the processor is further configured to: calculating the spatial relationship between the second projector, the second camera and the dome screen according to the second preset image and the second deformed image; and According to the third deformed image and the fourth deformed image, the spatial relationship between the first camera and the second camera is calculated to obtain the first projector, the second projector and the Describe the spatial relationship between the domes. 12. The projection system of claim 2, wherein the first preset image includes a plurality of control points, wherein the processor is configured to: obtaining a plurality of corresponding control points corresponding to the plurality of control points in the first deformed image; calculating a relative angle and a displacement direction between the first projector and the first camera according to the plurality of control points and the plurality of corresponding control points; and According to the relative angle, the displacement direction and the known radius of the spherical screen, it is calculated that the plurality of control points and the plurality of corresponding control points correspond to a plurality of coordinate points in the three-dimensional space, so as to calculate the The spatial relationship between the first projector, the first camera and the spherical screen. 13. A projector calibration method for imaging a spherical target image at a viewing point, wherein the projector calibration method comprises: Projecting a first preset image onto the dome screen by a first projector, and using a first camera to photograph the dome screen to obtain a first deformed image; calculating the spatial relationship between the first projector, the first camera and the dome screen according to the first preset image and the first deformed image; generating a first front distorted image according to the target image, the spatial relationship between the first projector and the dome, and the spatial relationship between the dome and the viewing point; and Projecting is performed by the first projector according to the first front distorted image. 14. The projector calibration method according to claim 13, wherein according to the target image, the spatial relationship between the first projector and the dome, and the relationship between the dome and the viewing point Spatial relationship, the step of generating the first pre-distorted image includes: calculating the spherical target image according to the target image and the spatial relationship between the dome and the viewing point; and Calculate the first front deformation image according to the spherical target image and the spatial relationship between the first projector and the spherical screen, The first projector projects the first pre-distorted image to the dome to form a first spherical image on the dome, and the spherical target image is the same as the first spherical image. 15. The projector calibration method according to claim 14, wherein the step of calculating the spherical target image according to the target image and the spatial relationship between the spherical screen and the viewing point comprises: The spherical target image is calculated according to the target image, the spatial relationship between the spherical screen and the viewing point, and the intrinsic parameters corresponding to the viewing point. 16. The projector calibration method according to claim 13, wherein according to the target image, the spatial relationship between the first projector and the dome, and the relationship between the dome and the viewing point Spatial relationship, the step of generating the first pre-distorted image includes: calculating a spherical target image according to the target image and the spatial relationship between the spherical screen and the viewing point; selecting a first partial image of the spherical target image according to the spatial relationship between the first projector and the dome and a first projection range of the first projector; and The first front deformation image is generated according to the first partial image and the spatial relationship between the first projector and the dome. 17. The projector calibration method of claim 16, further comprising: selecting a second partial image of the spherical target image according to the spatial relationship between the second projector and the spherical screen and the second projection range of the second projector; generating the second front deformed image according to the second partial image and the spatial relationship between the second projector and the dome; and Projecting is performed according to the second front distorted image by the second projector. 18. The projector calibration method of claim 17, further comprising: projecting a second preset image onto the dome screen by the second projector, and photographing the dome screen with the first camera to obtain a second deformed image; and According to the second preset image and the second deformed image, the spatial relationship between the second projector, the first camera and the dome is calculated. 19. The projector calibration method of claim 17, wherein the first projection range and the second projection range have an overlapping area on the dome screen, the projector calibration method further comprising: adjusting the brightness of the first pre-distorted image and the second pre-distorted image; and Projecting by the first projector according to the adjusted first front anamorphic image, and projecting by the second projector according to the adjusted second front anamorphic image, so that the A synthetic spherical image is formed on the spherical screen, The brightness of the synthetic spherical image corresponding to the overlapping area is the same as the brightness of the spherical target image corresponding to the overlapping area. 20. The projector calibration method of claim 19, wherein the step of adjusting the brightness of the first pre-distorted image and the second pre-distorted image comprises: A first luminance mask is established according to the first partial image and the spatial relationship between the viewing point and the dome, and a first luminance mask is established according to the second partial image and the space between the viewing point and the dome The relationship establishes a second luminance mask; normalizing the first brightness mask and the second brightness mask; generating a first front deformation brightness mask according to the normalized first brightness mask and the spatial relationship between the first projector, the viewing point and the dome; generating a second pre-distorted brightness mask according to the normalized second brightness mask and the spatial relationship between the second projector, the viewing point, and the dome; and The brightness of the first pre-distorted image and the second pre-distorted image are adjusted by using the first pre-distorted luminance mask and the second pre-distorted luminance mask, respectively. 21. The projector calibration method of claim 16, further comprising: projecting a second preset image onto the dome screen by the second projector, and using a second camera to photograph the dome screen to obtain a second deformed image; calculating the spatial relationship between the second projector, the second camera and the dome screen according to the second preset image and the second deformed image; Use the first camera to capture a third preset image displayed on the dome screen to obtain a third deformed image, and use the second camera to capture the third preset image displayed on the dome screen to obtain a fourth anamorphic image; and According to the third deformed image and the fourth deformed image, the spatial relationship between the first camera and the second camera is calculated to obtain the first projector, the second projector and the Describe the spatial relationship between the domes. 22. The projector calibration method of claim 13, wherein the first preset image includes a plurality of control points, and the first preset image and the first deformed image are calculated according to the first preset image and the first deformed image. The steps of the spatial relationship between the projector, the first camera and the spherical screen include: obtaining a plurality of corresponding control points corresponding to the plurality of control points in the first deformed image; calculating a relative angle and a displacement direction between the first projector and the first camera according to the plurality of control points and the plurality of corresponding control points; and According to the relative angle, the displacement direction and the known radius of the spherical screen, it is calculated that the plurality of control points and the plurality of corresponding control points correspond to a plurality of coordinate points in the three-dimensional space, so as to calculate the The spatial relationship between the first projector, the first camera and the spherical screen.

Images