JP6364715B2 - Transmission display device and control method of transmission display device - Google Patents

Description

  The present invention relates to a transmissive display device that allows a user to visually recognize content as a virtual image and a control method of the transmissive display device.

  2. Description of the Related Art Conventionally, a head-mounted display device (head mounted display (HMD)) that uses a glasses-shaped transmissive display mounted on the head is known as a type of transmissive display device. For example, the head-mounted display device of Non-Patent Document 1 can manually adjust the screen brightness according to the content to be viewed and the usage environment brightness. When reading an electronic book with a dark background color, the user adjusts the screen brightness to be increased because the external environment over the virtual image display area where the virtual image is displayed can be seen well, making it difficult to see the character information. On the other hand, when reading an electronic book with a bright background color, the external information over the virtual image display area is not seen so much that the character information looks clear. Therefore, the screen brightness is adjusted to be lowered.

See-Through Mobile Viewer MOVERIO BT-100 User's Guide Epson Sales Co., Ltd. Seiko Epson Corporation Page 12 (http://dl.epson.jp/support/manual/data/moverio/bt100/U_GUIDE_BT100.PDF)

  However, in the conventional head-mounted display device, since the user needs to manually adjust the screen brightness, the operation is troublesome. In recent years, attention has been focused on use for business purposes, such as displaying manuals and work instruction information on a head-mounted display device, but adjusting brightness during work is not only cumbersome but also work Improvement is desired because it may cause mistakes.

  SUMMARY OF THE INVENTION In view of the above problems, the present invention has an object to provide a transmissive display device and a transmissive display device control method capable of displaying content with appropriate screen brightness without requiring a cumbersome operation by a user. And

The transmissive display device of the present invention is a transmissive display device that allows the user to visually recognize the content as a virtual image, and includes a display unit that displays the content in a predetermined display area, and a background color determination that determines the background color of the content. And a display control unit that controls the display unit so that the luminance of the display region is adjusted according to the background color of the content, and the display control unit has a luminance of the background color of the content that is less than the threshold value In this case, the display unit is controlled so that the luminance of the display area is higher than that in the case where the luminance of the background color of the content is equal to or higher than the threshold value .
Another transmissive display device of the present invention is a transmissive display device that allows a user to visually recognize a content that is a book content as a virtual image, a display unit that displays the content in a predetermined display area, and a background of the content A background color determination unit that determines a color, and a display control unit that controls the display unit so that the luminance of the display area is adjusted according to the background color of the content. When the format is a format that specifies the background color as information, the background color of the content is determined based on the information.
Another transmissive display device of the present invention is a transmissive display device that allows a user to visually recognize content as a virtual image, and includes a display unit that displays content in a predetermined display area, and a background for determining the background color of the content. A color determination unit, and a display control unit that controls the display unit so that the luminance of the display region is adjusted according to the background color of the content, and the display control unit changes the luminance of the light source of the display unit Or the brightness of the display area is adjusted by applying a shutter to the light modulator of the display unit.
In transmissive display unit above SL, further comprising determining usage environment brightness determining unit usage environment brightness, the display control unit, according to the ambient brightness, as the brightness of the display region is adjusted, the display unit It is characterized by controlling.
In transmissive display unit above SL, further comprising character color determination unit determines the character color of the content, the display control unit, according to the character color, so that the brightness of the display region is adjusted, the display unit It is characterized by controlling.
In the above transmissive display device, the display control unit adjusts the transmittance of external light incident on the display unit in accordance with the background color of the content.
In the above transmissive display device, the display control unit changes at least one of the color of the electronic shade that adjusts the transmittance of external light and the color of the light source of the display unit according to the background color of the content. And
In the above transmissive display device, the display unit includes a right-eye transmissive display unit and a left-eye transmissive display unit, and the display control unit controls the right-eye transmissive display unit and the left-eye transmissive display unit independently of each other. It is characterized by that.
A control method for a transmissive display device according to the present invention is a control method for a transmissive display device including a display unit that displays a content to be visually recognized by a user as a virtual image in a predetermined display area. And a display control step for controlling the display unit so that the luminance of the display area is adjusted according to the background color of the content. In the display control step, the background color of the content is When the luminance is less than the threshold, the display unit is controlled so that the luminance of the display area is higher than when the luminance of the background color of the content is equal to or higher than the threshold .
Another control method for a transmissive display device according to the present invention is a control method for a transmissive display device including a display unit that displays a book content to be visually recognized by a user as a virtual image in a predetermined display area. A determination step for determining the background color of the content and a display control step for controlling the display unit so that the luminance of the display area is adjusted according to the background color of the book content are performed. When the content format is a format that specifies the background color as information, the background color of the book content is determined based on the information.
Another control method for a transmissive display device according to the present invention is a control method for a transmissive display device including a display unit that displays content to be visually recognized by a user as a virtual image in a predetermined display area. A determination step for determining a background color, and a display control step for controlling the display unit such that the luminance of the display region is adjusted according to the background color of the content. In the display control step, the display unit The brightness of the display area is adjusted by changing the brightness of the light source or by applying a shutter to the light modulator of the display unit.
The following configuration may be used.
The transmissive display device of the present invention is a transmissive display device that allows the user to visually recognize the content as a virtual image, and includes a display unit that displays the content in a predetermined display area, and a background color determination that determines the background color of the content. And a luminance adjustment unit that adjusts the luminance of the display area in accordance with the background color of the content.

  A control method for a transmissive display device according to the present invention is a control method for a transmissive display device that allows a user to visually recognize a content as a virtual image, in accordance with a determination step for determining the background color of the content, and the background color of the content And a luminance adjustment step of adjusting the luminance of the display area for displaying the content.

According to the configuration of the present invention, since the brightness of the display area (screen) for displaying the content is adjusted according to the background color of the content, even if the background color of the content is dark, it may be difficult to see characters and images. Absent. In addition, content can always be displayed with an appropriate screen brightness without requiring cumbersome operations by the user, so even if the transmissive display device is used for business purposes, the user's workload and work mistakes can be reduced. Mitigation can be achieved.
The “content background color” is a concept including a hue, luminance, brightness, saturation, and the like caused by the background color of the content.
Further, the luminance of the display area may be adjusted by changing the luminance of the light source (backlight) that illuminates the display unit, or by performing processing such as applying a shutter to the optical modulator. Also good.
“Content” refers to one image data in the case of still image content, one frame data or a collection of frame data in the case of moving image content, one page data or a collection of page data in the case of book content, etc. Point to. Therefore, for example, in the case of book content, the brightness may be adjusted by determining the background color for each page data, or the brightness adjustment for all page data in the content may be performed according to the background color determination result for any one page. May be performed.

  The transmissive display device further includes a use environment brightness determination unit that determines use environment brightness, and the brightness adjustment unit adjusts the brightness of the display area according to the use environment brightness.

According to the configuration of the present invention, it is possible to always display content with appropriate screen brightness regardless of the use environment brightness. For example, it is conceivable that adjustment is performed to increase the screen brightness in a bright use environment, and adjustment is performed to decrease the screen brightness (or the screen brightness is not changed) in a dark use environment.
Note that the detection unit may detect the use environment luminance around the user (in front) using an imaging sensor (camera) or a luminance sensor. Alternatively, the current position or work room of the user may be determined using GPS (Global Positioning System) technology, RFID (radio frequency identification) technology, or the like, and the use environment luminance may be specified from the determination result.
Moreover, you may determine environmental illumination intensity (amount of the light irradiated to the display surface of a transmissive display apparatus) around a transmissive display apparatus as use environment brightness | luminance.
Also, as the usage environment brightness, not only the illuminance of the place where the user is located but also the illuminance of the viewpoint (the illuminance ahead of the user's viewpoint) is determined, and the brightness of the display area is adjusted according to both determination results May be. According to this configuration, it is possible to adjust to a more optimal screen brightness according to the use situation such as when looking at a bright outdoor from the room or when using only indoors.

  In the above transmissive display device, the content is book content, and the background color determination unit determines a background color in a text area of the book content.

According to the configuration of the present invention, it is possible to solve the problem that characters are difficult to see due to the background color of the electronic book.
Note that when the electronic book is a document-centric book such as a paperback book, the background color of the book (paper background color) may be determined by setting the text area to the page area. As a determination method, for example, when the format of the electronic book is a format that specifies the background color as information (such as HTML), the background color may be determined based on the information. In the case of an image format such as PDF, a text region may be extracted by image analysis and a background color may be determined based on pixel information of the text region.
When a plurality of frames are present in the frame memory for storing content (for 3D content or moving image content), the background color may be determined for one of the frames. Further, the background color may be determined by reading the image light displayed on the transmissive display device with a CCD sensor or the like.

  The above transmissive display device further includes a character color determination unit that determines a character color in the text region, and the luminance adjustment unit adjusts the luminance of the display region according to the character color.

  According to the configuration of the present invention, content can be displayed with more appropriate screen brightness in accordance with not only the background color of the text area but also the character color. For example, even when the background color is dark, a method of not increasing the screen brightness depending on the character color is conceivable.

  In the above transmissive display device, the background color determination unit determines the color having the highest appearance frequency in the content as the background color of the content.

  According to the configuration of the present invention, the background color of content can be determined by simple image analysis even when the format of the electronic book is not a format that specifies the background color as information.

  In the above transmissive display device, the background color determination unit detects a same color area that is equal to or greater than a predetermined range from the content, and determines the color of the same color area as the background color of the content.

According to the configuration of the present invention, the background color of content can be determined by simple image analysis even when the format of the electronic book is not a format that specifies the background color as information.
Note that “detecting the same color region of a predetermined range or more” means detecting, for example, a predetermined number of consecutive pixels of the same color in the case of a liquid crystal display device.

  In the above transmissive display device, the content is book content, and the background color determination unit detects data at a predetermined position on a predetermined page of the book content, and determines the color of the data as the background color of the content. It is characterized by.

  According to the configuration of the present invention, in the case of document-centric book content such as a paperback book, data other than the cover is often the same paper color. By detecting, the background color can be accurately determined.

  The above transmissive display device further includes a transmittance adjusting unit that adjusts the transmittance of external light incident on the display unit, and the transmittance adjusting unit adjusts the transmittance of external light according to the background color of the content. It is characterized by adjusting.

According to the configuration of the present invention, since the transmittance of external light is adjusted according to the background color of the content, even when the background color of the content is dark, it is possible to make it easier to see characters and images. For example, when the background color of the content is dark, it may be possible to make the characters and images easier to see by adjusting the transmittance to be lowered.
Note that the transmittance of external light may be adjusted not only according to the background color of the content but also according to the usage environment luminance and the character color. Further, as a means for changing the transmittance, for example, it is conceivable to attach an electronic shade to the front surface of the display unit (the side opposite to the user side).

  In the transmissive display device described above, the display unit includes a right-eye transmissive display unit and a left-eye transmissive display unit, and the luminance adjustment unit includes the luminance of the right-eye display area that can be displayed by the right-eye transmissive display unit, and the left-eye display unit. The brightness of the display area for the left eye that can be displayed by the transmissive display unit is adjusted independently.

  According to the configuration of the present invention, when the present invention is applied to a transmissive display device having a transmissive display unit for the right eye and a transmissive display unit for the left eye, such as a head-mounted display, the luminance of the display region for the right eye and the display region for the left eye is increased. Since it can be adjusted independently, appropriate adjustment can be performed according to the characteristics of the eyes of the user (individual) (individual difference between left and right eyes, difference in visual acuity between left and right, etc.).

1 is an external perspective view of a head-mounted display device according to an embodiment of the present invention. (A) is a figure which shows an example of the content which has a dark background, (b) is a figure which shows the example of a display of the said content by a head-mounted display apparatus. (A) is a figure which shows an example of the content which has a bright background, (b) is a figure which shows the example of a display of the said content with a head-mounted display apparatus. It is a block diagram which shows the function structure of a head mounting | wearing type display apparatus. It is a detailed block diagram of a display control part. It is explanatory drawing of the background color determination method which concerns on the modification 4. FIG. 10 is a detailed block diagram of a display control unit according to Modification 5.

  Hereinafter, a transmissive display device and a control method of the transmissive display device according to an embodiment of the present invention will be described with reference to the accompanying drawings. FIG. 1 is an external perspective view of a head-mounted display device HM to which the transmissive display device of the present invention is applied. The head-mounted display device HM is worn by a user on the head and is also called a head mounted display (HMD). The head-mounted display device HM of this embodiment is an optically transmissive head-mounted display device HM (a transmissive head-mounted display device, a see-through type) that allows a user to visually recognize a virtual image and at the same time directly view an outside scene. -A head-mounted display) having a spectacle shape.

  The head-mounted display device HM includes a right holding unit 11, a right display driving unit 12, a left holding unit 13, a left display driving unit 14, a right optical image display unit 15, and a left optical image display unit 16. And an imaging sensor 17 (detection unit).

  The right optical image display unit 15 and the left optical image display unit 16 are disposed at positions corresponding to the right and left eyes of the user when the head-mounted display device HM is mounted. One end of the right optical image display unit 15 and one end of the left optical image display unit 16 are connected at positions corresponding to the user's eyebrows when the head-mounted display device HM is mounted. The right holding unit 11 extends from the end ER that is the other end of the right optical image display unit 15. Similarly, the left holding unit 13 extends from the end EL which is the other end of the left optical image display unit 16.

  The right holding unit 11 is substantially perpendicular to the right optical image display unit 15 from the end ER of the right optical image display unit 15 to a position corresponding to the user's temporal region when the head-mounted display device HM is mounted. It is a member provided by stretching so as to be formed. Similarly, the left holding unit 13 is connected to the left optical image display unit 16 from the end EL of the left optical image display unit 16 to a position corresponding to the user's temporal region when the head-mounted display device HM is worn. It is a member provided by extending so as to form a substantially right angle. The right holding unit 11 and the left holding unit 13 hold the head-mounted display device HM on the user's head like a temple of glasses.

  The right display drive unit 12 is on the inner side of the right holding unit 11, in other words, on the side facing the user's head when the head-mounted display device HM is mounted, and the end of the right optical image display unit 15. It is arranged on the ER side. The left display driving unit 14 is disposed inside the left holding unit 13 and on the end EL side of the left optical image display unit 16.

  The imaging sensor 17 is disposed at a position corresponding to the user's eyebrow when the head-mounted display device HM is worn. The imaging sensor 17 images an outside scene in the user's viewing direction (in front of the user) in a state where the head-mounted display device HM is mounted. In the present embodiment, the usage environment luminance is detected from the captured image captured by the imaging sensor 17. The imaging sensor 17 is also used for determining a user operation. The head-mounted display device HM recognizes an indicator operated by the user from the captured image, and generates an operation signal. As the indicator, for example, a fingertip of the user, a ring type input device worn by the user, a pen tip of the pen held by the user, a light emitter of a remote controller, or the like is used. The imaging range by the imaging sensor 17 is preferably wider than the user's field of view 21 (see FIGS. 2 and 3) when the head-mounted display device HM is worn.

  In the present embodiment, a configuration including one imaging sensor 17 is illustrated, but a configuration including a plurality of imaging sensors 17 may be used. Further, the detection means is not limited to the image sensor 17 and may be of any type as long as it can detect the usage environment luminance or the indicator. For example, a luminance sensor or an illuminance sensor may be used for use environment luminance detection, or a laser sensor or an ultrasonic sensor may be used for indicator detection.

  The head-mounted display device HM further includes a right earphone 18 for the right ear and a left earphone 19 for the left ear. The right earphone 18 and the left earphone 19 are respectively attached to the right and left ears when the user wears the head-mounted display device HM. Although not particularly illustrated, a control mechanism for realizing a control unit 30 described later is built in the right display drive unit 12 or the left display drive unit 14.

  Next, with reference to FIG. 2 and FIG. 3, information that can be visually recognized by the user wearing the head-mounted display device HM and a visual state that changes according to the background color of the content will be described. FIG. 2A shows an example of the book content Ca with a dark background. FIG. 2B shows information that can be visually recognized by the user when the book content Ca is displayed by the head-mounted display device HM. As shown in FIG. 2B, the visual field 21 of the user wearing the head-mounted display device HM is a virtual image 22 generated by the head-mounted display device HM, that is, the book content of FIG. Ca is displayed. In addition, the user can see the outside scene 23 in an area other than the area where the virtual image 22 is displayed in the field of view 21. The outside scene 23 is information that can be seen when the right optical image display unit 15 and the left optical image display unit 16 have translucency. Hereinafter, the area where the virtual image 22 is displayed is referred to as a “virtual image display area 25”, and the other area is referred to as an “outside scene visually recognizing area 26”. The “display area” in the claims refers to the “virtual image display area 25”.

  By the way, the head-mounted display device HM of the present embodiment is configured such that the outside scene 23 can be seen through the virtual image 22 in the virtual image display region 25. For this reason, character information or the like may be difficult to see depending on the background color (luminance) of the content displayed as the virtual image 22. For example, in the case of the book content Ca having a dark background color as shown in FIG. 2A, it is difficult to see the character information as shown in FIG. Become. On the contrary, in the case of the book content Cb having a bright background color as shown in FIG. 3A, the outside information 23 is not so visible, so that the character information can be seen clearly as shown in FIG. On the other hand, the visibility of character information also changes depending on the usage environment luminance. For example, character information becomes difficult to see in a bright usage environment. Conversely, in a dark usage environment, the character information looks clear. For this reason, in the present embodiment, the screen brightness is automatically adjusted according to the background color of the content and the usage environment brightness in order to maintain good visibility of the content (particularly character information). Details will be described later.

  Next, the functional configuration of the head-mounted display device HM will be described with reference to FIG. The head-mounted display device HM controls the display unit 40 that allows the user to visually recognize the virtual image 22, the imaging sensor 17, the right earphone 18, the left earphone 19, the power supply unit 71, the interface 72, and these units. And a control unit 30.

  The power supply unit 71 supplies power to each unit of the head-mounted display device HM. The interface 72 is for connecting various external devices OA that are sources of contents such as images (still images, moving images) and audio. Examples of the external device OA include a personal computer, a mobile terminal (such as a mobile phone and a smartphone), and a game terminal. As the interface 72, for example, a USB interface, a micro USB interface, a memory card interface, a wireless LAN interface, or the like can be employed.

  The control unit 30 includes a CPU (Central Processing Unit) 50 and a storage unit 60. The storage unit 60 includes a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and stores various computer programs such as an application (viewer) corresponding to a content format. The CPU 50 reads out and executes a computer program from the storage unit 60, thereby operating as an operating system (OS) 51, an image processing unit 52, a display control unit 53, an image analysis unit 54, a command execution unit 55, and a voice processing unit 56. Function.

  The image processing unit 52 generates a signal to be supplied to the display unit 40 based on the content input via the interface 72 and transmits the signal to the display unit 40. A signal supplied to the display unit 40 is different between an analog format and a digital format.

  For example, in the case of an analog format, the image processing unit 52 generates and transmits a clock signal PCLK, a vertical synchronization signal VSync, a horizontal synchronization signal HSync, and image data Data. Specifically, the image processing unit 52 acquires an image signal (for example, an analog signal composed of 30 frame images per second) included in the content, and the vertical synchronization signal VSync and horizontal synchronization are acquired from the acquired image signal. The synchronization signal such as the signal HSync is separated. Then, the image processing unit 52 generates a clock signal PCLK using a PLL circuit (not shown) or the like according to the separated vertical synchronization signal VSync, horizontal synchronization signal HSync, and the like. Further, the image processing unit 52 converts the analog image signal from which the synchronization signal is separated into a digital image signal using an A / D conversion circuit (not shown) or the like, and generates image data Data.

  On the other hand, in the case of the digital format, the image processing unit 52 generates the clock signal PCLK and the image data Data, and outputs them to the display unit 40. Specifically, when the content is in a digital format, the clock signal PCLK is output in synchronization with the image signal. For this reason, A / D conversion of the vertical synchronization signal VSync, the horizontal synchronization signal HSync, and the analog image signal becomes unnecessary.

  Note that the image processing unit 52 executes image processing such as resolution conversion processing, color tone correction processing such as adjustment of luminance and saturation, and keystone correction processing on the image data Data as necessary.

  The display control unit 53 controls the image display state on the display unit 40 by generating a control signal for controlling the right display driving unit 12 and the left display driving unit 14 and outputting the control signal to the display unit 40. Specifically, the display control unit 53 uses the control signal to turn on / off the drive of the right LCD 84 by the right LCD control unit 83, drive the ON / OFF of the right backlight 82 by the right backlight control unit 81, and the left LCD. The right display drive unit 12 and the left display drive unit 14 are individually controlled by controlling ON / OFF of the left LCD 94 by the control unit 93 and ON / OFF of the left backlight 92 by the left backlight control unit 91, respectively. The generation and emission of image light by each of the above are controlled. For example, when displaying side-by-side 3D video data, the display data for the left half is used as the left-eye display data for the single-screen display data in the side-by-side format, and the full-screen display, that is, the horizontal display size is doubled. Data is processed by the display control unit 53 so as to be input to the left display drive unit as display data. The same processing is performed for the display data for the right half as display data for the right eye. In addition, the display control unit 53 adjusts the screen brightness by adjusting the intensity of the right backlight 82 and the left backlight 92 according to the background color of the image data Data and the usage environment brightness. Here, the “screen” that is subject to brightness adjustment is the “virtual image display area 25” directly, but also affects the “outside view visual recognition area 26”. Both the “display area 25” and the “outside view visually recognizing area 26” are indicated (see FIGS. 2 and 3).

  The image analysis unit 54 analyzes an image captured by the image sensor 17. For example, the use environment luminance can be obtained from each pixel information of the captured image. Further, an indicator is detected from the captured image, and information (detection timing, detection position, etc.) is output to the image processing unit 52 and the display control unit 53. The image processing unit 52 and the display control unit 53 perform image processing and display control based on the output indicator information. In addition, the command execution unit 55 executes various commands set in advance in accordance with the operation of the indicator. The audio processing unit 56 acquires an audio signal included in the content, amplifies the acquired audio signal, and outputs the amplified audio signal to the right earphone 18 and the left earphone 19.

  The display unit 40 includes a right display drive unit 12 (right eye transmission display unit), a left display drive unit 14 (left eye transmission display unit), a right light guide plate 86 as the right optical image display unit 15, and a left optical image. And a left light guide plate 96 as the display unit 16. In the following description, the right optical image display unit 15 (right light guide plate 86) and the left optical image display unit 16 (left light guide plate 96) are also collectively referred to as “optical members”.

  The right display driving unit 12 includes a right backlight (BL) 82 that functions as a light source, a right backlight (BL) control unit 81 that controls driving of the right backlight (BL) 82, and a right LCD 84 that functions as a display element. A right LCD control unit 83 that controls driving of the right LCD 84, and a right projection optical system 85. The right backlight control unit 81 drives and controls the right backlight 82 based on the input control signal. The right backlight 82 is a light emitter such as an LED or electroluminescence (EL). The right LCD control unit 83 drives and controls the right LCD 84 based on the input clock signal PCLK, vertical synchronization signal VSync, horizontal synchronization signal HSync, and image data Data for the right eye. The right LCD 84 is a transmissive liquid crystal panel in which a plurality of pixels are arranged in a matrix.

  The right projection optical system 85 projects image light emitted from the right LCD 84, and is configured using, for example, a collimator lens. The right light guide plate 86 as the right optical image display unit 15 is formed of a light-transmitting resin material or the like, and reflects the image light output from the right projection optical system 85 along a predetermined optical path while being reflected by the user. Lead to the right eye. The right projection optical system 85 and the right light guide plate 86 are collectively referred to as “light guide unit”. The light guide unit can use any method as long as it forms a virtual image in front of the user's eyes using image light. For example, a diffraction grating (a hologram element as an example) may be used, or a transflective film May be used.

  As with the right display drive unit 12, the left display drive unit 14 includes a left backlight (BL) control unit 91, a left backlight (BL) 92, a left LCD control unit 93, a left LCD 94, and left projection optics. System 95. The left light guide plate 96 as the left optical image display unit 16 is formed of a light transmissive resin material or the like, and is used while reflecting the image light output from the left projection optical system 95 along a predetermined optical path. To the left eye.

  In this way, the image light guided to both eyes of the user of the head-mounted display device HM forms an image on the retina, so that the user can enter the virtual image display area 25 as shown in FIG. The virtual image 22 is visually recognized. In the present embodiment, making the user visually recognize the virtual image 22 in this way is also expressed as “display”.

  Next, the luminance adjustment by the display control unit 53 will be described in detail with reference to FIG. The display control unit 53 mainly functions as the determination unit 110 and the luminance adjustment unit 120 regarding luminance adjustment. The determination unit 110 includes a background color determination unit 111 and a use environment luminance determination unit 112. In addition, the brightness adjustment unit 120 includes a calculation unit 121 and a brightness control unit 122.

  The background color determination unit 111 determines the background color of the input content (image data Data). For example, when the content is book content (electronic book), the background color in the text area is determined. As a determination method, for example, when the format of an electronic book is a format that specifies a background color as information (for example, a format using a markup language such as XML or HTML), the background color is determined based on the information. When background color information is incorporated for each page, the background color determination unit 111 performs determination for each page. Note that only a predetermined page may be determined, and other pages within the same content may be regarded as having the same color.

  The background color determination unit 111 determines the background color of the text file when the format of the electronic book is a format including an image file and a text file. Further, when the input content is an image format such as PDF (Portable Document Format) or JPEG (Joint Photographic Experts Group), the background color determination unit 111 extracts a text area by image analysis, and the pixels of the text area The background color is determined based on the information.

  Note that it is determined whether or not the book content is based on additional information (metadata) added to the content. If it is determined that the content is a book content, the background color of the book (paper May be determined. When the book genre and book name can be determined from the additional information, the corresponding background color may be searched from a predetermined database using the determined book genre and book name as a search key. In this case, the predetermined database may be in the head-mounted display device HM or in a predetermined server accessible by the head-mounted display device HM.

  The use environment brightness determination unit 112 determines the brightness in front of the user, that is, the brightness of the outside scene that can be viewed through the head-mounted display device HM as the use environment brightness. In the present embodiment, the use environment luminance is detected by the image sensor 17, but the present invention is not limited to this, and it is used by using GPS (Global Positioning System) technology, RFID (radio frequency identification) technology, or the like. It is also possible to identify the current position and working room of the person and determine the usage environment luminance from the determination result. In this case, information in which the current position or the working room is associated with the luminance at the place is in the head-mounted display device HM or a predetermined server accessible by the head-mounted display device HM. Is a condition. Further, in the case of using the GPS technology, it is a condition that the GPS receiver is mounted on the head-mounted display device HM. In addition, in the case of using the RFID technology, it is a condition that a wireless IC tag is provided in the work room and a wireless IC reader is mounted on the head-mounted display device HM.

  The calculation unit 121 has a right backlight (BL) 82 and a left backlight (BL) 92 according to the background color determined by the background color determination unit 111 and the use environment brightness determined by the use environment brightness determination unit 112. Calculate the appropriate brightness (or luminous intensity, illuminance). As a calculation method, for example, when the brightness of the background color is equal to or higher than the threshold value La1 and the use environment brightness is less than the threshold value Lb1, the appropriate brightness is set to the first brightness or the brightness of the background color is not adjusted. If it is less than the threshold value La1 or the usage environment luminance is equal to or higher than the threshold value Lb1, it is conceivable that the appropriate luminance is the second luminance (where the first luminance is smaller than the second luminance). That is, when the brightness of the background color is low, it is preferable to increase the brightness of the backlights 82 and 92 compared to when the background color is high. When the use environment brightness is high, the brightness of the backlights 82 and 92 is increased compared to when the brightness is low. It is preferable to raise.

  The luminance control unit 122 outputs a command signal to the right backlight (BL) control unit 81 and the left backlight (BL) control unit 91 so that the backlights 82 and 92 have the luminance calculated by the calculation unit 121. To do. As a result, the brightness of the screen including the “virtual image display area 25” and the “outside scene visual recognition area 26” changes, and as a result, the visibility of content (particularly character information) is ensured. Note that the left and right backlights 82 and 92 are not necessarily controlled to have the same luminance, and may be controlled to have different luminance.

  As described above, according to the present embodiment, the brightness of the screen on which content is displayed is adjusted according to the background color of the content and the usage environment brightness. However, it does not hinder the visibility of the content. Further, even when the content is an electronic book such as a magazine, the text color is extracted and the background color is determined, so that the visibility of the characters can be reliably ensured. Furthermore, since no user operation is required to adjust the screen brightness, even when the head-mounted display device HM of this embodiment is used for business purposes, the user's workload and work errors are reduced. be able to.

Note that the present invention is not limited to the above-described embodiment, and the following modification examples can be employed.
(Modification 1)
In the above embodiment, the screen brightness is adjusted according to the background color of the text area and the usage environment brightness. However, the character color in the text area is further determined (character color determination unit), and the screen brightness is determined according to the character color. You may adjust it. According to this configuration, content can be displayed with more appropriate screen luminance in consideration of not only the background color of the text area but also the character color. For example, even when the background color is dark, the screen brightness may not be increased depending on the character color. Conversely, even when the background color is bright, the screen brightness may be increased depending on the character color. As for the character color, pixels constituting the character may be specified by image analysis, and the color of the pixel may be determined as the character color. Further, when the content format is a format in which a character color is specified as information, the character color may be determined based on the information.

(Modification 2)
In the above embodiment, the text region is extracted from the content and the background color is determined. However, the extraction of the text region may be omitted. For example, the color with the highest appearance frequency in the content (for example, page data in the case of book content or a collection of page data) may be determined as the background color of the content. According to this configuration, even if the format of the electronic book is an image format that handles one page as one image, the background color of the content can be determined by simple image analysis.

(Modification 3)
Further, instead of the second modification, the same color area of a predetermined range or more may be detected from the content (for example, page data in the case of book content), and the color of the same color area may be determined as the background color of the content. . For example, in the case of the liquid crystal display method, a predetermined number or more of the same color pixels may be detected, and the color of the pixels may be determined as the background color of the content. In this case, the continuous direction may be arbitrary, or may be configured to detect only a specific direction such as a vertical direction / horizontal direction. According to this configuration, the same effect as that of the second modification can be obtained. It should be noted that the background color of the content may be determined more accurately by combining this modification and the above-described modification 2.

(Modification 4)
Further, based on the additional information added to the content, it is determined whether or not it is book content. When it is determined that the content is book content, data (pixels) at a predetermined position in the predetermined page is detected, and the data May be determined as the background color of the content. FIG. 6 is an explanatory diagram of this modification. In the figure, one page data D1 of the book content is a cover, and the book content starting from the two page data D2 is illustrated. In this case, the pixels 210a to 210d at the four corners of the two-page data D2 may be detected, and the background color may be determined from those colors. For example, it is conceivable that the most common color among four pixels is the background color. According to this configuration, in the case of document-centric book content such as a paperback book, the same paper color is often used except for the cover. Therefore, the background color can be accurately determined by detecting a pixel at a predetermined position on a predetermined page. Can do. Further, by performing color determination by detecting a plurality of pixels at predetermined positions, erroneous determination can be prevented even when an incorrect color is detected by one pixel.

(Modification 5)
In the above embodiment, the visibility of the content is ensured by adjusting the screen brightness. However, other methods may be combined. FIG. 7 is a block diagram of the display control unit 53 according to this modification. This modification is different from the example shown in FIG. 5 in that a transmittance control unit 123 is added. Further, an electronic shade (right shade 131 and left shade 132) is added to the front surfaces (external world side) of the right optical image display portion 15 and the left optical image display portion 16, respectively. The “transmittance adjusting unit” in the claims corresponds to the calculation unit 121 and the transmittance control unit 123 of this modification.

  In this case, the calculation unit 121 calculates the transmittance of the left and right electronic shades 131 and 132 according to the background color of the content and the usage environment luminance. Further, the transmittance control unit 123 controls the transmittance of the left and right electronic shades 131 and 132 so as to obtain the transmittance calculated by the calculation unit 121 (varies the voltage value applied to the electronic shades 131 and 132). ). Specifically, when the background color of the content is dark or the usage environment luminance is bright (when the screen luminance needs to be increased), the transmittance control unit 123 is configured to display the right optical image display unit 15 and the left optical image display unit 16. It adjusts so that the transmittance | permeability of the external light which injects into may be reduced. According to this configuration, the visibility of the content can be more reliably ensured even when the background color of the content is dark or the usage environment luminance is bright.

  It should be noted that, as in the right shade 131 and the left shade 132, an integrated shade may be used instead of a separated shape. Also, the adjustment of the screen brightness and the adjustment of the transmittance of the electronic shades 131 and 132 may not be performed simultaneously, but only one of them may be selectively performed. Further, when the colors of the electronic shades 131 and 132 can be changed, the color may be changed in addition to the transmittance. Further, the left and right electronic shades 131 and 132 may be controlled independently.

(Modification 6)
In the above embodiment, when the background color brightness is less than the threshold value La1 or the usage environment brightness is the threshold value Lb1 or more, the brightness of the backlights 82 and 92 is increased. The brightness may be determined not under the or condition but under the and condition, and the appropriate brightness of the backlights 82 and 92 may be calculated based on the determination result. Similarly, the transmittance of the electronic shades 131 and 132 may be calculated based on the determination result based on the and condition.

(Modification 7)
In the above embodiment, the screen brightness is adjusted according to whether the background color brightness and the usage environment brightness are each equal to or higher than a predetermined threshold value. However, a plurality of threshold values are provided, and the screen brightness is adjusted in a plurality of stages. You may adjust it. Further, the screen brightness may be linearly adjusted according to the brightness of the background color and / or the usage environment brightness. Similarly, the transmittance of the electronic shades 131 and 132 may be adjusted in a plurality of stages or linearly.

(Modification 8)
In the above embodiment, the screen brightness is adjusted by changing the brightness of the backlights 82 and 92. However, the process of applying a shutter to the LCDs 84 and 94 (the light transmittance is adjusted by the polarizing plate and the alignment film). Screen luminance (in this case, the luminance of the virtual image display area 25) may be adjusted by performing the processing.

(Modification 9)
In the above embodiment, the case where the size of the virtual image display area 25 is smaller than the size of the outside scene visual recognition area 26 is illustrated, but the present invention is not limited to this. When the virtual image display area 25 and the outside scene visual recognition area 26 have the same size (in other words, when the virtual image display area 25 has the same size as the visual field 21), or the virtual image display area 25 is larger than the external scene visual recognition area 26 In the case (in other words, when the virtual image display area 25 is larger than the field of view 21), the present invention can be applied.

(Modification 10)
In the above embodiment, the head mounted display device HM is mounted by mounting the display unit 40 that allows the user to visually recognize a virtual image, the control unit 30 that controls the display unit 40, and the like in one housing (device). Although implemented, these may be implemented as separate devices. That is, the head-mounted display device HM main body including the display unit 40, the imaging sensor 17, and the earphones 18 and 19 and the controller (not shown) including the control unit 30 may be configured to be communicable. In this case, the main body and the controller may be connected by wire, or may be connected by a wireless LAN, infrared communication, Bluetooth (registered trademark), or the like. Further, the function of the controller may be realized by a personal computer, a portable information terminal (PAD, cellular phone, watch-type portable terminal) or the like.

(Other variations)
In the above embodiment, the visibility of the content is ensured by adjusting the screen luminance according to the background color of the content and the usage environment luminance. However, in addition to the adjustment of the screen luminance, the color of the content is changed. It may be variable. For example, it is conceivable to increase the visibility of text by strengthening a color that is complementary to the background color of the content (a color that is diagonal to the background color in the hue circle). The content color variable control may be performed only on the text area or on the entire screen.

  In the above embodiment, the display control unit 53 adjusts the intensity of the right backlight 82 and the left backlight 92 at the same time. However, the display control unit 53 may be independently adjustable (luminance adjustment unit). According to this configuration, appropriate adjustment can be performed according to the characteristics of the individual eyes (individual differences between left and right eyes, differences in visual acuity between left and right, etc.). In this case, it is preferable that the user can set or select the brightness adjustment of the left and right display areas, the brightness adjustment amount of the left and right display areas, the brightness adjustment method of the left and right display areas, and the like. Further, when the content color variable control is performed as described above, the color variable control may be performed independently for the left and right display frames. Furthermore, when the colors of the backlights 82 and 92 and the colors of the electronic shades 131 and 132 can be adjusted, variable control of the light color and shade color may be performed independently on the left and right. Note that the color change control of the backlight / electronic shade also has a color after the variable control (for example, a color complementary to the background color so that the visibility of characters in the text area is improved according to the background color of the content. It is preferable to determine the relevant color).

  Various display modes may be selectable by the user, including the brightness adjustment item and the content / backlight / electronic shade color variable control item. For example, any one of the first mode in which the screen brightness is adjusted only according to the background color of the content, the second mode in which the screen brightness is adjusted according to the background color of the content and the use environment brightness, and the third mode in which the screen brightness is not adjusted. It is good also as being selectable. Further, it is determined whether or not the content is an electronic book, and a first mode in which screen brightness adjustment is performed only in the case of an electronic book, a second mode in which screen brightness adjustment is performed regardless of whether the content is an electronic book, or screen brightness adjustment is not performed. Any one of the third modes may be selectable. In addition, the first mode in which the illuminance at the place where the user is present and the illuminance at the viewpoint are determined and the screen brightness is adjusted according to both determination results, the screen brightness is adjusted only according to the illuminance at the place where the user is present. Either the second mode or the third mode in which screen brightness adjustment is not performed may be selectable. Further, a first mode for adjusting screen brightness and content / backlight / electronic shade color variable control according to the background color and / or usage environment brightness of the content, a second mode for performing only screen brightness adjustment, and screen brightness adjustment. Alternatively, any one of the third modes in which neither content / backlight / electronic shade color variable control is performed may be selectable.

  In the above embodiment, the binocular head-mounted display device HM (optical transmission type head-mounted display device) is exemplified as the transmission type display device, but the present invention is, for example, a video transmission type. It can also be applied to other types of head mounted display devices such as monocular type. In the above embodiment, the eyeglass-type head-mounted display device HM that covers the front of the eye is illustrated, but the present invention is not limited to this, and the type that does not completely cover the front of the eye (the front of the eye). The present invention can also be applied to a head-mounted display device of a type that partially covers). Furthermore, the present invention is not limited to the head-mounted display device, but can be applied to other transmissive display devices such as a head-up display.

  In the above embodiment, the backlight control units 81 and 91, the backlights 82 and 92, the LCD control units 83 and 93, and the display drive units 12 and 14 including the LCDs 84 and 94 are illustrated. However, this aspect is merely an example. In addition to or in place of these components, a component for realizing another method may be provided.

  For example, an organic EL (Organic Electro-Luminescence) display and an organic EL control unit may be provided. In place of the LCD, LCOS (Liquid crystal on silicon, LCoS is a registered trademark), a digital micromirror device, or the like may be used. Further, the present invention can be applied to a laser retinal projection type head-mounted display device. In the case of the laser retinal projection type, the “display area” can be defined as an image area recognized by the user's eyes. The optical system of the head-mounted display device HM transmits external light through the optical member and enters the user's eye together with the image light. The optical member guides the image light into the member. It is not limited to this, and may be refracted or reflected in the direction of the user's eyes.

  In the present embodiment, the projection optical systems 85 and 95 are applied as the optical system, but this aspect is merely an example. For example, a scanning optical system composed of a MEMS mirror may be employed as the optical system, and a signal light modulation unit that emits signal light to the scanning optical system may be provided. In this case, the signal light formed and emitted by the signal light modulation unit is incident on a scanning optical system (MEMS mirror) that is a scanning unit. The scanning optical system emits signal light as scanning light toward a light guide having a half mirror layer, and the scanning light is scanned on the surface of the half mirror layer to form a virtual image by image light. The user can recognize the image by grasping with the eyes.

  Moreover, it is possible to provide each function of the head-mounted display device HM shown in the above embodiment as a program. Further, the program can be provided by being stored in various recording media (CD-ROM, flash memory, etc.). That is, a program for causing a computer to function as each component of the head-mounted display device HM and a recording medium recording the program are also included in the scope of the right of the present invention. In addition, regardless of the above-described embodiments, the device configuration and processing steps of the head-mounted display device HM can be changed as appropriate without departing from the scope of the present invention.

  DESCRIPTION OF SYMBOLS 11 ... Right holding part 12 ... Right display drive part 13 ... Left holding part 14 ... Left display drive part 15 ... Right optical image display part 16 ... Left optical image display part 17 ... Imaging sensor 21 ... Visual field 22 ... Virtual image 23 ... Outside scene 25 ... Virtual image display area 26 ... Outside view visual recognition area 110 ... Determination part 111 ... Background color determination part 112 ... Usage environment luminance determination part 120 ... Luminance adjustment part 121 ... Calculation part 122 ... Luminance control part 123 ... Transmittance control part 131 ... Right shade 132 ... Left shade HM ... Head-mounted display device

Claims (11)

A transmissive display device that allows a user to visually recognize content as a virtual image,
A display unit for displaying the content in a predetermined display area;
A background color determination unit for determining a background color of the content;
A display control unit that controls the display unit so that the luminance of the display region is adjusted according to the background color of the content ,
The display control unit controls the display unit so that the luminance of the display area is higher when the luminance of the background color of the content is less than the threshold than when the luminance of the background color of the content is equal to or higher than the threshold. A transmissive display device characterized by that. A transmissive display device that allows a user to visually recognize a content that is a book content as a virtual image,
A display unit for displaying the content in a predetermined display area;
A background color determination unit for determining a background color of the content;
A display control unit that controls the display unit so that the luminance of the display region is adjusted according to the background color of the content ,
When the content format is a format that specifies a background color as information, the background color determination unit determines the background color of the content based on the information . A transmissive display device that allows a user to visually recognize content as a virtual image,
A display unit for displaying the content in a predetermined display area;
A background color determination unit for determining a background color of the content;
A display control unit that controls the display unit so that the luminance of the display region is adjusted according to the background color of the content ,
The display control unit adjusts the luminance of the display region by changing the luminance of the light source of the display unit or by applying a shutter to the light modulator of the display unit. apparatus. A use environment brightness determination unit for determining the use environment brightness;
The said display control part controls the said display part so that the brightness | luminance of the said display area may be adjusted according to the said use environment brightness | luminance, The Claim 1 thru | or 3 characterized by the above-mentioned. A transmissive display device. Further comprising a character color determination unit determines the character color before Kiko content,
The transmissive display apparatus according to claim 2 , wherein the display control unit controls the display unit so that luminance of the display region is adjusted according to the character color. The display controller in accordance with the background color of the content, transmission according to any one of claims 1 to 5, characterized in that to adjust the transmittance of external light incident on the display unit Display device. The display control unit changes at least one of a color of an electronic shade for adjusting a transmittance of the external light and a color of a light source of the display unit according to a background color of the content. 7. A transmissive display device according to 6 . The display unit includes a right eye transparent display unit and a left eye transparent display unit,
The display controller, the transmissive display part and the left eye transmissive display unit for the right eye, the transmission type display device according to any one of claims 1 to 7, characterized in that independently controlled. A control method for a transmissive display device including a display unit that displays content displayed as a virtual image to a user in a predetermined display area,
A determination step of determining a background color of the content;
Performing a display control step of controlling the display unit so that the luminance of the display area is adjusted according to the background color of the content ,
In the display control step, when the luminance of the background color of the content is less than a threshold value, the display unit is controlled so that the luminance of the display area is higher than when the luminance of the background color of the content is equal to or higher than the threshold value. A control method for a transmissive display device. A control method of a transmissive display device including a display unit that displays a book content to be visually recognized by a user as a virtual image in a predetermined display area,
A determination step of determining a background color of the book content;
Executing a display control step of controlling the display unit so that the brightness of the display area is adjusted according to the background color of the book content ,
In the determination step, if the format of the book content is a format that specifies a background color as information, the background color of the book content is determined based on the information. . A control method for a transmissive display device including a display unit that displays content displayed as a virtual image to a user in a predetermined display area,
A determination step of determining a background color of the content;
Performing a display control step of controlling the display unit so that the luminance of the display area is adjusted according to the background color of the content ,
In the display control step, the luminance of the display region is adjusted by changing the luminance of the light source of the display unit or by applying a shutter to the light modulator of the display unit. Device control method.

Images