JP7192561B2 - Audio output device and audio output method - Google Patents

Description

The present invention relates to an audio output device and an audio output method for outputting audio.

2. Description of the Related Art In recent years, many vehicles are equipped with navigation devices that provide vehicle travel guidance. Patent Literature 1 discloses a technique of acquiring medical information of a driver and increasing the sound volume of a guidance voice of a navigation device when the driver's hearing is impaired.

JP 2009-254544 A

When there is a lot of noise inside and outside the vehicle, it may be difficult for the passenger to hear the voice of the navigation device. In addition, when the voice of the route guidance of the navigation device includes the information of the landmark of the route guidance point, the passenger may not be able to visually recognize the landmark. In these cases, it is difficult for the passenger to fully understand the content of the voice. Therefore, when it is difficult for the passenger to understand the contents of the voice of the navigation device, it is desirable to output an easy-to-understand voice.

SUMMARY OF THE INVENTION The present invention has been made in view of such circumstances, and its object is to provide an audio output device and an audio output method capable of outputting an easy-to-understand audio when the content of the output audio is difficult for a passenger to understand. It is in.

In order to solve the above-described problems, an audio output device according to one aspect of the present invention includes an acquisition unit that acquires an utterance of a vehicle occupant, a determination unit that determines whether the acquired utterance is a reflection, and a classification unit that classifies the type of reflection when the utterance is determined to be a reflection; an output unit that outputs speech according to the classified reflection type based on the content of the speech to be reflected; an image recognition unit that recognizes an image in the vehicle interior to detect a possibly sleeping passenger . The output unit re-outputs the voice to be listened to again at a higher volume if the image recognition unit does not detect the possibly sleeping occupant in the case where the type of feedback is a type indicating missed hearing. However, if the type of feedback is a type that indicates a missed hearing, and the image recognition unit detects a possibly sleeping passenger, the volume is maintained and the voice to be reviewed is re-output. .

According to this aspect, the feedback type of the passenger is classified, and the sound corresponding to the classified feedback type is output based on the content of the speech to be asked back. is difficult to comprehend, and when the feedback is repeated, an easy-to-understand voice can be output.

The voice output device may include a specifying unit that specifies the contents of the voice to be listened back to based on the voice output from the output unit immediately before the feedback.

The output unit may output another voice related to the content of the voice to be reflected when the feedback type is a type indicating that the meaning of the content of the voice is not understood.

The output unit may re-output the speech to be listened back to when the feedback type is a type indicating a missed listening.

The output unit may re-output the speech to be listened to again at a higher volume when the feedback type is a type indicating that the speech cannot be heard.

Another aspect of the present invention is an audio output method. This method is a voice output method executed by a computer, and includes an acquisition step of acquiring an utterance of a vehicle occupant, a determination step of determining whether or not the acquired utterance is a reflection, and a step of determining whether the utterance is a reflection. If it is determined that there is, a classification step of classifying the type of reflection, an image recognition step of recognizing the image in the vehicle interior to detect possible sleeping occupants, and a speech recognition target for reflection. and an output step of outputting speech according to the type of categorized reflection based on the content. In the output step, when the type of feedback is a type indicating a missed hearing, the voice to be reviewed is re-output at a higher volume if the passenger who may be asleep is not detected in the image recognition step. However, if the feedback type is a type that indicates a missed hearing, and if a possibly sleeping occupant is detected in the image recognition step, the volume is maintained and the voice to be reviewed is re-output. .

According to this aspect, the feedback type of the passenger is classified, and the sound corresponding to the classified feedback type is output based on the content of the speech to be asked back. is difficult to comprehend, and when the feedback is repeated, an easy-to-understand voice can be output.

ADVANTAGE OF THE INVENTION According to this invention, when it is difficult for a passenger to understand the content of the output voice, it is possible to output an easy-to-understand voice.

1 is a block diagram of a navigation device according to an embodiment; FIG. 2 is a flowchart showing audio output processing of the audio output device of FIG. 1;

FIG. 1 is a block diagram of a navigation device 10 according to an embodiment. The navigation device 10 is mounted on a vehicle, which is an automobile. The navigation device 10 includes a microphone 12 , a speaker 14 , a navigation section 16 and an audio output device 18 .

The microphone 12 is installed in the vehicle interior of the vehicle, converts the voice in the vehicle interior such as the utterance of the passenger into an audio signal, and outputs the converted audio signal to the audio output device 18 . The speaker 14 is installed in the passenger compartment of the vehicle, converts an audio signal output from the audio output device 18 into audio, and outputs the audio.

The navigation unit 16 sets a guidance route for route guidance using a well-known technology, displays the guidance route and a map on a display unit (not shown), outputs voice for driving guidance to the voice output device 18, and outputs the guidance route. Follow the route guidance. The voice for travel guidance includes information on landmarks such as intersections where travel guidance should be provided. When the position of the vehicle reaches the point on the guide route to which the travel guidance should be given, the navigation unit 16 provides travel guidance by voice, for example, "You are about to turn right. Convenience store ABC is a landmark." The navigation unit 16 may cause the audio output device 18 to output audio of various information for improving the driver's convenience, such as information on traffic jams and construction work on the travel route, weather forecast for the destination, and information on facilities near the current location. good.

The audio output device 18 includes a processing section 20 and a storage section 22 . The processing unit 20 includes an acquisition unit 30 , a determination unit 32 , a classification unit 34 , an identification unit 36 and an output unit 38 . The configuration of the processing unit 20 can be implemented in terms of hardware by a CPU, memory, and other LSIs of any computer, and in terms of software is implemented by a program or the like loaded into the memory. It depicts the functional blocks realized by cooperation. Therefore, those skilled in the art will understand that these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

The output unit 38 outputs audio through the speaker 14 or the like based on the audio information for driving guidance supplied from the navigation unit 16 .

Acquisition unit 30 acquires the speech of the vehicle occupant via microphone 12 . When the passenger cannot hear the voice for driving guidance by the navigation device 10 or cannot understand the contents of the voice, he/she may utter an utterance such as "Huh?" Acquisition unit 30 recognizes the utterance of the passenger based on the audio signal output from microphone 12 , acquires the utterance as text data, and supplies the text data of the utterance to determination unit 32 and classification unit 34 .

The storage unit 22 prestores text data of a plurality of reflections as a database. The determination unit 32 refers to the database of the storage unit 22 , determines whether or not the utterance acquired by the acquisition unit 30 is a reflection, and supplies the determination result to the classification unit 34 and the identification unit 36 . If the text data of the speech matches the text data of the reflection in the database, the determination unit 32 determines that the speech is reflection. If the text data of the speech does not match the text data of the reflection in the database, the determination unit 32 determines that the speech is not the reflection. If the text data of the utterance matches the text data of the reflection in the database and the ending of the utterance rises, the determination unit 32 may determine that the utterance is reflection. This can improve the determination accuracy.

When the utterance is determined to be a reflection, the identification unit 36 identifies the content of the speech to be reflected based on the sound output from the output unit 38 immediately before the reflection, and confirms the content of the identified speech. It is supplied to the output section 38 . As a result, it is easy to correctly specify which voice content the passenger listened to again.

When the utterance is determined to be a reflection, the classification unit 34 classifies the reflection type and supplies the classified reflection type to the output unit 38 . Reflection types include a first type indicating that the meaning of the content of the speech is not understood, a second type indicating that the listener did not hear the content, and a third type indicating that the listener did not hear the content.

The storage unit 22 also pre-stores as a database the correspondence relationship between each of the plurality of text data of reflection and the type of reflection. There is a one-to-one correspondence between reflections and types of reflections. For example, reflections such as "Where?", "Which?" are associated with the first type. Reflections such as "Huh? What did you say?" are associated with the second type. Reflections such as "What? It's hard to hear" are associated with the third type. Reflections such as "What?", which may apply to any of the first to third types and are difficult to classify, are associated with the second type. Reflections that do not fit into either the first type or the third type are associated with the second type.

The classification unit 34 refers to the database of the storage unit 22 to identify the type of reflection text data that matches the text data of the utterance, and sets the identified type as the reflection type.

The output unit 38 outputs, through the speaker 14 or the like, a sound according to the type of reflection classified by the classification unit 34 based on the content of the reflection target speech supplied from the specifying unit 36 .

When the feedback type is the first type, the output unit 38 outputs another voice related to the content of the voice to be reviewed. The storage unit 22 preliminarily stores, as a database, one or more pieces of separate speech data associated with the speech content for each speech content to be listened back to. For example, if the content of the voice to be reflected is "Convenience store ABC is a landmark", rephrase the content of the voice to be reviewed such as "The red sign is the landmark" or "The red building is the landmark". Audio data is associated. In other words, in this example, the signboard and building of "Convenience Store ABC" are assumed to be red. If the passenger cannot see the letters "Convenience store ABC" and cannot understand the meaning of the voice saying "Convenience store ABC is a landmark" because they do not know the signboard or the color of the building, they will not be able to understand the meaning. It is possible to hear a voice such as "The red sign is a landmark" related to the content of the voice that was not heard, and it is possible that the content can be understood.

If the type of reflection is the second type, the output unit 38 re-outputs the speech that is the object of reflection. As a result, when the passenger misses hearing the voice, the passenger can listen to the missed voice again, and can easily grasp the contents of the voice. The voice is re-output even in the case of reflections that are difficult to classify, and reflections that do not fall into either the first or third type. It may be possible to make the crew understand.

When the type of reflection is the third type, the output unit 38 re-outputs the speech to be the object of reflection with a higher volume. As a result, the occupant can hear the voice again at a volume that is easier to hear, and can easily grasp the content of the voice, which the occupant could not hear due to the influence of ambient noise or his/her own hearing loss.

Next, the overall operation of the audio output device 18 configured as above will be described. FIG. 2 is a flowchart showing audio output processing of the audio output device 18 of FIG. The processing in FIG. 2 is repeatedly executed.

If the acquisition unit 30 has not acquired the utterance of the passenger (N in S10), the process waits in step S10. If an utterance is acquired (Y of S10), if the utterance is not a repeat (N of S12), the process returns to step S10. If the utterance is a reflection (Y of S12), the identification unit 36 identifies the content of the speech to be reflected (S14), the classification unit 34 classifies the type of reflection (S16), and the output unit 38 repeats the reflection. type is confirmed (S18).

If the feedback type is the first type, the output unit 38 outputs another voice related to the content of the voice to be reviewed (S20), and ends the process. If the feedback type is the second type, the output unit 38 re-outputs the speech to be reviewed (S22), and ends the process. If the feedback type is the third type, the output unit 38 re-outputs the voice to be reviewed at a higher volume (S24), and ends the process.

According to this embodiment, it is difficult for the occupant to understand the content of the voice output from the voice output device 18, and when the passenger repeats the voice, an easy-to-understand voice can be output. In addition, since the type of reflection is classified and the sound corresponding to the classified type of reflection is output, the configuration can be simpler than generating the output sound for each of a plurality of reflections. Therefore, an increase in cost can be suppressed, and the navigation device 10 suitable for in-vehicle use can be provided.

The present invention has been described above based on the embodiments. It should be understood by those skilled in the art that the embodiment is merely an example, and that various modifications are possible in combination of each component and each treatment process, and that such modifications are within the scope of the present invention.

For example, the audio output device 18 may include an image recognition unit that recognizes an image of the interior of the vehicle captured by a camera in the vehicle to detect a possibly sleeping passenger. A well-known technique can be used for image recognition. When the feedback type is the second type, the output unit 38 re-outputs the voice to be reviewed at a higher volume if the image recognition unit does not detect the possible sleeping occupant. good too. This makes it easier for the occupant to grasp the content of the missed voice than when the voice is re-output while maintaining the volume. On the other hand, when the type of reflection is the second type, the output unit 38 maintains the volume and re-outputs the sound to be the object of reflection when a possibly sleeping passenger is detected. good too. This makes it possible to consider sleeping passengers.

In the embodiment, the database is referred to determine whether or not the utterance is a reflection, and the type of reflection is classified. Depending on the result, it may be determined whether or not the utterance is a reflection, and the type of reflection may be classified. A well-known technique can be used for intention understanding. In this modification, the degree of freedom in configuring the audio output device 18 can be improved.

DESCRIPTION OF SYMBOLS 10... Navigation apparatus, 18... Voice output device, 30... Acquisition part, 32... Judgment part, 34... Classification part, 36... Identification part, 38... Output part.

Claims (5)

an acquisition unit that acquires an utterance of an occupant of the vehicle;
a determination unit that determines whether the acquired utterance is a reflection;
a classification unit that classifies the type of reflection when the utterance is determined to be reflection;
an output unit for outputting speech according to the classified feedback type based on the content of the speech to be reviewed;
an image recognition unit that recognizes images in the vehicle interior and detects occupants who may be asleep;
with
The output unit re-outputs the voice to be listened to again at a higher volume if the image recognition unit does not detect the possibly sleeping occupant in the case where the type of feedback is a type indicating missed hearing. However, if the type of feedback is a type that indicates a missed hearing, and the image recognition unit detects a possibly sleeping passenger, the volume is maintained and the voice to be reviewed is re-output. ,
An audio output device characterized by: 2. The voice output device according to claim 1, further comprising a specifying unit that specifies the contents of the voice to be reflected based on the voice output from the output unit immediately before the feedback. The output unit is characterized in that, when the type of reflection is a type indicating that the meaning of the content of the speech is not understood, another speech related to the content of the speech to be reflected is output. 3. The audio output device according to claim 1 or 2. 4. The output unit according to any one of claims 1 to 3 , wherein, when the type of feedback is a type indicating that the feedback cannot be heard, the output unit re-outputs the speech to be reviewed at a higher volume. audio output device. A computer-implemented audio output method comprising:
an acquisition step of acquiring an utterance of an occupant of the vehicle;
a determination step of determining whether the acquired utterance is a reflection;
a classification step of classifying the type of reflection if the utterance is determined to be reflection;
an image recognition step of recognizing images in the vehicle interior to detect possible sleeping occupants;
an output step of outputting a speech corresponding to a type of categorized reflection based on the content of the speech targeted for reflection;
with
In the output step, when the type of feedback is a type indicating a missed hearing, the voice to be reviewed is re-output at a higher volume if the passenger who may be asleep is not detected in the image recognition step. However, if the feedback type is a type that indicates a missed hearing, and if a possibly sleeping occupant is detected in the image recognition step, the volume is maintained and the voice to be reviewed is re-output. ,
An audio output method characterized by:

Images