JP6754184B2 - Voice recognition device and voice recognition method - Google Patents

Description

The present disclosure relates to a voice recognition device and a voice recognition method that remove noise contained in voice information and perform voice recognition on the voice information from which noise has been removed.

Conventionally, a function of controlling a terminal by voice or a function of searching for a keyword by voice has been studied. In order to realize these functions, a microphone is conventionally mounted on a remote controller for operating a terminal, and sound is collected by the microphone. In recent years, for the purpose of further improving convenience, a technique has been studied in which a terminal has a built-in microphone so that the terminal can be operated even if the user speaks from a place away from the terminal. However, when the user speaks from a place away from the terminal, the volume difference between the voice spoken by the user and the unnecessary sound (noise) disappears, and it becomes difficult for the terminal to correctly recognize the utterance content. Conventionally, the voice recognition processing function of the terminal has been used to speed up the response to the utterance related to the terminal operation, and the voice recognition processing function of the server responds to the utterance related to the information search that requires a dictionary with a huge number of vocabularies. Has been used to do.

For example, in Patent Document 1, the server is provided with a dictionary-scale priority voice recognition means, and the client is provided with a speed priority voice recognition means.

Japanese Unexamined Patent Publication No. 2013-64777

However, Patent Document 1 has a problem that when a user speaks from a position away from the microphone, the voice contains noise and voice recognition is not executed accurately. Further, in Patent Document 1, the noise removal processing is not distributed between the terminal and the server, and the configuration or condition for executing both the noise removal processing and the voice recognition processing is not examined.

The present disclosure has been made to solve the above problems, and it is possible to improve the accuracy of speech recognition in a high noise environment, and to realize high-speed speech recognition in a low noise environment. It is an object of the present invention to provide an apparatus and a voice recognition method.

The voice recognition device according to one aspect of the present disclosure uses a voice acquisition unit that acquires the first voice information and a first removal method that removes noise contained in the first voice information acquired by the voice acquisition unit. The noise removal processing unit that removes the noise by using and outputs the voice information from which the noise has been removed as the second voice information, and the second voice information output by the noise removal processing unit perform voice recognition. A voice recognition unit that outputs the voice recognition result as the first voice recognition result information and the first voice information acquired by the voice acquisition unit are transmitted to the server, and the server uses the first removal method. The noise contained in the first voice information is generated by using the second removal method of removing a larger amount of noise from the first voice information than the amount of noise removed from the first voice information. A communication unit that receives the voice recognition result as the second voice recognition result information, which is the result of voice recognition on the third voice information from which the noise has been removed, and the voice. It includes a mediation unit for selecting which of the first voice recognition result information output by the recognition unit and the second voice recognition result information received by the communication unit is to be output.

According to the present disclosure, in a noisy environment, voice can be removed by removing noise from the first voice information in a server capable of removing more noise from the first voice information than a voice recognition device. The accuracy of recognition can be improved, and in a silent environment, the speed of voice recognition can be realized by removing noise from the first voice information in the voice recognition device.

FIG. 1 is a diagram showing an overall configuration of a voice recognition system according to the first embodiment of the present disclosure. FIG. 2 is a diagram showing a functional configuration of the voice recognition system according to the first embodiment of the present disclosure. FIG. 3 is a flowchart showing an example of the operation of the voice recognition system according to the first embodiment of the present disclosure. FIG. 4 is a diagram showing an example of a display screen for displaying non-speech recognition information. FIG. 5 is a diagram showing an example of a display screen for displaying server transmission information. FIG. 6 is a diagram showing an example of a display screen for displaying transmission confirmation information. FIG. 7 is a diagram showing a functional configuration of a voice recognition system in a modified example of the first embodiment of the present disclosure. FIG. 8 is a diagram showing a functional configuration of the voice recognition system according to the second embodiment of the present disclosure. FIG. 9 is a first flowchart showing an example of the operation of the voice recognition system according to the second embodiment of the present disclosure. FIG. 10 is a second flowchart showing an example of the operation of the voice recognition system according to the second embodiment of the present disclosure. FIG. 11 is a third flowchart showing an example of the operation of the voice recognition system according to the second embodiment of the present disclosure. FIG. 12 is a first flowchart showing an example of the operation of the voice recognition system in the modified example of the second embodiment of the present disclosure. FIG. 13 is a second flowchart showing an example of the operation of the voice recognition system in the modified example of the second embodiment of the present disclosure. FIG. 14 is a diagram showing a functional configuration of the voice recognition system according to the third embodiment of the present disclosure. FIG. 15 is a diagram showing a functional configuration of the voice recognition system according to the fourth embodiment of the present disclosure. FIG. 16 is a diagram showing a functional configuration of the voice recognition system according to the fifth embodiment of the present disclosure. FIG. 17 is a diagram showing a functional configuration of a voice recognition system in a modified example of the fifth embodiment of the present disclosure.

(Knowledge that became the basis of the present invention)
Patent Document 1 has a problem that when a user speaks from a position away from a microphone, the voice contains noise and voice recognition does not operate correctly. Further, in Patent Document 1, the noise removal processing is not distributed between the terminal and the server, and the configuration or condition in which the noise removal processing and the voice recognition processing are used in combination is not examined.

The present disclosure has been made to solve the above problems, and it is possible to improve the accuracy of speech recognition in a high noise environment, and to realize high-speed speech recognition in a low noise environment. A device and a voice recognition method are provided.

The voice recognition device according to one aspect of the present disclosure uses a voice acquisition unit that acquires the first voice information and a first removal method that removes noise contained in the first voice information acquired by the voice acquisition unit. The noise removal processing unit that removes the noise by using and outputs the voice information from which the noise has been removed as the second voice information, and the second voice information output by the noise removal processing unit perform voice recognition. A voice recognition unit that outputs the voice recognition result as the first voice recognition result information and the first voice information acquired by the voice acquisition unit are transmitted to the server, and the server uses the first removal method. The noise contained in the first voice information is generated by using the second removal method of removing a larger amount of noise from the first voice information than the amount of noise removed from the first voice information. A communication unit that receives the voice recognition result as the second voice recognition result information, which is the result of voice recognition on the third voice information from which the noise has been removed, and the voice. It includes a mediation unit for selecting which of the first voice recognition result information output by the recognition unit and the second voice recognition result information received by the communication unit is to be output.

According to this configuration, the first voice information is acquired. The noise contained in the acquired first voice information is removed by using the first removal method, and the voice information from which the noise has been removed is output as the second voice information. Voice recognition is performed on the output second voice information, and the voice recognition result is output as the first voice recognition result information. In addition, the acquired first voice information is transmitted to the server, and the server produces a larger amount of noise than the amount of noise removed from the first voice information by the first removal method. This is the result of voice recognition being performed on the third voice information from which the noise has been removed by removing the noise contained in the first voice information by using the second removal method for removing from the voice information of. The voice recognition result is received from the server as the second voice recognition result information. Which of the output first voice recognition result information and the received second voice recognition result information is to be output is selected.

Therefore, in a noisy environment, it is possible to improve the accuracy of speech recognition by removing noise in a server that can remove more noise from the first speech information than a speech recognition device. In a low noise environment, the voice recognition device can remove noise from the first voice information, so that the speed of voice recognition can be increased.

Further, in the voice recognition device, the voice recognition unit calculates a first likelihood indicating the likelihood of the first voice recognition result information, and the calculated first likelihood is applied to the mediation unit. Output, the communication unit receives the second likelihood indicating the likelihood of the second voice recognition result information calculated by the server, and outputs the received second likelihood to the mediation unit. Then, the arbitration unit determines which of the first voice recognition result information and the second voice recognition result information is output, at least of the first likelihood and the second likelihood. You may choose based on one.

According to this configuration, the first likelihood indicating the plausibility of the first speech recognition result information is calculated, and the calculated first likelihood is output. In addition, a second likelihood indicating the likelihood of the second speech recognition result information calculated by the server is received, and the received second likelihood is output. Then, which of the first voice recognition result information and the second voice recognition result information is output is selected based on at least one of the first likelihood and the second likelihood.

Therefore, since the output voice recognition result is selected based on the likelihood, a more accurate voice recognition result can be output.

Further, in the voice recognition device, the arbitration unit outputs the first voice recognition result information when the first likelihood is larger than a predetermined first threshold value, and the first likelihood is obtained. Is equal to or less than the first threshold value, and when the second likelihood is greater than the predetermined second threshold value, the second voice recognition result information is output, and the first likelihood is the first. When it is equal to or less than the threshold value of and the second likelihood is equal to or less than the second threshold value, neither the first voice recognition result information nor the second voice recognition result information may be output. ..

According to this configuration, when the first likelihood is larger than a predetermined first threshold value, the first speech recognition result information is output, the first likelihood is equal to or less than the first threshold value, and the second When the likelihood of is greater than a predetermined second threshold value, the second speech recognition result information is output, the first likelihood is equal to or less than the first threshold value, and the second likelihood is the second threshold value. In the following cases, neither the first voice recognition result information nor the second voice recognition result information is output.

Therefore, since the voice recognition result is selected by comparing the likelihood and the threshold value, the voice recognition result to be output can be selected with a simpler configuration.

Further, the voice recognition device further includes a utterance section detection unit that detects an utterance section spoken by the user in the first voice information acquired by the voice acquisition unit, and the utterance section detection unit further includes the utterance section. If is not detected, the noise removing processing unit does not remove the noise contained in the first voice information and does not output the second voice information, and the communication unit outputs the first voice information. It does not have to be sent to the server.

According to this configuration, the utterance section spoken by the user in the acquired first voice information is detected. If the utterance section is not detected, the noise included in the first voice information is not removed, the second voice information is not output, and the first voice information is not transmitted to the server.

Therefore, if it is not the utterance section spoken by the user, the noise contained in the first voice information is not removed, the second voice information is not output, and the first voice information is not transmitted to the server, which is unnecessary. It is possible to prevent the arithmetic processing from being performed and to prevent unnecessary information from being transmitted.

Further, in the above-mentioned voice recognition device, when the utterance section is detected by the utterance section detection unit, the utterance duration is measured, which is the duration of the utterance section detected by the utterance section detection unit. Further including a measuring unit, when the utterance section is detected by the utterance section detection unit, the noise removal processing unit removes noise included in the first voice information, and the communication unit removes the noise included in the first voice information. The first utterance information is transmitted to the server, and the arbitration unit transmits the first utterance recognition result information output by the utterance recognition unit and the second utterance received by the communication unit. Which of the recognition result information to output may be selected by using at least the information regarding the length of the utterance duration.

According to this configuration, when the utterance section is detected, the utterance duration, which is the duration of the detected utterance section, is measured. When the utterance section is detected, the noise included in the first voice information is removed, and the first voice information in the utterance section is transmitted to the server. Then, which of the output first voice recognition result information and the received second voice recognition result information is to be output is selected by using at least the information regarding the length of the utterance duration. ..

Therefore, since the voice recognition result is selected by using at least the information regarding the length of the utterance duration, the voice recognition result to be output can be selected with a simpler configuration.

Further, in the above-mentioned voice recognition device, the arbitration unit multiplies the second likelihood indicating the likelihood of the second voice recognition result information when the utterance duration is longer than a predetermined length. , May be higher than the weighting multiplied by the first likelihood indicating the likelihood of the first speech recognition result information.

According to this configuration, when the utterance duration is longer than a predetermined length, the weighting by multiplying the second likelihood indicating the likelihood of the second speech recognition result information is the likelihood of the first speech recognition result information. It is higher than the weighting that is multiplied by the first likelihood that indicates the likelihood. If the utterance duration is long, it is highly possible that advanced voice instructions with a large number of words are being performed. Therefore, when the utterance duration is long, erroneous recognition can be prevented by adopting the voice recognition result output from the server.

Further, in the voice recognition device, the communication unit receives the third voice information from the server, outputs the received third voice information to the voice recognition unit, and the voice recognition unit receives the third voice information. Voice recognition is performed on the third voice information received by the communication unit, the voice recognition result is output as the fourth voice recognition result information, and the second voice output by the noise removal processing unit. The information is transmitted to the server, and the voice recognition result obtained by performing voice recognition for the second voice information is received from the server as the third voice recognition result information, and the received third voice recognition result information. Is output to the arbitration unit, and the arbitration unit receives the first voice recognition result information output by the voice recognition unit, the second voice recognition result information received by the communication unit, and the communication. You may select whether to output the third voice recognition result information received by the unit or the fourth voice recognition result information output by the voice recognition unit.

According to this configuration, the third voice information is received from the server, and the received third voice information is output to the voice recognition unit. Voice recognition is performed on the received third voice information, and the voice recognition result is output as the fourth voice recognition result information. Further, the output second voice information is transmitted to the server, and the voice recognition result obtained by voice recognition for the second voice information is received from the server as the third voice recognition result information and received. The third voice recognition result information is output to the arbitration unit. Then, the output first voice recognition result information, the received second voice recognition result information, the received third voice recognition result information, and the output fourth voice recognition result information Which of them is output is selected.

Therefore, the first voice recognition result in which the noise removal processing and the voice recognition processing are performed by the voice recognition device, the second voice recognition result in which the noise removal processing and the voice recognition processing are performed by the server, and the voice recognition device A third voice recognition result in which noise removal processing is performed and voice recognition processing is performed by the server, and a fourth voice recognition result in which noise removal processing is performed by the server and voice recognition processing is performed by the voice recognition device. Since either of the above is output, the optimum voice recognition result can be obtained according to the state of the environmental sound and the performance of voice recognition.

Further, in the voice recognition device, the voice recognition unit calculates a first likelihood indicating the likelihood of the first voice recognition result information, and the calculated first likelihood is applied to the mediation unit. The communication unit receives a second likelihood indicating the likelihood of the second voice recognition result information calculated by the server, and outputs the received second likelihood to the mediation unit. Then, the communication unit receives the third likelihood indicating the likelihood of the third voice recognition result information calculated by the server, and outputs the received third likelihood to the mediation unit. The voice recognition unit calculates a fourth likelihood indicating the likelihood of the fourth voice recognition result information, outputs the calculated fourth likelihood to the mediation unit, and the mediation unit receives the calculated fourth likelihood. Which of the first voice recognition result information, the second voice recognition result information, the third voice recognition result information, and the fourth voice recognition result information is output is determined. The selection may be based on at least one of a first likelihood, a second likelihood, a third likelihood and a fourth likelihood.

According to this configuration, the first likelihood indicating the plausibility of the first speech recognition result information is calculated, and the calculated first likelihood is output. In addition, a second likelihood indicating the likelihood of the second speech recognition result information calculated by the server is received, and the received second likelihood is output. Further, a third likelihood indicating the likelihood of the third speech recognition result information calculated by the server is received, and the received third likelihood is output. Furthermore, a fourth likelihood indicating the likelihood of the fourth speech recognition result information is calculated, and the calculated fourth likelihood is output. Then, which of the first voice recognition result information, the second voice recognition result information, the third voice recognition result information, and the fourth voice recognition result information is output is the first. It is selected based on at least one of a likelihood, a second likelihood, a third likelihood and a fourth likelihood.

Therefore, since the output voice recognition result is selected based on the likelihood, a more accurate voice recognition result can be output.

Further, the voice recognition device further includes a utterance section detection unit that detects an utterance section spoken by the user in the first voice information acquired by the voice acquisition unit, and the utterance section detection unit further includes the utterance section. If is not detected, the noise removing processing unit does not remove the noise contained in the first voice information and does not output the second voice information, and the communication unit outputs the first voice information. It does not have to be sent to the server.

According to this configuration, the utterance section spoken by the user in the acquired voice information is detected. If the utterance section is not detected, the noise included in the first voice information is not removed, and the first voice information is not transmitted to the server.

Therefore, if it is not the utterance section spoken by the user, the noise contained in the first voice information is not removed, the second voice information is not output, and the first voice information is not transmitted to the server, which is unnecessary. It is possible to prevent the arithmetic processing from being performed and to prevent unnecessary information from being transmitted.

Further, in the above-mentioned voice recognition device, when the utterance section is detected by the utterance section detection unit, the utterance duration is measured, which is the duration of the utterance section detected by the utterance section detection unit. Further including a measuring unit, when the utterance section is detected by the utterance section detection unit, the noise removal processing unit removes noise included in the first voice information, and the communication unit removes the noise included in the first voice information. The first utterance information is transmitted to the server, and the arbitration unit uses the first utterance recognition result information, the second utterance recognition result information, and the third utterance recognition result information. Which of the fourth voice recognition result information is to be output may be selected by using at least the information regarding the length of the utterance duration.

According to this configuration, when the utterance section is detected, the utterance duration, which is the duration of the detected utterance section, is measured. When the utterance section is detected, the noise included in the first voice information is removed, and the first voice information in the utterance section is transmitted to the server. Then, which of the first voice recognition result information, the second voice recognition result information, the third voice recognition result information, and the fourth voice recognition result information is output is at least the utterance continuation. Selected with information about the length of time.

Therefore, since the voice recognition result is selected by using at least the information regarding the length of the utterance duration, the voice recognition result to be output can be selected with a simpler configuration.

Further, in the voice recognition device, the arbitration unit has a second likelihood and a third likelihood of indicating the likelihood of the second voice recognition result information when the utterance duration is longer than a predetermined length. The weighting by multiplying the third likelihood indicating the likelihood of the speech recognition result information is the likelihood of the first likelihood and the fourth speech recognition result information indicating the likelihood of the first speech recognition result information. It may be higher than the weighting that multiplies the fourth likelihood that indicates.

According to this configuration, when the utterance duration is longer than a predetermined length, the second likelihood indicating the likelihood of the second speech recognition result information and the third likelihood indicating the likelihood of the third speech recognition result information are shown. The weighting multiplied by the likelihood of is greater than the weighting multiplied by the first likelihood indicating the likelihood of the first speech recognition result information and the fourth likelihood indicating the likelihood of the fourth speech recognition result information. Can be raised. If the utterance duration is long, it is highly possible that advanced voice instructions with a large number of words are being performed. Therefore, when the utterance duration is long, erroneous recognition can be prevented by adopting the voice recognition result calculated by the server.

Further, in the voice recognition device, the arbitration unit multiplies the second likelihood by the weighting obtained by multiplying the third likelihood when the utterance duration is longer than a predetermined length. May also be raised.

According to this configuration, when the utterance duration is longer than a predetermined length, the weight to be multiplied by the second likelihood is higher than the weight to be multiplied by the third likelihood.

Therefore, the second voice recognition result information in which the noise removal processing and the voice recognition processing are performed by the server is the third voice recognition result in which the noise removal processing is performed by the voice recognition device and the voice recognition processing is performed by the server. Since a higher priority is given than information, it is possible to prevent misrecognition.

The voice recognition device according to another aspect of the present disclosure is a voice recognition unit that acquires the first voice information, and a first removal method that removes noise contained in the first voice information acquired by the voice acquisition unit. The noise removal processing unit that removes the noise using the above and outputs the voice information from which the noise has been removed as the second voice information, and the first voice information acquired by the voice acquisition unit is transmitted to the server, and the server The first removal method is used to remove a larger amount of noise from the first voice information than the amount of noise removed from the first voice information by the first removal method. The communication unit that receives the third voice information from which the noise contained in the voice information is removed from the server and the second voice information output by the noise removal processing unit are voice-recognized to perform voice recognition. The recognition result is output as the first voice recognition result information, voice recognition is performed on the third voice information received by the communication unit, and the voice recognition result is output as the second voice recognition result information. It includes a voice recognition unit and an arbitration unit that selects which of the first voice recognition result information and the second voice recognition result information output by the voice recognition unit is to be output.

According to this configuration, the first voice information is acquired. The noise contained in the acquired first voice information is removed by using the first removal method, and the voice information from which the noise has been removed is output as the second voice information. The acquired first voice information is transmitted to the server, and the first voice information causes a larger amount of noise than the amount of noise removed from the first voice information by the first removal method in the server. The third voice information from which the noise contained in the first voice information has been removed by using the second removal method for removing from the first voice information is received from the server. Voice recognition is performed on the output second voice information, the voice recognition result is output as the first voice recognition result information, and voice recognition is performed on the received third voice information. , The voice recognition result is output as the second voice recognition result information. Which of the output first voice recognition result information and the second voice recognition result information is to be output is selected.

Therefore, in a high noise environment, the accuracy of voice recognition can be improved by removing noise in a server that can remove a larger amount of noise than a voice recognition device, and voice recognition in a quiet environment. By removing noise in the device, it is possible to realize high-speed voice recognition.

The voice recognition method according to another aspect of the present disclosure is a voice recognition method in a server including a communication unit, a noise removal processing unit, a voice recognition unit, and a mediation unit, and performing voice recognition for voice information acquired by a terminal. Therefore, the communication unit receives the first voice information acquired by the terminal, and the noise removal processing unit uses the first removal method to remove the noise contained in the received first voice information. The voice information from which the noise has been removed is output as the second voice information, the voice recognition unit performs voice recognition on the second voice information, and the voice recognition result is the first voice recognition. A second removal method, which is output as result information and in which the communication unit removes less noise than the amount of noise removed from the first voice information by the first removal method in the terminal. The voice recognition result, which is the result of voice recognition being performed on the third voice information from which the noise has been removed by removing the noise contained in the first voice information, is used as the second voice recognition result. Received from the terminal as information, the arbitration unit selects which of the first voice recognition result information and the second voice recognition result information is output.

According to this configuration, the first voice information is received. The noise contained in the received first voice information is removed by using the first removal method, and the noise-removed voice information is output as the second voice information. Voice recognition is performed on the output second voice information, and the voice recognition result is output as the first voice recognition result information. In addition, the noise contained in the first voice information is generated by using the second removal method for removing a smaller amount of noise than the noise removed from the first voice information by the first removal method in the terminal. The voice recognition result, which is the result of voice recognition performed on the third voice information from which the noise has been removed, is received from the terminal as the second voice recognition result information. Which of the output first voice recognition result information and the received second voice recognition result information is to be output is selected.

Therefore, in a noisy environment, the accuracy of speech recognition can be improved by removing the noise in a server that can remove more noise from the first speech information than the speech recognition device. In a silent environment, the voice recognition device can increase the speed of voice recognition by removing noise from the first voice information.

Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. The following embodiments are examples that embody the present disclosure, and do not limit the technical scope of the present disclosure.

(Embodiment 1)
FIG. 1 is a diagram showing an overall configuration of a voice recognition system according to the first embodiment of the present disclosure.

The voice recognition system shown in FIG. 1 includes a terminal 100 and a server 200. The terminal 100 is communicably connected to the server 200 via a network (for example, the Internet) 300.

The terminal 100 is, for example, a device for controlling a television or an air conditioner arranged in a home. The terminal 100 recognizes the voice spoken in a predetermined language. Further, the terminal 100 recognizes the voice and controls the television or the air conditioner arranged in the home based on the result of the voice recognition.

The terminal 100 may be a separate body from, for example, a device to be controlled (for example, a television or an air conditioner arranged in a home), or may be included in the device to be controlled.

The terminal 100 includes a communication unit 101, a microphone 102, a speaker 103, a control unit 104, a memory 105, and a display unit 106. The microphone 102, the speaker 103, and the display unit 106 do not have to be built in the terminal 100.

The communication unit 101 transmits information to the server 200 via the network 300, and receives information from the server 200 via the network 300. The method of connecting the communication unit 101 to the network 300 does not matter. The microphone 102 collects ambient sounds and acquires voice information. The speaker 103 outputs sound.

The control unit 104 has, for example, a CPU (central processing unit), and functions as the control unit 104 when the CPU executes a control program stored in a memory 105, which will be described later. The control unit 104 processes, for example, various data (information) received by the communication unit 101, and controls the operation of each configuration in the terminal 100.

The memory 105 is, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), an HDD (Hard Disk Drive), or the like, and includes data (information) received by the communication unit 101 and data calculated by the control unit 104. (Information) or a control program, etc. is stored. The display unit 106 is, for example, a liquid crystal display device, and displays various information.

Further, the terminal 100 may have a translation unit (not shown) that translates a word or sentence resulting from voice recognition into a word or sentence in another language. The result translated by the translation unit may be displayed on the display unit 106, for example. Further, the result translated by the translation unit may be displayed on the display screen of a device such as a television arranged in the home, which is the control target of the terminal 100, for example.

The server 200 includes a communication unit 201, a control unit 202, and a memory 203.

The communication unit 201 transmits information to the terminal 100 via the network 300, and receives information from the terminal 100 via the network 300.

The control unit 202 has, for example, a CPU, and functions as the control unit 202 when the CPU executes a control program stored in the memory 203 described later. The control unit 202 processes, for example, various data (information) received by the communication unit 201, and controls the operation of each configuration in the server 200.

The memory 203 is, for example, a ROM, a RAM, an HDD, or the like, and stores data (information) received by the communication unit 201, data (information) processed by the control unit 202, a control program, or the like.

FIG. 2 is a diagram showing a functional configuration of the voice recognition system according to the first embodiment of the present disclosure. As shown in FIG. 2, the voice recognition system includes a terminal 100 and a server 200. The terminal 100 includes a voice acquisition unit 11, a first sound collection processing unit 12, a first voice recognition unit 13, and an arbitration unit 14. The server 200 includes a second sound collection processing unit 21 and a second voice recognition unit 22.

The voice acquisition unit 11 is realized by the microphone 102, and the first sound collection processing unit 12, the first voice recognition unit 13, and the arbitration unit 14 are realized by the control unit 104. Further, the second sound collection processing unit 21 and the second voice recognition unit 22 are realized by the control unit 202.

The voice acquisition unit 11 acquires the first voice information. Here, the voice information is, for example, a voice signal waveform, but it may be a feature amount of voice obtained by frequency-analyzing the signal waveform. The communication unit 101 (not shown) transmits the first voice information acquired by the voice acquisition unit 11 to the server 200. The communication unit 201 of the server 200 receives the first voice information transmitted by the terminal 100.

The first sound collecting processing unit 12 removes the noise contained in the first voice information acquired by the voice acquisition unit 11 by using the first removing method, and the voice information from which the noise is removed is used as the second voice. Output as information.

The first voice recognition unit 13 performs voice recognition on the second voice information output by the first sound collection processing unit 12, and outputs the voice recognition result as the first voice recognition result information. The first voice recognition unit 13 calculates a first likelihood indicating the likelihood of the first voice recognition result information when voice recognition is performed, and the calculated first likelihood is used as the first voice recognition. It is output to the arbitration unit 14 together with the result information.

The first voice recognition unit 13 performs voice recognition for the second voice information from which noise has been removed by the first sound collection processing unit 12. The first voice recognition unit 13 refers to the acoustic model and the language model stored in advance and the terminal dictionary, and performs voice recognition for the second voice information. The voice recognition result includes character string data composed of a plurality of words, which is a character string of the result of voice recognition of the second voice information. The first likelihood indicates the likelihood of the speech recognition result (that is, the first speech recognition result information) of the second speech information. Specifically, the first likelihood indicates, for example, the likelihood of the entire character string data or the likelihood of each word constituting the character string data.

The first voice recognition unit 13 calculates the degree of agreement (likelihood) between the utterance content obtained from the second voice information and each of the plurality of vocabularies included in the terminal dictionary. The first voice recognition unit 13 selects a vocabulary having the highest degree of matching with the spoken content from the vocabularies included in the terminal dictionary, and includes the selected vocabulary in the voice recognition result.

When a plurality of words are included in the utterance content, the first speech recognition unit 13 selects the vocabulary having the highest degree of matching for each word, and includes the selected vocabulary in the speech recognition result.

The first speech recognition unit 13 sets the likelihood corresponding to the selected vocabulary as the first likelihood.

Alternatively, when the voice recognition result includes a plurality of vocabularies, the first voice recognition unit 13 calculates the likelihood for the entire plurality of vocabularies based on the likelihood corresponding to each vocabulary, and calculates the calculated likelihood. It may be the first likelihood.

The value of the first likelihood becomes higher as the degree of matching between the vocabulary selected by the first voice recognition unit 13 and the utterance content is higher.

Then, the first voice recognition unit 13 outputs the voice recognition result to the arbitration unit 14 as the first voice recognition result information. Further, the first voice recognition unit 13 outputs the first likelihood to the arbitration unit 14.

The terminal dictionary is a list in which words to be recognized are registered and are stored in the terminal 100. The terminal dictionary mainly contains, for example, words for controlling the operation of the terminal 100. The terminal dictionary may contain, for example, words for the terminal 100 to control a device arranged in the home.

The second sound collecting processing unit 21 removes noise contained in the first voice information received by the communication unit 201 at a higher level (or a larger amount) than that of the first removing method. Remove using the removal method of 2.

Conversely, the first sound collecting processing unit 12 removes the noise contained in the first voice information at a lower level (or less amount) than the second removing method. Remove using a removal method.

The second sound collection processing unit 21 outputs the voice information obtained by removing noise from the first voice information as the third voice information by using the second removal method. The second sound collecting processing unit 21 removes a larger amount of noise from the first voice information than the amount of noise removed by the first sound collecting processing unit 12.

The second voice recognition unit 22 performs voice recognition on the third voice information output by the second sound collection processing unit 21, and outputs the voice recognition result as the second voice recognition result information. The second voice recognition unit 22 calculates a second likelihood indicating the likelihood of the second voice recognition result information when voice recognition is performed, and the calculated second likelihood is used as the second voice recognition. It is output to the communication unit 201 together with the result information. The communication unit 201 transmits the second voice recognition result information output by the second voice recognition unit 22 and the second likelihood to the terminal 100. The communication unit 101 of the terminal 100 receives the second voice recognition result information transmitted by the server 200. The communication unit 101 receives the second likelihood indicating the likelihood of the second voice recognition result information calculated when the server 200 performs voice recognition, and the arbitration unit 14 receives the received second likelihood. Output to.

The second voice recognition unit 22 performs voice recognition for the third voice information from which noise has been removed by the second sound collection processing unit 21. The second voice recognition unit 22 refers to the sound model and the language model stored in advance and the server dictionary, and performs voice recognition for the third voice information. The voice recognition result includes character string data composed of a plurality of words, which is a character string of the result of voice recognition of the third voice information. The second likelihood indicates the likelihood of the speech recognition result (that is, the second speech recognition result information) of the third speech information. Specifically, the second likelihood indicates, for example, the likelihood of the entire character string data or the likelihood of each word constituting the character string data.

The second voice recognition unit 22 calculates the degree of agreement (likelihood) between the utterance content obtained from the third voice information and each of the plurality of vocabularies included in the server dictionary. The second voice recognition unit 22 selects the vocabulary having the highest degree of matching with the spoken content from the vocabularies included in the server dictionary, and includes the selected vocabulary in the voice recognition result.

When the utterance content includes a plurality of words, the second voice recognition unit 22 selects the vocabulary having the highest degree of matching for each word, and includes the selected vocabulary in the voice recognition result.

The second speech recognition unit 22 sets the likelihood corresponding to the selected vocabulary as the second likelihood.

Alternatively, when the voice recognition result includes a plurality of vocabularies, the second voice recognition unit 22 calculates the likelihood for the entire plurality of vocabularies based on the likelihood corresponding to each vocabulary, and calculates the calculated likelihood. It may be the second likelihood.

The value of the second likelihood becomes higher as the degree of matching between the vocabulary selected by the second voice recognition unit 22 and the utterance content is higher.

Then, the second voice recognition unit 22 transmits the voice recognition result as the second voice recognition result information to the terminal 100 via the communication unit 201. The second voice recognition unit 22 transmits the second likelihood to the terminal 100 via the communication unit 201.

Further, the communication unit 101 outputs the received second voice recognition result information and the second likelihood to the arbitration unit 14.

The server dictionary is a list of words to be recognized registered and stored in the server 200. The server dictionary includes not only words for controlling the operation of the terminal 100 but also various search keywords and the like. The server dictionary may include information for the terminal 100 to control the device. The number of vocabularies in the server dictionary is larger than the number of vocabularies in the terminal dictionary. The number of vocabularies in the server dictionary is, for example, 100,000 to hundreds of thousands of words, while the number of vocabularies in the terminal dictionary is, for example, tens to hundreds of words.

Here, the difference between the first sound collecting processing unit 12 and the second sound collecting processing unit 21 will be described. The first sound collecting processing unit 12 removes noise by signal processing using the power or correlation of a plurality of voice signals. On the other hand, in addition to the above signal processing, the second sound collection processing unit 21 statistically models a signal indicating voice or a signal indicating noise, and uses the probabilistic plausibility of the separated signal to generate noise. Remove.

In modeling, parameters are set in advance in order to separate the first voice information (voice signal) received by the second sound collection processing unit 21 into a voice signal corresponding to the speaker's voice and a signal corresponding to noise. You need to decide.

For example, the following processing is performed in advance. First, the parameters required for modeling are determined in advance. Then, in a noisy environment, the above model is applied to the first voice information obtained by the speaker speaking, a process of removing the signal corresponding to the noise is performed, and the voice signal obtained by this process is performed. To evaluate.

Alternatively, in a noisy environment, the first voice information obtained by the speaker speaking is subjected to the same processing as that of the first sound collection processing unit 12, and noise is removed from the first voice information. The above model may be applied to the voice signal, a process for removing the signal corresponding to the noise may be performed, and the voice signal obtained by this process may be evaluated.

If the evaluation value for the voice signal obtained by this processing is lower than the predetermined evaluation value, the first parameter obtained by modifying the above parameters and again speaking by the speaker in a noisy environment is obtained. The process of removing the signal corresponding to noise from the voice information and the evaluation of the voice signal are performed.

If the evaluation value for the voice signal obtained by this processing is higher than the predetermined evaluation value, the parameter used for obtaining the above-mentioned voice signal is used for the processing of the second sound collection processing unit 21 (pre-learning). It is retained as a parameter).

Then, using the pre-learned parameters, a signal indicating voice or a signal indicating noise is statistically modeled. The second sound collecting processing unit 21 holds the pre-learned parameters.

The pre-learned parameters statistically model a signal indicating voice or a signal indicating noise included in the first voice information acquired by the terminal 100 when the speaker speaks in a noisy environment. , It is a parameter suitable for removing noise by using the probabilistic plausibility of the separated signals.

When voice recognition is actually required, the second sound collection processing unit 21 statistically models a signal indicating voice or a signal indicating noise using pre-learned parameters, and separates the signals. Noise is removed from the first speech information using probabilistic plausibility.

At this time, the second sound collecting processing unit 21 may update the pre-learned parameters as necessary by using the first voice information acquired from the terminal 100.

By doing so, the pre-learned parameters are updated to the parameters more suitable for the environment in which the speaker is speaking.

As a general finding, a signal indicating audio or a signal indicating noise is statistically modeled and separated rather than a method of removing noise from the first audio information by signal processing using the power or correlation of the audio signal. The amount of noise that can be removed by the method of removing noise from the first audio information by using the probabilistic plausibility of the signal is larger.

As a matter of course, after the noise is removed from the first voice information by signal processing using the power or correlation of the voice signal, the signal indicating the voice included in the first voice information from which the noise is removed by this method or By statistically modeling the signal indicating noise and removing the noise by using the probabilistic plausibility of the separated signal, the noise is removed from the first audio information by signal processing using the power or correlation of the audio signal. More noise can be removed from the first audio information as compared with the case where only the removal method is used.

That is, the amount of noise removed by the second sound collecting processing unit 21 is larger than the amount of noise removed by the first sound collecting processing unit 12. Therefore, the second sound collecting processing unit 21 can sufficiently remove noise (noise) even in a noisy environment, and can extract only the user's utterance. The second sound collecting processing unit 21 holds, for example, more pre-learned parameters than the first sound collecting processing unit 12, and performs more arithmetic processing. Therefore, the time required for the second sound collecting processing unit 21 to remove noise is longer than the time required for the first sound collecting processing unit 12 to remove noise, for example, about several tens of ms to several hundreds ms longer. Further, the second sound collecting processing unit 21 can update the noise removing processing algorithm in real time, whereas the first sound collecting processing unit 12 updates the noise removing processing algorithm. The program needs to be updated.

As described above, the first sound collection processing unit 12 removes noise by using the power or correlation of the audio signal, and the second sound collection processing unit 21 statistically models the signal indicating sound or the signal indicating noise. Noise is removed using the probabilistic plausibility of the separated signals. However, these sound collecting processing units may remove noise by another method.

That is, if the second sound collecting processing unit 21 removes a larger amount of noise from the first voice information than the first sound collecting processing unit 12, the first sound collecting processing unit 12 and the first Any specific process for removing noise from the sound collecting processing unit 21 of 2 may be used.

Subsequently, the difference between the first voice recognition unit 13 and the second voice recognition unit 22 will be described. As described above, the first voice recognition unit 13 and the second voice recognition unit 22 have different numbers of vocabularies in the dictionary used for voice recognition, and the number of vocabularies in the server dictionary is that of the terminal dictionary. More than the number of vocabulary. Therefore, the number of recognizable words of the second voice recognition unit 22 is larger than the number of recognizable words of the first voice recognition unit 13. The first voice recognition unit 13 may simply convert the voice into characters without using a dictionary. The time required for the second voice recognition unit 22 to perform voice recognition is longer than the time required for the first voice recognition unit 13 to perform voice recognition, for example, it is several tens of ms to several hundred ms longer. Further, the second voice recognition unit 22 can update the speech recognition processing algorithm in real time, whereas the first voice recognition unit 13 has a program for updating the speech recognition processing algorithm. Needs an update.

The arbitration unit 14 selects which of the first voice recognition result information output by the first voice recognition unit 13 and the second voice recognition result information received by the communication unit 101 is output. To do. The arbitrator 14 makes this selection based on at least one of a first likelihood and a second likelihood. That is, the arbitration unit 14 outputs the first voice recognition result information when the first likelihood is larger than the predetermined first threshold value. Further, the arbitration unit 14 outputs the second voice recognition result information when the first likelihood is equal to or less than the first threshold value and the second likelihood is larger than the predetermined second threshold value. Further, in the arbitration unit 14, when the first likelihood is equal to or less than the first threshold value and the second likelihood is equal to or less than the second threshold value, the first speech recognition result information and the second speech recognition are performed. No result information is output. The first threshold value and the second threshold value are stored in, for example, the memory 105 of the terminal 100.

The arbitration unit 14 compares the likelihood and the threshold value of the previously input information among the first voice recognition result information and the second voice recognition result information. For example, when the first voice recognition result information is input to the arbitration unit 14 before the second voice recognition result information, the arbitration unit 14 has a first likelihood corresponding to the first voice recognition result information. Is compared with the first threshold value, and if the first likelihood is larger than the first threshold value, the first voice recognition result information is output. On the other hand, when the first likelihood is equal to or less than the first threshold value, the arbitration unit 14 waits for the second voice recognition result information to be input, and then the second voice recognition result information is input. In this case, the second likelihood corresponding to the second speech recognition result information is compared with the second threshold value, and if the second likelihood is larger than the second threshold value, the second speech recognition result information is used. Output. At this time, if the second likelihood is equal to or less than the second threshold value, the arbitration unit 14 determines that a satisfactory voice recognition result has not been obtained, and determines that the first voice recognition result information and the second voice recognition are performed. No result information is output. The above processing is also performed in the same manner when the second voice recognition result information is input before the first voice recognition result information.

FIG. 3 is a flowchart showing an example of the operation of the voice recognition system according to the first embodiment of the present disclosure.

First, in step S1, the voice acquisition unit 11 of the terminal 100 acquires the first voice information.

Next, in step S2, the communication unit 101 transmits the first voice information acquired by the voice acquisition unit 11 to the server 200.

Next, in step S3, the first sound collection processing unit 12 removes the noise included in the first voice information acquired by the voice acquisition unit 11, and outputs the second voice information from which the noise has been removed. ..

Next, in step S4, the first voice recognition unit 13 performs voice recognition on the second voice information output by the first sound collection processing unit 12, and the voice recognition result is the first voice recognition. It is output to the arbitration unit 14 as result information. Further, the first voice recognition unit 13 outputs the first likelihood indicating the likelihood of the first voice recognition result information to the arbitration unit 14.

Next, in step S5, the arbitration unit 14 determines whether or not the first likelihood indicating the likelihood of the first voice recognition result information is larger than the first threshold value. The first threshold value is a threshold value at which it can be determined that the first voice recognition result information is a correct recognition result. Here, when it is determined that the first likelihood is larger than the first threshold value (YES in step S5), the arbitration unit 14 outputs the first voice recognition result information in step S6.

On the other hand, when it is determined that the first likelihood is equal to or less than the first threshold value (NO in step S5), the process proceeds to step S7.

Here, the processes of steps S31 to S34 of the server 200 are performed in parallel with the processes of steps S3 to S5 of the terminal 100.

In step S31, the communication unit 201 of the server 200 receives the first voice information transmitted by the terminal 100.

Next, in step S32, the second sound collecting processing unit 21 removes the noise included in the first voice information received by the communication unit 201, and outputs the third voice information from which the noise has been removed.

Next, in step S33, the second voice recognition unit 22 performs voice recognition on the third voice information output by the second sound collection processing unit 21, and recognizes the voice recognition result as the second voice recognition. It is output to the communication unit 201 as the result information. Further, the second voice recognition unit 22 outputs a second likelihood indicating the likelihood of the second voice recognition result information to the communication unit 201.

Next, in step S34, the communication unit 201 transmits the second voice recognition result information and the second likelihood output from the second voice recognition unit 22 to the terminal 100.

Next, in step S7, the communication unit 101 of the terminal 100 receives the second voice recognition result information and the second likelihood transmitted by the server 200. Further, the communication unit 101 outputs the second voice recognition result information and the second likelihood to the arbitration unit 14.

Next, in step S8, the arbitration unit 14 determines whether or not the second likelihood indicating the likelihood of the second voice recognition result information is larger than the second threshold value. The second threshold value is a threshold value at which it can be determined that the second voice recognition result information is the correct recognition result. Here, when it is determined that the second likelihood is larger than the second threshold value (YES in step S8), in step S9, the arbitration unit 14 outputs the second voice recognition result information.

On the other hand, when it is determined that the second likelihood is equal to or less than the second threshold value (NO in step S8), in step S10, the display unit 106 provides voice recognition non-recognition information indicating that voice recognition could not be performed. Notify the user.

FIG. 4 is a diagram showing an example of a display screen for displaying non-speech recognition information.

As shown in FIG. 4, when the mediation unit 14 determines that the first likelihood is equal to or less than the first threshold value and the second likelihood is equal to or less than the second threshold value, the display unit 106 is determined. The voice recognition non-recognizable information 1061 is displayed on the display screen. In FIG. 4, the voice recognition impossible information 1061 is composed of the character information "Voice recognition could not be performed".

In the present embodiment, the terminal 100 notifies the user by displaying the voice recognition unrecognizable information on the display unit 106, but the terminal 100 may notify the user by another method. The present disclosure is not particularly limited to this, and the user may be notified by outputting voice from the speaker 103.

Further, in the present embodiment, the terminal 100 may notify the user of the server transmission information indicating that the first voice information is transmitted to the server 200.

FIG. 5 is a diagram showing an example of a display screen for displaying server transmission information.

As shown in FIG. 5, when the first voice information is transmitted to the server 200, the display unit 106 displays the server transmission information 1062 on the display screen. In FIG. 5, the server transmission information 1062 is composed of an icon indicating that the voice information is transmitted via the network. The display of the server transmission information 1062 may be started at the transmission start timing of the first voice information and may end at the transmission end timing. Further, the display of the server transmission information 1062 may be started at the transmission start timing of the first voice information and may end at the reception timing of the second voice recognition result information.

Note that some users may not want to transmit voice information to the server 200. Therefore, it is preferable to present to the user in advance transmission confirmation information for confirming whether or not to transmit voice information to the server 200.

FIG. 6 is a diagram showing an example of a display screen for displaying transmission confirmation information.

As shown in FIG. 6, the display unit 106 displays the transmission confirmation information 1063 on the display screen at the time of initial setting. In FIG. 6, the transmission confirmation information 1063 is composed of the character information "Can the voice be uploaded to the cloud?". The transmission confirmation information 1063 may be displayed at the time of initial setting of the terminal 100, or may be displayed at the time of first transmitting the first voice information.

In the present embodiment, the first voice recognition unit 13 has been described as an example of outputting the first voice recognition result information and the first likelihood to the arbitration unit 14, but the present invention is not limited thereto. ..

The first voice recognition unit 13 calculates the first likelihood when performing voice recognition for the second voice information. For example, the first voice recognition unit 13 may output the voice recognition result of the second voice information and the first voice recognition result information including the first likelihood to the arbitration unit 14.

In this case, the arbitration unit 14 may take out the voice recognition result and the first likelihood of the second voice information from the first voice recognition result information as necessary and perform processing.

In the present embodiment, the second voice recognition unit 22 has been described as an example of outputting the second voice recognition result information and the second likelihood to the communication unit 201, respectively, but the present invention is not limited thereto. ..

The second voice recognition unit 22 calculates the second likelihood when performing voice recognition for the third voice information. For example, the second voice recognition unit 22 may output the voice recognition result of the third voice information and the second voice recognition result information including the second likelihood to the communication unit 201.

In this case, any one of the communication unit 201, the communication unit 101, and the arbitration unit 14 obtains the voice recognition result or the second likelihood of the third voice information from the second voice recognition result information, if necessary. It may be taken out and processed.

Further, in the present embodiment, the first voice recognition unit 13 outputs the first voice recognition result information and the first likelihood to the arbitration unit 14, and the communication unit 101 is transmitted by the server 200. The voice recognition result information of the above and the one that outputs the second likelihood to the arbitration unit 14 have been described as an example, but the description is not limited to this.

If the first likelihood is input, the arbitration unit 14 can determine whether or not to output the first voice recognition result information according to the comparison result with the first threshold value held in advance.

Further, if the second likelihood is input, the arbitration unit 14 can determine whether or not to output the second voice recognition result information according to the comparison result with the second threshold value held in advance.

For example, the first voice recognition unit 13 may store the first voice recognition result information in the memory 105 of the terminal 100 instead of outputting it to the arbitration unit 14. In this case, the first voice recognition unit 13 outputs the first likelihood to the arbitration unit 14.

Further, for example, the communication unit 101 may store the second voice recognition result information in the memory 105 of the terminal 100 instead of outputting it to the arbitration unit 14. In this case, the communication unit 101 outputs the second likelihood to the arbitration unit 14.

Further, the arbitration unit 14 may take out the first voice recognition result information or the second voice recognition result information determined to be output from the memory 105 and output the information. Further, when the arbitration unit 14 determines that the first voice recognition result information and the second voice recognition result information are not output, the mediation unit 14 outputs the first voice recognition result information and the second voice recognition result information from the memory 105. You may delete it.

Further, in the present embodiment, the terminal 100 includes the arbitration unit 14, but the present disclosure is not particularly limited to this, and the server 200 may include the arbitration unit.

FIG. 7 is a diagram showing a functional configuration of a voice recognition system in a modified example of the first embodiment of the present disclosure. As shown in FIG. 7, the terminal 100 includes a voice acquisition unit 11, a first sound collection processing unit 12, and a first voice recognition unit 13. The server 200 includes a second sound collection processing unit 21, a second voice recognition unit 22, and an arbitration unit 23.

The communication unit 101 of the terminal 100 transmits the first voice recognition result information and the first likelihood output from the first voice recognition unit 13 to the server 200. The communication unit 201 of the server 200 receives the first voice recognition result information and the first likelihood transmitted by the terminal 100 and outputs them to the arbitration unit 23.

The second voice recognition unit 22 performs voice recognition on the third voice information output by the second sound collection processing unit 21, and sends the voice recognition result to the arbitration unit 23 as the second voice recognition result information. Output. Further, the second voice recognition unit 22 outputs the second likelihood to the arbitration unit 23.

The arbitration unit 23 selects which of the first voice recognition result information received by the communication unit 201 and the second voice recognition result information output by the second voice recognition unit 22 is output. To do. Since the processing of the arbitration unit 23 is the same as the processing of the arbitration unit 14, the description thereof will be omitted.

The communication unit 201 of the server 200 transmits the selection result output from the arbitration unit 23 to the terminal 100. The selection result includes either one of the first voice recognition result information and the second voice recognition result information, or information indicating that the voice recognition could not be performed. The communication unit 101 of the terminal 100 receives the selection result transmitted by the server 200.

As described above, the arbitration unit may be provided by either the terminal 100 or the server 200. When the arbitration unit is located on the server 200, the calculation amount of the terminal 100 can be reduced. Further, when the arbitration unit is located in the terminal 100, it is not necessary to receive the selection result via the network, so that the processing time can be shortened.

The first sound collecting processing unit 12 removes the noise contained in the first voice information by the first removing method, and the second sound collecting processing unit 21 removes the noise contained in the first voice information. The method of removing has been described as the second removing method. However, the first removal method and the second removal method are names of methods for removing noise contained in the first voice information performed by the first sound collection processing unit 12 and the second sound collection processing unit 21. Is.

Therefore, the first sound collecting processing unit 12 removes the noise contained in the first voice information by the second removing method, and the second sound collecting processing unit 21 removes the noise contained in the first voice information. The method of removing may be called the first removing method.

(Embodiment 2)
Subsequently, the voice recognition system according to the second embodiment will be described. Since the overall configuration of the voice recognition system according to the second embodiment is the same as that in FIG. 1, the description thereof will be omitted.

FIG. 8 is a diagram showing a functional configuration of the voice recognition system according to the second embodiment of the present disclosure. As shown in FIG. 8, the voice recognition system includes a terminal 100 and a server 200. The terminal 100 includes a voice acquisition unit 11, a first sound collection processing unit 12, a first voice recognition unit 13, and an arbitration unit 14. The server 200 includes a second sound collection processing unit 21 and a second voice recognition unit 22.

The communication unit 201 of the server 200 transmits the third voice information output from the second sound collection processing unit 21 to the terminal 100. The communication unit 101 of the terminal 100 receives the third voice information from the server 200, and outputs the received third voice information to the first voice recognition unit 13. The first voice recognition unit 13 performs voice recognition on the third voice information received by the communication unit 101, and outputs the voice recognition result to the arbitration unit 14 as the fourth voice recognition result information.

In this case, the voice recognition result includes the voice recognition result of the third voice information. Further, the first voice recognition unit 13 calculates a fourth likelihood indicating the plausibility of the recognition result, and outputs the calculated fourth likelihood to the arbitration unit 14.

In the first voice recognition unit 13 described in the first embodiment, the third voice information may be used instead of the second voice information in the voice recognition and the calculation of the likelihood. Detailed description of voice recognition for the third voice information and calculation of the fourth likelihood in the voice recognition unit 13 of 1 will be omitted.

Further, the communication unit 101 transmits the second voice information output by the first sound collection processing unit 12 to the server 200. The communication unit 201 of the server 200 receives the second voice information transmitted by the terminal 100 and outputs it to the second voice recognition unit 22. The second voice recognition unit 22 performs voice recognition on the second voice information received by the communication unit 201, and outputs the voice recognition result to the communication unit 201 as the third voice recognition result information.

In this case, the voice recognition result includes the voice recognition result of the second voice information. Further, the second voice recognition unit 22 calculates a third likelihood indicating the plausibility of the voice recognition result, and outputs the calculated third likelihood to the communication unit 201.

In the second voice recognition unit 22 described in the first embodiment, in the voice recognition and the calculation of the likelihood, the second voice information may be used instead of the third voice information. Detailed description of voice recognition for the second voice information and calculation of the third likelihood in the voice recognition unit 22 of 2 will be omitted.

The communication unit 201 transmits the third voice recognition result information and the third likelihood output from the second voice recognition unit 22 to the terminal 100. The communication unit 101 receives the third voice recognition result information, which is the voice recognition result of the second voice information, from the server 200, and outputs the received third voice recognition result information to the arbitration unit 14.

The arbitration unit 14 has the first voice recognition result information output by the first voice recognition unit 13, the second voice recognition result information received by the communication unit 101, and the third voice recognition result information received by the communication unit 101. The voice recognition result information of the above and the fourth voice recognition result information output by the first voice recognition unit 13 are selected.

The first voice recognition unit 13 calculates the first likelihood indicating the plausibility of the first voice recognition result information, and outputs the calculated first likelihood to the arbitration unit 14.

The communication unit 101 receives the second likelihood indicating the likelihood of the second voice recognition result information transmitted from the server 200, and outputs the received second likelihood to the arbitration unit 14. Further, the communication unit 101 receives the third likelihood indicating the likelihood of the third voice recognition result information transmitted from the server 200, and outputs the received third likelihood to the arbitration unit 14.

Further, the first voice recognition unit 13 calculates a fourth likelihood indicating the plausibility of the fourth voice recognition result information, and outputs the calculated fourth likelihood to the arbitration unit 14.

The arbitration unit 14 determines which of the first voice recognition result information, the second voice recognition result information, the third voice recognition result information, and the fourth voice recognition result information is output. Selection is based on at least one of a first likelihood, a second likelihood, a third likelihood, and a fourth likelihood.

FIG. 9 is a first flowchart showing an example of the operation of the voice recognition system according to the second embodiment of the present disclosure, and FIG. 10 shows an example of the operation of the voice recognition system according to the second embodiment of the present disclosure. 2 is a flowchart of FIG. 11, and FIG. 11 is a third flowchart showing an example of the operation of the voice recognition system according to the second embodiment of the present disclosure.

First, in step S41, the voice acquisition unit 11 of the terminal 100 acquires the first voice information.

Next, in step S42, the communication unit 101 transmits the first voice information acquired by the voice acquisition unit 11 to the server 200.

Next, in step S43, the first sound collection processing unit 12 removes noise included in the first voice information acquired by the voice acquisition unit 11, and outputs the second voice information from which the noise has been removed. ..

Next, in step S44, the communication unit 101 transmits the second voice information from which noise has been removed by the first sound collection processing unit 12 to the server 200.

Next, in step S45, the first voice recognition unit 13 performs voice recognition on the second voice information output by the first sound collection processing unit 12, and recognizes the voice recognition result as the first voice recognition. It is output to the arbitration unit 14 as result information. Further, the first voice recognition unit 13 calculates the first likelihood indicating the plausibility of the first voice recognition result information, and outputs the calculated first likelihood to the arbitration unit 14.

Next, in step S46, the arbitration unit 14 determines whether or not the first likelihood indicating the likelihood of the first voice recognition result information is larger than the first threshold value. The first threshold value is a threshold value at which it can be determined that the first voice recognition result information is a correct recognition result. Here, when it is determined that the first likelihood is larger than the first threshold value (YES in step S46), in step S47, the arbitration unit 14 outputs the first voice recognition result information.

On the other hand, when it is determined that the first likelihood is equal to or less than the first threshold value (NO in step S46), the process proceeds to step S48.

Here, the processes of steps S61 to S63 of the server 200 are performed in parallel with the processes of steps S44 to S46 of the terminal 100.

In step S61, the communication unit 201 of the server 200 receives the second voice information transmitted by the terminal 100.

Next, in step S62, the second voice recognition unit 22 performs voice recognition on the second voice information received by the communication unit 201, and uses the voice recognition result as the third voice recognition result information in the communication unit. Output to 201. Further, the second voice recognition unit 22 calculates a third likelihood indicating the likelihood of the third voice recognition result information, and outputs the calculated third likelihood to the communication unit 201.

Next, in step S63, the communication unit 201 transmits the third voice recognition result information and the third likelihood output from the second voice recognition unit 22 to the terminal 100.

Next, in step S48, the communication unit 101 of the terminal 100 receives the third voice recognition result information and the third likelihood transmitted by the server 200. Further, the communication unit 101 outputs the third voice recognition result information and the third likelihood to the arbitration unit 14.

Next, in step S49, the arbitration unit 14 determines whether or not the third likelihood indicating the likelihood of the third voice recognition result information is greater than the third threshold value. The third threshold value is a threshold value at which it can be determined that the third voice recognition result information is the correct recognition result. Here, when it is determined that the third likelihood is larger than the third threshold value (YES in step S49), in step S50, the arbitration unit 14 outputs the third voice recognition result information.

On the other hand, when it is determined that the third likelihood is equal to or less than the third threshold value (NO in step S49), the process proceeds to step S51.

Here, the processes of steps S71 to S73 of the server 200 are performed in parallel with the processes of steps S42 to S49 of the terminal 100.

In step S71, the communication unit 201 of the server 200 receives the first voice information transmitted by the terminal 100.

Next, in step S72, the second sound collecting processing unit 21 removes the noise included in the first voice information received by the communication unit 201, and outputs the third voice information from which the noise has been removed.

Next, in step S73, the communication unit 201 transmits the third voice information output from the second sound collection processing unit 21 to the terminal 100.

Next, in step S51, the communication unit 101 of the terminal 100 receives the third voice information transmitted by the server 200.

Next, in step S52, the first voice recognition unit 13 performs voice recognition on the third voice information received by the communication unit 101, and the voice recognition result is used as the fourth voice recognition result information in the mediation unit. Output to 14. Further, the first voice recognition unit 13 calculates a fourth likelihood indicating the plausibility of the fourth voice recognition result information, and outputs the calculated fourth likelihood to the arbitration unit 14.

Next, in step S53, the arbitration unit 14 determines whether or not the fourth likelihood indicating the likelihood of the fourth voice recognition result information is greater than the fourth threshold value. The fourth threshold value is a threshold value at which it can be determined that the fourth voice recognition result information is a correct recognition result. Here, when it is determined that the fourth likelihood is larger than the fourth threshold value (YES in step S53), in step S54, the arbitration unit 14 outputs the fourth voice recognition result information.

On the other hand, when it is determined that the fourth likelihood is equal to or less than the fourth threshold value (NO in step S53), the process proceeds to step S55.

Here, the processes of steps S74 to S75 of the server 200 are performed in parallel with the processes of steps S52 to S53 of the terminal 100.

In step S74, the second voice recognition unit 22 performs voice recognition on the third voice information output from the second sound collection processing unit 21, and uses the voice recognition result as the second voice recognition result information. Output to communication unit 201. Further, the second voice recognition unit 22 calculates the second likelihood indicating the likelihood of the second voice recognition result information, and outputs the calculated second likelihood to the communication unit 201.

Next, in step S75, the communication unit 201 transmits the second voice recognition result information and the second likelihood output from the second voice recognition unit 22 to the terminal 100.

Next, in step S55, the communication unit 101 of the terminal 100 receives the second voice recognition result information and the second likelihood transmitted by the server 200. Further, the communication unit 101 outputs the second voice recognition result information and the second likelihood to the arbitration unit 14.

Next, in step S56, the arbitration unit 14 determines whether or not the second likelihood indicating the likelihood of the second voice recognition result information is larger than the second threshold value. The second threshold value is a threshold value at which it can be determined that the second voice recognition result information is the correct recognition result. Here, when it is determined that the second likelihood is larger than the second threshold value (YES in step S56), in step S57, the arbitration unit 14 outputs the second voice recognition result information.

On the other hand, when it is determined that the second likelihood is equal to or less than the second threshold value (NO in step S56), in step S58, the display unit 106 provides voice recognition non-recognition information indicating that voice recognition could not be performed. Notify the user.

The first threshold value, the second threshold value, the third threshold value, and the fourth threshold value are stored in advance in, for example, the memory 105 of the terminal 100.

In the present embodiment, the first voice recognition unit 13 outputs the first voice recognition result information, the first likelihood, the fourth voice recognition result information, and the fourth likelihood to the arbitration unit 14, respectively. The explanation was given using a thing as an example, but it is not limited to this.

The first voice recognition unit 13 calculates the first likelihood when performing voice recognition for the second voice information. For example, the first voice recognition unit 13 may output the voice recognition result of the second voice information and the first voice recognition result information including the first likelihood to the arbitration unit 14.

In this case, the arbitration unit 14 may take out the voice recognition result and the first likelihood of the second voice information from the first voice recognition result information as necessary and perform processing.

The first voice recognition unit 13 calculates the fourth likelihood when performing voice recognition for the third voice information. For example, the first voice recognition unit 13 may output the voice recognition result of the third voice information and the fourth voice recognition result information including the fourth likelihood to the arbitration unit 14.

In this case, the arbitration unit 14 may take out the voice recognition result and the fourth likelihood of the third voice information from the fourth voice recognition result information as necessary and perform processing.

In the present embodiment, the second voice recognition unit 22 outputs the second voice recognition result information, the second likelihood, the third voice recognition result information, and the third likelihood to the communication unit 201, respectively. The explanation was given using a thing as an example, but it is not limited to this.

The second voice recognition unit 22 calculates the second likelihood when performing voice recognition for the third voice information. For example, the second voice recognition unit 22 may output the voice recognition result of the third voice information and the second voice recognition result information including the second likelihood to the communication unit 201.

In this case, any one of the communication unit 201, the communication unit 101, and the arbitration unit 14 obtains the voice recognition result or the second likelihood of the third voice information from the second voice recognition result information, if necessary. It may be taken out and processed.

The second voice recognition unit 22 calculates the third likelihood when performing voice recognition for the second voice information. For example, the second voice recognition unit 22 may output the voice recognition result of the second voice information and the third voice recognition result information including the third likelihood to the communication unit 201.

In this case, any one of the communication unit 201, the communication unit 101, and the arbitration unit 14 obtains the voice recognition result or the third likelihood of the second voice information from the third voice recognition result information, if necessary. It may be taken out and processed.

Further, in the present embodiment, the first voice recognition unit 13 transfers the first voice recognition result information, the first likelihood, the fourth voice recognition result information, and the fourth likelihood to the arbitration unit 14. An example in which the communication unit 101 outputs the second voice recognition result information, the second likelihood, the third voice recognition result information, and the third likelihood transmitted by the server 200 to the arbitration unit 14. I explained to. However, it is not limited to this.

Upon receiving the first likelihood, the arbitration unit 14 can determine whether or not to output the first voice recognition result information according to the comparison result with the first threshold value held in advance.

Upon receiving the second likelihood, the arbitration unit 14 can determine whether or not to output the second voice recognition result information according to the comparison result with the second threshold value held in advance.

Upon receiving the third likelihood, the arbitration unit 14 can determine whether or not to output the third voice recognition result information according to the comparison result with the third threshold value held in advance.

Further, when the arbitration unit 14 receives the fourth likelihood, it can determine whether or not to output the fourth voice recognition result information according to the comparison result with the fourth threshold value held in advance.

For example, the first voice recognition unit 13 may store the first voice recognition result information and the fourth voice recognition result information in the memory 105 of the terminal 100 instead of outputting them to the arbitration unit 14. In this case, the communication unit 101 outputs the first likelihood and the fourth likelihood to the arbitration unit 14.

Further, for example, the communication unit 101 may store the second voice recognition result information and the third voice recognition result information in the memory 105 of the terminal 100 instead of outputting them to the arbitration unit 14. In this case, the communication unit 101 outputs the second likelihood and the third likelihood to the arbitration unit 14.

Further, for example, the arbitration unit 14 may take out any one of the first voice recognition result information to the fourth voice recognition result information determined to be output from the memory 105 and output the information.

Further, when the arbitration unit 14 determines that none of the first voice recognition result information to the fourth voice recognition result information should be output, the first voice recognition result information to the fourth voice recognition result information to the fourth from the memory 105 The voice recognition result information may be deleted.

In the second embodiment, the process of transmitting the third voice recognition result information in step S63 is performed before the process of transmitting the third voice information in step S73. However, the process of transmitting the third voice information may be performed before the process of transmitting the third voice recognition result information.

Therefore, a modified example of the second embodiment in which the process of transmitting the third voice information is performed before the process of transmitting the third voice recognition result information will be described.

FIG. 12 is a first flowchart showing an example of the operation of the voice recognition system in the modified example of the second embodiment of the present disclosure, and FIG. 13 is a diagram of the voice recognition system in the modified example of the second embodiment of the present disclosure. It is a 2nd flowchart which shows an example of operation. The processing before step S46 in FIG. 12 is the same as the processing in steps S41 to S45 in FIG. 9, and the processing before step S101 in FIG. 12 is the same as the processing in steps S71 to S72 in FIG. The processing before step S111 in FIG. 13 is the same as the processing in steps S61 to S62 in FIG.

In step S101, the communication unit 201 transmits the third voice information output from the second sound collection processing unit 21 to the terminal 100.

Next, in step S81, the communication unit 101 of the terminal 100 receives the third voice information transmitted by the server 200.

Next, in step S82, the first voice recognition unit 13 performs voice recognition on the third voice information received by the communication unit 101, and the voice recognition result is used as the fourth voice recognition result information in the mediation unit. Output to 14. Further, the first voice recognition unit 13 calculates a fourth likelihood indicating the plausibility of the fourth voice recognition result information, and outputs the calculated fourth likelihood to the arbitration unit 14.

Next, in step S83, the arbitration unit 14 determines whether or not the fourth likelihood indicating the likelihood of the fourth voice recognition result information is greater than the fourth threshold value. The fourth threshold value is a threshold value at which it can be determined that the fourth voice recognition result information is a correct recognition result. Here, when it is determined that the fourth likelihood is larger than the fourth threshold value (YES in step S83), in step S84, the arbitration unit 14 outputs the fourth voice recognition result information.

On the other hand, when it is determined that the fourth likelihood is equal to or less than the fourth threshold value (NO in step S83), the process proceeds to step S85.

Here, in parallel with the processing of steps S44 to S83 of the terminal 100, the processing of steps S61 to S111 of the server 200 is performed.

Next, in step S111, the communication unit 201 transmits the third voice recognition result information and the third likelihood output from the second voice recognition unit 22 to the terminal 100.

Next, in step S85, the communication unit 101 of the terminal 100 receives the third voice recognition result information and the third likelihood transmitted by the server 200. The communication unit 101 outputs the third voice recognition result information and the third likelihood to the arbitration unit 14.

Next, in step S86, the arbitration unit 14 determines whether or not the third likelihood indicating the likelihood of the third voice recognition result information is larger than the third threshold value. The third threshold value is a threshold value at which it can be determined that the third voice recognition result information is the correct recognition result. Here, when it is determined that the third likelihood is larger than the third threshold value (YES in step S86), in step S87, the arbitration unit 14 outputs the third voice recognition result information.

On the other hand, when it is determined that the third likelihood is equal to or less than the third threshold value (NO in step S86), the process proceeds to step S88.

Here, in parallel with the processing of steps S82 to S86 of the terminal 100, the processing of steps S102 to S103 of the server 200 is performed.

In step S102, the second voice recognition unit 22 performs voice recognition on the third voice information output from the second sound collection processing unit 21, and uses the voice recognition result as the second voice recognition result information. Output to communication unit 201. Further, the second voice recognition unit 22 calculates the second likelihood indicating the likelihood of the second voice recognition result information, and outputs the calculated second likelihood to the communication unit 201.

Next, in step S103, the communication unit 201 transmits the second voice recognition result information and the second likelihood output from the second voice recognition unit 22 to the terminal 100.

Next, in step S88, the communication unit 101 of the terminal 100 receives the second voice recognition result information and the second likelihood transmitted by the server 200. The communication unit 101 outputs the second voice recognition result information and the second likelihood to the arbitration unit 14.

Next, in step S89, the arbitration unit 14 determines whether or not the second likelihood indicating the likelihood of the second voice recognition result information is larger than the second threshold value. The second threshold value is a threshold value at which it can be determined that the second voice recognition result information is the correct recognition result. Here, when it is determined that the second likelihood is larger than the second threshold value (YES in step S89), in step S90, the arbitration unit 14 outputs the second voice recognition result information.

On the other hand, when it is determined that the second likelihood is equal to or less than the second threshold value (NO in step S89), in step S91, the display unit 106 provides voice recognition non-recognition information indicating that voice recognition could not be performed. Notify the user.

(Embodiment 3)
Subsequently, the voice recognition system according to the third embodiment will be described. Since the overall configuration of the voice recognition system according to the third embodiment is the same as that in FIG. 1, the description thereof will be omitted.

FIG. 14 is a diagram showing a functional configuration of the voice recognition system according to the third embodiment of the present disclosure. As shown in FIG. 14, the voice recognition system includes a terminal 100 and a server 200. The terminal 100 includes a voice acquisition unit 11, a first sound collection processing unit 12, a first voice recognition unit 13, and an arbitration unit 14. The server 200 includes a second sound collecting processing unit 21.

The difference between the voice recognition system according to the first embodiment and the voice recognition system according to the third embodiment is whether or not the server 200 includes the second voice recognition unit 22.

The second sound collection processing unit 21 removes noise included in the first voice information received by the communication unit 201, and outputs the third voice information from which the noise has been removed.

The communication unit 201 of the server 200 transmits the third voice information output from the second sound collection processing unit 21 to the terminal 100.

The first voice recognition unit 13 performs voice recognition on the second voice information output by the first sound collection processing unit 12, and uses the voice recognition result as the first voice recognition result information in the mediation unit 14. Output. Further, the first voice recognition unit 13 calculates the first likelihood indicating the plausibility of the first voice recognition result information, and outputs the calculated first likelihood to the arbitration unit 14.

Further, the first voice recognition unit 13 performs voice recognition on the third voice information received by the communication unit 101, and outputs the voice recognition result to the arbitration unit 14 as the fourth voice recognition result information. Further, the first voice recognition unit 13 calculates a fourth likelihood indicating the plausibility of the fourth voice recognition result information, and outputs the calculated fourth likelihood to the arbitration unit 14.

The arbitration unit 14 outputs either the first voice recognition result information output by the first voice recognition unit 13 or the fourth voice recognition result information output by the first voice recognition unit 13. Select whether to do it. Since the processing of the arbitration unit 14 is the same as that of the other embodiments, the description thereof will be omitted.

(Embodiment 4)
Subsequently, the voice recognition system according to the fourth embodiment will be described. Since the overall configuration of the voice recognition system according to the fourth embodiment is the same as that in FIG. 1, the description thereof will be omitted.

FIG. 15 is a diagram showing a functional configuration of the voice recognition system according to the fourth embodiment of the present disclosure. As shown in FIG. 15, the voice recognition system includes a terminal 100 and a server 200. The terminal 100 includes a voice acquisition unit 11 and a first sound collection processing unit 12. The server 200 includes a second sound collection processing unit 21, a second voice recognition unit 22, and an arbitration unit 23.

The difference between the voice recognition system (FIG. 7) in the modified example of the first embodiment and the voice recognition system in the fourth embodiment is whether or not the terminal 100 includes the first voice recognition unit 13.

The communication unit 101 transmits the second voice information output by the first sound collection processing unit 12 to the server 200. The communication unit 201 of the server 200 receives the second voice information transmitted by the terminal 100 and outputs it to the second voice recognition unit 22. The second voice recognition unit 22 performs voice recognition on the second voice information received by the communication unit 201, and outputs the voice recognition result to the arbitration unit 23 as the third voice recognition result information.

The second voice recognition unit 22 performs voice recognition on the third voice information output by the second sound collection processing unit 21, and sends the voice recognition result to the arbitration unit 23 as the second voice recognition result information. Output.

The arbitration unit 23 outputs either the third voice recognition result information output from the second voice recognition unit 22 or the second voice recognition result information output from the second voice recognition unit 22. Select whether to do it. Since the processing of the arbitration unit 23 is the same as that of the other embodiments, the description thereof will be omitted.

(Embodiment 5)
Subsequently, the voice recognition system according to the fifth embodiment will be described. Since the overall configuration of the voice recognition system according to the fifth embodiment is the same as that in FIG. 1, the description thereof will be omitted.

FIG. 16 is a diagram showing a functional configuration of the voice recognition system according to the fifth embodiment of the present disclosure. As shown in FIG. 16, the voice recognition system includes a terminal 100 and a server 200. The terminal 100 includes a voice acquisition unit 11, a first sound collection processing unit 12, a first voice recognition unit 13, an arbitration unit 14, an utterance section detection unit 15, and an utterance duration measurement unit 17. The server 200 includes a second sound collection processing unit 21 and a second voice recognition unit 22.

The difference between the voice recognition system according to the first embodiment and the voice recognition system according to the fifth embodiment is whether or not the terminal 100 includes the utterance section detection unit 15 and the utterance duration measurement unit 17.

The utterance section detection unit 15 detects the utterance section spoken by the user in the first voice information acquired by the voice acquisition unit 11. The utterance section detection unit 15 detects the utterance section by using a general utterance section detection (VAD: Voice Activity Detection) technique. For example, the utterance section detection unit 15 detects whether or not the frame is a voice section based on the amplitude and the number of zero crossings in the frame composed of the time series of the input voice signal. Further, for example, the utterance section detection unit 15 calculates the probability that the user is speaking by the voice model based on the feature amount of the input voice information, and calculates the probability that the user is not speaking by the noise model. A section in which the probability obtained from the voice model is higher than the probability obtained from the noise model may be determined to be the utterance section.

The utterance duration measurement unit 17 measures the time from the start to the end of the section (frame) determined to have voice when the utterance section is detected by the utterance section detection unit 15.

When the utterance section detection unit 15 does not detect the utterance section, the first sound collection processing unit 12 does not remove the noise included in the first voice information and does not output the second voice information. Further, the communication unit 101 does not transmit the first voice information to the server 200 when the utterance section is not detected by the utterance section detection unit 15.

When the utterance section is detected by the utterance section detection unit 15, the first sound collection processing unit 12 removes noise included in the first voice information. Further, when the utterance section is detected by the utterance section detection unit 15, the communication unit 101 transmits the first voice information in the utterance section to the server 200.

The arbitration unit 14 determines which of the first voice recognition result information output by the voice recognition unit and the second voice recognition result information received by the communication unit 101 is output, at least the utterance duration. Select using information about the length of. That is, the arbitration unit 14 is weighted by multiplying the second likelihood indicating the likelihood of the second speech recognition result information when the utterance duration detected by the utterance duration measurement unit 17 is longer than a predetermined length. Is higher than the weighting that multiplies the first likelihood, which indicates the likelihood of the first speech recognition result information. If the utterance duration is longer than the predetermined length of time, it is highly likely that advanced voice instructions with a large number of words are being performed. In this case, erroneous recognition can be prevented by increasing the weighting multiplied by the voice recognition result output from the server 200.

In the voice recognition system according to the second embodiment shown in FIG. 8, the terminal 100 may include the utterance section detection unit 15. Further, in the voice recognition system according to the second embodiment shown in FIG. 8, the terminal 100 may include the utterance section detection unit 15 and the utterance duration measurement unit 17.

In this case, the arbitration unit 14 outputs any of the first voice recognition result information, the second voice recognition result information, the third voice recognition result information, and the fourth voice recognition result information. At least with information about the length of speech duration.

Further, the arbitration unit 14 shows the likelihood of the second speech recognition result information and the likelihood of the third speech recognition result information when the utterance duration is longer than a predetermined length. From the weighting that multiplies the likelihood of 3 by the first likelihood that indicates the likelihood of the first speech recognition result information and the fourth likelihood that indicates the likelihood of the fourth speech recognition result information. Also raise.

Further, the arbitration unit 14 raises the weight to be multiplied by the second likelihood more than the weight to be multiplied by the third likelihood when the utterance duration is longer than a predetermined length.

Further, in the fifth embodiment, the utterance section detection is performed on the acquired voice information, but the utterance section detection may be performed on the voice information from which noise has been removed. Therefore, a modified example of the fifth embodiment in which the utterance section is detected for the voice information from which noise has been removed will be described.

FIG. 17 is a diagram showing a functional configuration of a voice recognition system in a modified example of the fifth embodiment of the present disclosure. As shown in FIG. 17, the terminal 100 includes a voice acquisition unit 11, a first sound collection processing unit 12, a first voice recognition unit 13, an arbitration unit 14, a speech section detection unit 15, and a voice transmission determination unit 16. ..

The utterance section detection unit 15 detects the utterance section spoken by the user in the second voice information from which noise has been removed by the first sound collection processing unit 12. The utterance section detection unit 15 detects the utterance section by using a general utterance section detection technique.

The voice transmission determination unit 16 determines whether or not to transmit the first voice information acquired by the voice acquisition unit 11 based on the detection result of the utterance section by the utterance section detection unit 15. When the utterance section detection unit 15 detects the utterance section, the voice transmission determination unit 16 determines that the first voice information acquired by the voice acquisition unit 11 is transmitted, and the utterance section detection unit 15 detects the utterance section. If not, it is determined that the first voice information acquired by the voice acquisition unit 11 is not transmitted. The communication unit 101 transmits the first voice information acquired by the voice acquisition unit 11 based on the determination result by the voice transmission determination unit 16.

In this way, by detecting the utterance section for the second voice information which is the voice information from which noise has been removed, the utterance section can be detected with higher accuracy.

The voice recognition device and the voice recognition method according to the present disclosure can improve the accuracy of voice recognition in a noisy environment, can realize high-speed voice recognition in a quiet environment, and can reduce noise contained in voice information. It is useful as a voice recognition device and a voice recognition method for performing voice recognition on voice information from which noise has been removed.

11 Voice acquisition unit 12 1st sound collection processing unit 13 1st voice recognition unit 14 Mediation unit 15 Speech section detection unit 16 Voice transmission judgment unit 21 2nd sound collection processing unit 22 2nd voice recognition unit 23 Mediation unit 100 Terminal 101 Communication unit 102 Microphone 103 Speaker 104 Control unit 105 Memory 106 Display unit 200 Server 201 Communication unit 202 Control unit 203 Memory 300 Network

Claims (16)

The voice acquisition unit that acquires the first voice information,
A noise removal processing unit that removes noise contained in the first voice information acquired by the voice acquisition unit by using the first removal method, and outputs the voice information from which the noise has been removed as second voice information. When,
A voice recognition unit that performs voice recognition on the second voice information output by the noise removal processing unit and outputs the voice recognition result as the first voice recognition result information.
The first voice information acquired by the voice acquisition unit is transmitted to the server, and the amount of noise removed from the first voice information by the first removal method in the server is larger than the amount of noise. The noise contained in the first voice information is removed by using the second removal method for removing the noise from the first voice information, and voice recognition is performed for the third voice information from which the noise is removed. The communication unit that receives the voice recognition result as the second voice recognition result information from the server, which is the result of the above.
An arbitration unit that selects which of the first voice recognition result information output by the voice recognition unit and the second voice recognition result information received by the communication unit is to be output.
An utterance section detection unit that detects an utterance section spoken by a user in the first voice information acquired by the voice acquisition unit, and a speech section detection unit.
With
When the utterance section is not detected by the utterance section detection unit, the noise removal processing unit does not remove the noise included in the first voice information and does not output the second voice information.
The communication unit does not transmit the first voice information to the server,
The voice recognition unit calculates a first likelihood indicating the plausibility of the first voice recognition result information, outputs the calculated first likelihood to the arbitration unit, and outputs the calculated first likelihood.
The communication unit receives the second likelihood indicating the likelihood of the second voice recognition result information calculated by the server, outputs the received second likelihood to the arbitration unit, and outputs the received second likelihood.
The arbitration unit determines which of the first voice recognition result information and the second voice recognition result information is output, at least one of the first likelihood and the second likelihood. Select based on,
Voice recognition device. The arbitration section
When the first likelihood is larger than a predetermined first threshold value, the first voice recognition result information is output.
When the first likelihood is equal to or less than the first threshold value and the second likelihood is larger than the predetermined second threshold value, the second voice recognition result information is output.
When the first likelihood is equal to or less than the first threshold value and the second likelihood is equal to or less than the second threshold value, the first voice recognition result information and the second voice recognition result Does not output any information,
The voice recognition device according to claim 1. The voice acquisition unit that acquires the first voice information,
A noise removal processing unit that removes noise contained in the first voice information acquired by the voice acquisition unit by using the first removal method, and outputs the voice information from which the noise has been removed as second voice information. When,
A voice recognition unit that performs voice recognition on the second voice information output by the noise removal processing unit and outputs the voice recognition result as the first voice recognition result information.
The first voice information acquired by the voice acquisition unit is transmitted to the server, and the amount of noise removed from the first voice information by the first removal method in the server is larger than the amount of noise. The noise contained in the first voice information is removed by using the second removal method for removing the noise from the first voice information, and voice recognition is performed for the third voice information from which the noise is removed. The communication unit that receives the voice recognition result as the second voice recognition result information from the server, which is the result of the above.
An arbitration unit that selects which of the first voice recognition result information output by the voice recognition unit and the second voice recognition result information received by the communication unit is to be output.
An utterance section detection unit that detects an utterance section spoken by a user in the first voice information acquired by the voice acquisition unit, and a speech section detection unit.
When the utterance section is detected by the utterance section detection unit, the utterance duration measurement unit that measures the utterance duration, which is the duration of the utterance section detected by the utterance section detection unit,
With
When the utterance section is not detected by the utterance section detection unit, the noise removal processing unit does not remove the noise included in the first voice information and does not output the second voice information.
The communication unit does not transmit the first voice information to the server,
When the utterance section is detected by the utterance section detection unit, the noise removal processing unit removes noise included in the first voice information.
The communication unit transmits the first voice information in the utterance section to the server.
The arbitration unit determines at least which of the first voice recognition result information output by the voice recognition unit and the second voice recognition result information received by the communication unit is output. Select using the information on the length of speech duration,
Voice recognition device. When the utterance duration is longer than a predetermined length, the arbitration unit multiplies the second likelihood indicating the likelihood of the second voice recognition result information by the weighting of the first voice recognition result information. Higher than the weighting that multiplies the first likelihood, which indicates the likelihood of
The voice recognition device according to claim 3 . The voice acquisition unit that acquires the first voice information,
A noise removal processing unit that removes noise contained in the first voice information acquired by the voice acquisition unit by using the first removal method, and outputs the voice information from which the noise has been removed as second voice information. When,
A voice recognition unit that performs voice recognition on the second voice information output by the noise removal processing unit and outputs the voice recognition result as the first voice recognition result information.
The first voice information acquired by the voice acquisition unit is transmitted to the server, and the amount of noise removed from the first voice information by the first removal method in the server is larger than the amount of noise. The noise contained in the first voice information is removed by using the second removal method for removing the noise from the first voice information, and voice recognition is performed for the third voice information from which the noise is removed. The communication unit that receives the voice recognition result as the second voice recognition result information from the server, which is the result of the above.
An arbitration unit that selects which of the first voice recognition result information output by the voice recognition unit and the second voice recognition result information received by the communication unit is to be output.
With
The communication unit receives the third voice information from the server, outputs the received third voice information to the voice recognition unit, and outputs the received third voice information to the voice recognition unit.
The voice recognition unit performs voice recognition on the third voice information received by the communication unit, and outputs the voice recognition result as the fourth voice recognition result information.
The communication unit transmits the second voice information output by the noise removal processing unit to the server, and the voice recognition result obtained by voice recognition for the second voice information is the third voice recognition. Received from the server as the result information, and output the received third voice recognition result information to the arbitration unit.
The arbitration unit includes the first voice recognition result information output by the voice recognition unit, the second voice recognition result information received by the communication unit, and the third voice recognition result information received by the communication unit. Which of the voice recognition result information of the above and the fourth voice recognition result information output by the voice recognition unit is to be output.
Voice recognition device. The voice recognition unit calculates a first likelihood indicating the plausibility of the first voice recognition result information, outputs the calculated first likelihood to the arbitration unit, and outputs the calculated first likelihood.
The communication unit receives the second likelihood indicating the likelihood of the second voice recognition result information calculated by the server, outputs the received second likelihood to the arbitration unit, and outputs the received second likelihood.
The communication unit receives a third likelihood indicating the likelihood of the third voice recognition result information calculated by the server, outputs the received third likelihood to the arbitration unit, and outputs the third likelihood.
The voice recognition unit calculates a fourth likelihood indicating the plausibility of the fourth voice recognition result information, outputs the calculated fourth likelihood to the arbitration unit, and outputs the calculated fourth likelihood.
The arbitration unit outputs any of the first voice recognition result information, the second voice recognition result information, the third voice recognition result information, and the fourth voice recognition result information. The choice is made based on at least one of the first likelihood, the second likelihood, the third likelihood and the fourth likelihood.
The voice recognition device according to claim 5 . The utterance section detection unit for detecting the utterance section spoken by the user in the first voice information acquired by the voice acquisition unit is further provided.
When the utterance section is not detected by the utterance section detection unit, the noise removal processing unit does not remove the noise included in the first voice information and does not output the second voice information.
The communication unit does not transmit the first voice information to the server.
The voice recognition device according to claim 5 or 6 . When the utterance section is detected by the utterance section detection unit, the utterance duration measuring unit further comprises a utterance duration measuring unit that measures the utterance duration, which is the duration of the utterance section detected by the utterance section detection unit.
When the utterance section is detected by the utterance section detection unit, the noise removal processing unit removes noise included in the first voice information.
The communication unit transmits the first voice information in the utterance section to the server.
The arbitration unit outputs any of the first voice recognition result information, the second voice recognition result information, the third voice recognition result information, and the fourth voice recognition result information. The choice is made using at least the information regarding the length of the speech duration.
The voice recognition device according to claim 7 . The arbitration unit shows the likelihood of the second voice recognition result information and the likelihood of the third voice recognition result information when the utterance duration is longer than a predetermined length. The weighting to be multiplied by the third likelihood is multiplied by the first likelihood indicating the plausibility of the first speech recognition result information and the fourth likelihood indicating the plausibility of the fourth speech recognition result information. Raise more than the weight to do
The voice recognition device according to claim 8 . When the utterance duration is longer than a predetermined length, the arbitrator increases the weight to be multiplied by the second likelihood more than the weight to be multiplied by the third likelihood.
The voice recognition device according to claim 9 . The voice acquisition unit that acquires the first voice information,
A noise removal processing unit that removes noise contained in the first voice information acquired by the voice acquisition unit by using the first removal method, and outputs the voice information from which the noise has been removed as second voice information. When,
The first voice information acquired by the voice acquisition unit is transmitted to the server, and the amount of noise is larger than the amount of noise removed from the first voice information by the first removal method in the server. A communication unit that receives a third voice information from the server from which noise contained in the first voice information has been removed by using a second removal method that removes noise from the first voice information.
Voice recognition is performed on the second voice information output by the noise removal processing unit, the voice recognition result is output as the first voice recognition result information, and the third voice information received by the communication unit is output. A voice recognition unit that performs voice recognition on voice information and outputs the voice recognition result as second voice recognition result information.
An arbitration unit that selects which of the first voice recognition result information and the second voice recognition result information output by the voice recognition unit is to be output.
An utterance section detection unit that detects an utterance section spoken by a user in the first voice information acquired by the voice acquisition unit, and a speech section detection unit.
With
When the utterance section is not detected by the utterance section detection unit, the noise removal processing unit does not remove the noise included in the first voice information and does not output the second voice information.
The communication unit does not transmit the first voice information to the server,
The voice recognition unit calculates a first likelihood indicating the plausibility of the first voice recognition result information, outputs the calculated first likelihood to the arbitration unit, and outputs the calculated first likelihood.
The voice recognition unit calculates a second likelihood indicating the plausibility of the second voice recognition result information, outputs the calculated second likelihood to the arbitration unit, and outputs the calculated second likelihood.
The arbitration unit determines which of the first voice recognition result information and the second voice recognition result information is output, at least one of the first likelihood and the second likelihood. Select based on,
Voice recognition device. It is a voice recognition method in a server having a communication unit, a noise removal processing unit, a voice recognition unit , a mediation unit, and a speech section detection unit , and performing voice recognition for voice information acquired by a terminal.
The communication unit receives the first voice information acquired by the terminal, and receives the first voice information.
The noise removal processing unit removes the noise contained in the received first voice information by using the first removal method, and outputs the voice information from which the noise has been removed as the second voice information.
The voice recognition unit performs voice recognition on the second voice information and outputs the voice recognition result as the first voice recognition result information.
The first removal method is used by the communication unit to remove a smaller amount of noise than the amount of noise removed from the first voice information by the first removal method in the terminal. The noise included in the voice information is removed, and the voice recognition result, which is the result of voice recognition being performed on the third voice information from which the noise is removed, is received from the terminal as the second voice recognition result information. And
The arbitration unit selects which of the first voice recognition result information and the second voice recognition result information is output.
The utterance section detection unit detects the utterance section spoken by the user in the first voice information received by the communication unit.
When the utterance section is not detected by the utterance section detection unit, the noise removal processing unit does not remove the noise included in the first voice information and does not output the second voice information.
The voice recognition unit calculates a first likelihood indicating the plausibility of the first voice recognition result information, and outputs the calculated first likelihood to the arbitration unit.
The communication unit receives the second likelihood indicating the likelihood of the second voice recognition result information calculated by the terminal, outputs the received second likelihood to the arbitration unit, and outputs the received second likelihood.
Which of the first voice recognition result information and the second voice recognition result information is output by the arbitration unit is determined by at least one of the first likelihood and the second likelihood. Select based on,
Speech recognition method. The voice acquisition unit that acquires the first voice information,
A noise removal processing unit that removes noise contained in the first voice information acquired by the voice acquisition unit by using the first removal method, and outputs the voice information from which the noise has been removed as second voice information. When,
The first voice information acquired by the voice acquisition unit is transmitted to the server, and the amount of noise is larger than the amount of noise removed from the first voice information by the first removal method in the server. A communication unit that receives a third voice information from the server from which noise contained in the first voice information has been removed by using a second removal method that removes noise from the first voice information.
Voice recognition is performed on the second voice information output by the noise removal processing unit, the voice recognition result is output as the first voice recognition result information, and the third voice information received by the communication unit is output. A voice recognition unit that performs voice recognition on voice information and outputs the voice recognition result as second voice recognition result information.
An arbitration unit that selects which of the first voice recognition result information and the second voice recognition result information output by the voice recognition unit is to be output.
An utterance section detection unit that detects an utterance section spoken by a user in the first voice information acquired by the voice acquisition unit, and a speech section detection unit.
When the utterance section is detected by the utterance section detection unit, the utterance duration measurement unit that measures the utterance duration, which is the duration of the utterance section detected by the utterance section detection unit,
With
When the utterance section is not detected by the utterance section detection unit, the noise removal processing unit does not remove the noise included in the first voice information and does not output the second voice information.
The communication unit does not transmit the first voice information to the server,
When the utterance section is detected by the utterance section detection unit, the noise removal processing unit removes noise included in the first voice information.
The communication unit transmits the first voice information in the utterance section to the server.
The arbitration unit determines which of the first voice recognition result information and the second voice recognition result information output by the voice recognition unit is output, at least with respect to the length of the utterance duration. Select with information,
Voice recognition device. It is a voice recognition method in a server that includes a communication unit, a noise removal processing unit, a voice recognition unit, a mediation unit, a utterance section detection unit, and a utterance duration measurement unit, and performs voice recognition on voice information acquired by a terminal. ,
The communication unit receives the first voice information acquired by the terminal, and receives the first voice information.
The noise removal processing unit removes the noise contained in the received first voice information by using the first removal method, and outputs the voice information from which the noise has been removed as the second voice information.
The voice recognition unit performs voice recognition on the second voice information and outputs the voice recognition result as the first voice recognition result information.
The first removal method is used by the communication unit to remove a smaller amount of noise than the amount of noise removed from the first voice information by the first removal method in the terminal. The noise included in the voice information is removed, and the voice recognition result, which is the result of voice recognition being performed on the third voice information from which the noise is removed, is received from the terminal as the second voice recognition result information. And
The arbitration unit selects which of the first voice recognition result information and the second voice recognition result information is output.
The utterance section detection unit detects the utterance section spoken by the user in the first voice information received by the communication unit.
When the utterance section is detected by the utterance section detection unit, the utterance duration measuring unit measures the utterance duration, which is the duration of the utterance section detected by the utterance section detection unit.
When the utterance section is not detected by the utterance section detection unit, the noise removal processing unit does not remove the noise included in the first voice information and does not output the second voice information.
When the utterance section is detected by the utterance section detection unit, the noise removal processing unit removes the noise included in the first voice information.
Which of the first voice recognition result information and the second voice recognition result information output by the voice recognition unit is output by the arbitration unit is related to at least the length of the utterance duration. Select with information,
Speech recognition method. The voice acquisition unit that acquires the first voice information,
A noise removal processing unit that removes noise contained in the first voice information acquired by the voice acquisition unit by using the first removal method, and outputs the voice information from which the noise has been removed as second voice information. When,
The first voice information acquired by the voice acquisition unit is transmitted to the server, and the amount of noise is larger than the amount of noise removed from the first voice information by the first removal method in the server. A communication unit that receives a third voice information from the server from which noise contained in the first voice information has been removed by using a second removal method that removes noise from the first voice information.
Voice recognition is performed on the second voice information output by the noise removal processing unit, the voice recognition result is output as the first voice recognition result information, and the third voice information received by the communication unit is output. A voice recognition unit that performs voice recognition on voice information and outputs the voice recognition result as second voice recognition result information.
An arbitration unit that selects which of the first voice recognition result information and the second voice recognition result information output by the voice recognition unit is to be output.
With
The communication unit receives the voice recognition result, which is the result of voice recognition for the third voice information in the server, as the third voice recognition result information from the server, and receives the third voice information. Output the voice recognition result information of 3 to the arbitration unit,
The communication unit transmits the second voice information output by the noise removal processing unit to the server, and the voice recognition result obtained by voice recognition for the second voice information is the fourth voice. Received from the server as recognition result information, and output the received fourth voice recognition result information to the arbitration unit.
The arbitration unit includes the first voice recognition result information output by the voice recognition unit, the second voice recognition result information output by the voice recognition unit, and the first voice recognition result information received by the communication unit. Select which of the voice recognition result information of 3 and the fourth voice recognition result information received by the communication unit is to be output.
Voice recognition device. It is a voice recognition method in a server having a communication unit, a noise removal processing unit, a voice recognition unit, and a mediation unit, and performing voice recognition for voice information acquired by a terminal.
The communication unit receives the first voice information acquired by the terminal, and receives the first voice information.
The noise removal processing unit removes the noise contained in the received first voice information by using the first removal method, and outputs the voice information from which the noise has been removed as the second voice information.
The voice recognition unit performs voice recognition on the second voice information and outputs the voice recognition result as the first voice recognition result information.
The first removal method is used by the communication unit to remove a smaller amount of noise than the amount of noise removed from the first voice information by the first removal method in the terminal. The noise included in the voice information is removed, and the voice recognition result, which is the result of voice recognition being performed on the third voice information from which the noise is removed, is received from the terminal as the second voice recognition result information. And
The arbitration unit selects which of the first voice recognition result information and the second voice recognition result information is output.
The communication unit transmits the second voice information to the terminal,
At the terminal, the communication unit removes noise contained in the second voice information by using the second removal method, and voice recognition is performed on the fourth voice information from which the noise has been removed. The voice recognition result, which is the result of the noise recognition, is received from the terminal as the third voice recognition result information.
The communication unit receives the third voice information from the terminal and outputs the received third voice information to the voice recognition unit.
The voice recognition unit performs voice recognition on the third voice information received by the communication unit, and outputs the voice recognition result as the fourth voice recognition result information.
The arbitration unit has the first voice recognition result information output by the voice recognition unit, the second voice recognition result information received by the communication unit, and the third voice recognition result information received by the communication unit. Which of the voice recognition result information of the above and the fourth voice recognition result information output by the voice recognition unit is to be output.
Speech recognition method.