KR20200044175A - Electronic apparatus and assistant service providing method thereof - Google Patents

Abstract

An electronic device is disclosed. According to the present invention, the electronic device comprises a communication unit, a memory, and a processor connected to the communication unit and the memory and controlling the electronic device. By executing at least one instruction stored in the memory, the processor transmits information about a user voice acquired by the electronic device to a plurality of servers providing different secretary services through the communication unit when a user input for running a secretary service is received, and provides, when a plurality of response information is received from the plurality of servers, a response for the user voice based on at least one piece of the plurality of received response information. The plurality of servers provide secretary services using an artificial intelligence agent.

Description

Electronic device and its secretary service provision method {ELECTRONIC APPARATUS AND ASSISTANT SERVICE PROVIDING METHOD THEREOF}

The present disclosure relates to an electronic device and a method of providing a secretarial service thereof, and more specifically, to an electronic device providing a secretarial service and a method of providing a secretarial service thereof.

In recent years, artificial intelligence systems that embody human-level intelligence have been used in various fields. The artificial intelligence system is a system in which a machine learns, judges, and becomes smart unlike a rule-based smart system. As the artificial intelligence system is used, the recognition rate is improved and the user's taste can be more accurately understood, and the existing rule-based smart system is gradually being replaced by a deep learning-based artificial intelligence system.

Artificial intelligence technology is composed of machine learning (eg, deep learning) and elemental technologies utilizing machine learning.

Machine learning is an algorithm technology that classifies / learns the characteristics of input data by itself, and element technology is a technology that simulates functions such as cognition and judgment of the human brain by using machine learning algorithms such as deep learning. It consists of technical fields such as understanding, reasoning / prediction, knowledge expression, and motion control.

Currently, artificial intelligence technology is used in various fields, and in particular, an assistant service (e.g., voice assistant) that provides a response to user input (especially user voice) using an artificial intelligence model in recent years. ) Or a voice assistant service is provided.

Although these secretary services have different domains each having excellent performance, the current electronic device provides only one secretary service, so the search for a method for providing the secretary service to users more usefully Is requested.

The present invention is in accordance with the above-mentioned needs, and an object of the present invention is to provide an electronic device capable of providing various secretary services and a method for providing secretary services thereof.

An electronic device according to an embodiment of the present disclosure includes a communication unit, a memory, and a processor connected to the communication unit and the memory to control the electronic device, wherein the processor executes at least one instruction stored in the memory. When a user input for executing a secretary service is received, information on the user voice acquired by the electronic device is transmitted to a plurality of servers providing different secretary services through the communication unit, and a plurality of When response information is received, a response to the user voice is provided based on at least one of the received response information, and the plurality of servers provide a secretary service using an artificial intelligence agent do.

Here, when the user input for executing the first secretary service is received, the processor transmits information on the user voice to the first server providing the first secretary service and the second server providing the second secretary service. And, a response to the user voice may be provided based on the response information received from the first server among the plurality of response information received from the first and second servers.

In addition, when the first wakeup word for executing the first secretary service is confirmed from the first user voice acquired by the electronic device, the processor may transmit information about the first user voice to the first secretary. Transmitted to the first server providing the service and the second server providing the second secretary service, and received from the first server and the second server in response to transmission of information about the first user voice A response to the first user voice may be provided based on the response information received from the first server among the plurality of response information.

Here, the processor may check whether the second user voice obtained after the first user voice corresponds to a voice requesting a response result of another secretary service to the first user voice.

In addition, when the second user voice corresponds to a voice requesting a response result of the second secretary service to the first user voice, the processor is received in response to transmission of information about the first user voice. A response to the second user voice may be provided based on the response information received from the second server among the plurality of response information.

In addition, when the second user voice does not correspond to the voice requesting the response result of the other secretary service, the processor transmits information on the second user voice to the first server and the second server, and In response to the transmission of information on the second user voice, the response to the second user voice is based on the response information received from the first server among the plurality of response information received from the first server and the second server. Can provide.

Meanwhile, when the user voice is obtained after the user input is received by selecting a preset button, the processor transmits information on the user voice to the plurality of servers and a plurality of response information from the plurality of servers. When is received, a plurality of responses to the user voice may be provided based on the plurality of response information.

Here, the processor may output the plurality of responses in different forms based on at least one of a user's preference for a plurality of secretary services provided by the plurality of servers and the accuracy of the plurality of response information.

On the other hand, a method for providing a secretarial service of an electronic device according to an embodiment of the present disclosure, when a user input for executing a secretary service is received, a plurality of different secretary service providing information about the user voice acquired by the electronic device And providing a response to the user's voice based on at least one of the received plurality of response information, when the plurality of response information is received from the plurality of servers. The server provides a secretary service using an artificial intelligence agent.

Here, in the transmitting step, when a user input for executing the first secretary service is received, the first server providing the first secretarial service and the second server providing the second secretarial service for information on the user's voice And providing the response, the response to the user voice may be provided based on the response information received from the first server among the plurality of response information received from the first and second servers.

In addition, when the first wakeup word for executing the first secretary service is confirmed from the first user voice acquired by the electronic device, the transmitting step may include information on the first user voice. 1 transmitting to the first server providing the secretarial service and the second server providing the second secretarial service, and the providing step comprises the first server and the first server in response to the transmission of information on the first user voice A response to the first user voice may be provided based on the response information received from the first server among the plurality of response information received from the second server.

Here, the method for providing a secretary service according to the present disclosure determines whether a second user voice obtained after the first user voice corresponds to a voice requesting a response result of another secretary service to the first user voice It may further include a step.

In addition, in the method for providing a secretarial service according to the present disclosure, when the second user voice corresponds to a voice requesting a result of the response of the second secretary service to the first user voice, information regarding the first user voice The method may further include providing a response to the second user voice based on the response information received from the second server among the plurality of response information received in response to the transmission.

In addition, in the method for providing a secretarial service according to the present disclosure, when the second user voice does not correspond to a voice requesting a response result of the other secretary service, information on the second user voice is provided to the first server and the second A second response based on the response information received from the first server among a plurality of response information received from the first server and the second server in response to transmission of information about the second user voice. And providing a response to the user's voice.

On the other hand, when the user voice is obtained after the user input is received by selecting a preset button, the transmitting step transmits information on the user voice to the plurality of servers, and the providing step comprises the plurality of steps. When a plurality of response information is received from the server of, it is possible to provide a plurality of responses to the user voice based on the plurality of response information.

Here, the providing step may output the plurality of responses in different forms based on at least one of a user's preference for a plurality of secretary services provided by the plurality of servers and accuracy of the plurality of response information. .

According to various embodiments of the present disclosure, satisfaction and convenience of a user who uses a secretary service may be improved in that answers to various secretary services can be provided.

1 is a view for explaining a system for providing a secretary service according to an embodiment of the present disclosure;
2A and 2B are block diagrams illustrating a configuration of an electronic device according to an embodiment of the present disclosure;
3 to 18 are diagrams for explaining a method of providing a secretary service according to various embodiments of the present disclosure;
19 is a block diagram illustrating the configuration of an electronic device according to an embodiment of the present disclosure;
20 and 21 are diagrams for explaining a method of providing a guide related to a secretary service according to various embodiments of the present disclosure, and
22 is a flowchart illustrating a secretary service providing method according to an embodiment of the present disclosure.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings. However, this is not intended to limit the techniques described in this document to specific embodiments, and should be understood to include various modifications, equivalents, and / or alternatives of embodiments of the document. . In connection with the description of the drawings, similar reference numerals may be used for similar components.

In this document, expressions such as "have," "can have," "includes," or "can contain," the existence of a corresponding feature (eg, a component such as a numerical value, function, operation, or part) And does not exclude the presence of additional features.

In this document, expressions such as “A or B,” “at least one of A or / and B,” or “one or more of A or / and B”, etc. may include all possible combinations of the items listed together. . For example, “A or B,” “at least one of A and B,” or “at least one of A or B,” (1) includes at least one A, (2) includes at least one B, Or (3) all cases including both at least one A and at least one B.

Expressions such as "first," "second," "first," or "second," as used herein may modify various components, regardless of order and / or importance, and denote one component. It is used to distinguish from other components, but does not limit the components.

Some component (eg, first component) is "(functionally or communicatively) coupled with / to" another component (eg, second component), or " When referred to as "connected to", it should be understood that any of the above components may be directly connected to the other component or may be connected through another component (eg, a third component). On the other hand, when it is mentioned that a component (eg, a first component) is “directly connected” or “directly connected” to another component (eg, a second component), the component and the component It can be understood that there are no other components (eg, the third component) between the other components.

As used herein, the expression "configured to" is, depending on the context, for example, "having the capacity to" , "" Designed to, "" adapted to, "" made to, "or" capable of "can be used interchangeably. The term "configured (or set) to" may not necessarily mean only "specifically designed to" in hardware. Instead, in some situations, the expression "a device configured to" may mean that the device "can" with other devices or parts. For example, the phrase “subprocessor configured (or set) to perform A, B, and C” executes a dedicated processor (eg, an embedded processor) to perform the operation, or one or more software programs stored in the memory device. By doing so, it may mean a general-purpose processor (eg, a CPU or application processor) capable of performing the corresponding operations.

An electronic device according to various embodiments of the present disclosure includes, for example, a smart phone, a tablet PC, a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a workstation, a server, a PDA, and a PMP. (portable multimediaplayer), an MP3 player, a medical device, a camera, or a wearable device. Wearable devices are accessories (e.g. watches, rings, bracelets, anklets, necklaces, glasses, contact lenses, or head-mounted-device (HMD)), fabrics or garments (e.g. electronic clothing), It may include at least one of a body-attached type (eg, a skin pad or a tattoo), or a bio-implantable circuit In some embodiments, the electronic device may be, for example, a television, a digital video disk (DVD) player, Speaker, audio, refrigerator, air conditioner, cleaner, oven, microwave, washing machine, air purifier, set-top box, home automation control panel, security control panel, media box (e.g. Samsung HomeSync ^TM , Apple TV ^TM , or Google TV) ^TM ), a game console (eg, Xbox ^TM , PlayStation ^TM ), an electronic dictionary, an electronic key, a camcorder, or at least one of an electronic picture frame.

In another embodiment, the electronic device includes various medical devices (eg, various portable medical measurement devices (such as a blood glucose meter, heart rate monitor, blood pressure meter, or body temperature meter), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), CT (computed tomography), camera, or ultrasound, etc., navigation device, global navigation satellite system (GNSS), event data recorder (EDR), flight data recorder (FDR), automotive infotainment device, marine electronic equipment (E.g. navigational devices for ships, gyro compasses, etc.), avionics, security devices, head units for vehicles, robots for industrial or domestic use, drones, ATMs in financial institutions, point-of-sale at stores of sales), or at least one of Internet of Things devices (eg light bulbs, various sensors, sprinkler devices, fire alarms, thermostats, street lights, toasters, exercise equipment, hot water tanks, heaters, boilers, etc.). .

In this document, the term user may refer to a person using an electronic device or a device using an electronic device (eg, an artificial intelligence electronic device).

1 is a view for explaining a system for providing a secretary service according to an embodiment of the present disclosure.

1A to 1D, the electronic device 100 is illustrated as a television, but this is only an example, and the electronic device 100 may be implemented as various types of devices as described above.

First, referring to FIG. 1A, the electronic device 100 may provide a secretary service (or a voice secretary service).

To this end, the electronic device 100 may acquire a user voice and transmit information about the obtained user voice to a plurality of servers 200-1, 200-2, ..., 200-n.

For example, information about the user's voice may be a voice file. That is, the electronic device 100 may generate a voice file including the acquired user voice, such as a voice wave file, and transmit it to a plurality of servers 200-1, 200-2, ..., 200-n. have.

As another example, the information on the user's voice may be a compressed voice file. That is, the electronic device 100 compresses the acquired user voice with a codec or the like to generate a compressed voice file and transmits the compressed voice file to a plurality of servers 200-1, 200-2, ..., 200-n. have.

As another example, the electronic device 100 may remove, for example, a noise removing operation, an operation of removing a voice of a speaker other than the voice of a specific speaker, and an audio signal prior to the time when the actual voice starts. At least one pre-processing operation may be performed, and the pre-processed user voice may be transmitted to a plurality of servers 200-1, 200-2, ..., 200-n. In this case, the pre-processed user voice may be transmitted to a plurality of servers 200-1, 200-2, ..., 200-n in the form of a voice file or a compressed voice file.

As another example, the electronic device 100 may extract a feature point from the acquired user voice, and transmit information about the extracted feature point to a plurality of servers 200-1, 200-2, ..., 200-n. .

On the other hand, a plurality of servers (200-1, 200-2, ..., 200-n) may be a server that provides a secretary service using an artificial intelligence agent (Artificial intelligence agent).

To this end, a plurality of servers 200-1, 200-2, ..., 200-n may include a dialogue system that can provide a response to a user's voice using an artificial intelligence model. have.

Specifically, the plurality of servers 200-1, 200-2, ..., 200-n perform pre-processing for the user's voice, and perform voice recognition for the user's voice to convert the user's voice into text, Based on the result of the speech recognition, the intent and entity of the user's voice may be identified. Then, a plurality of servers (200-1, 200-2, ..., 200-n) acquires information on the response to the user's voice based on the natural language understanding result, and based on the obtained information to the user's voice As response information, a natural language can be obtained. Then, the plurality of servers 200-1, 200-2, ..., 200-n may transmit response information to the electronic device 100.

To this end, a plurality of servers (200-1, 200-2, ..., 200-n) includes an Automatic Speech Recognition (ASR) module, a Natural Language Understanding (NLU) module, and conversation management ( Dialogue Management (DM) module, Natural Language Generation (NLG) module, and the like.

Meanwhile, the electronic device 100 may provide a response to the user's voice based on the received response information.

For example, the electronic device 100 converts text included in the received response information into voice through text to speech (TTS), and through a speaker of the electronic device 100 or a speaker connected to the electronic device 100 It is possible to output or display a user interface including the corresponding text through the display of the electronic device 100.

Thereby, the conversation system can provide a response to the user's voice, so that the user can perform a conversation with the electronic device 100.

On the other hand, a plurality of servers (200-1, 200-2, ..., 200-n) is the user's voice includes the intention to control the electronic device 100 or an external electronic device, the electronic target of control A command for controlling the device may be transmitted to the electronic device 100 or an electronic device (not shown) capable of controlling the electronic device to be controlled.

For example, assume that the user voice includes an intention to turn on a light (not shown) constituting the Internet of Things (IoT) environment together with the electronic device 100. In this case, the plurality of servers 200-1, 200-2, ..., 200-n may transmit a command to turn on a light (not shown) to the electronic device 100. In this case, the electronic device 100 transmits a corresponding command to a light (not shown), and the light (not shown) may be turned on by a command received from the electronic device 100.

However, this is only an example, and the plurality of servers 200-1, 200-2, ..., 200-n may transmit control commands in various ways.

Specifically, the plurality of servers 200-1, 200-2, ..., 200-n may transmit a control command to an electronic device (not shown) to be controlled. In the above example, the plurality of servers 200-1, 200-2, ..., 200-n transmit a command to turn on a light (not shown) to a light (not shown), and a light (not shown) ) May be turned on by commands received from a plurality of servers 200-1, 200-2, ..., 200-n.

Also, a plurality of servers 200-1, 200-2, ..., 200-n transmits a control command to another electronic device (not shown), and another electronic device (not shown) controls the corresponding command. It can be transmitted to the electronic self-governing (not shown). In the above example, it is assumed that a refrigerator (not shown) among a plurality of electronic devices constituting the Internet of Things environment performs a function of controlling an electronic device constituting the Internet of Things environment. In this case, the plurality of servers 200-1, 200-2, ..., 200-n may transmit a command to turn on a light (not shown) to a refrigerator (not shown). In this case, the refrigerator (not shown) transmits the corresponding command to a light (not shown), and the light (not shown) can be turned on by a command received from the refrigerator (not shown).

On the other hand, a plurality of servers (200-1, 200-2, ..., 200-n) may store an AI agent for operating the conversation system. Specifically, a plurality of servers 200-1, 200-2, ..., 200-n may use an artificial intelligence agent to generate natural language in response to a user's voice. The AI agent is a dedicated program for providing AI (Artificial Intelligence) based services (eg, voice recognition service, secretary service, translation service, search service, etc.), and is a conventional general-purpose processor (for example, CPU). Alternatively, it may be executed by a separate AI-only processor (eg, GPU, etc.). In particular, the AI agent can control various modules.

Further, according to an embodiment of the present disclosure, a plurality of servers 200-1, 200-2, ..., 200-n may store an artificial intelligence model trained to generate (or acquire) natural language. The artificial intelligence model learned in the present disclosure may be constructed in consideration of application fields of a recognition model or computer performance of a device. In order to generate natural natural language, the trained artificial intelligence model may be, for example, a model based on a neural network. The artificial intelligence model can be designed to simulate a human brain structure on a computer and can include a plurality of network nodes with weights, simulating neurons of a human neural network. A plurality of network nodes may each form a connection relationship so that neurons simulate synaptic activity of neurons that send and receive signals through synapses. In addition, the trained artificial intelligence model may include, for example, a neural network model or a deep learning model developed from a neural network model. In the deep learning model, a plurality of network nodes may be located at different depths (or layers) and exchange data according to a convolutional connection relationship. Examples of the learned artificial intelligence model may include, but are not limited to, Deep Neural Network (DNN), Recurrent Neural Network (RNN), and Bidirectional Recurrent Deep Neural Network (BRDNN).

Meanwhile, as described above, for voice recognition, operations such as pre-processing the user voice, converting the user voice to text, and semantic recognition (ie, intention and entity identification) in the text are performed. In the above example, these operations are described as being performed by a plurality of servers 200-1, 200-2, ..., 200-n, but this is only an example, and at least some of these operations are different electronic devices. (Not shown).

Specifically, at least some of the operations for speech recognition may be performed in the electronic device 100. In this case, a module for performing the corresponding operation may be provided in an electronic device (not shown).

For example, the electronic device 100 may perform pre-processing of the user's voice and convert the user's voice into text. In addition, the electronic device 100 may transmit information on the text in which the user's voice is converted to a plurality of servers 200-1, 200-2, ..., 200-n. In this case, the plurality of servers 200-1, 200-2, ..., 200-n grasp the intention and entities of the user's voice from the text, and respond to the user's voice based on the natural language understanding result. It can be transmitted to the electronic device 100.

As another example, the electronic device 100 performs pre-processing of the user's voice, converts the user's voice into text, grasps the intent and entities for the user's voice in the text, and obtains information about the natural language understanding result from a plurality of servers (200-1, 200-2, ..., 200-n). In this case, the plurality of servers 200-1, 200-2, ..., 200-n acquire information on the response to the user's voice based on the information received from the electronic device 100, and the obtained As a response information to the user's voice based on the information, a natural language can be obtained. Then, the plurality of servers 200-1, 200-2, ..., 200-n may transmit response information to the electronic device 100.

Meanwhile, at least some of the operations for speech recognition may be performed in another electronic device (not shown). In this case, a module for performing the corresponding operation may be provided in another electronic device (not shown).

For example, as illustrated in FIG. 1B, it is assumed that the electronic device 100 configures the Internet of Things environment in the home together with the refrigerator 10, the washing machine 20, the light fixture 30, and the air conditioner 40.

In this case, the electronic device 100 may transmit the acquired user voice to the refrigerator 10. At this time, the refrigerator 10 may perform pre-processing on the user voice received from the electronic device 100, convert the user voice into text, and transmit information on the text in which the user voice is converted to the electronic device 100. have. In this case, the electronic device 100 may transmit text information received from the refrigerator 10 to a plurality of servers 200-1, 200-2, ..., 200-n. The plurality of servers 200-1, 200-2, ..., 200-n identify the intention and entities of the user's voice from the text, and provide response information for the user's voice based on the natural language understanding result ( 100).

In addition, at least some of the operations for speech recognition may be performed in a plurality of electronic devices (not shown).

For example, in FIG. 1B, the electronic device 100 may transmit the acquired user voice to the refrigerator 10. At this time, the refrigerator 10 may perform pre-processing on the user voice received from the electronic device 100 and transmit the pre-processed user voice to the electronic device 100. In this case, the electronic device 100 may transmit the user's voice received from the refrigerator 10 to the air conditioner 40. At this time, the air conditioner 40 may convert the user voice received from the electronic device 100 into text, and transmit information on the text converted from the user voice to the electronic device 100. In this case, the electronic device 100 may transmit text information received from the air conditioner 40 to a plurality of servers 200-1, 200-2, ..., 200-n. The plurality of servers 200-1, 200-2, ..., 200-n identify the intention and entities of the user's voice from the text, and provide response information for the user's voice based on the natural language understanding result ( 100).

Meanwhile, in the above-described example, although the electronic device 100 is described as being implemented as a television, the electronic device 100 may be implemented as various types of devices.

For example, as illustrated in FIG. 1C, the electronic device may be implemented as a speaker 100 having a microphone. In this case, the speaker 100 may acquire a user voice through a microphone and transmit the obtained user voice to a plurality of servers 200-1, 200-2, ..., 200-n. In addition, the speaker 100 may convert and output response information received from a plurality of servers 200-1, 200-2, ..., 200-n into voice.

Meanwhile, the response information may be provided by an electronic device (not shown) other than the electronic device 100.

For example, assume that the electronic device is implemented as a speaker 100 equipped with a microphone, as shown in FIG. 1D.

In this case, the speaker 100 may receive a user voice through a microphone and transmit information on the user voice to a plurality of servers 200-1, 200-2, ..., 200-n. Accordingly, the plurality of servers 200-1, 200-2, ..., 200-n may generate response information for the user's voice and transmit it to the speaker 100. In this case, the speaker 100 transmits response information to the television 50 equipped with a display (not shown), and the television 50 includes a user interface including text included in the response information received from the speaker 100 Can be displayed on the display of the television 50.

Meanwhile, the plurality of servers 200-1, 200-2, ..., 200-n may be servers that provide different secretary services.

For example, in recent years, various companies, such as Samsung Bixby, provide secretarial services. As such, various secretary services exist, and according to an embodiment of the present disclosure, the server may be a server operated to provide secretary services in each enterprise.

Meanwhile, currently provided secretarial services may have different domains (e.g., questions / answers, device control, etc.) each exhibiting excellent performance. Accordingly, if a user can use a secretary service capable of providing a more accurate response, satisfaction and convenience of a user using the secretary service can be improved.

Accordingly, the electronic device 100 according to an embodiment of the present disclosure may obtain response information from a plurality of servers 200-1, 200-2, ..., 200-n and provide it to a user. It will be described in more detail below.

2 is a diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure.

Referring to FIG. 2A, the electronic device 100 may include a communication unit 110, a memory 120, and a processor 130.

The communication unit 110 may communicate with various types of external devices. For example, the communication unit 110 may communicate with an external electronic device or server. In this case, the communication unit 110 may use various types of short-range communication such as Wi-Fi, Bluetooth and NFC, or use various types of remote communication such as wireless mobile communication and Ethernet. To this end, the communication unit 110 may include at least one of a Wi-Fi chip, an Ethernet chip, a Bluetooth chip, a wireless mobile communication chip, and an NFC chip.

Here, the server may include a server capable of providing a secretary service using the learned artificial intelligence model.

In this case, the communication unit 110 may transmit information about the user voice acquired by the electronic device 100 to the server, and receive response information corresponding to the user voice obtained through the artificial intelligence model from the server. Here, the user voice information may be at least one of a voice file, a compressed voice file, pre-processed user voice, and feature point information extracted from the user voice.

The memory 120 may store instructions or data related to at least one other component of the electronic device 100. In particular, the memory 120 may store at least one instruction related to execution of a secretary service.

The processor 130 may be connected to the communication unit 110 and the memory 120 to control the electronic device 100. Specifically, the processor 130 may be electrically connected to the communication unit 110 and the memory 120 to control overall operations and functions of the electronic device 100.

In particular, the processor 130 executes at least one command stored in the memory 120, and when a user input for executing a secretary service is received, the electronic device 100 receives information about the user's voice acquired by the communication unit 110 ) To multiple servers (200-1, 200-2, ..., 200-n) providing different secretary services, and multiple servers (200-1, 200-2, ..., When a plurality of response information is received from 200-n), a response to a user voice may be provided based on at least one of the received plurality of response information.

Here, when a user input for executing a secretary service is received, a user voice including a wakeup word (or a wakeup command) is input, or by selecting a preset button. It may include a case where a user input is received.

In this case, the wake-up word means a trigger word for executing a secretary service or requesting a response from the secretary service, and the user uses the wake-up word corresponding to a voice call command in order to use the secretary service in the electronic device 100. Must ignite. These wake-up words may be determined based on the secretarial service. For example, the wake-up word may include the name of the artificial intelligence that each company uses for secretarial services. Therefore, the wake-up word may be different depending on the type of secretary service.

Also, the wakeup word may be changed according to user input. That is, the user may change the wake-up word by accessing the server through the electronic device 100 or another electronic device (not shown). For example, the user may request to change the wake-up word “hi” Bixby ”to“ hello ”Bixby, and the server may register by changing the wake-up word according to user input. . However, if all or part of the new wake-up word entered by the user for change is the same as all or part of the wake-up word of the secretary service provided by another server, the server rejects the user's change request and the wake-up word You can not change it.

Meanwhile, the user's voice is input through a microphone (not shown) provided in the electronic device 100, or an external electronic device (not shown) connected to the electronic device 100 receives the user's voice, and the electronic device 100 ).

Meanwhile, the preset button may be provided on the electronic device 100 or may be provided on an external electronic device (not shown) connected to the electronic device 100. In this case, when the preset button is selected, the external electronic device (not shown) may transmit a signal indicating that the preset button is selected to the electronic device 100.

Hereinafter, a method in which the processor 130 provides a response to a user voice through a plurality of secretary services will be described in more detail with reference to FIG. 2B.

First, a case in which a user voice including a specific wakeup word is input as a user input for executing a secretary service according to an embodiment of the present disclosure will be described.

When the user input for executing the first secretary service is received, the processor 130 transmits information on the user's voice to the first server providing the first secretary service and the second server providing the second secretary service, The response to the user voice may be provided based on the response information received from the first server among the plurality of response information received from the first and second servers.

That is, when the first wake-up word for executing the first secretary service is identified in the first user voice acquired by the electronic device 100, the processor 130 provides the information about the first user voice to the first secretary service A first server providing a first server and a second server providing a second secretary service, the first of the plurality of response information received from the first server and the second server in response to the transmission of information about the first user voice A response to the first user voice may be provided based on the response information received from the server.

In this case, the processor 130 may use the wakeup engine 131 to check whether a wakeup word exists in the user's voice. Here, the wake-up engine 131 may include an artificial intelligence model trained to recognize various wake-up words, wherein the wake-up word may be additionally designated or changed. Meanwhile, in FIG. 2B, the wake-up engine 131 and the voice recognition module 132 are described as being implemented in hardware, but they may be implemented in software, and in this case, may be stored in the memory 120.

Specifically, the processor 130 stores the input user voice in the memory 120 and performs a voice recognition on the user voice in a certain section through the wake-up engine 131, so that a wake-up word exists in the user voice You can check whether or not.

Here, the wake-up word may be a wake-up word for executing a secretary service of any one of a plurality of secretary services provided by a plurality of servers 200-1, 200-2, ..., 200-n. You can.

Accordingly, when the wake-up word exists in the user's voice, the processor 130 transmits information about the user's voice through the communication unit 110 to a plurality of servers 200-1, 200-2, ..., 200-n ).

In this case, the processor 130 may obtain information about the user voice from the user voice acquired by the electronic device 100.

Specifically, the processor 130 generates a voice file for the user voice or a compressed voice file, or performs pre-processing for the user voice, and generates a voice file or compressed voice file for the pre-processed user voice. can do. In addition, the processor 130 may extract feature points from the user's voice to obtain information about the user's voice.

On the other hand, the processor 130 may transmit information on the remaining user voices except for the wake-up word from the user voices to a plurality of servers 200-1, 200-2, ..., 200-n. For example, the processor 130 may transmit information about the user's voice that exists after the wake-up word to a plurality of servers 200-1, 200-2, ..., 200-n.

To this end, the processor 130 may execute (or launch, activate) the applications 121-1, 121-2, ..., 121-n stored in the memory 120.

Here, the application (121-1, 121-2, ..., 121-n) is a plurality of servers (200-1, 200-2, ..., 200-n) in conjunction with each of the secretary service function As a program for providing, a plurality of servers 200-1, 200-2, ..., 200-n, and various information for providing secretary services can be transmitted and received, and responses to user voices can be provided.

These applications (121-1, 121-2, ..., 121-n) are downloaded through a plurality of servers (200-1, 200-2, ..., 200-n) or separate application stores. It may be installed on the device 100 or the electronic device 100 may access a plurality of servers 200-1, 200-2, ..., 200-n to use the corresponding function as a cloud service method.

In this case, the processor 130 executes all other applications 121-2, ..., 121-n, as well as an application 121-1 for providing a secretary service corresponding to the wake-up word, and the application Information about the user voice may be transmitted to a plurality of servers 200-1, 200-2, ..., 200-n through (121-1, 121-2, ..., 121-n).

That is, even when a wake-up word for executing a specific secretary service is input, the processor 130 not only the server 200-1 providing a specific secretary service, but also other servers 200-2 different from the corresponding server (200-2, ..., 200-n) can also transmit information about the user's voice.

In this case, the plurality of servers 200-1, 200-2, ..., 200-n generate response information for the user voice based on the information about the user voice received from the electronic device 100 and , Response information may be transmitted to the electronic device 100.

Meanwhile, when the response information for the user's voice is received from the plurality of servers 200-1, 200-2, ..., 200-n, the processor 130 is a secretary corresponding to the wake-up word among the plurality of response information A response to the user's voice may be provided based on the response information generated by the service.

That is, the fact that the user uttered a wake-up word indicating a specific secretary service is a request for a response from the corresponding secretary service, and the processor 130 provides a secretary service matching the wake-up word among the plurality of response information received. Based on the response information received from the server, a response to the user's voice can be provided. In this case, the processor 130 may store other response information in the memory 120.

Meanwhile, the processor 130 may check whether the second user voice obtained after the first user voice corresponds to a voice requesting a response result of another secretary service to the first user voice.

In this case, the processor 130 may use the voice recognition module 132 to check whether the user voice corresponds to a voice requesting a response result of another secretary service.

Specifically, the voice recognition module 132 performs voice recognition on the user's voice to convert the user's voice into text, and whether the user's voice intention requests a response result of another secretary service based on the voice recognition result. Can grasp. In this case, the speech recognition module 132 may include an artificial intelligence model trained to confirm whether the user's voice requests a response result of another secretary service through natural language understanding.

Meanwhile, the processor 130 performs recognition of the user's voice only when the applications 121-1, 121-2, ..., 121-n are executed, and the response result of the secretary service whose user voice is different You can also check whether you have the intention to request.

Accordingly, when the second user voice does not correspond to the voice requesting the response result of the second secretary service to the first user voice, the processor 130 transmits information about the second user voice to the first server and the second To the server and provide a response to the second user voice based on the response information received from the first server among the plurality of response information received from the first server and the second server in response to the transmission of the second user voice. You can.

That is, when the user does not request information provided by another secretary service, and utters a voice with a different intention, the processor 130 continues to use the secretary service that previously provided a response, to the subsequent user voice. Response. However, even in this case, since the user may request an answer provided by another secretary service for the user's voice uttered later, the processor 130 also transmits information about the user's voice acquired later to a plurality of servers. Is done.

Meanwhile, when the second user voice corresponds to a voice requesting a response result of the second secretary service to the first user voice, the processor 130 may receive a plurality of received responses in response to transmission of information about the first user voice. The response to the second user voice may be provided based on the response information received from the second server providing the second secretary service among the response information.

That is, when a user utters a voice requesting information provided by another secretary service for the previous user voice, the processor 130 provides a secretary service requested by the user among a plurality of response information received for the previous user voice The response to the user voice may be provided based on the response information received from the server.

On the other hand, in this way, when the secretary service is changed according to the user's request, the processor 130 may provide a response to the user voice thereafter using the changed secretary service. That is, the processor 130 provides a response to the user's voice using the changed secretary service until the user's voice requesting the change to another secretary service is input again, and the user requesting the change to another secretary service When a voice is input, a response to the user voice may be provided using a secretary service requested by the user.

As described above, according to an embodiment of the present disclosure, the user transmits information about the user's voice to a plurality of servers, and provides a response to the user's voice by using the plurality of response information received from the plurality of servers. It can be provided seamlessly (semaless) a plurality of secretary services, in that the user can be provided with the answers of various secretary services, the satisfaction and convenience of the user using the secretary service can be improved.

3 to 10 are diagrams for explaining a method for providing a response to a user voice according to an embodiment of the present disclosure.

First, as shown in FIG. 3, it is assumed that the user has spoken as “XXX, what time is it now?”. Here, XXX may be a wake-up word for the secretary service provided by the first server 200-1.

In this case, as shown in ① in FIG. 3, the processor 130 executes the applications 121-1, 121-2, ..., 121-n, and the applications 121-1, 121-2, ... , 121-n), information about the user's voice "what time is now" except the wake-up word can be transmitted to a plurality of servers 200-1, 200-2, ..., 200-n.

Accordingly, as shown in ② of FIG. 3, a plurality of servers 200-1, 200-2, ..., 200-n generate response information for "what time now" received from the electronic device 100. Then, the generated response information may be transmitted to the electronic device 100.

For example, the first server 200-1 transmits "It is 9:00 in the morning" as the response information to the electronic device 100, and the second server 200-2 asks "Do you want to be informed in Korean time?" The response information may be transmitted to the electronic device 100, and the nth server 200-n may transmit “nine o'clock” to the electronic device 100 as response information.

In this case, the processor 130 is based on the response information received from the first server 200-1, as shown in FIG. 4, since the secretary service (or voice secretary) called by the user is XXX, “Morning 9 It is possible to display a screen including the poetry "410 on the display of the electronic device 100.

Thereafter, as shown in FIG. 5, it is assumed that the user has spoken as “How is the weather today?”.

In this case, the processor 130 requests the response result of another secretary service with "what is the weather today" in that the applications 121-1, 121-2, ..., 121-n are executed. You can check whether it corresponds to the voice. At this time, "how is the weather today" does not correspond to the voice requesting the response result of the other secretary service, as shown in ① in Fig. 5, the user voice "how about the weather today" a plurality of servers (200 -1, 200-2, ..., 200-n).

Accordingly, as shown in ② of FIG. 5, a plurality of servers 200-1, 200-2, ..., 200-n receive response information for "What is the weather today" received from the electronic device 100? It can generate and transmit the generated response information to the electronic device 100.

For example, the first server 200-1 transmits "I will rain once today" as the response information to the electronic device 100, and the second server 200-2 says, "The probability of raining today is 60 % Is sent to the electronic device 100 as response information, and the nth server 200-n transmits "today is clear, and it is expected to rain between 6 and 7 pm" as response information. It can be transmitted to the device 100.

In this case, the processor 130, based on the response information received from the first server 200-1, as shown in FIG. 6 in that the response to the previous user voice through the XXX secretary service, "Today is It will rain once. ”A screen including 610 may be displayed on the display of the electronic device 100.

Thereafter, as shown in FIG. 7A, when the user utters “tell me the YYY result”, the processor 130 is in the state that the applications 121-1, 121-2, ..., 121-n are executed. , It can be confirmed whether "tell me the YYY result" corresponds to the voice requesting the response result of another secretary service. Here, YYY may be a wake-up word for the secretary service provided by the second server 200-2.

At this time, "tell me the YYY result" in that it corresponds to the voice requesting the response result of the secretary service YYY, the processor 130 is previously received from the second server 200-2, as shown in Figure 7B Based on the response information on "How is the weather today," a screen including "710% probability of rain today" is displayed on the display of the electronic device 100.

Thereafter, as shown in FIG. 8, it is assumed that the user utters as “Is it raining on the weekend this week?”.

In this case, the processor 130 requests the response result of the secretarial service different from "it rains even on the weekend this week" in that the applications 111-1, 111-2, ..., 111-n are executed. You can check whether it corresponds to the voice. At this time, in the point that "it rains even on this weekend" does not correspond to the voice requesting the response result of the other secretary service, as shown in ① in Fig. 8, a plurality of information about the user voice "it rains also on this weekend" Servers 200-1, 200-2, ..., 200-n.

Accordingly, as shown in ② of FIG. 8, the plurality of servers 200-1, 200-2, ..., 200-n respond to "Raining rain on the weekend this week" received from the electronic device 100. To generate and transmit the generated response information to the electronic device 100.

In this case, the processor 130 is based on the response information received from the second server 200-2, as shown in FIG. 9, since the secretary service previously called by the user is YYY, "the rain will not rain on the weekend." It is possible to display a screen including the "see" (910) on the display of the electronic device (100).

Meanwhile, in the above-described example, the processor 130 may provide a user interface indicating that a response from another secretary service can be provided.

Specifically, when providing the response of the secretary service called by the user, the processor 130 displays the name of the secretary service other than the secretary service called by the user along with the response of the secretary service called by the user, and other secretary It is possible to guide the user that the response of the service can be provided.

For example, as shown in FIG. 10A, when the user utters "XXX, how is the weather today?", The processor 130 may use the response information provided by the secretary service XXX, “Today it will rain once. 1010 can be displayed on the display. In this case, the processor 130 may display YYY 1020 and ZZZ 1030, which are names of other secretary services, on the display.

Thereafter, as shown in FIG. 10B, when the user utters “tell me the YYY result”, the processor 130 is based on the response information received from the server providing the secretary service YYY, as shown in FIG. 10C, “Today is empty Probability is 60% "(1040) on the display. In this case, the processor 130 may display the names of other secretary services XXX (1050) and ZZZ (1030) on the display.

Meanwhile, a case where a user input for selecting a preset button is input as a user input for executing a secretary service according to an embodiment of the present disclosure.

Specifically, when a user voice is obtained after a user input is received by selecting a preset button, the processor 130 receives information about the user voice from a plurality of servers 200-1, 200-2, ..., 200 -n).

Here, since the preset button is a button for triggering a secretary service, the processor 130, when the preset button is selected, the application (121-1, 121-2, ...) stored in the memory 120 121-n), and then input the information about the user voice through the application (121-1, 121-2, ..., 121-n) a plurality of servers (200-1, 200-2 ,. .., 200-n).

Meanwhile, the preset button may be provided on the electronic device 100 or may be provided on an external electronic device (not shown) connected to the electronic device 100. In this case, when the preset button is pressed, the external electronic device (not shown) may transmit a signal indicating that the preset button was pressed to the electronic device 100.

On the other hand, a plurality of servers (200-1, 200-2, ..., 200-n) generates response information for the user voice received from the electronic device 100, the response information to the electronic device 100 Can transmit.

Meanwhile, when the response information for the user voice is received from the plurality of servers 200-1, 200-2, ..., 200-n, the processor 130 may perform a plurality of user voices based on the plurality of response information. Can provide a response.

In this case, the processor 130 responds to the user's voice based on a plurality of response information provided from a plurality of secretary services, in that the user does not request a response by specifying a secretary service such as a specific wake-up word is spoken. Can provide

In this case, the processor 130 may provide a plurality of responses in different forms based on at least one of a user's preference for a plurality of secretary services provided by a plurality of servers and accuracy of the plurality of response information.

Specifically, the processor 130 displays the response of the secretary service with the highest user preference to the largest, or a color different from other secretary services, based on the user's preference for the plurality of secretary services provided by the plurality of servers. It can be marked with or by highlighting. Alternatively, the processor 130 may display the response of the secretary service with the highest user preference, and only the name of the secretary service for other secretary services. Alternatively, the processor 130 may largely display the response of the secretary service in the order of high user preference. Alternatively, when displaying responses of a plurality of secretary services, the processor 130 may display them in order of user preference.

To this end, the processor 130 may store information about the secretary service used by the user in the memory 120 and update the information whenever the user uses the secretary service.

Accordingly, the processor 130 may determine that the user has a high preference in the order of secretary services having a high frequency of use.

On the other hand, the processor 130, based on the accuracy of a plurality of secretary services provided by a plurality of servers, the response of the secretary service that does not provide an accurate response is displayed smaller than the responses of other secretary services, or only the name of the secretary service It may be displayed, or only the name of the secretary service and a portion of the response.

For example, the plurality of servers 200-1, 200-2, ..., 200-n, for example, cannot analyze the meaning of the user's voice, and thus cannot configure the response information. In this case, the plurality of servers 200-1, 200-2, ..., 200-n may transmit response information such as "This service is not supported" to the electronic device 100.

In this case, the processor 130 may determine that the secretary service provided by the corresponding server does not provide an accurate response to the user's voice, and may display the response of the secretary service differently from the response of the other secretary service.

Meanwhile, after providing a plurality of responses, the processor 130 may check whether a wakeup word exists in the user voice using the wakeup engine 131 when the user voice is input.

Accordingly, when the wakeup word does not exist, the processor 130 transmits information about the user's voice to a plurality of servers 200-1, 200-2, ..., 200-n, and a plurality of servers A plurality of responses may be provided based on a plurality of response information received from (200-1, 200-2, ..., 200-n). In this case, the processor 130 may differentially display a plurality of responses, as described above.

However, when the wake-up word exists, the processor 130 may transmit information on the user's voice excluding the wake-up word to a plurality of servers 200-1, 200-2, ..., 200-n. have.

Then, when the processor 130 receives the response information for the user's voice from the plurality of servers 200-1, 200-2, ..., 200-n, the processor 130 corresponds to a wake-up word among the plurality of response information A response to the user's voice may be provided based on the response information generated by the service.

That is, after the user utters a wake-up word indicating a specific secretary service, rather than providing responses from a plurality of secretary services, the user provides a secretary service that requests a response, that is, a server providing a secretary service matching the wake-up word Based on the response information received from, a response to the user voice may be provided. In this case, the processor 130 may store other response information in the memory 120 and then, when requesting a response result of another secretary service, use the response information stored in the memory 120 to provide a response. have.

On the other hand, the detailed operation of the processor 130 when the wake-up word exists in the voice spoken by the user is the same as described above, and thus, a detailed overlapping description will be omitted.

As described above, according to an embodiment of the present disclosure, when a user requests execution of a secretary service without specifying a specific secretary service, responses of a plurality of secretary services to a user voice are provided, and then selected by the user In that the response of the secretary service can be provided, satisfaction and convenience of a user using the secretary service can be improved.

11 to 17 are diagrams for describing a method of providing a response to a user voice according to an embodiment of the present disclosure.

First, as shown in FIG. 11, it is assumed that a user presses a button provided on the remote controller 1000 and then utters a message as “What time is it now?” In this case, the remote controller 1000 may transmit a signal indicating that the button is pressed to the electronic device 100, and the processor 130 determines that a user input for executing a secretary service is input based on the received signal. can do.

Accordingly, as shown in ① of FIG. 11, the processor 130 executes the applications 121-1, 121-2, ..., 121-n, and information about the acquired user voice "what time is now" It can be transmitted to a plurality of servers (200-1, 200-2, ..., 200-n) through the application (121-1, 121-2, ..., 121-n).

Accordingly, as shown in ② of FIG. 11, a plurality of servers 200-1, 200-2, ..., 200-n generate response information for "what time now" received from the electronic device 100. Then, the generated response information may be transmitted to the electronic device 100.

In this case, the processor 130 electronically transmits a plurality of responses to " what time now " based on a plurality of response information received from a plurality of servers 200-1, 200-2, ..., 200-n. It can be displayed on the display of the device 100.

For example, as illustrated in FIG. 12A, the processor 130 may inform the user of “Korean morning time” based on the response information of YYY and ZZZ based on the response information of XXX, which has high user preference, based on the response information of YYY and ZZZ. It can be displayed in a size larger than (1220) and "It is 9 o'clock" (1230). In addition, the processor 130 is provided as "No service" (1240, 1250) in response to aaa and bbb that do not configure the response information, but may be displayed smaller than the responses of other secretary services.

Thereafter, as shown in FIG. 13, it is assumed that the user has spoken as “How is the weather today?”.

In this case, the processor 130 may use the wake-up module 131 to check whether a wake-up word exists in “How is the weather today?”.

At this time, in the absence of a wake-up word in "How is today's weather?", The processor 130, as shown in ① in Fig. 13, the user voice, "What is the weather today?" 200-1, 200-2, ..., 200-n), and a plurality of servers 200-1, 200-2, ..., 200-n) are electronic as shown in ② in FIG. The response information for “what is the weather today” received from the device 100 may be generated, and the generated response information may be transmitted to the electronic device 100.

In this case, the processor 130 electronically responds to "How is the weather today?" Based on the plurality of response information received from the plurality of servers 200-1, 200-2, ..., 200-n. It can be displayed on the display of the device 100.

At this time, the processor 130 is a plurality of received from a plurality of servers (200-1, 200-2, ..., 200-n) in that the "how is the weather today?" Multiple responses to "How is the weather today?" May be displayed in the response information of.

For example, as shown in FIG. 14, the processor 130 sets the “today will rain once” 1410 based on the response information of XXX having a high user preference, and the “today today” based on the response information of YYY, ZZZ, and bbb. Probability is 60% "(1420)," Today is clear, but it is expected to rain between 6 and 7 pm "(1430) and" Today is expected to rain "(1440) can do. In addition, the processor 130 provides a "No service" 1450 in response to aaa, which has failed to configure response information, but may be displayed smaller than the response of other secretary services.

Then, as shown in FIG. 15, it is assumed that the user utters "ZZZ, rain on weekends?". Here, ZZZ may be a wake-up word for a specific secretary service.

In this case, the processor 130 may use the wake-up module 131 to check whether a wake-up word exists in "ZZZ, rain on weekends?".

At this time, in the "ZZZ, does it rain on the weekend?", The wake-up word ZZZ exists, so that the processor 130, as shown in ① in Fig. 15, the user voice, except for "ZZZ," Information can be transmitted to a plurality of servers (200-1, 200-2, ..., 200-n), and a plurality of servers (200-1, 200-2, ..., 200-n) As in (2) of 15, response information for “it rains even on weekends” received from the electronic device 100 may be generated, and the generated response information may be transmitted to the electronic device 100.

In this case, as shown in FIG. 16, the processor 130 receives response information received from a server providing a secretary service ZZZ uttered by a user among a plurality of servers 200-1, 200-2, ..., 200-n. Based on the, “weekends appear to be sunny” 1610 may be displayed on the display of the electronic device 100.

In this way, the processor 130 may display only the response of the secretary service requested by the user when the user requests a response of a specific secretary service even when the user selects a preset button to execute the secretary service. In addition, the processor 130 may provide a continuous conversation with the corresponding secretary service or provide a response from another secretary service according to whether a wakeup word is included in the user's voice.

Meanwhile, in the above-described example, the response of the secretary service having a high user preference is displayed as larger than the response of the other secretary service according to the user preference, but this is only an example. That is, the processor 130 may display the response of the secretary service having the highest user preference, and may display only the name of the secretary service for the other secretary services.

For example, when the user speaks, such as "What time is it now?", The processor 130, based on the response information of the secretarial service XXX with high user preference, as shown in FIG. 17A, is "9:00 am" (1710). In this case, the processor 130 may display the names of other secretary services, YYY (1720), ZZZ (1730), aaa (1740), and bbb (1750).

Thereafter, as shown in FIG. 17B, when the user utters “tell me the ZZZ result”, the processor 130 is based on the response information received from the server providing the secretary service ZZZ, as in FIG. "(1760) may be displayed on the display. In this case, the processor 130 may display the names of other secretary services, XXX (1770), YYY (1720), aaa (1740), and bbb (1750). You can.

Meanwhile, in the above-described example, it has been described that the response to the user's voice is provided through the display of the electronic device 100, but this is only an example. That is, the electronic device 100 may output a response to the user's voice as a voice through a speaker provided in the electronic device 100. In this case, the electronic device 100 may provide a response in at least one form of a screen or voice.

In addition, the response to the user's voice may be provided by an electronic device (not shown) other than the electronic device 100. That is, the user voice is acquired by the electronic device 100, but a response to the user voice may be provided by another electronic device (not shown) connected to the electronic device 100 rather than the electronic device 100.

For example, assume that the electronic device 100 is implemented as a speaker with a microphone, as shown in FIGS. 18A to 18D.

First, as shown in FIG. 18A, it is assumed that the user has spoken as “XXX, what time is it now?”.

In this case, when the user voice is received through the microphone, the processor 130 may transmit information about the user voice to a plurality of servers 200-1, 200-2, ..., 200-n. Accordingly, the plurality of servers 200-1, 200-2, ..., 200-n may generate response information for the user's voice and transmit it to the electronic device 100.

In this case, the processor 130 among the response information received from a plurality of servers (200-1, 200-2, ..., 200-n), response information received from a server providing a secretary service XXX called by a user Can be transmitted to the television 50.

Accordingly, as shown in FIG. 18B, the television 50 may display “10 am in the morning” 1810 on the display based on the response information received from the electronic device 100.

Thereafter, as shown in FIG. 18C, it is assumed that the user utters as “tell me the YYY result”.

In this case, when the user voice is received through the microphone, the processor 130 may determine whether the user voice corresponds to a voice requesting a response result of another secretary service. At this time, "tell me also the YYY results" in that it corresponds to the voice requesting the response result of the secretary service YYY, the processor 130 among the previously received response information is the response information received from the server providing the secretary service YYY Can be transmitted to the television 50.

Accordingly, as illustrated in FIG. 18D, the television 50 may display “Would you like to be informed in Korean time?” 1820 on the display based on the response information received from the electronic device 100.

Meanwhile, in the above-described example, it has been described that the operations for speech recognition are performed on a plurality of servers 200-1, 200-2, ..., 200-n.

However, this is only an example, and at least some of the operations for speech recognition may be performed in another electronic device (not shown).

Specifically, the processor 130 may preprocess the user's voice and convert the user's voice into text. Then, the processor 130 may transmit information on the text in which the user's voice is converted to a plurality of servers 200-1, 200-2, ..., 200-n. In this case, the plurality of servers 200-1, 200-2, ..., 200-n grasp the intention and entities of the user's voice from the text, and respond to the user's voice based on the natural language understanding result. It can be transmitted to the electronic device 100.

As another example, the processor 130 performs pre-processing of the user's voice, converts the user's voice into text, understands the intent and entities for the user's voice in the text, and provides information about the natural language understanding results from a plurality of servers ( 200-1, 200-2, ..., 200-n). In this case, the plurality of servers 200-1, 200-2, ..., 200-n acquire information on the response to the user's voice based on the information received from the electronic device 100, and the obtained As a response information to the user's voice based on the information, a natural language can be obtained. Then, the plurality of servers 200-1, 200-2, ..., 200-n may transmit response information to the electronic device 100.

19 is a block diagram illustrating a configuration of an electronic device according to various embodiments of the present disclosure.

Referring to FIG. 19, the electronic device 100 includes a communication unit 110, a memory 120, a processor 130, a display 140, a microphone 150, an audio output unit 160, and a user input unit 170. It can contain. These components can be controlled by processor 130.

On the other hand, since the communication unit 110, the memory 120, and the processor 130 perform the same functions as the communication unit 110, the memory 120, and the processor 130 shown in FIG. 2, overlapping of these components The description is omitted.

The communication unit 110 may communicate with various types of external devices according to various types of communication methods. The communication unit 110 may include at least one of a Wi-Fi chip 111, a Bluetooth chip 112, a wireless communication chip 113, an NFC chip 114, and an Ethernet chip 115.

In this case, the processor 130 may communicate with an external server or various external devices using the communication unit 110.

For example, the processor 130 may transmit the user voice acquired by the electronic device 100 to the server through the communication unit 110, and receive response information corresponding to the user voice obtained through the artificial intelligence model from the server can do. Also, the processor 130 may receive a user voice acquired by an external electronic device (not shown) through the communication unit 110. Further, the processor 130 may receive a signal indicating that a preset button is pressed in an external electronic device (not shown) through the communication unit 110.

The memory 120 may store instructions or data related to at least one other component of the electronic device 100. The memory 120 may be implemented as a non-volatile memory, volatile memory, flash-memory, hard disk drive (HDD), or solid state drive (SSD). The memory 120 is accessed by the processor 130, and data read / write / modify / delete / update may be performed by the processor 130. In this document, the term memory is a memory card (not shown) mounted on a memory 120, a ROM (not shown), a RAM (not shown), or an electronic device 100 in the processor 130 (eg, micro SD) Card, memory stick).

Further, programs and data for configuring various screens to be displayed on the display area of the display 140 may be stored in the memory 120. In addition, the memory 120 may store various AI models of this document, for example, a model for confirming whether a user voice corresponds to a voice requesting a response result of another secretary service. However, this is only an example, and the corresponding model may be stored in an external electronic device (not shown). In this case, the processor 130 transmits the user's voice to the external electronic device (not shown) through the communication unit 110 and receives the result recognized by the external electronic device (not shown) from the external electronic device (not shown). It might be.

The display 140 may display various screens. Specifically, the processor 130 may display a response to the user voice on the display 140 based on the response information received from the plurality of servers 200-1, 200-2, ..., 200-n. You can. In this case, the screen displayed on the display 140 may include text indicating an answer to the user's voice.

Meanwhile, at least a portion of the display 140 may be coupled to at least one of a front region, a side region, and a rear region of the electronic device 100 in the form of a flexible display. The flexible display may be characterized by being able to bend, bend, or roll without damage through a thin and flexible substrate like paper.

In addition, the display 140 may be implemented as a touch screen having a layer structure in combination with the touch panel 171. The touch screen may have a function of detecting not only a display function but also a touch input position, a touched area as well as a touch input pressure, and also a function of detecting a proximity touch as well as a real-touch. You can.

The microphone 150 may acquire a user voice. The microphone 150 may receive a user voice and generate it as a voice signal. In this case, the processor 130 may provide a response to the user's voice using the voice signal.

The audio output unit 160 is implemented as a speaker and can output various notification sounds or voice messages. In this case, the processor 130 may convert a response to the user's voice into voice, and output it through the audio output unit 160.

The user input unit 170 may receive various user inputs and transmit them to the processor 130. The user input unit 170 may include, for example, a touch panel 171 or a key 172. The touch panel 171 may use, for example, at least one of capacitive, pressure-sensitive, infrared, and ultrasonic methods. Also, the touch panel 171 may further include a control circuit. The touch panel 171 may further include a tactile layer to provide a tactile reaction to the user. The key 172 may include, for example, a physical button, an optical key, or a keypad.

In particular, the user input unit 170 may transmit a user input to select a specific button provided in the user input unit 170 to the processor 130, and when the user input is received, the processor 130 uses the acquired user voice To provide secretarial services.

The processor 130 (or the control unit) may control the overall operation of the electronic device 100 using various programs stored in the memory 120.

The processor 130 may be composed of a RAM 133, a ROM 134, a graphics processor 136, a main CPU 135, first to n interfaces 137-1 to 137-n, and a bus 138. have. At this time, the RAM 133, the ROM 134, the graphic processing unit 136, the main CPU 135, and the first to n interfaces 137-1 to 137-n may be connected to each other through the bus 138. .

Meanwhile, the electronic device 100 according to an embodiment of the present disclosure may provide various guides related to a secretary service, and will be described in more detail with reference to FIGS. 20 and 21.

According to an embodiment of the present disclosure, even when a user utters a specific wake-up word, the electronic device 100 may transmit to another server in addition to a server providing a secretary service corresponding to the wake-up word.

Accordingly, service providers providing secretarial services to a plurality of servers 200-1, 200-2, ..., 200-n may include a plurality of servers 200-1, 200-2, ..., 200- By using the user's voice received by n), it is possible to check which voice the user spoke a lot when using the secretary service.

Accordingly, service providers can learn the secretarial service to provide the function or answer when their secretarial service does not provide the function or answer requested by the user. In addition, when learning is completed, the service providers may transmit a guide to the electronic device 100 that a corresponding function or answer may be provided through the server.

In this case, the electronic device 100 may display information received from the server. For example, as shown in FIG. 20, a guide such as "XXX has learned the lighting control function. In the future," You can control the lighting with "Turn off XXX lighting" "(2010) can be displayed.

In addition, service providers may transmit a guide to the electronic device 100 that a function or answer requested by the user can be provided in their secretary service.

In this case, the electronic device 100 may display information received from the server. For example, as shown in FIG. 21, a guide such as “XXX can tell the weather. In the future,“ You can know the weather with “tell me about the weather tomorrow” ”(2110) can be displayed.

22 is a flowchart illustrating a method of providing a secretarial service of an electronic device according to an embodiment of the present disclosure.

First, when a user input for executing a secretary service is received, information on the user voice acquired by the electronic device is transmitted to a plurality of servers providing different secretary services (S2210).

Then, when a plurality of response information is received from a plurality of servers, a response to a user voice is provided based on at least one of the received response information (S2220).

Here, the plurality of servers may provide a secretary service using an artificial intelligence agent.

Meanwhile, in step S2210, when a user input for executing the first secretary service is received, information about the user's voice is transmitted to the first server providing the first secretary service and the second server providing the second secretary service, In step S2220, a response to a user voice may be provided based on the response information received from the first server among the plurality of response information received from the first and second servers.

In addition, in step S2210, when the first wakeup word for executing the first secretary service is confirmed in the first user voice acquired by the electronic device, information on the first user voice is provided to the first secretary service. The first server and the second server providing the second secretary service, and step S2220 is selected from among a plurality of response information received from the first server and the second server in response to the transmission of information about the first user voice. A response to the first user's voice may be provided based on the response information received from the 1 server.

Here, it can be confirmed whether the second user voice obtained after the first user voice corresponds to a voice requesting a response result of another secretary service to the first user voice.

In addition, when the second user voice corresponds to a voice requesting a response result of the second secretary service to the first user voice, the second of the plurality of response information received in response to the transmission of the information about the first user voice A response to the second user voice may be provided based on the response information received from the server.

In addition, when the second user voice does not correspond to the voice requesting the response result of the other secretary service, information on the second user voice is transmitted to the first server and the second server, and the information on the second user voice The response to the second user voice may be provided based on the response information received from the first server among the plurality of response information received from the first server and the second server in response to the transmission.

On the other hand, in step S2210, when a user voice is acquired after a user input is received by selecting a preset button, information on the user voice is transmitted to a plurality of servers, and step S1920 receives a plurality of response information from a plurality of servers. If so, a plurality of responses to the user's voice may be provided based on the plurality of response information.

In this case, step S2220 may output a plurality of responses in different forms based on at least one of a user's preference for a plurality of secretary services provided by a plurality of servers and accuracy of the plurality of response information.

As used herein, the term "module" includes units composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic blocks, components, or circuits. The module may be an integrally configured component or a minimum unit that performs one or more functions or a part thereof. For example, the module can be configured with an application-specific integrated circuit (ASIC).

Various embodiments of the present document may be implemented with software including instructions stored in a machine (eg, computer) readable storage media. A device that can be called and operated according to the called command, and may include an electronic device (eg, electronic device A) according to the disclosed embodiments. When the command is executed by a processor, the processor directly Or, under the control of the processor, other components may be used to perform the function corresponding to the instruction, which may include code generated or executed by a compiler or interpreter. , It may be provided in the form of a non-transitory storage medium, where 'non-transitory' means that the storage medium does not contain a signal. It means that it is tangible and does not distinguish between data being stored semi-permanently or temporarily on a storage medium.

According to a date and time example, a method according to various embodiments disclosed in this document may be provided as being included in a computer program product. Computer program products are commodities that can be traded between sellers and buyers. The computer program product may be distributed online in the form of a storage medium readable by the device (eg compact disc read only memory (CD-ROM)) or through an application store (eg Play StoreTM). In the case of online distribution, at least a portion of the computer program product may be temporarily stored at least temporarily on a storage medium such as a memory of a manufacturer's server, an application store's server, or a relay server, or may be temporarily generated.

Each component (eg, module or program) according to various embodiments may be composed of a singular or a plurality of entities, and some of the aforementioned sub-components may be omitted, or other sub-components may be omitted. It may be further included in various embodiments. Alternatively or additionally, some components (eg, modules or programs) may be integrated into one entity, performing the same or similar functions performed by each corresponding component before being integrated. According to various embodiments, operations performed by a module, program, or other component may be sequentially, parallel, iteratively or heuristically executed, at least some operations may be executed in a different order, omitted, or other operations may be added. Can be.

100: electronic device 110: communication unit
120: memory 130: processor

Claims (16)

In the electronic device,
Communication department;
Memory; And
And a processor connected to the communication unit and the memory to control the electronic device.
The processor, by executing at least one instruction stored in the memory,
When a user input for executing a secretary service is received, information on a user voice obtained by the electronic device is transmitted to a plurality of servers providing different secretary services through the communication unit,
When a plurality of response information is received from the plurality of servers, a response to the user voice is provided based on at least one of the received plurality of response information,
The plurality of servers, the electronic device to provide a secretary service using an artificial intelligence agent (Artificial intelligence agent). According to claim 1,
The processor,
When a user input for executing a first secretary service is received, information on the user voice is transmitted to a first server providing a first secretary service and a second server providing a second secretary service, and the first and An electronic device that provides a response to the user voice based on response information received from the first server among a plurality of response information received from a second server. According to claim 1,
The processor,
When the first wakeup word for executing the first secretary service is identified in the first user voice acquired by the electronic device, the information for the first user voice is provided to the first secretary service Among the plurality of response information transmitted from the first server and the second server in response to transmission of information on the first user voice, transmitted to the first server and the second server providing the second secretary service And providing a response to the first user voice based on the response information received from the first server. According to claim 3,
The processor,
The electronic device confirms whether the second user voice obtained after the first user voice corresponds to a voice requesting a response result of another secretary service to the first user voice. According to claim 4,
The processor,
If the second user voice corresponds to a voice requesting a response result of the second secretary service to the first user voice, among the plurality of response information received in response to the transmission of the information about the first user voice And provide a response to the second user voice based on the response information received from the second server. According to claim 4,
The processor,
If the second user voice does not correspond to the voice requesting the response result of the other secretary service, information on the second user voice is transmitted to the first server and the second server, and the second user voice is transmitted to the second user voice. An electronic device that provides a response to the second user voice based on the response information received from the first server among the plurality of response information received from the first server and the second server in response to the transmission of the information about the . According to claim 1,
The processor,
When the user voice is obtained after the user input is received by selecting a preset button, information about the user voice is transmitted to the plurality of servers, and when a plurality of response information is received from the plurality of servers, the An electronic device that provides a plurality of responses to the user voice based on a plurality of response information. The method of claim 7,
The processor,
The electronic device outputs the plurality of responses in different forms based on at least one of a user's preference for a plurality of secretary services provided by the plurality of servers and the accuracy of the plurality of response information. In the method for providing secretarial services of electronic devices,
When a user input for executing a secretary service is received, transmitting information on user voices obtained by the electronic device to a plurality of servers providing different secretary services; And
Includes; when receiving a plurality of response information from the plurality of servers, providing a response to the user voice based on at least one of the received plurality of response information; includes,
The plurality of servers, using an artificial intelligence agent (Artificial intelligence agent) to provide a secretary service, secretary service providing method. The method of claim 9,
The transmitting step,
When a user input for executing a first secretary service is received, the information on the user voice is transmitted to a first server providing a first secretary service and a second server providing a second secretary service,
The providing step,
A method of providing a secretary service, wherein a response to the user's voice is provided based on response information received from the first server among a plurality of response information received from the first and second servers. The method of claim 9,
The transmitting step,
When the first wakeup word for executing the first secretary service is identified in the first user voice acquired by the electronic device, the information for the first user voice is provided to the first secretary service Transmitting to the first server and the second server providing the second secretary service,
The providing step,
Response to the first user voice based on the response information received from the first server among a plurality of response information received from the first server and the second server in response to the transmission of the information about the first user voice Provided, secretary service provision method. The method of claim 11,
And confirming whether the second user voice obtained after the first user voice corresponds to a voice requesting a response result of another secretary service to the first user voice. The method of claim 12,
If the second user voice corresponds to a voice requesting a response result of the second secretary service to the first user voice, among the plurality of response information received in response to the transmission of the information about the first user voice And providing a response to the second user voice based on the response information received from the second server. The method of claim 12,
If the second user voice does not correspond to the voice requesting the response result of the other secretary service, information on the second user voice is transmitted to the first server and the second server, and the second user voice is transmitted to the second user voice. Providing a response to the second user voice based on the response information received from the first server among a plurality of response information received from the first server and the second server in response to the transmission of the information on the; A method of providing a secretarial service, further comprising. The method of claim 9,
The transmitting step,
When the user voice is obtained after the user input is received by selecting a preset button, information on the user voice is transmitted to the plurality of servers,
The providing step,
When a plurality of response information is received from the plurality of servers, a method for providing a secretary service is provided based on the plurality of response information. The method of claim 15,
The providing step,
And outputting the plurality of responses in different forms based on at least one of a user's preference for a plurality of secretary services provided by the plurality of servers and the accuracy of the plurality of response information.

Images