KR102734876B1 - User interfaces for customizing graphic objects - Google Patents

Abstract

In some embodiments, the electronic device displays a virtual keyboard, wherein the virtual keyboard includes a plurality of selectable representations of a plurality of graphical objects. In some embodiments, the electronic device receives user input selecting a first selectable representation of a first graphical object, wherein the first graphical object is associated with a first customizable visual element and a second customizable visual element. In some embodiments, in response to receiving the user input and upon a determination that one or more first criteria are satisfied, the electronic device displays, via the display device, a customization user interface including a first section for customizing the first customizable visual element, a second section for customizing the second customizable visual element, and a third section for displaying a first individual representation of the first graphical object indicating a current customization state of the first graphical object.

Description

User interfaces for customizing graphic objects

Cross-reference to related applications

This application claims the benefit of U.S. Provisional Application No. 62/907,437, filed September 27, 2019, the contents of which are hereby incorporated by reference in their entirety for all purposes.

Technical field

The present disclosure generally relates to user interfaces that enable a user to customize and insert graphical objects, such as emojis, on an electronic device.

User interaction with electronic devices has increased significantly in recent years. These devices may be computers, smart phones, tablet computers, televisions, multimedia devices, mobile devices, wearable devices, etc.

In some situations, users input graphical objects, such as emojis, using a virtual keyboard on the electronic device. In some situations, users want to customize different aspects of certain graphical objects or emojis. Enhancing these interactions can improve the user experience with the device and reduce user interaction time, which is especially important when the input devices are battery-operated.

It is well understood that the use of personally identifiable information should follow privacy policies and practices generally recognized as meeting or exceeding industry or governmental requirements for maintaining the privacy of users. In particular, personally identifiable information data should be managed and processed to minimize the risks of unintended or unauthorized access or use, and the nature of authorized use should be clearly indicated to users.

Some embodiments described in the present disclosure relate to presenting customized user interfaces for customizing and inserting graphical objects such as emojis. It is to be understood that a sufficient description of the embodiments is provided in the drawings and the detailed description for carrying out the invention, and that the disclosure provided above does not limit the scope of the present disclosure in any way.

For a better understanding of the various described embodiments, reference should be made to the following detailed description taken in conjunction with the drawings in which like reference numerals represent corresponding parts throughout the drawings.
FIG. 1A is a block diagram illustrating a portable multifunction device having a touch-sensitive display, according to some embodiments.
FIG. 1b is a block diagram illustrating exemplary components for event processing according to some embodiments.
FIG. 2 illustrates a portable multifunction device having a touch screen, according to some embodiments.
FIG. 3 is a block diagram of an exemplary multifunction device having a display and a touch-sensitive surface, according to some embodiments.
FIG. 4A illustrates an exemplary user interface for a menu of applications on a portable multifunction device according to some embodiments.
FIG. 4b illustrates an exemplary user interface for a multifunction device having a touch-sensitive surface separate from a display, according to some embodiments.
FIG. 5A illustrates a personal electronic device according to some embodiments.
FIG. 5b is a block diagram illustrating a personal electronic device according to some embodiments.
FIGS. 5C and 5D illustrate exemplary components of a personal electronic device having a touch-sensitive display and intensity sensors, according to some embodiments.
FIGS. 5E-5H illustrate exemplary components and user interfaces of a personal electronic device according to some embodiments.
FIGS. 6A through 6XX illustrate exemplary ways in which an electronic device presents a customized user interface for customizing and inserting emojis, according to some embodiments of the present disclosure.
FIGS. 7A through 7M are flowcharts illustrating a method of presenting a customized user interface for customizing and inserting emojis, according to some embodiments of the present disclosure.

The following description sets forth exemplary methods, parameters, etc. However, it should be recognized that such description is not intended to be a limitation on the scope of the present invention, but instead is provided as a description of exemplary embodiments.

There is a need for electronic devices that provide efficient methods for customizing graphical objects, such as emojis, having multiple customizable components. Such techniques may reduce the cognitive burden on users of such devices. Additionally, such techniques may reduce processor and battery power otherwise wasted on excessive user inputs.

Although the following description uses terms such as "first," "second," etc. to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of the various embodiments described, a first touch could be referred to as a second touch, and similarly, a second touch could be referred to as a first touch. Although the first touch and the second touch are both touches, they are not the same touch.

The terminology used in the description of the various described embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used in the description of the various described embodiments and in the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and/or," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "include," "including," "comprise," and/or "comprising," when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

The term "if" is optionally interpreted to mean "when" or "upon" or "in response to determining" or "in response to detecting", as the context requires. Similarly, the phrase "if it is determined that" or "if [the stated condition or event] is detected" is optionally interpreted to mean "when determining that" or "in response to determining that" or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]", as the context requires.

Embodiments of electronic devices, user interfaces for such devices, and associated processes for using such devices are described. In some embodiments, the device is a portable communications device, such as a mobile telephone, that also includes other functionality, such as PDA and/or music player functionality. Exemplary embodiments of portable multifunction devices include, without limitation, the iPhone®, iPod Touch®, and iPad® devices from Apple Inc. of Cupertino, Calif. Other portable electronic devices, such as laptop or tablet computers having touch-sensitive surfaces (e.g., touch screen displays and/or touchpads), are optionally used. It should also be understood that in some embodiments, the device is a desktop computer having a touch-sensitive surface (e.g., a touch screen display and/or touchpad) rather than a portable communications device.

In the discussion below, an electronic device including a display and a touch-sensitive surface is described. However, it should be understood that the electronic device optionally includes one or more other physical user interface devices, such as a physical keyboard, mouse, and/or joystick.

The device typically supports a variety of applications, such as one or more of the following: a drawing application, a presentation application, a word processing application, a website creation application, a disk authoring application, a spreadsheet application, a gaming application, a telephony application, a video conferencing application, an email application, an instant messaging application, a fitness assistance application, a photo management application, a digital camera application, a digital video camera application, a web browsing application, a digital music player application, and/or a digital video player application.

Various applications running on the device optionally use at least one common physical user interface device, such as a touch-sensitive surface. One or more functions of the touch-sensitive surface, as well as corresponding information displayed on the device, are optionally adjusted and/or varied from one application to the next and/or within individual applications. In this way, the common physical architecture of the device (such as the touch-sensitive surface) optionally supports various applications using user interfaces that are intuitive and transparent to the user.

Now, we turn to embodiments of portable devices having touch-sensitive displays. FIG. 1A is a block diagram illustrating a portable multifunction device (100) having a touch-sensitive display system (112), according to some embodiments. The touch-sensitive display (112) is sometimes conveniently referred to as a "touch screen" and sometimes known or referred to as a "touch-sensitive display system." The device (100) includes memory (102) (optionally including one or more computer-readable storage media), a memory controller (122), one or more processing units (CPUs) (120), a peripheral interface (118), RF circuitry (108), audio circuitry (110), a speaker (111), a microphone (113), an input/output (I/O) subsystem (106), other input control devices (116), and an external port (124). The device (100) optionally includes one or more optical sensors (164). The device (100) optionally includes one or more contact intensity sensors (165) for detecting the intensity of contacts on the device (100) (e.g., a touch-sensitive surface such as the touch-sensitive display system (112) of the device (100). The device (100) optionally includes one or more tactile output generators (167) for generating tactile outputs on the device (100) (e.g., for generating tactile outputs on a touch-sensitive surface such as the touch-sensitive display system (112) of the device (100) or the touchpad (355) of the device (300). These components optionally communicate via one or more communication buses or signal lines (103).

As used in the specification and claims, the term "intensity" of a contact on a touch-sensitive surface refers to the force or pressure (force per unit area) of a contact (e.g., a finger contact) on the touch-sensitive surface, or a proxy for the force or pressure of a contact on the touch-sensitive surface. The intensity of a contact has a range of values, including at least four distinct values, and more typically hundreds (e.g., at least 256) distinct values. The intensity of a contact is optionally determined (or measured) using various approaches, and various sensors or combinations of sensors. For example, one or more force sensors under or adjacent the touch-sensitive surface are optionally used to measure force at various points on the touch-sensitive surface. In some implementations, force measurements from multiple force sensors are combined (e.g., weighted averaged) to determine an estimated force of the contact. Similarly, the pressure-sensitive tip of the stylus is optionally used to determine the pressure of the stylus on the touch-sensitive surface. Alternatively, the size of the contact area detected on the touch-sensitive surface and/or changes therein, the capacitance of the touch-sensitive surface near the contact and/or changes therein, and/or the resistance of the touch-sensitive surface near the contact and/or changes therein are optionally used as surrogates for the force or pressure of the contact on the touch-sensitive surface. In some implementations, the surrogate measurements of the contact force or pressure are used directly to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is described in units corresponding to the surrogate measurements). In some implementations, the surrogate measurements of the contact force or pressure are converted to an estimated force or pressure, and the estimated force or pressure is used to determine whether an intensity threshold has been exceeded (e.g., the intensity threshold is a pressure threshold measured in units of pressure). Using intensity of contact as a property of user input allows users access to additional device functionality that might not otherwise be accessible to users on reduced-size devices with limited real-world area for displaying affordances (e.g., on a touch-sensitive display) and/or receiving user input (e.g., via a touch-sensitive display, a touch-sensitive surface, or a physical/mechanical control such as a knob or button).

As used in the specification and claims, the term "tactile output" refers to a physical displacement of the device relative to its previous position, a physical displacement of a component (e.g., a touch-sensitive surface) of the device relative to another component (e.g., a housing) of the device, or a displacement of a component relative to the center of mass of the device that is detectable by a user using the sense of touch. For example, in situations where the device or a component of the device contacts a touch-sensitive surface of the user (e.g., a finger, palm, or other part of the user's hand), the tactile output generated by the physical displacement would be interpreted by the user as a tactile sensation corresponding to a perceived change in the physical characteristics of the device or component of the device. For example, movement of the touch-sensitive surface (e.g., a touch-sensitive display or track pad) would optionally be interpreted by the user as a "down click" or "up click" of a physical actuator button. In some cases, a user will feel a tactile sensation such as a "down click" or an "up click" even when there is no movement of a physical actuator button associated with the touch-sensitive surface that is physically pressed (e.g., displaced) by the user's movements. As another example, the movement of the touch-sensitive surface is optionally interpreted or sensed by the user as a "roughness" of the touch-sensitive surface, even when there is no change in the flatness of the touch-sensitive surface. Although such interpretations of touch by the user will likely be influenced by the user's individualized sensory perception, there are many touch sensory perceptions that are common to most users. Accordingly, when a tactile output is described as corresponding to a particular sensory perception of a user (e.g., an "up click," a "down click," a "roughness"), unless otherwise noted, the generated tactile output corresponds to a physical displacement of the device or a component thereof that would produce the described sensory perception for a typical (or average) user.

It should be understood that the device (100) is only one example of a portable multi-function device, and that the device (100) may optionally have more or fewer components than those illustrated, optionally combine two or more components, or optionally have different configurations or arrangements of the components. The various components illustrated in FIG. 1A are implemented in hardware, software, or a combination of both hardware and software, including one or more signal processing and/or application-specific integrated circuits.

The memory (102) optionally includes high-speed random access memory, and optionally also includes non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid state memory devices. The memory controller (122) optionally controls access to the memory (102) by other components of the device (100).

A peripheral interface (118) may be used to couple input and output peripherals of the device to the CPU (120) and memory (102). One or more processors (120) drive or execute various software programs and/or sets of instructions stored in the memory (102) to perform various functions for the device (100) and to process data. In some embodiments, the peripheral interface (118), CPU (120), and memory controller (122) are optionally implemented on a single chip, such as chip (104). In some other embodiments, they are optionally implemented on separate chips.

The RF (radio frequency) circuitry (108) receives and transmits RF signals, also referred to as electromagnetic signals. The RF circuitry (108) converts electrical signals to and from electromagnetic signals, and communicates with communication networks and other communication devices via the electromagnetic signals. The RF circuitry (108) optionally includes well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, etc. The RF circuitry (108) optionally communicates wirelessly with networks, such as the Internet, also referred to as the World Wide Web (WWW), an intranet, and/or a wireless network, such as a cellular telephone network, a wireless local area network (LAN) and/or a metropolitan area network (MAN), and other devices. The RF circuitry (108) optionally includes well-known circuitry for detecting near field communication (NFC) fields, for example by a short-range communication radio. Wireless communications may optionally include Global System for Mobile Communications (GSM), Enhanced Data GSM Environment (EDGE), high-speed downlink packet access (HSDPA), high-speed uplink packet access (HSUPA), Evolution, Data-Only (EV-DO), HSPA, HSPA+, Dual-Cell HSPA (DC-HSPDA), long term evolution (LTE), near field communication (NFC), wideband code division multiple access (W-CDMA), code division multiple access (CDMA), time division multiple access (TDMA), Bluetooth, Bluetooth Low Energy (BTLE), Wireless Fidelity (Wi-Fi) (e.g., IEEE 802.11a, IEEE 802.11b, IEEE 802.11g, IEEE 802.11n, and/or IEEE 802.11ac); Protocols for voice over Internet Protocol (VoIP), Wi-MAX, and email (e.g., Internet message access (IMAP)) protocol) and/or POP (post office protocol)), instant messaging (e.g., extensible messaging and presence protocol (XMPP), Session Initiation Protocol for Instant Messaging and Presence Leveraging Extensions (SIMPLE), Instant Messaging and Presence Service (IMPS)), and/or Short Message Service (SMS), or any other suitable communication protocol, including communication protocols not yet developed as of the filing date of this document.

The audio circuitry (110), the speaker (111), and the microphone (113) provide an audio interface between a user and the device (100). The audio circuitry (110) receives audio data from the peripheral interface (118), converts the audio data into an electrical signal, and transmits the electrical signal to the speaker (111). The speaker (111) converts the electrical signal into a sound wave that can be heard by a human. The audio circuitry (110) also receives the electrical signal converted from the sound wave by the microphone (113). The audio circuitry (110) converts the electrical signal into audio data, and transmits the audio data to the peripheral interface (118) for processing. The audio data is optionally retrieved from the memory (102) and/or the RF circuitry (108) by the peripheral interface (118) and/or transmitted to the memory (102) and/or the RF circuitry (108). In some embodiments, the audio circuitry (110) also includes a headset jack (e.g., 212 of FIG. 2). The headset jack provides an interface between the audio circuitry (110) and detachable audio input/output peripherals, such as output-only headphones, or a headset having both output (e.g., headphones for one or both ears) and input (e.g., a microphone).

The I/O subsystem (106) couples input/output peripherals on the device (100), such as a touch screen (112) and other input control devices (116), to the peripheral interface (118). The I/O subsystem (106) optionally includes one or more input controllers (160) for a display controller (156), a light sensor controller (158), an intensity sensor controller (159), a haptic feedback controller (161), and other input or control devices. The one or more input controllers (160) receive/transmit electrical signals from/to the other input control devices (116). The other input control devices (116) optionally include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, etc. In some alternative embodiments, the input controller(s) (160) are optionally coupled to any (or none) of a keyboard, an infrared port, a USB port, and a pointer device such as a mouse. One or more buttons (e.g., 208 of FIG. 2) optionally include up/down buttons for volume control of the speaker (111) and/or microphone (113). The one or more buttons optionally include a push button (e.g., 206 of FIG. 2).

A quick press of the push button optionally initiates processes to unlock the touch screen (112), or optionally use gestures on the touch screen to unlock the device, as described in U.S. Patent Application No. 11/322,549, filed December 23, 2005, entitled “Unlocking a Device by Performing Gestures on an Unlock Image,” (U.S. Pat. No. 7,657,849), which is hereby incorporated by reference in its entirety. A longer press of the push button (e.g., 206) optionally turns the device (100) on or off. The functionality of one or more of the buttons is optionally user customizable. The touch screen (112) is used to implement virtual or soft buttons and one or more soft keyboards.

A touch-sensitive display (112) provides an input interface and an output interface between the device and a user. A display controller (156) receives and/or transmits electrical signals from/to the touch screen (112). The touch screen (112) displays visual output to the user. The visual output optionally includes graphics, text, icons, video, and any combination thereof (collectively referred to as “graphics”). In some embodiments, some or all of the visual output optionally corresponds to user interface objects.

The touch screen (112) has a touch-sensitive surface, sensor, or set of sensors that accepts input from a user based on haptic and/or tactile contact. The touch screen (112) and the display controller (156) (along with any associated modules and/or sets of instructions in the memory (102)) detect contact (and any movement or cessation of contact) on the touch screen (112) and translate the detected contact into interaction with user interface objects displayed on the touch screen (112) (e.g., one or more soft keys, icons, web pages, or images). In an exemplary embodiment, the point of contact between the touch screen (112) and the user corresponds to the user's finger.

The touch screen (112) optionally uses liquid crystal display (LCD) technology, light emitting polymer display (LPD) technology, or light emitting diode (LED) technology, although other display technologies are used in other embodiments. The touch screen (112) and display controller (156) optionally detect contact and any movement or cessation thereof using any of a number of touch sensing technologies now known or later developed, including but not limited to capacitive, resistive, infrared, and surface acoustic wave technologies, as well as other proximity sensor arrays, or other elements for determining one or more points of contact with the touch screen (112). In an exemplary embodiment, projected mutual capacitance sensing technology is used, such as found in the iPhone® and iPod touch® from Apple Inc. of Cupertino, Calif.

In some embodiments of the touch screen (112), the touch-sensitive display is optionally similar to the multi-touch-sensitive touchpads described in U.S. Pat. Nos. 6,323,846 (Westerman et al.), 6,570,557 (Westerman et al.), and/or 6,677,932 (Westerman), and/or U.S. Patent Publication No. 2002/0015024A1, each of which is incorporated herein by reference in its entirety. However, the touch screen (112) displays visual output from the device (100), whereas the touch-sensitive touchpads do not provide visual output.

Touch-sensitive displays in some embodiments of the touch screen (112) are described in the following applications: (1) U.S. Patent Application No. 11/381,313, filed May 2, 2006, entitled "Multipoint Touch Surface Controller"; (2) U.S. Patent Application No. 10/840,862, filed May 6, 2004, entitled "Multipoint Touchscreen"; (3) U.S. Patent Application No. 10/903,964, filed July 30, 2004, entitled "Gestures For Touch Sensitive Input Devices"; (4) U.S. Patent Application No. 11/048,264, filed January 31, 2005, entitled "Gestures For Touch Sensitive Input Devices"; (5) U.S. Patent Application No. 11/038,590, filed Jan. 18, 2005, entitled "Mode-Based Graphical User Interfaces For Touch Sensitive Input Devices"; (6) U.S. Patent Application No. 11/228,758, filed Sep. 16, 2005, entitled "Virtual Input Device Placement On A Touch Screen User Interface"; (7) U.S. Patent Application No. 11/228,700, filed Sep. 16, 2005, entitled "Operation Of A Computer With A Touch Screen Interface"; (8) U.S. Patent Application No. 11/228,737, filed Sep. 16, 2005, entitled "Activating Virtual Keys Of A Touch-Screen Virtual Keyboard"; and (9) U.S. patent application Ser. No. 11/367,749, filed March 3, 2006, entitled “Multi-Functional Hand-Held Device”. All of which applications are incorporated herein by reference in their entirety.

The touch screen (112) optionally has a video resolution greater than 100 dpi. In some embodiments, the touch screen has a video resolution of approximately 160 dpi. The user optionally makes contact with the touch screen (112) using any suitable object or appendage, such as a stylus, a finger, or the like. In some embodiments, the user interface is designed to operate primarily using finger-based contacts and gestures, which may be less precise than stylus-based input due to the larger contact area of a finger on the touch screen. In some embodiments, the device converts rough finger-based input into precise pointer/cursor positions or commands for performing desired actions by the user.

In some embodiments, in addition to the touch screen, the device (100) optionally includes a touchpad (not shown) for activating or deactivating certain features. In some embodiments, the touchpad is a touch-sensitive area of the device that, unlike the touch screen, does not display visual output. The touchpad is optionally a touch-sensitive surface separate from the touch screen (112) or an extension of the touch-sensitive surface formed by the touch screen.

The device (100) also includes a power system (162) for supplying power to the various components. The power system (162) optionally includes a power management system, one or more power sources (e.g., batteries, alternating current (AC)), a recharging system, a power fault detection circuit, a power converter or inverter, a power status indicator (e.g., a light emitting diode (LED)), and any other components associated with generating, managing and distributing power within portable devices.

The device (100) also optionally includes one or more optical sensors (164). FIG. 1A illustrates a optical sensor coupled to a optical sensor controller (158) within the I/O subsystem (106). The optical sensor (164) optionally includes charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) phototransistors. The optical sensor (164) receives light from the surrounding environment, which is projected through one or more lenses, and converts the light into data representing an image. In conjunction with an imaging module (143) (also referred to as a camera module), the optical sensor (164) optionally captures still images or video. In some embodiments, the optical sensor is located on the back surface of the device (100), opposite the touch screen display (112) on the front surface of the device, such that the touch screen display can be used as a viewfinder for capturing still and/or video images. In some embodiments, the optical sensor is positioned on the front of the device so that the user's image is optionally acquired for video conferencing while viewing other video conferencing participants on the touch screen display. In some embodiments, the position of the optical sensor (164) can be changed by the user (e.g., by rotating the lens and sensor within the device housing), allowing a single optical sensor (164) to be used for both video conferencing and still and/or video image acquisition in conjunction with the touch screen display.

The device (100) optionally also includes one or more contact intensity sensors (165). FIG. 1A illustrates a contact intensity sensor coupled to an intensity sensor controller (159) within the I/O subsystem (106). The contact intensity sensor (165) optionally includes one or more piezoresistive strain gauges, capacitive force sensors, electrical force sensors, piezoelectric force sensors, optical force sensors, capacitive touch-sensitive surfaces, or other intensity sensors (e.g., sensors used to measure the force (or pressure) of a contact on a touch-sensitive surface). The contact intensity sensor (165) receives contact intensity information (e.g., pressure information or a proxy for pressure information) from the environment. In some embodiments, at least one contact intensity sensor is collocated with or proximate to the touch-sensitive surface (e.g., the touch-sensitive display system (112)). In some embodiments, at least one contact intensity sensor is positioned on the back surface of the device (100) opposite the touch screen display (112) positioned on the front surface of the device (100).

The device (100) also optionally includes one or more proximity sensors (166). FIG. 1A illustrates a proximity sensor (166) coupled to a peripheral interface (118). Alternatively, the proximity sensor (166) is optionally coupled to an input controller (160) within the I/O subsystem (106). The proximity sensor (166) may optionally be configured as described in U.S. Patent Application Nos. 11/241,839, "Proximity Detector In Handheld Device"; 11/240,788, "Proximity Detector In Handheld Device"; 11/620,702, "Using Ambient Light Sensor To Augment Proximity Sensor Output"; 11/586,862, "Automated Response To And Sensing Of User Activity In Portable Devices"; As described in U.S. Pat. No. 11/638,251, "Methods And Systems For Automatic Configuration Of Peripherals", which is hereby incorporated by reference in its entirety. In some embodiments, the proximity sensor turns off and disables the touch screen (112) when the multifunction device is placed near a user's ear (e.g., when the user is on a phone call).

The device (100) also optionally includes one or more tactile output generators (167). FIG. 1A illustrates a tactile output generator coupled to a haptic feedback controller (161) within the I/O subsystem (106). The tactile output generator (167) optionally includes one or more electroacoustic devices, such as speakers or other audio components, and/or electromechanical devices that convert energy into linear motion, such as a motor, a solenoid, an electroactive polymer, a piezoelectric actuator, an electrostatic actuator, or other tactile output generating component (e.g., a component that converts electrical signals into tactile outputs on the device). The contact intensity sensor (165) receives tactile feedback generating instructions from the haptic feedback module (133) and generates tactile outputs on the device (100) that can be sensed by a user of the device (100). In some embodiments, at least one tactile output generator is positioned with or proximate to a touch-sensitive surface (e.g., a touch-sensitive display system (112)) and optionally generates a tactile output by moving the touch-sensitive surface vertically (e.g., in/out of a surface of the device (100)) or laterally (e.g., back and forth in the same plane as the surface of the device (100)). In some embodiments, the at least one tactile output generator sensor is positioned on a back surface of the device (100) opposite the touch screen display (112) that is positioned on the front surface of the device (100).

The device (100) also optionally includes one or more accelerometers (168). FIG. 1A illustrates an accelerometer (168) coupled to a peripheral interface (118). Alternatively, the accelerometer (168) is optionally coupled to an input controller (160) within the I/O subsystem (106). The accelerometer (168) is optionally implemented as described in U.S. Patent Publication No. 20050190059, entitled “Acceleration-based Theft Detection System for Portable Electronic Devices,” and U.S. Patent Publication No. 20060017692, entitled “Methods And Apparatuses For Operating A Portable Device Based On An Accelerometer,” both of which are incorporated herein by reference in their entireties. In some embodiments, information is displayed on the touch screen display in either a portrait view or a landscape view based on analysis of data received from the one or more accelerometers. The device (100) optionally includes, in addition to the accelerometer(s) (168), a magnetometer (not shown), and a GPS (or GLONASS or other global navigation system) receiver (not shown) for obtaining information regarding the position and orientation (e.g., vertical or horizontal) of the device (100).

In some embodiments, the software components stored in the memory (102) include an operating system (126), a communications module (or set of instructions) (128), a contact/motion module (or set of instructions) (130), a graphics module (or set of instructions) (132), a text entry module (or set of instructions) (134), a GPS module (or set of instructions) (135), and applications (or sets of instructions) (136). Furthermore, in some embodiments, the memory (102 of FIG. 1A or 370 of FIG. 3) stores a device/global internal state (157), as illustrated in FIGS. 1A and 3 . The device/global internal state (157) includes an active application state, which indicates which applications, if any, are currently active; a display state, which indicates which applications, views, or other information occupy various regions of the touch screen display (112); Sensor status including information obtained from various sensors and input control devices (116) of the device; and one or more of position information regarding the position and/or attitude of the device.

An operating system (126) (e.g., an embedded operating system such as Darwin, RTXC, LINUX, UNIX, OS X, iOS, WINDOWS, or VxWorks) includes various software components and/or drivers to control and manage general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitate communication between various hardware and software components.

The communications module (128) enables communication with other devices via one or more external ports (124), and also includes various software components for processing data received by the RF circuitry (108) and/or the external port (124). The external port (124) (e.g., USB, FIREWIRE, etc.) is configured to couple directly to other devices or indirectly via a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector that is identical to, similar to, and/or compatible with the 30-pin connector used on iPod® (a trademark of Apple Inc.) devices.

The contact/motion module (130) optionally detects contact with the touch screen (112) (along with the display controller (156)) and other touch-sensitive devices (e.g., a touchpad or physical click wheel). The contact/motion module (130) includes various software components for performing various operations related to detecting contact, such as determining whether contact has occurred (e.g., detecting a finger-down event), determining the magnitude of the contact (e.g., the force or pressure of the contact, or a surrogate for the force or pressure of the contact), determining whether there is movement of the contact and tracking the movement across the touch-sensitive surface (e.g., detecting one or more finger-dragging events), and determining whether contact has ceased (e.g., detecting a finger-up event or a cessation of contact). The contact/motion module (130) receives contact data from the touch-sensitive surface. Determining the movement of a contact point represented by a series of contact data optionally includes determining the speed (magnitude), velocity (magnitude and direction), and/or acceleration (change in magnitude and/or direction) of the contact point. These operations optionally apply to single contacts (e.g., single finger contacts) or multiple simultaneous contacts (e.g., “multiple contacts”/multiple finger contacts). In some embodiments, the contact/motion module (130) and the display controller (156) detect contacts on the touchpad.

In some embodiments, the contact/motion module (130) uses a set of one or more intensity thresholds to determine whether an action has been performed by a user (e.g., to determine whether the user has "clicked" on an icon). In some embodiments, at least a subset of the intensity thresholds are determined by software parameters (e.g., the intensity thresholds are not determined by the activation thresholds of specific physical actuators, and can be adjusted without changing the physical hardware of the device (100). For example, a mouse "click" threshold of a trackpad or touch screen display can be set to any of a wide range of predefined threshold values without changing the trackpad or touch screen display hardware. Additionally, in some implementations, a user of the device is provided with software settings to adjust one or more of the set of intensity thresholds (e.g., by adjusting individual intensity thresholds and/or by adjusting multiple intensity thresholds at once as a system-level click "intensity" parameter).

The contact/motion module (130) optionally detects a gestural input by the user. Different gestures on the touch-sensitive surface have different contact patterns (e.g., different motions, timings, and/or intensities of the detected contacts). Accordingly, a gesture is optionally detected by detecting a particular contact pattern. For example, detecting a finger tap gesture includes detecting a finger-down event, followed by detecting a finger-up (liftoff) event at the same location (or substantially the same location) as the finger-down event (e.g., at the location of an icon). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger-down event, followed by detecting one or more finger-drag events, followed by detecting a finger-up (liftoff) event.

The graphics module (132) includes various known software components for rendering and displaying graphics on the touch screen (112) or other display, including components for changing the visual effects (e.g., brightness, transparency, saturation, contrast, or other visual properties) of the graphics being displayed. As used herein, the term "graphics" includes any object that can be displayed to a user, including without limitation text, web pages, icons (e.g., user interface objects including soft keys), digital images, videos, animations, and the like.

In some embodiments, the graphics module (132) stores data representing the graphics to be used. Each graphic is optionally assigned a corresponding code. The graphics module (132) receives one or more codes specifying the graphics to be displayed from applications, etc., along with coordinate data and other graphic attribute data, if necessary, and then generates screen image data and outputs it to the display controller (156).

The haptic feedback module (133) includes various software components for generating commands used by the haptic output generator(s) (167) to generate tactile outputs at one or more locations on the device (100) in response to user interactions with the device (100).

Optionally, a text input module (134), which is a component of the graphics module (132), provides soft keyboards for entering text into various applications (e.g., contacts (137), email (140), IM (141), browser (147), and any other application requiring text input).

The GPS module (135) determines the location of the device and provides this information for use in various applications (e.g., to the phone (138) for use in location-based dialing; to the camera (143) as photo/video metadata; and to applications that provide location-based services such as weather widgets, local yellow pages widgets, and map/navigation widgets).

The applications (136) optionally include the following modules (or sets of instructions), or a subset or superset thereof:

연락처 모듈(137)(때때로 주소록 또는 연락처 목록으로 지칭됨);

Contacts module (137) (sometimes referred to as an address book or contact list);

Telephone module (138);

Video conferencing module (139);

Email client module (140);

Instant messaging (IM) module (141);

Exercise support module (142);

Camera module (143) for still and/or video images;

Image management module (144);

Video player module;

Music player module;

Browser module (147);

Calendar module (148);

Widget modules (149) optionally including one or more of a weather widget (149-1), a stock widget (149-2), a calculator widget (149-3), an alarm clock widget (149-4), a dictionary widget (149-5), and other widgets obtained by the user as well as user-created widgets (149-6);

A widget generator module (150) for creating user-created widgets (149-6);

Search module (151);

A video and music player module (152) integrating a video player module and a music player module;

Memo module (153);

map module (154); and/or

Online video module (155).

Examples of other applications (136) optionally stored within memory (102) include other word processing applications, other image editing applications, drawing applications, presentation applications, JAVA-enabled applications, encryption, digital rights management, speech recognition and speech transcription.

In conjunction with the touch screen (112), the display controller (156), the contact/motion module (130), the graphics module (132) and the text input module (134), the contacts module (137) is optionally used to manage an address book or list of contacts (e.g., stored in the application internal state (192) of the contacts module (137) within the memory (102) or memory (370), including: adding name(s) to the address book; deleting name(s) from the address book; associating phone number(s), email address(es), physical address(es) or other information with a name; associating an image with a name; categorizing and sorting names; providing phone numbers or email addresses to initiate and/or facilitate communication by phone (138), video conferencing module (139), email (140) or IM (141), etc.

In conjunction with the RF circuitry (108), the audio circuitry (110), the speaker (111), the microphone (113), the touch screen (112), the display controller (156), the contact/motion module (130), the graphics module (132), and the text entry module (134), the telephone module (138) is optionally used to input a sequence of characters corresponding to a phone number, access one or more phone numbers within the contacts module (137), modify an input phone number, dial an individual phone number, engage in a conversation, and disconnect or hang up when the conversation is complete. As noted above, the wireless communications optionally utilize any of a plurality of communication standards, protocols and technologies.

In conjunction with the RF circuitry (108), the audio circuitry (110), the speaker (111), the microphone (113), the touch screen (112), the display controller (156), the light sensor (164), the light sensor controller (158), the contact/motion module (130), the graphics module (132), the text input module (134), the contact module (137), and the phone module (138), the video conferencing module (139) includes executable instructions for initiating, conducting, and terminating a video conference between a user and one or more other participants according to user instructions.

In conjunction with the RF circuitry (108), touch screen (112), display controller (156), contact/motion module (130), graphics module (132), and text input module (134), the email client module (140) includes executable instructions to compose, send, receive, and manage emails in response to user instructions. In conjunction with the image management module (144), the email client module (140) makes it very easy to create and send emails having still or video images captured by the camera module (143).

In conjunction with the RF circuitry (108), the touch screen (112), the display controller (156), the contact/motion module (130), the graphics module (132), and the text input module (134), the instant messaging module (141) includes executable instructions to input a sequence of characters corresponding to an instant message, to modify previously input characters, to send individual instant messages (e.g., using the Short Message Service (SMS) or Multimedia Message Service (MMS) protocols for phone-based instant messages, or XMPP, SIMPLE or IMPS for Internet-based instant messages), to receive instant messages, and to view received instant messages. In some embodiments, the sent and/or received instant messages optionally include graphics, photographs, audio files, video files, and/or other attachments as supported by MMS and/or Enhanced Messaging Service (EMS). As used herein, “instant messaging” refers to both phone-based messages (e.g., messages sent using SMS or MMS) and Internet-based messages (e.g., messages sent using XMPP, SIMPLE or IMPS).

In conjunction with the RF circuitry (108), the touch screen (112), the display controller (156), the contact/motion module (130), the graphics module (132), the text entry module (134), the GPS module (135), the map module (154), and the music player module, the workout assistance module (142) includes executable instructions to: design workouts (e.g., with time, distance, and/or calorie consumption goals); communicate with workout sensors (sports devices); receive workout sensor data; calibrate sensors used to monitor workouts; select and play music for workouts; and display, store, and transmit workout data.

In conjunction with the touch screen (112), the display controller (156), the light sensor(s) (164), the light sensor controller (158), the contact/motion module (130), the graphics module (132), and the image management module (144), the camera module (143) includes executable instructions to capture still images or video (including a video stream) and store them in the memory (102), modify characteristics of a still image or video, or delete a still image or video from the memory (102).

In conjunction with the touch screen (112), display controller (156), contact/motion module (130), graphics module (132), text input module (134), and camera module (143), the image management module (144) includes executable instructions to arrange, modify (e.g., edit), or otherwise manipulate, label, delete, present (e.g., in a digital slide show or album), and store still and/or video images.

In conjunction with the RF circuitry (108), the touch screen (112), the display controller (156), the contact/motion module (130), the graphics module (132), and the text input module (134), the browser module (147) includes executable instructions for browsing the Internet according to user instructions, including retrieving, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.

In conjunction with the RF circuitry (108), the touch screen (112), the display controller (156), the contact/motion module (130), the graphics module (132), the text input module (134), the email client module (140), and the browser module (147), the calendar module (148) includes executable instructions that cause calendars and data associated with the calendars (e.g., calendar entries, to-do lists, etc.) to be created, displayed, modified, and stored in accordance with user instructions.

Along with the RF circuitry (108), the touch screen (112), the display controller (156), the contact/motion module (130), the graphics module (132), the text input module (134), and the browser module (147), the widget modules (149) are mini-applications that are optionally downloaded and used by the user (e.g., the weather widget (149-1), the stock widget (149-2), the calculator widget (149-3), the alarm clock widget (149-4), and the dictionary widget (149-5)), or created by the user (e.g., the user-created widget (149-6)). In some embodiments, the widget includes a Hypertext Markup Language (HTML) file, a Cascading Style Sheets (CSS) file, and a JavaScript file. In some embodiments, the widget includes an Extensible Markup Language (XML) file and a JavaScript file (e.g., Yahoo! widgets).

In conjunction with the RF circuitry (108), touch screen (112), display controller (156), contact/motion module (130), graphics module (132), text input module (134), and browser module (147), a widget generator module (150) is optionally used to create widgets by the user (e.g., changing a user-specified portion of a web page into a widget).

In conjunction with the touch screen (112), the display controller (156), the contact/motion module (130), the graphics module (132), and the text input module (134), the search module (151) includes executable instructions that cause the memory (102) to search for text, music, sound, images, video, and/or other files that match one or more search criteria (e.g., one or more user-specific search terms) according to user instructions.

In conjunction with the touch screen (112), the display controller (156), the contact/motion module (130), the graphics module (132), the audio circuitry (110), the speaker (111), the RF circuitry (108), and the browser module (147), the video and music player module (152) includes executable instructions that enable a user to download and play recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files, and executable instructions that cause videos to be displayed, played, or otherwise played (e.g., on the touch screen (112) or on an externally connected display via the external port (124). In some embodiments, the device (100) optionally includes the functionality of an MP3 player, such as an iPod (a trademark of Apple Inc.).

In conjunction with the touch screen (112), display controller (156), contact/motion module (130), graphics module (132) and text input module (134), the memo module (153) includes executable instructions to create and manage memos, to-do lists, etc. according to user instructions.

In conjunction with the RF circuitry (108), touch screen (112), display controller (156), contact/motion module (130), graphics module (132), text entry module (134), GPS module (135), and browser module (147), the map module (154) is optionally used to receive, display, modify, and store maps and data associated with the maps (e.g., driving directions, data regarding stores and other points of interest at or near a particular location, and other location-based data) in accordance with user instructions.

In conjunction with the touch screen (112), the display controller (156), the contact/motion module (130), the graphics module (132), the audio circuitry (110), the speaker (111), the RF circuitry (108), the text input module (134), the email client module (140), and the browser module (147), the online video module (155) includes instructions that enable a user to access, browse, receive (e.g., by streaming and/or downloading), play (e.g., on the touch screen or on an external connected display via the external port (124), send an email with a link to a particular online video, and otherwise manage online videos in one or more file formats, such as H.264. In some embodiments, an instant messaging module (141), rather than the email client module (140), is used to send a link to a particular online video. Additional descriptions of online video applications may be found in U.S. Provisional Patent Application No. 60/936,562, filed June 20, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos” and U.S. Patent Application No. 11/968,067, filed December 31, 2007, entitled “Portable Multifunction Device, Method, and Graphical User Interface for Playing Online Videos”, the contents of which are hereby incorporated herein by reference in their entireties.

Each of the modules and applications identified above corresponds to a set of executable instructions for performing one or more of the functions described above and the methods described herein (e.g., the computer-implemented methods and other information processing methods described herein). These modules (e.g., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules may optionally be combined or otherwise rearranged in various embodiments. For example, a video player module may optionally be combined with a music player module into a single module (e.g., the video and music player module (152) of FIG. 1A ). In some embodiments, memory (102) optionally stores a subset of the modules and data structures identified above. Furthermore, memory (102) optionally stores additional modules and data structures not described above.

In some embodiments, the device (100) is a device in which operation of a predefined set of functions on the device is exclusively performed via a touch screen and/or a touchpad. By using the touch screen and/or the touchpad as the primary input control device for operation of the device (100), the number of physical input control devices (such as push buttons, dials, etc.) on the device (100) is optionally reduced.

A predefined set of functions performed entirely via the touch screen and/or touchpad optionally includes navigation between user interfaces. In some embodiments, the touchpad, when touched by a user, navigates the device (100) to a main, home, or root menu from any user interface displayed on the device (100). In such embodiments, a "menu button" is implemented using the touchpad. In some other embodiments, the menu button is a physical push button or other physical input control device instead of the touchpad.

FIG. 1B is a block diagram illustrating exemplary components for event processing, according to some embodiments. In some embodiments, a memory (102 of FIG. 1A or 370 of FIG. 3) includes an event classifier (170) (e.g., in an operating system (126)) and an individual application (136-1) (e.g., any of the applications (137-151, 155, 380-390) described above).

An event classifier (170) receives event information and determines an application (136-1) to which the event information will be delivered, and an application view (191) of the application (136-1). The event classifier (170) includes an event monitor (171) and an event dispatcher module (174). In some embodiments, the application (136-1) includes an application internal state (192) that indicates the current application view(s) displayed on the touch-sensitive display (112) when the application is active or running. In some embodiments, the device/global internal state (157) is used by the event classifier (170) to determine which application(s) is currently active, and the application internal state (192) is used by the event classifier (170) to determine the application views (191) to which the event information will be delivered.

In some embodiments, the application internal state (192) includes additional information such as one or more of: resume information to be used when the application (136-1) resumes execution, user interface state information indicating information being displayed or ready to be displayed by the application (136-1), a state queue to enable a user to return to a previous state or view of the application (136-1), and a redo/undo queue of previous actions taken by the user.

The event monitor (171) receives event information from the peripheral interface (118). The event information includes information about a sub-event (e.g., a user touch on the touch-sensitive display (112) as part of a multi-touch gesture). The peripheral interface (118) transmits information received from the I/O subsystem (106) or sensors, such as proximity sensors (166), accelerometer(s) (168), and/or microphones (113) (via the audio circuitry (110)). The information that the peripheral interface (118) receives from the I/O subsystem (106) includes information from the touch-sensitive display (112) or the touch-sensitive surface.

In some embodiments, the event monitor (171) transmits requests to the peripheral interface (118) at predetermined intervals. In response, the peripheral interface (118) transmits event information. In other embodiments, the peripheral interface (118) transmits event information only when there is a significant event (e.g., receiving input exceeding a predetermined noise threshold and/or input for more than a predetermined duration).

In some embodiments, the event classifier (170) also includes a hit view determination module (172) and/or an active event recognizer determination module (173).

The hit view determination module (172) provides software procedures for determining where a sub-event occurred within one or more views when the touch-sensitive display (112) displays more than one view. The views consist of controls and other elements that a user can see on the display.

Another aspect of the user interface associated with the application is a set of views, sometimes referred to herein as application views or user interface windows, in which information is displayed and touch-based gestures occur. The application views (of an individual application) in which touches are detected optionally correspond to programmatic levels within the application's program or view hierarchy. For example, the lowest level view in which a touch is detected is optionally referred to as a hit view, and the set of events that are recognized as appropriate inputs are optionally determined based at least in part on the hit view of the initial touch that initiates the touch-based gesture.

The hit view determination module (172) receives information related to sub-events of a touch-based gesture. When an application has multiple views organized in a hierarchy, the hit view determination module (172) identifies a hit view as the lowest level view in the hierarchy that should process a sub-event. In most situations, the hit view is the lowest level view in which an initiating sub-event occurs (e.g., the first sub-event in a sequence of sub-events forming an event or potential event). Once a hit view is identified by the hit view determination module (172), the hit view typically receives all sub-events associated with the same touch or input source that caused it to be identified as the hit view.

The active event recognizer determination module (173) determines which view or views within the view hierarchy should receive a particular sequence of sub-events. In some embodiments, the active event recognizer determination module (173) determines that only the hit view should receive the particular sequence of sub-events. In other embodiments, the active event recognizer determination module (173) determines that all views that include the physical location of the sub-event are actively involved views, and thus all actively involved views should receive the particular sequence of sub-events. In other embodiments, even if the touch sub-events are confined entirely to a region associated with a particular view, the higher views within the hierarchy will still remain as actively involved views.

The event dispatcher module (174) dispatches event information to an event recognizer (e.g., an event recognizer (180)). In embodiments that include an active event recognizer determination module (173), the event dispatcher module (174) passes the event information to the event recognizer determined by the active event recognizer determination module (173). In some embodiments, the event dispatcher module (174) stores the event information in an event queue, from which the event information is retrieved by individual event receivers (182).

In some embodiments, the operating system (126) includes the event classifier (170). Alternatively, the application (136-1) includes the event classifier (170). In still other embodiments, the event classifier (170) is a standalone module or is part of another module stored in memory (102), such as the contact/motion module (130).

In some embodiments, the application (136-1) includes a plurality of event handlers (190) and one or more application views (191), each of which includes instructions for processing touch events that occur within a respective view of the application's user interface. Each application view (191) of the application (136-1) includes one or more event recognizers (180). Typically, an individual application view (191) includes a plurality of event recognizers (180). In other embodiments, one or more of the event recognizers (180) are part of a separate module, such as a user interface kit (not shown) or a higher level object from which the application (136-1) inherits methods and other properties. In some embodiments, an individual event handler (190) includes one or more of the following: a data updater (176), an object updater (177), a GUI updater (178), and/or an event classifier (170). The event handler (190) optionally updates the application internal state (192) by utilizing or calling a data updater (176), an object updater (177), or a GUI updater (178). Alternatively, one or more of the application views (191) include one or more individual event handlers (190). Additionally, in some embodiments, one or more of the data updater (176), the object updater (177), and the GUI updater (178) are included within an individual application view (191).

An individual event recognizer (180) receives event information (e.g., event data (179)) from an event classifier (170) and identifies an event from the event information. The event recognizer (180) includes an event receiver (182) and an event comparator (184). In some embodiments, the event recognizer (180) also includes at least a subset of metadata (183) and event delivery instructions (188) (optionally including sub-event delivery instructions).

The event receiver (182) receives event information from the event classifier (170). The event information includes information about a sub-event, for example, a touch or a touch movement. Depending on the sub-event, the event information also includes additional information, such as the location of the sub-event. When the sub-event relates to the motion of a touch, the event information optionally also includes speed and direction of the sub-event. In some embodiments, the events include rotation of the device from one orientation to another (e.g., from a portrait orientation to a landscape orientation, or vice versa), and the event information includes corresponding information about the current orientation of the device (also referred to as the device pose).

An event comparator (184) compares event information with predefined event or sub-event definitions and, based on the comparison, determines an event or sub-event, or determines or updates a status of an event or sub-event. In some embodiments, the event comparator (184) includes event definitions (186). The event definitions (186) include definitions of events (e.g., predefined sequences of sub-events), for example, Event 1 (187-1), Event 2 (187-2), etc. In some embodiments, the sub-events within an event (187) include, for example, Touch Start, Touch End, Touch Move, Touch Cancel, and Multi-Touch. In one example, a definition for Event 1 (187-1) is a double tap on a displayed object. A double tap includes, for example, a first touch (touch start) on the displayed object during a predetermined phase, a first liftoff (touch end) on the displayed object during the predetermined phase, a second touch (touch start) on the displayed object during the predetermined phase, and a second liftoff (touch end) on the displayed object during the predetermined phase. In another example, the definition for event 2 (187-2) is dragging on the displayed object. Dragging includes, for example, a touch (or contact) on the displayed object during a predetermined phase, movement of the touch across the touch-sensitive display (112), and a liftoff (touch end) of the touch. In some embodiments, the event also includes information about one or more associated event handlers (190).

In some embodiments, the event definition (187) includes definitions of events for individual user interface objects. In some embodiments, the event comparator (184) performs a hit test to determine which user interface object is associated with the sub-event. For example, in an application view where three user interface objects are displayed on a touch-sensitive display (112), when a touch is detected on the touch-sensitive display (112), the event comparator (184) performs a hit test to determine which of the three user interface objects is associated with the touch (sub-event). If each displayed object is associated with a respective event handler (190), the event comparator uses the results of the hit test to determine which event handler (190) should be activated. For example, the event comparator (184) selects the object that triggers the hit test and the event handler associated with the sub-event.

In some embodiments, the definition for an individual event (187) also includes delayed actions that delay delivery of event information until after it has been determined whether the sequence of sub-events corresponds to or does not correspond to an event type of the event recognizer.

When an individual event recognizer (180) determines that a series of sub-events do not match any of the events within the event definitions (186), the individual event recognizer (180) enters an event not possible, event failed, or event ended state, and the individual event recognizer then ignores subsequent sub-events of the touch-based gesture. In this situation, other event recognizers, if any, that remain active for the hit view continue to track and process sub-events of the ongoing touch-based gesture.

In some embodiments, individual event recognizers (180) include metadata (183) having configurable properties, flags, and/or lists that indicate how the event delivery system should perform sub-event delivery to the actively participating event recognizers. In some embodiments, metadata (183) includes configurable properties, flags, and/or lists that indicate how the event recognizers interact, or are caused to interact, with each other. In some embodiments, metadata (183) includes configurable properties, flags, and/or lists that indicate whether sub-events are delivered to various levels in a view or program hierarchy.

In some embodiments, an individual event recognizer (180) activates an event handler (190) associated with an event when one or more specific sub-events of the event are recognized. In some embodiments, an individual event recognizer (180) passes event information associated with the event to the event handler (190). Activating the event handler (190) is separate from sending (and delaying sending) the sub-events to each hit view. In some embodiments, an event recognizer (180) sends a flag associated with a recognized event, and an event handler (190) associated with that flag catches the flag and performs a predefined process.

In some embodiments, the event delivery commands (188) include sub-event delivery commands that deliver event information about a sub-event without activating an event handler. Instead, the sub-event delivery commands deliver event information to event handlers associated with a series of sub-events or to actively participating views. The event handlers associated with the series of sub-events or actively participating views receive the event information and perform predetermined processing.

In some embodiments, the data updater (176) creates and updates data used in the application (136-1). For example, the data updater (176) updates a phone number used in the contact module (137) or stores a video file used in the video player module. In some embodiments, the object updater (177) creates and updates objects used in the application (136-1). For example, the object updater (177) creates a new user interface object or updates the location of a user interface object. The GUI updater (178) updates the GUI. For example, the GUI updater (178) prepares display information for display on a touch-sensitive display and transmits it to the graphics module (132).

In some embodiments, the event handler(s) (190) include or access a data updater (176), an object updater (177), and a GUI updater (178). In some embodiments, the data updater (176), the object updater (177), and the GUI updater (178) are contained within a single module of an individual application (136-1) or application view (191). In other embodiments, they are contained within two or more software modules.

The above discussion of event processing of user touches on a touch-sensitive display also applies to other forms of user input for operating multifunction devices (100) having input devices, although it should be understood that not all of them are initiated on touch screens. For example, mouse movements and mouse button presses, optionally in combination with single or multiple keyboard presses or holds; contact movements such as taps, drags, scrolls, etc. on a touchpad; pen stylus inputs; movement of the device; spoken commands; detected eye movements; biometric inputs; and/or any combination thereof, are optionally utilized as inputs corresponding to sub-events defining an event to be recognized.

FIG. 2 illustrates a portable multifunction device (100) having a touch screen (112), according to some embodiments. The touch screen optionally displays one or more graphics within a user interface (UI) (200). In this embodiment, as well as others described below, a user is enabled to select one or more of the graphics by making a gesture over the graphics, for example, using one or more fingers (202) (not drawn to scale in the drawing) or one or more styli (203) (not drawn to scale in the drawing). In some embodiments, selection of the one or more graphics occurs when the user breaks contact with the one or more graphics. In some embodiments, the gesture optionally includes one or more taps, one or more swipes (left to right, right to left, up, and/or down), and/or a roll (right to left, left to right, up, and/or down) of a finger in contact with the device (100). In some implementations or situations, inadvertent contact with a graphic does not select the graphic. For example, a swipe gesture that sweeps over an application icon when the corresponding gesture for selection is a tap does not optionally select the corresponding application.

In some embodiments, the stylus (203) is an active device and includes one or more electronic circuitry. For example, the stylus (203) includes one or more sensors, and one or more communication circuitry (e.g., the communication module (128) and/or the RF circuitry (108)). In some embodiments, the stylus (203) includes one or more processors and power systems (e.g., similar to the power system (162)). In some embodiments, the stylus (203) includes an accelerometer (e.g., the accelerometer (168)), a magnetometer, and/or a gyroscope, which can determine the position, angle, location, and/or other physical characteristics of the stylus (203) (e.g., whether the stylus is lying down, tilted toward or away from the device, and/or close to or away from the device). In some embodiments, the stylus (203) communicates with the electronic device (e.g., via a wireless communication protocol such as Bluetooth, via communication circuitry) and transmits sensor data to the electronic device. In some embodiments, the stylus (203) may determine whether the user is holding the device (e.g., via an accelerometer or other sensors). In some embodiments, the stylus (203) may accept tap inputs (e.g., a single tap or a double tap) on the stylus (203) from the user (e.g., received by an accelerometer or other sensors) and may interpret the input as a command or request to perform a function or change to a different input mode.

The device (100) also optionally includes one or more physical buttons, such as a “home” or menu button (204). As previously described, the menu button (204) is optionally used to navigate to any application (136) within the set of applications optionally running on the device (100). Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on the touch screen (112).

In some embodiments, the device (100) includes a touch screen (112), a menu button (204), a push button (206) to power the device on/off and lock the device, volume control button(s) (208), a subscriber identity module (SIM) card slot (210), a headset jack (212), and a docking/charging external port (124). The push button (206) is optionally used to power the device on/off by pressing the button and holding it pressed for a predefined time interval; to lock the device by pressing the button and releasing the button before the predefined time interval has elapsed; and/or to unlock the device or initiate an unlocking process. In an alternative embodiment, the device (100) also accepts verbal input via the microphone (113) to activate or deactivate certain features. The device (100) also optionally includes one or more contact intensity sensors (165) for detecting the intensity of contacts on the touch screen (112) and/or one or more tactile output generators (167) for generating tactile outputs for a user of the device (100).

FIG. 3 is a block diagram of an exemplary multifunction device having a display and a touch-sensitive surface, according to some embodiments. The device (300) need not be portable. In some embodiments, the device (300) is a laptop computer, a desktop computer, a tablet computer, a multimedia player device, a navigation device, an educational device (such as a children's learning toy), a gaming system, or a control device (e.g., a home or industrial controller). The device (300) typically includes one or more processing units (CPUs) (310), one or more network or other communication interfaces (360), memory (370), and one or more communication buses (320) for interconnecting these components. The communication buses (320) optionally include circuitry (sometimes referred to as a chipset) for interconnecting the system components and controlling communications therebetween. The device (300) includes an input/output (I/O) interface (330) that typically includes a display (340), which is a touch screen display. The I/O interface (330) also optionally includes a keyboard and/or mouse (or other pointing device) (350) and a touchpad (355), a tactile output generator (357) for generating tactile outputs on the device (300) (e.g., similar to the tactile output generator(s) (167) described above with reference to FIG. 1A ), and sensors (359) (e.g., an optical sensor, an acceleration sensor, a proximity sensor, a touch-sensitive sensor, and/or a contact intensity sensor similar to the contact intensity sensor(s) (165) described above with reference to FIG. 1A ). The memory (370) includes high-speed random access memory, such as DRAM, SRAM, DDR RAM, or other random access solid state memory devices; and optionally nonvolatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other nonvolatile solid state storage devices. The memory (370) optionally includes one or more storage devices located remotely from the CPU(s) (310). In some embodiments, the memory (370) stores programs, modules, and data structures similar to the programs, modules, and data structures stored in the memory (102) of the portable multifunction device (100) (FIG. 1A), or a subset thereof. Furthermore, the memory (370) optionally stores additional programs, modules, and data structures that are not present in the memory (102) of the portable multifunction device (100). For example, the memory (370) of the device (300) optionally stores a drawing module (380), a presentation module (382), a word processing module (384), a website creation module (386), a disk authoring module (388), and/or a spreadsheet module (390), while the memory (102) of the portable multifunction device (100) (FIG. 1A) optionally does not store such modules.

Each of the elements identified above in FIG. 3 is optionally stored in one or more of the previously mentioned memory devices. Each of the modules identified above corresponds to a set of instructions for performing the function described above. The modules or programs (e.g., sets of instructions) identified above need not be implemented as separate software programs, procedures or modules, and thus, in various embodiments, various subsets of these modules may be optionally combined or otherwise rearranged. In some embodiments, the memory (370) optionally stores a subset of the modules and data structures identified above. Furthermore, the memory (370) optionally stores additional modules and data structures not described above.

Now, let us consider embodiments of user interfaces optionally implemented on, for example, a portable multifunction device (100).

FIG. 4A illustrates an exemplary user interface for a menu of applications on a portable multifunction device (100), according to some embodiments. Similar user interfaces are optionally implemented on the device (300). In some embodiments, the user interface (400) includes the following elements, or a subset or superset thereof:

Signal strength indicator(s) (402) for wireless communication(s), such as cellular and Wi-Fi signals;

Time (404);

Bluetooth indicator (405);

Battery status indicator (406);

A tray (408) with icons for frequently used applications, such as:

An icon (416) for a phone module (138) labeled “Phone” optionally including an indicator (414) of the number of missed calls or voicemail messages;

An icon (418) for an email client module (140) labeled “Mail” optionally including an indicator (410) of the number of unread emails;

o Icon (420) for a browser module (147) labeled "Browser"; and

An icon (422) for a video and music player module (152) labeled “iPod”, also referred to as an iPod (a trademark of Apple Inc.) module (152); and

Icons for other applications, such as:

Icon (424) for IM module (141) labeled “Message”;

o Icon (426) for the calendar module (148) labeled “Calendar”;

Icon (428) for the image management module (144) labeled “Photo”;

Icon (430) for a camera module (143) labeled “Camera”;

o Icon (432) for an online video module (155) labeled “Online Video”;

Icon (434) for the stock widget (149-2) labeled “Stocks”;

Icon (436) for the map module (154) labeled “map”;

Icon (438) for a weather widget (149-1) labeled “Weather”;

Icon (440) for an alarm clock widget (149-4) labeled "Clock";

o Icon (442) for the exercise support module (142) labeled “exercise support”;

o Icon (444) for a note module (153) labeled "Note"; and

An icon (446) for a settings application or module, labeled “Settings”, which provides access to settings for the device (100) and its various applications (136).

It should be noted that the icon labels illustrated in FIG. 4A are merely examples. For example, the icon (422) for the video and music player module (152) is labeled "Music" or "Music Player." Other labels are optionally used for various application icons. In some embodiments, the label for an individual application icon includes the name of the application corresponding to the individual application icon. In some embodiments, the label for a particular application icon is separate from the name of the application corresponding to the particular application icon.

FIG. 4B illustrates an exemplary user interface on a device (e.g., device (300) of FIG. 3) having a touch-sensitive surface (451) (e.g., a tablet or touchpad (355) of FIG. 3) separate from a display (450) (e.g., a touch screen display (112)). The device (300) optionally also includes one or more contact intensity sensors (e.g., one or more of the sensors (359)) for detecting the intensity of contacts on the touch-sensitive surface (451) and/or one or more tactile output generators (357) for generating tactile outputs to a user of the device (300).

While some of the following examples will be provided with reference to inputs on a touch screen display (112) (which is a combination of a touch-sensitive surface and a display), in some embodiments, the device detects inputs on a touch-sensitive surface that is separate from the display, as illustrated in FIG. 4B . In some embodiments, the touch-sensitive surface (e.g., 451 of FIG. 4B ) has a principal axis (e.g., 452 of FIG. 4B ) that corresponds to a principal axis (e.g., 453 of FIG. 4B ) on the display (e.g., 450 ). According to these embodiments, the device detects contacts (e.g., 460 and 462 of FIG. 4B ) with the touch-sensitive surface (451) at locations that correspond to respective locations on the display (e.g., 460 corresponds to 468 and 462 corresponds to 470 in FIG. 4B ). In this manner, user inputs (e.g., contacts (460, 462) and movements thereof) detected by the device on the touch-sensitive surface (e.g., 451 of FIG. 4b ) are used by the device to manipulate a user interface on the display of the multifunction device (e.g., 450 of FIG. 4b ) when the touch-sensitive surface is separate from the display. It should be appreciated that similar methods may optionally be used for other user interfaces described herein.

Additionally, while the following examples are provided primarily with reference to finger inputs (e.g., finger contacts, finger tap gestures, finger swipe gestures), it should be understood that in some embodiments, one or more of the finger inputs are replaced with input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced with a mouse click (e.g., instead of a contact) followed by movement of the cursor along the path of the swipe (e.g., instead of movement of the contact). As another example, a tap gesture is optionally replaced with a mouse click while the cursor is positioned over the location of the tap gesture (e.g., instead of detection of the contact and subsequent detection of the contact being interrupted). Similarly, it should be understood that when multiple user inputs are detected simultaneously, multiple computer mice are optionally used simultaneously, or that mice and finger contacts are optionally used simultaneously.

FIG. 5A illustrates an exemplary personal electronic device (500). The device (500) includes a body (502). In some embodiments, the device (500) may include some or all of the features described for devices (100, 300) (e.g., FIGS. 1A-4B ). In some embodiments, the device (500) has a touch-sensitive display screen (504) (hereinafter, “touch screen 504”). Alternatively or additionally to the touch screen (504), the device (500) has a display and a touch-sensitive surface. Like the devices (100, 300), in some embodiments, the touch screen (504) (or touch-sensitive surface) optionally includes one or more intensity sensors for detecting the intensity of applied contacts (e.g., touches). One or more intensity sensors of the touch screen (504) (or touch-sensitive surface) can provide output data representing the intensity of touches. The user interface of the device (500) can respond to touches based on the intensity of the touches, meaning that touches of different intensities can invoke different user interface actions on the device (500).

Exemplary techniques for detecting and processing touch intensity can be found, for example, in related applications: International Patent Application No. PCT/US2013/040061, filed May 8, 2013, entitled "Device, Method, and Graphical User Interface for Displaying User Interface Objects Corresponding to an Application" (published as WIPO Publication No. WO/2013/169849), and International Patent Application No. PCT/US2013/069483, filed November 11, 2013, entitled "Device, Method, and Graphical User Interface for Transitioning Between Touch Input to Display Output Relationships" (published as WIPO Publication No. WO/2014/105276), each of which is herein incorporated by reference in its entirety.

In some embodiments, the device (500) has one or more input mechanisms (506, 508). The input mechanisms (506, 508) (if included) can be physical. Examples of physical input mechanisms include push buttons and rotatable mechanisms. In some embodiments, the device (500) has one or more attachment mechanisms. These attachment mechanisms (if included) allow the device (500) to be attached to, for example, a hat, glasses, earrings, a necklace, a shirt, a jacket, a bracelet, a watch strap, a chain, pants, a belt, a shoe, a purse, a backpack, and the like. These attachment mechanisms allow the device (500) to be worn by a user.

FIG. 5B illustrates an exemplary personal electronic device (500). In some embodiments, the device (500) may include some or all of the components described with respect to FIGS. 1A, 1B, and 3. The device (500) has a bus (512) operatively coupling an I/O section (514) with one or more computer processors (516) and a memory (518). The I/O section (514) may be coupled to a display (504), which may have a touch-sensitive component (522), and optionally, an intensity sensor (524) (e.g., a contact intensity sensor). Additionally, the I/O section (514) may be coupled to a communications unit (530) to receive application and operating system data using Wi-Fi, Bluetooth, near field communication (NFC), cellular, and/or other wireless communication techniques. The device (500) may include input mechanisms (506 and/or 508). The input mechanism (506) is, optionally, a rotatable input device or, for example, a pressable and rotatable input device. In some examples, the input mechanism (508) is, optionally, a button.

In some examples, the input mechanism (508) is, optionally, a microphone. The personal electronic device (500) optionally includes various sensors, such as a GPS sensor (532), an accelerometer (534), an orientation sensor (540) (e.g., a compass), a gyroscope (536), a motion sensor (538), and/or combinations thereof, all of which may be operably connected to the I/O section (514).

The memory (518) of the personal electronic device (500) may include one or more non-transitory computer-readable storage media for storing computer-executable instructions that, when executed by, for example, one or more computer processors (516), cause the computer processors to perform the techniques described below, including the method (700) (FIG. 7). The computer-readable storage media may be any media that tangibly contains or can store computer-executable instructions for use by or in connection with an instruction execution system, apparatus, or device. In some examples, the storage media is a transitory computer-readable storage media. In some examples, the storage media is a non-transitory computer-readable storage media. The non-transitory computer-readable storage media may include, but are not limited to, magnetic, optical, and/or semiconductor storage. Examples of such storage include magnetic disks, optical disks based on CD, DVD, or Blu-ray technologies, as well as persistent solid-state memory such as flash, solid-state drives, and the like. The personal electronic device (500) is not limited to the components and configurations of FIG. 5b and may include other or additional components in a number of configurations.

As used herein, the term "affordance" refers to a user-interactive graphical user interface object that is optionally displayed on a display screen of the devices (100, 300, and/or 500) (FIGS. 1A, 3, and 5A and 5B). For example, an image (e.g., an icon), a button, and text (e.g., a hyperlink) each optionally constitute an affordance.

As used herein, the term "focus selector" refers to an input element that indicates the current portion of a user interface with which a user is interacting. In some implementations that include a cursor or other location marker, when input (e.g., a press input) is detected on a touch-sensitive surface (e.g., the touchpad (355) of FIG. 3 or the touch-sensitive surface (451) of FIG. 4B) while the cursor is over a particular user interface element (e.g., a button, a window, a slider, or other user interface element), the cursor functions as a "focus selector" such that the particular user interface element is adjusted in response to the detected input. In some implementations that include a touch screen display that enables direct interaction with user interface elements on the touch screen display (e.g., the touch-sensitive display system (112) of FIG. 1A or the touch screen (112) of FIG. 4A), when an input (e.g., a press-by-contact input) is detected on the touch screen display at a location of a particular user interface element (e.g., a button, a window, a slider, or other user interface element), the contact detected on the touch screen functions as a "focus selector" such that the particular user interface element is adjusted in response to the detected input. In some implementations, focus is moved from one area of the user interface to another area of the user interface without movement of a corresponding cursor or contact on the touch screen display (e.g., by moving focus from one button to another using the tab key or arrow keys); in these implementations, the focus selector moves in response to movement of focus between different areas of the user interface. Regardless of the particular form taken by the focus selector, a focus selector is typically a user interface element (or a contact on a touch screen display) that is controlled by the user to communicate the user's intended interaction with the user interface (e.g., by displaying on the device an element of the user interface with which the user wishes to interact). For example, the position of a focus selector (e.g., a cursor, contact, or selection box) over an individual button while a press input is detected on a touch-sensitive surface (e.g., a touchpad or touch screen) will indicate that the user intends to activate that individual button (as opposed to other user interface elements visible on the device's display).

As used in the specification and claims, the term "characteristic intensity" of a contact refers to a characteristic of a contact based on one or more intensities of the contact. In some embodiments, the characteristic intensity is based on a plurality of intensity samples. The characteristic intensity is optionally based on a predefined number of intensity samples, or a set of intensity samples collected over a predefined period of time (e.g., 0.05, 0.1, 0.2, 0.5, 1, 2, 5, 10 seconds) for a predefined event (e.g., after detecting the contact, before detecting liftoff of the contact, before or after detecting the start of movement of the contact, before or after detecting the end of the contact, before or after detecting an increase in intensity of the contact, and/or before or after detecting a decrease in intensity of the contact). The characteristic intensity of the contact is optionally based on one or more of a maximum value of the intensities of the contact, a mean value of the intensities of the contact, an average value of the intensities of the contact, a top 10 percentile value of the intensities of the contact, a value of half the maximum value of the intensities of the contact, a value of 90 percent of the maximum value of the intensities of the contact, etc. In some embodiments, the duration of the contact is used to determine the characteristic intensity (e.g., when the characteristic intensity is an average of the intensities of the contact over time). In some embodiments, the characteristic intensity is compared to a set of one or more intensity thresholds to determine whether an action has been performed by a user. For example, the set of one or more intensity thresholds optionally includes a first intensity threshold and a second intensity threshold. In these examples, a first action is performed as a result of a contact having a characteristic intensity that does not exceed a first threshold, a second action is performed as a result of a contact having a characteristic intensity that exceeds the first intensity threshold but does not exceed a second intensity threshold, and a third action is performed as a result of a contact having a characteristic intensity that exceeds the second threshold. In some embodiments, a comparison between the characteristic intensity and the one or more thresholds is used to determine whether to perform one or more actions (e.g., whether to perform an individual action or to refrain from performing an individual action), rather than to determine whether to perform the first action or the second action.

FIG. 5C illustrates detecting a plurality of contacts (552A-552E) on a touch-sensitive display screen (504) using a plurality of intensity sensors (524A-524D). FIG. 5C additionally includes intensity diagrams showing current intensity measurements of the intensity sensors (524A-524D) in terms of intensity units. In this example, the intensity measurements of the intensity sensors (524A, 524D) are each 9 intensity units, and the intensity measurements of the intensity sensors (524B, 524C) are each 7 intensity units. In some implementations, the total intensity is the sum of the intensity measurements of the plurality of intensity sensors (524A-524D), which in this example is 32 intensity units. In some embodiments, each contact is assigned an individual intensity that is a portion of the total intensity. FIG. 5d illustrates assigning total intensities to contacts (552A through 552E) based on their distance from a center of force (554). In this example, contacts (552A, 552B, 552E) are each assigned a contact intensity of eight intensity units of the total intensity, and contacts (552C, 552D) are each assigned a contact intensity of four intensity units of the total intensity. More generally, in some implementations, each contact (j) is assigned an individual intensity (Ij) that is a portion of the total intensity (A) according to a predefined mathematical function Ij = A (Dj/ΣDi), where Dj is the distance of the individual contact (j) to the center of force, and ΣDi is the sum of the distances of all the individual contacts to the center of force (e.g., i=1 to the last). The operations described with reference to FIGS. 5C and 5D can be performed using an electronic device similar or identical to the device (100, 300, or 500). In some embodiments, the characteristic intensity of the contact is based on one or more intensities of the contact. In some embodiments, intensity sensors are used to determine a single characteristic intensity (e.g., a single characteristic intensity of a single contact). It should be noted that the intensity diagrams are not part of the displayed user interface, but are included in FIGS. 5C and 5D to assist the reader.

In some embodiments, a portion of a gesture is identified for purposes of determining a characteristic intensity. For example, the touch-sensitive surface optionally receives a continuous swipe contact that transitions from a starting position to an ending position (at which point the intensity of the contact increases). In such an example, the characteristic intensity of the contact at the ending position is optionally based on only a portion of the continuous swipe contact (e.g., only a portion of the swipe contact at the ending position) rather than the entire swipe contact. In some embodiments, a smoothing algorithm is optionally applied to the intensities of the swipe contact prior to determining the characteristic intensity of the contact. For example, the smoothing algorithm optionally includes one or more of an unweighted sliding-average smoothing algorithm, a triangular smoothing algorithm, a median filter smoothing algorithm, and/or an exponential smoothing algorithm. In some situations, these smoothing algorithms remove narrow spikes or dips in the intensities of the swipe contact to determine the characteristic intensity.

The intensity of a contact on the touch-sensitive surface is optionally characterized for one or more intensity thresholds, such as a contact-detection intensity threshold, a light press intensity threshold, a deep press intensity threshold, and/or one or more other intensity thresholds. In some embodiments, the light press intensity threshold corresponds to an intensity at which the device would perform actions typically associated with clicking a button of a physical mouse or a trackpad. In some embodiments, the deep press intensity threshold corresponds to an intensity at which the device would perform actions that are different from actions typically associated with clicking a button of a physical mouse or a trackpad. In some embodiments, when a contact is detected with a characteristic intensity below the light press intensity threshold (e.g., and above a nominal contact detection intensity threshold below which the contact is no longer detected) the device will move the focus selector in response to movement of the contact on the touch-sensitive surface without performing an action associated with the light press intensity threshold or the deep press intensity threshold. In general, unless otherwise noted, these century thresholds are consistent across different sets of user interface drawings.

An increase in the characteristic intensity of a contact from below a light press intensity threshold to an intensity between the light press intensity threshold and the deep press intensity threshold is sometimes referred to as a "light press" input. An increase in the characteristic intensity of a contact from below a deep press intensity threshold to an intensity above the deep press intensity threshold is sometimes referred to as a "deep press" input. An increase in the characteristic intensity of a contact from below a contact detection intensity threshold to an intensity between the contact detection intensity threshold and the light press intensity threshold is sometimes referred to as detecting a contact on the touch surface. A decrease in the characteristic intensity of a contact from above the contact detection intensity threshold to below the contact detection intensity threshold is sometimes referred to as detecting a liftoff of the contact from the touch surface. In some embodiments, the contact detection intensity threshold is zero (0). In some embodiments, the contact detection intensity threshold is greater than zero.

In some embodiments described herein, one or more actions are performed in response to detecting a gesture comprising an individual press input or in response to detecting an individual press input performed with an individual contact (or multiple contacts), wherein the individual press input is detected based at least in part on detecting an increase in intensity of the contact (or multiple contacts) above a press-input intensity threshold. In some embodiments, the individual action is performed in response to detecting an increase in intensity of the individual contact above the press-input intensity threshold (e.g., a “down stroke” of the individual press input). In some embodiments, the press input comprises an increase in intensity of the individual contact above the press-input intensity threshold and a subsequent decrease in intensity of the contact below the press-input intensity threshold, and the individual action is performed in response to detecting a subsequent decrease in intensity of the individual contact below the press-input threshold (e.g., an “up stroke” of the individual press input).

FIGS. 5E-5H illustrate detection of a gesture including a press input corresponding to an increase in intensity of a contact (562) from an intensity below a light press intensity threshold (e.g., "IT _L ") of FIG. 5E to an intensity above a deep press intensity threshold (e.g., "IT _D ") of FIG. 5H . A gesture performed using a contact (562) is detected on a touch-sensitive surface (560) while a cursor (576) is displayed over an application icon (572B) corresponding to App 2 on a displayed user interface (570) including application icons (572A-572D) displayed in a predefined region (574). In some embodiments, the gesture is detected on the touch-sensitive display (504). Intensity sensors detect the intensity of contacts on the touch-sensitive surface (560). The device determines that the intensity of the contact (562) peaks at a deep press intensity threshold (e.g., "IT _D "). The contact (562) is maintained on the touch-sensitive surface (560). In response to detecting the gesture, and in response to the contact (562) having an intensity exceeding the deep press intensity threshold (e.g., "IT _D ") during the gesture, scaled representations (578A-578C) (e.g., thumbnails) of recently opened documents for App 2 are displayed, as illustrated in FIGS. 5F-5H . In some embodiments, the intensity compared to one or more intensity thresholds is a characteristic intensity of the contact. It should be noted that the intensity diagram for the contact (562) is not part of the displayed user interface, but is included in FIGS. 5E-5H to assist the reader.

In some embodiments, the display of representations (578A through 578C) includes an animation. For example, as illustrated in FIG. 5F , representation (578A) is initially displayed proximate to application icon (572B). As the animation progresses, representation (578A) moves up and representation (578B) is displayed proximate to application icon (572B), as illustrated in FIG. 5G . Subsequently, as illustrated in FIG. 5H , representation (578A) moves up, representation (578B) moves up toward representation (578A), and representation (578C) is displayed proximate to application icon (572B). Representations (578A through 578C) form an array over icon (572B). In some embodiments, as illustrated in FIGS. 5F and 5G , the animation progresses based on the intensity of the contact (562), with representations (578A through 578C) appearing and moving upward as the intensity of the contact (562) increases toward a deep press intensity threshold (e.g., “IT _D ”). In some embodiments, the intensity on which the animation progresses is based is a characteristic intensity of the contact. The operations described with reference to FIGS. 5E through 5H can be performed using an electronic device similar to or identical to the device (100, 300, or 500).

In some embodiments, the device utilizes intensity hysteresis to avoid accidental inputs, sometimes referred to as "jitter", wherein the device defines or selects a hysteresis intensity threshold having a predefined relationship to a press-input intensity threshold (e.g., the hysteresis intensity threshold is X intensity units lower than the press-input intensity threshold, or the hysteresis intensity threshold is 75%, 90%, or some suitable fraction of the press-input intensity threshold). Thus, in some embodiments, a press input comprises an increase in intensity of an individual contact above the press-input intensity threshold and a subsequent decrease in intensity of the contact below the hysteresis intensity threshold corresponding to the press-input intensity threshold, and the individual action is performed in response to detecting a subsequent decrease in intensity of the individual contact below the hysteresis intensity threshold (e.g., an "upstroke" of the individual press input). Similarly, in some embodiments, a press input is detected only when the device detects an increase in intensity of the contact from an intensity below a hysteresis intensity threshold to an intensity above the press-input intensity threshold, and optionally, a subsequent decrease in intensity of the contact to an intensity below the hysteresis intensity, and the respective action is performed in response to detecting the press input (e.g., an increase in intensity of the contact or a decrease in intensity of the contact depending on the environment).

For convenience of explanation, the description of actions performed in response to a press input associated with a press input intensity threshold or in response to a gesture including a press input is optionally triggered in response to detecting any one of: an increase in intensity of the contact exceeding the press input intensity threshold, an increase in intensity of the contact from an intensity below a hysteresis intensity threshold to an intensity above the press input intensity threshold, a decrease in intensity of the contact below the press input intensity threshold, and/or a decrease in intensity of the contact below a hysteresis intensity threshold corresponding to the press input intensity threshold. Additionally, in examples where the action is described as being performed in response to detecting a decrease in intensity of the contact below the press input intensity threshold, the action is optionally performed in response to detecting a decrease in intensity of the contact below a hysteresis intensity threshold corresponding to and lower than the press input intensity threshold.

As used herein, an "installed application" refers to a software application that has been downloaded onto an electronic device (e.g., devices (100, 300, and/or 500)) and is ready to be launched (e.g., opened) on the device. In some embodiments, a downloaded application is an application that is installed via an installer program that extracts program portions from a downloaded package and integrates the extracted portions with the operating system of a computer system.

As used herein, the terms "open application" or "running application" refer to a software application that has state information maintained (e.g., as part of the device/global internal state (157) and/or the application internal state (192)). An open or running application is, optionally, any of the following application types:

The active application currently displayed on the display screen of the device on which the application is being used;

A background application (or background process) that is not currently displayed, but in which one or more processes for the application are being processed by one or more processors; and

A suspended or hibernated application that is not running, but has state information that is stored in memory (volatile and non-volatile, respectively) and can be used to resume execution of the application.

As used herein, the term "closed application" refers to software applications that do not have any retained state information (e.g., no state information is stored in the memory of the device for closed applications). Thus, closing an application includes stopping and/or removing application processes for the application, and removing state information for the application from the memory of the device. In general, opening a second application while in a first application does not close the first application. When the second application is displayed and the first application ceases to be displayed, the first application becomes a background application.

Now, attention is drawn to embodiments of user interfaces (“UIs”) and associated processes implemented on an electronic device, such as a portable multifunction device (100), device (300), or device (500).

User interfaces and associated processes

Customizing elements of graphic objects

Users interact with electronic devices in a variety of ways, including entering text into the electronic device. In some embodiments, the electronic device provides a virtual keyboard (e.g., a soft keyboard) that mimics the layout of a physical keyboard and allows the user to select characters (e.g., letters) to enter. In some embodiments, the virtual keyboard includes an emoji keyboard for selecting and inserting emojis (and other graphical objects). In some embodiments, the device provides user interfaces for customizing and inserting emojis with a number of customizable elements, thereby enhancing user interactions with the device. Enhancing interactions with the device reduces the time required by the user to perform actions, thereby reducing power usage of the device and increasing battery life for battery-operated devices. It is understood that people use the devices. When a person uses the device, the person is, optionally, referred to as a user of the device.

FIGS. 6A through 6XX illustrate exemplary ways in which an electronic device presents a customized user interface for customizing and inserting graphical objects, such as emojis. The embodiments in these drawings are used to illustrate the processes described below, including the processes described with reference to FIGS. 7A through 7M.

FIG. 6A illustrates an exemplary device (500) including a touch screen (504). As shown in FIG. 6A, the electronic device (500) displays a user interface (600). In some embodiments, the user interface (600) is any user interface that includes one or more text input fields (e.g., text input zones). In some embodiments, a text input field (e.g., a text input zone) is a user interface element into which a user can enter text (e.g., letters, characters, words, etc.). For example, a text input field may be a text field within an application, a text field on a form, a URL input element within a browser, a login field, etc. It is to be understood that a text input field (e.g., a text input zone) is not limited to a user interface element that accepts only text, but may also accept and display audio and/or visual media and/or any other type of visual or graphical object.

In FIG. 6A , a user interface (600) is of a text messaging application displaying a text message conversation. In some embodiments, the user interface (600) includes a conversation area (602), a text input area (604), and a virtual keyboard (606). In some embodiments, the virtual keyboard (606) is displayed in response to user input requesting display of the virtual keyboard (606). In some embodiments, the virtual keyboard (606) is a user interface element having a plurality of selectable options for inserting objects (e.g., text, emojis, images, etc.) into the text input area (604). In FIG. 6A , the virtual keyboard (606) presents an emoji selection user interface (hereinafter referred to as the emoji keyboard (606)) that includes one or more selectable options for inserting emojis into the text input area (604). In some embodiments, an emoji is a graphical object. In some embodiments, emojis are images, icons, symbols, or ideograms that represent people, places, animals, objects, flags, signs, and the like. In some embodiments, emojis are recognizable as text characters, for example, by the Unicode standard (e.g., Unicode 6.0 or later). In some embodiments, emojis can be inserted into a text input area and transmitted to other electronic devices. In some embodiments, the emoji keyboard (606) includes smiley emojis, thumbs up and thumbs down emojis, person emojis, group emojis, family emojis, and the like. In some embodiments, the available emojis are organized into one or more categories. In some embodiments, the emoji keyboard (606) displays each category on a different page or tab. In some embodiments, one or more selectable options (610) are displayed on the emoji keyboard (606) for navigating to individual pages or tabs.

In some embodiments, certain emojis on the emoji keyboard (606) are customizable. In some embodiments, certain customizable emojis have one customizable element. For example, a single human emoji is optionally customizable to change the skin tone of the person. In some embodiments, certain customizable emojis have multiple customizable elements. For example, a group emoji (e.g., a two-person emoji) or a family emoji (e.g., a three-person or four-person emoji) is optionally customizable to change the skin tone of each of the people represented in the emoji. In some embodiments, other customizations are possible. In some embodiments, certain customizable emojis have one or more accessories. For example, an emoji of people is optionally customizable to add a hat, jewelry, etc. In general, some emojis optionally include multiple customizable visual elements (e.g., colors, visual elements such as objects, etc.) that are customizable using the user interfaces of the present disclosure.

As illustrated in FIG. 6A, the emoji keyboard (606) includes eight emojis (608-1 to 608-8). Emojis (608-1, 608-2) correspond to smile and frown emojis and do not have customizable elements. Emojis (608-3, 608-4) correspond to male character and female character emojis and have a single customizable element (e.g., a person represented in the emoji). Emojis (608-5, 608-6) correspond to female-male groups and female-female groups and have two customizable elements (e.g., two people represented in the emoji). The emoji keyboard (606) optionally includes other emojis, such as emojis (608-7, 608-8), and is optionally scrollable to reveal other emojis.

It is to be understood that the emojis illustrated herein are merely exemplary, and that emojis other than those illustrated and described herein may have no customizable elements or may have any of one or more customizable elements. It is to be understood that the examples illustrated in FIGS. 6A-6XX are exemplary, and should not be considered limiting to the illustrated user interfaces and/or applications.

In FIG. 6b, a user input (603) selecting an emoji (608-1) corresponding to a smiley emoji is received (e.g., detected) on the touch screen (504). In some embodiments, the user input (603) selecting the emoji (608-1) corresponds to a request to insert the emoji (608-1) into the text input area (604). In some embodiments, the user input (603) is a tap input. For example, the user input (603) illustrated in FIG. 6b is a contact for less than a predetermined threshold (e.g., 0.1 second, 0.25 seconds, 0.5 seconds, 1 second, 3 seconds, 5 seconds, etc.), as illustrated by the timer (612). In some embodiments, the emoji (608-1) does not have any customizable elements. Accordingly, in some embodiments, in response to a user input, the device (500) inserts an emoji (608-1) into the text input area (604), as illustrated in FIG. 6C . In some embodiments, in response to the text input area receiving the inserted character (e.g., the emoji (608-1)), a selectable transmission option (616) is displayed to transmit (e.g., send) the text in the text input area (604) (e.g., as a text message).

In FIG. 6C , a user input (603) is received on the touch screen (504) selecting an emoji (608-4) corresponding to a female character emoji. As described above, the emoji (608-4) has a single customizable element. In the embodiment illustrated in FIG. 6C , the emoji (608-4) has not previously been customized (e.g., the user has not previously selected a variation of the customizable element). In some embodiments, the user input (603) selecting the emoji (608-4) corresponds to a request to customize the emoji (608-4) (e.g., because the emoji has not yet been customized). In some embodiments, the user input (603) selecting the emoji (608-4) corresponds to a request to insert the emoji (608-4) into the text input area (604). In some embodiments, the user input (603) is a tap input. For example, the user input (603) illustrated in FIG. 6c is a contact for less than a predetermined threshold (e.g., 0.1 second, 0.25 seconds, 0.5 seconds, 1 second, 3 seconds, 5 seconds, etc.), as illustrated by the timer (612). In some embodiments, in response to the user input (603), the device (500) displays a single-element customization user interface (618), as illustrated in FIG. 6d, because the emoji (608-4) has not yet been customized by the user. In some embodiments, the single-element customization user interface (618) is displayed in response to a tap input only if the user has not previously customized the emoji. In some embodiments, the single-element customization user interface (618) is always displayed in response to a tap input, regardless of whether the user has previously customized the emoji. In some embodiments, the single-element customization user interface (618) is displayed only in response to user input having a contact duration longer than a predetermined threshold (e.g., a “long press” or “tap and hold” input).

As illustrated in FIG. 6d , the single-element customization user interface (618) is a user interface element that includes one or more representations of one or more variations of the emoji (608-4). For example, the customizable element of the emoji (608-4) is the skin tone of the female character in the emoji (608-4). As illustrated in FIG. 6d , the single-element customization user interface (618) includes variations (620, and 622-1 through 622-3). In some embodiments, variation (620) corresponds to a default variation of the emoji (608-4) (e.g., an emoji having a default color that does not represent any particular skin tone, such as yellow). In some embodiments, variations (622-1 through 622-3) correspond to customizable skin tones of the emoji (608-4). In some embodiments, the single-element customization user interface (618) includes a visual delineation between the default variant (620) and the custom variants (622-1 through 622-3). In some embodiments, the default variant (620) is displayed as visually separated from the custom variants (622-1 through 622-3). In some embodiments, the variants (620 and 622-1 through 622-3) are selectable to both: 1) select an individual variant; and 2) insert an individual variant into the text input field (618). For example, in FIG. 6d , user input (603) is received selecting (e.g., tapping) a variant (622-2). In some embodiments, in response to user input (603) selecting a variant (622-2), the device (500) inserts the variant (622-2) of the emoji (608-4) as a character (614-2) into the text input field (604), as illustrated in FIG. 6E . In addition to inserting the variant (622-2) into the text input field (604), as illustrated in FIG. 6E , the device (500) updates the emoji keyboard (606) such that the emoji (608-4) is displayed as the selected variant (622-2).

FIGS. 6E-6EE illustrate methods of customizing and inserting emojis having multiple customizable elements. In FIG. 6E, a user input (603) is received selecting (e.g., tapping) an emoji (608-6) corresponding to a female-female group emoji. In some embodiments, the emoji (608-6) has two customizable elements. For example, the emoji (608-6) is a representation of two people (e.g., a group emoji). In some embodiments, the skin tone of each person in the emoji (608-6) is customizable. In the embodiment illustrated in FIG. 6E, the emoji (608-6) has not previously been customized (e.g., the user has not previously selected an option or variation for any of the customizable elements). In some embodiments, the user input (603) selecting an emoji (608-6) corresponds to a request to customize the emoji (608-6) (e.g., because the emoji (608-6) has not yet been customized). In some embodiments, the user input (603) selecting an emoji (608-6) corresponds to a request to insert the emoji (608-6) into the text input area (604). In some embodiments, the user input (603) is a tap input. For example, the user input (603) illustrated in FIG. 6E is a contact for less than a predetermined threshold (e.g., 0.1 seconds, 0.25 seconds, 0.5 seconds, 1 second, 3 seconds, 5 seconds, etc.), as illustrated by the timer (612). In some embodiments, in response to user input (603), the device (500) displays a multi-element customization user interface (624), as illustrated in FIG. 6F. In some embodiments, the multi-element customization user interface (624) is displayed in response to a tap input only if the user has not yet completed customizing the emoji. In some embodiments, the multi-element customization user interface (624) is always displayed in response to a tap input, regardless of whether the user has previously customized the emoji. In some embodiments, the multi-element customization user interface (624) is only displayed in response to a user input having a contact duration longer than a predetermined threshold.

As illustrated in FIG. 6F , the multi-element customization user interface (624) includes two rows of variations for customizing the emoji (608-6). In some embodiments, the first row includes variations (626-1 through 626-4) of the emoji (608-6) that correspond to variations of a first customizable element (e.g., the person on the left side of the emoji). In some embodiments, the second row includes variations (628-1 through 628-4) of the emoji (608-6) that correspond to a second customizable element (e.g., the person on the right side of the emoji). For example, the first customizable element for the emoji (608-6) is the skin tone of the left person in the emoji (608-6), and the second customizable element for the emoji (608-6) is the skin tone of the right person in the emoji (608-6). In some embodiments, the variations within the first and second rows of the multi-element customization user interface (624) display each of the customizable elements (e.g., displaying the entire emoji without cropping away portions of the emoji that are not customized by the individual rows). In some embodiments, the representations of the variations within the first row are displayed as individual variations for the left person (e.g., each selectable option displays the left person with a different skin tone). In some embodiments, the representations within the first row are displayed with the right person having a placeholder appearance (e.g., grayed out) to indicate that the first row is not intended for customizing the right person (e.g., the right person is not selectable to customize). In some embodiments, the representations within the second row are displayed as individual variations for the right person (e.g., each selectable option displays the right person with a different skin tone). In some embodiments, the expressions in the second row are displayed with the left person having a placeholder appearance (e.g., grayed out) to indicate that the second row is not intended for customizing the left person (e.g., is not selectable to customize the left person). In some embodiments, any of the transformation options for the left and right people described above may be a default transformation described below (e.g., a yellow coloring corresponding to the default coloration of emojis). In some embodiments, none of the transformation options described above are default transformations (e.g., they are not listed as options in the rows and are not selectable as a customization for either the left or right person). In such embodiments, the default transformation is not an option for customizing the selectable elements. In such embodiments, a user cannot select a default transformation for either of the selectable elements (e.g., a default transformation for one element and a skin tone transformation for another element, or a default transformation for both elements), and the user can only select a skin tone transformation for the selectable elements. As described herein, the default variation (e.g., a yellow tint corresponding to the default tint of the emoji) is not a skin tone variation (such as those selectable to customize the emoji), but is merely the default tint of the emoji and does not represent a particular skin tone.

In some embodiments, the third row includes a default variation (630-1) for emoji (608-6). For example, the default variation (630-1) has a default coloring for both the left and right person (e.g., the default coloring for emojis that do not express any particular skin tone, such as yellow). In some embodiments, the third row includes a customized variation (630-2) for emoji (608-6). In some embodiments, if emoji (608-6) did not have any customizations (e.g., no options from the first and second rows were selected), the customized variation (630-2) is displayed with placeholder elements for both people (e.g., both customizable elements). For example, in FIG. 6F , both people in the customized variation (630-2) are displayed with a placeholder appearance (e.g., grayed out) to indicate that no customizations were selected. As described in more detail below, the customized variation (630-2) may be optionally inserted into a text input field when customization is complete.

In FIG. 6f, a user input (603) is received selecting (e.g., tapping) a default variation (630-1). In some embodiments, in response to the user input (603), the device (500) inserts the default variation (630-1) of the emoji (608-6) into the text input area (604), as illustrated in FIG. 6g. As illustrated in FIG. 6g, the emoji (608-6) on the emoji keyboard (606) is displayed as the default variation (630-1) because the variation (630-1) is the variation that the user most recently selected and/or inserted into the text input area (604).

In FIG. 6g, a user input (603) is received selecting an emoji (608-6) on an emoji keyboard (606). In some embodiments, the user input (603) is a tap input. For example, the user input (603) illustrated in FIG. 6g is a contact for less than a predetermined threshold (e.g., 0.1 second, 0.25 second, 0.5 second, 1 second, 3 seconds, 5 seconds, etc.), as illustrated by a timer (612). In some embodiments, in response to the user input (603), the device (500) displays a multi-element personalized user interface (624), as illustrated in FIG. 6h. In some embodiments, the emoji (608-6) is not fully customized because variations for each of the customizable elements have not yet been selected, and thus, selection of the emoji (608-6) in the emoji keyboard (606) causes display of the multi-element customization user interface (624).

As illustrated in FIG. 6h , the multi-element customization user interface (624) retains the previous state of the customized variation (630-2). For example, both people (e.g., both customizable elements) within the customized variation (630-2) have a placeholder appearance (e.g., grayed out, grayed out, etc.) to indicate that no customizations have yet been selected. In FIG. 6h , a user input (603) is received selecting (e.g., tapping) the customized variation (630-2). In some embodiments, because the emoji (608-6) is not customized, selecting the customized expression (630-2) does not result in insertion of the customized variation (630-2) into the text input field (604). In some embodiments, in response to user input (603), the device (500) displays an indicia (632) to indicate that individual customizable elements have not yet been customized and that selection of individual customizable elements is required to insert a customized expression into the text input field, as illustrated in FIG. 6i . For example, as illustrated in FIG. 6i , the indicia (632) is an animation. In some embodiments, the animation comprises momentarily increasing the size of the selectable options for each customizable variation that has not yet been customized. In some embodiments, the animation is at least partially staggered such that the animation begins at one end and moves to the other end (e.g., similar to a wave). For example, variation (626-1) first increases in size, and as variation (626-1) decreases in size, variation (626-2) begins to increase in size, and so on. In some embodiments, each of the variations within the first and second rows is animated. In some embodiments, the animation is performed simultaneously on the first and second rows (e.g., the wave moves down the first row of transformations at the same time that the wave moves down the second row of transformations). As described in more detail below, the row(s) corresponding to customizable transformations that require customization (e.g., have not yet been customized) are animated, while the row(s) corresponding to customizable transformations that have already been customized are not animated.

Thus, in some embodiments, until customization of all of the customizable elements of the emoji (608-6) is complete, the customized variation (630-2) is not selectable to insert the customized variation of the emoji (608-6) into the text input area. In some embodiments, the default customization is not selected for the customized variation (630-2) because the customization is optionally of a skin tone of the person within the emoji (608-6), and the default skin tone for the customized variation (630-2) is not selected by the manufacturer (or provider) of the electronic device. In some embodiments, it is desirable to allow the user to make initial selections of the skin tone of each person in the customizable emoji. Thus, in some embodiments, using placeholder elements instead of selecting default skin tones for the customizable elements maintains a neutral position with respect to the available skin tone options. In this way, the customized variant (630-2) actually reflects the user's customizations and preferences without any bias (implicit or not).

In FIG. 6j , a user input (603) is received selecting (e.g., tapping) a variant (626-1). In some embodiments, the variant (626-1) corresponds to a customizable option for the left person of emoji (608-6). In some embodiments, the variant (626-1) is a customized variant for the left person of emoji (e.g., a customizable skin tone for the left person). In some embodiments, a default variant (e.g., a yellow tint that does not represent a skin tone) is not provided as an option. In some embodiments, in response to the user input (603) selecting the variant (626-1), the variant (626-1) is updated with a visual indication that the variant (626-1) is selected (e.g., as shown in dashed lines), as illustrated in FIG. 6k . In some embodiments, in response to user input (603) selecting a variant (626-1), the customized variant (630-2) is updated to reflect the user's selection of the variant (626-1), as illustrated in FIG. 6K . In some embodiments, the visual indication includes highlighting the variant (e.g., changing the background color of the box around the emoji to blue, yellow, red, etc.). In some embodiments, other visual characteristics or indications are possible. As illustrated in FIG. 6K , the customized variant (630-2) is updated such that the left person in the customized variant (630-2) has the selected skin tone of variant (626-1), while the right person remains as a placeholder element (e.g., has a placeholder appearance, such as being grayed out) because the right person has not yet been customized by the user.

In FIG. 6k , a user input (603) is received selecting (e.g., tapping) a customized variation (630-2). In some embodiments, because the emoji (608-6) is not yet fully customized (e.g., only the left person is customized, but the right person is not yet customized), selecting the customized variation (630-2) does not result in insertion of the customized variation (630-2) into the text input field (604). In some embodiments, in response to the user input (603), the device (500) displays an indicia (632) to indicate that the right-hand customizable element is not yet customized, and that selection of an option for the right-hand customizable element is required to enable insertion of the customized expression into the text input field, as illustrated in FIG. 6l . In some embodiments, the indicia (632) is similar to the indicia (632) described above with respect to FIG. 6i . In some embodiments, only the variants within the second row will display the indication (632) because the second row corresponds to a customizable element that has not yet been customized. Similarly, if the device (500) receives user input selecting a customized variant (630-2) and the customizable element associated with the first row has not yet been customized, the variants within the first row will display the indication.

In FIG. 6m, user input (603) (e.g., a tap) is received on a user interface (600) outside of a multi-element customization user interface (624). In some embodiments, the user input corresponds to a request to dismiss the multi-element customization user interface (624). In some embodiments, in response to the user input (603), the multi-element customization user interface (624) is dismissed (e.g., stops being displayed), as illustrated in FIG. 6n. In some embodiments, the device (500) displays a user interface that was displayed prior to the multi-element customization user interface (624) being displayed (e.g., the entire user interface (600). As illustrated in FIG. 6n, the emoji (608-6) on the emoji keyboard (606) does not change (e.g., continues to display the default variation) because the user did not complete the customization and/or the user did not select a customized variation (630-2), even if the user performed a partial customization. In some embodiments, user input on the paddle portion of the customization user interface (624) (e.g., the portion of the multi-element customization user interface (624) that is overlaid on the emoji (608-6)) may also be interpreted as a request to dismiss the customization user interface, causing the multi-element customization user interface (624) to be dismissed.

In FIG. 6n, a user input (603) is received selecting (e.g., tapping) an emoji (608-6). In some embodiments, because the emoji (608-6) is not fully customized (e.g., a variation for the left person has been selected, but a variation for the right person has not yet been selected), in response to the user input (603), a multi-element customization user interface (624) is displayed as illustrated in FIG. 6o. In some embodiments, the multi-element customization user interface (624) is always displayed in response to a user input selecting an emoji (608-6), regardless of whether the emoji (608-6) is fully customized or not fully customized.

In FIG. 6o, a user input (603) is received that selects (e.g., taps) a variant (628-3) corresponding to a variant for the right person of emoji (608-6). As described above, the variant (628-3) is a skin tone variant for the right person of emoji (608-6). In some embodiments, in response to the user input (603), the customized variant (630-2) is updated such that the right person is displayed with visual characteristics associated with the variant (628-3) (e.g., the skin tone of the variant (628-3)), as illustrated in FIG. 6p. In some embodiments, the user's selection of the variant (628-3) completes the customization of the emoji (608-6) (e.g., a variant for each customizable element has been selected). Accordingly, in some embodiments, in response to completing the customization, an indicia (634) is displayed on the customized variant (630-2) to indicate that the customization of the variant (630-2) is complete and that the variant (630-2) is now selectable for insertion into the text input area. In some embodiments, the indicia (634) is a graphical animation. In some embodiments, the animation includes temporarily increasing and decreasing the size of the variant (630-2), similar to the indicia (632) described above with reference to FIGS. 6I and 6L.

In FIG. 6q, a user input (603) is received selecting (e.g., tapping) a customized variation (630-2). In some embodiments, since the customization of the emoji (608-6) is complete, in response to the user input (603), the customized variation (630-2) is inserted into the text input area (604) as a character (614-4), as illustrated in FIG. 6r. In FIG. 6r, since the customized variation (630-2) is the user's most recently selected variation of the emoji (608-6) for insertion into the text input field (604), the emoji (608-6) on the emoji keyboard (606) is updated to display the customized variation (630-2).

In FIG. 6r , a user input (603) is received selecting (e.g., tapping) the emoji (608-6) while it is displayed as a customized variation (630-2). In some embodiments, in response to the user input (603) selecting the emoji (608-6) while it is displayed as a customized variation (630-2), because the customization of the emoji (608-6) is complete (e.g., because the variation (630-2) is fully customized), the device (500) inserts the customized variation (630-2) as a character (614-5) into the text input area (604), as illustrated in FIG. 6s (e.g., the emoji (608-6) having variations of the first and second customizable elements implemented by the customized variation (630-2) is inserted). Thus, in some embodiments, since the emoji (608-6) is fully customized, a user input of tapping on the emoji (608-6) on the emoji keyboard (606) does not display the multi-element customization user interface (624). In some embodiments, tapping on the emoji (608-6) results in insertion of the most recently selected variation of the emoji (608-6) (e.g., and optionally, a variation displayed on the emoji keyboard (606).

In FIG. 6t , a touchdown of a contact (603) is received on the emoji (608-6) while the emoji (608-6) is displayed as a customized variation (630-2). In FIG. 6u , the contact (603) on the emoji (608-6) is held for a predetermined threshold duration (e.g., 0.1 seconds, 0.25 seconds, 0.5 seconds, 1 second, 3 seconds, 5 seconds, etc.), as indicated by the timer (612). As illustrated in FIG. 6u , in response to detecting that the contact (603) has been held for a predetermined threshold duration, a multi-element customization user interface (624) is displayed. In some embodiments, the multi-element customization user interface (624) displays a current customization state for the emoji (608-6). For example, customization selections for each of the customizable elements are maintained. In FIG. 6u, the variants (626-1, 628-3) are highlighted, indicating that the variants (626-1, 628-3) are selected and are currently applied to the customized variant (630-2). In FIG. 6v, a liftoff of the contact (603) from the touch screen (504) is detected. In some embodiments, in response to detecting a liftoff of the contact (603), the device (500) maintains the display of the multi-element customization user interface (624) (e.g., holds off dismissing the multi-element customization user interface (624)), as illustrated in FIG. 6v.

In FIG. 6w , a user input (603) is received selecting (e.g., tapping) a variant (626-2) corresponding to a second skin tone variant for the left person of the emoji (608-6). In some embodiments, in response to the user input (603), the customized variant (630-2) is updated to reflect the skin tone variant of the left person in the variant (626-2), as illustrated in FIG. 6x (e.g., the previous selection of variant (626-1) is replaced with variant (626-2). As illustrated in FIG. 6x , variant (626-2) is displayed as highlighted, and variant (626-1) is no longer displayed as highlighted. Thus, in some embodiments, the user may update the customized variant (630-2) by selecting other variants for the customized elements and overriding the previous selection. In some embodiments, after updating the customized variant (630-2) with new variant selections, if the user dismisses the multi-element customization user interface (624) while the emoji (608-6) is displaying the customized variant (630-2), the emoji (608-6) on the emoji keyboard (606) is also updated to display the updated customized variant (630-2) (e.g., as a left person variant (626-2) and as a right person variant (628-3)). In some embodiments, after updating the customized variant (630-2) with new variant selections, if the user dismisses the multi-element customization user interface (624) while the emoji (608-6) is displaying the customized variant (630-2), optionally, the emoji (608-6) on the emoji keyboard (606) is not updated with the updated customized variant (630-2) because the user did not select the updated customized variant (630-2) for insertion into the text input area from the multi-element customization user interface (624), but rather retains its previous customization (e.g., the non-updated customized variant (630-2)).

In FIG. 6x , a user input (603) is received selecting (e.g., tapping) a customized variant (630-2) with updated variant selections (e.g., variant 626-2 as a left person and variant 628-3 as a right person). In some embodiments, in response to the user input (603), the updated customized variant (630-2) is inserted into the text input area (604) as a character (614-6), as illustrated in FIG. 6y . As illustrated in FIG. 6y , the emoji (608-6) on the emoji keyboard (606) is updated to reflect the updated customized variant (630-2).

In FIG. 6z, a touchdown of the contact (603) is detected on the emoji (608-6). In FIG. 6aa, the contact (603) on the emoji (608-6) is maintained for more than a predetermined threshold duration (e.g., 0.1 seconds, 0.25 seconds, 0.5 seconds, 1 second, 3 seconds, 5 seconds, etc.), as indicated by the timer (612). In some embodiments, in response to detecting that the contact (603) on the emoji (608-6) is maintained for more than the predetermined threshold duration, as illustrated in FIG. 6aa, a multi-element customization user interface (624) is displayed. In FIG. 6bb, a contact (603) moving to the default variant (630-1) while maintaining contact with the touch screen (504) is detected (e.g., a swipe gesture to the default variant (630-1)). In some embodiments, a liftoff of the contact (603) following a swipe gesture that moves the contact (603) to the default variant (630-1) is interpreted as a selection input of the default variant (630-1). In some embodiments, in response to detecting the gesture that moves the contact (603) to the default variant (630-1) and the liftoff of the contact (603) while positioned on the default variant (630-1), the default variant (630-1) is inserted into the text input area (604) as a character (614-7), as illustrated in FIG. 6cc. In some embodiments, the emoji (608-6) on the emoji keyboard (606) is updated to display the default variant (630-1) (e.g., as the most recently selected variant of the emoji (608-6)).

In FIG. 6dd, a user input (603) is received selecting (e.g., tapping) the emoji (608-6) while the emoji (608-6) is displaying a default variation (630-1) (e.g., default emoji coloring for both the left and right person, e.g., yellow). In some embodiments, in response to the user input (603), the default variation (630-1) is inserted into the text input area (604) as a character (614-8), as illustrated in FIG. 6ee. Thus, in some embodiments, after the emoji (608-6) is fully customized, user input selecting the emoji (608-6) (e.g., the user has previously selected a variant for each of the customizable elements of the emoji (608-6), independent of whether the current display state of the emoji (608-6) is the default variant or a customized variant) will cause the currently displayed variant (e.g., either the default variant or the customized variant) to be inserted into the text input area. As illustrated above, after the customization for an individual emoji is complete, the multi-element customization user interface (624) is optionally displayed for that individual emoji only in response to a long-press user input (e.g., a contact for more than a predetermined duration threshold). It is understood that each emoji on the emoji keyboard may maintain its own customized state. Accordingly, whether an individual emoji is partially or fully customized is specific and unique to the individual emoji, and the behavior of the customized user interface is also specific and unique to the individual emoji. For example, completing customization of a first emoji with a plurality of customizable elements does not change the customized state of the second emoji with a plurality of customizable elements, such that if the second emoji was not customized when the user customized the first emoji, the customized user interface for the second emoji would still be displayed in response to a tap on the second emoji, including placeholder elements on the customized variation of the second emoji.

In some embodiments, other behaviors are possible. For example, in some embodiments, the multi-element customization user interface is always displayed in response to user input (e.g., a tap), regardless of the customization state of individual emojis. In other embodiments, the multi-element customization user interface is displayed only in response to a long-press user input, regardless of the customization state of individual emojis, and selection (e.g., a tap) of an emoji having multiple customizable elements causes insertion of the default variation until the user defines and inserts different variations (e.g., via the multi-element customization user interface).

FIGS. 6ff-6nn illustrate methods of customizing and inserting emojis having only one customizable element. In FIG. 6ff, a user input (603) is detected selecting an emoji (608-4) while the emoji (608-4) is displayed as a variation (622-2) (e.g., a previously selected variation of the emoji (608-4)). In some embodiments, the user input (603) is a contact for less than a predetermined threshold (e.g., 0.1 seconds, 0.25 seconds, 0.5 seconds, 1 second, 3 seconds, 5 seconds, etc.), as exemplified by a timer (612). In some embodiments, in response to user input (603), the device (500) inserts a variation (622-2) of emoji (608-4) (e.g., a previously selected variation of emoji (608-4)) as a character (614-9) into the text input field (604), as illustrated in FIG. 6gg.

In FIG. 6hh, a touchdown of a contact (603) is detected on the emoji (608-4) while the emoji (608-4) is displayed as a variant (622-2). In FIG. 6ii, the contact (603) on the emoji (608-4) is held for a predetermined threshold duration (e.g., 0.1 seconds, 0.25 seconds, 0.5 seconds, 1 second, 3 seconds, 5 seconds, etc.), as indicated by the timer (612). In some embodiments, in response to detecting that the contact (603) is held for a predetermined threshold duration, as illustrated in FIG. 6ii, a single-element customization user interface (618) is displayed. In FIG. 6jj , a lift-off of the contact (603) from the touch screen (504) is detected while the contact (603) is not positioned over any of the variations (620, 622-1, 622-2, or 622-3). In some embodiments, in response to detecting a lift-off of the contact (603), the device (500) dismisses the display of the single-element customization user interface (618), as illustrated in FIG. 6jj , and inserts the variation (622-2) of the emoji (608-4) (e.g., the previously selected variation of the emoji (608-4)) as the character (614-10) into the text input field (604). Thus, in some embodiments, the lift-off of the contact (603) does not cause the display of the single-element customization user interface (618) to remain.

FIGS. 6kk-6ll illustrate a touchdown of a contact (603) on an emoji (608-4) for greater than a predetermined threshold duration (e.g., 0.1 second, 0.25 second, 0.5 second, 1 second, 3 seconds, 5 seconds, etc.) that causes display of a single-element customization user interface (618). In FIG. 6mm, a contact (603) is detected that moves to a variant (622-1) while maintaining contact with the touch screen (504) (e.g., a swipe gesture to variant (622-1)). In some embodiments, a liftoff of the contact (603) following a swipe gesture that moves the contact (603) to variant (622-1) is interpreted as a selection input of the variant (622-1). In some embodiments, in response to a gesture moving the contact (603) to the variant (622-1), and detecting liftoff of the contact (603) while positioned over the variant (622-1), the variant (622-1) is inserted into the text input area (604) as a character (614-11), as illustrated in FIG. 6nn. In some embodiments, the emoji (608-4) on the emoji keyboard (606) is updated to display the variant (622-1) (e.g., as the most recently selected variant of emoji (608-4)).

In FIG. 6nn, a touchdown of the contact (603) is detected over the emoji (608-1). In some embodiments, the emoji (608-1) is not associated with any customizable elements (e.g., does not have any customizable elements). In FIG. 6oo, the contact (603) over the emoji (608-1) is maintained for more than a predetermined threshold duration (e.g., 0.1 seconds, 0.25 seconds, 0.5 seconds, 1 second, 3 seconds, 5 seconds, etc.), as indicated by the timer (612). In some embodiments, in response to detecting that the contact (603) is maintained for more than the predetermined threshold duration, the device (500) does not display a customizable user interface or insert the emoji (608-1) into the text input area (604). In some embodiments, in response to the contact being held for a predetermined threshold period of time (e.g., optionally while the contact (603) is held), an overlay is displayed with a preview of the emoji (608-1) (e.g., a larger preview than that which appears on the virtual keyboard (606)). In FIG. 6pp, a liftoff of the contact (603) from the touch screen (504) is detected. In some embodiments, in response to detecting the liftoff of the contact (603), the device (500) inserts the emoji (608-1) as a character (614-12) into the text input area (604), as illustrated in FIG. 6pp. Thus, in some embodiments, short taps and long presses on emojis that do not have any customizable elements do not cause the display of a customizable user interface, and optionally insert the emoji into the text input area.

FIG. 6QQ illustrates an embodiment of a multi-element customization user interface (624) for an emoji having three customizable elements. In the embodiment displayed in FIG. 6QQ, emoji (608-6) has three customizable elements: a skin tone for the left person, a skin tone for the right person, and a headwear accessory for the left person. In some embodiments, as shown in FIG. 6QQ, the multi-element customization user interface (624) includes three rows of variations for customizing emoji (606-8). In some embodiments, a first row includes variations (626-1 through 626-4) of emoji (608-4) corresponding to variations of the first customizable element (e.g., similar to the first row of variations described above). In some embodiments, the second row includes variations (628-1 through 628-4) of emoji (608-4) corresponding to the second customizable element (e.g., similar to the second row of variations described above). In some embodiments, the third row includes variations (636-1 through 636-4) of emoji (608-4) corresponding to the third customizable element. In some embodiments, the third customizable element is optional. For example, as illustrated by variation (636-1), a user may select variation (636-1) to not include a hat for the left person of emoji (608-4). In some embodiments, variations (636-2 through 636-4) correspond to different hat options for the left person of emoji (608-4). Similar to variations (626-1 through 626-4 and 628-1 through 628-4), the left and right people in variations (636-1 through 636-4) are displayed with placeholder appearances (e.g., gray) to indicate that the left and right people are not customizable via variations (636-1 through 636-4). In some embodiments, the fourth row includes a default variation (630-1) and a customized variation (630-2) (e.g., similar to the default variation (630-1) and customized variation (630-2) described above). Thus, the multi-element customization user interface (624) optionally includes any number of sections based on the number of customizable elements for each emoji. It is understood that emoji (608-4) optionally has any number of customizable elements, such as accessory options for the right person. Users can customize/insert/etc. multi-element emojis in the same manner as described above with reference to FIGS. 6e to 6ee.

FIGS. 6rr through 66tt illustrate one embodiment of a multi-element customizable user interface (624) that extends horizontally in a single row (e.g., when the device (500) is in landscape mode and/or the touch-sensitive display can support only one or a limited number of rows of content). In FIG. 6rr, the device (500) is displaying the user interface (600) in landscape mode having a conversation area (602), a text input field (604), and an emoji keyboard (606). It is understood that the emoji keyboard (606) is displayed in FIG. 6rr as a single row of emojis for illustrative purposes only, and that any number of rows are possible. In some embodiments, the emoji keyboard includes five emojis (608-1, 608-3, 608-5, 608-7, 608-8).

In FIG. 6ss, a user input (603) is received selecting an emoji (608-5). In some embodiments, the user input (603) is a contact for less than a predetermined threshold (e.g., 0.1 seconds, 0.25 seconds, 0.5 seconds, 1 second, 3 seconds, 5 seconds, etc.), as exemplified by a timer (612). In the embodiment illustrated in FIG. 6ss, the emoji (608-5) has two customizable elements (e.g., a skin tone of a left person and a skin tone of a right person) and is not yet customized. In response to the user input (603), the device (500) displays a multi-element customization user interface (638), as illustrated in FIG. 6tt. In some embodiments, the multi-element customization user interface (638) is a multi-element customization user interface that is used when the device (500) is in landscape mode. As illustrated in FIG. 6tt, the multi-element customization user interface (638) includes a single row of selectable options. In some embodiments, the single row includes a default variant (630-1) and a customized variant (630-2) (similar to the default variant (630-1) and customized variant (630-2) described above), variants for customizing the first customizable element (626-1 through 626-4), and variants for customizing the second customizable element (628-1 through 628-4). In some embodiments, the multi-element customization user interface is divided into three sections: one section for variants of each of the two customizable elements, and one section for variants for inserting emoji into the text input area (e.g., the default variant and the customized variant). In some embodiments, the sections are divided using a visual divider (e.g., a bar) and/or are divided by whitespace. In some embodiments, the variants (626-1 to 626-4, 628-1 to 628-4) and the default variant (630-1) and the customized variant (630-2) behave similarly to the corresponding variants described above.

FIGS. 6UU through 6XX illustrate an embodiment of a multi-element customization user interface (624) when the multi-element customization user interface is displayed in full-screen mode. In some embodiments, the device (500) is a wearable device and has a touch screen (504). In some embodiments, the touch screen (504) has limited screen real estate. In some embodiments, the elements displayed on the user interface are large relative to the size of the display (e.g., because the display is small). In FIG. 6UU, the device (500) is displaying an emoji keyboard (602) (e.g., a virtual keyboard that displays an emoji selection page). Thus, in some embodiments, the emoji keyboard (602) is displayed in full-screen mode (e.g., instead of being displayed concurrently with the text input area and/or any other user interface elements). In some embodiments, selection of an emoji on the emoji keyboard (602) causes insertion of the individual emoji into the text input area and dismissal of the emoji keyboard (e.g., and optionally, displays the user interface that was displayed prior to the emoji keyboard (602) being displayed). In FIG. 6UU, the emoji keyboard (602) includes nine emojis (608-1 through 608-9).

In FIG. 6vv, a user input (603) is received selecting (e.g., tapping) an emoji (608-5). In the embodiment illustrated in FIG. 6vv, the emoji (608-5) has two customizable elements and is not yet customized. In some embodiments, in response to the user input (603), the device (500) displays a multi-element customization user interface (642), as illustrated in FIG. 6ww. In some embodiments, the multi-element customization user interface (642) is displayed in full-screen mode and replaces the display of the virtual keyboard (602) (e.g., rather than as an overlay). In some embodiments, the multi-element customizable user interface (642) includes a selectable option (644) that is selectable to dismiss the multi-element customizable user interface (642) and revert to a user interface that was displayed before the multi-element user interface was displayed (e.g., the emoji keyboard (602)). In some embodiments, the multi-element customizable user interface (642) includes three sections of selectable options (e.g., similar to the multi-element customizable user interface (624) described above). In some embodiments, the first section includes a default variation (630-1) and a customized variation (630-2). In some embodiments, the second section includes variations (626-1 through 626-4) for customizing the first customizable element. In some embodiments, the second section includes variations (628-1 through 628-4) (not shown) for customizing the second customizable element. In some embodiments, the variants (630-1, 630-2, 626-1 through 626-4 and 628-1 through 628-4) behave similarly to the variants described above.

In some embodiments, the multi-element customization user interface (642) is scrollable. For example, FIG. 6xx illustrates a user input (603) corresponding to an upward swipe gesture for scrolling the user interface upward. In some embodiments, in response to the user input (603), the multi-element customization user interface (642) scrolls upward to reveal a third section that includes variations (628-1 through 628-4) for customizing the second customizable element. In some embodiments, rotating a dial (e.g., a crown) of the wearable device is also interpreted as an upward scrolling input. In some embodiments, selection of the default variant (630-1) or (if customization is complete) a custom variant (630-2) causes dismissal of the multi-element customization user interface (642), display of the user interface that was displayed prior to display of the emoji keyboard (602) (e.g., the user interface into which the emoji is to be inserted), and insertion of the individual variant into the text input area within the user interface. In some embodiments, selection of the default variant (630-1) or (if customization is complete) a custom variant (630-2) causes dismissal of the multi-element customization user interface (642), and display of the emoji keyboard (602) with the individual emoji updated to display the selected variant (e.g., the selected variant is then selectable to insert the individual variant into the text input area).

FIGS. 7A through 7M are flowcharts illustrating a method of presenting a customized user interface for customizing and inserting emojis. The method (700) is optionally performed in an electronic device, such as device (100), device (300), and device (500), as described above with reference to FIGS. 1A and 1B, FIGS. 2, 3, FIGS. 4A and 4B, and FIGS. 5A through 5H. Some of the operations in the method (700) are optionally combined and/or the order of some of the operations is optionally changed.

As described below, the method (700) provides ways to present customized user interfaces for customizing and inserting graphical objects, such as emojis. The method reduces the cognitive burden on a user when interacting with a user interface of a device of the present disclosure, thereby creating a more efficient human-machine interface. For battery-operated electronic devices, increasing the efficiency of user interaction with a user interface saves power and increases the time between battery charges.

In some embodiments, an electronic device (e.g., an electronic device, a mobile device (e.g., a tablet, a smartphone, a media player, or a wearable device), or a computer (e.g., a touchscreen) that communicates with a display device and one or more input devices, such as device (100), device (300), or device (500)) displays a user interface via the display device, such as the user interface (600) of FIG. 6A , a text input area and a virtual keyboard (e.g., a virtual keyboard displaying an emoji keyboard), wherein the virtual keyboard includes a plurality of selectable representations of a plurality of graphical objects (702). In some embodiments, the user interface is any user interface through which a user can enter text using the virtual keyboard. In some examples, the emoji keyboard displays a plurality of selectable emojis for inserting a selected emoji into the text input area. In some embodiments, the emoji keyboard includes a plurality of categories of emojis.

In some embodiments, the electronic devices receive, via one or more input devices, a first user input (e.g., a tap, a long press, or any other suitable selection input on a first emoji on an emoji keyboard) selecting a first selectable representation of a first graphical object from among a plurality of representations of graphical objects, such as user input (603) of FIG. 6E , wherein the first graphical object is associated with a plurality of customizable visual elements including a first customizable visual element and a second customizable visual element (e.g., the first emoji includes a plurality of customizable elements) (704). In some embodiments, the first user input corresponds to a request to insert the first emoji into a text input area. In some embodiments, the first user input corresponds to a request to customize the first emoji, as described below. In some embodiments, the first emoji optionally includes multiple people (e.g., group emojis, “family” emojis, etc.), wherein each person within the emoji is a customizable element. In some embodiments, the skin tone of each person within the multi-person emoji is individually customizable. In some embodiments, the first emoji includes customizable accessories (e.g., hat type, hat color, etc.). In some examples, the user can select a customization option for each customizable element.

In some embodiments, in response to receiving the first user input (706), and upon a determination that one or more first criteria are satisfied (e.g., if the one or more first criteria are satisfied, a customization user interface for the user to customize elements of the first emoji is displayed), the electronic device displays, via the display device, a customization user interface, such as the multi-element customization user interface (624) of FIG. 6F (e.g., displays a user interface for customizing customizable elements of the first emoji) (708).

In some embodiments, if one or more of the first criteria are not satisfied, the personalized user interface is not displayed. In some embodiments, the personalized user interface is displayed regardless of whether the one or more of the first criteria are satisfied. In some embodiments, if one or more of the first criteria are not satisfied, the first emoji is inserted into the text input area. In some embodiments, if the user has not previously completed customization of the first emoji, the one or more of the first criteria are satisfied. In some embodiments, if the user input is a long press (e.g., a touchdown and continued contact having a duration greater than a predetermined threshold, such as 0.1 second, 0.25 seconds, 0.5 seconds, 1 second, 3 seconds, 5 seconds, etc.), the one or more of the first criteria are satisfied. In some embodiments, if the user has previously completed customization of the first emoji (e.g., selected a customization option for each customizable element) or, optionally, if the user has previously selected a default variation of the first emoji and inserted the default variation of the first emoji into the text input field, one or more of the first criteria are not satisfied. In some embodiments, if the first emoji is not a customizable emoji or is not an emoji with multiple customizable elements (e.g., an emoji with only a single customizable element), one or more of the first criteria are not satisfied. In some embodiments, the customizable user interface is a user interface element that overlays elements of a user interface that was displayed before the customizable user interface was displayed (e.g., a pop-up user interface). In some embodiments, the customizable user interface is a full-screen user interface. In some embodiments, the customizable user interface is dismissible to display a user interface that was displayed before the customizable user interface was displayed.

In some embodiments, the customization user interface includes a first section (710) including one or more first selectable options for customizing a first customizable visual element, such as variations (626-1 through 626-4) of FIG. 6f (e.g., a first row displaying one or more variations for the first customizable visual element), a second section (712) including one or more second selectable options for customizing a second customizable visual element, such as variations (628-1 through 628-4) of FIG. 6f (e.g., a second row displaying one or more variations for the second customizable visual element), and a third section (714) for displaying a first individual representation of the first graphical object indicating a current customization state of the first graphical object, such as customized variations (630-2) of FIG. 6f (e.g., a third row displaying a representation of a current customization state of an emoji).

In some examples, the selectable options within the first row are visual previews of individual variations for the first customizable visual element. In some examples, the visual elements that are not customizable by the first selectable options are displayed with different visual characteristics than the visual elements that are customizable by the first selectable options. For example, if the first emoji is a group emoji (e.g., two person emoji), the first section is optionally for customizing the left person. In such examples, the selectable options within the first section are optionally displayed with the right person having a placeholder appearance (e.g., grayed out to indicate that the first section is not for customizing the right person), and the left person is displayed as an individual variation. For example, the first section optionally includes one or more skin tone options for the left person. In some examples, the selectable options within the second row are visual previews of individual variations for the second customizable visual element. In some examples, visual elements that are not customizable by the second selectable options are displayed with different visual characteristics than visual elements that are customizable by the second selectable options. For example, if the first emoji is a group emoji (e.g., two person emoji), the second section is optionally for customizing the right person. In such examples, the selectable options within the second section are optionally displayed with the left person having a placeholder appearance (e.g., grayed out to indicate that the second section is not for customizing the left person), and the right person is displayed as an individual variation. For example, the second section optionally includes one or more skin tone options for the right person. In some embodiments, if the emoji has not yet been customized, the representation of the current customization state indicates that the emoji is not customized. For example, the representation may be displayed with different visual characteristics than if the emoji was partially or fully customized. In some embodiments, the representation is displayed with visual elements having placeholder appearances (e.g., grayed out). In some embodiments, non-customized emojis are not selectable for insertion into the text input area. In some embodiments, the three sections are displayed as separate rows on the customized user interface. In some embodiments, the three sections are displayed as a single row, but are visually separated (e.g., by whitespace or a descriptive marker).

The above-described method of providing a customization user interface for an emoji having multiple customizable components (e.g., by receiving a user selection of an emoji on an emoji keyboard and, if the emoji has not yet been customized, displaying a customization user interface having different sections for selecting options for each of the customizable components of the emoji) provides a quick and efficient way to customize a selected emoji when a user has indicated a desire to insert an emoji (e.g., by presenting the user with customization options for each of the customizable components of the selected emoji), which simplifies the interaction between a user and an electronic device, improves the operability of the electronic device, and makes the user-device interface more efficient (e.g., by allowing the user to view available options for each of the customizable components, view the current customization status of the emoji, and select desired customization options, without requiring the user to navigate to a separate user interface to customize the components of the emoji before selecting an individual emoji on an emoji keyboard), which additionally allows the user to use the electronic device more quickly and efficiently. It reduces power consumption of electronic devices and improves battery life by reducing errors in device usage.

In some embodiments, the first individual representation reflects a first customizable visual element based on one or more first selectable options and a second customizable visual element based on one or more second selectable options (716), for example, the customized variation (630-2) of FIG. 6p reflects a selection of variations (626-1 and 628-3) (e.g., if the emoji is partially or fully customized, the first individual representation displays the results of the user's customization (e.g., the representation reflects the selected customizations)).

In some embodiments, the first individual representation displays a preview of the user's selections from the first and second sections. For example, if the first emoji is a group emoji (e.g., two person emoji), the representation optionally initially displays both people with placeholder appearances (e.g., gray) to indicate that no person has yet been customized. In such examples, in response to the user selecting an option in the first section, the representation optionally updates to show the left person as the selected option (e.g., because the user has not yet selected an option for the right person) and to still display the right person with the placeholder appearance. In some embodiments, in response to the user selecting an option in the second section, the representation optionally updates to show the right person as the selected option. In some embodiments, selection of another option after completing customization overrides the previous selection, and the representation is updated to reflect the new selection. In some embodiments, if the user has completed customization (e.g., selected an option for each customizable element), selection of the representation inserts the customized emoji into the text input field. In some embodiments, if the user has not completed customization (e.g., has not selected an option for each requested customizable element), selecting an expression does not result in insertion of the customized emoji into the text input field.

The above-described manner of displaying a preview of a user's customization choices (e.g., by displaying a representation of an emoji that reflects the user's customizations on the same user interface in which the user made the selections of the customizations) provides a quick and efficient way to display the results of the user's customizations when the user selects customization options (e.g., by displaying a representation that reflects the user's customizations on the same user interface in which the user made the selections of the customizations), which simplifies the interaction between the user and the electronic device (e.g., by allowing the user to see a preview of a customized emoji as he or she customizes the emoji without requiring the user to navigate to a separate user interface to preview the results of his or her customizations (which avoids user errors in generating customized emojis), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, the first individual representation of the first graphical object includes a representation of a first customizable visual element and a representation of a second customizable visual element, such as the left and right people on the customized variation (630-2) of FIG. 6h (718) (e.g., the first individual representation is an image of an individual emoji that includes customizable visual elements).

In some embodiments, upon a determination that the first customizable visual element and the second customizable visual element are not customized (e.g., no variations for either the left person or the right person of the two person emoji were selected), the representation of the first customizable visual element and the representation of the second customizable visual element are displayed (720) as placeholder elements indicating that the first customizable visual element and the second customizable visual element are not customized, such as characters outlined in gray within the customized variation (630-2) of FIG. 6H (e.g., if the visual elements have not yet been customized, the left and right people within the first individual representation are placeholder elements).

In some embodiments, the first individual representation is a full image of the two person emoji, without cropping of either the left or right person. In some embodiments, the placeholder elements are representations of the customizable element that have different visual characteristics than the valid variations. For example, the left and right people are displayed with placeholder appearances, such as gray, shade, or any other visual characteristics that are different than the valid variations (e.g., the placeholder elements do not reflect the valid skin tone options, and optionally display an outline of each of the customizable elements).

The above-described method of displaying a preview of a graphical object having placeholder elements instead of customizable visual elements (e.g., by displaying placeholder elements instead of valid customizable variations) provides a quick and efficient way to indicate to a user that certain customizable visual elements have not yet been customized, which simplifies the interaction between a user and an electronic device, improves the operability of the electronic device, and makes the user-device interface more efficient (e.g., by allowing the user to see that certain customizable visual elements have not yet been customized without requiring the user to perform additional inputs or navigate to a separate user interface to determine whether additional customization is required before the first individual expression can be inserted into a text input area), which additionally reduces power usage and improves battery life of the electronic device by enabling the user to use the electronic device more quickly and efficiently while reducing errors in use of the device.

In some embodiments, while a first individual representation of a first graphical object is displayed with placeholder elements for the first customizable visual element and the second customizable visual element, a second user input, e.g., user input (603 of FIG. 6J ), is received (722) via one or more input devices selecting an individual selectable option on the customization user interface (e.g., while the individual emoji does not have any customizations, and thus the individual representation only displays placeholder elements (e.g., outlines of people), selecting a transform option on the customization user interface).

In some embodiments, in response to receiving a second user input selecting an individual selectable option (724), and in response to a determination that the individual selectable option is a first selectable option of the one or more first selectable options, the electronic device updates (726) the first individual representation of the first graphical object to replace the placeholder element for the first customizable visual element with a first variation of the first customizable visual element corresponding to the first selectable option, for example, the left person in the customized variation (630-2) is updated to represent the skin tone option of variation (626-1) of FIG. 6K (e.g., in response to the user selecting a variation for the first customizable visual element (e.g., the left person), updating the first individual representation with the selected variation).

For example, if the user selected a skin tone option for the left person, the left person in the first individual representation is updated to have the selected skin tone option. Accordingly, in some embodiments, the first individual representation displays a preview of the user's selections. In some embodiments, the customizable visual element that was not selected continues to be displayed as a placeholder element (e.g., the right person continues to be displayed with a placeholder appearance, such as gray). In some embodiments, in response to the user selecting a variation for the first customizable visual element, the selected variation is displayed with an indication that it has been selected. For example, the selected variation is optionally highlighted in a particular color, enlarged, or displayed with any other suitable visual characteristic.

In some embodiments, in response to receiving a second user input selecting an individual selectable option (724), and in response to a determination that the individual selectable option is a second selectable option of the one or more second selectable options, the electronic device updates (728) the first individual representation of the first graphical object to replace the placeholder element for the second customizable visual element with a first variation of the second customizable visual element corresponding to the second selectable option, for example, the person on the right in the customized variation (630-2) is updated to represent the skin tone option of variation (628-3) of FIG. 6p (e.g., in response to the user selecting a variation for the second customizable visual element (e.g., the person on the right) to update the first individual representation with the selected variation).

For example, if the user selected a skin tone option for the right person, the right person in the first individual representation is updated to have the selected skin tone option. Thus, in some embodiments, the first individual representation displays a preview of the user's selections. In some embodiments, the customizable visual element that was not selected continues to be displayed as a placeholder element (e.g., the left person continues to be displayed with a placeholder appearance, such as gray). In some embodiments, in response to the user selecting a variation for the second customizable visual element, the selected variation is displayed with an indication that it has been selected. For example, the selected variation is optionally highlighted in a particular color, enlarged, or has any other suitable visual characteristic. In some embodiments, if both options are now selected, the first individual representation no longer displays any of the placeholder elements. In some embodiments, when the user input completes the customization, an indication that the customization is complete is displayed. In some embodiments, the indication is an animation on the first individual representation similar to the animation of the variations described above (e.g., temporarily enlarging the size of the representation).

The above-described method of replacing placeholder elements in a preview of a graphical object with the user's selections when the user selects variations for the first and second customizable visual elements (e.g., by replacing the placeholder elements with the user's selected variations when the user selects variations) provides a quick and efficient way for the user to preview what their selections will look like in combination, which simplifies the interaction between the user and the electronic device (e.g., by allowing the user to preview the results of their customizations without requiring the user to insert an emoji, perform additional inputs, or navigate to a separate user interface (which also reduces user errors when customizing/inserting an emoji)), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by allowing the user to use the electronic device more quickly and efficiently while reducing errors in use of the device.

In some embodiments, after updating the first individual representation of the first graphical object with the first variation of the first customizable visual element corresponding to the first selectable option, the electronic device receives (730) a third user input, such as user input (603 of FIG. 6w), selecting a third selectable option from the one or more first selectable options via one or more input devices (e.g., while the individual representation has a skin tone customized for the left person, receiving an option to select a different skin tone for the left person).

In some embodiments, in response to receiving the third user input, the electronic device updates (732) the first individual representation of the first graphical object to replace the first variation of the first customizable visual element with a second variation that is different from the first variation of the first customizable visual element corresponding to the third selectable option, for example, the left person in the customized variation (630-2) is updated to reflect the skin tone of the variation (626-2) of FIG. 6x (e.g., updating the first individual representation with the new skin tone selection).

For example, if the left person in the first individual representation was displaying the first skin tone and the user selected the second skin tone option, the left person in the first individual representation would be updated to have the second skin tone. In some embodiments, user input selecting a new skin tone variation for the right person (e.g., the second customizable element) would similarly cause the first individual representation to update the right person's skin tone to the newly selected skin tone. Thus, in some embodiments, previously selected variations are overridden or replaced by the selection of a different variation. It is understood that changing the customization of an individual emoji optionally does not cause previously inserted instances of the individual emoji to be updated with the new customization (e.g., previous versions of the individual emoji inserted into text input areas retain the variations that were applied when they were inserted), and optionally only applies to future insertions of the individual emoji. In some embodiments, after the customization is complete, the customization of the emoji may be modified or updated with the selection of a different variation. In some embodiments, emoji customizations can only be modified or replaced, but cannot be cleared or reset (e.g., reset to a state where no customization or placeholder elements appear).

The above-described method of updating personalized selections (e.g., by receiving user input selecting a different variation than the currently selected variation and updating the preview to reflect the newly selected variation) provides a fast and efficient way to sample different variations and change the user's selections (e.g., by updating the preview of the variation based on the user's new selections), which simplifies the interaction between the user and the electronic device (e.g., by allowing the user to change their previous selections and preview the changes without requiring the user to navigate to a separate user interface or perform additional inputs), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, while the customization state of the first graphical object is a partially customizable state and a first individual representation of the first graphical object is displayed with a first variation for the first customizable visual element and a placeholder element for the second customizable visual element (e.g., the person on the left is customized, but the person on the right is not yet customized), the electronic device receives (734) a second user input, via one or more input devices, corresponding to a request to dismiss the customization user interface, such as user input (603) of FIG. 6m.

In some embodiments, in response to receiving the second user input, the electronic device ceases displaying (736) the customized user interface (e.g., dismisses the customized user interface or otherwise ceases displaying the customized user interface, and optionally, displays the user interface that was displayed before the customized user interface was displayed), as in FIG. 6n. In some embodiments, because the emoji has not completed customization, the individual emoji displayed on the virtual keyboard retains the display of the default variation for both the left and right person (e.g., the emoji having a default color, such as yellow).

In some embodiments, after ceasing to display the personalized user interface and while the personalized state of the first graphical object is in a partially personalized state, the electronic device receives (738) a third user input, e.g., user input (603 of FIG. 6n), selecting a first selectable representation of the first graphical object via one or more input devices (e.g., receiving user input selecting a partially personalized emoji on a virtual keyboard).

In some embodiments, in response to receiving the third user input, the electronic device displays a customizable user interface via the display device, as in FIG. 6o , wherein the first individual representation of the first graphical object is displayed (740) in the customizable user interface with a first variation for the first customizable visual element and a placeholder element for the second customizable visual element (e.g., re-displaying the customizable user interface). In some embodiments, the customizable user interface preserves the customization state. For example, since the left person of the individual emoji was previously customized but the right person of the individual emoji was not customized, the individual representation displays the customized variation for the left person and the placeholder element for the right person. In some embodiments, the selectable options within the customizable user interface also preserve their selection state. For example, the variation for the left person that was previously selected (and is being applied to the first individual representation) continues to be displayed with an indication that it is selected (e.g., highlighted, enlarged, etc.).

The above-described manner of retaining a user's customization selections (e.g., by storing the user's selections even if the user only partially customized a graphical object, and, after dismissal of the customization user interface, displaying the user's previous selections upon subsequent displays of the customization user interface) provides a quick and efficient way for a user to continue customizing a previously-partially customized graphical object, which simplifies the interaction between the user and the electronic device (e.g., by allowing the user to continue a customization process where they left off without requiring the user to perform additional user inputs to reselect the variation(s) they have already previously selected), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, upon a determination that the plurality of customizable visual elements include a third customizable visual element, the customization user interface includes a fourth section (742) comprising one or more third selectable options for customizing the third customizable visual element, as in FIG. 6qq (e.g., if the individual emoji has a third customizable visual element, the customization user interface includes another row of selectable options for customizing the third customizable visual element). Thus, in some embodiments, the customization user interface includes a section (e.g., row) of selectable options for each customizable visual element of the individual emoji. In some embodiments, any number of customizable visual elements is possible (e.g., two, three, four, five, ten customizable visual elements, etc.).

The above-described manner of displaying sections of selectable options for each customizable visual element within a graphical object (e.g., by displaying more or fewer sections of variation options based on how many customizable visual elements the graphical object has) provides a quick and efficient way to present different variations for each customizable visual element, which simplifies the interaction between a user and the electronic device (e.g., by adjusting the number of sections within the customizable user interface based on the number of visual elements to be customized, while maintaining consistency in the user interaction with the customization user interface, and without requiring the user to navigate to a separate user interface or perform additional inputs to view and select all available variations for each customizable visual element), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, the customizable user interface includes a second individual representation of the first graphical object (744) corresponding to predefined variations of the first customizable visual element and the second customizable visual element, e.g., the default variation (630-1) of FIG. 6F (e.g., the customizable user interface includes a predefined default representation that is selectable to insert the default variation into the text input area). For example, the predefined default representation is optionally a default color for each element within the emoji, e.g., yellow. In some embodiments, the predefined default representation is not customizable, and selection of variations in the customizable user interface customizes only the first individual representation and not the second individual representation. In some embodiments, the second individual representation is displayed in a third section of the customizable user interface, optionally adjacent to the first individual representation.

The above-described method of providing a predefined default version of a first graphical object (e.g., by including a representation of a first graphical object with default variations for customizable visual elements) provides a quick and efficient way to select a default version of a first graphical object without selecting a customization - even after customization has been performed - which simplifies the interaction between a user and the electronic device, improves the operability of the electronic device, and makes the user-device interface more efficient (e.g., by allowing the user to easily select default variations if he or she is not interested in customizing the first graphical object, without requiring the user to perform additional inputs to configure the default version of the first graphical object), which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, while the customization state of the first graphical object is not a completed customization state (e.g., partially customized or not customized), the electronic device receives (746) a second user input, via one or more input devices, selecting a first individual representation of the first graphical object, e.g., user input (603) of FIGS. 6H and 6K (e.g., user input selecting the first individual representation while the first individual representation is displayed as placeholder elements for any or all of the customizable visual elements).

In some embodiments, in response to receiving the second user input (748), and determining that the first customizable visual element is not customized, the electronic device displays (750) a visual indication via the display device that the first customizable visual element is not customized, such as the indication (632) of FIG. 6i (e.g., if the first customizable visual element is not customized, displaying the visual indication over the first selectable options for customizing the first customizable visual element). In some embodiments, in response to the second user input, the first individual expression is not inserted into the text input field.

In some embodiments, in response to receiving the second user input (748), and determining that the second customizable visual element is not customized, the electronic device displays (752) a visual indication via the display device that the second customizable visual element is not customized, such as the indication (632) of FIG. 6L (e.g., if the second customizable visual element is not customized, displaying the visual indication on the second selectable options for customizing the second customizable visual element). In some embodiments, in response to the second user input, the individual emoji are not inserted into the text input area. In some embodiments, the customization user interface continues to be displayed (e.g., is not dismissed).

The above-described manner of indicating that a customization is not complete (e.g., by displaying an indication that individual customizable visual elements have not yet been customized when a user selects and attempts to insert a customized version of a graphical object) provides a quick and efficient way to indicate that a customization is not complete (e.g., by indicating to the user that selection of a first individual representation still requires selection of individual variations before inserting a customized version of the graphical object), which simplifies the interaction between a user and the electronic device (e.g., by displaying an indication that the customization is not complete without requiring the user to navigate to a separate user interface or perform additional inputs to determine whether the customization is complete and a customized version of the graphical object is available for insertion), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, the visual indication that the first customizable visual element is not customized comprises animating a change in a visual characteristic of one or more of the first selectable options for customizing the first customizable visual element (e.g., animating the first selectable options), as in FIG. 6i (754). In some embodiments, the animation comprises temporarily increasing and decreasing the size of the first selectable options. In some embodiments, the animation moves in a wavelike motion from one end of the first selectable options to the other end. For example, a first selectable option of the first selectable options increases in size while the other selectable options of the first selectable options maintain their original sizes, and then, as the first selectable option of the first selectable options decreases in size, a second selectable option of the first selectable options increases in size, and so on, until the animation reaches the last selectable option of the first selectable options.

In some embodiments, the visual indication that the second customizable visual element is not customized includes animating a change in a visual characteristic of one or more of the second selectable options for customizing the second customizable visual element (e.g., animating the second selectable options), as in FIG. 6l (756). In some embodiments, the animation includes temporarily increasing and decreasing the size of the second selectable options. In some embodiments, the animation moves in a wavelike motion from one end of the second selectable options to the other end. For example, a first selectable option of the second selectable options increases in size while the other selectable options of the second selectable options maintain their original sizes, and then, as the first selectable option of the second selectable options decreases in size, a second selectable option of the second selectable options increases in size, and so on, until the animation reaches the last selectable option of the second selectable options.

The above-described manner of indicating that a customization is not complete and that selection of certain selectable options is required to complete the customization (e.g., by animating selectable options for individual customizable visual elements that require selection to complete the customization) provides a quick and efficient way to indicate which options are to be selected to complete the customization (e.g., by displaying an animation over the selectable options for individual customizable visual elements that have not yet been customized), which simplifies the interaction between a user and the electronic device (e.g., by displaying an indication over the individual selectable options themselves without requiring the user to determine which of the customizable elements have not been customized and which selectable options are for customizing the not-yet-customizable elements), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, while the customization state of the first graphical object is a completed customization state in which an individual variation of the first customizable visual element and an individual variation of the second customizable visual element have been selected (e.g., customization of the emoji is completed and the first individual representation is displayed with valid variations for the first and second customizable visual elements without any placeholder elements), the electronic device receives (758) a second user input, via one or more input devices, selecting a first individual representation of the first graphical object, such as user input (603) of FIG. 6q (e.g., a user input selecting a completed first individual representation).

In some embodiments, in response to receiving the second user input, the electronic device inserts (760) a first graphical object having a respective variation of the first customizable visual element and a respective variation of the second customizable visual element into the text input area, for example, inserting the personalized variation (630-2) into the text input area (604) as character (614-4) of FIG. 6r (e.g., inserting an individual emoji having respective variations for the first and second customizable visual elements into the text input area). In some embodiments, the version of the emoji that is inserted is identical to the preview provided by the first individual representation. In some embodiments, in response to the second user input, the personalized user interface is dismissed (e.g., ceases to be displayed).

The above-described method of inserting a customized graphical object (e.g., in response to a user selecting a representation of a first graphical object, and after the user has completed customizing the first graphical object, inserting a first graphical object having the user's customizations) provides a quick and efficient way to insert customized graphical objects (e.g., by displaying a preview of the customizations applied to the first graphical object on the same user interface used to select the customizations, and in response to the user selecting the preview, inserting the customized first graphical object), which simplifies the interaction between the user and the electronic device (e.g., by allowing the user to preview and insert the customized first graphical object without requiring the user to navigate to a separate user interface or perform additional inputs to customize the graphical element and insert the customized graphical element), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, after receiving a second user input selecting a first individual representation of the first graphical object, the electronic device updates (762) the first selectable representation of the first graphical object on the virtual keyboard to display a representation of the first graphical object having an individual variation of the first customizable visual element and an individual variation of the second customizable visual element, for example, emoji (608-6) is updated to display the customized variation (630-2) of FIG. 6r (e.g., after the user selects and inserts a customized version of the emoji into the text input area, the individual emoji on the virtual keyboard is updated to reflect the customized version of the emoji).

For example, an individual emoji on the virtual keyboard includes individual variations for the first and second customizable visual elements (e.g., the left and right people have skin tones selected on the customization user interface). In some embodiments, if an individual emoji is displaying a customized version of the emoji, then if the user subsequently updates or modifies the customization (e.g., by selecting different variations on the customization user interface) but does not insert the customized version of the emoji into the text input area, the virtual keyboard will be updated to display the customized version. Accordingly, in some embodiments, an individual emoji on the virtual keyboard is set to a "customized" mode that will always display the most recent customized state of the individual emoji (e.g., regardless of whether the user has inserted a customized emoji since updating the customization) until the user changes the individual emoji to a "default" mode that displays default variations of the emoji on the virtual keyboard (e.g., by selecting a default version of the individual emoji on the customization user interface (which causes the default emoji to be inserted into the text input area)).

The above-described manner of displaying a graphical object on a virtual keyboard (e.g., by displaying a graphical object having variations of the most recently selected and inserted customizable visual elements on the virtual keyboard) provides a quick and efficient way to preview a version of a graphical object to be inserted upon selection of a representation of the graphical object (e.g., by displaying a preview of the customizations applied to the graphical object on the virtual keyboard itself, selectable to insert a customized version of the graphical object), which simplifies the interaction between a user and the electronic device (e.g., by allowing the user to preview and insert a customized first graphical object without requiring the user to navigate to a separate user interface or perform additional inputs to determine which version of the graphical element to insert upon selection of the representation), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, while the customization state of the first graphical object is in a completed customization state, the electronic device receives (764) a second user input, e.g., user input (603 of FIG. 6r), selecting a first selectable representation of the first graphical object via one or more input devices (e.g., receiving user input selecting an individual emoji on a virtual keyboard after the individual emoji has been fully customized).

In some embodiments, in response to receiving a second user input (766), and upon determining that the second user input includes selection of a first selectable representation of the first graphical object for less than a predetermined duration threshold (e.g., 0.1 second, 0.25 seconds, 0.5 seconds, 1 second, 3 seconds, 5 seconds, etc.), the electronic device inserts (768) the first graphical object into the text input area without displaying a customized user interface, for example, inserting the customized variation (630-2) into the text input area (604) as character (614-5) of FIG. 6s (e.g., if the user input is a tap input, inserting an individual emoji into the text input area and not displaying the customized user interface).

In some embodiments, the version of the inserted emoji is the same version displayed on the virtual keyboard. For example, if an individual emoji on the virtual keyboard includes individual variations for the first and second customizable visual elements, the inserted emoji has individual variations for the first and second customizable visual elements. In some embodiments, if an individual emoji on the virtual keyboard includes default variations for the first and second customizable visual elements (e.g., an emoji having a default color such as yellow), the inserted emoji has default variations for the first and second customizable visual elements.

In some embodiments, in response to receiving the second user input (766), and upon determining that the second user input includes selection of a first selectable representation of the first graphical object for a predetermined duration threshold, the electronic device displays (770) a customization user interface via the display device without inserting the first graphical object into the text input area (e.g., if the user input is a press-and-hold input, displaying the customization user interface and not inserting the individual emoji into the text input area), as in FIG. 6u . In some embodiments, after customization of the individual emoji is complete, the press-and-hold user input is the only way to cause display of the customization user interface (e.g., to allow the user to update the customization). In some embodiments, the customization user interface displays a customization state for the first graphical object. For example, individual variations for the first and second customizable elements are highlighted, and the first individual representation (e.g., a preview) displays the user's customizations.

The above-described method of inserting and customizing a graphical object (e.g., inserting a graphical object if the user input is a short tap, or displaying a customization user interface if the user input is a long press) provides a quick and efficient way to either insert a graphical object (e.g., by providing the user with the option to either insert a graphical object or to customize the graphical object using the same representation, based on the duration of the user input) or to provide a user interface for viewing or updating the customization of the graphical object, which simplifies the interaction between the user and the electronic device, improves the operability of the electronic device, and makes the user-device interface more efficient (e.g., by allowing the user to perform either the functions of inserting a graphical object or further customizing a graphical object without requiring the user to perform additional inputs or navigate to a separate user interface to perform the customization), which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, inserting the first graphical object into the text input area (772) comprises inserting (774) the first graphical object having default variations into the first text input area, for example, inserting the default variation (630-1) into the text input area (604) as character (614-8) of FIG. 6ee (e.g., the inserted emoji is the same version that is displayed on the virtual keyboard). In some embodiments, if an individual emoji on the virtual keyboard includes default variations for the first and second customizable visual elements (e.g., an emoji having a default color such as yellow), the inserted emoji has default variations for the first and second customizable visual elements.

In some embodiments, inserting the first graphical object into the text input area (772) comprises inserting (776) the first graphical object with customized variations into the first text input area, for example, inserting the customized variation (630-2) into the text input area (604) as character (614-5) of FIG. 6s , based on a determination that the first selectable representation of the first graphical object is displayed with customized variations for the first customizable visual element and the second customizable visual element (e.g., if an individual emoji on the virtual keyboard includes individual variations for the first customizable visual element and the second customizable visual element, the inserted emoji has individual variations for the first customizable visual element and the second customizable visual element). Thus, in some embodiments, selecting an individual emoji on the virtual keyboard will automatically insert the previously selected version of the individual emoji.

The above-described method of inserting a graphical object (e.g., inserting a version of the graphical object when a version of the graphical object is displayed on a virtual keyboard) provides a quick and efficient way to verify that the graphical object to be inserted is the graphical object displayed to the user (e.g., by inserting a graphical object having either the customized variations, if the customized variations are displayed on the virtual keyboard, or the default variations, if the default variations are displayed on the virtual keyboard), which simplifies the interaction between the user and the electronic device (e.g., by allowing the user to preview and verify the version of the graphical object to be inserted without requiring the user to perform additional inputs or navigate to a separate user interface to determine which version to insert upon selection of the first selectable expression), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, the first user input comprises a touchdown and liftoff of a contact from a touch-sensitive surface of one or more input devices, and the customization user interface is displayed after the touchdown of the contact on the touch-sensitive surface and before the liftoff of the contact from the touch-sensitive surface, for example, in FIG. 6u, display of a multi-element customization user interface (624) is displayed (778) in response to the contact (603) being held for more than a predetermined duration threshold (e.g., the customization user interface is displayed in response to detecting a touchdown of the contact on an emoji within a virtual keyboard associated with a plurality of customizable elements). In some embodiments, if customization is not complete, the customization user interface is displayed in response to any touchdown of the contact. In some embodiments, if customization is complete, the custom user interface is displayed in response to a touchdown being made in contact with the touch screen and maintained for a threshold time duration (e.g., 0.1 second, 0.25 second, 0.5 second, 1 second, 3 seconds, 5 seconds, etc.).

In some embodiments, while displaying a customized user interface (e.g., displayed in response to either a short contact if customization is not complete or a long contact if customization is complete), the electronic device detects liftoff of the contact from the touch-sensitive surface, as in FIG. 6v (780).

In some embodiments, in response to detecting a liftoff of contact from the touch-sensitive surface, the electronic device maintains display of a customized user interface via the display device (782), as in FIG. 6v (e.g., when the user releases contact with the touch screen, the customized user interface is not dismissed). In some embodiments, the customized user interface is displayed until the user selects an emoji to insert into the text input area or selects a location outside the customized user interface. In some embodiments, if the user input comprises a swipe from an original touchdown (e.g., while maintaining contact) to a location on the customized user interface for either the default version of the emoji or a customized version of the emoji, upon liftoff of the user input, the respective (e.g., default or customized) version of the emoji is selected and inserted into the text input area, and the customized user interface is dismissed. In some embodiments, for single-element customizable user interfaces (e.g., emojis with only one customizable element), detecting liftoff of a contact always causes the customizable user interface to be dismissed, and a version of the emoji (e.g., either the user's previous selection or the user's new selection, if the user input included a swipe to a new variation prior to liftoff) to be inserted into the text input area.

The above-described method of displaying a customized user interface (e.g., displaying the customized user interface in response to detecting a touchdown of a contact, and maintaining the display of the customized user interface after detecting a liftoff of the contact) provides a quick and efficient way to customize a graphical object (e.g., by maintaining the display of the customized user interface so that a user can make additional inputs to select desired variations on the customized user interface), which simplifies the interaction between a user and the electronic device (e.g., by allowing a user to customize multiple elements of the graphical object after displaying the customized user interface without requiring the user to re-display the customized user interface for each element to be customized), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling a user to use the electronic device more quickly and efficiently.

In some embodiments, a customized user interface, such as the multi-element customized user interface (624) of FIG. 6F, is overlaid (786) over a user interface that includes a text input area and a virtual keyboard (e.g., the customized user interface is a pop-up user interface element that is displayed as an overlay on top of everything that was previously displayed when user input to display the customized user interface is received).

The above-described manner of displaying a customized user interface (e.g., displaying the customized user interface as an overlay over a previously displayed user interface when the customized user interface was displayed) provides contextual continuity of the user interface, which simplifies the interaction between the user and the electronic device (e.g., by visually indicating to the user that the customization and insertion of graphical objects are related to a previously displayed interface, and by providing the user with a customized user interface with minimally distracting user interface elements), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, based on a determination that the customized user interface comprises less than a predetermined amount of the display area of the display device (e.g., the customized user interface occupies less than 25%, 50%, 66%, 75% of the total display area of the display device), the customized user interface, e.g., the multi-element customized user interface (624) of FIG. 6F, is overlaid (788) over the user interface that includes the text input area and the virtual keyboard (e.g., the customized user interface is displayed as a pop-up user interface element that is overlaid on top of everything that was being displayed when user input to display the customized user interface was received).

In some embodiments, upon a determination that the customized user interface comprises more than a predetermined amount of the display area of the display device (e.g., the customized user interface occupies more than 25%, 50%, 66%, 75% of the total display area of the display device), the customized user interface, e.g., the multi-element customized user interface (642) of FIG. 6ww, replaces the display of the user interface (790) (e.g., the customized user interface is displayed as a full-screen user interface). In some embodiments, the customized user interface includes a selectable option to dismiss the customized user interface. In some embodiments, in response to user input to dismiss the customized user interface, the device displays the user interface that was displayed when the user input to display the customized user interface was received (e.g., resumes display of the user interface).

The above-described manner of displaying a customized user interface (e.g., displaying the customized user interface as an overlay when the customized user interface occupies less than a certain amount of screen area or as a full-screen user interface when the customized user interface occupies more than a certain amount of screen area) provides, where appropriate, contextual continuity of the user interface, which simplifies the interaction between a user and the electronic device (e.g., by automatically adjusting the style of the customized user interface to enable a user to view and perform customizations regardless of the size of the display device), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by enabling a user to use the electronic device more quickly and efficiently while reducing errors in use of the device.

In some embodiments, while displaying the customized user interface, the electronic device receives (792) a second user input, such as user input (603) of FIG. 6m, via one or more input devices, corresponding to a location outside the customized user interface (e.g., receiving the user input on a location on a touch screen outside the customized user interface).

In some embodiments, in response to receiving the second user input, the electronic device ceases displaying (794) the customized user interface (e.g., dismisses the customized user interface and redisplays the entire user interface, including the virtual keyboard and text input area (e.g., unobstructed by the customized user interface), as in FIG. 6n). In some embodiments, user input selecting the “paddle” portion of the customized user interface also causes the customized user interface to be dismissed. In some embodiments, the “paddle” portion of the customized user interface is the portion that is overlaid over the corresponding emoji on the virtual keyboard. Thus, in some embodiments, selecting the same location that caused the display of the customized user interface to be dismissed. In some embodiments, after discontinuing displaying the customized user interface, additional input selecting an individual emoji from the virtual keyboard will cause the customized user interface to be re-displayed, if customization has not yet been completed.

The above-described method of dismissing a customized user interface (e.g., dismissing a customized user interface in response to user input on a touch screen outside of the customized user interfaces) provides an intuitive and efficient way to dismiss a customized user interface (e.g., by quickly dismissing the customized user interface when the user indicates that he or she is not interested in performing customizations or viewing the customized user interface by tapping outside of the customized user interface), which simplifies the interaction between the user and the electronic device, improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, the plurality of selectable representations of the plurality of graphical objects included in the virtual keyboard further includes a second selectable representation of a second graphical object, wherein the second graphical object, such as the emoji (608-1) of FIG. 6A , is not associated with a plurality of customizable visual elements (796) (e.g., the virtual keyboard displays a plurality of different emojis selectable to customize or insert individual emojis). In some embodiments, the plurality of different emojis include emojis without customizable visual elements, emojis with one customizable visual element, and emojis with multiple customizable visual elements.

The above-described manner of presenting graphical objects on a virtual keyboard (e.g., displaying graphical objects associated with a plurality of customizable visual elements on the virtual keyboard next to graphical objects that are not associated with the plurality of graphical objects) provides a quick and efficient way to select different graphical objects (e.g., by providing the user with the option to select from the plurality of graphical objects, regardless of whether the plurality of graphical objects are associated with the plurality of customizable objects), which simplifies the interaction between a user and the electronic device (e.g., by allowing the user to select from the plurality of graphical objects from the same user interface without requiring the user to navigate to a user interface for accessing graphical objects that are not associated with the plurality of graphical objects and a different user interface for accessing graphical objects associated with the plurality of graphical objects), improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces power usage and improves battery life of the electronic device by reducing errors in use of the device while enabling the user to use the electronic device more quickly and efficiently.

In some embodiments, the electronic device receives, via one or more input devices, a second user input, e.g., user input (603) of FIG. 6B , selecting an individual selectable representation of the plurality of selectable representations on the virtual keyboard, wherein each selectable representation is associated with a respective graphical object (798) (e.g., receiving user input selecting an emoji on the virtual keyboard).

In some embodiments, in response to receiving a second user input (799), upon determining that the individual graphical object is associated with a plurality of customizable visual elements and one or more first criteria are satisfied (e.g., the selected emoji has a plurality of customizable visual elements and customization is not complete or the user input is a contact for more than a predetermined duration threshold), the electronic device displays, via the display device, a first customization user interface, such as a multi-element customization user interface (624) of FIG. 6F , that includes selectable options for customizing the plurality of customizable visual elements of the individual graphical object, and an individual representation of the individual graphical object reflecting the plurality of customizable visual elements based on the selectable options, wherein the individual representation of the individual graphical object is selectable to insert the individual representation into the text input area (797) (e.g., including a plurality of sections for customizing the plurality of customizable visual elements, and an individual representation selectable to preview the user's customizations and insert the customized emoji into the text input area). (Displaying a customizable user interface similar to the described multi-element customizable user interface). In some embodiments, the selectable options for customizing the multiple customizable visual elements of an individual graphical object are selectable to customize the individual graphical object without inserting the individual graphical object into the text input area.

In some embodiments, in response to receiving the second user input (799), and determining that the individual graphical object is associated with only one customizable visual element and one or more second criteria are satisfied (e.g., the selected emoji has only one customizable visual element and customization is not complete, or the user input is a contact for more than a predetermined duration threshold), the electronic device displays, via the display device, a second customizable user interface, such as the single-element customizable user interface (618) of FIG. 6D , that includes individual representations corresponding to individual variations of the one customizable visual element, wherein the individual representations corresponding to the individual variations are selectable to customize one customizable visual element of the individual graphical object and insert the individual graphical object into the text input area (795) (e.g., displaying a single-element customizable user interface that lacks multiple options for customizing multiple customizable visual elements).

In some embodiments, the single-element customization user interface displays multiple variations of a single customizable visual element in a single row. In some embodiments, the single-element customization user interface does not include individual representations that preview the user's customizations. In some embodiments, if the selected emoji has not been customized (e.g., no variations have been selected), upon detecting liftoff of the contact, the display of the single-element customization user interface is maintained. In some embodiments, selection of an individual variation of the multiple variations causes insertion of the individual variation into the text input area (e.g., the preview may be selectable to cause insertion, as in the case of a graphical object with multiple customizable elements, as opposed to updating the preview). In some embodiments, if the selected emoji has already been personalized (e.g., if the user previously selected one of the variations for a customizable visual element), upon detecting liftoff of the contact, the emoji is inserted into the text input area and the single-element personalization user interface is dismissed (e.g., the single-element personalization user interface is displayed only while the user input is in contact with the touch screen). In some embodiments, the inserted emoji is the previously selected variation of the emoji. In some embodiments, if the contact was moved to an individual variation of the multiple variations, upon liftoff, the inserted emoji is the individual variation.

In some embodiments, in response to receiving the second user input (799), and upon determining that the individual graphical object is not associated with any customizable visual elements (e.g., the selected emoji does not have any customizable visual elements), the electronic device inserts the individual graphical object into the first text input area (793) without displaying a customizable user interface via the display device, for example, inserts the emoji (608-1) into the text input area (604) as character (614-1) of FIG. 6C (e.g., inserts the selected emoji into the text input area without displaying a customizable user interface). In some embodiments, if the user input is a contact exceeding a predetermined duration threshold, a pop-up is displayed. In some embodiments, the pop-up includes a representation of the selected emoji. In some embodiments, upon detecting liftoff of the contact, the emoji is inserted into the text input area. In some embodiments, the pop-up does not include any selectable options for customizing the emoji. In some embodiments, the pop-up simply provides the user with a larger preview of the emoji to be inserted (e.g., larger than the display of the emoji within the virtual keyboard).

The above-described manner of customizing and inserting graphical objects (e.g., by receiving user input on a graphical object on a virtual keyboard, and displaying a multi-element customization user interface if the graphical object is associated with multiple customizable elements, a single-element customization user interface if the graphical object is associated with only one customizable element, or by simply inserting the graphical object if the graphical object is not associated with any customizable elements) provides a quick and efficient way to insert graphical objects and present customization options only when necessary (e.g., by determining whether a selected graphical object can be customized and how many customizations are available, and automatically displaying (or not displaying) the appropriate customization user interface), which facilitates interaction between a user and an electronic device (e.g., by automatically presenting the appropriate customization user interface if customizations are available, or inserting the graphical object if customizations are not available, without requiring the user to separately determine whether a particular graphical object supports customizations, and without requiring additional inputs to navigate to a separate user interface to customize and/or insert graphical objects that support customizations). It simplifies the interaction between the electronic device and the user, improves the operability of the electronic device, and makes the user-device interface more efficient, which additionally reduces the power usage of the electronic device and improves the battery life by reducing errors in the use of the device while enabling the user to use the electronic device more quickly and efficiently.

It should be understood that the particular order in which the operations described in FIGS. 7A through 7M are exemplary only and are not intended to indicate that the described order is the only order in which the operations may be performed. Those skilled in the art will recognize various ways to reorder the operations described herein.

The operations in the information processing methods described above are optionally implemented by driving one or more functional modules within an information processing device, such as general purpose processors or application specific chips (e.g., as described with respect to FIGS. 1A, 1B, 3, and 5A to 5H). Additionally, the operations described above with reference to FIGS. 7A to 7M are optionally implemented by the components illustrated in FIGS. 1A and 1B. For example, the display operations (702, 708, 740, 750, 752, 770, 797, 795) and the receiving operations (704, 730, 734, 738, 746, 758, 764, 792, 798) are optionally implemented by an event classifier (170), an event recognizer (180), and an event handler (190). When a particular predefined event or sub-event is detected, the event recognizer (180) activates an event handler (190) associated with the detection of the event or sub-event. The event handler (190) optionally updates the application's internal state (192) by utilizing or calling a data updater (176) or an object updater (177). In some embodiments, the event handler (190) accesses a particular GUI updater (178) to update what is displayed by the application. Similarly, it will be apparent to those skilled in the art how other processes may be implemented based on the components illustrated in FIGS. 1A and 1B.

As described above, one aspect of the present technology potentially involves the collection and use of data available from specific and legitimate sources to facilitate customization of graphical objects. The present disclosure contemplates that, in some examples, such collected data may include personal data that uniquely identifies or can be used to identify a particular person. Such personal data may include demographic data, location-based data, online identifiers, usage history, skin tone (or other visual appearance information), skin tone (or other visual appearance) preferences, customization preferences, or any other personal information.

The present disclosure recognizes that such use of personal information data in the present technology may be used to benefit users. For example, the personal information data may be used to automatically perform actions for interacting with an electronic device using a virtual keyboard (e.g., customizing and/or inserting graphical objects using the virtual keyboard). Thus, the use of such personal information data may enable users to make fewer inputs to perform actions on graphical objects. Additionally, other uses of personal information data that benefit users are also contemplated by the present disclosure. For example, the user's customization selections may be used to generate suggested customizations for the user.

This disclosure contemplates that such entities responsible for collecting, analyzing, disclosing, transmitting, storing, or otherwise using such personal data will adhere to well-established privacy policies and/or privacy practices. In particular, such entities will be expected to implement and consistently apply privacy practices that are generally recognized as meeting or exceeding industry or administrative requirements for maintaining the privacy of users. Such information regarding the use of personal data should be prominent and readily accessible to users, and should be updated as the collection and/or use of the data changes. Personal information from users should be collected only for lawful uses. Additionally, such collection/sharing should occur only after receiving consent from users or other lawful grounds specified in applicable law. Additionally, such entities should consider taking any necessary steps to protect and secure access to such personal data and to ensure that others who have access to the personal data adhere to their privacy policies and procedures. Additionally, such entities may themselves be assessed by third parties to demonstrate their adherence to widely accepted privacy policies and practices. In addition, policies and practices should be adapted to the specific types of personal data being collected and/or accessed, and to applicable laws and standards, including jurisdiction-specific considerations that may serve to impose higher standards. For example, in the United States, collection of or access to certain health data may be governed by federal and/or state law, such as the Health Insurance Portability and Accountability Act (HIPAA); whereas health data in other countries may be subject to, and must be handled in accordance with, different laws and policies.

Notwithstanding the foregoing, the present disclosure also contemplates embodiments in which users may selectively block use of, or access to, personal information data. That is, the present disclosure contemplates that hardware and/or software elements may be provided to prevent or block access to such personal information data. For example, a user may configure one or more electronic devices to change the discovery or privacy settings of the electronic devices. For example, a user may select a setting that only allows a particular electronic device to access the user's customized selections (e.g., not allowing the customized selections to be shared with other electronic devices and/or not allowing the selections to be transmitted to a server). In another example, users may manage the privacy of their customized selections.

Moreover, it is the intent of the present disclosure that personal data should be managed and processed in a manner that minimizes the risks of unintended or unauthorized access or use. Risks can be minimized by limiting the collection of data and deleting it when it is no longer needed. Additionally, and where applicable, including for certain health-related applications, data de-identification can be used to protect the privacy of users. Where appropriate, de-identification can be facilitated by removing identifiers, controlling the amount or specificity of data stored (e.g., by collecting location data at the city level rather than at the address level), controlling the way data is stored (e.g., by aggregating data across users), and/or by other methods such as differential privacy.

Accordingly, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may also be implemented without the need for access to such personal information data. That is, various embodiments of the present technology are not rendered inoperable by the absence of all or part of such personal information data. For example, a user may be able to interact with graphical objects and insert graphical objects into text input areas without having to provide any customization selections.

The foregoing description has been described with reference to specific embodiments, for purposes of illustration. However, the above illustrative discussions are not intended to be exhaustive or to limit the invention to the precise forms disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments were chosen and described in order to best explain the principles of the invention and its practical applications, thereby enabling others skilled in the art to best utilize the invention and the various described embodiments with various modifications as are suited to the particular use contemplated.

Claims (28)

As a method,
In an electronic device communicating with a display device and one or more input devices,
A step of displaying a text input area and a virtual keyboard in a user interface via said display device, said virtual keyboard including a plurality of selectable representations of a plurality of graphical objects;
receiving, via one or more of the input devices, a first user input selecting a first selectable representation of a first graphical object from among a plurality of selectable representations of the graphical objects, wherein the first graphical object includes a plurality of customizable visual elements corresponding to a plurality of different portions of the first graphical object, the plurality of customizable visual elements including a first customizable visual element corresponding to a first portion of the first graphical object and a second customizable visual element corresponding to a second portion of the first graphical object that is different from the first portion; and
In response to receiving the first user input,
comprising the step of displaying a customized user interface via the display device, based on a determination that one or more first criteria are satisfied;
The above customized user interface,
A first section comprising one or more first selectable options for customizing the first portion of the first graphical object corresponding to the first customizable visual element;
a second section comprising one or more second selectable options for customizing the second portion of the first graphic object corresponding to the second customizable visual element; and
a third section for displaying a first individual representation of said first graphic object, said first graphic object indicating a current customization state of said first graphic object;
The first individual representation of the first graphic object comprises a representation of the first customizable visual element and a representation of the second customizable visual element,
A method wherein, based on a determination that the first customizable visual element and the second customizable visual element are not customized, the representation of the first customizable visual element and the representation of the second customizable visual element are displayed as placeholder elements indicating that the first customizable visual element and the second customizable visual element are not customized. In the first paragraph,
A method wherein said first individual representation reflects said first customizable visual element based on said one or more first selectable options and said second customizable visual element based on said one or more second selectable options. delete In the first paragraph,
receiving, via one or more input devices, a second user input selecting an individual selectable option in the customizable user interface while the first individual representation of the first graphical object is displayed with placeholder elements for the first customizable visual element and the second customizable visual element; and
In response to receiving said second user input selecting said individual selectable option,
updating the first individual representation of the first graphical object to replace the placeholder element for the first customizable visual element with a first variation of the first customizable visual element corresponding to the first selectable option, based on a determination that the individual selectable option is a first selectable option of the one or more first selectable options; and
A method further comprising: updating the first individual representation of the first graphical object to replace the placeholder element for the second customizable visual element with a first variation of the second customizable visual element corresponding to the second selectable option, based on a determination that the individual selectable option is a second selectable option of the one or more second selectable options. In paragraph 4,
After updating said first individual representation of said first graphical object with said first variation of said first customizable visual element corresponding to said first selectable option, receiving a third user input selecting a third selectable option from among said one or more first selectable options via said one or more input devices; and
A method further comprising, in response to receiving said third user input, updating said first individual representation of said first graphical object to replace said first variation of said first customizable visual element with a second variation of said first customizable visual element that is different from said first variation of said first customizable visual element corresponding to said third selectable option. In the first paragraph,
While the customization state of the first graphical object is a partial customization state and the first individual representation of the first graphical object is displayed with a first variation for the first customizable visual element and a placeholder element for the second customizable visual element, receiving a second user input via the one or more input devices corresponding to a request to dismiss the customizable user interface;
In response to receiving said second user input, a step of stopping display of said customized user interface;
After stopping the display of the customized user interface and while the customized state of the first graphical object is in the partially customized state, receiving a third user input selecting the first selectable representation of the first graphical object via the one or more input devices; and
In response to receiving said third user input, further comprising the step of displaying said customized user interface through said display device,
A method wherein said first individual representation of said first graphical object is displayed in said customizable user interface as said first variation for said first customizable visual element and as said placeholder element for said second customizable visual element. In the first paragraph,
A method wherein, in response to a determination that the plurality of customizable visual elements include a third customizable visual element, the customizable user interface includes a fourth section including one or more third selectable options for customizing the third customizable visual element. In the first paragraph,
A method wherein said customizable user interface comprises a second individual representation of said first graphical object corresponding to predefined variations of said first customizable visual element and said second customizable visual element. In the first paragraph,
receiving a second user input selecting the first individual representation of the first graphic object via the one or more input devices while the customization state of the first graphic object is not a completed customization state; and
In response to receiving the second user input,
Upon determining that the first customizable visual element is not customized, a step of displaying a visual indication via the display device that the first customizable visual element is not customized; and
A method further comprising: displaying, via the display device, a visual indication that the second customizable visual element is not customized, upon a determination that the second customizable visual element is not customized. In Article 9,
The visual indication that the first customizable visual element is not customized comprises animating a change in a visual characteristic of the one or more first selectable options for customizing the first customizable visual element;
A method wherein the visual indication that the second customizable visual element is not customized comprises animating a change in a visual characteristic of the one or more second selectable options for customizing the second customizable visual element. In the first paragraph,
While the customization state of the first graphical object is a completed customization state in which an individual variation of the first customizable visual element and an individual variation of the second customizable visual element have been selected, receiving a second user input, via the one or more input devices, selecting the first individual representation of the first graphical object; and
A method further comprising, in response to receiving said second user input, inserting into said text input area said first graphical object having said individual variation of said first customizable visual element and said individual variation of said second customizable visual element. In Article 11,
A method further comprising: receiving said second user input selecting said first individual representation of said first graphical object; updating said first selectable representation of said first graphical object on said virtual keyboard to display a representation of said first graphical object having said individual variation of said first customizable visual element and said individual variation of said second customizable visual element. In the first paragraph,
While the customization state of the first graphic object is in a completed customization state, receiving a second user input selecting the first selectable representation of the first graphic object via the one or more input devices; and
In response to receiving the second user input,
inserting the first graphical object into the text input area without displaying the customized user interface, based on a determination that the second user input includes selection of the first selectable representation of the first graphical object for less than a predetermined duration threshold; and
A method further comprising: displaying, via the display device, the customized user interface without inserting the first graphical object into the text input area, upon determining that the second user input comprises selection of the first selectable representation of the first graphical object for a period exceeding the predetermined duration threshold. In Article 13,
The step of inserting the first graphic object into the text input area is:
inserting the first graphical object having the default variations into the text input area, based on a determination that the first selectable representation of the first graphical object is displayed with default variations for the first customizable visual element and the second customizable visual element; and
A method comprising: inserting the first graphical object having the customized variations into the text input area, wherein the first selectable representation of the first graphical object is displayed with customized variations for the first customizable visual element and the second customizable visual element. In the first paragraph,
wherein said first user input comprises a touchdown and liftoff of a contact from a touch-sensitive surface of said one or more input devices, and wherein said customized user interface is displayed after said touchdown of said contact on said touch-sensitive surface and before said liftoff of said contact from said touch-sensitive surface;
The above method,
detecting a lift-off of said contact from said touch-sensitive surface while displaying said customized user interface; and
A method further comprising: maintaining display of the customized user interface via the display device in response to detecting the lift-off of the contact from the touch-sensitive surface. In the first paragraph,
A method wherein said customized user interface is overlaid over said user interface including said text input area and said virtual keyboard. In the first paragraph,
In response to a determination that the customized user interface comprises less than a predetermined amount of a display area of the display device, the customized user interface is overlaid over the user interface including the text input area and the virtual keyboard;
A method wherein the customized user interface replaces the display of the user interface, based on a determination that the customized user interface includes more than the predetermined amount of the display area of the display device. In the first paragraph,
While displaying the customized user interface, receiving a second user input corresponding to a location outside the customized user interface via the one or more input devices; and
A method further comprising: in response to receiving said second user input, stopping display of said customized user interface. In the first paragraph,
The plurality of selectable representations of the plurality of graphic objects included in the virtual keyboard further include a second selectable representation of a second graphic object,
A method wherein the second graphic object is not associated with a plurality of customizable visual elements. In Article 19,
receiving a second user input selecting an individual selectable expression of the plurality of selectable expressions on the virtual keyboard via the one or more input devices, wherein the individual selectable expressions are associated with individual graphical objects; and
In response to receiving the second user input,
In response to a determination that the individual graphical object is associated with a plurality of customizable visual elements and that the one or more first criteria are satisfied, displaying, via the display device, a first customizable user interface including selectable options for customizing the plurality of customizable visual elements of the individual graphical object, and an individual representation of the individual graphical object reflecting the plurality of customizable visual elements based on the selectable options, wherein the individual representation of the individual graphical object is selectable to insert the individual representation within the text input area;
displaying, via the display device, a second customized user interface comprising individual representations corresponding to individual variations of the one customizable visual element, wherein the individual representations corresponding to the individual variations are selectable to customize the one customizable visual element of the individual graphical object and to insert the individual graphical object into the text input area, upon a determination that the individual graphical object is associated with only one customizable visual element and that one or more second criteria are satisfied; and
A method further comprising the step of inserting the individual graphical object into the text input area without displaying a customizable user interface via the display device, based on a determination that the individual graphical object is not associated with any customizable visual elements. As an electronic device,
One or more processors;
memory; and
Contains one or more programs,
The one or more programs are stored in the memory and configured to be executed by the one or more processors,
An electronic device, wherein said one or more programs include commands for performing the method of any one of claims 1, 2, 4 to 20. A non-transitory computer-readable storage medium storing one or more programs,
A non-transitory computer-readable storage medium, wherein said one or more programs include instructions that, when executed by one or more processors of an electronic device, cause the electronic device to perform the method of any one of claims 1, 2, 4 to 20. delete delete delete delete delete delete

Images