CN102068237A - Systems and methods for monitoring eye movement, systems and methods for controlling computing devices - Google Patents

Abstract

The invention provides a system and a method for monitoring eye movement and a system and a method for controlling a computing device. The system for monitoring eye movement comprises: means for wearing on a person's head; a light source for directing light towards the eye of the person when the apparatus is worn; a camera on the device for monitoring the movement of the eye, the camera for generating a video signal representing a video image of the eye acquired by the camera; a processor in communication with the camera for processing the video signal and identifying a pupil of the eye from the video signal; and a display connected to the processor for displaying a video image of the eye from the video signal processed by the processor, the display superimposing a graphic onto the video image to identify the pupil.

Description

System and method for monitoring eye movement, system and method for controlling computing device

This application is based on the divisional application of the early Chinese patent application No. 200580017565.X of the international patent application PCT/US2005/011104 filed on April 1, 2005. The title of the invention of the original application is "Biological Sensors, Communicators, and Controllers and Methods of Use". the

technical field

The present invention generally relates to devices, systems, and methods for monitoring the movement of a human eye, for example, for monitoring fatigue, intentional communication, and/or for monitoring motion based on the eyes, eyelids, and/or one or both eyes of a person. Apparatus, systems and methods for controlling the motion of other components of a device. the

Background technique

It has been proposed to use the movement of a person's eyes to monitor unconscious states, such as the person's wakefulness or drowsiness. For example, US Patent No. 3,863,243 discloses a device that can sound an alarm to warn a person using the device that he is about to fall asleep. The device includes a frame similar to a set of lenses on which optical fibers and photocells are mounted, wherein when the frame is worn, the lenses face the user's eyes. The photoelectric element detects the intensity of light reflected by the human eye, that is, the intensity of light reflected by the eyelid when the eye is closed or by the eye surface when the eye is open. The timer differentiates between the duration of normal blinking and eye closure periods (ie, representing the time a person falls asleep). When a critical time has elapsed, an alarm is issued to notify and/or wake up the user. the

Another device is the Alterness Monitor from MTI Research Inc., which can be mounted on safety glass and along the eyelid axis at strategic locations where the beam is not interrupted by eyelashes except during blinking. Sends out a continuous beam of infrared light, giving it the ability to measure the frequency of eye blinks. Other devices, such as those disclosed in US Pat. Nos. 5,469,143 and 4,359,724, directly use the user's eyelids or eyebrows to detect eye movement and activate an alarm when a drowsy state is detected. Such devices may include mechanical devices, such as robotic arms, or piezoelectric membranes resting on the eyelids. the

It has been proposed to install cameras or other devices on the dashboard, roof, or other locations of vehicles to monitor the driver's awareness. However, such devices require the user's eyes to remain in constant contact with the camera. Additionally, they cannot monitor eyelid movement if the user turns their head sideways or down, swivels, out of the vehicle, if the user moves back and forth quickly, or if the camera moves relative to the person. Additionally, such cameras may be invasive of privacy and/or have problems seeing through glasses, sunglasses, or even contact lenses, and they do not operate effectively in daylight. the

Contents of the invention

The present invention relates to devices, systems, and methods for monitoring the movement of one or more eyes, eyelids, and/or pupils of a subject. Typically, humans blink at least about 5-30 times per minute, or about 7,000-43,000 times per day. Each involuntary blink lasts approximately 200-300 milliseconds, with an overall average of approximately 250 milliseconds, resulting in approximately 1,750-10,800 seconds of eye closure due to involuntary blinking per day. When fatigue or drowsiness occurs, the blinks may be longer and slower and/or the blink rate may vary, and/or the eyelids may begin to droop gradually due to small eyelid blinks, for example, until the eyes begin to slump due to short-term "microsleeping" (that is, a sleep condition lasting about 3-5 seconds or longer), or due to a prolonged sleep. Additionally, the pupil may constrict more slowly, exhibit erratic fluctuations in size, gradually contract in diameter, and/or exhibit a delayed response to light flashes as sleepiness and fatigue progress (ie, delayed pupillary response latency). In addition, other visual manifestations of drowsiness can occur, such as slow or delayed saccade eye-tracking responses to, for example, stimuli (i.e., delayed saccade response latencies), and and/or inability to direct gaze regardless of converging or diverging eyes, misalignment, or esophoria. the

In one embodiment, an apparatus for monitoring eyelid, pupil, and/or eye movement is provided, comprising: a device for wearing on the user's head; a light source for directing light to the person when the device is worn and first and second fiber optic bundles connected to the device, the first fiber optic bundle for viewing the first eye of the person wearing the device and the second fiber optic bundle for viewing the second eye of the person wearing the device . The device may also include a camera coupled to the first and second fiber optic bundles for acquiring images of the first and second eyes. the

Optionally, the device may also include a third optical beam positioned away from the user, for example, for viewing an area where the user's head will turn. Additionally or alternatively, the device may have one or more spatial sensors. The camera can be connected to first and second fiber optic bundles for capturing images of the first and second eyes, and to a third fiber optic bundle for capturing images of the area the user's head and/or eyes are facing . Spatial sensors may allow simultaneous measurement or tracking of, for example, a user's head movement relative to the user's eye movement. In addition, a transmitter and/or sensor array connected to the camera may allow the measurement of various eye movement parameters of one or both eyes, such as velocity, acceleration and deceleration of the eyelids, blink rate, "PERCLOS" (time the eyelids are open) % of the eye), the vertical height of the palpebral fissure (i.e., the area between the eyelids not covering the pupil), eg, the distance or percentage associated with a fully open eye, etc. the

In another embodiment, there is provided a self-contained device for detecting movement of a person's eyelids, the self-contained device comprising: a device for wearing on a person's head; a transmitter located on the device for directing light towards a person's eye when the device is worn; and a camera for detecting light from the emitter. The sensor generates an output signal that indicates when the person's eyes are open or closed, and a transmitter on the frame is connected to the sensor for wireless transmission of the output signal to a remote location. The spectacle frame may also include a processor for comparing the output signal to a predetermined threshold to detect eyelid movement caused by drowsiness. Similar to the previous embodiments, the emitters and sensors may be solid-state biosensor devices for emitting and detecting infrared light, or alternatively arrays, for example, one-dimensional or two-dimensional arrays of emitters and/or sensors pre-configured on the frame. A dimensional array, for example, a vertical, horizontal, diagonal, or other linear or other geometric array of more than one emitter and/or sensor positioned toward one or both eyes. In particular, an array of emitters and/or sensors may allow measurement of eye movement parameters, such as those identified herein. the

Transmitters and/or sensors may be affixed to various points on the frame, for example, around the lenses and/or in the bridge of the nose, or alternatively anywhere along the frame, including on the nose guard portion of the frame, the frame The fittings of the temple pieces, and/or are mounted near or on top of the surface of the lenses of the eyeglasses. Optionally, emitters and/or sensors can be mounted into the lenses of the glasses so that they can be operated through the lenses. Thus, emitters and/or sensors can be mounted on eyeglass frames so that they can move with the wearer's head movement and stay focused on the user's eyes in any position, whether the user is in a car, outdoors, etc. or any other environment. the

In another embodiment, a system for monitoring human eye movement is provided. The system includes: a device for wearing on a person's head; one or more emitters on the device for directing light to the person's eyes when the device is worn; and a camera, such as a CCD or CMOS device. Emitters can be used to project a frame of reference towards the eye. A camera may be positioned toward the eye for monitoring eye movement relative to a frame of reference. The camera may be located on the device, or may be located at a location remote from the device, but relatively closer to the user. the

Light from the emitter can be directed at the eyes of a user wearing the device to illuminate the user's eyes while projecting a frame of reference onto the eyes. The emitter may project light towards the user "invisibly", that is, outside the scotopic (night vision) or photopic (photopic) range of normal vision, e.g., in the infrared range, thereby enabling illumination and/or Or the frame of reference basically does not interfere with the user's vision. The camera can image the light produced by the emitter, for example, in the infrared range, thereby detecting the projected light as spots, bands or other "reflections". The movement of the eyes relative to the frame of reference can be monitored using a camera. Graphical output of motion monitored by the camera may be monitored, for example, relative to a frame of reference projected onto the eye. For example, infrared light from the emitter can be reflected off the retina as a "red reflex" in the case of white light, and as a white or dark pupil in the case of infrared light (including using the pupils known in the art). image of a dark pupil using the subtractive method). the

For example, a processor using one or more of these methods may detect movement of the pupil of the eye, eg, measure movement relative to a frame of reference. This movement can be displayed graphically, showing the movement of the pupil of the eye relative to the frame of reference. Optionally, output signals from one or more sensors may be correlated with video signals produced by cameras monitoring eye movement relative to a frame of reference, for example, to determine a person's level of drowsiness, or to correlate with information about consciousness. A state related to mental or neuro-physiological cognition, emotion, and/or vigilance. the

In another embodiment, a method is provided for controlling a computing device or other electronic or electro-mechanical device (e.g., radio, television, wheelchair, telephone, alarm system, auditory , visual or tactile alarm systems, etc.). The apparatus may include one or more components, similar to other embodiments described herein, including a camera having at least one objective lens focused on at least one eye of the user. A computing device may include a display including a pointer displayed on the display. Displays can include: heads-up or heads-up displays, attached to, or attached to, the user's head; desktop computer monitors, placed in front of the user and digitally projecting an image on a screen (eg, like in a driving or flight simulator), etc. The user's eye movement can be monitored using a camera, and the eye movement can be associated with a pointer on the display so that the pointer can follow the eye movement, eg, similar to a computer mouse. Optionally, the camera may monitor the user's eyes for predetermined eye movements, e.g., blinking for a predetermined length of time, which may correspond to instructions for executing one or more commands identified by a pointer on the display, e.g., with a computer The "double click" of the mouse is similar. the

Other aspects and features of the present invention will be better understood through the following detailed description in conjunction with the accompanying drawings. the

Description of drawings

1 is a perspective view of a patient wearing an embodiment of an apparatus for monitoring a patient based on movement of the patient's eyes and/or eyelids in a hospital;

Figure 2 is an enlarged perspective view of the embodiment of Figure 1 comprising a detection device and a processing box;

Figure 3 is a schematic diagram of an exemplary embodiment of a circuit for sending an output signal corresponding to a series of eyelid movements;

Figure 4 is a schematic diagram of an exemplary embodiment of circuitry for controlling equipment in response to output signals corresponding to a series of eyelid movements;

Figure 5 is a schematic diagram of an exemplary embodiment of a circuit for detecting eyelid movement;

6A-6C are cross-sectional and front views of an alternative embodiment of an apparatus for emitting light toward and detecting light reflected from a surface of an open eye;

7A-7C are cross-sectional and frontal views, respectively, of the apparatus for emitting light toward and detecting light reflected from a closed eyelid in FIGS. 6A-6C;

Figure 8 is a perspective and block diagram of another embodiment of a system for monitoring a user based on movement of the user's eyes and/or eyelids;

Figure 9 is a block diagram of components of another embodiment of a system for monitoring a user based on movement of the user's eyes and/or eyelids;

Figure 10A is a perspective view of yet another embodiment of a system for monitoring a user based on movement of the user's eyes and/or eyelids;

Figure 10B is a schematic detail view of a portion of the system shown in Figure 10A;

Figure 10C is a detailed view of an exemplary array of emitters and sensors that may be placed on the nose bridge of an eyeglass frame such as in Figure 10A;

Figure 10D is a cross-sectional view of the array of emitters and sensors shown in Figure 10C for emitting light and detecting light reflected from the eye;

11A is a schematic diagram of a system for selectively controlling multiple devices from a remote location based on eyelid movement;

Figure 11B is a schematic diagram of an additional device that may be controlled by the system shown in Figure 11A;

Figure 12A is a table showing the relationship between the activation of the sensor array shown in Figures 10A-10D and the eye being monitored by the array as the eye changes between the open and closed states;

Figure 12B is a graph showing the data flow provided by the sensor array as shown in Figures 10A-10D, the graph showing the percentage of eye field of view ("PERCLOS") as a function of time;

Figure 12C is a graphical display of multiple physiological parameters (including PERCLOS) of a person being monitored by a system including a device such as that shown in Figures 10A-10D;

Figure 12D is a table showing the relationship between the activation of the two-dimensional sensor array and the eye being monitored as the eye changes between open and closed states;

13 is a perspective view of another system for monitoring a user based on user eye and/or eyelid movement;

Fig. 14 is the detailed view of the camera on the picture frame shown in Fig. 13;

15A-15I are graphical displays of various parameters that can be monitored using the system shown in FIG. 13;

Figure 16 is a detailed view of the video output from the camera on the frame shown in Figure 13;

17 is a schematic diagram showing an exemplary embodiment of a circuit for processing signals from a five-element sensor array;

Figures 18A and 18B show another embodiment of a device inserted into a pilot's helmet for monitoring eye movement;

Figure 19 is a schematic diagram of a camera that may be included in the device shown in Figures 18A and 18B;

20A and 20B are diagrams showing simultaneous output from multiple cameras showing the user's eyes open and closed, respectively;

21A-21C are graphical displays showing oval figures created to identify pupil circumference to facilitate monitoring of eye movement;

22A and 22B are flow diagrams illustrating a method for vigilance testing a user wearing a device for monitoring user eye movement;

23 is a flowchart illustrating a method for controlling a computing device based on eye movement; and

Figure 24 is a view of a device for transcutaneously emitting light into the eye and detecting the emitted light exiting the pupil of the eye. the

Detailed ways

Referring to the drawings, FIG. 1 shows a patient 10 on a hospital bed 12 wearing a detection device 30 for detecting eye and/or eyelid movement of the patient 10 . Detection means 30 may include any of the biosensor means described herein, which may be used to monitor, for example, intentional eye movement for intentional communication of information, for monitoring unintentional eye movement (e.g., drowsiness or other conditions), and and/or for controlling one or more electronic devices (not shown). The detection device 30 may be connected to a processing box 130 for converting detected eye and/or eyelid movements into a data stream, an intelligible message, and/or into other information, which may communicate, for example, using a video display 50 Medical care provider 40 communication. the

Referring to FIGS. 2 , 6A , and 7A , an exemplary embodiment of an apparatus or system 14 is shown that includes an aimable and focusable detection device 30 connectable to a commonly used pair of eyeglasses 20 . Eyeglasses 20 include a pair of lenses 21 attached to a frame 22 that includes a bridgework 24 extending between the lenses 21, and side members or temple pieces that support ear pieces 26. (temple piece)25, they are all commonly used. Optionally, the frame 22 may not have the lens 21 because the lens 21 is not necessary. the

The detection device 30 includes a clamp or other mechanism 27 for attachment to one of the side beams 25 and an adjustable arm 31 on which is mounted one or more transmitters 32 and sensors 33 (not shown). Emitter 32 and sensor 33 are mounted in a predetermined relationship such that emitter 32 can send a signal to eye 300 of a person wearing glasses 20 and sensor 33 can detect signals reflected from the surface of eye 300 and eyelid 302 . In the exemplary embodiment shown in FIGS. 6A and 7A , emitter 32 and sensor 33 may be mounted adjacent to each other. the

Alternatively, for example, as shown in FIGS. 6B and 7B , the emitter 32 ′ and the sensor 33 ′ can be mounted on the frame separately from each other so that the emitter 32 ′ and the sensor 33 ′ are substantially transverse relative to each other. place. In another alternative, as shown in FIGS. 6C and 7C , the emitter 32 ″ and the sensor 33 ″ may be mounted across the eye 300 in axial alignment with each other in the axial direction. When the eyelid 302 is closed, it may interrupt the beam 340 detected by the sensor 33".

In one embodiment, emitter 32 and sensor 33 respectively generate and detect continuous or pulsed light, eg, in the infrared range, to minimize disruption or interference with the wearer's normal vision. The emitter 32 can emit pulsed light of a predetermined frequency, and the sensor 33 is used to detect the light pulse of the predetermined frequency. This pulsing operation can reduce the energy consumption of the emitter 32 and/or can minimize interference with other light sources. Alternatively, light of other predetermined frequency ranges outside or within the visible spectrum, such as ultraviolet light, or other forms of energy, such as radio waves, sound waves, etc., may be used. the

The processing box 130 is connected to the detection device 30 by a cable 34 comprising one or more metal wires (not shown). As shown in FIG. 9 , processing box 130 may include a central processing unit (CPU) 140 and/or other circuitry, such as the exemplary circuitry shown in FIGS. 3-5 , for receiving and/or processing output from sensors 33 Signal 142 (such as a light intensity signal). The processing box 130 may also include a control circuit 141 for controlling the emitter 32 and/or the sensor 33, or the CPU 140 may include an internal control circuit. the

For example, in one embodiment, control circuit 141 may control transmitter 32 to generate pulses at a predetermined frequency (up to thousands of pulses per second to as little as 4-5 pulses per second, e.g., a minimum of about 5-5 pulses per second) 20 pulses) to facilitate the detection of inadvertent or intentional blinks of only about 200 milliseconds per blink. The sensor 33 can be controlled to detect only pulses of light pulsed at a predetermined frequency specific to the flashing frequency of the emitter 32 . Thus, by synchronizing the transmitter 32 and sensor 33 to a predetermined frequency, the system 10 can be used in a variety of environmental conditions without the output signal 142 being substantially affected by, for example, bright sunlight, total darkness, ambient infrared light background , or other transmitters operating at different flicker frequencies. Blink frequency can be adjusted to maximize the number of blinks per unit of time (e.g., about ten to twenty blinks per minute), the duration of each blink (e.g., about 200 milliseconds to approximately 300 milliseconds), and/or PERCLOS (ie, the percentage of time the eyelids are fully or partially closed), or to maximize the efficiency of the system. the

Control circuit 141 and/or processing box 130 may include manual and/or software adjustment means (not shown) for adjusting the frequency, focus, or intensity of light emitted by emitter 32 to turn off or on emitter 32, Adjusting the threshold sensitivity of the sensor 33, and/or for autofocusing with maximum infrared reflection off the closed eyelid will be understood by those skilled in the art. the

In addition, the processing box 130 may also include a power supply 160 for providing power to the transmitter 32, the sensor 33, the CPU 144, and/or other components in the processing box 130. The process box 130 may be powered by a conventional DC battery (e.g., a nine-volt battery or a rechargeable lithium, cadmium, or hydrogen generating battery), and/or by a solar cell connected to or built into the system 14 . Optionally, an adapter (not shown) may be connected to the process box 130, such as a conventional AC adapter or a twelve volt vehicle ignition adapter. the

The CPU 140 may include a timer circuit 146 for comparing the length of the individual components of the output signal 142 to a predetermined threshold to distinguish between normal blinking and other eyelid movements. Timer circuit 146 may be a separate discrete component or may be built into CPU 140, as will be appreciated by those skilled in the art. CPU 140 may convert output signal 142 into a data stream 144 that may be used to communicate with other persons or equipment. For example, data stream 144 generated by CPU 140 may be a binary signal, such as Morse code or ASCI code. Alternatively, CPU 140 may be capable of generating other outputs, such as synthesized speech signals, control signals for equipment, or graphical representations. the

To facilitate communication, processing box 130 may include various output devices for using data stream 144 . For example, an internal speaker 150 capable of generating an alarm tone or a synthesized voice may be provided. An output port 148 may be provided, which may be hardwired directly to various equipment, such as the video display 50 shown in FIG. 1 . the

Additionally or alternatively, processing box 130 may include a transmitter 152 coupled to CPU 144 for wireless communication of data stream 144 with a remote location. For example, as shown in FIG. 9, system 14 may include a receiving and processing unit 154, such as a computer or other control or display system. Transmitter 152 may be a radio frequency (" RF") transmitter. Optionally, other transmitters may be provided, eg infrared transmitters. the

Transmitter 152 may also be connected to an amplifier (not shown) to enable streaming of data hundreds or thousands of feet or more, for example using Bluetooth or other RF protocols. For example, the amplifier and transmitter 152 may communicate via telephone communication lines, satellite, etc., to transmit the data stream to a remote location at a greater distance from the system, where the data may be monitored, analyzed in real time, or stored (e.g. , stored in the truck or aircraft "black box" recorder) for future or retrospective analysis. The system may include, or may be connected to, a Global Positioning System (GPS) for monitoring the location, movement, and/or cognitive alertness, state of arousal, drowsiness, or emotional/behavioral performance of the individual wearing the detection device 30 and / or security. the

The receiving and processing unit 154 may include a receiver 156 (eg, a radio frequency receiver) for receiving a signal 153 transmitted by the transmitter 152, the signal comprising a data stream. Processor 158 is coupled to receiver 156 for interpreting, storing, and/or utilizing information in the data stream, and processor 158 is coupled to memory circuitry 160 , communication device 162 , and/or control system 164 . For example, receiving and processing unit 154 may include memory circuitry 160 in which processor 158 may simply store the data stream for subsequent retrieval and analysis. the

Processor 158 may translate the data stream by, for example, converting binary code in the data stream into an intelligible message (i.e., a series of letters, words, and/or commands), and/or may use a large communication device or software ( such as KE:NX or Words Plus) to facilitate communication. The resulting message may be displayed on the communication device 162, which may include a video display for displaying text, pictures and/or symbols, a synthesized speech module for providing electronic speech, and the like. the

Alternatively, the data stream can be displayed digitally as a "real-time" message signal graphically on a computer display or other electronic display device (e.g., showing blink rate, blink duration, PERCLOS, etc.), or with a graph similar to an EKG or EEG traces to display graphically. Additionally, as shown in Figure 12C, the data stream can be displayed along with other physiological data, such as skin conductance, body temperature, cardiovascular related data (e.g., heart rate, blood pressure), respiratory related data (e.g., respiration rate, oxygen in the blood, and carbon dioxide levels), electromyography (EMG) and/or actigraphy data sets (ie, body movement, position), and/or other polysomnographic (PSG) or electroencephalographic (EEG) variables. Optionally, the data stream can be combined with a controller for monitoring vehicle or machinery operation (e.g., vehicle speed, acceleration, brake function, torque, roll or tilt, engine or motor speed, etc.) and/or vehicle conditions, and functions related to the state of the driver's or machine operator's awareness, arousal, attention, and/or alertness responses manifested in real time, make an informed decision about whether to slow down or accelerate the vehicle. the

Additionally, messages may be interpreted by processor 158 for directing control system 164 to control one or more pieces of machinery or equipment. For example, data streams may include instructions for directing control system 164 to control relay switches or other devices to turn off and on electrical devices (such as appliances, electric wheelchairs, motors, lights, alarms, phones, televisions, computers, tactile vibrating seats, etc.) , or commands for operating an eye-triggered computer mouse or other controller. the

Alternatively, processor 158 may use data streams to control PCs, IBM, Macintosh, and other computers, and/or compatible computer software and/or hardware, for example, to communicate with objects similar to a mouse, "enter" key, and/or "joystick" input devices. For example, the data stream may include commands for launching commonly used commercially available software and computer games, as well as those used in special communication software (such as WORDS-PLUS or Ke:Nx) from which submenus or individual programs can be selected. A sequence of menu items. Processor 158 may control, scroll, or select items of a computer software program, operate a printer, or other peripheral device (e.g., select font, paragraph, markup, or other symbolic operators, select items such as "Edit," "Find," " Format", "Insert", "Help", or to control the operation of a CD-ROM or disk drive, and/or other Windows and non-Windows functions). the

Alternatively, receiver 156 may be directly connected to various devices (not shown), such as radio or television controls, lights, fans, heaters, motors, vibrating tactile seats, remotely controlled vehicles, vehicle monitoring or control devices, Computers, printers, telephones, lifeline units, electronic toys, or large communication systems to provide a direct link between the user and the device. the

During use, the detection device 30 may be placed on the user's head, ie the detection device may be placed on the user's head by wearing glasses 20 as shown in FIG. 1 . The adjustable arm 31 and/or clamp 27 can be adjusted to optimally orient the emitter 32 and sensor 33 toward the user's eye 300 (shown in FIGS. 6A-6C and 7A-7C ). Emitter 32 may be activated to direct light beam 340 from emitter 32 to eye 300 . Next, the intensity and/or frequency of emitter 32 and/or the threshold sensitivity or other focus of sensor 33 may be adjusted (eg, manually or automatically using a self-adjusting feature). the

Due to the different reflective characteristics of the surface of the eye 300 itself and the eyelids 302, the intensity of light reflected by the eye 300 depends on whether the eye 300 is open or closed. For example, FIGS. 6A and 6B show an open eye where a ray of light 340 generated by emitter 32 hits the surface of the eye 300 itself and then spreads out as shown by ray 350 . Consequently, the resulting light intensity detected by the sensor 33 is relatively low, that is, the sensor 33 may not receive any actual feedback signal. the

In FIGS. 7A and 7B , the eye 300 is shown with the eyelids 302 closed, which may be closed when blinking normally, while dozing off, blinking intentionally, or other eyelid movements. Because light 340 hits eyelid 302, substantially all of the light is reflected back to sensor 33, as indicated by ray 360, causing sensor 33 to detect a relatively high light intensity. Alternatively, the light beam 340 may be interrupted or cut by the eyelid 302 when the eye 300 is closed, as shown in FIG. 7C . the

Accordingly, the sensor 33 generates a light intensity signal that indicates when the eye 300 is open or closed, ie corresponding to times when the sensor 33 detects no or detects reflected light, respectively. Typically, the intensity of infrared light reflected from the eyelid surface is largely unaffected by skin pigment. If it is desired to adjust the intensity of light reflected from the eyelids, foil, glitter, reflective moisturizer, etc. may be applied to increase reflectivity, or black eyeliner, absorbing or reflective cream, etc. may be applied to decrease reflectivity. the

Referring to FIG. 9, the light intensity detected by the sensor 33 generates an output signal 142 comprising a train of time-varying light intensity signals (eg, as shown in FIG. 12B). The output signals 142 are received by a CPU 140 connected to the sensor 33, which compares the time length of each light intensity signal 142 corresponding, for example, to a closed eye state with a predetermined threshold. The timing circuit 146 may provide the critical time for the CPU 140 to distinguish normal blinks from voluntary and/or other involuntary eyelid movements, from which the CPU 140 then filters the output signal 142. Next, CPU 140 generates a data stream 144 that can be used for intentional and/or unintentional communication. the

In one useful application, the detection device 30 may be used to detect the approaching drowsiness or "microsleep" of a user through the processing box 130 for triggering an alarm to alert the user and others in the location and for monitoring the onset of drowsiness. ” (i.e., entering the wakeful sleep state for several minutes). The threshold of the timing circuit 146 can be adjusted so that the CPU 140 can detect relatively long periods of eye closure, such as may occur when a person falls asleep. the

For example, because the normal blink time is relatively short, the threshold can be set to a time ranging from close to zero seconds to a few seconds, for example, at about two hundred to three hundred milliseconds (200-300ms), or, at other In one embodiment, for example, approximately two hundred and fifty milliseconds (250 ms) to distinguish normal blinks from eyelid movements caused by drowsiness. When the CPU 140 detects a drowsy state, ie, detects a high brightness intensity signal exceeding a predetermined critical time, it may activate an alarm device. An alarm device such as a speaker 150 may be included in the processing box 130, or alternatively provided (for example) by mounting an alarm light (not shown) or an alarm speaker (not shown in FIG. 9 , see FIG. 10C ) on the mirror frame. on the frame. In another alternative embodiment, the alerting device may be a tactile device such as a vibrating seat or the like described elsewhere herein. the

Alternatively, the detection device 30 can be used to silently record or monitor sleepiness-induced eyelid movement by the CPU 140, which generates a data stream 144 that can be transmitted by a transmitter 152 to a receiving and processing unit 154 (FIG. 9). For example, device 30 may be used in conjunction with vehicle safety systems to monitor a driver's level of awareness or attention. The data stream 144 may be communicated to a receiving and processing unit 154 installed in the vehicle, which may store data regarding driver drowsiness and/or may use the data to make decisions in response to predetermined algorithms to control the vehicle, e.g., to adjust vehicle speed, Even turn off the vehicle engine. Thus, the detection device 30 may be used to monitor truck drivers, taxi drivers, crew members or pilots, train conductors or engineers, radar or airport control tower operators, operators of heavy equipment or factory machinery, divers, students, astronauts, recreational participant or observer etc. the

Detection device 30 and system 14 may also be used in medical diagnostic, therapeutic, research, or professional settings to monitor wakefulness, sleep patterns, and/or stress states or stimulants (e.g., caffeine, nicotine, dextroamphetamine, , methylphenidate, modafinil), the sympathetic and parasympathetic effects of sedatives (eg, benzodiazepine, Ambien), or the circadian stimulating effects of light and dark or melatonin, which can affect patients Or the blink rate, blink rate, blink duration, or PERCLOS of the vehicle operator. Signals regarding changes in trends measured over time can be stored and analyzed in real time to predict drowsiness effects in individuals using the device. the

Similar to the methods described above, the CPU 140 can generate a data stream 144 that the transmitter can send to a remote receiving and processing unit 154. Receiving and processing unit 154 may store data stream 144 in storage circuitry 160 for later retrieval and analysis by researchers, medical professionals, or security personnel (e.g., as opposed to storing flight recorder data in an aircraft "black box" logger in a similar way). The receiving and processing unit 154 may also display the data stream 144, such as at a nursing station, as an additional parameter to continuously monitor the patient's physical, mental, or emotional state. Unit 154 may store and/or generate a signal that must be responded to (eg, exhibiting vigilance monitoring) within a predetermined time, for example, through a series of algorithms to prevent false positives and false negatives. the

A variety of medical conditions can be monitored by the detection device 30 and system 14, such as petit mal seizures (in which the eyes blink at a rate of approximately 3 revolutions per second), grand mal seizures, or psychometric seizures (where the eyes can blink in Repeated staring or closing in a saccade pattern), myoclonic seizures (where the eyelids can open and close in a saccade pattern), or spasms, or other eye movements such as those experienced by people with Tourette's syndrome. The system can be used to monitor g-LOC (loss of consciousness) of a pilot due to positive or negative G-effects, hypoxemia of passengers or crew in an aircraft due to loss of cabin pressure, nitrogen anesthesia of divers or Flexion, or the effects of gases, chemicals, narcotics, and/or biological agents on military personnel or others. the

The system can also be used to monitor mental states, for example, during hypnosis, to detect stress or whether a person is lying (for example, closing or moving their eyes when lying), to monitor attention, and to measure one of the following: One or more: "Negative" side effects and/or "Positive" therapeutic effects of anesthesia or agents where the function of the eye is compromised (e.g. L-dopa for improving blink rate in Parkinson's disease; for the treatment of conditions such as ALS eye spasms or myasthenia gravis or anesthetics for neuromuscular disorders); anesthetics or alcohol markers based on correlative eye measurements (eg, nystagmus to flashes or delayed pupillary response); anticonvulsants, anesthetics, alcohol, toxins negative effects, or the effects of hypoxia or oxygenation, etc. Patients of all ages can also be monitored for neurological conditions that can affect the neurological or mechanical effects of the eye or eyelids, such as those with Parkinson's disease, muscular disorders (e.g., myotonia, tense muscular dystrophy, facial spasms), phobias Patients with photosensitivity or photosensitivity, encephalopathy, seizures, Bell's palsy, or where the condition can cause vision loss (eg, macular degeneration), ptosis, or drooping (such as third cranial nerve palsy or partial paralysis) , brainstem damage or stroke, neoplasm, infectious disease, metabolic disease, trauma, degenerative conditions (eg, multiple sclerosis), amyotrophic lateral sclerosis, polyneuropathy, myasthenia gravis, food poisoning, tetanus, tetany, tardy Dyskinesia, brainstem encephalitis, and other primary eyelid conditions such as exophthalmos, thyrotoxicosis, or other thyroid. In a similar manner, detection device 30 may be used in an ambulatory fashion to study the progression and/or regression of any of the above neuro-ophthalmic and ophthalmic disorders. the

Similarly, detection device 30 may be used in biofeedback applications, eg, biofeedback, hypnosis, or psychotherapy for certain diseases (eg, tic disorders). The detection device 30 may generate a stimulus (eg, activate a light or a speaker), and may receive a response in advance (eg, a specific sequence of eye blinks) and then determine the stimulus within a predetermined time to monitor the user's eyelid movement. If the user fails to respond, the processor may store the response, eg, including the response time, and/or may automatically send a signal, such as an alert signal. the

Additionally, the detection device 30 may be used to monitor individuals in non-medical settings, such as during normal activities in a user's home or anywhere else. For example, individuals with unconscious medical disorders such as epilepsy or narcolepsy may be monitored, or individuals such as infants and children, prison inmates, mentally ill patients (e.g., with Alazheimer's disease), law enforcement officers, military personnel, bank tellers clerks, clerks, casino workers, students, middle or night shift workers, and more. A similar application can be applied to sleep testing for monitoring sleeping patients during multiple sleep recording procedures (e.g., PSG, MSLT, or MWT) to measure things such as sleep onset, sleep latency, eyelid opening or closing Parameters such as the time of waking up at night, the time of awakening at night, etc., are used in animal tests that use blinking, pupil changes, and/or slow or fast eye movements as research aspects, or in the development of visual nerves in infants. the

Detection device 30 may be used to study or monitor the drowsiness, wakefulness, or alarming effects of prescribed drugs (eg, stimulants), alcohol or other illicit narcotics, toxins, poisons, and other relaxation, sedation, or alarm techniques or devices. Similarly, user performance and vigilance can be tested and analyzed as a direct function of PERCLOS, or in relation to PERCLOS. the

When the CPU 140 detects the occurrence of specific eyelid movements, such as prolonged eye closure, which may occur, for example, during a seizure, during syncope, during narcolepsy, or when dozing off while driving or working , the CPU 140 can generate an output signal that initiates an alarm. Alternatively, the transmitter 152 may emit an output signal to turn off the power being used, to notify medical personnel, such as by automatically activating a phone to dial emergency services, to signal a remote site, such as a police station, ambulance, vehicle Control Center, Admin, and more. the

System 14 may also provide efficient applications for intentional communications. A user wearing detection device 30 may consciously blink in a predetermined pattern, eg, in Morse code or other blink codes, to convey an intelligible message to a person or device (eg, to notify an emergency). The CPU 140 may convert the light intensity signal 142 received from the sensor 33 and corresponding to the blink code into a data stream 144, or possibly directly into an incomprehensible message comprising letters, words and/or commands. the

Next, data stream 144 may be displayed on video display 50 (see FIG. 1 ) connected to output 148 or emitted as synthesized speech on internal speaker 150 . The data stream 144 may be sent by a transmitter 152 via a signal 153 to a receiving and processing unit 154 for displaying messages, or controlling a device connected to a control system 164, such as a household device. In addition to residential settings, the system 14 may also be used by a medically or nursing caregiver, for example, by an intubated, ventilated, restrained, paralyzed or debilitated patient, to communicate with attending medical staff and/or to consciously Signal to the nurse's station. These individuals include all patients who do not have the physical ability to communicate verbally, but who retain the ability to communicate using the blink of one or both eyes (e.g., those with ALS, transverse myelitis, obstructive syndrome, cerebrovascular stroke, advanced muscular dystrophy, and those on ventilators). the

The device can be used in any environment or field, for example, in water or other substantially transparent liquids. Additionally, device 30 may also be used as an emergency notification and/or discrete security appliance. If a situation occurs where normal communication (ie, speaking) is not a viable option, the device 30 may be worn by a person capable of normal speaking. For example, a bank or retail employee who is being robbed or is at the scene of a crime could intermittently blink a pre-programmed alarm to notify security personnel or call law enforcement. Alternatively, people with certain medical conditions may wear the device if they are physically incapacitated, ie unable to move to call emergency medical care, but still able to actively move their eyes. In this case, a pre-recorded message or identifying data (eg, user name, location, type of emergency) may be sent to the remote location via a predetermined blink code set or pre-programmed message. In this manner, the detection device 30 can be used to monitor patients in an ICU environment, patients on ventilators, prisoners, the elderly or disabled, heavy equipment operators, truck drivers, motorists, ship and aircraft pilots, train Drivers, radar or aircraft control tower operators, or as a nonverbal or subconscious tool of communication for military guards, bank tellers on duty, shop assistants, taxi drivers, etc. The detection device 30 can also be used as an entertainment device, for example, as a children's toy similar to a walkie-talkie, or for operating a remote control toy vehicle. the

Additionally, the CPU 140 is required to perform additional threshold comparisons to ensure continued use of the detection device 30. For example, an additional timing circuit can be connected to the CPU 140 so that the CPU 140 can compare the light intensity signal received from the sensor 33 with a second predetermined threshold provided by the timing circuit. The second predetermined threshold may correspond to a period of time during which a person normally blinks. If CPU 140 fails to detect a normal blink within this time period or if the user fails to respond to a predetermined stimulus (e.g., a flash of light or a sound), CPU 140 may generate a signal to activate speaker 150 or use transmitter 152 to send an alarm. the

This may be useful, for example, to prevent "false alarms" or to measure a user's "attention state" if the detection device 30 is removed by a criminal during a crime, but falls due to a medical event. Optionally, the user is required to perform a vigilance figure to determine whether the transmitted signal is meaningful or a "false alarm" signal, to measure attention or drowsiness levels for biofeedback, and also to measure the compliance of the user wearing the device . the

Optionally, the polarity of the output signal 142 can be reversed so that the data stream is generated only when the eyes are open, for example, when monitoring patients in a sleep laboratory to measure sleep onset, sleep latency, time to eyelid closure, etc. , or monitor sleeping prisoners. It will be appreciated by those skilled in the art that for these uses, the CPU 140 can only activate the alarm when detecting an open eye state. the

Referring to Figure 8, another embodiment of a detection device 30 is shown. In this embodiment, both the emitter and sensor are separate solid state light emitting and detecting biosensor devices 132 that are mounted directly on the glasses 20 . The biosensor device 132 that generates and detects infrared light can be as small as 2 millimeters by 4 millimeters (2 mm x 4 mm) and weigh only a few grams, thereby enhancing the convenience, comfort and/or discrimination of the detection device 30 . Due to the small size, the biosensor device 133 can be directly installed in the lens 21, as shown in FIG. Another location that facilitates eye movement detection. The biosensor device 132 can measure a surface area of less than about 5 millimeters by 5 millimeters, and can weigh only about an ounce, thereby providing a transmitter/sensor that can be unobtrusive, portable, and conveniently incorporated into lightweight eyeglass frames. sensor combination. Because the entire system can be mounted on the frame, it can move with the user regardless of the direction the user is looking, and can be operated in various situations or fields such as day or night, underwater, etc. the

Hamamatsu manufactures various infrared emitter and monitor devices that can be used as biosensor devices 132, such as Model Nos. L1909, L1915-01, L2791-02, L2792-02, L2959, and 5482 -No. 11, or alternatively, a Radio Shack infrared transmitter (Radio Shack infrared emitter), Model No. 274-142 may be used. Suitable multi-component rays for use (e.g., linear light scanning sensor rays) are available from Texas Advanced Optoelectronic Solutions, Inc. (TAOS) of Plano, TX, such as Model No. TSL 201 (64 pixels x 1 pixel), TSL No. 202 ( 128×1), TSL 208(512×1), TSL 2301(102×1). These sensors can be used in conjunction with mirror arrays to facilitate the focusing of the detected light, such as the Selfoc mirror arrays for line scan applications manufactured by NSG America, Inc. of Irvine, CA. the

In addition, a plurality of biosensor devices 132 may be provided on the glasses 20, for example, as shown in FIG. 8, a pair of biosensor devices 132 may be provided for detecting the eyelid movement of each eye of the user (not shown). Cables 134 may extend from each biosensor device 132 to a processing tank 130 similar to the processing tank 130 described above. CPU 140 (not shown in FIG. 8 ) of processing box 130 may receive output signals from each biosensor device 132 and compare the output signals to further facilitate differentiating other eyelid movements from normal eye blinks. the

Those skilled in the art will appreciate that the pair of biosensor devices 132 may allow the user to use more complex codes, such as blinking each eye individually or together, for more efficient or convenient communication. In one form, a blink of one eye could correspond to a "dot", while a blink of the other eye could correspond to a "dash", to facilitate the use of Morse codes. The signals output from each eye can be interpreted by the CPU 140 and then converted into intelligible information. the

In another form, a blink (or continuous blink) of the right eye can cause the electric wheelchair to move to the right, a blink of the left eye (or continuous blinks) can move the wheelchair to the left, and two simultaneous right and left eye blinks One movement can cause the wheelchair to move forward, and/or four simultaneous left and right eye blinks can cause the wheelchair to move backward. A similar combination or sequence of eye blinks can be used to control on/off functions, or volume or channel controls for televisions, AM/FM radio stations, VCRs, tape recorders or other electronic or electromechanical devices, any large communication or control device, Or any device that can be operated by a simple "on/off" switch (e.g. wireless TV remote control single switch TV control unit, universal remote control, single switch with AC plug/wall outlet or wall switch module multiple application units, computer input adapters, ignition signaling buzzers or vibration signal boxes, all types of switch modules, video game entertainment controller switch modules and electronic toys controlled by switches). the

In another option, one or more lenses or filters may be provided to control the light emitted and/or detected by the biosensor device, individual emitters, and/or detectors. For example, prisms or other lenses may be used to change the angle of emitted light, or light may be collected or focused through slits to produce a predetermined shaped beam of light directed at the eye or reflected by a sensor. An adjustable lens array may be provided to control the shape (eg width) of the emitted beam, or to adjust the sensitivity of the sensor. Those skilled in the art will appreciate that the lens may be packaged with the transmitter in plastic or the like, or provided as a separate accessory. the

Referring to FIG. 10A, another embodiment of a system 414 is shown comprising a frame 422 having a biosensor device 432 with an associated processor and transmitter circuitry 430 disposed directly on the frame 422, e.g., the frame 422 Used to increase the convenience and discretion of the system 414. The mirror frame 422 may include a bridge 424 and a pair of ear supports 423 , 425 , wherein the biosensor device 432 may be fixedly, movably, and/or adjustably mounted on the bridge 424 . the

One of the earstays 423 may have a larger size than the other earstay 425, for example, to accommodate the processor and transmitter circuitry 430 plugged into or mounted thereon. A processor 440 similar to the CPU 140 in the aforementioned processing box 130 may be provided on the mirror frame 422, and a power source such as a lithium battery 460 may be inserted or attached to the bracket 423. A radio frequency or other transmitter 452 (e.g., using Bluetooth or other protocols) including an antenna 453 is disposed on the ear support 423, wherein the antenna may be embedded or fixed in a temple piece in the frame 422 along the ear support 423 or otherwise location. the

The system 414 may also include a manual adjustment device (not shown) on the ear support 423 on the frame 422 or elsewhere (not shown), for example, to power off and on, or to adjust the intensity and/or threshold of the biosensor 432 . Thus, system 414 may be substantially self-contained on frame 422 which may or may not include lenses (not shown) similar to eyeglasses. External cables or wires can be eliminated, providing a more convenient and comfortable system for communicating and/or monitoring the user. the

In another alternative embodiment, as shown in FIGS. 10B, 10C, and 10D, a linear array 530 of emitters 532 and sensors 533 may be arranged, for example, mounted to the eyepiece frame 522 in a vertical arrangement. On the bridge of the nose 524 . CPU 540, battery 460, transmitter antenna 543, and alarm indicator 550 may also be provided on frame 522, for example, on temple piece 525, similar to the previously described embodiments. LEDs 542 or similar stimulation devices may also be provided at predetermined locations on the spectacle frame 522 to enable conventional biofeedback responses to be obtained from the user. Additionally, a receiver 544 may be provided for receiving the data stream generated by the CPU 540 and sent by the transmitter 543. the

As particularly shown in FIG. 10C , each of the sensor 533 and the emitter 532 may be connected to a CPU 540 or other control for controlling the emitter 532 and/or for processing the light intensity signal generated by the sensor 532. circuit. Thus, the CPU 540 may cycle through the sensors 533 in the array 530, processing the signals from each sensor 533 in turn, similar to the processors described elsewhere herein. As shown in FIG. 10D , emitter 532 includes optics 534 for focusing light beams (represented by respective rays 360 a , 360 b ) onto eye 300 , eg, toward pupil 301 . The sensors 533 are included in the bridge of the nose 524 , and one slit 535 is provided for each sensor, the slits 535 having predetermined dimensions to control the reflected light detected by each sensor 533 . Thus, each sensor 535 can detect movement of the eyelid 302 over a particular portion of the eye 300, for example, measuring PERCLOS as shown in FIG. 12A . A sensor or emitter may have optics or focusing means to focus emitted or reflected light. the

In addition to slight eye closure, the linear array 530 also facilitates the measurement of other parameters related to eyelid movement, for example, to measure the speed at which the eyelid opens or closes, i.e., the rate at which the eye closes, the CPU 540 can compare the The time delay between the startup process. Additionally, the signal output from sensor 553 may be processed to measure the percentage of pupil coverage of eyelid 302 as a function of time (e.g., due to partial eye closure), e.g., to monitor when the eye is partially rather than fully closed, and/or To monitor the percentage of eye closure time relative to the user's maximum eye open benchmark (PERCLOS), as shown in Figures 12A-12C. the

Referring to Figure 12D, in another embodiment, a two-dimensional sensor array may be provided. While a 5x5 array 633 and a 9x11 array 733 are shown as exemplary embodiments, other arrays including any number of elements in the array may also be provided. For example, as will be described further below, the sensor may be in the form of a CMOS or CCD device comprising hundreds or thousands of pixels in a grid or other pattern. The sensors 633, 733 can then be used to measure the surface area reflectance of light from the emitter 632, i.e., a processor (not shown) can process the signals from each sensor in the array 633, 733 to generate The percentage of surface area of the eye 300 covered by the eyelid 302 and/or the relative position of the pupil is streamed. the

The sensors in the arrays 633, 733 may be sufficiently sensitive or of sufficient resolution that they can detect "red reflections" or equivalent infrared "bright pupil" reflections due to the reflection of light reflected from the retina through the pupil 301. Thus, the sensor may generate a light intensity signal including a value of substantially zero (which indicates no red reflex or a bright pupil), a low output (indicating a red reflex or a white pupil reflex), and a high output (indicating a reflection of a closed eyelid 302) . The red reflex can appear as a bright white pupil (generated by infrared light from the emitter reflecting off the retina when the eyelids are open, or as a dark or "black pupil" if the processor uses subtraction). The processor can process the light intensity signal to detect when the pupil 301 is covered by the eyelid 302, i.e. at what point in time the user cannot see even though the user's eye 300 is not completely covered by the eyelid 302, typically this PERCLOS value accounts for The original staring percentage was 50-75. Optionally, when the eyelids, eyes, and pupils are drooping, the sensor can detect a red reflex or a bright pupil even though the PERCLOS measurement can reach a percentage of 75-80 or greater, e.g. When looking down, look through a palpebral fissure as narrow as a slit. the

In another alternative embodiment, a processor and/or transmitter circuit (such as CPU 140 in processor box 130 shown in FIG. 2 , or CPUs 440, 540 shown in FIGS. 10A and 10B ) may include A separate memory slice or other circuit element, or an identification circuit (not shown) exists within the CPU itself. The identification circuitry may be pre-programmed with a fixed identification code or be programmable, for example, to include selected identification information such as the user's identity, user location, user-specific biometric measurements (e.g., a unique iris or retina pattern, Fingerprint, voice identification), and identification codes for each detection device, etc. the

The CPU may selectively add identifying information to the transmitted data stream 553, or identifying information may be included in the data stream 553 automatically or periodically, continuously or discontinuously, thereby enabling the data stream 553 to be associated with a particular detection device, personality users, and/or specific locations. The identification information may be used by a processor (e.g., a processor at a remote location) to differentiate between data streams received from multiple detection devices, which may then be stored, displayed, etc., as described above wait. Thus, the detection device may not require the user to consciously communicate certain identifying or other standard information when using the system. the

As shown in FIG. 11A , the receiver 544 may allow a user to control one or more devices connected to the receiver 544 through a single switch multi-application control unit 550 . The control unit 550 includes its own transmitter suitable for sending on/off or other control signals which may be received by the respective control modules 552a-55sf. The user 10 can blink to generate a data stream 553 to be sent including on and off commands, or use the control unit 550 and control modules 552a-552f to control selected devices, such as a radio 554, television 556, lights 558a, A light 562 controlled by a mounted switch 560, a fan 566 plugged into a wall socket 564, etc. the

Optionally, as shown in FIG. 11B , receiver 554 may interface with other systems, such as computer 570 and printer 572 , vehicle integration system 574 , lifeline unit 576 , GPS or other satellite transmitter 578 , and the like. The transmitted data stream 553 may be processed alone or with other data such as other vehicle sensor information 573, and/or human factors, such as human factors such as : EKG, EEG, EOG, pulse, blood pressure, respiration rate, oximetry, actigraphy, head position, voice analysis, body temperature, skin conductance, self-assessment measurement and performance vigilance response, others through fixed unavailable Observations made with control pads or helmet-mounted camera systems, and more. the

Referring to Figure 13, another embodiment of a system 810 for monitoring eye movement is shown. Generally, the system 810 includes: a frame 812, which may include a bridge 814 and a pair of ear supports 816; one or more transmitters 820; one or more sensors 822; and/or one or more cameras 830 , 840. Frame 812 may include a pair of lenses (not shown), such as regular, tinted, or protective lenses, although they may be omitted. Alternatively, the system can be provided on other devices that can be worn on the user's head (such as: pilot's oxygen mask, protective eye gear, patient respirator, scuba or swimming mask, helmet, cap, headband, helmet, protective headgear), or placed in a sealed suit for protecting the head and/or face, etc. (not shown). Components of the system may be located at various locations on the device that generally minimize interference with the user's vision and/or with normal use of the device. the

As shown, arrays of emitters 820 are disposed on mirror frame 812 in, for example, vertical array 820a and horizontal array 820b. Additionally or alternatively, emitters 820 may be arranged in other configurations, such as circular arrays (not shown), and may or may not include filters and/or diffusers (also not shown). In an exemplary embodiment, emitter 820 is an infrared emitter for emitting pulses at a predetermined frequency, similar to other embodiments described elsewhere herein. Emitter 820 may be positioned on the frame such that it projects a frame of reference over the area including one of the user's eyes. As shown, the frame of reference includes a pair of intersecting bands 850a, 850b that divide the area into four quadrants. In an exemplary embodiment, the point of intersection of the crossing strips may be positioned at a location that substantially corresponds to the pupil of the eye at the original gaze (ie, when the user is looking substantially straight ahead). Alternatively, other frames of reference may be provided, for example, frames of reference that include vertical and horizontal components, angular and radial components, or other rectangular components. Optionally, even one or two reference points that remain substantially fixed may provide a sufficient frame of reference for determining relative motion of the eyes, as explained further below. the

An array of sensors 822 may also be provided on the frame 812 for detecting light from the emitters 820 that reflects off the user's eyelids. Sensor 822 may generate an output signal having an intensity for identifying whether the eyelid is closed or open, similar to other embodiments described elsewhere herein. Sensors 822 may be placed near each emitter 820 for detecting light reflected off various portions of the eyelid. Alternatively, sensors 822 may be provided in a vertical array only (eg, along bridge 814 ) for monitoring the number of eyelid closures, similar to embodiments described elsewhere herein. In another alternative embodiment, emitter 820 and sensor 822 may be a solid-state biosensor (not shown) that has both emitting and sensing functions in a single device. Optionally, emitter 820 and/or sensor 822 may be eliminated, eg, when cameras 830, 840 provide sufficient information, as further explained below. the

Circuitry and/or software can be configured to measure PERCLOS or other parameters using the signals generated by the sensor array. For example, Figure 17 shows an exemplary schematic that may be used to process signals from a quintuple array, for example, to obtain PERCLOS measurements or other alarm parameters. the

Referring to FIG. 13, the system 810 also includes one or more cameras 830 positioned generally toward one or both eyes of the user. Each camera 830 may include a fiber optic bundle 832 that includes a first optical fiber mounted on or near bridge 814 (or elsewhere on frame 812, e.g., in a location that minimizes interference with the user's vision). terminal, and a second terminal 837 connected to a detector 838 (eg, a CCD or CMOS sensor) that can convert the image into a digital video signal. As shown in FIG. 14 , for example, an objective lens 834 may be disposed on a first end of the fiber optic bundle 832 to focus an image onto the fiber optic bundle 832 . Optionally, as also shown in FIG. 14 , fiber optic bundle 832 may include one or more illumination fibers terminating proximate to lens 834 to locate emitter 836 . An illumination fiber can be connected to a light source (not shown), for example, similar to the embodiment shown in Figures 22A and 22B and described further below. While only one camera 830 is shown in FIG. 13 (e.g., for monitoring the user's left eye), it should be understood that similar components (e.g., fiber optic bundles, lenses, transmitters, and/or or detector (although, as described below, the cameras may share a common detector) another camera (not shown) for monitoring the user's other eye (eg, the right eye). Optionally, there may be a need for multiple cameras (not shown) facing each eye, for example from different angles facing the eye. Optionally, these cameras may include fiber optic extensions, prism sheets, and/or mirrors (eg, to reflect infrared light), opaque or closed mirrors on the side of the lens facing the eye, and the like. These attachments can be set up to bend, rotate, reflect, or invert the image that is ideally sent to the camera's eye. the

Camera 830 may be used to detect the frequency of light (eg, infrared light or other light outside the visible range) emitted by emitters 820 and/or 836 . Optionally, the emitter 820 on the frame 812 can be eliminated if the fiber optic bundle 832 includes one or more illumination fibers for the emitter 836 . In this embodiment, it is also possible to remove the sensor 822 and use a camera 830 to monitor the user's eye movement, eg, as further explained below. Optionally, for example, the system 810 may include a second camera 840 positioned towards a direction away from the user's head for monitoring the user's surroundings, such an area being directly in front of the user's face. Camera 840 may include components similar to those of camera 830, such as a fiber optic bundle 841, a lens (not shown), and/or a transmitter (also not shown). Optionally, the camera 830 may be sensitive enough to produce an image under ambient lighting conditions, and the emitter may be omitted. As shown in Figure 13, the camera 840 may be connected to a separate monitor 839, or may share a detector 838 with the camera 830, as further described below. the

For example, each of fiber optic bundles 832, 841 may include between five thousand and one hundred thousand (5,000-100,000) pixel-carrying fibers, or between ten thousand and fifty thousand (10,000-50,000) optical fiber. The number of fibers can be determined according to the specific needs of a given application (e.g., to provide the desired optical fiber in the acquired image of the user's eye (i.e., for the "internal camera" fiber) and the surrounding environment (i.e., the user's "external camera" fiber). optical resolution). Optionally, the fibers of the fiber optic bundle may include one or more illumination fibers. In an exemplary embodiment, five thousand (5,000) fibers (providing 75 x 75 pixel resolution), ten thousand (10,000) fibers (providing 100 x 100 pixel resolution), fifty thousand (50,000 ) fibers (providing a pixel resolution of 224×224), and fiber bundles of one hundred thousand (100,000) fibers (providing a pixel resolution of 316×316). The resulting fiber optic bundles 832, 841 may have, for example, a diameter between 3 and 5 millimeters (3-5 mm), with or without cladding. The fiber optic bundles 832 , 841 may be fixed along the spectacle frame 812 or may be disposed within the spectacle frame 812 . For example, the frame 812 may be formed with one or more channels extending, eg, from the bridge 814 to the ear support 816, for receiving the fiber optic bundles 832, 841 therethrough. Optionally, the fiber optic bundles 832, 841 may be cast, fused, or embedded in the frame 812, for example, when the frame 812 is manufactured. Optionally, fiber optic bundles 832, 841 may extend from the mirror frame 812 to a detector and/or processor (not shown) separate from the mirror frame 812, similar to the embodiment described below with reference to Figures 18A and 18B. the

One or both ear braces 816 may include a panel 818 for housing one or more components (e.g., a controller or processor such as the exemplary processor 842), a transmitter 844, an antenna 845, detectors 838, 389, and/or battery 846. Processor 840 may be coupled to emitter 820, sensor 822, and/or cameras 830, 840 (eg, to detectors 838, 839) for controlling their operation. As described below, the transmitter 844 may be connected, for example, to the processor 842 and/or detectors 838, 839 for receiving output signals from the sensor 822 and/or cameras 830, 840 for sending the signals to a remote location. Alternatively, the transmitter 844 may be connected directly to the output leads from the sensor 822 and/or cameras 830, 840. For example, the frame 812 may also include manual controls (not shown) on the earstays 816, for example, to turn power on and off, or to adjust the intensity of the sensors 822, cameras 830, 840, and/or transmitter 820 and/or critical values. the

For example, system 810 may also include one or more additional sensors on frame 812, if desired, for the synthesis and cross-correlation of additional biological or neurophysiological data related to the user's perceptual, emotional, and/or behavioral states (eg, physiological sensors). The sensors can be connected to the processor 842 and/or the transmitter 844 to monitor, record, and/or transmit signals from the sensors to a remote location. For example, one or more position sensors 852a, 852b may be provided, eg, for determining the spatial orientation of the spectacle frame 812, and thus the spatial orientation of the user's head. For example, a change recording sensor may be provided to measure head tilt or movement, for example, to monitor whether a user's head is drooping forward or tilted to the side. An acoustic sensor (eg, microphone 854) may be provided for detecting ambient noise or sounds produced by the user. the

Additionally or alternatively, for example, for physiological system integration and/or cross-correlation, the frame 812 may include one or more sensors for measuring one or more physical characteristics of the user. For example, EEG electrodes 856 may be placed on ear braces 816, above or below the nasion, on the mastoid behind the ear, on the occipital region, and/or other areas that may contact the user's skin (e.g., wet surface-contacting electrodes), or may not touch the user's skin (e.g., dry wireless electrodes), to measure and transmit brain activity at different frequencies ranging from about one to five hundred Hertz (1-500 Hz) (e.g., awake, sleepy, or other sleep-related brain activity) for visual, short-term, or long-term spatial and/or temporal trend analysis (eg, fast Fourier or spectral analysis). EKG electrodes (not shown) may be provided to measure the heartbeat through the skin contact point. Cardiovascular beats may be measured using a pulse sensor (not shown), or oxygen saturation may be measured using an oximetry sensor 858 . A thermistor or other sensor can measure, for example, the airflow of breathing through the user's nose. A thermistor, thermocouple, or other temperature sensor (not shown) may be provided for measuring the user's skin temperature. A moisture detector (sweat detector) (not shown) may be provided to measure the humidity on the user's skin, and/or a microphone or other acoustic sensor (also not shown) may be attached to the frame 812 to detect the user's mouth or breath sounds. One or more electrooculographic (EOG) electrodes may be placed in contact with the user's skin at desired areas to measure fluctuations in electrical potential during eye movement. the

Additionally, system 810 may include one or more feedback devices on frame 812 . These devices may provide feedback to the user, for example, to alert and/or wake the user when a predetermined condition is detected (eg, a state of drowsiness or lack of consciousness). The feedback device may be connected to the processor 842 for controlling its activation. For example, the mechanical vibration device 860 that can provide tactile vibration stimulation through skin contact can be provided at a position that can contact the user, for example, on the ear support 816 . Electrodes (not shown) capable of generating relatively low power electrical stimulation may be provided. Visible white or colored light emitters such as one or more LEDs may be provided at desired locations, eg, on bridge 814 . Optionally, an audio device 862 such as a buzzer or other alarm may be provided, similar to other embodiments described elsewhere herein. In another optional embodiment, the scent emitter may be disposed on the frame 810, for example, on or near the bridge 814. the

Additionally or alternatively, one or more feedback devices may be provided separate from the frame 812, but in a manner that provides a feedback response to the user. For example, audio, video, tactile (eg, vibrating seat), or olfactory transmitters, such as any of the devices described elsewhere herein, may be placed in the user's vicinity. In another optional embodiment, a heating or cooling device capable of generating a temperature stimulus to the user may be provided, such as a remote-controlled fan or air conditioning unit. the

The system 810 may also include components remote from the frame 812, similar to other embodiments described elsewhere herein. For example, system 810 may include a receiver, processor, and/or display (not shown) at a location remote from frame 812 (e.g., in the same room, at a nearby monitoring station, or at a remote location). ). The receiver may receive signals sent by the transmitter 842 , including signals output from the sensor 822 , the cameras 830 , 840 , or any other sensor disposed on the frame 812 . the

The processor may be connected to a receiver for analyzing signals from components on the frame 812, for example, to modulate signals for graphical display. For example, the processor may modulate signals from the sensor 822 and/or cameras 830, 840 for display on a monitor, thereby enabling the user to be monitored by others. At the same time, other parameters can be displayed on a single or separate display. For example, FIGS. 15A-15I show outputs indicative of various sensors that may be provided on the frame 812, which may be displayed along a common time axis, or otherwise, for example, in relation to the user's eye movement and/or drowsiness levels. Associated. The processor can overlay or otherwise simultaneously display the video signal along with other sensory parameters to enable a physician or other person to monitor and personally correlate these parameters with user behavior. the

In another optional embodiment, the processor may automatically process the signals to monitor and/or study user behavior. For example, the processor can use the output signal to monitor various eye movement parameters related to eye movement, such as blink duration (EBD), blink frequency, blink rate, blink acceleration, internal blink (interblink) time (IBD), PERCLOS, PEROP (percent eyelid opening), fluctuations in pupil size, eye gaze and eye movements, etc., are described in US Patent No. 6,542,081, the entire contents of which are incorporated herein by reference. the

Video signals from camera 830 may be processed to monitor various eye parameters, such as pupil size, position (eg, within the four quadrants defined by intersection band 850), eye tracking motion, eye gaze distance, and the like. For example, because camera 830 is capable of detecting light emitted from emitter 822, camera 830 may detect a frame of reference projected by the emitter onto the area of the user's eyes. Figure 16 shows an exemplary video output from a camera included in a system having 20 emitters arranged in a vertical array. The camera can detect twenty discrete light zones arranged as a vertical band. The camera can also detect "flickering" dots G, and/or moving bright pupils P. Thus, pupil movement may be monitored relative to glint point G, and/or relative to vertical bands 1-20. the

Because the emitter 822 is fixed to the frame 812, the frame of reference 850 may remain substantially fixed relative to the user. Thus, the processor may determine the location of the pupil from Cartesian coordinates (eg, x-y or angular extent) with respect to the frame of reference 850 . Alternatively, if the frame of reference is removed, the position of the pupil can be determined relative to any fixed stationary "blinking" point or other predetermined reference point on the user's eye. For example, the camera 830 itself may project a point of light onto the eye, where the point of light may be reflected and detected by the camera. Because the camera 830 is fixed to the frame 812, this "blinking" point can remain substantially fixed, thereby providing a desired reference point from which subsequent movements of the eye can be determined. the

Additionally, the video signal from a remote camera separate from the frame 812 may be imaged from a distance (e.g., on the dashboard of a car, driver, airliner, or other simulator, or in a sonar, radar, or other monitoring station) The user's face, for example, to monitor various facial measurements (such as facial expression, frequency of yawning, etc.), in addition, or alternatively, to monitor the frame of reference of the projected light from the emitter. Additionally or alternatively, parameters from other sensors, such as head orientation, tilt, body movement, physiological parameters, etc., may be processed and correlated. In one embodiment, the processor may correlate two or more of these parameters to generate a Composite Fatigue Index ("COFI"). For example, when blinking or covering pupils by eyelids exceeds a critical duration, the processor may monitor position sensors to detect head tilt and/or physiological sensors regarding brain activity, potentially indicating that the user is falling asleep or otherwise becoming able to Drive or operate equipment. The processor may use an empirical algorithm stored in memory to assign values to these parameters, and then add or otherwise correlate the parameters to assign a numerical COFI to the user's current state. the

When a predetermined COFI threshold is exceeded, the system 810 may activate an alarm or otherwise notify the user and/or another party at the remote location. Thus, the system 810 can provide a more efficient way to monitor a user's fatigue, drowsiness, vigilance, mental state, etc. In another optional embodiment, system 810 may be used to generate a predetermined output, eg, to activate or deactivate a device, such as a vehicle being operated by a user when system 810 determines a predetermined condition (eg, COFI value). the

Optionally, a processor may be provided on the spectacle frame 812, for example, as part of the processor 842, for monitoring parameters regarding a predetermined event to occur, such as exceeding a predetermined COFI threshold. In another alternative embodiment, the eye-tracking parameters described above may be monitored by a remote camera, for example, in a fixed location in front of the user, such as a vehicle's dashboard or the like. Remote cameras can be connected to the processor directly or via their own transmitters, which can also be integrated into the COFI determination and/or monitored by third parties along with algorithmically defined response measurements. Additional information regarding various devices, systems, and methods of use are found in U.S. Patent Nos. 6,163,281 (published December 19, 2000), 6,246,344 (published June 12, 2001), and 6,542,081 (published April 1, 2003) described in No., the entire contents of which are hereby expressly incorporated by reference. the

Referring to Figure 13, in an alternative embodiment, the cameras 832, 840 may be connected to a single detector (not shown), similar to the configuration shown in Figures 22A and 22B. Fiber optic bundles 832, 841 may be connected to one or more lenses for transmitting and/or focusing images from cameras 830, 840 onto various regions of the detector. The detector may be a CCD or CMOS chip with an active imaging area, for example, in cross-section between 5 and 10 millimeters (5-10 mm). In an exemplary embodiment, the effective imaging area of the detector may be square, rectangular, circular, or elliptical, as long as it has enough area to receive simultaneous images from the camera 830 and the camera 840 . Exemplary outputs showing synchronized video images from the cameras 830, 840 are shown in FIGS. 20A and 20B and are described further below. In this alternative embodiment, with sufficient resolution and processing, it is possible to eliminate emitter 820 and/or sensor 822 from system 810 . the

Referring to Figures 18A and 18B, another embodiment of a device 910 for monitoring eyelid movement of a person wearing the device 910 is shown. As described elsewhere herein, device 910 may be used as a biosensor, communicator, and/or controller, and/or may be included in a system, for example, to monitor intentional and/or or unintentional movement. the

As shown, device 910 includes a helmet 912 that may be worn on a user's head, and a biosensor assembly 920 . Helmet 912 may be a standard pilot's helmet, such as those used by helicopter or jet pilots, for example, including a pair of night vision tubes or other goggles 914 mounted thereon. Optionally, helmet 912 may include one or more hazard warning displays, such as small flat panel LCDs mounted in front of or near one or both eyes. Optionally, helmet 912 may be replaced with a mirror frame (not shown, see, e.g., FIG. Other examples described in . The frame may include a pair of lenses (also not shown), such as regular, tinted, and/or protective lenses, although they are not necessary for the operation of the device. Alternatively, one or both lenses may be replaced by a display (eg, a relatively small flat-panel LCD that may be used as a simulator and/or entertainment device, as explained further below). In another alternative embodiment, device 910 may include other devices that may be worn on the user's head, such as hats, caps, headbands, helmets, eye and head protection, face masks, oxygen masks, ventilation hoods , respirator or swimming cover etc. (not shown). the

Components of device 910 may be positioned at various locations on helmet 912 (or other head-worn device), for example, to generally minimize interference with a user's vision and/or normal activities while wearing device 910. As shown, biosensor assembly 920 includes camera 922 mounted on top of helmet 912, eg, using Velcro, straps, and/or other temporary or removable attachments (not shown). This allows the camera 922 to be removed when not in use. Alternatively, the camera 922 may be substantially permanently attached to the helmet 912, incorporated directly into the helmet 912 (or other frame), connected to a head-mounted television, LCD monitor, or other digital display, etc., as described herein Other examples described in are similar. the

Biosensor assembly 920 may also include one or more fiber optic bundles 924 extending from camera 922 to the front of helmet 912 to provide one or more "internal cameras" for imaging the user's eyes. As shown, a pair of fiber optic bundles 924 extending from the camera 922 to the respective tubes of the goggles 914 are shown. In an exemplary embodiment, fiber optic bundle 924 may be long enough (e.g., between 12 to 18 inches long) to extend from camera 922 to goggles 914, or alternatively, fiber optic bundle 924 may be longer (e.g., between about 12 and 18 inches in length). 2 to 4 feet), or could be shorter, depending on the location of the camera 922 on the helmet 910 (or if the camera 922 is placed separately from the helmet 910). the

An end 926 of the fiber optic bundle 924 may be permanently or removably connected to the goggle 914 , eg, to a bracket 916 connected to or otherwise extending from the goggle 914 . Optionally, fiber optic bundle 924 may be temporarily or substantially permanently secured to goggle 914 using clips, fasteners, adhesives, etc. (not shown). As shown, the end 926 of the fiber optic bundle 924 is mounted under the goggles 914 and angled upwardly toward the user's eyes. The angle of the tip 926 can be adjustable, for example, about 15 degrees up or down from a base angle of about forty-five degrees. Optionally, the end 926 of the fiber optic bundle 924 may be positioned elsewhere on the helmet 912 and/or goggles 914, but still directly toward the user's eyes. the

With additional reference to FIG. 19 , each fiber optic bundle 924 can include a fiber optic image guide 928 (i.e., an optical imaging fiber bundle), and an illumination fiber optic bundle 930 (e.g., a fiber optic bundle that is housed to extend between the camera 922 and the end 926 of the fiber optic bundle 924). shrink tube (not shown)). Each illumination fiber optic bundle 930 may include one or more optical fibers connected to a light source, eg, internal to camera 922 . For example, the camera 922 may include a light emitting diode (LED) housing 932 with one or more LEDs 934 (one shown for brevity), and an illumination fiber optic bundle 930 may be connected to the LED housing 932 to deliver light to the tip 926. the

The light emitted by light source 934 may be beyond the range of normal human vision, for example, in the infrared range (e.g., with a very small output wavelength between about 840 and 880 nanometers (840-880 nm)), such that the emitted light is substantially It will not interfere with the normal vision of the user. The light source may generate substantially continuous light or pulses of light at a desired frequency, similar to the embodiments described elsewhere herein. Alternatively, other light sources for illuminating the user's face and/or one or both eyes may be provided in place of the illuminating fiber optic bundle 930 . For example, similar to other embodiments described elsewhere herein, one or more transmitters (not shown) may be provided, for example, along one or more regions of the helmet 912 and/or visor 914 array. the

The end 926 of each fiber optic bundle 924 may include one or more lenses, such as an objective lens 936 (shown in FIG. 18A ), capable of focusing the image guide 928 in a desired manner, for example, toward a user's eye. Each image guide 928 may have a forward line of sight (zero degree (0°) field of view), and objective lens 936 may provide a wider field of view, eg, approximately forty-five degrees (45°). Optionally, the line of sight may be adjustable, for example, by adjusting the objective lens 936 between approximately 30 and 60 degrees (30-60°). Additionally, the objective lens 936 can optimize the viewing distance, eg, to about 2 inches (2 in.), thereby improving focus on the user's eyes. Thus, the image guide 928 may transmit the image of the user's eye to the camera 922 via the fiber optic bundle 924 . the

As shown in FIG. 19 , camera 922 may include one or more lenses (e.g., magnifying portion 928 ) for directing and/or focusing images from image guide 928 (and/or camera 944 ) onto an effective lens of imaging device 940 . Display area 942 above. Imaging device 940 may be a variety of known devices that provide a two-dimensional active display area for receiving images, such as a CMOS or CCD detector. In an exemplary embodiment, imaging device 940 may be a CMOS device such as Model cmos SamBaHR-130 manufactured by Sensovation, or Fast Camera 13, Model MI-MV13 manufactured by Micron Imaging. The magnifying part 938 may be mechanically paired with the camera 922 via C-tape or other connectors (not shown). the

In an exemplary embodiment, each image guide 928 is capable of providing up to ten thousand to fifty thousand (10,000 to 50,000) pixels of image data, e.g., similar to fiber optic bundles described elsewhere herein, which can be projected onto the active display area 942 of the imaging device 940 . For the device 910 shown in Figures 18A and 18B, the images from the two fiber optic bundles 924 are projected onto a single imaging device 940, as shown in Figure 19, that is, so that the images from each eye of the user The image occupies less than half of the active display area 942 . the

Optionally, device 910 may include an "external camera" 944 positioned away from the user's head, eg, to monitor the user's surroundings, similar to embodiments described elsewhere herein. For example, as shown in Figure 18A, another fiber optic bundle 945 extending from the camera 922 may be provided. As shown, fiber optic bundle 945 is oriented “forward”, ie generally in the same direction as when the user is looking straight ahead, and terminates at microlens 946 . Fiber optic bundle 945 may be relatively short and/or substantially fixed such that its field of view remains substantially constant relative to helmet 912 . Optionally, external camera 944 may be located elsewhere on helmet 912 and/or goggles 914 , eg, including a flexible fiber optic bundle, similar to external camera 840 described above. Thus, external camera 944 may provide an image away from the user, eg, directly in front of the user's face. the

External camera 944 may or may not include one or more illumination fibers, but may include an image guide connected to imaging device 940, for example, through magnification 938 or separately. Thus, images from external cameras 944 may be transmitted to the same active display area 942 as the images received from image guide 928 for each user's eye, similar to other embodiments described herein. Such a configuration may allow or facilitate temporal and/or spatial synchronization, or identification of "where" the user's eyes are looking, "looking" relative to the directional position of the user's head through "triangulation" or other algorithms for eye tracking. What", and/or "how long" (duration of gaze). the

Thus, camera 922 may simultaneously capture images from one or more "internal cameras," ie, from fiber optic bundle 924 and from external camera 944 . This ensures that the images captured by each device are synchronized with each other, ie linked together at the same time, so that an image of one eye taken at a particular time corresponds to an image of the other eye taken at substantially the same time. Additionally, these images can be substantially synchronized with data from other sensors (eg, one or more physiological sensors), which can enhance the ability to monitor and/or diagnose a user, and/or predict user behavior. Because of this synchronization, image data can be captured at a relatively high rate (eg, between approximately five hundred and seven hundred and fifty frames per second or Hertz (500-750 Hz)). Alternatively, a separate detector may be provided which captures image data which may be synchronized, for example, by a processor receiving the data. In this alternative embodiment, a slower capture rate (eg, between about 30 to 60 Hertz (30-60 Hz)) may be used to facilitate synchronization of the processor or other means used for capture afterwards. Optionally, the camera 922 and/or associated processor may be capable of capturing relatively slow vision, eg, at a rate between approximately fifteen and sixty (15-60) frames per second. the

20A and 20B illustrate exemplary outputs of a camera receiving synchronized image signals from two internal cameras 2010 and one external camera 2020 (or from a device for compiling images from a single camera and/or sensor). As shown, the inner cameras are directed toward each of the user's eyes, and the outer cameras are directed outward toward the user's surroundings (ie, generally directly in front of the user's face). In FIG. 20A, both of the user's eyes 2010L, 2010R are open, and the external image 2020 shows a horizontal view of the room in front of the user. In contrast, in Figure 20B, the user has one eye 2010L fully closed and the other eye 2010R partially closed such that the eyelid covers most of the pupil. External image 2020 shows the user's head beginning to tilt to the left and drooping forward. Optionally, additional information may be displayed and/or stored with these images (such as real-time data from other sensors on device 910), similar to that shown in FIGS. 12A-12C or 15A-15I. the

Referring to Figures 18A, 18B, and 19, images from camera 922 (and/or camera 944) may be sent from device 910 via cable 948 (best seen in Figure 18A). For example, imaging device 940 may convert optical images from active display area 942 into electrical signals, which may be sent via cable 948 to one or more processors and/or controllers (not shown), as described herein. Other embodiments described elsewhere are similar. Optionally, images from fiber optic bundle 924 and/or external camera 944 may be sent from device 910 to one or more remote devices, e.g., cameras, detectors, and/or processors (not shown), similar to those herein Other examples described in . In this alternative embodiment, the length of the fiber optic bundle 924 may be between about 2 and 6 feet, for example, to provide sufficient length to enable the user to move normally but still remain connected to the remote device. the

Alternatively or in addition, device 910 may include a wireless transmitter (not shown), such as a short-range or long-range radio frequency (RF) transmitter, that may connect to camera 922, for example, using Bluetooth or other protocols. The transmitter may be located in camera 922 or elsewhere on helmet 912 . A transmitter may send an image signal representing image data to a receiver at a remote location, similar to other embodiments described elsewhere herein. In another optional embodiment, in addition to the transmitter and/or cable 948, the device 910 may also include a memory (also not shown) for storing image data, or the transmitter and/or may be replaced by a memory or cables. For example, data could be stored in a recorder device, e.g. similar to a "black box" recorder used in an aircraft, to be retrieved at a later time, e.g., in the event of a vehicle accident , Medical events, etc., are analyzed afterward. the

Optionally, device 910 may include one or more controllers (not shown), eg, within camera 922 and/or in helmet 912 , for controlling various components of device 910 . For example, a controller may be coupled to one or more LEDs 934 to cause the LEDs 934 to pulse at a predetermined frequency, for example, to reduce power consumption of the device 910. Additionally, device 910 may include one or more power sources, such as batteries and/or cables, for providing power to one or more components of device 910 . For example, one or more batteries (not shown) may be provided in camera 922 for powering imaging device 940 and/or LED 934. the

Device 910 may also include one or more additional sensors (not shown), eg, located on helmet 910, if desired. The sensors may be connected to biosensor assembly 920, cable 948, and/or wireless transmitters (not shown), if included, so that signals from the sensors may be monitored, recorded, and/or sent to a remote location. For example, one or more position sensors may be provided, eg, to determine the spatial orientation of the helmet 912, and thus the spatial orientation of the user's head, as described elsewhere herein. Additionally or alternatively, device 910 may include: one or more physiological sensors (not shown), for example, for measuring one or more physical characteristics of the user; such as one or more EEG electrodes, EKG electrodes, EOG electrodes , pulse sensor, oxygen sensor, thermistor, thermocouple, or other temperature sensor, for example, to measure the temperature of the user's skin; a moisture detector, to measure the humidity of the user's skin; and/or to measure the humidity of the user's skin A sensor for airflow (eg, through the user's nose). the

Additionally, device 910 may include one or more feedback devices (also not shown), for example, to alert and/or wake up the user, such as mechanical vibration devices that may provide tactile vibration stimulation through skin contact, may generate relatively low power One or more electrodes for electrical stimulation, one or more light emitters (such as LEDs), audio devices, odor emitters, and the like. Optionally, a feedback device may be provided separately from device 910, but placed in such a way as to provide a feedback response to the user. For example, audio, video, tactile (eg, vibrating seat), or scent emitters may be placed near the user, such as any of the devices described above. In another optional embodiment, a heat generating or cooling device capable of generating temperature stimulation to the user may be provided, for example, a remote-controlled fan or an air conditioning unit. the

A system including device 910 may include components remote from device 910, similar to other embodiments described elsewhere herein. For example, the system may include one or more receivers, processors, and/or displays (not shown), located at a remote location away from the device 910, for example, in the same room, at a nearby monitoring station, or at a farther away. A receiver may receive signals sent by transmitters on device 910 , including image signals from camera 922 and/or signals from other sensors on device 910 . the

A processor may be connected to the receiver for analyzing the signal from the device 910, for example, to prepare the signal for graphical display. For example, the processor may prepare video signals from the camera 922 for display on a monitor, similar to the images shown in FIGS. 20A and 20B , thereby enabling the user to be monitored by a third party (e.g., doctor, administrator , or other colleagues, etc.). At the same time, other parameters may be displayed, either on a single monitor or on separate displays, similar to other embodiments described elsewhere herein. The processor may overlay or simultaneously display the displayed video signals of the user's eyes and/or external camera images either on their own or with other sensory parameters to enable a doctor or other person to monitor and personally correlate these parameters with the user's behavior. the

In another optional embodiment, the processor may automatically process the signals to monitor or study user behavior. For example, the processor may use the output signal to monitor various parameters related to eye movement, such as blink duration (EBD), blink frequency, blink rate, blink acceleration, internal blink duration (IBD), PERCLOS, PEROP (eyelid open percentage) and so on. The video signal from the camera 922 can be processed to monitor single or multiple eye parameters such as pupil size, relative position, eye tracking motion, eye gaze distance, continuously or discontinuously and/or sequentially in any combination or mode of acquisition etc., as described elsewhere in this article. Accordingly, the device 910 and/or system may monitor one or more visual or other parameters, such as disclosed in US Patent No. 6,542,081, the contents of which are incorporated herein by reference. the

To facilitate monitoring of pupil size (eg, dilation, constriction, and/or eccentricity) and/or eye movement, the system may include a processor in communication with camera 922 for processing video signals and identifying pupils of the eyes from the video signals. For example, using higher resolution cameras (such as CMOS and CCD detectors), the processor can identify the edges of the pupil and thus the circumference, diameter, and/or cross-sectional area of the pupil. A display may be coupled to the processor for displaying a video image of the eye from a video signal processed by the processor. the

Additionally, referring to FIGS. 21A-21C , the processor may overlay graphics on the display, eg, over a video image, to facilitate identification and/or monitoring of the pupil 301 of the eye 300 . As shown, because of the difference between the edge of the pupil 301 and the surrounding iris 304, the processor can simulate this boundary and produce a graphical halo, ellipse, or Other graphics 306 (only one eye 300 is shown in FIGS. 21A-21C for simplicity). Observers can use this graphic 306 for the convenience of the user of the monitoring device 910 . the

Additionally or alternatively, the processor may automatically analyze information related to the size and/or shape of the pupil 301 (or graphic 306) to correlate the video signals to determine the person's drowsiness level or other physical and/or mental state. This analysis may include monitoring the relative position of the pupils, the size of the pupils, and/or the eccentricity of the pupils, eg, over time. For example, the processor may monitor the diameter of the pupil 300 over time, which may be displayed graphically, e.g., as shown in FIG. 15E , stored in memory as a function of time, and/or overlaid in real time, for example. image of the eye. the

For example, FIG. 21A may show pupil 301 in a relaxed state under ambient conditions, eg, corresponding to figure 306 having diameter "d ₁ ". As shown in FIG. 21B, if the user blinks or closes the eye 300, the pupil 310 may dilate so that the pupil 301 begins to dilate when the eye 300 reopens, as shown by the graph 306 having a diameter " _d2 " . The processor may compare graph 306 or the change in diameter of pupil 301 itself to determine the delay of pupil 301 for returning to diameter "d ₁ " after blinking or other eye closure. This delay or loss of response to visible or invisible flashes of light may indicate, at least in part, the degree of drowsiness, the degree of injury, e.g., poisoning, and/or the occurrence of a medical accident, including, for example, due to hypoxemia, hypoglycemia, Fatal or late events of brain injury or brain death caused by stroke, myocardial infarction, toxins, poisons, etc.

Additionally or alternatively, the processor may determine the approximate eccentricity of the pupil, for example, when it is partially covered by the eyelid 302 . For example, as shown in FIG. 21C, when the eyelid 302 is partially closed, the halo 306 superimposed on the image may take an elliptical shape corresponding to the width "w" and height "h" of the exposed portion of the pupil 301. The height "h" may be related to the diameter "d ₁ ", i.e., the ratio of the height "h" to the diameter "d ₁ " may be equal to or less than 1 (h/d ₁ ≥ 1), as the extent to which the pupil 301 is covered by the eyelid 302 instructions. For example, this ratio may drop from 1 to 0 when the pupil 301 is fully covered by the eyelid 302 .

Similarly, the width "w" may also be related to the diameter "d ₁ " (w//d ₁ ≧1) as an indication of the extent to which the pupil 301 is covered by the eyelid 302, e.g., when the eyelid 302 begins to cover more than the pupil 301 half the time. Additionally or alternatively, the ratio of height to width (h/w≧1) may be correlated with eccentricity information of the pupil 301 , eg based on the coverage of the eyelid 302 . These parameters may be analyzed individually, collectively, and/or in combination with other eye movement and/or physiological parameters to monitor, analyze, and/or predict further user behavior. The data can be compared to empirical or other data maintained or accessed by the processor to provide information about the user's status (eg, COFI value), as described elsewhere herein.

For example, if analysis of the user's pupils results in a determination that the user's alertness level falls below a predetermined level, an alert may be provided to the user and/or one or more third parties, similar to other embodiments described herein. These methods may also be used to determine whether a user is under the influence of narcotics, alcohol, and/or a medical condition, as described elsewhere herein. the

Additionally, any device or system such as described herein may test the user's pupillary response, as a margin and/or to test the user's vigilance. Such testing can confirm that the user is active while wearing the device, ie, not falling asleep, or even dead. For example, if a light source flashes at a user's eye for a predetermined duration and/or pulse rate, the user's pupils may temporarily dilate from their relaxed state (based on ambient light) and then contract back for a predictable period of time. Relaxed state. the

Referring to FIG. 22A , an exemplary method for testing the vigilance of a user of any of the devices and systems described herein is shown. For example, a user may wear a device 810 shown in FIG. 18 for monitoring one or both eyes of the user (as further described above). At step 2210, the basis or parameters of the user's eyes may be determined in the relevant state. For example, the relaxed diameter of the pupil can be measured or monitored under ambient conditions. the

At step 2220, one or more pulses of light may be delivered to the eye, which may cause the eye to open and/or contract from a relaxed state, for example, at substantially the same frequency as the pulsed light flashes. For example, one or more transmitters on device 810 may be activated in a predetermined sequence to cause the eyes to open wide. Thereafter, in step 2230, the user's eyes may be subconsciously or unconsciously monitored, eg, using camera 830 or sensor 822, to determine a reaction time for the eyes to return to a relaxed state. The reaction time may be compared to an experience database or other data to confirm that the user is conscious, awake, and/or alive. Steps 2220 and 2230 may be repeated one or more times, if desired, to determine reaction times and/or provide average reaction times, if desired, eg, to avoid wrong negative decisions. the

If the light source is outside the visible range, eg in the infrared range, the pupil can still react to the flash in this way. One advantage of using infrared light is that it does not distract or disturb the user since the user will not be consciously observing the light. However, the pupils can still respond sufficiently to such flashes that a system for monitoring the eye can recognize when the pupils dilate and contract in response to these flashes. the

For example, during critical testing, it is sufficient to generate a flash of light as well as monitor pupillary responses. Alternatively, a burst of flashes can be used to monitor pupillary response over time, for example, to study trends or to weed out erroneous data that can be produced by a single flash. For a burst of flashes, the pulse frequency should be greater than the time required for the pupil to naturally return to its relaxed state after dilation in response to the flashes, eg, at least between about fifty and one hundred milliseconds (50-100 ms). Alternatively, pulses of light, eg, near infrared light (having a wavelength between about 640-700 nanometers) may be directed at the user's eye. The system can detect rhythmic fluctuations during pupillary responses. These responses can arise from an initial visual response and may be related to night vision, for example, "seeing" in the dark or sensing infrared light in the dark. the

Such pupillary response tests may also be used to identify false positives, eg, when the user is dead and the system does not detect any eye closure and/or movement. Similarly, the pupillary response test can also determine whether the user is asleep or unconscious. Additionally, the pupillary response test can be used to determine if the user is under the influence of alcohol, narcotics, etc., which may affect the rate at which the pupils contract back to their relaxed state after dilating in response to a flash of light. Additionally or alternatively, the pupillary response test may also be used to determine the blood level or amount of narcotics or alcohol in the user's body based on the relationship between eye movement tests and corresponding scientifically determined blood levels. the

Referring to Figure 22B, another method for testing borderline vigilance is shown. This method generally includes: step 2240, providing a stimulus for instructing the user to move their eyes in a desired manner on purpose; and step 2250, monitoring the eyes, e.g. Intentional movement. Any of the devices described herein may include one or more stimulation devices, such as speakers, lights, vibratory, or other tactile-type devices. Alternatively, these devices may be located remotely from the user, eg, on a vehicle dashboard, video display, etc. the

For example, if a visible light on the device is activated, the user may be instructed to close their eyes for a predetermined amount of time. Once the light is activated, the system can monitor the eyes to confirm that the user is responding within a predetermined time frame and/or in a predetermined manner (eg, blinking one or more times in a predetermined sequence). Alternatively, other stimuli may be provided in place of flashes, such as visual indications on a display (on or off the device), audible signals (e.g., verbal commands from a speaker on or near the device), tactile signals, etc. wait. In these embodiments, the user may be instructed to perform a sequence of actions, eg, look up or down, look left or right, blink in a desired sequence, close eyes until instructed, follow a pointer on the display, etc. For example, such tests may be used to confirm that a test subject is awake, conscious, and/or alert during a battery of tests or while performing various actions. the

In another embodiment, devices and systems such as those described elsewhere herein may be used to control a computer system, eg, similar to a computer mouse, joystick, and the like. For example, with reference to apparatus 810 shown and described in FIG. 13 , camera 830 may be used to monitor the position of a user's pupils to indicate and/or trigger a mouse pointer on a computer screen or other display. A processor for receiving image data from the camera 922 may analyze the image data to determine the relative position of the pupils within the active display area 942 of the monitor 940. Optionally, one or more displays may be fixed relative to a frame 812 positioned in or in front of one or both eyes of the user. For example, a flat panel LCD or other display (not shown) may be mounted on the frame 812 instead of a lens. Such devices may be used, for example, as simulations within medical or other research facilities, as well as for recreational use, eg, as video game controllers, and the like. the

Referring to FIG. 23 , there is shown an example method for controlling a computing device based on eye movement detected using any of the devices or systems described herein. For example, a device 910 shown in FIG. 18A that includes a fiber optic bundle 924 for imaging one or both eyes of a user may be used. Optionally, as explained further below, the device may also have one or more external cameras, eg, placed adjacent one or both eyes of the user, which may be positioned outwardly along the user's forward field of view. First, at step 2310, it is necessary to initialize the system including the means to establish a frame of reference, such as a base or reference position, a frame of reference with orthogonal components, and the like. For example, the user may be instructed to look at a pointer or other predetermined location on the display, thereby maintaining the state of the user's eyes, thereby keeping the user's pupils substantially unchanged. When the user's eyes are substantially unchanged, the processor may analyze the image data from the camera 830, for example, to determine the position of the pupil on the image corresponding to a reference point or "base position". For example, the pointer or base position may be positioned substantially straight in front of the user's pupils. Optionally, the user may be instructed to look sequentially at two or more identified locations on the display, thereby providing a gauge of the relative movement of the user's eyes. In this alternative embodiment, it is desirable to have the user look at opposite corners of the display, for example, to identify the limits of appropriate eye movement relative to the display. the

Once initialization is complete, the user is free to move their eyes, for example, to the left and right, up and down, relative to the pointer and/or rests of the display. At step 2320, the system can monitor eye movements such that the processor can analyze the image data to determine the relative position of the user's pupils from the base location. For example, if the user moves his/her eyes up and to the right from a base position, ie relative to a pointer on the computer screen, the processor can determine this movement. In response to this, step 2330, the processor can move the pointer up and to the right, i.e., thereby tracking the user's gaze. When the user stops moving his/her eyes, the processor stops the pointer once it reaches the position the user is currently seeing on the display. the

Optionally, step 2340, once the pointer moves to a desired location on the display, the user can execute a command, eg, similar to actuating a button on a mouse. For example, the processor may monitor image data for a signal from the user, eg, a predetermined sequence of one or more intentional blinks. This could be as simple as a single blink of a predetermined duration (eg, a few seconds long), or it could be a complex sequence of blinks, eg involving one or both of the user's eyes. Alternatively, the signal may be a predetermined period without any eye blinks, for example, three seconds, five seconds, or more seconds long. When the processor recognizes the signal, the processor can trigger the command. For example, when reaching an icon on the display, a message command, etc., the user can stop moving their eyes, and the processor can move the pointer until it is placed or positioned at the icon or command. The user may then blink or perform an action, as described above, similar to a "double click" of a button on a computer mouse, instructing the processor to complete or send the selected command to the intended recipient. For example, selected commands may result in the execution of a computer program, or the activation, deactivation, or control of a piece of equipment or other apparatus in a desired manner. Thus, the system can be used to accomplish a variety of tasks from controlling a computer device connected to a processor and/or display, to turning a light switch or vehicle on or off. Such a device and/or system may thus provide a method for using a computer automatically, ie using only the movement of the user's eyes. the

For example, in one application, the system may be used to maneuver a vehicle, such as a helicopter, jet, or other aircraft, for example, to trigger or control weapons, navigation, or other onboard systems. In another application, the system could be used in video games or other simulations, for example, to enhance virtual realism. For example, the system can enable the user to quickly navigate through multiple menus, scenes, or other actions while the user's hands are free to perform other functions, for example, perform other actions in addition to or in conjunction with eye-controlled functions. Simultaneously perform other behaviors, which may allow more and/or more complex tasks to be performed simultaneously. the

Additionally, one or more external cameras may be used to enhance and/or facilitate tracking eye movement relative to a pointer on the display. For example, an external camera may be placed proximate to at least one eye, eg, a predetermined distance from or otherwise in relationship to the eye, oriented toward the display. Thus, the external camera can provide an image of the display showing, for example, the movement of a pointer in real time (possibly in sync with the movement of the eyes monitored using the internal camera). The processor can describe this data using triangulation or other algorithms to increase the accuracy of tracking the pointer using eye movements. This can ensure the accuracy of actually selecting the intended command when the user intends to execute the command using the pointer on the command by blinking, eg when the display shows multiple commands available. the

Additionally, such systems and methods may be used in medical or other diagnostic procedures such as vigilance testing. For example, the processor can analyze data from internal and external cameras to correlate eye movements with images on the display to study various oculomotor parameters such as slow rolling eye movement, eye fixation and/or tracking, Trance gaze, increased blinking, hypnotic gaze, prolonged eyelid drooping or blinking, slow eyelid opening and closing, startled eyelid opening rate, chronic pupillary constriction changes, unstable pupillary parameters, pupillary responses evoking blurred vision, and/or this article Other parameters described elsewhere in . These procedures can be used to study an individual's response to various environmental, alcohol or drug stimuli, and/or other situations. the

Referring to FIG. 24 , in another embodiment, a device 2410 may be provided for percutaneously illuminating the eye 300 of a user wearing the device 2410 . Device 2410 may generally be similar to any of the embodiments described herein, such as mirror frame 812 shown in FIG. 13 . As shown, device 2410 may include one or more emitters or other light sources 2422 that contact a user's head 308 (e.g., a user's temple near one or both eyes 300 (one is shown for brevity)) . Additionally, device 2410 may include one or more sensors, such as fiber optic bundle 2430 terminating in lens 2434 , for acquiring images of user's eye 300 . the

In an exemplary embodiment, infrared emitters 2422 may be fixedly or adjustably mounted on one or both earpieces 2422 of the frame 2412 such that the emitters 2422 contact the user's skin and transmit light to the user's skin. For example, emitter 2422 may include multiple LEDs that emit parallel, non-coherent (non-laser) infrared light. Emitter 2422 may generally be positioned toward eye 300 so that light from emitter 2422 may travel through or along user's head 308 into eye 300 as indicated by dashed arrow 2450 . For example, it has been found that skin or other tissue can be transparent (translucent) to at least certain frequencies of light, such as infrared light. Thus, at least some light from emitter 2422 may be sent transcutaneously along or through the user's skin and/or other tissue until at least some light enters eye 300 . the

Light may reflect off the retina (not shown) within eye 300 such that at least some of the light exits pupil 301 as indicated by arrow 2452 . Lens 2434 and fiber optic bundle 2430 can pass an image of the eye to a detector (not shown), which, like other embodiments described herein, can identify pupil 301 as a bright spot on the image due to light 2452 . A processor or other device connected to the detector may monitor the pupil 301, such as its relative position, size, and/or shape on the image, using, for example, the "white pupil" or "dark pupil" techniques described elsewhere herein , similar to other embodiments described herein. Additionally or alternatively, the transcutaneous light can illuminate the entire eye 300, especially the retina, which can be viewed from the front of the user's face, for example, as a bright spot with the pupil 301 (shown as a bright spot surrounded by a dimmer shape). Dimmer spherical or other shapes. the

This embodiment may have the advantage that one or more emitters are not necessarily positioned in front of the user's eyes, which may partially block or divert the user's field of view. Additionally, because this system uses an infrared sensing camera and/or detector that operate independently of the angle of incident infrared light, this embodiment can eliminate technical difficulties arising from loss of proper positioning of the emitter to the detector. Additionally, the absence of an emitter in front of the eye eliminates flicker or reflections away from the eye, which can facilitate image data analysis. the

It should be appreciated that various components and features described using a particular embodiment may be added, deleted, and/or substituted with other embodiments, depending on the intended use of the embodiment. For example, Appendices A and B of Provisional Application Serial No. 60/559,135, the contents of which are incorporated herein by reference, provide information on possible structural features, and/or methods for using the devices and systems described herein. The entire disclosures of Appendices A and B are expressly incorporated herein by reference. the

While the invention is susceptible to various modifications, and alternative forms, specific examples thereof have been shown in the drawings and described in detail herein. It should be understood that the invention is not to be limited to the particular forms or methods disclosed herein, but on the contrary, the invention is to cover all modifications, equivalents and alternatives falling within the scope of the appended claims. the

Claims (21)

1. the system of the motion of eyes that are used for the monitor, it comprises:

Install, be used to be worn over people's head;

Light source is used for when described device is put on, with the eyes of photoconduction to described people;

Camera, it is positioned on the described device, is used to monitor the motion of described eyes, and described camera is used to produce the video signal of the video image of the described eyes that expression obtained by described camera;

Processor is communicated by letter with described camera, the pupil that is used for handling described video signal and discerns described eyes from described video signal; And

Display, it is connected to described processor, is used for from being shown the video image of described eyes by the handled described video signal of described processor, and described display is added to figure on the described video image, to discern described pupil.

2. system according to claim 1, wherein, described processor also is used to make described video signal to be associated, so that described people's the health or the mental status are determined in small part ground according to the analysis that the described pupil by described processor identification is carried out.

3. system according to claim 2, wherein, described processor is used to analyze described video signal, and is following at least a to monitor: the eccentricity of the relative position of described pupil, the size of described pupil and described pupil.

4. system according to claim 1, wherein, described camera comprises CCD or CMOS detector and at least one fibre bundle, and described fibre bundle comprises first end that is installed on the described device, when being put on, make the eye location of described first end towards described people with the described device of box lunch.

5. system according to claim 4, wherein, described light source comprises that at least one is by lighting fiber that described fibre bundle carried.

6. system according to claim 1 further comprises transmitter, and it is positioned on the described device, is used for the output signal from described camera is wirelessly transmitted to remote location.

7. system according to claim 6, wherein, described processor is positioned at the position away from described device, and described processor is connected to the receptor that is used to receive the described output signal that is sent by described transmitter.

8. system according to claim 1, wherein, described processor is set on the described device.

9. system according to claim 1, further comprise one or more pick offs, it is positioned on the described device, be used to measure one or more physiological features of described people, described processor is connected to described one or more pick off, be used to make described one or more physiological feature to be associated, to determine described people's the health or the mental status with described video signal.

10. system according to claim 1, further comprise one or more pick offs, it is positioned on the described device, be used to measure one or more physiological features of described user, described processor is connected to described one or more pick off, is used for showing on described display described video image and described one or more physiological feature.

11. a method that is used to use the motion of the eyes that the biosensor assembly comes the monitor, described biosensor assembly comprises: light source is used for when described biosensor assembly is put on, with the eyes of photoconduction to described people; And camera, towards described eye location, described method comprises:

Launch light from described light source to described eyes;

Use described camera to monitor the motion of described eyes, to produce the video image of described eyes; And

Generation is by the figure output of the described motion of described phase machine monitoring, and described figure is exported the analog image on the described video image that comprises the described eyes that are added to, to discern the pupil of described eyes.

12. a system that is used to control accountant, it comprises:

Install, be used to be worn over people's head;

Camera, it is positioned on the described device, is used to monitor the motion of described people's at least one eyes, and described camera is used to produce the signal of the video image of the described eyes that expression obtained by described camera;

Display is placed near described device, and the described people who makes pointer on the described display can be had on described device sees; And

Processor, be connected to described camera, described processor is used to handle described signal, to monitor of the motion of described eyes with respect to the referential on the described display, described processor is connected to described display, be used to make described pointer according to of the motion of described eyes, on described display, move with respect to described referential.

13. system according to claim 12, wherein, described display is a HUD, and it is fixed on the described device, so that described display is placed on the preset distance place in described eyes the place ahead of the described people who wears described device.

14. system according to claim 12, wherein, described processor is connected to described camera, be used to monitor the described signal that described people that expression wears described device is just watching the video image of the specific location on the described display, described processor is used for described pointer is moved to described ad-hoc location.

15. system according to claim 14, wherein, described processor is used to monitor described signal, to detect video image, described video image represents that the described people who wears described device blinks request execution command with predefined procedure when seeing described ad-hoc location, and described processor is configured for based on detecting described video image and carry out described order on described accountant.

16. system according to claim 12 further comprises light source, is used for when described device is put on, with the eyes of photoconduction to described people.

17. system according to claim 16, wherein, described light source is configured for described referential is projected on the described people's who wears described device the described eyes.

18. system according to claim 12, wherein, described processor is configured for the base location place of pictorial display at described display, described processor also is configured for the described signal of monitoring, to detect video image, described video image represents that the described people who wears described device is just seeing described base location place, thereby the subsequently described referential of motion of described eyes with respect to described base location is provided.

19. method that is used to use device on the head that is worn over the user to control accountant, described device comprises camera, described camera comprises the object lens at least one the eyes that point to described user, described accountant comprises display, described display is included in pointer shown on the described display, and described method comprises:

Use described camera to monitor the motion of described at least one eyes; And

Described pointer the motion of described at least one eyes is associated, so that can be followed the motion of described at least one eyes with described pointer on the described display.

20. method according to claim 19, further comprise: when described at least one eyes are being seen described pointer, determine the base location of described at least one eyes on effective viewing area of described camera, and wherein, by determining of the change of the position of described at least one eyes on described effective viewing area with respect to described base location, the described motion of described at least one eyes is associated, so that the described change that described pointer can response position is moved.

21. method according to claim 19 further comprises: monitor the eye motion of the predefined procedure of described at least one eyes, thereby provide signal to trigger the order of discerning by the described pointer on the described display.

Images