US11554781B2 - Driver alertness monitoring including a predictive sleep risk factor - Google Patents

Driver alertness monitoring including a predictive sleep risk factor Download PDF

Info

Publication number
US11554781B2
US11554781B2 US16/827,309 US202016827309A US11554781B2 US 11554781 B2 US11554781 B2 US 11554781B2 US 202016827309 A US202016827309 A US 202016827309A US 11554781 B2 US11554781 B2 US 11554781B2
Authority
US
United States
Prior art keywords
vehicle
condition
passenger
driver
alertness
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US16/827,309
Other versions
US20210291838A1 (en
Inventor
Santhosh Tamilarasan
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aptiv Technologies AG
Original Assignee
Aptiv Technologies Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Aptiv Technologies Ltd filed Critical Aptiv Technologies Ltd
Assigned to APTIV TECHNOLOGIES LIMITED reassignment APTIV TECHNOLOGIES LIMITED ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: Tamilarasan, Santhosh
Priority to US16/827,309 priority Critical patent/US11554781B2/en
Priority to EP21156287.1A priority patent/EP3886069A1/en
Priority to CN202110301523.XA priority patent/CN113425299B/en
Publication of US20210291838A1 publication Critical patent/US20210291838A1/en
Publication of US11554781B2 publication Critical patent/US11554781B2/en
Application granted granted Critical
Assigned to APTIV TECHNOLOGIES (2) S.À R.L. reassignment APTIV TECHNOLOGIES (2) S.À R.L. ENTITY CONVERSION Assignors: APTIV TECHNOLOGIES LIMITED
Assigned to APTIV MANUFACTURING MANAGEMENT SERVICES S.À R.L. reassignment APTIV MANUFACTURING MANAGEMENT SERVICES S.À R.L. MERGER Assignors: APTIV TECHNOLOGIES (2) S.À R.L.
Assigned to Aptiv Technologies AG reassignment Aptiv Technologies AG ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: APTIV MANUFACTURING MANAGEMENT SERVICES S.À R.L.
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/18—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state for vehicle drivers or machine operators
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
    • G08B21/02—Alarms for ensuring the safety of persons
    • G08B21/06—Alarms for ensuring the safety of persons indicating a condition of sleep, e.g. anti-dozing alarms
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6887—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient mounted on external non-worn devices, e.g. non-medical devices
    • A61B5/6893—Cars
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271—Specific aspects of physiological measurement analysis
    • A61B5/7275—Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient; User input means
    • A61B5/746—Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B60W40/04—Traffic conditions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B60W40/06—Road conditions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/18—Eye characteristics, e.g. of the iris
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0818—Inactivity or incapacity of driver
    • B60W2040/0827—Inactivity or incapacity of driver due to sleepiness
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0881—Seat occupation; Driver or passenger presence
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/225—Direction of gaze

Definitions

  • Contemporary passenger vehicles include a variety of electronic components. Advances in technology have made it possible to include a variety of sensing and control features.
  • driver monitoring systems are capable of detecting when a driver is drowsy or asleep. Such systems often include a camera that provides image information regarding the driver's eyes.
  • a computing device uses an algorithm to process the image information to make a determination regarding the driver's alertness or attention on the road. For example, if the image information indicates that the driver's eyes are frequently closing or the direction of the driver's gaze is not on the road in front of the vehicle, the system may determine that the driver is drowsy. Some such systems provide an alert to the driver under such circumstances.
  • An illustrative example system includes at least one alertness detector that is configured to detect an alertness condition of a driver of a vehicle and an alertness condition of a passenger in the vehicle.
  • a controller is configured to determine a sleep risk factor based on the alertness condition of the passenger.
  • the controller is also configured to determine a likelihood that the driver is sleepy based on the alertness condition of the driver and the sleep risk factor.
  • the controller is configured to control a feature of the vehicle to assist the driver when the determined likelihood satisfies a predetermined criterion.
  • the controller is configured to determine the sleep risk factor based on the alertness condition of the passenger and at least one other condition.
  • the at least one other condition is based on at least one of a number of passengers in the vehicle, a location of the passenger in the vehicle, time information, and information regarding a road the vehicle is traveling on.
  • the at least one other condition is the number of passengers in the vehicle
  • the controller is configured to determine an alertness condition of each of the passengers
  • the controller is configured to alter a significance of the at least one other condition for determining the sleep risk factor based on the number of the passengers that are drowsy or asleep.
  • the controller is configured to assign a first significance when a first number of passengers is asleep or drowsy, and assign a second significance when a second number of passengers is asleep or drowsy; the second number of passengers is higher than the first number of passengers; and the second significance is greater than the first significance.
  • the at least one other condition is the location of the passenger in the vehicle
  • the controller is configured to assign a first a significance of the at least one other condition for determining the sleep risk factor when the passenger is in a front seat of the vehicle
  • the controller is configured to assign a second a significance of the at least one other condition for determining the sleep risk factor when the passenger is in a rear seat of the vehicle, and the first significance is greater than the second significance.
  • the at least one other condition is the time information
  • the time information includes an amount of time the driver has been driving the vehicle
  • the time information includes a time of day when the likelihood is being determined
  • the controller is configured to alter a significance of the at least one other condition for determining the sleep risk factor based on the amount of time the driver has been driving and the time of day.
  • the at least one other condition is the information regarding the road the vehicle is traveling on, the information indicates a type of the road, the information indicates a location of the vehicle, the information indicates a traffic condition of the road in a vicinity of the vehicle, and the controller is configured to alter a significance of the at least one other condition for determining the sleep risk factor based on the type of the road, the location of the vehicle and the traffic condition.
  • the controller is configured to determine the sleep risk factor based on a combination of respective values assigned by the controller to a passenger alertness factor, a time factor, and a road factor.
  • the controller is configured to: determine the respective value of the passenger alertness factor based on the determined alertness condition of the passenger, a location of the passenger in the vehicle, and a number of passengers in the vehicle; determine the respective value of the time factor based on an amount of time the driver has been driving the vehicle and a time of day when the sleep risk factor is being determined; and determine the respective value of the road factor based on a type of road the vehicle is traveling along, a location of the vehicle, and a traffic condition in a vicinity of the vehicle.
  • An illustrative example method of monitoring a driver includes detecting an alertness condition of a driver of a vehicle and an alertness condition of a passenger in the vehicle; determining a sleep risk factor based on the alertness condition of the passenger; determining a likelihood that the driver is sleepy based on the alertness condition of the driver and the sleep risk factor; and controlling a feature of the vehicle to assist the driver when the determined likelihood satisfies a predetermined criterion.
  • determining the sleep risk factor is based on the alertness condition of the passenger and at least one other condition.
  • the at least one other condition is based on at least one of a number of passengers in the vehicle, a location of the passenger in the vehicle, time information, and information regarding a road the vehicle is traveling on.
  • the at least one other condition is the number of passengers in the vehicle and the method comprises determining an alertness condition of each of the passengers, and altering a significance of the at least one other condition for determining the sleep risk factor based on the number of the passengers that are drowsy or asleep.
  • An example embodiment having at least one feature of the method of any of the previous paragraphs includes assigning a first significance when a first number of passengers is asleep or drowsy, and assigning a second significance when a second number of passengers is asleep or drowsy; wherein the second number of passengers is higher than the first number of passengers, and the second significance is greater than the first significance.
  • the at least one other condition is the location of the passenger in the vehicle and the method comprises assigning a first a significance of the at least one other condition for determining the sleep risk factor when the passenger is in a front seat of the vehicle, and assigning a second a significance of the at least one other condition for determining the sleep risk factor when the passenger is in a rear seat of the vehicle, wherein the first significance is greater than the second significance.
  • the at least one other condition is the time information
  • the time information includes an amount of time the driver has been driving the vehicle
  • the time information includes a time of day when the likelihood is being determined
  • the method comprises altering a significance of the at least one other condition for determining the sleep risk factor based on the amount of time the driver has been driving and the time of day.
  • the at least one other condition is the information regarding the road the vehicle is traveling on, the information indicates a type of the road, the information indicates a location of the vehicle, the information indicates a traffic condition of the road in a vicinity of the vehicle, and the method comprises altering a significance of the at least one other condition for determining the sleep risk factor based on the type of the road, the location of the vehicle and the traffic condition.
  • determining the sleep risk factor is based on a combination of respective values assigned by the controller to a passenger alertness factor, a time factor, and a road factor.
  • An example embodiment having at least one feature of the method of any of the previous paragraphs includes: determining the respective value of the passenger alertness factor based on the determined alertness condition of the passenger, a location of the passenger in the vehicle, and a number of passengers in the vehicle; determining the respective value of the time factor based on an amount of time the driver has been driving the vehicle and a time of day when the sleep risk factor is being determined; and determining the respective value of the road factor based on a type of road the vehicle is traveling along, a location of the vehicle, and a traffic condition in a vicinity of the vehicle.
  • FIG. 1 diagrammatically illustrates selected portions of a vehicle including a driver monitor system designed according to an embodiment of this invention.
  • FIG. 2 is a flowchart diagram summarizing an example process of monitoring a driver of a vehicle.
  • FIG. 1 diagrammatically illustrates selected portions of an interior of a vehicle 20 .
  • a driver alertness detector 22 has a field of view 24 directed toward a driver's seat 26 .
  • the driver alertness detector 22 is situated to observe at least one characteristic of a driver (not illustrated) seated in the driver's seat 26 .
  • the driver alertness detector 22 includes a camera that provides image information regarding the driver's eyes, such as whether they are closed and whether the driver's gaze is directed toward the road in front of the vehicle 20 .
  • a passenger alertness detector 28 has a field of view 30 directed toward a front passenger seat 32 next to the driver's seat 26 .
  • the passenger alertness detector 28 in the illustrated example includes a camera that provides image information regarding an individual seated in the front passenger seat 32 .
  • the passenger alertness detector 28 provides information regarding the passenger's face, which can be used to determine whether the passenger is drowsy or sleeping.
  • Additional alertness detectors 34 each include a field of view 36 for observing a passenger located in a rear passenger seat 38 .
  • the alertness monitors 34 provide information regarding whether a passenger in the seat 38 is drowsy or asleep, for example.
  • While the illustrated embodiment includes multiple alertness detectors 22 , 28 and 34 , some embodiments include a single detector that is capable of providing information regarding an individual in more than one of the seats 26 , 32 and 38 . Given this description, those skilled in the art will be able to select an arrange appropriate detector components to meet the needs of their particular situation.
  • a controller 40 receives information from the alertness detectors 22 , 28 and 34 regarding the alertness condition of the driver and any passenger in the vehicle 20 .
  • the controller 40 is configured to predictively determine a likelihood that the driver is sleepy or asleep based on the information from the alertness detectors regarding the driver and at least one passenger in the vehicle 20 .
  • FIG. 2 includes a flowchart diagram 50 that summarizes an example approach.
  • the controller 40 determines an alertness condition of the driver and any passenger in the vehicle 20 .
  • the alertness condition is based on information, such as image data, from the alertness detectors 22 , 28 and 34 .
  • the alertness condition of the driver is determined based upon image information regarding the driver's eyes and a direction of the driver's gaze. Other information regarding a posture of the driver and a tilt of the driver's head may be used.
  • Some example embodiments include a known algorithm for processing image information regarding a driver's face to determine an alertness condition of the driver.
  • the controller 40 uses the same or similar techniques for determining the alertness condition of a passenger.
  • the controller 40 also determines a location of each passenger in the vehicle 20 , such as whether a passenger is in the front passenger seat 32 or rear passenger seat 38 .
  • the controller 40 determines a sleep risk factor based on the alertness condition of at least one passenger. If there is more than one passenger in the vehicle, then the sleep risk factor is based on the alertness condition of more than one of the passengers. In some embodiments, the alertness condition of every passenger is taken into account by the controller 40 when determining the sleep risk factor at 54 .
  • the sleep risk factor determined by the controller 40 allows the controller 40 to determine a likelihood that the driver is sleepy in a predictive manner because the sleep risk factor corresponds to conditions or situations within the vehicle 20 that have an influence on predicting whether a driver might fall asleep.
  • Determining the sleep risk factor at 54 is based on at least the alertness condition of at least one passenger within the vehicle 20 .
  • the sleep risk factor is a weighted summation of the alertness condition of the passenger and at least one other condition.
  • Example conditions that are taken into account by the controller 40 in combination with the alertness condition of any passengers include a number of passenger in the vehicle, a location of the passengers in the vehicle, time information, and information regarding a road the vehicle is traveling on. The information regarding the road includes location or map information and traffic information in some embodiments.
  • the weighting factors a, b, c and d may be predetermined constants or may be adjusted dynamically by the controller 40 depending upon the particular condition at the time when the controller 40 is determining the sleep risk factor.
  • the controller when the number of passengers asleep in the vehicle is a condition contributing to the sleep risk factor, the controller is configured to determine an alertness condition of each of the passengers and to alter a significance of that condition for determining the sleep risk factor based on the number of the passengers that are drowsy or asleep.
  • the controller When more than one other individual in a vehicle is asleep or drowsy, there is an increased chance that the driver will become sleepy.
  • the sleep risk factor will increase based on an increased value of Info passenger because of an increased number of sleepy or drowsy passengers and the influence or significance of that number will increase as the weighting factor a increases.
  • the controller 40 is configured to assign a first weight or significance to the weighting factor a when a first number of passengers is asleep or drowsy and assign a second, greater significance when a second, higher number of passengers is asleep or drowsy.
  • the controller 40 in such embodiments is configured to increase the value of the sleep risk factor when an increased number of passengers in the vehicle 20 is asleep or drowsy by more than just the increase in the value of Info passenger .
  • Those individual weighting factors may vary to place differing weights on the different information contributing to Info passenger
  • the values of a 1 , a 2 and a 3 may be adjusted similarly to how the value of the weighting factor a is adjusted as described above, such as increasing when the corresponding characteristic of the passenger(s) increases in value.
  • a 1 , a 2 and a 3 are predetermined constants.
  • the example controller 40 is also configured to take into account the location of a passenger in the vehicle.
  • the controller assigns a first significance a 1 to the passenger location information, which is seating positon in the preceding equation. If the passenger is in the rear passenger seat 38 , the controller assigns a second significance a 1 to that condition.
  • a passenger in the front passenger seat 32 is asleep or drowsy, that has a different effect on the likelihood that the driver may be sleepy and the different significances assigned by the controller 40 take that into account.
  • the weighting factor b may increase as the time of day becomes later so that the time information has a higher influence on the sleep risk factor at night compared to during the day.
  • the time information utilized by the controller 40 in this example embodiment includes information such as the time of day, the amount of time that the driver has been driving and the duration of recent trips taken by the driver.
  • b 1 , b 2 and b 3 are weights or significance values that may be predetermined or adjusted by the controller 40 .
  • Information regarding the road contributes to the potential fatigue level of the driver. For example, when a driver is driving along a highway there tends to be a higher likelihood that the driver may become drowsy compared to when a driver is in an urban location driving along city streets. Traffic information also is useful for determining a likelihood whether a driver is sleepy. Heavier traffic conditions typically require a higher level of alertness and a driver tends to be less relaxed compared to situations in which there is very light or relatively no traffic. The controller 40 takes such information into account in the illustrated example embodiment.
  • the weighting factors a, b, c, and d and the individual weighting factors a 1 , a 2 , a 3 , b 1 , b 2 and b 3 also may vary independently of changes to the corresponding or related values. In some embodiments, the weighting factors are adjusted based on other selected criteria. The weighting factors allow for adjusting the sleep risk factor in a customized or enhanced manner than just simply increasing a number of sleeping passengers or number of hours that the driver has been driving.
  • the controller 40 determines a likelihood that the driver is sleepy based on the alertness condition of the driver and the sleep risk factor.
  • the likelihood that the driver is sleepy accounts for a driver that is asleep, drowsy or falling asleep. For example, if the driver alertness condition corresponds to the driver not being fully awake or not being fully attentive to driving, the alertness condition of the driver suggests a likelihood that the driver is sleepy.
  • the sleep risk factor corresponds to conditions indicating a higher risk that the driver may be sleepy, there is a higher likelihood that the driver is sleepy.
  • the driver alertness condition corresponds to the driver being fully alert and awake, the likelihood that the driver is sleepy will be less even though the sleep risk factor may be relatively high based upon the condition of the passengers.
  • the controller 40 controls at least one vehicle feature to assist the driver when there is a likelihood that the driver is sleepy that satisfies at least one predetermined criterion.
  • Vehicle features that may be controlled include, for example, turning on or brightening lights within the cabin of the vehicle 20 , providing an audible or tactile alert and increasing the volume of any infotainment that may be active within the vehicle 20 .
  • Other vehicle features that are controlled in some embodiments include automated vehicle maneuvers, such as slowing down the vehicle and automatically maneuvering the vehicle onto a shoulder of the road where the vehicle is automatically stopped.
  • Various combinations of vehicle features may be controlled in response to a determination that there is a high likelihood that the driver is sleepy.
  • the controller 40 takes information into account including the alertness condition of at least one passenger in the vehicle for determining the sleep risk factor that allows for predictively determining a likelihood that the driver is sleeping. Using information apart from the alertness condition of the driver provides improved driver alertness determinations and allows for predicting potential sleep conditions.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Mechanical Engineering (AREA)
  • Transportation (AREA)
  • Automation & Control Theory (AREA)
  • General Health & Medical Sciences (AREA)
  • Mathematical Physics (AREA)
  • Biophysics (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Pathology (AREA)
  • Veterinary Medicine (AREA)
  • Physiology (AREA)
  • General Physics & Mathematics (AREA)
  • Psychiatry (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychology (AREA)
  • Social Psychology (AREA)
  • Hospice & Palliative Care (AREA)
  • Ophthalmology & Optometry (AREA)
  • Human Computer Interaction (AREA)
  • Educational Technology (AREA)
  • Developmental Disabilities (AREA)
  • Child & Adolescent Psychology (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Traffic Control Systems (AREA)
  • Auxiliary Drives, Propulsion Controls, And Safety Devices (AREA)

Abstract

An illustrative example system includes at least one alertness detector that is configured to detect an alertness condition of a driver of a vehicle and an alertness condition of a passenger in the vehicle. A controller is configured to determine a sleep risk factor based on the alertness condition of the passenger. The controller is also configured to determine a likelihood that the driver is sleepy based on the alertness condition of the driver and the sleep risk factor. The controller is configured to control a feature of the vehicle to assist the driver when the determined likelihood satisfies a predetermined criterion.

Description

BACKGROUND
Contemporary passenger vehicles include a variety of electronic components. Advances in technology have made it possible to include a variety of sensing and control features. For example, driver monitoring systems are capable of detecting when a driver is drowsy or asleep. Such systems often include a camera that provides image information regarding the driver's eyes. A computing device uses an algorithm to process the image information to make a determination regarding the driver's alertness or attention on the road. For example, if the image information indicates that the driver's eyes are frequently closing or the direction of the driver's gaze is not on the road in front of the vehicle, the system may determine that the driver is drowsy. Some such systems provide an alert to the driver under such circumstances.
One limitation of such systems is that they do not have predictive capability. Another shortcoming is that the determination of the driver's alertness is often based only on information regarding the driver's gaze.
SUMMARY
An illustrative example system includes at least one alertness detector that is configured to detect an alertness condition of a driver of a vehicle and an alertness condition of a passenger in the vehicle. A controller is configured to determine a sleep risk factor based on the alertness condition of the passenger. The controller is also configured to determine a likelihood that the driver is sleepy based on the alertness condition of the driver and the sleep risk factor. The controller is configured to control a feature of the vehicle to assist the driver when the determined likelihood satisfies a predetermined criterion.
In an example embodiment having at least one feature of the system of the previous paragraph, the controller is configured to determine the sleep risk factor based on the alertness condition of the passenger and at least one other condition.
In an example embodiment having at least one feature of the system of any of the previous paragraphs, the at least one other condition is based on at least one of a number of passengers in the vehicle, a location of the passenger in the vehicle, time information, and information regarding a road the vehicle is traveling on.
In an example embodiment having at least one feature of the system of any of the previous paragraphs, the at least one other condition is the number of passengers in the vehicle, the controller is configured to determine an alertness condition of each of the passengers, and the controller is configured to alter a significance of the at least one other condition for determining the sleep risk factor based on the number of the passengers that are drowsy or asleep.
In an example embodiment having at least one feature of the system of any of the previous paragraphs, the controller is configured to assign a first significance when a first number of passengers is asleep or drowsy, and assign a second significance when a second number of passengers is asleep or drowsy; the second number of passengers is higher than the first number of passengers; and the second significance is greater than the first significance.
In an example embodiment having at least one feature of the system of any of the previous paragraphs, the at least one other condition is the location of the passenger in the vehicle, the controller is configured to assign a first a significance of the at least one other condition for determining the sleep risk factor when the passenger is in a front seat of the vehicle, the controller is configured to assign a second a significance of the at least one other condition for determining the sleep risk factor when the passenger is in a rear seat of the vehicle, and the first significance is greater than the second significance.
In an example embodiment having at least one feature of the system of any of the previous paragraphs, the at least one other condition is the time information, the time information includes an amount of time the driver has been driving the vehicle, the time information includes a time of day when the likelihood is being determined, and the controller is configured to alter a significance of the at least one other condition for determining the sleep risk factor based on the amount of time the driver has been driving and the time of day.
In an example embodiment having at least one feature of the system of any of the previous paragraphs, the at least one other condition is the information regarding the road the vehicle is traveling on, the information indicates a type of the road, the information indicates a location of the vehicle, the information indicates a traffic condition of the road in a vicinity of the vehicle, and the controller is configured to alter a significance of the at least one other condition for determining the sleep risk factor based on the type of the road, the location of the vehicle and the traffic condition.
In an example embodiment having at least one feature of the system of any of the previous paragraphs, the controller is configured to determine the sleep risk factor based on a combination of respective values assigned by the controller to a passenger alertness factor, a time factor, and a road factor.
In an example embodiment having at least one feature of the system of any of the previous paragraphs, the controller is configured to: determine the respective value of the passenger alertness factor based on the determined alertness condition of the passenger, a location of the passenger in the vehicle, and a number of passengers in the vehicle; determine the respective value of the time factor based on an amount of time the driver has been driving the vehicle and a time of day when the sleep risk factor is being determined; and determine the respective value of the road factor based on a type of road the vehicle is traveling along, a location of the vehicle, and a traffic condition in a vicinity of the vehicle.
An illustrative example method of monitoring a driver includes detecting an alertness condition of a driver of a vehicle and an alertness condition of a passenger in the vehicle; determining a sleep risk factor based on the alertness condition of the passenger; determining a likelihood that the driver is sleepy based on the alertness condition of the driver and the sleep risk factor; and controlling a feature of the vehicle to assist the driver when the determined likelihood satisfies a predetermined criterion.
In an example embodiment having at least one feature of the method of any of the previous paragraphs, determining the sleep risk factor is based on the alertness condition of the passenger and at least one other condition.
In an example embodiment having at least one feature of the method of any of the previous paragraphs, the at least one other condition is based on at least one of a number of passengers in the vehicle, a location of the passenger in the vehicle, time information, and information regarding a road the vehicle is traveling on.
In an example embodiment having at least one feature of the method of any of the previous paragraphs, the at least one other condition is the number of passengers in the vehicle and the method comprises determining an alertness condition of each of the passengers, and altering a significance of the at least one other condition for determining the sleep risk factor based on the number of the passengers that are drowsy or asleep.
An example embodiment having at least one feature of the method of any of the previous paragraphs includes assigning a first significance when a first number of passengers is asleep or drowsy, and assigning a second significance when a second number of passengers is asleep or drowsy; wherein the second number of passengers is higher than the first number of passengers, and the second significance is greater than the first significance.
In an example embodiment having at least one feature of the method of any of the previous paragraphs, the at least one other condition is the location of the passenger in the vehicle and the method comprises assigning a first a significance of the at least one other condition for determining the sleep risk factor when the passenger is in a front seat of the vehicle, and assigning a second a significance of the at least one other condition for determining the sleep risk factor when the passenger is in a rear seat of the vehicle, wherein the first significance is greater than the second significance.
In an example embodiment having at least one feature of the method of any of the previous paragraphs, the at least one other condition is the time information, the time information includes an amount of time the driver has been driving the vehicle, the time information includes a time of day when the likelihood is being determined, and the method comprises altering a significance of the at least one other condition for determining the sleep risk factor based on the amount of time the driver has been driving and the time of day.
In an example embodiment having at least one feature of the method of any of the previous paragraphs, the at least one other condition is the information regarding the road the vehicle is traveling on, the information indicates a type of the road, the information indicates a location of the vehicle, the information indicates a traffic condition of the road in a vicinity of the vehicle, and the method comprises altering a significance of the at least one other condition for determining the sleep risk factor based on the type of the road, the location of the vehicle and the traffic condition.
In an example embodiment having at least one feature of the method of any of the previous paragraphs, determining the sleep risk factor is based on a combination of respective values assigned by the controller to a passenger alertness factor, a time factor, and a road factor.
An example embodiment having at least one feature of the method of any of the previous paragraphs includes: determining the respective value of the passenger alertness factor based on the determined alertness condition of the passenger, a location of the passenger in the vehicle, and a number of passengers in the vehicle; determining the respective value of the time factor based on an amount of time the driver has been driving the vehicle and a time of day when the sleep risk factor is being determined; and determining the respective value of the road factor based on a type of road the vehicle is traveling along, a location of the vehicle, and a traffic condition in a vicinity of the vehicle.
The various features and advantages of at least one disclosed example embodiment will become apparent to those skilled in the art from the following detailed description. The drawings that accompany the detailed description can be briefly described as follows.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 diagrammatically illustrates selected portions of a vehicle including a driver monitor system designed according to an embodiment of this invention.
FIG. 2 is a flowchart diagram summarizing an example process of monitoring a driver of a vehicle.
 DETAILED DESCRIPTION
FIG. 1 diagrammatically illustrates selected portions of an interior of a vehicle 20. In this example embodiment, a driver alertness detector 22 has a field of view 24 directed toward a driver's seat 26. The driver alertness detector 22 is situated to observe at least one characteristic of a driver (not illustrated) seated in the driver's seat 26. In some examples, the driver alertness detector 22 includes a camera that provides image information regarding the driver's eyes, such as whether they are closed and whether the driver's gaze is directed toward the road in front of the vehicle 20.
A passenger alertness detector 28 has a field of view 30 directed toward a front passenger seat 32 next to the driver's seat 26. The passenger alertness detector 28 in the illustrated example includes a camera that provides image information regarding an individual seated in the front passenger seat 32. For example, the passenger alertness detector 28 provides information regarding the passenger's face, which can be used to determine whether the passenger is drowsy or sleeping.
Additional alertness detectors 34 each include a field of view 36 for observing a passenger located in a rear passenger seat 38. The alertness monitors 34 provide information regarding whether a passenger in the seat 38 is drowsy or asleep, for example.
While the illustrated embodiment includes multiple alertness detectors 22, 28 and 34, some embodiments include a single detector that is capable of providing information regarding an individual in more than one of the seats 26, 32 and 38. Given this description, those skilled in the art will be able to select an arrange appropriate detector components to meet the needs of their particular situation.
A controller 40 receives information from the alertness detectors 22, 28 and 34 regarding the alertness condition of the driver and any passenger in the vehicle 20. The controller 40 is configured to predictively determine a likelihood that the driver is sleepy or asleep based on the information from the alertness detectors regarding the driver and at least one passenger in the vehicle 20.
FIG. 2 includes a flowchart diagram 50 that summarizes an example approach. At 52, the controller 40 determines an alertness condition of the driver and any passenger in the vehicle 20. The alertness condition is based on information, such as image data, from the alertness detectors 22, 28 and 34. For example, the alertness condition of the driver is determined based upon image information regarding the driver's eyes and a direction of the driver's gaze. Other information regarding a posture of the driver and a tilt of the driver's head may be used. Some example embodiments include a known algorithm for processing image information regarding a driver's face to determine an alertness condition of the driver.
The controller 40 uses the same or similar techniques for determining the alertness condition of a passenger. The controller 40 also determines a location of each passenger in the vehicle 20, such as whether a passenger is in the front passenger seat 32 or rear passenger seat 38.
At 54, the controller 40 determines a sleep risk factor based on the alertness condition of at least one passenger. If there is more than one passenger in the vehicle, then the sleep risk factor is based on the alertness condition of more than one of the passengers. In some embodiments, the alertness condition of every passenger is taken into account by the controller 40 when determining the sleep risk factor at 54.
The sleep risk factor determined by the controller 40 allows the controller 40 to determine a likelihood that the driver is sleepy in a predictive manner because the sleep risk factor corresponds to conditions or situations within the vehicle 20 that have an influence on predicting whether a driver might fall asleep.
Determining the sleep risk factor at 54 is based on at least the alertness condition of at least one passenger within the vehicle 20. In the illustrated example embodiment, the sleep risk factor is a weighted summation of the alertness condition of the passenger and at least one other condition. Example conditions that are taken into account by the controller 40 in combination with the alertness condition of any passengers include a number of passenger in the vehicle, a location of the passengers in the vehicle, time information, and information regarding a road the vehicle is traveling on. The information regarding the road includes location or map information and traffic information in some embodiments.
The controller 40 determines the sleep risk factor using a weighted sum of passenger information, time information, map information and traffic information using the following relationship:
Sleepriskfactor =a(Infopassenger)+b*(InfoTime)+c(InfoMap)+d*(InfoTraffic)
where, a, b, c, and d are weighting factors and a+b+c+d=1. The weighting factors a, b, c and d may be predetermined constants or may be adjusted dynamically by the controller 40 depending upon the particular condition at the time when the controller 40 is determining the sleep risk factor. For example, when the number of passengers asleep in the vehicle is a condition contributing to the sleep risk factor, the controller is configured to determine an alertness condition of each of the passengers and to alter a significance of that condition for determining the sleep risk factor based on the number of the passengers that are drowsy or asleep. When more than one other individual in a vehicle is asleep or drowsy, there is an increased chance that the driver will become sleepy.
In embodiments that include dynamically adjustable weighting factors, the sleep risk factor will increase based on an increased value of Infopassenger because of an increased number of sleepy or drowsy passengers and the influence or significance of that number will increase as the weighting factor a increases. For example, the controller 40 is configured to assign a first weight or significance to the weighting factor a when a first number of passengers is asleep or drowsy and assign a second, greater significance when a second, higher number of passengers is asleep or drowsy. In other words, the controller 40 in such embodiments is configured to increase the value of the sleep risk factor when an increased number of passengers in the vehicle 20 is asleep or drowsy by more than just the increase in the value of Infopassenger.
In an example embodiment, the controller 40 utilizes information regarding the number of passengers that are drowsy or asleep and the seating location of each. In some embodiments, the controller 40 also uses information regarding an amount of time that any of the passengers has been asleep or drowsy. In such embodiments, the controller 40 uses a relationship such as:
Infopassenger =a 1*seatingpositinn +a 2*sleepduration +a 3*sleepingnumber
for determining the value of Infopassenger used for the sleep risk factor. The values of a1, a2 and a3 are individual weighting factors for each of the values that contribute to Infopassenger. Those individual weighting factors may vary to place differing weights on the different information contributing to Infopassenger The values of a1, a2 and a3 may be adjusted similarly to how the value of the weighting factor a is adjusted as described above, such as increasing when the corresponding characteristic of the passenger(s) increases in value. In some embodiments, a1, a2 and a3 are predetermined constants.
The example controller 40 is also configured to take into account the location of a passenger in the vehicle. When the passenger is in the front passenger seat 32, the controller assigns a first significance a1 to the passenger location information, which is seatingpositon in the preceding equation. If the passenger is in the rear passenger seat 38, the controller assigns a second significance a1 to that condition. When a passenger in the front passenger seat 32 is asleep or drowsy, that has a different effect on the likelihood that the driver may be sleepy and the different significances assigned by the controller 40 take that into account.
The weighting factor b may increase as the time of day becomes later so that the time information has a higher influence on the sleep risk factor at night compared to during the day.
The time information utilized by the controller 40 in this example embodiment includes information such as the time of day, the amount of time that the driver has been driving and the duration of recent trips taken by the driver. The controller 40 may use the following relationship for determining the time information:
InfoTime =b 1*Timedriving +b 2*Timeduration +b 3*historyprevioustripsduration.
where b1, b2 and b3 are weights or significance values that may be predetermined or adjusted by the controller 40.
For example, when a driver has been driving for multiple hours during the night, that contributes to a higher likelihood that the driver is sleepy compared to a driver that has been driving for less time and during the afternoon.
Information regarding the road, such as the location of the vehicle, contributes to the potential fatigue level of the driver. For example, when a driver is driving along a highway there tends to be a higher likelihood that the driver may become drowsy compared to when a driver is in an urban location driving along city streets. Traffic information also is useful for determining a likelihood whether a driver is sleepy. Heavier traffic conditions typically require a higher level of alertness and a driver tends to be less relaxed compared to situations in which there is very light or relatively no traffic. The controller 40 takes such information into account in the illustrated example embodiment.
The weighting factors a, b, c, and d and the individual weighting factors a1, a2, a3, b1, b2 and b3 also may vary independently of changes to the corresponding or related values. In some embodiments, the weighting factors are adjusted based on other selected criteria. The weighting factors allow for adjusting the sleep risk factor in a customized or enhanced manner than just simply increasing a number of sleeping passengers or number of hours that the driver has been driving.
At 56, the controller 40 determines a likelihood that the driver is sleepy based on the alertness condition of the driver and the sleep risk factor. The likelihood that the driver is sleepy accounts for a driver that is asleep, drowsy or falling asleep. For example, if the driver alertness condition corresponds to the driver not being fully awake or not being fully attentive to driving, the alertness condition of the driver suggests a likelihood that the driver is sleepy. When the sleep risk factor corresponds to conditions indicating a higher risk that the driver may be sleepy, there is a higher likelihood that the driver is sleepy. On the other hand, if the driver alertness condition corresponds to the driver being fully alert and awake, the likelihood that the driver is sleepy will be less even though the sleep risk factor may be relatively high based upon the condition of the passengers.
At 58, the controller 40 controls at least one vehicle feature to assist the driver when there is a likelihood that the driver is sleepy that satisfies at least one predetermined criterion. Vehicle features that may be controlled include, for example, turning on or brightening lights within the cabin of the vehicle 20, providing an audible or tactile alert and increasing the volume of any infotainment that may be active within the vehicle 20. Other vehicle features that are controlled in some embodiments include automated vehicle maneuvers, such as slowing down the vehicle and automatically maneuvering the vehicle onto a shoulder of the road where the vehicle is automatically stopped. Various combinations of vehicle features may be controlled in response to a determination that there is a high likelihood that the driver is sleepy.
The controller 40 takes information into account including the alertness condition of at least one passenger in the vehicle for determining the sleep risk factor that allows for predictively determining a likelihood that the driver is sleeping. Using information apart from the alertness condition of the driver provides improved driver alertness determinations and allows for predicting potential sleep conditions.
The preceding description is exemplary rather than limiting in nature.
Variations and modifications to the disclosed examples may become apparent to those skilled in the art that do not necessarily depart from the essence of this invention. The scope of legal protection given to this invention can only be determined by studying the following claims.

Claims (15)

I claim:
1. A driver monitoring system, comprising:
at least one alertness detector configured to detect an alertness condition of a driver of a vehicle and an alertness condition of a passenger in the vehicle; and
a controller configured to
determine a sleep risk factor based on the alertness condition of the passenger and a number of passengers in the vehicle,
determine an alertness condition of each of the passengers,
alter a significance of at least one other condition for determining the sleep risk factor based on the number of the passengers that are drowsy or asleep,
determine a likelihood that the driver is sleepy based on the alertness condition of the driver and the sleep risk factor, and
control a feature of the vehicle to assist the driver when the determined likelihood satisfies a predetermined criterion.
2. The system of claim 1, wherein the controller is configured to determine the sleep risk factor based on at least one other condition that is based on at least one of
a location of the passenger in the vehicle,
time information, and
information regarding a road the vehicle is traveling on.
3. The system of claim 1, wherein
the controller is configured to
assign a first significance when a first number of passengers is asleep or drowsy, and
assign a second significance when a second number of passengers is asleep or drowsy;
the second number of passengers is higher than the first number of passengers; and
the second significance is greater than the first significance.
4. The system of claim 2, wherein
the at least one other condition is based on the location of the passenger in the vehicle,
the controller is configured to assign a first a significance of the at least one other condition for determining the sleep risk factor when the passenger is in a front seat of the vehicle,
the controller is configured to assign a second significance of the at least one other condition for determining the sleep risk factor when the passenger is in a rear seat of the vehicle, and
the first significance is greater than the second significance.
5. The system of claim 2, wherein
the at least one other condition is based on the time information,
the time information includes an amount of time the driver has been driving the vehicle,
the time information includes a time of day when the likelihood is being determined, and
the controller is configured to alter a significance of the at least one other condition for determining the sleep risk factor based on the amount of time the driver has been driving and the time of day.
6. The system of claim 2, wherein
the at least one other condition is based on the information regarding the road the vehicle is traveling on,
the information indicates a type of the road,
the information indicates a location of the vehicle,
the information indicates a traffic condition of the road in a vicinity of the vehicle, and
the controller is configured to alter a significance of the at least one other condition for determining the sleep risk factor based on the type of the road, the location of the vehicle and the traffic condition.
7. The system of claim 1, wherein the controller is configured to determine the sleep risk factor based on a combination of respective values assigned by the controller to a passenger alertness factor, a time factor, and a road factor.
8. The system of claim 7, wherein the controller is configured to
determine the respective value of the passenger alertness factor based on
the determined alertness condition of the passenger,
a location of the passenger in the vehicle, and
a number of passengers in the vehicle;
determine the respective value of the time factor based on
an amount of time the driver has been driving the vehicle and
a time of day when the sleep risk factor is being determined; and
determine the respective value of the road factor based on
a type of road the vehicle is traveling along,
a location of the vehicle, and
a traffic condition in a vicinity of the vehicle.
9. A method of monitoring a driver, the method comprising:
detecting an alertness condition of a driver of a vehicle and an alertness condition of a passenger in the vehicle;
determining a sleep risk factor based on a combination of respective values assigned by a controller to a passenger alertness factor, a time factor, and a road factor;
determining the respective value of the passenger alertness factor based on
the determined alertness condition of the passenger,
a location of the passenger in the vehicle, and
a number of passengers in the vehicle;
determining the respective value of the time factor based on
an amount of time the driver has been driving the vehicle and
a time of day when the sleep risk factor is being determined; and
determining the respective value of the road factor based on
a type of road the vehicle is traveling along,
a location of the vehicle, and
a traffic condition in a vicinity of the vehicle;
determining a likelihood that the driver is sleepy based on the alertness condition of the driver and the sleep risk factor; and
controlling a feature of the vehicle to assist the driver when the determined likelihood satisfies a predetermined criterion.
10. The method of claim 9, wherein there is a plurality of passengers in the vehicle and the method comprises
determining an alertness condition of each of the passengers, and
altering a significance of the passenger alertness factor for determining the sleep risk factor based on the number of the passengers that are drowsy or asleep.
11. The method of claim 10, comprising
assigning a first significance when a first number of passengers is asleep or drowsy, and
assigning a second significance when a second number of passengers is asleep or drowsy;
wherein
the second number of passengers is higher than the first number of passengers, and
the second significance is greater than the first significance.
12. The method of claim 9, comprising
assigning a first a significance of the passenger alertness factor for determining the sleep risk factor when the passenger is in a front seat of the vehicle, and
assigning a second significance of the passenger alertness factor for determining the sleep risk factor when the passenger is in a rear seat of the vehicle,
wherein the first significance is greater than the second significance.
13. The method of claim 9, comprising altering a significance of the time factor for determining the sleep risk factor based on the amount of time the driver has been driving and the time of day.
14. The method of claim 9, comprising altering a significance of the road factor for determining the sleep risk factor based on the type of the road, the location of the vehicle and the traffic condition.
15. A driver monitoring system, comprising:
at least one alertness detector configured to detect an alertness condition of a driver of a vehicle and an alertness condition of a passenger in the vehicle; and
a controller configured to
determine a sleep risk factor based on the alertness condition of the passenger,
determine a likelihood that the driver is sleepy based on the alertness condition of the driver and the sleep risk factor, and
control a feature of the vehicle to assist the driver when the determined likelihood satisfies a predetermined criterion,
wherein:
the controller is configured to determine the sleep risk factor based on the alertness condition of the passenger and at least one other condition;
the at least one other condition is based on at least one of
a number of passengers in the vehicle,
a location of the passenger in the vehicle,
time information, and
information regarding a road the vehicle is traveling on,
the at least one other condition is the location of the passenger in the vehicle;
the controller is configured to assign a first a significance of the at least one other condition for determining the sleep risk factor when the passenger is in a front seat of the vehicle;
the controller is configured to assign a second significance of the at least one other condition for determining the sleep risk factor when the passenger is in a rear seat of the vehicle; and
the first significance is greater than the second significance.
US16/827,309 2020-03-23 2020-03-23 Driver alertness monitoring including a predictive sleep risk factor Active 2040-10-10 US11554781B2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US16/827,309 US11554781B2 (en) 2020-03-23 2020-03-23 Driver alertness monitoring including a predictive sleep risk factor
EP21156287.1A EP3886069A1 (en) 2020-03-23 2021-02-10 Driver alertness monitoring including a predictive sleep risk factor
CN202110301523.XA CN113425299B (en) 2020-03-23 2021-03-22 Driver alertness monitoring including predictive sleep risk factors

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US16/827,309 US11554781B2 (en) 2020-03-23 2020-03-23 Driver alertness monitoring including a predictive sleep risk factor

Publications (2)

Publication Number Publication Date
US20210291838A1 US20210291838A1 (en) 2021-09-23
US11554781B2 true US11554781B2 (en) 2023-01-17

Family

ID=74586902

Family Applications (1)

Application Number Title Priority Date Filing Date
US16/827,309 Active 2040-10-10 US11554781B2 (en) 2020-03-23 2020-03-23 Driver alertness monitoring including a predictive sleep risk factor

Country Status (3)

Country Link
US (1) US11554781B2 (en)
EP (1) EP3886069A1 (en)
CN (1) CN113425299B (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11180158B1 (en) * 2018-07-31 2021-11-23 United Services Automobile Association (Usaa) Routing or driving systems and methods based on sleep pattern information
JP2023096862A (en) * 2021-12-27 2023-07-07 パナソニックＩｐマネジメント株式会社 Awakening Support Device and Awakening Support Method
US20240109553A1 (en) * 2022-09-30 2024-04-04 Resmed Digital Health Inc. Vehicle operator sleep condition remediation
EP4408010A1 (en) * 2023-01-27 2024-07-31 Aptiv Technologies AG Method and apparatus for controlling the exposure time of an imaging device

Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040090334A1 (en) 2002-11-11 2004-05-13 Harry Zhang Drowsiness detection system and method
US7423540B2 (en) 2005-12-23 2008-09-09 Delphi Technologies, Inc. Method of detecting vehicle-operator state
US7646422B2 (en) 2006-10-04 2010-01-12 Branislav Kisacanin Illumination and imaging system with glare reduction and method therefor
US20100219955A1 (en) 2009-02-27 2010-09-02 Toyota Motor Engineering & Manufacturing NA (TEMA) System, apparatus and associated methodology for interactively monitoring and reducing driver drowsiness
US9727056B2 (en) 2015-06-24 2017-08-08 Delphi Technologies, Inc. Automated vehicle control with time to take-over compensation
US20170344620A1 (en) * 2016-05-27 2017-11-30 Adobe Systems Incorporated Feature Summarization Filter With Applications Using Data Analytics
US9988055B1 (en) 2015-09-02 2018-06-05 State Farm Mutual Automobile Insurance Company Vehicle occupant monitoring using infrared imaging
US10025316B1 (en) 2017-03-23 2018-07-17 Delphi Technologies, Inc. Automated vehicle safe stop zone use notification system
WO2018146266A1 (en) 2017-02-10 2018-08-16 Koninklijke Philips N.V. Driver and passenger health and sleep interaction
US20180319279A1 (en) * 2016-03-16 2018-11-08 Mitsubishi Electric Corporation On-vehicle apparatus, drowsy driving prevention method, and computer readable medium
US20190011918A1 (en) * 2017-07-06 2019-01-10 Lg Electronics Inc. Driving system for vehicle and vehicle thereof
US20200086891A1 (en) 2019-08-31 2020-03-19 Lg Electronics Inc. Method for controlling vehicle and intelligent computing device for controlling vehicle
US20200359934A1 (en) * 2009-09-14 2020-11-19 Sotera Wireless, Inc. Body-worn monitor for measuring respiration rate
US20210331681A1 (en) * 2019-05-31 2021-10-28 Lg Electronics Inc. Vehicle control method and intelligent computing device for controlling vehicle

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8698639B2 (en) * 2011-02-18 2014-04-15 Honda Motor Co., Ltd. System and method for responding to driver behavior
WO2015174963A1 (en) * 2014-05-13 2015-11-19 American Vehicular Sciences, LLC Driver health and fatigue monitoring system and method
EP3416147B1 (en) * 2017-06-13 2020-01-15 Volvo Car Corporation Method for providing drowsiness alerts in vehicles

Patent Citations (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7202792B2 (en) 2002-11-11 2007-04-10 Delphi Technologies, Inc. Drowsiness detection system and method
US20040090334A1 (en) 2002-11-11 2004-05-13 Harry Zhang Drowsiness detection system and method
US7423540B2 (en) 2005-12-23 2008-09-09 Delphi Technologies, Inc. Method of detecting vehicle-operator state
US7646422B2 (en) 2006-10-04 2010-01-12 Branislav Kisacanin Illumination and imaging system with glare reduction and method therefor
US20100219955A1 (en) 2009-02-27 2010-09-02 Toyota Motor Engineering & Manufacturing NA (TEMA) System, apparatus and associated methodology for interactively monitoring and reducing driver drowsiness
US20200359934A1 (en) * 2009-09-14 2020-11-19 Sotera Wireless, Inc. Body-worn monitor for measuring respiration rate
US9727056B2 (en) 2015-06-24 2017-08-08 Delphi Technologies, Inc. Automated vehicle control with time to take-over compensation
US9988055B1 (en) 2015-09-02 2018-06-05 State Farm Mutual Automobile Insurance Company Vehicle occupant monitoring using infrared imaging
US20180319279A1 (en) * 2016-03-16 2018-11-08 Mitsubishi Electric Corporation On-vehicle apparatus, drowsy driving prevention method, and computer readable medium
US20170344620A1 (en) * 2016-05-27 2017-11-30 Adobe Systems Incorporated Feature Summarization Filter With Applications Using Data Analytics
US20190357834A1 (en) * 2017-02-10 2019-11-28 Koninklijke Philips N.V. Driver and passenger health and sleep interaction
WO2018146266A1 (en) 2017-02-10 2018-08-16 Koninklijke Philips N.V. Driver and passenger health and sleep interaction
US10025316B1 (en) 2017-03-23 2018-07-17 Delphi Technologies, Inc. Automated vehicle safe stop zone use notification system
US20190011918A1 (en) * 2017-07-06 2019-01-10 Lg Electronics Inc. Driving system for vehicle and vehicle thereof
US20210331681A1 (en) * 2019-05-31 2021-10-28 Lg Electronics Inc. Vehicle control method and intelligent computing device for controlling vehicle
US20200086891A1 (en) 2019-08-31 2020-03-19 Lg Electronics Inc. Method for controlling vehicle and intelligent computing device for controlling vehicle

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
American Automobile Association Foundation for Traffic Safety, 2010 Asleep at the wheel: the prevalence and impact of drowsy driving, http://www.aaafoundation.org/pdf/2010DrowsyDrivingReport.pdf, accessed Jan. 5, 2011.
Extended European Search Report for Application No. EP 21 15 6287 dated Jul. 30, 2021.
Leger, D., 1994 The cost of sleep-related accidents: a report for the National Commission on Sleep Disorders Research, Sleep 17(1):84-93.
Sciencenorway, tuesday Apr. 25, 2017: What makes us tired in a car if other passengers are sleeping?, Nancy Bazilchuk.
Volvo will be the First Manufacturer to install Driver Monitoring and Intervention: http://bestride.com/news/volvo-will-be-the-first-manufacturer-to-install-driver-monitoring-and-intervention-to-stop-drunk-and-impaired-drivers to Stop Drunk and Impaired Drivers.

Also Published As

Publication number Publication date
US20210291838A1 (en) 2021-09-23
EP3886069A1 (en) 2021-09-29
CN113425299B (en) 2024-10-11
CN113425299A (en) 2021-09-24

Similar Documents

Publication Publication Date Title
US11554781B2 (en) Driver alertness monitoring including a predictive sleep risk factor
US20210009057A1 (en) Vehicle cabin monitoring system and temperature control
JP6627811B2 (en) Concentration determination device, concentration determination method, and program for concentration determination
JP2008070965A (en) Vehicle control apparatus and vehicle control method
US11427205B2 (en) Vehicle stop support system
WO2018168051A1 (en) Degree of concentration determination device, degree of concentration determination method, and program for determining degree of concentration
WO2018168049A1 (en) Concentration degree determination device, concentration degree determination method, and program for determining concentration degree
KR20220012490A (en) Motion sickness reduction system and method for vehicle occupants
CN113200005A (en) Method and system for controlling motor vehicle functions
JP7251524B2 (en) Drowsiness Sign Notification System, Drowsiness Sign Notification Method, and Drowsiness Sign Notification Program
CN114572238A (en) Vehicle alarm reminding method and device based on automatic driving, automobile and storage medium
WO2018168099A1 (en) Concentration degree determination device, concentration degree determination method, and program for determining concentration degree
US11203336B2 (en) Vehicle stop support system
US11541751B2 (en) Vehicle stop support system
KR20230117710A (en) Vehicle monitoring system and method for detecting and mitigatign rider safety risk in vehicle
WO2018168050A1 (en) Concentration level determination device, concentration level determination method, and program for determining concentration level
CN119343281A (en) Information processing device, information processing method and system
JP2023108495A5 (en) automatic navigation device
JP7062083B2 (en) Notification control device, notification device, notification system, and notification control method
WO2018168048A1 (en) Degree of concentration determination device, degree of concentration determination method, and program for determining degree of concentration
US12246669B2 (en) Method for combating drowsiness of a driver of a motor vehicle and electronic computing device
CN115736951B (en) Information warning method and system based on brain waves
US20250108834A1 (en) Vehicle integration control device and vehicle integration control method
US20250100348A1 (en) Method for mitigating driver distraction
JP6979311B2 (en) Visibility control device

Legal Events

Date Code Title Description
AS Assignment

Owner name: APTIV TECHNOLOGIES LIMITED, BARBADOS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:TAMILARASAN, SANTHOSH;REEL/FRAME:052198/0850

Effective date: 20200320

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: FINAL REJECTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: ADVISORY ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE AFTER FINAL ACTION FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: AWAITING TC RESP., ISSUE FEE NOT PAID

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

AS Assignment

Owner name: APTIV TECHNOLOGIES (2) S.A R.L., LUXEMBOURG

Free format text: ENTITY CONVERSION;ASSIGNOR:APTIV TECHNOLOGIES LIMITED;REEL/FRAME:066746/0001

Effective date: 20230818

Owner name: APTIV MANUFACTURING MANAGEMENT SERVICES S.A R.L., LUXEMBOURG

Free format text: MERGER;ASSIGNOR:APTIV TECHNOLOGIES (2) S.A R.L.;REEL/FRAME:066566/0173

Effective date: 20231005

Owner name: APTIV TECHNOLOGIES AG, SWITZERLAND

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:APTIV MANUFACTURING MANAGEMENT SERVICES S.A R.L.;REEL/FRAME:066551/0219

Effective date: 20231006

OSZAR »