JPH11345392A - Device and method for detecting obstacle - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect another vehicle running on the adjacent lane by correctly separating it from road marking, a vehicle shadow or the like. SOLUTION: The presence or absence of the adjacent lane and the position of a lane mark which indicates a lane boundary with the adjacent lane are detected by an adjacent lane state detecting part 9 based on an image picked up by an image pickup device 1, presence or absence of a mobile object in an area separated from one's own vehicle by prescribed distance and the correlative position and speed of the mobile object as against one's own vehicle are detected by a mobile object detecting part 11 and the processing results of the both are compared by an obstacle detecting part 11 so that excess information is separated so as to detect another vehicle to be an obstacle.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to an obstacle detecting device and an obstacle detecting method.

[0002]

2. Description of the Related Art Conventionally, as an apparatus for monitoring a predetermined area from an own vehicle and detecting an obstacle, for example, Japanese Patent Application Laid-Open No. 9-25 / 1990
As disclosed in Japanese Patent Application Laid-Open No. 9287, there is known a device that extracts an edge of an input image from a road image captured by a vehicle-mounted camera and detects a white line and an obstacle in a traveling lane based on the edge information. .

[0003]

However, in such a conventional obstacle detection apparatus, when an obstacle is detected from edge information in an image obtained by a single-lens camera, other obstacles are detected based on an edge distribution gathering at the rear of the vehicle. Since the vehicle is recognized, road markings, shadows of other vehicles, and the like are also detected as moving objects, and there has been a problem that erroneous detection occurs in the detection result of the obstacle.

The present invention has been made in view of such a conventional problem, and an obstacle detecting apparatus and an obstacle detecting apparatus capable of accurately detecting another vehicle as an obstacle separately from a road marking, a vehicle shadow, and the like. An object is to provide an object detection method.

[0005]

An obstacle detecting device according to the present invention is mounted on an own vehicle and includes an image pickup device for picking up an image of a predetermined area from the own vehicle, and an obstacle detecting device based on an image picked up by the image pickup device. An adjacent lane state detecting means for detecting the presence or absence of an adjacent lane and a position of a lane mark indicating a lane boundary of the adjacent lane which is farther from the own vehicle; Moving object detecting means for detecting the presence or absence of a moving object in a region separated by the distance of the moving object, and the relative position and relative speed of the moving object with respect to the own vehicle, the detection result by the adjacent lane state detecting means, and the moving object detecting means Obstacle detection means for detecting another vehicle that becomes an obstacle based on the detection result.

In the obstacle detecting device according to the first aspect of the present invention, a predetermined area is picked up from the own vehicle by the image pickup means, and the presence or absence of the adjacent lane is determined by the adjacent lane state detection means based on the picked-up image. And the position of the lane mark indicating the lane boundary far from the own vehicle,
In parallel with this, based on the captured image, the presence / absence of a moving object and a relative position of the moving object with respect to the own vehicle in an area (an area corresponding to an adjacent lane) separated from the own vehicle by a predetermined distance by the moving object detecting means. And the relative speed. Then, the obstacle detecting means detects the other vehicle serving as an obstacle by mutually considering the detection result by the adjacent lane state detecting means and the detection result by the moving object detecting means.

[0007] Thus, based on the image of the predetermined area taken from the own vehicle, the detection of the lane mark of the adjacent lane and the detection of the moving object are performed independently, and the detection result of the moving object is used as the lane mark of the adjacent lane. By using the detection result of the above, the extra information included in the detection of the moving object is separated from the information, and the other vehicle serving as an obstacle is accurately detected.

[0008] An obstacle detection device according to a second aspect of the present invention is provided with an image pickup means mounted on the own vehicle, for picking up an image of a predetermined area from the own vehicle, a running state detecting means for detecting a running speed of the own vehicle,
Detecting the presence or absence of an adjacent lane and the position of a lane mark indicating a lane boundary of the adjacent lane that is farther from the own vehicle based on the image captured by the imaging unit and the vehicle speed detected by the traveling state detection unit; Adjacent lane state detecting means, and the presence or absence of a moving object in a region separated by a predetermined distance from the own vehicle, and the relative position and relative speed of the moving object with respect to the own vehicle, based on the image taken by the imaging means. A moving object detecting means for detecting, and an obstacle detecting means for detecting another vehicle serving as an obstacle based on a detection result by the adjacent lane state detecting means and a detection result by the moving object detecting means. is there.

In the obstacle detecting device according to the second aspect of the present invention, a predetermined area is imaged from the own vehicle by the image pickup means, and the traveling speed of the own vehicle is detected by the traveling means detecting means. The adjacent lane state detecting means detects the presence or absence of the adjacent lane and the position of the lane mark indicating the lane boundary of the adjacent lane farther from the own vehicle, and in parallel with this, moves based on the captured image. The presence / absence of a moving object in a region (a region corresponding to an adjacent lane) away from the own vehicle by a predetermined distance and the relative position and relative speed of the moving object with respect to the own vehicle are detected by the object detecting means. Then, the obstacle detecting means detects the other vehicle serving as an obstacle by mutually considering the detection result by the adjacent lane state detecting means and the detection result by the moving object detecting means.

Thus, based on the image of the predetermined area taken from the own vehicle, the detection of the lane mark of the adjacent lane and the detection of the moving object are performed independently, and the detection result of the moving object is determined based on the lane mark of the adjacent lane. Correction is made using the detection result, and using the information on the traveling speed of the own vehicle detected by the traveling state detection means, the broken part of the adjacent lane mark is distinguished from the part that is hidden by the moving object and cannot be seen. By integrating these, extra information included in the detection of the moving object is separated from the information, and another vehicle that becomes an obstacle is detected with high accuracy.

According to a third aspect of the present invention, in the obstacle detecting device according to the first or second aspect, the adjacent lane state detecting means includes an area for detecting a lane mark in an image taken by the imaging means. Lane mark detection area setting means for setting a plurality of locations, lane mark detection means for calculating a coordinate value at which a lane mark exists from among the areas set by the lane mark detection area setting means, and lane mark detection means Based on the calculation result of
And an adjacent lane presence / absence determining means for determining the presence / absence of an adjacent lane.

In the obstacle detecting device according to the third aspect of the present invention, a plurality of regions for detecting lane marks are set in the captured image by the lane mark detecting region setting device, and the lane mark detecting device sets the region from each region. The coordinate value where the lane mark exists is calculated, and the presence or absence of the adjacent lane is determined by the adjacent lane presence / absence determining means based on the calculation result.

Thus, by detecting lane marks in a plurality of areas, the presence or absence of lane marks is reliably detected, and the detection result is used to accurately detect other vehicles that are obstacles.

According to a fourth aspect of the present invention, in the obstacle detection device according to the third aspect, the lane mark detection area setting means reduces the width of the lane from the host vehicle toward the edge of the road by half the normal lane width. Multiple locations separated by a corresponding distance,
Further, an area for detecting a lane mark is set at a plurality of locations separated by a normal lane width, and a lane mark detection area closer to the traveling lane of the own vehicle is set in a road from the own vehicle to the road. Toward the end, a plurality of points are set at positions separated by a distance corresponding to a half of the normal lane width, and the detection area of the lane mark far from the traveling lane of the own vehicle is set to the traveling lane of the own vehicle. A plurality of locations are set at positions further away from the closer detection area by the normal lane width.

Thus, the lane mark indicating the lane boundary of the adjacent lane which is farther from the own vehicle is reliably detected, and the other vehicle which becomes an obstacle is accurately detected by using the detection result.

According to a fifth aspect of the present invention, in the obstacle detecting device according to the third or fourth aspect, the lane mark detecting means sets a detection target to a white line, and the width of the white line or the contrast between the road surface and the white line. The lane mark is reliably detected using the ratio, and the other vehicle that becomes an obstacle is accurately detected using the detection result.

According to a sixth aspect of the present invention, in the obstacle detecting device according to any one of the first to fifth aspects, the moving object detecting means is provided at a predetermined distance from the own vehicle in an image taken by the image taking means. Moving object detection area setting means for setting an inspection line which is an area for detecting a moving object at a position distant from the object, and a luminance value on the inspection line set by the moving object detection area setting means, the elapsed time being changed by one line at a time. A spatio-temporal image creating means for creating a spatio-temporal image by sequentially arranging the moving object and the relative position and relative position of the moving object with respect to the own vehicle based on data in the spatio-temporal image created by the spatio-temporal image creating means. And a moving object candidate point searching means for detecting speed.

In the obstacle detecting apparatus according to the sixth aspect of the present invention, the moving object detection area setting means sets an inspection line serving as an area for detecting a moving object at a position separated from the own vehicle by a predetermined distance in the captured image. A spatio-temporal image is created by sequentially arranging the luminance values on the inspection line line by line with respect to the elapsed time by a spatio-temporal image creating means, and the data in the spatio-temporal image is created by a moving object candidate point searching means. , The relative position and the relative speed of the moving object with respect to the own vehicle are detected.

Thus, using spatiotemporal image processing, the relative position and relative speed of the other vehicle, which is an obstacle, with respect to the own vehicle can be quickly and simultaneously calculated.

According to a seventh aspect of the present invention, in the obstacle detecting device according to the sixth aspect, the moving object detection area setting means is located at a position away from the host vehicle by an ordinary lane width toward the edge of the road. An inspection line is set, whereby an area for detecting a moving object is reliably set in an adjacent lane, and only an obstacle in the adjacent lane is detected.

According to an eighth aspect of the present invention, in the obstacle detecting device according to any one of the first to seventh aspects, the obstacle detecting means is a processing result of the adjacent lane state detecting means and the moving object detecting means. , Image coordinates having the same horizontal angle information are calculated with reference to the imaging means, each detection region including the same image coordinate value is set as a current region of interest, and the processing results in the region of interest are compared. Neighboring areas for comparing the processing results of the respective detection areas including the same image coordinate value in the vicinity of the target area with respect to the processing results of the state comparing means, the adjacent lane state detecting means, and the moving object detecting means A comparison unit, and an obstacle determination unit configured to determine the presence or absence of an obstacle based on a processing result of each of the state comparison unit and the nearby area comparison unit. It is.

In the obstacle detecting device according to the eighth aspect of the present invention, the processing results of the adjacent lane state detecting means and the moving object detecting means have the same horizontal angle information based on the image pickup means by the state comparing means. Calculate the coordinates,
Each detection area including the same image coordinate value is set as a current attention area, the processing results in the attention area are compared, and the comparison result is passed to the obstacle determination unit. If there is a contradiction in the processing results, the detection results of the processing performed by the neighboring area comparison means and the moving lane detection means and the moving object detection means may include respective detection areas including the same image coordinate values in the vicinity of the noted area. Are compared with each other, and the comparison result is passed to the obstacle determining means. The obstacle determining means determines the presence or absence of an obstacle based on the result of comparison between the state comparing means and the nearby area comparing means.

In this way, a detection process using spatial information is performed in two stages to detect an obstacle, and the obstacle is separated from unnecessary information and detected accurately.

According to a ninth aspect of the present invention, in the obstacle detecting device according to any one of the first to seventh aspects, the obstacle detecting means is a processing result of the adjacent lane state detecting means and the moving object detecting means. , Image coordinates having the same horizontal angle information are calculated with reference to the imaging means, each detection region including the same image coordinate value is set as a current region of interest, and the processing results in the region of interest are compared. Neighboring areas for comparing the processing results of the respective detection areas including the same image coordinate value in the vicinity of the target area with respect to the processing results of the state comparing means, the adjacent lane state detecting means, and the moving object detecting means Based on the comparison means and the relative position and relative speed of the obstacle with respect to the own vehicle calculated based on the result of the past processing, the present area of the obstacle is predicted and calculated. That the history extraction unit, which is constituted by the respective processing results to determine obstacle determining means whether the obstacle on the basis of said state comparing means and the neighboring region comparison means said history extraction means.

In the obstacle detecting device according to the ninth aspect of the present invention, the processing results of the adjacent lane state detecting means and the moving object detecting means have the same horizontal angle information based on the imaging means by the state comparing means. Calculate the coordinates,
Each detection area including the same image coordinate value is set as a current attention area, the processing results in the attention area are compared, and the comparison result is passed to the obstacle determination unit. If there is a contradiction in the processing results, the detection results of the processing performed by the neighboring area comparison means and the moving lane detection means and the moving object detection means may include respective detection areas including the same image coordinate values in the vicinity of the noted area. Are compared with each other, and the comparison result is passed to the obstacle determining means. Further, if the presence or absence of an obstacle cannot be determined even by the vicinity area comparing means, the history extracting means determines the current position of the obstacle based on the relative position and the relative speed of the obstacle with respect to the own vehicle calculated based on the past processing result. Predicting and calculating the existence area of the obstacle, and determining whether an obstacle is detected in the corresponding area, and passing the determination result to the obstacle determination means. The obstacle determining means determines the presence or absence of an obstacle based on the processing results of the state comparing means, the nearby area comparing means, and the history extracting means.

In order to detect an obstacle, a detection process using spatial information is first performed in two stages, and then a detection process using temporal information is also performed. Separate and accurately detect.

According to a tenth aspect of the present invention, in the obstacle detection device according to any one of the first to ninth aspects, the imaging means captures an image of a rear side area of the vehicle as a predetermined area. Thus, an obstacle in the rear side area which becomes a problem when the lane of the own vehicle is changed is detected.

According to an eleventh aspect of the present invention, in the obstacle detecting method, a predetermined area is imaged from the own vehicle by the image pickup means, and the presence or absence of an adjacent lane and the lane boundary of the adjacent lane farther from the own vehicle are determined based on the image. In parallel with this, based on the image, the presence or absence of a moving object in an area corresponding to an adjacent lane and the relative position and relative speed of the moving object with respect to the own vehicle are detected. And detecting the presence or absence of another vehicle serving as an obstacle based on the detection result of the lane mark and the detection result of the moving object.

Thus, the detection of the lane mark of the adjacent lane and the detection of the moving object are independently performed based on the image of the predetermined area taken from the own vehicle, and the detection result of the moving object is converted to the lane mark of the adjacent lane. By using the detection result of the above, the extra information included in the detection of the moving object is separated from the information, and the other vehicle serving as an obstacle is accurately detected.

In the obstacle detecting method according to a twelfth aspect of the present invention, the predetermined area is imaged from the own vehicle by the image pickup means, and the traveling speed of the own vehicle is detected. The presence and absence of the lane mark indicating the lane boundary of the adjacent lane that is farther from the host vehicle are detected, and in parallel with this, based on the captured image, the presence or absence of a moving object in the area corresponding to the adjacent lane A relative position and a relative speed of the moving object with respect to the own vehicle are detected, and the presence or absence of another vehicle serving as an obstacle is determined based on the detection result of the lane mark and the detection result of the moving object.

Thus, the detection of the lane mark of the adjacent lane and the detection of the moving object are independently performed based on the image of the predetermined area taken from the own vehicle, and the detection result of the moving object is determined based on the lane mark of the adjacent lane. Correction is made using the detection result, and using the information on the traveling speed of the own vehicle detected by the traveling state detection means, the broken part of the adjacent lane mark is distinguished from the part that is hidden by the moving object and cannot be seen. By integrating these, extra information included in the detection of the moving object is separated from the information, and another vehicle that becomes an obstacle is detected with high accuracy.

The invention of claim 13 is the invention of claim 11 or 12
In the obstacle detection method according to the above, regarding the detection result of the lane mark and the detection result of the moving object, image coordinates having the same horizontal angle information are calculated based on the imaging unit, and the same image coordinate values are calculated. Each of the detection areas including the current area of interest is determined, the presence or absence of another vehicle that is an obstacle is determined by comparing the detection results in the area of interest, and if there is a contradiction between the detection results, The presence / absence of another vehicle as an obstacle is determined by comparing the detection results in the respective detection areas including the same image coordinate value in the vicinity of the target area.

As a result, a detection process using spatial information for detecting an obstacle is performed in two stages, and the obstacle is separated from unnecessary information and accurately detected.

The invention of claim 14 is the invention of claim 11 or 12
In the obstacle detection method according to the above, regarding the detection result of the lane mark and the detection result of the moving object, image coordinates having the same horizontal angle information are calculated based on the imaging unit, and the same image coordinate values are calculated. Each of the detection areas including the current area of interest is determined, the presence or absence of another vehicle that is an obstacle is determined by comparing the detection results in the area of interest, and if there is a contradiction between the detection results, The presence or absence of another vehicle that is an obstacle is determined by comparing the detection results in the respective detection areas including the same image coordinate value in the vicinity of the attention area, and the detection area in the detection area in the vicinity of the attention area is determined. If there is a contradiction between the detection results, the presence of the obstacle at the current time is further determined based on the relative position and relative speed of the obstacle with respect to the host vehicle calculated in the past. Predicting calculated range, it is obtained so as to determine the presence or absence of another vehicle which is the final obstacle depending on whether there is a moving object in the corresponding area.

In order to detect an obstacle, a detection process using spatial information is first performed in two stages, and then a detection process using temporal information is also performed. Separate and accurately detect.

[0036]

As described above, according to the obstacle detecting device of the first aspect of the present invention, the detection of the state of the adjacent lane and the detection of the moving object are performed based on the image of the predetermined area captured from the own vehicle. Independently, by correcting the detection result of this moving object using the detection result of the state of the adjacent lane, other vehicles that become obstacles can be extracted from extra information included at the time of detection of the moving object. It can be separated and detected accurately.

According to the obstacle detecting device of the second aspect of the present invention, using the information on the traveling speed of the own vehicle detected by the traveling state detecting means, a broken portion of the adjacent lane mark is found, and this is detected by the moving object. It can be distinguished from a part that is hidden behind and cannot be seen, and another vehicle that becomes an obstacle can be separated and detected with high accuracy from extra information included when a moving object is detected.

According to the obstacle detecting device of the present invention, since the lane mark is detected in a plurality of areas, the presence or absence of the lane mark can be reliably detected. The other vehicle can be accurately detected.

According to the obstacle detecting device of the fourth aspect of the present invention, a lane mark indicating a lane boundary of the adjacent lane which is farther from the vehicle can be reliably detected. The other vehicle can be accurately detected.

According to the obstacle detecting apparatus of the present invention, the lane mark can be reliably detected by using the width of the white line or the contrast ratio between the road surface and the white line, and the detected result is used as an obstacle. Another vehicle can be accurately detected.

According to the obstacle detecting device of the present invention, the relative position and the relative speed of the other vehicle, which is an obstacle, with respect to the own vehicle can be quickly and simultaneously calculated using the spatiotemporal image processing.

According to the obstacle detecting device of the present invention, the area for detecting the moving object can be reliably set in the adjacent lane, and only the obstacle in the adjacent lane can be detected.

According to the obstacle detecting device of the present invention, since the detection process using spatial information is performed in two stages to detect the obstacle, the obstacle is separated from unnecessary information. Can be detected accurately.

According to the obstacle detecting apparatus of the ninth aspect, in order to detect an obstacle, first, detection processing using spatial information is performed in two stages, and then detection using temporal information is performed. Since processing is also performed, obstacles can be accurately detected by separating them from unnecessary information.

According to the obstacle detecting device of the tenth aspect, it is possible to detect an obstacle in the rear side area which becomes a problem when the lane of the own vehicle is changed.

According to the obstacle detecting method of the eleventh aspect, based on the image of the predetermined area taken from the own vehicle,
The detection of the state of the adjacent lane and the detection of the moving object are performed independently, and the detection result of the moving object is corrected using the detection result of the state of the adjacent lane, so that another vehicle that becomes an obstacle can be moved. Can be accurately detected separately from extra information included in the detection.

According to the obstacle detecting method of the twelfth aspect, the information on the traveling speed of the own vehicle detected by the traveling state detecting means is used to find a broken portion of the adjacent lane mark, and to detect the broken portion of the adjacent lane mark. Other vehicles that can be distinguished from those that are hidden behind and become obstacles,
It is possible to separate and detect with high accuracy from extra information included in the detection of a moving object.

According to the obstacle detecting method of the thirteenth aspect, the detection process using spatial information is performed in two stages to detect an obstacle, so that the obstacle is separated from extra information. Can be detected accurately.

According to the obstacle detecting method of the fourteenth aspect, in order to detect an obstacle, first, detection processing using spatial information is performed in two stages, and then detection using temporal information is performed. Since processing is also performed, obstacles can be accurately detected by separating them from unnecessary information.

[0050]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below in detail with reference to the drawings. FIG. 1 shows one embodiment of the present invention. An obstacle detection device according to this embodiment is mounted on an own vehicle, and an imaging device 1 that images a predetermined area from the own vehicle, and a traveling state detection device 3 that detects a traveling speed of the own vehicle.
And an image processing device 5 that performs an obstacle detection process based on the image captured by the imaging device 1 and the traveling speed of the vehicle detected by the traveling state detection device 3, and a processing result of the image processing device 5. Is displayed.

A video camera is adopted as the image pickup device 1,
It is installed on a door mirror portion of the own vehicle or a center pillar portion in the vehicle in a direction in which a predetermined rear region (region 41 shown in FIG. 2) can be imaged. The traveling state detection device 3 employs a vehicle speed sensor, a wheel speed sensor, and the like. The image processing device 5 is constituted by a microcomputer, and its processing function will be described later. The display device 7 employs a CRT display device, a liquid crystal display device, or the like, and is installed in a vehicle.

The image processing apparatus 5 composed of a microcomputer is roughly divided into an adjacent lane state detecting section 9 for detecting the presence or absence of an adjacent lane and a position of a lane mark of the adjacent lane farther from the own vehicle, A moving object detecting unit 11 for detecting the presence or absence of a moving object in a region separated by a predetermined distance and a relative position and a relative speed of the moving object with respect to the own vehicle; an adjacent lane state detecting unit 9 and a moving object detecting unit 11; And a processing unit including a detection unit 13 that determines the presence or absence of an obstacle based on the detection result.

The adjacent lane state detecting section 9 includes a first area setting section 15 for setting a plurality of areas for detecting lane marks in an image picked up by the image pickup apparatus 1, and a first area setting section 15. A lane mark detection unit 17 that calculates a coordinate value at which a lane mark exists from each set area, and an adjacent lane presence / absence determination unit 19 that determines the presence or absence of an adjacent lane based on the calculation result of the lane mark detection unit 17 Consists of

The moving object detecting unit 11 sets a second area setting unit for setting an inspection line which is an area for detecting a moving object at a position separated from the own vehicle by a predetermined distance in an image picked up by the image pickup apparatus 1. 21, a spatio-temporal image creation unit 23 that creates a spatio-temporal image by sequentially arranging the luminance values on the inspection line one line at a time with respect to the elapsed time, and a movement with respect to the vehicle based on the data in the spatio-temporal image. A moving object candidate point search unit 25 for detecting the relative position and relative speed of the object.

The obstacle detecting section 13 calculates image coordinates having the same horizontal angle information with respect to the image pickup apparatus 1 based on the processing results of the adjacent lane state detecting section 9 and the moving object detecting section 11, and calculates the same image. The respective detection regions including the coordinate values are set as the current region of interest, and the state comparison unit 27 that compares the respective processing results in the region of interest and the processing results of the adjacent lane state detection unit 9 and the moving object detection unit 11 A neighborhood area comparison unit 29 that compares processing results in respective detection areas including the same image coordinate value in the vicinity of the area of interest; and a relative position and relative position of the obstacle with respect to the own vehicle calculated based on the processing results in the past. A history extracting unit 31 that predicts and calculates the current area of the obstacle based on the speed.
And an obstacle determination unit 33 that determines the presence or absence of an obstacle based on the processing results of the state comparison unit 27, the nearby area comparison unit 29, and the history extraction unit 31.

Next, the operation of the obstacle detecting device according to the present embodiment will be described. As shown in FIG. 2, the rear side region 41 is imaged from the own vehicle 37 by the imaging device 1 of the own vehicle 37, and the traveling speed of the own vehicle 37 is detected by the traveling state detection device 3. These pieces of information are sent to the image processing device 5.

The image sent to the image processing device 5 is processed by the adjacent lane state detecting section 9 and the moving object detecting section 11. In the adjacent lane state detection section 9, the first area setting section 15 sets a plurality of areas for detecting lane marks of adjacent lanes in the image. The set position is, for example, in the lane mark detection area closer to the traveling lane of the own vehicle 37, from the own vehicle 37 toward the edge of the road by a distance equivalent to 1/2 of a normal lane width. The lane mark detection area which is set at a plurality of locations along the traveling line passing through the distant position and which is far from the traveling lane of the own vehicle 37 is further increased from the detection area closer to the traveling lane of the own vehicle 37. Are set at a plurality of locations along the traveling line, passing through a position separated by a normal lane width toward the end of the road.

From each of the detection areas thus set, the coordinates at which the lane mark exists are calculated by the lane mark detection unit 17. The target to be detected as a lane mark is a white line. Straight line detection using Hough transform is performed in each area, and the presence / absence information of the lane mark for each area and the coordinates where at least one lane mark exists when the lane mark is detected are obtained.

Then, based on the number of areas where white lines as lane marks have been detected in the preprocessing, the presence / absence of an adjacent lane in the set area is determined by the adjacent lane presence / absence determining unit 19. At this time, if the number of areas where lane marks are detected is equal to or greater than a predetermined value, it is determined that an adjacent lane exists in the set area. If it is determined that there is an adjacent lane, but there are some areas in the set area where no lane mark is detected, those areas may be areas where lane marks are hidden by obstacles or lane marks. Is one of the broken parts. Therefore, the traveling speed of the vehicle 37 sent from the traveling state detection device 3 is compared with the temporal change amount of the area where the lane mark is not detected, and those areas are compared with those of the area where the lane mark is interrupted. Is determined. If it is determined that the area is not a part where the lane mark is interrupted, it is determined that the area has the lane mark hidden by the obstacle, and the processing results are sent to the obstacle detection unit 13. .

On the other hand, with respect to the image sent from the imaging device 1 to the moving object detection unit 11, the second area setting unit 21 follows a traveling line passing a position away from the host vehicle 37 by a predetermined distance. Thus, an inspection line 49 that is an area for detecting a moving object is set. The inspection line 49 is set at a position away from the host vehicle 37 by the normal lane width.

The spatio-temporal image creating section 23 creates a spatio-temporal image by sequentially arranging the luminance values on the inspection line 49 one line at a time every fixed time. The data in the spatiotemporal image is converted into a space representing the relative position and relative speed of the moving object with respect to the own vehicle 37 by the moving object candidate point search unit 25, and the relative position and relative speed are calculated simultaneously. You. This processing result is sent to the obstacle detection unit 13.

The processing results sent from the adjacent lane presence / absence determining section 19 in the adjacent lane state detecting section 9 and the moving object candidate point searching section 25 in the moving object detecting section 11 to the obstacle detecting section 13 are as follows: The state comparison unit 27 performs comparison processing.
At this time, image coordinates having the same horizontal angle information are calculated with reference to the imaging device 1, and the respective detection regions including the same image coordinate values are moved to the adjacent lane presence / absence determination unit 19 as current regions of interest. The processing result with the object candidate point search unit 25 is compared. If the comparison results in inconsistencies in the respective processing results, the processing results are sent to the neighboring area comparing unit 29.

In the neighboring area comparing section 29, at least one of the detection areas including the same coordinate value in the vicinity of the current area of interest is used by the adjacent lane presence / absence determining section 19 and the moving object candidate point searching section 25. The processing results are compared, and an obstacle is determined based on the presence or absence of spatial continuity. If inconsistency arises in each processing result even by this comparison processing, the processing result is sent to the history extracting unit 31.

The history extracting unit 31 predicts and calculates the present area of the obstacle based on the relative position and relative speed of the obstacle with respect to the host vehicle 37 calculated at least one time before. The result is sent to the obstacle determining unit 33.

The obstacle determining unit 33 determines whether or not there is an obstacle, the relative position of the obstacle with respect to the host vehicle 37, and the position of the obstacle based on the processing results of the state comparing unit 27, the nearby area comparing unit 29, and the history extracting unit 31. The relative speed is detected, and the result is sent to the display device 7 or the external device 35.

In the external device 35, the processing result sent from the image processing device 5 transmits, for example, a lane departure warning by an alarm device (not shown), or performs automatic obstacle avoidance control by a vehicle control device. Used for

The above operation will be described in more detail. FIG.
Assuming that the own vehicle 37 and the other vehicle 39 are traveling in the positional relationship shown in FIG. 7, the processing operation of detecting the other vehicle 39 as an obstacle will be described.

First, the image information of the image pickup area 41 picked up by the image pickup device 1 installed in the own vehicle 37 and the information on the running speed of the own vehicle 37 measured by the running state detecting device 3 are converted into an image processing device. Sent to 5.

FIG. 3 shows an image sent to the image processing device 5. In the image processing device 5, a plurality of lane mark detection areas 45 are set on the adjacent white line 43 in the image, and the presence or absence of the adjacent white line 43 is detected for each lane mark detection area 45. In addition, an inspection line 49 which is a detection area of a moving object is almost at the center of the adjacent lane 47.
Is set, and the presence or absence of a moving object on the inspection line 49 is detected. Then, each detection processing result and the own vehicle 3
7 based on information on the traveling speed of the other vehicle 39.
The relative position and relative speed of the vehicle 37 with respect to the own vehicle 37 are detected, and the information is sent to the display device 7 and the external device 35.

FIG. 4 is a flowchart showing a process for detecting an obstacle in the rear side area. First, an image of the rear side of the host vehicle 37 is captured by the imaging device 1, and an edge image is created by performing a spatial differentiation process on the image.
This image information is sent to the image processing device 5 (S
1).

Next, the adjacent lane state detecting section 9
An area that is considered to include the adjacent white line 43 in the image sent from the imaging apparatus 1 is divided into a plurality of areas, each of which is set as a lane mark detection area 45, and the lane mark detection area 45 The presence or absence is detected. At this time, the lane mark detection area 45 in which the adjacent white line 43 is not detected is either a portion where the adjacent white line 43 is hidden by an obstacle or a portion where the adjacent white line 43 is interrupted. Then, the traveling speed of the vehicle 37 sent from the traveling state detection device 3 is compared with the temporal change amount of the area where the adjacent white line 43 is not detected, and the area is determined by the interruption of the adjacent white line 43. It is determined whether or not it is the part that has been set. Accordingly, when it is determined that the area is not a part where the adjacent white line 43 is interrupted, it is determined that the adjacent white line is hidden by the obstacle, and the processing results are transmitted to the obstacle detection unit 13. (S2).

On the other hand, the moving object detecting section 11 detects a moving object such as another vehicle 39 in the adjacent lane 47 based on the image sent from the imaging device 1, and detects the moving object relative to the own vehicle 37. The position and the relative speed are calculated, and the result is sent to the obstacle detection unit 13 (S3).

The processes from (S1) to (S3) can use, for example, the method described in Japanese Patent Application Laid-Open No. 9-240397.

The obstacle detecting section 13 compares the position of the area where the obstacle is obtained by the adjacent lane state detecting section 9 with the relative position of the moving object obtained by the moving object detecting section 11. Then, in the area including the same image coordinate value as the area where the moving object is detected by the moving object detection unit 11, the adjacent white line 43
If is not detected, it is determined that an obstacle having a three-dimensional effect is present, and the relative position and relative speed of the obstacle are output as the final result. If the adjacent white line 43 is detected in an area including the same image coordinate value as the area where the moving object is detected, the moving object is determined to have no three-dimensional effect such as a road surface display and is excluded from the processing target. Is performed (S4).

Thereafter, the process returns to (S1) and the process is repeated, whereby the presence or absence of an obstacle and the relative positional relationship in the rear side area are continuously detected.

The processing of (S4) will be described in more detail. FIG. 5 is a conceptual diagram of the detection processing in the obstacle detection unit 13. In the present invention, there are two processes in which the information of the other vehicle 39 is included in the process result. One is an adjacent white line (lane mark) detection process, and the other is a moving object candidate point search process. The detection process in the obstacle detection unit 13 integrates the detection process result of the adjacent white line 43 and the search result of the candidate point of the moving object to determine the candidate point of the object having no height such as noise and road surface display. The moving object is deleted from the candidate point list.

For this purpose, first, the result of the process of detecting the adjacent white line 43 in the region of interest and the result of the process of searching for candidate points of a moving object are input to the obstacle detection unit 13.

The adjacent white line detection processing is an algorithm for searching for a non-existent area of another vehicle 39, and the processing result is position information of the adjacent white line 43. Adjacent lane 47
When there is another vehicle 39 inside, the adjacent white line 43 is not detected in the lane mark detection area 45 of the portion hidden by the other vehicle 39. Therefore, this processing result also includes information on the area where the other vehicle 39 exists. However, as shown in FIG. 5, there is a case where the adjacent white line 43 is erroneously detected in an area where the adjacent white line 43 does not actually exist due to extra information such as the lateral edge of the other vehicle 39 based on the processing result alone.

On the other hand, the moving object candidate point search processing is an algorithm for searching for an existing area of another vehicle 39, and the processing result is a moving object candidate point in the adjacent lane 47. As shown in FIG. 5, this processing result may include extra information such as road surface display and joints, image noise, and the like, in addition to the information on the area where the other vehicle 39 is present.

The obstacle detector 13 compares and integrates these two processing results to remove extra information included in each processing result, and to accurately determine whether or not another vehicle 39 exists, and Comprehensive determination for calculating the relative position and relative speed of the other vehicle 39 with respect to the own vehicle 37 is performed.

FIG. 6 shows a flow of the comprehensive judgment in the obstacle detecting section 13. In order to prevent leakage of information about the other vehicle 39 in the moving object candidate point search process in the previous stage, the search threshold is set to a small value. As a result, extra information included in the processing result also increases, but this information is sent to the obstacle detection unit 13 as a moving object candidate point list as it is.

In the obstacle detection unit 13, first, as shown in FIG. 7A, a detection result of a candidate point of a moving object and a lane mark detection are set with each detection area including the same image coordinate value as a current attention area. The result is compared.

As a result, if the adjacent white line 43 as the lane mark is not detected in the lane mark detection area 45 including the x address on the same image plane as the candidate point of the moving object detected on the inspection line 49. The candidate point of the moving object is left as it is (in the case of a-3).

On the other hand, when the adjacent white line 43 is detected in the lane mark detection area 45 including the x address on the same image plane as the candidate point of the moving object (case a-2),
There are two reasons. One is a case where a road surface display, a seam, an image noise or the like is erroneously detected as a moving object on the inspection line 49, and the other is a case where a lateral edge of the vehicle is adjacent to the lane mark detection area 45. This is a case where the white line 43 is erroneously detected.

Then, the process of FIG. 13B is started, and the vicinity of the currently processed x address in the Hough plane is examined. If another candidate point of the moving object is not detected in the vicinity, it is determined in the pre-processing that the road surface display, the seam, the image noise, and the like have been erroneously detected as the moving object (in the case of b-4). . On the other hand, when another candidate point of the moving object is detected in the vicinity, the same x as the candidate point of the moving object
It is checked whether or not the adjacent white line 43 is detected in the lane mark detection area 45 including the address. If the adjacent white line 43 is not detected, the candidate point of the moving object is left as it is (in the case of b-3).

If the adjacent white line 43 is detected (in the case of b-2), the process of FIG. 9C is started, and the candidate point of the moving object is detected in the area by the previous detection process. The current candidate point is left as it is (in the case of c-1), and if it is not detected, the current candidate point is deleted (c-2).
in the case of).

In the process shown in FIG. 9A, when no candidate point of the moving object is detected in the target area and the adjacent white line 43 is detected in the lane mark detection area 45 including the same x address. , The candidate point of the moving object is deleted (a-4).

In the process of FIG. 11A, although the candidate point of the moving object is detected in the region of interest, and the adjacent white line 43 is detected by the lane mark detection result, (a-2)
In the case of (b), when neither the candidate point of the moving object nor the adjacent white line 43 is detected near the region of interest in the process of FIG. It is determined that the horizontal edge has been erroneously detected as the adjacent white line 43. Then, next, it is verified whether or not the candidate point of the moving object has been detected in the area by the previous detection processing, and if it has been detected, the current candidate point is left as it is (in the case of c-1), If not detected, the current candidate point is deleted (in the case of c-2).

Further, in the process shown in FIG. 11A, even when neither a candidate point of the moving object nor the adjacent white line 43 is detected in the region of interest (case a-1), the process goes through the same steps. If the processing result finally indicates that a moving object is present, the moving object is newly added to the candidate point list.

As a result of this determination processing, a new moving object candidate point list is obtained in which extra information has been deleted from the candidate point list obtained by the moving object candidate point search processing. In this new moving object candidate point list, the x address of each candidate point indicates the relative position of the vehicle. By re-searching the Hough plane based on the x address, the y address of the pixel having the maximum value in one vertical line of the x address is calculated as the relative speed of the vehicle.

As described above, according to the obstacle detecting device of the present embodiment, when detecting another vehicle that is an obstacle, the detection of the moving object on the adjacent lane is performed using the detection result of the lane mark of the adjacent lane. Since the other vehicle as an obstacle is detected by correcting the detection result, erroneous detection of extra information such as a road sign and a shadow of the vehicle can be greatly reduced. Furthermore, even if inconsistency occurs in both the processing of moving object detection and lane mark detection, the detection rate is analyzed by analyzing the detection results in terms of spatial continuity and temporal continuity. Can be greatly improved. Then, the relative position and the relative speed of the obstacle with respect to the own vehicle 37 are quickly and simultaneously calculated by the spatiotemporal image processing, so that it is possible to easily determine the risk of the obstacle exerting on the own vehicle 37. In addition, spatial information is given priority over two stages to make a decision, and if necessary, temporal information is used as a supplement to make a decision. Obstacles can be accurately detected.

In the above embodiment, as shown in FIG. 6, the obstacle is detected in three stages (a) to (c). However, the obstacle is detected in two stages (a) and (b). It may be configured to end with That is, in the vicinity detection processing, the presence or absence of an obstacle is determined based on the result of detecting a candidate point of a moving object.
As a result, the processing speed of the system can be increased.

Further, in the above-described embodiment, the broken portion of the lane mark is determined using the own vehicle speed. However, this function can be omitted. In this case, the break portion of the lane mark is also regarded as a region where the lane mark is not detected, and the subsequent processing is performed. As a result, the system processing can be simplified and speeded up.

Further, in the above embodiment, the inspection line 49 for the moving object is set at a position away from the host vehicle 37 by the normal lane width. This simplifies the arithmetic processing. However, this set position is obtained by obtaining information of the adjacent lane from the adjacent lane state detection unit 9 side.
If there is an adjacent lane, the first processing or the own vehicle 3 if there is no adjacent lane.
7 may be set at a position apart from the normal lane by the normal lane width. Thereby, the detection accuracy of the moving object can be improved.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an embodiment of an obstacle detection device according to the present invention.

FIG. 2 is an explanatory diagram of a situation example of the embodiment.

FIG. 3 is an image diagram taken in the above situation example.

FIG. 4 is a flowchart illustrating an obstacle detection process.

FIG. 5 is a conceptual diagram of an obstacle detection process.

FIG. 6 is a diagram showing a flow of an obstacle presence / absence determination process.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 imaging device 3 running state detection device 5 image processing device 7 display device 9 adjacent lane state detection unit 11 moving object detection unit 13 obstacle detection unit 15 first area setting unit 17 lane mark detection unit 19 adjacent lane presence / absence determination unit 21 Second region setting unit 23 Spatiotemporal image creation unit 25 Moving object candidate point search unit 27 State comparison unit 29 Neighborhood region comparison unit 31 History extraction unit 33 Obstacle determination unit 35 External device 37 Own vehicle 39 Other vehicle 41 Imaging region 43 Adjacent white line 45 Lane mark detection area 47 Adjacent lane 49 Inspection line

Claims (14)

[Claims] 1. An image pickup unit mounted on an own vehicle to image a predetermined area from the own vehicle, and based on an image picked up by the image pickup unit, the presence or absence of an adjacent lane and a lane of the adjacent lane farther from the own vehicle. An adjacent lane state detecting means for detecting a position of a lane mark indicating a boundary, and the presence or absence of a moving object in a region separated from the own vehicle by a predetermined distance based on an image taken by the imaging means, and A moving object detecting means for detecting a relative position and a relative speed with respect to the own vehicle; and detecting another vehicle serving as an obstacle based on a detection result by the adjacent lane state detecting means and a detection result by the moving object detecting means. An obstacle detection device comprising: an obstacle detection unit. 2. An image pickup device mounted on the own vehicle and imaging a predetermined area from the own vehicle, a running state detecting device for detecting a running speed of the own vehicle, an image picked up by the imaging device, and the running state detection. An adjacent lane state detecting means for detecting the presence or absence of an adjacent lane and a position of a lane mark indicating a lane boundary of the adjacent lane farther from the own vehicle based on the own vehicle speed detected by the means; Moving object detecting means for detecting the presence or absence of a moving object and a relative position and a relative speed of the moving object with respect to the own vehicle in an area separated by a predetermined distance from the own vehicle based on the obtained image; Detecting means for detecting another vehicle that is an obstacle based on the detection result by the detecting means and the detection result by the moving object detecting means. Location. 3. The lane mark detection area setting means for setting a plurality of lane mark detection areas in an image picked up by the image pickup means, and the lane mark detection area setting means. Lane mark detecting means for calculating a coordinate value at which a lane mark exists from each of the areas set by the method, and an adjacent lane presence / absence determining means for determining the presence or absence of an adjacent lane based on the calculation result of the lane mark detecting means The obstacle detection device according to claim 1, comprising: 4. The lane mark detection area setting means,
From the vehicle to the edge of the road, 1/1 of the normal lane width
4. The obstacle according to claim 3, wherein lane mark detection areas are set at a plurality of positions separated by a distance equivalent to 2 and further at a plurality of positions separated by a normal lane width. Object detection device. 5. The obstacle detection device according to claim 3, wherein the lane mark detection means sets a detection target to a white line. 6. The moving object detection means for setting an inspection line serving as an area for detecting a moving object at a position separated from the own vehicle by a predetermined distance in an image taken by the imaging means. Area setting means; spatio-temporal image creating means for creating a spatio-temporal image by sequentially arranging the luminance values on the inspection line set by the moving object detection area setting means line by line with respect to elapsed time; 2. A moving object candidate point searching means for detecting a relative position and a relative speed of the moving object with respect to the own vehicle based on data in the spatiotemporal image created by the spatial image creating means. 6. The obstacle detection device according to any one of claims 5 to 5. 7. The moving object detection area setting means sets an inspection line at a position away from a host vehicle toward an edge of a road by a normal lane width.
An obstacle detection device according to claim 1. 8. The obstacle detecting means calculates image coordinates having the same horizontal angle information with respect to the imaging means, based on the processing results of the adjacent lane state detecting means and the moving object detecting means, Regarding the respective detection areas including the same image coordinate value as the current area of interest, the state comparison means for comparing the processing results in the area of interest, and the processing results of the adjacent lane state detection means and the moving object detection means A neighborhood area comparison means for comparing the processing results in the respective detection areas including the same image coordinate value in the vicinity of the area of interest; and a processing result of the state comparison means and the neighborhood area comparison means. The obstacle detection device according to any one of claims 1 to 7, further comprising: an obstacle determination unit configured to determine presence / absence of an obstacle. 9. The obstacle detection means calculates image coordinates having the same horizontal angle information with respect to the image pickup means, based on processing results of the adjacent lane state detection means and the moving object detection means, Regarding the respective detection areas including the same image coordinate value as the current area of interest, the state comparison means for comparing the processing results in the area of interest, and the processing results of the adjacent lane state detection means and the moving object detection means A neighborhood area comparison means for comparing the processing results in the respective detection areas including the same image coordinate value in the vicinity of the target area, and a relative position of the obstacle with respect to the own vehicle, calculated based on the past processing results, and History extracting means for predicting and calculating an existing area of the obstacle based on the relative speed, the state comparing means, the neighboring area comparing means, and the history extracting means; The obstacle detection device according to any one of claims 1 to 7, further comprising: an obstacle determination unit configured to determine the presence or absence of an obstacle based on a processing result of each of the means and the unit. 10. The imaging device according to claim 1, wherein the imaging unit captures an image of a rear side area of the vehicle as the predetermined area.
The obstacle detection device according to any one of claims 9 to 13. 11. A predetermined area is picked up from an own vehicle by an image pickup means, and the presence or absence of an adjacent lane and a position of a lane mark indicating a lane boundary of the adjacent lane farther from the own vehicle are detected based on the picked-up image. In parallel with this, based on the image, the presence or absence of a moving object in an area corresponding to an adjacent lane and the relative position and relative speed of the moving object with respect to the own vehicle are detected. An obstacle detection method, wherein the presence or absence of another vehicle serving as an obstacle is determined based on the detection result of the moving object. 12. An image of a predetermined area from an own vehicle by an image pickup means, a traveling speed of the own vehicle is detected, and, based on the captured image and the own vehicle speed, the presence or absence of an adjacent lane and a direction of the adjacent lane farther from the own vehicle. In parallel with this, based on the captured image, the presence or absence of a moving object in the area corresponding to the adjacent lane, the relative position of the moving object with respect to the own vehicle, and An obstacle detection method comprising: detecting a relative speed; and determining the presence or absence of another vehicle serving as an obstacle based on the detection result of the lane mark and the detection result of the moving object. 13. The obstacle detection method according to claim 11, wherein the detection result of the lane mark and the detection result of the moving object have the same horizontal angle information based on the imaging unit. The coordinates are calculated, each detection region including the same image coordinate value is set as a current region of interest, and the presence or absence of another vehicle that is an obstacle is determined by comparing the detection results in the region of interest with the detection result. If there is a contradiction between the two, it is further determined whether or not there is another vehicle that is an obstacle by comparing the detection results in the respective detection areas including the same image coordinate value in the vicinity of the target area. An obstacle detection method comprising: 14. The obstacle detection method according to claim 11, wherein the detection result of the lane mark and the detection result of the moving object have the same horizontal angle information based on the imaging unit. The coordinates are calculated, each detection region including the same image coordinate value is set as a current region of interest, and the presence or absence of another vehicle that is an obstacle is determined by comparing the detection results in the region of interest with the detection result. If there is a contradiction between, further determine the presence or absence of other vehicles that are obstacles by comparing the detection results in each detection area including the same image coordinate value in the vicinity of the attention area, If there is a contradiction between the detection results in the detection area in the vicinity of the area of interest, furthermore, the relative position and the relative speed of the obstacle calculated in the past with respect to the own vehicle are determined. An obstacle detection method comprising: predicting and calculating an existing area of an obstacle on the basis of the above, and finally determining the presence or absence of another vehicle as an obstacle based on whether or not there is a moving object in the area.