CN109801265B - A real-time foreign object detection system for power transmission equipment based on convolutional neural network - Google Patents

Abstract

The invention relates to a system for detecting foreign objects (including kites, plastic bags, etc.) in real-time power transmission equipment based on a convolutional neural network. In low-cost embedded devices, a set of deep learning models that have been trained on the server are used to detect whether there are foreign objects and the location of foreign objects in the power transmission equipment in real time, and then transmit this information back to the server. The deployed recognition model utilizes a deep separable convolutional neural network to extract image features for efficient real-time detection in the Faster R‑CNN method optimized for foreign object features. The invention can detect whether there are foreign objects in the power transmission equipment in real time, save a lot of manpower to detect the condition of the power transmission equipment on site, and ensure the stable operation of the power transmission equipment.

Description

A real-time foreign object detection system for power transmission equipment based on convolutional neural network

technical field

The invention relates to the fields of electric power systems and computer vision, in particular to a system for real-time foreign object detection in power transmission equipment based on a convolutional neural network.

Background technique

If there are foreign objects in power transmission equipment (including kites, plastic bags and other things that power transmission equipment should not have), it may affect the stability of power transmission, and even cause serious safety problems. It is very important to check and clean up these foreign objects in time. necessary. At present, the inspection and maintenance of power transmission equipment mainly relies on on-site inspection, and the human eye can identify whether there is an abnormal situation. In recent years, due to the development of drone technology, drones can be used to take pictures, and then these photos can be screened by personnel. , which saves some manpower, but still cannot meet the needs of intelligence, and the efficiency of testing and maintaining power transmission equipment is still not high.

In recent years, the performance of embedded devices has been continuously improved, and a number of embedded chips with superior performance have appeared, such as TX2, which makes it possible to do deep learning on embedded devices, but these devices are relatively expensive and not suitable for large-scale application. Raspberry Pi is an ARM-based micro computer motherboard with all the basic functions of a PC. The price and cost of Raspberry Pi are very low compared to TX2, even less than one tenth, but its performance is poor, especially for deep learning. If you want to deploy deep learning models on Raspberry Pi, you need to Optimize the model.

Faster R-CNN is a target detection algorithm based on convolutional neural network. Faster R-CNN improves FastR-CNN and incorporates the extraction part of candidate regions into the entire neural network to realize an end-to-end target detection model. . Faster R-CNN consists of two parts, Fast R-CNN detector and region proposal network (RPN, region proposal network). By sharing the convolutional neural network with Fast R-CNN, RPN can extract candidate regions with only a small increase in time. In addition, the co-training of these two sub-networks improves the ability of the entire CNN to extract features and improve the detection accuracy. RPN is essentially a fully convolutional network that takes the convolutional feature map matrix extracted from the original image as input and outputs a series of rectangular candidate regions and objectness scores of whether the rectangular candidate region is an object. The RPN network performs sliding window processing on the feature map matrix output by the last layer of the convolutional layer of the original image with an n*n window. , the convolution layer composed of convolution kernels with stride 1 is mapped to a d-dimensional feature vector, and the d-dimensional feature vector is used as the input of the two fully connected layers. Then use these two fully connected layers to do front/background classification and regression box position respectively. Map the proposed window to the last layer of convolutional feature map of CNN, generate a fixed-size feature map for each RoI through the RoI pooling layer, and then classify in the obtained candidate frame to determine what category it belongs to.

Faster r-cnn is a very mature method in the field of target detection, but this method requires a lot of memory consumption, low-cost embedded devices cannot be well supported, and if this method is not optimized, it will The running speed in the CPU of the device is very slow and cannot meet the actual application requirements.

In addition to Faster R-CNN, there are also some faster lightweight target detection models, but the recognition accuracy will decrease, and the effect on small object detection is particularly bad, and many foreign objects on power transmission equipment are relatively small.

The depthwise separable convolution structure is an improvement on the ordinary convolution structure, which decomposes the traditional convolution operation into a depthwise convolution and a 1*1 pointwise convolution operation. Each filter in Depthwise convolution is responsible for convolution in a channel of input, and 1*1 convolution is responsible for merging the results of depthwise convolution. In this way, the calculation amount of the traditional convolution operation is greatly reduced. For a 3*3 convolution kernel, the calculation amount can be reduced to about 1/9.

SUMMARY OF THE INVENTION

Aiming at the technical problems existing in the prior art, the present invention provides a system for real-time foreign object detection in power transmission equipment based on a convolutional neural network.

The technical solution of the present invention to solve the above-mentioned technical problems is as follows: a system for real-time foreign object detection in power transmission equipment on embedded equipment based on a depthwise separable convolutional neural network, comprising the following modules:

The foreign object image sample library module obtains the images of the power transmission equipment in advance, manually labels them, and uses these data to build a sample library. The server stores the image information returned from the embedded equipment, and regularly reviews these data manually. If the detection result is If it is correct, mark the original image, store it in the sample library, and regularly train the existing model with new data to update the parameters of the existing model;

The neural network model module builds a target detection model based on a depthwise separable convolutional neural network, and uses the data in the sample library to train the target detection model;

The device module is detected in real time, and the camera facing the power transmission device is used to periodically take pictures and transmit them to the embedded device. Since the embedded device uses a low-cost Raspberry Pi, each camera can be equipped with a set of embedded devices. The Raspberry Pi board can be inserted into the SD card to expand the storage, and the trained neural network model is used in the embedded device to detect the incoming image in real time;

The information return module transmits the data of the detected foreign objects including the original image back to the server, and the server makes corresponding processing after receiving the information of the power defect intelligent identification device;

Further, the foreign body image sample library module includes:

The images of power transmission equipment are captured by drones, cameras and cameras, mainly images with foreign objects, these images are preprocessed, the positions of foreign objects are marked manually, and these data are used to establish a foreign object detection sample library for power transmission equipment, which is stored on the server.

Further, the neural network model module includes:

Build a target detection model based on a deep separable convolutional neural network. This model is an improvement and optimization of the Faster R-CNN model for foreign objects in power transmission equipment. Improvements and optimizations have been made in the following aspects:

Replace all ordinary convolutional structures in Faster R-CNN with depthwise separable convolutional structures, which can reduce the amount of computation of each convolutional layer. Assuming that the original convolutional layer is 3*3, it can be reduced to the original 1/9 of;

Reduce the number of candidate frames retained after non-maximum suppression in the RPN structure, because in the case of foreign object detection in power transmission equipment, the number of detected targets will not be too many, so there is no need to generate too many candidate frames, which can save time;

Replacing the last classified two-layer fully connected layer structure with a convolutional structure can further reduce the amount of parameters and speed up the detection speed;

Further, the information return module includes:

The data of detected foreign objects is sent back to the server. After receiving the information of detected foreign objects, the server will generate corresponding alarm information, and save the detected images of foreign objects in the cluster. The interactive server accesses the data in the cluster, The user is reminded in time to deal with the equipment failure, and the remote server can also directly retrieve the content of the camera.

Compared with the prior art, the innovation of the present invention is that the deep separable convolutional neural network is applied to the scenario of foreign object detection in power transmission equipment, and optimized for this scenario, so that it can be deployed in low-cost The equipment can also achieve relatively good results, which greatly saves the manpower of on-site detection of foreign objects in power transmission equipment or checking the situation of power transmission equipment one by one according to the original image taken by the camera.

Description of drawings

Fig. 1 is the system structure diagram of real-time foreign object detection of power transmission equipment on embedded equipment provided by the present invention;

FIG. 2 is a structural diagram of Faster R-CNN after improvement and optimization for foreign object detection in power transmission equipment provided by the present invention.

Detailed ways

Referring to Figure 1, a system for real-time foreign object detection in power transmission equipment based on convolutional neural network includes the following modules:

The foreign object image sample library module takes images of power transmission equipment, mainly shooting images of foreign objects (including kites and plastic bags), preprocesses these images, manually marks the position of foreign objects, and uses these data to establish a foreign object detection sample library for power transmission equipment, Stored on the server, the server will also store the image information returned from the embedded device, and the data will be reviewed manually on a regular basis. If the detection result is correct, the original image will be marked and stored in the sample library;

The neural network model module builds a target detection model based on a depthwise separable convolutional neural network, and uses the data in the sample library to train the target detection model;

The device module is detected in real time, and the camera facing the power transmission device is used to periodically take pictures and transmit them to the embedded device. Since the embedded device uses a low-cost Raspberry Pi, each camera can be equipped with a set of embedded devices. The Raspberry Pi board can be inserted into the SD card to expand the storage, and the trained model is used in the embedded device to detect the incoming image in real time;

The information return module transmits the data of the detected foreign object including the original image to the server, and the server makes corresponding processing after receiving the information of the detected foreign object.

Further, the foreign body image sample library module includes:

The images of power transmission equipment are captured by drones, cameras and cameras, mainly images with foreign objects, these images are preprocessed, the positions of foreign objects are marked manually, and these data are used to establish a foreign object detection sample library for power transmission equipment, which is stored on the server.

Further, the neural network model module includes:

Build a target detection model based on a deep separable convolutional neural network. This model is an improvement and optimization of the Faster R-CNN model for foreign objects in power transmission equipment. See Figure 2. Improvements and optimizations have been made in the following aspects:

Replace all ordinary convolutional structures in Faster R-CNN with depthwise separable convolutional structures (including Convlayers, convolutional layers in RPN, and convolutional layers that replace fully connected layers after ROIPooling);

Reduce the number of candidate boxes retained after non-maximum suppression in the RPN structure, for example, from 300 to 50;

Change the fully connected layer structure after ROIPooling to a convolutional structure, for example, replace the fully connected layer with 2 layers of 4096 outputs to 2 consecutive convolution layers plus relu layer;

Further, the information return module includes:

The data of detected foreign objects is sent back to the server. After receiving the information of detected foreign objects, the server will generate corresponding alarm information, and save the detected images of foreign objects in the cluster. The interactive server accesses the data in the cluster, The user is reminded in time to deal with the foreign matter of the power transmission equipment, and the remote server can also directly retrieve the content of the camera.

Claims (2)

1. a system based on the real-time power transmission equipment foreign body detection of convolutional neural network, is characterized in that, comprises following module: The foreign object image sample library module takes images of power transmission equipment and images with foreign objects, preprocesses these images, manually marks the position of foreign objects, and uses these data to establish a foreign object detection sample library for power transmission equipment, which is stored on the server, and the server will also Store the image information returned from the embedded device, and periodically review the data manually. If the detection result is correct, mark the original image and store it in the sample library; The neural network model module builds a target detection model based on a depthwise separable convolutional neural network, and uses the data in the sample library to train the target detection model. This model is an improvement and optimization of the Faster R-CNN model for foreign objects in power transmission equipment; Replace all ordinary convolutional structures in Faster R-CNN with depthwise separable convolutional structures; reduce the number of candidate frames retained after non-maximum suppression in the RPN structure; replace the last classified two-layer fully connected layer The structure is replaced by a convolution structure; Real-time detection of the device module, using the camera facing the power transmission equipment to periodically take pictures and transfer them to the embedded device, equip each camera with a set of embedded processing equipment, insert the sd card to expand the storage, in the embedded device The trained neural network model is used to detect the incoming image in real time; The information return module sends the data of detected foreign objects back to the server. After the server receives the information of detected foreign objects, it will generate corresponding alarm information, and save the detected images of foreign objects in the cluster, and the interactive server can access the The data in the cluster reminds the user to deal with the equipment failure in time, and the remote server directly retrieves the content of the camera. 2 . The system according to claim 1 , wherein the detected target is foreign objects in power transmission equipment. 3 .

Images