Detecting anomalies on UAV photos from scratch using YOLOv5

We had a task to detect anomalies (oil spill, excavation, materials storage etc) from UAV photos. There is no dataset of UAV photos containing such anomalies. In this post I will tell how to training YOLOv5 object detection model from scratch on own dataset.

Task definition

Oil company using drones for monitoring oil production areas. There some anomalies could take place like oil spill, excavation, storage of material,
work of special equipment. This process is done manually which is time consuming. Artificial intelligence can be used here to automatically detect such anomalies from UAV photos. Our task was to develop such a solution for automatic anomalies detection.

Data exploration

We were provided with very small dataset of images 6000x4000 separated into 4 categories presented above. Classes are highly imbalanced: many samples for storage of material (15 samples with high appearance frequency (like 5 places on a single photo) and very few samples for special equipment (6 images and low appearance frequency). There is also a lot of negative samples without any anomalies. Moreover there is no case when different types of anomalies will appear on the same photo.

Training data preparation

As I said we were provided with a small dataset of images 6000x4000: 15 samples for storage of material, 8 for excavation, 15 for oil spill, 6 for special equipment and 121 samples without anomaly. Object detection models need thousands of samples for training to get appropriate results. So we decided to split all images into smaller tiles 600x400 with overlap 25% following sliding window method.

Example of tile cropped from original images

We then perform labeling using makesense.ai. Interface is very simple, no need to register. Simply click Get started, then drag-n-drop files or upload folder with images through file uploader, create a list of labels clicking on text “Your label list is empty” and click Start project. If you want you can use AI to automate labeling.

Once labeling is done just export labels in YOLO format.

Initially we had 27K images, but YOLOv5 model took long to load such a big dataset so we reduced dataset to 10K samples.

We then apply data augmentation using imgaug library to increase the size of dataset and improve detection accuracy. More on importance of data augmentation in object detection is here. YOLOv5 applying some data augmentation techniques out of box (e.g. novel mosaic data augmentation). We here apply rotation, horizontal and vertical flip, shift in both x and y axes. Here is my script for data augmentation. You can also use roboflow service for data augmentation.

Examples

Rotation (left) and shift (right) augmentation examples

Choosing model

Here we have chosen YOLOv5 released recently. YOLO model outperforms another SOTA models in accuracy on COCO dataset is a good trade-off between precision and performance. Here is a good comparison of SOTA models.

Training model

We used this github repo of YOLOv5. Tutorial: https://github.com/ultralytics/yolov5/wiki/Train-Custom-Data

Simply clone repo and install dependencies:

pip install -qr requirements.txt

Copy coco128.yaml file in data folder for custom use

cd data && cp coco128.yaml anomalies.yaml

and edit this file. Set paths to train and vlidation datasets

train: ../data_aug/images/train/  
val: ../data_aug/images/val/

Set number of classes

# number of classes
nc: 4

and specify class names

# class names
names: ['excavation', 'service', 'spill', 'storage']

Then edit number of classes in file models/yolov5s.yaml

nc: 4 # number of classes

And start training

python train.py --img 416 --batch 64 --epochs 20 --data ./data/anomalies.yaml --cfg ./models/yolov5s.yaml --weights ''

Here we use yolov5s model.

yolov5 model will log results of training in folder runs

cd yolov5 && ls -l runs
drwx------ 2 root root 4096 Sep 21 09:52 exp0
drwx------ 2 root root 4096 Sep 21 15:07 exp1
...

metrics will be saved in results.png

Labels distribution in data will be saved in labels.png

Some initial detection results on our dataset for default YOLOv5 training parameters

Performance optimization

Here you can tune batch size, try different input image sizes. Moreover you can tune parameters of ahchors in file yolov5s.yaml.

That’s it. Good luck in object detection using YOLOv5.