BoxCars: Improving Fine-Grained Recognition of Vehicles Using 3-D Bounding Boxes in Traffic Surveillance [IEEE T-ITS]

Jakub Sochor, Jakub Špaňhel, Adam Herout
GRAPH@FIT, Brno University of Technology
Corresponding authors: {ispanhel; herout} [at] fit.vutbr.cz

Abstract

In this paper, we focus on fine-grained recognition of vehicles mainly in traffic surveillance applications. We propose an approach that is orthogonal to recent advancements in fine-grained recognition (automatic part discovery and bilinear pooling). In addition, in contrast to other methods focused on fine-grained recognition of vehicles, we do not limit ourselves to a frontal/rear viewpoint, but allow the vehicles to be seen from any viewpoint. Our approach is based on 3-D bounding boxes built around the vehicles. The bounding box can be automatically constructed from traffic surveillance data. For scenarios where it is not possible to use precise construction, we propose a method for an estimation of the 3-D bounding box. The 3-D bounding box is used to normalize the image viewpoint by “unpacking” the image into a plane. We also propose to randomly alter the color of the image and add a rectangle with random noise to a random position in the image during the training of convolutional neural networks (CNNs). We have collected a large fine-grained vehicle data set BoxCars116k, with 116k images of vehicles from various viewpoints taken by numerous surveillance cameras. We performed a number of experiments, which show that our proposed method significantly improves CNN classification accuracy (the accuracy is increased by up to 12% points and the error is reduced by up to 50% compared with CNNs without the proposed modifications). We also show that our method outperforms the state-of-the-art methods for fine-grained recognition.

Downloads

  • Paper
  • BoxCars116k dataset – information about structure can be found in the github repo bellow
    • The dataset is for non-commercial usage. For commercial license, please contact corresponding authors.
  • Supplementary material
  • Github repository with implementation of this method and BoxCars116k dataset information
  • For commercial use please contact us to {ispanhel, herout} [at] fit.vutbr.cz

Datasets License


Except where otherwise noted, this work is licensed under
https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode.
© 2018, SOCHOR, ŠPAŇHEL, HEROUT. Some Rights Reserved.

Citation

@ARTICLE{Sochor2018, 
author={J. Sochor and J. Špaňhel and A. Herout}, 
journal={IEEE Transactions on Intelligent Transportation Systems}, 
title={BoxCars: Improving Fine-Grained Recognition of Vehicles Using 3-D Bounding Boxes in Traffic Surveillance}, 
year={2018}, 
volume={PP}, 
number={99}, 
pages={1-12}, 
doi={10.1109/TITS.2018.2799228}, 
ISSN={1524-9050}
}

Acknowledgment

This work was supported by The Ministry of Education, Youth and Sports of the Czech Republic from the National Programme of Sustainability (NPU II); project IT4Innovations excellence in science — LQ1602.