An Appearance Based Techinque for Detecting Obstacles

Majority of the upcoming smart cars are employing safety techniques that detect pedestrians, obstacles, vehicles in the path. For example, the car detection algorithm employed by Vicomtech (Video Link) tracks cars in front and calculates collision time. Other than this, an obstacle detection system can be incorporated in smartphones which can be used by the blind for safely navigating an indoor environment.

During my recent internship at Soliton Technologies, Bangalore, known for its work on smart cameras and factory automation using vision systems, I worked on developing a system that can be used for detecting obstacles in a video input from a monocular camera and for segmenting walk-able floor region. The technique may be used for assisting the blind or for moving robots indoors.

We majorly concentrated on using color cues for finding the floor region. Once the floor region is detected, we simply highlight all other regions in an image as obstacles. Our technique was inspired from the research work of Iwan Ulrich and Illah Nourbakhsh, in their paper “Appearance-Based Obstacle Detection with Monocular Color Vision” (Paper Link).

We begin by assuming the bottom most region of an image to contain the floor region as shown. The region inside the red rectangle is taken as reference region that contains the floor.

Figure 1: Marked rectangle (in red) is assumed to contain floor region.

In the next step, we calculate the histogram of this region, and for every pixel in the entire image, we find the probability of that pixel being similar to pixels in the red rectangle region (based on naive bayes theorem). Interested viewers can look into the "backprojection" method implemented in OpenCV. 

Finally, All pixels dissimilar in color to the pixels in the rectangular region are highlighted as obstacles. 

The first approach was based on comparing the RGB values of the pixels. Pixels having high probability of belonging to the histogram of the RGB image inside the rectangular region were considered as non-obstacles.

Figure 2: The pixels which did not match the pixels
inside the rectangular region are highlighted in red

To get a cleaner result, we modify the algorithm to consider larger non-obstacle area for better histogram estimate.

Figure 3: Improved detection results on the same video frame

This technique still suffered from some major drawbacks: the floor regions suffering from specular reflections (due to shining light of the surface) and floor regions containing shadows are still highlighted as obstacles. 

Hence, we investigated the same algorithm, but using the HSV model as it can handle the cases of specular reflection better. The technique is similar to the Mean-Shift algorithm used for tracking moving objects. 

Figure 4: Obstacles highlighted using HSV model

The HSV model handles cases of specular reflection and shadows well, but fails at many locations. Specially regions with darker intensities are not detected as obstacle.

We switched back to the RGB model and find regions of specular reflection (based on intensity and gradient magnitude) and mark them as non-obstacles and regions of shadows are blurred to soften their effects. A technique for storing and retrieving previous histograms was implemented for better performance. The algorithm was implemented on various video in both indoor and outdoor settings. Following were results:

Figure 5 a): Results in outdoor setting

Figure 5 b): Results in Indoor setting

Figure 5 c): Results in Indoor setting

A segmentation based technique was also tried but was rejected owing to the huge computation time. The algorithm is currently being converted into a viable product that can be used for navigation using only a video input and no other sensors. 

My sincere advice for people looking for work in the field of Computer Vision to definitely get in touch with the folks working at Solion Technologies for its awesome start-up like culture and great office environment.