Negative Obstacle Detection and Localisation: A Combined Deep Learning and Computer Vision Framework

The burgeoning advancements in robotics and Artificial Intelligence (AI) have propelled a pervasive application in diverse domains, paving the path towards countless different applications ranging from path planning and object detection to autonomous and semi-autonomous navigation. However, for autonomy in navigation to be achievable, robotic platforms are expected to be able to traverse different types of thoroughfares. For this to be achieved, systems should be able to take into account different obstacles that are needed and localised in order for them to be avoided. One of the main platforms being widely and heavily used are assistive technologies such as mobility scooters, segways, and electric-powered wheelchairs (EPWs). Recent advancements in technology have made positive obstacle avoidance possible and very accurate, as there have been many stable and highly reliable systems with the help of different technologies ranging from multimodal sensing techniques to computer vision. However, one of the main challenges which persist is negative road anomalies, upward and downward-inclined paths, and curbs when it comes to assistive technologies.

For these systems to be reliable, novel techniques and approaches are a must due to the fact that assistive technologies are universally considered safety-critical systems, i.e. systems that could directly impact human safety, whereby failure could potentially lead to serious injury or death.

This thesis introduces three different approaches aimed at tackling various problems that render autonomous and semi-autonomous navigation possible. The first problem is Negative Road Anomalies, depressions, or irregular paths and roads caused by wear and tear within the ground surface, including different types of pavement and road imperfections. One of the most universal and fundamental examples of road anomalies is potholes, or road depressions, which form the highest threat to assistive technologies. As a solution to this problem, this thesis proposes a novel object detection and localisation algorithm based on deep learning and machine vision. The second problem addressed within this thesis is upward and downward path inclinations such as ramps and dropped curbs. This thesis proposes a multimodal sensing technique that uses stereo vision and depth vision along with an inertial measurement unit (IMU) in order to detect, localise, and assess inclined planes by processing the point cloud generated as a result of the mentioned sensing techniques. The proposed approach can detect, segment, and localise the inclined plane, whether it is upward-facing or downward-facing. It can also calculate the inclination angle and any ground offset, as well as estimate the width of the inclined plane in order to assess whether it is traversable or not.

As for the third problem addressed within this thesis, it is edge detection, localisation, and avoidance, in order to avoid the risk of falls. This enables assistive technologies to detect the edge of the road whether it is a cliffside, curb, or any physical end of a road or path. This can be achieved by using the same sensing techniques proposed for inclined planes traversal, but with a completely different assessment logic.

The proposed systems were tested in a real-life scenario by mounting them onto a widely and heavily used assistive robotic system from the realm of assistive mobility, an Electric-Powered Wheelchair, or EPW. The three proposed systems were mounted onto an EPW, which was driven in real-life conditions indoors and outdoors as a representative of everyday user contexts as well as in isolated and controlled laboratory settings for in-depth performance assessment of reliability and accuracy. Due to the critical nature of the systems, the ground truth has been measured with the use of indisputable techniques such as a tape measure and a protractor in order to safely assess the system's performance to the millimetre and decimal degree accuracy.