Soutenance de thèse Séverine Cloix

Mme Séverine Cloix soutiendra, en vue de l'obtention du grade de docteur ès sciences, mention informatique, sa thèse intitulée:

Sparse Multi-View 3D Computer Vision - Application to Embedded Assistive Technologies



External members:

  • Dr. Christian Perwass, Clutec, Germany;
  • Dr. Loïc Baboulaz, ArtMyn, Switzerland;
  • Prof. Andres Perez-Uribe, HEIG-VD, Switzerland.

Computer Science department representatives:

  • Prof. Sviatoslav Voloshynovskiy;
  • Dr. Guido Bologna.

Thesis directors:

  • Dr. David Hasler, Bobst Graphics;
  • Prof. Thierry Pun.

This PhD was supported by the Hasler Foundation, CSEM Neuchâtel, and the Computer Science department.


The advances in computing power and the affordability of today's imaging sensors gave rise to the development of complex and highly efficient algorithms. The domain of 3D computer vision has benefited from this growth for a broad range of applications, including assistive technologies. The latter field is of high concern, especially for the elderly. At the same time, the United Nations yearly reports the trends in population aging. It reveals the necessity to develop solutions that compensate impairment due to old age including the one related to mobility. Today's assistive devices, however, are bulky and require large batteries to run for just a couple of hours. This is how the EyeWalker project came out along with the research studies reported in this thesis, whose common objective is to design new computer-vision-based approaches dedicated to embedded and real-time applications with limited resources.

The EyeWalker project aims at developing a low-cost, ultra-light computer-vision-based device for rollator users with mobility problems. Thereby this thesis proposes novel strategies for rapid object detection and recognition under practical constraints. These limitations are for example the number of sensors and their resolution, algorithm complexity and mobile battery-life. We defined use cases based on data collected from occupational therapists. We could therefore narrow the research scope to specific objects and obstacles detection from off-the-shelf stereo and plenoptic cameras. The research work thus investigates two areas of computer vision from multi-view imaging, namely exploiting (i) sparse 3D keypoint clouds from stereo vision and (ii) light field imaging for low-complexity and efficient algorithms. 

We address the problems of detecting obstacles and specific objects for aids to navigation. We propose two approaches using sparse 3D object cues from stereo vision in a boosting classification framework. The first estimates poses of a 3D object to reduce the ambiguity of its appearance for its detection in a 2D image. The second allows specific 2D and 3D feature extraction for obstacle detection. We also present a method to detect generic descending stairs. We studied the influence of two parameters on the algorithm performance: the reduction of the image resolution and the type of imaging systems, i.e. passive and active stereo cameras in various illumination conditions. We analysed the power consumption versus the resolution with regard to considerations on hardware and embedded real-time programming. It assesses the robustness and the low computation time of our algorithm. 

Regarding (ii), the recent advances in optics and light field vision have given rise to off-the-shelf plenoptic cameras that embed the equivalent of about 7900 viewpoints. We present a state-of-the-art scale-invariant object recognition method using a Raytrix® camera evaluated on a novel and versatile light field dataset. A final study presents a method to characterize 2D points using the surrounding rays captured by a light field. Applied to a single pixel, it independently gives an accurate depth estimate and predicts if it is a keypoint lying on a region of depth discontinuities. The low complexity of the algorithm makes it an ideal candidate for real-time applications intended for embedded platforms. 

In conclusion this thesis is in the opposite direction of the recent trend for big data. It highlights the capabilities of exploiting sparse cues from multiple views to efficiently perform computer vision tasks. While we faced the limits of sparse 3D point clouds from low-resolution sensors to estimate poses of planar objects in 3D space, we have demonstrated the practical performance of such sets of points to detect descending stairs in real-time on a low-power embedded vision system. By drastically increasing the number of views with an off-the-shelf plenoptic camera, our approaches to object recognition and 3D keypoint detection position our research among the state-of-the-art work on light field for computer vision. In addition to the continuous improvement in mobile computing power, the low-complexity of all the proposed methods comply with restrictive technical requirements to design not only affordable and useful assistive devices for the elderly but also navigation and surveillance systems, and retrofitting in existing manufacturing lines for quality inspection to name a few.

Date: Lundi 19 juin 2017 à 13h30

Lieu: Battelle bâtiment A - Auditoire rez-de-chaussée

30 mai 2017

À la Une