Ph.D. in Applied Mathematics

(Artificial Intelligence)

Version Française
English Version

Learning Representations using Neural Networks and Optimal Transport

September 2016 - October 2020

Warith HARCHAOUI

Ph.D. Defense (in French) - 8^th October 2020, 2 PM at the MAP5 lab in Paris
Webcasting and realization: Jean Defontaine - Pierre Chosson // O·H·N·K
 

Chapters in the video:

• Introduction
• Clustering with Wasserstein criteria
• Prediction with Uncertainty
• Unsupervised Features Importance

Discussion with members of the jury:

• Introduction with Pr. Julie Delon, chairwoman
• Pr. Nicolas Courty, Referee
• Pr. Erwann Le Pennec, Referee
• Dr. Laure Soulier
• Dr. Pierre-Alexandre Mattei
• Pr. Andrés Almansa
• Pr. Julie Delon, chairwoman
• Pr. Charles Bouveyron, Ph.D. supervisor
• Pierre-Noël Luiggi, founder of Oscaro.com that funded this research
• Reading of the jury's report

I have spent four wonderful years diverting neural networks (with so-called deep learning techniques) from their normal use:

• Clustering with generative and discriminative and Wasserstein distances criteria through Generative Adversarial Networks (GANs)
• Unsupervised feature importance with infinitesimal Wasserstein maximal distortion
• Prediction with uncertainty

In layman's terms, this Ph.D. work was respectively about:

• how to make a few groups among a dataset that are both very numerous and big. For example, we explore the feasibility of grouping hundreds of thousands (or more) photographs each made of millions pixels a.k.a color dots
• how to show the distinctive attributes of a dataset. Indeed, we investigate the opportunity of interpreting raw data attributes (or coordinates) without annotations nor labels
• how to estimate the confidence of an automatic decision. In real-world applications (e.g. industrial constraints, health, security, even justice), a stastical approach to confidence in decision making is crucial for giving back the actual responsibility to human beings for difficult cases typically.

These problems share a common scientific questioning: how should we represent data? For that purpose, we revisit mathematical concept called Optimal Transport with a widely known algorithmical tool called Neural Networks (nicknamed “Deep Learning” since 2010 approximately).

Great people like Pr. Charles Bouveyron (my academic thesis supervisor), Dr. Stéphane Raux (my corporate thesis supervisor), Dr. Pierre-Alexandre Mattei, Pr. Andrés Almansa, Thi Thanh Yen Nguyen, Pr. Olivier Bouaziz and Pr. Antoine Chambaz did me an honour by helping me accomplish this work in the warmth of the MAP5 lab and with the pugnacity of the Oscaro company.

Ph.D. elements:

• Full Manuscript
   
  
• Code Repository
   
  
• Abstract (in English and French)
   
  
• Ph.D. Defense Slides
   
  
• Members of the Jury:
  Pr. Julie Delon, Chairwoman
  Pr. Erwann Le Pennec, Referee
  Pr. Nicolas Courty, Referee
  Dr. Laure Soulier
  Dr. Pierre-Alexandre Mattei
  Pr. Andrés Almansa, Guest
  Pr. Charles Bouveyron, Supervisor
  Dr. Stéphane Raux, Co-supervisor
   
  
• Ph.D. reports from the referees and jury:
  by Pr. Nicolas Courty
  by Pr. Erwann Le Pennec
  by the jury chaired by Pr. Julie Delon
  

For ease of reading, the chapters are separated here:

• State of the Art
   
  
• Wasserstein Clustering
  joint work with Pr. Charles Bouveyron, Dr. Pierre-Alexandre Mattei and Pr. Andrés Almansa
   
  
• Unsupervised Feature Importance
   
  
• Prediction with Uncertainty
   
  
• Appendix (Auto-encoders for GANs initialization)
   
  

Unpublished works have begun during this thesis without appearing in the manuscript (and are currently in progress):

• Grammar and Deep Learning, project in Natural Language Processing towards the reconciliation between grammar from linguistics and neural networks via auto-encoders and modern embedding techniques, as an extension of a research internship that I designed and supervised for Maxime Haddouche.
   
  
• GaDeMI: Gaussian and Decorrelated Means Independence, project for advanced Data Analysis with Dr. Joan Alexis Glaunès regarding a new kind of auto-encoders built in successive layers to extract a representation whose coordinates are quasi-gaussian and decorrelated (and therefore approximately independent)
  Extension of the Auto-encoder-based GAN Initialization appendix
   
  
• StaReLefOU: State Representation Learning from Observation Uncertainty, project in Robotics with Astrid Merckling towards the construction of a state representation built without reward while still being useful for any unknown task in the same environment thanks to exploration
  Extension of the Prediction with Uncertainty chapter
   
  
• Wasserstein Co-Clustering, project in Computatinal Biology for studying the Huttington disease using co-clustering for pairing RNA molecules of different kinds with Optimal Transport with Thi Thanh Yen Nguyen, Pr. Olivier Bouaziz and Pr. Antoine Chambaz
  Extension of the Wasserstein Clustering chapter
   
  
• Cerbero: Early Real-Time Credit Card Fraud Detection at Oscaro.com, project of applied AI with a pragmatic numerical optimization that is cost-oriented that had some press coverage with colleagues Roland Thiollière, Romain Nio, Nils Grunwald, Jérémie Thomas, Julien Gaunon, and Dr. Stéphane Raux (random order)
  Extension of the Prediction with Uncertainty chapter
   
  
• i2nn: An Invitation to Neural Networks, talk given several times to convince people working both in Statistics and Programming to use Deep Learning
  Extension of the State of the Art chapter
   
  

Warith Harchaoui