Advanced Machine Learning for Sparse Representations in Pattern Recognition Applications

Recent developments in the field of Machine Learning (ML) have accelerated research leveraging sparse representation in pattern recognition. Sparse representation or sparse coding refers to the compact yet sufficient depiction of data where the most descriptive components are preserved. This thesis develops advanced machine learning methods leveraging sparse representations in various pattern recognition applications. First, we focus on the sparse Support Estimation (SE) which aims to estimate non-zero coefficient locations for a sparsely represented signal. Conventional sparse SE methods rely on iterative minimization techniques where the signal is first recovered, and then thresholded to find out the support locations. A learning-based and non-iterative method, Convolutional Support Estimator Network (CSEN) is proposed for performing direct SE without signal recovery. In this way, the performance of sparse SE is greatly improved with enhanced computational complexity over simulated Compressive Sensing (CS) measurements in experimental evaluations.

In representation-based classification tasks, the proposed CSENs have been used in face recognition and early chest X-ray image based Coronavirus disease 2019 (COVID-19) detection applications demonstrating high classification performances with minimum computational complexity. In face recognition, they achieved greater than 90% accuracy; whereas, in COVID-19 detection, they resulted in 97% sensitivity. The latter classification task has been especially challenging considering the limited amount of chest X-ray image training data. To address this problem, the pioneer Early-QaTa-COV19 dataset has been compiled for the purpose of developing an ML-based advance COVID-19 warning system using the proposed CSEN approach. The publicly available dataset includes ground truth labels of early-cases of chest X-ray COVID-19 images by medical doctors. The next contribution of the thesis consists of a novel approach, called, Representation-based Regression (RbR), which uses representative dictionaries in regression task for object distance estimation. RbR-based distance estimation achieved around 4m RMSE using monocular visual data.

Operational Support Estimator Networks (OSENs) consisting of operational layers are the next contribution in the thesis. An operational layer is formed by the "generative" neuron model which approximates non-linear function obtained via Taylor series expansion. Moreover, when the kernel location of the filtering operations is not fixed and it is actively learned during training, the underlying neuron model is called the "super" neuron. The proposed OSENs introduce a hybrid loss topology for the representation-based classification task where the classification and estimated support masks both contribute to the learning process. The proposed support estimator OSEN with enhanced learning capability achieves state-of-the-art performance in different applications, e.g., with over 8% and 15% improvements in F1−Score and accuracy, reaching nearly 45 dB for SE from CS measurements, representation-based classification, and learning-aided signal reconstruction, respectively.

Both CSEN and OSEN require a coarse initial estimation called the proxy computation. Although proxy mapping is computationally efficient, it has negatively affected the learning performance. To this end, Compressive Learning CSEN (CL-CSEN) and Non-linear Compressive Learning OSEN (NCL-OSEN) are proposed to jointly optimize the proxy mapping and support estimator parts. Such joint optimization has boosted the performance levels in the underlying applications. NCL module of NCL-OSEN introduces Self-Organized Generalized Operational Perceptrons (Self-GOPs) which are a generalized version of the Generalized Operational Perceptrons.

The next contribution of the thesis consists of a novel 1-D Sparse Operational Autoencoder (SOA) model for the sparse self-representation learning problem. SOA achieved the best classification results in the band selection problem of Hyperspectral Image (HSI) data with 95% overall classification accuracy.

The final contribution of the thesis includes the design of a novel approach to leverage neural networks in reconstruction tasks using a learning-aided signal reconstruction approach. Using the output probability maps of CSEN and OSEN, which provide prior information for traditional iterative signal reconstruction techniques, both the convergence time and accuracy have been improved in the following three major applications, the recovery of sparse signals, natural images, and CS Magnetic Resonance Imaging (MRI). In the last application, complex and 2-D Self-GOPs are proposed that can learn proxy mapping directly in the complex domain for CS MRI.