Advances in Sparse Representation: Efficient modeling and applications

This dissertation investigates the analysis of linear systems of equations, particularly focusing on underdetermined systems where the solution vector is sparse or approximately sparse. With its roots in Compressive Sensing (CS), this dissertation identifies several critical challenges in the literature of Sparse Representation theory and introduces a number of novel methodologies. The methods proposed in this dissertation are applicable to a wide range of application areas.

In this dissertation, a considerable portion of the research focuses on the task of sparse support estimation, exploring methods to estimate the locations of the non-zero elements of the sparse vector without the need to fully recover it. This dissertation demonstrates that learning such a direct mapping from the measurement vector to the location indices of sparse signals, which is also called as support set, is indeed possible via a compact neural network even if only a small or moderate-size training dataset is available. A new classifier is presented as a key application, which integrates dictionary-based and neural network approaches into an efficient hybrid method. The classifier proved its effectiveness in data-scarce scenarios, particularly in COVID-19 detection from chest X-RAY images task, which was developed in the early stage of the pandemic when large training sets were unavailable. Despite such training data scarcity, the CSEN-based classification approach achieved remarkably good performance in COVID-19 detection from X-ray images, achieving over 98% sensitivity and over 95% specificity on the QaTa-Cov19 dataset.

Specifically focused on the challenge of handling large-scale and multi-dimensional signals, the dissertation introduces a novel factorization method, the Generalized Tensorial Sum (GTS-T), to represent the CS matrix with far fewer parameters than the conventional CS system, formulated by matrix-vector multiplication. This method enables the training of neural networks for efficient optimization of the CS matrix, which is especially advantageous for large-scale and multi-dimensional signals. The proposed GTS-T factorization remarkably reduces the complexity of optimizing such large-scale CS matrices by using separable multi-linear learning and showed significant performance enhancement in signal recovery, especially at low measurement rates compared to commonly used factorization approaches such as block-wise learning.

This dissertation also investigates methods that perform signal processing directly on the measurement vector rather than first requiring the recovery of the sparse vector. It first investigates two different methods of direct classification of CS measurements, one of which is for multi-linear compressive learning where CS matrix is learned in the form of GTS-1, and the other is for more vectorized compressive learning where CS system is in conventional matrix-vector multiplication form. Second, by reforming the direct data hiding over CS measurement technology, the dissertation provides both single-level and multi-level CS-based encryption for compressive and secure sensing based monitoring. The proposed methodology is specifically important for privacy-preserving surveillance systems, with its effectiveness in applying increased security to the sensitive part of the signals at a low cost. The experimental validations showed the effectiveness of the proposed approach in robustly hiding sensitive data during joint compression and encryption at a very low cost in the sensory part while maintaining high-quality signal recovery in the receiver. This scheme successfully balanced privacy protection with image fidelity, demonstrating its potential for use in real-world surveillance scenarios with face anonymization on top of compressive encryption.

The dissertation further presents a method for domain transition strategy based on sparse representation and personalized dictionary learning. The method is demonstrated for a specific case study of zero-shot ECG anomaly detection, providing evidence of its effectiveness in efficiently transferring samples representing both normal and abnormal beats from the source to the target directly in the signal domain. The proposed zero-shot classification approach achieved a remarkable accuracy of 98.2% and an F1-Score of 92.8% on the MIT-BIH ECG dataset, significantly outperforming the existing approaches and demonstrating its usefulness for personalized, energy-efficient ECG monitoring.