TY - GEN
T1 - Variance Preserving Initialization for Training Deep Neuromorphic Photonic Networks with Sinusoidal Activations
AU - Passalis, Nikolaos
AU - Mourgias-Alexandris, George
AU - Tsakyridis, Apostolos
AU - Pleros, Nikos
AU - Tefas, Anastasios
N1 - EXT="Tefas, Anastasios"
PY - 2019/5/1
Y1 - 2019/5/1
N2 - Photonic neuromorphic hardware can provide significant performance benefits for Deep Learning (DL) applications by accelerating and reducing the energy requirements of DL models. However, photonic neuromorphic architectures employ different activation elements than those traditionally used in DL, slowing down the convergence of the training process for such architectures. An initialization scheme that can be used to efficiently train deep photonic networks that employ quadratic sinusoidal activation functions is proposed in this paper. The proposed initialization scheme can overcome these limitations, leading to faster and more stable training of deep photonic neural networks. The ability of the proposed method to improve the convergence of the training process is experimentally demonstrated using two different DL architectures and two datasets.
AB - Photonic neuromorphic hardware can provide significant performance benefits for Deep Learning (DL) applications by accelerating and reducing the energy requirements of DL models. However, photonic neuromorphic architectures employ different activation elements than those traditionally used in DL, slowing down the convergence of the training process for such architectures. An initialization scheme that can be used to efficiently train deep photonic networks that employ quadratic sinusoidal activation functions is proposed in this paper. The proposed initialization scheme can overcome these limitations, leading to faster and more stable training of deep photonic neural networks. The ability of the proposed method to improve the convergence of the training process is experimentally demonstrated using two different DL architectures and two datasets.
KW - Neuromorphic Hardware
KW - Photonic Neural Networks
KW - Sinusoidal Activations
U2 - 10.1109/ICASSP.2019.8682218
DO - 10.1109/ICASSP.2019.8682218
M3 - Conference contribution
AN - SCOPUS:85064389224
SP - 1483
EP - 1487
BT - 2019 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Proceedings
PB - IEEE
T2 - IEEE International Conference on Acoustics, Speech, and Signal Processing
Y2 - 12 May 2019 through 17 May 2019
ER -