Memory Requirement Reduction of Deep Neural Networks for Field Programmable Gate Arrays Using Low-Bit Quantization of Parameters

Niccoló Nicodemo; Gaurav Naithani; Konstantinos Drossos; Tuomas Virtanen; Roberto Saletti

doi:10.23919/Eusipco47968.2020.9287739

Memory Requirement Reduction of Deep Neural Networks for Field Programmable Gate Arrays Using Low-Bit Quantization of Parameters

Niccoló Nicodemo, Gaurav Naithani, Konstantinos Drossos, Tuomas Virtanen, Roberto Saletti

Computing Sciences

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

1 Citation (Scopus)

Abstract

Effective employment of deep neural networks (DNNs) in mobile devices and embedded systems, like field programmable gate arrays, is hampered by requirements for memory and computational power. In this paper we propose a method that employs a non-uniform fixed-point quantization and a virtual bit shift (VBS) to improve the accuracy of the quantization of the DNN weights. We evaluate our method in a speech enhancement application, where a fully connected DNN is used to predict the clean speech spectrum from the input noisy speech spectrum. A DNN is optimized, its memory requirement is calculated, and its performance is evaluated using the short-time objective intelligibility (STOI) metric. The application of the low-bit quantization leads to a 50% reduction of the DNN memory requirement while the STOI performance drops only by 2.7%.

Original language	English
Title of host publication	2020 28th European Signal Processing Conference (EUSIPCO)
Publisher	IEEE
Pages	466-470
Number of pages	5
ISBN (Electronic)	978-9-0827-9705-3
DOIs	https://doi.org/10.23919/Eusipco47968.2020.9287739
Publication status	Published - 2021
Publication type	A4 Article in conference proceedings
Event	European Signal Processing Conference - Beurs van Berlage, Amsterdam, Netherlands Duration: 18 Jan 2021 → 22 Jan 2021 Conference number: 28 https://eusipco2020.org

Publication series

Name	European Signal Processing Conference
ISSN (Electronic)	2076-1465

Conference

Conference	European Signal Processing Conference
Abbreviated title	EUSIPCO2020
Country/Territory	Netherlands
City	Amsterdam
Period	18/01/21 → 22/01/21
Internet address	https://eusipco2020.org

Keywords

Quantization (signal)
Neural networks
Memory management
Speech enhancement
Logic gates
Table lookup
Field programmable gate arrays
neural network quantization
memory footprint reduction
FPGA
hardware accelerators

Publication forum classification

Publication forum level 1

Access to Document

10.23919/Eusipco47968.2020.9287739

Cite this

Nicodemo, N., Naithani, G., Drossos, K., Virtanen, T., & Saletti, R. (2021). Memory Requirement Reduction of Deep Neural Networks for Field Programmable Gate Arrays Using Low-Bit Quantization of Parameters. In 2020 28th European Signal Processing Conference (EUSIPCO) (pp. 466-470). (European Signal Processing Conference). IEEE. https://doi.org/10.23919/Eusipco47968.2020.9287739

@inproceedings{5c7fa0a7e30349339abf6086b431f88a,

title = "Memory Requirement Reduction of Deep Neural Networks for Field Programmable Gate Arrays Using Low-Bit Quantization of Parameters",

abstract = "Effective employment of deep neural networks (DNNs) in mobile devices and embedded systems, like field programmable gate arrays, is hampered by requirements for memory and computational power. In this paper we propose a method that employs a non-uniform fixed-point quantization and a virtual bit shift (VBS) to improve the accuracy of the quantization of the DNN weights. We evaluate our method in a speech enhancement application, where a fully connected DNN is used to predict the clean speech spectrum from the input noisy speech spectrum. A DNN is optimized, its memory requirement is calculated, and its performance is evaluated using the short-time objective intelligibility (STOI) metric. The application of the low-bit quantization leads to a 50% reduction of the DNN memory requirement while the STOI performance drops only by 2.7%.",

keywords = "Quantization (signal), Neural networks, Memory management, Speech enhancement, Logic gates, Table lookup, Field programmable gate arrays, neural network quantization, memory footprint reduction, FPGA, hardware accelerators",

author = "Niccol{\'o} Nicodemo and Gaurav Naithani and Konstantinos Drossos and Tuomas Virtanen and Roberto Saletti",

note = "jufoid=55867; European Signal Processing Conference, EUSIPCO2020 ; Conference date: 18-01-2021 Through 22-01-2021",

year = "2021",

doi = "10.23919/Eusipco47968.2020.9287739",

language = "English",

series = "European Signal Processing Conference",

publisher = "IEEE",

pages = "466--470",

booktitle = "2020 28th European Signal Processing Conference (EUSIPCO)",

url = "https://eusipco2020.org",

}

Nicodemo, N, Naithani, G, Drossos, K, Virtanen, T & Saletti, R 2021, Memory Requirement Reduction of Deep Neural Networks for Field Programmable Gate Arrays Using Low-Bit Quantization of Parameters. in 2020 28th European Signal Processing Conference (EUSIPCO). European Signal Processing Conference, IEEE, pp. 466-470, European Signal Processing Conference, Amsterdam, Netherlands, 18/01/21. https://doi.org/10.23919/Eusipco47968.2020.9287739

Memory Requirement Reduction of Deep Neural Networks for Field Programmable Gate Arrays Using Low-Bit Quantization of Parameters. / Nicodemo, Niccoló; Naithani, Gaurav; Drossos, Konstantinos et al.
2020 28th European Signal Processing Conference (EUSIPCO). IEEE, 2021. p. 466-470 (European Signal Processing Conference).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Memory Requirement Reduction of Deep Neural Networks for Field Programmable Gate Arrays Using Low-Bit Quantization of Parameters

AU - Nicodemo, Niccoló

AU - Naithani, Gaurav

AU - Drossos, Konstantinos

AU - Virtanen, Tuomas

AU - Saletti, Roberto

N1 - Conference code: 28

PY - 2021

Y1 - 2021

N2 - Effective employment of deep neural networks (DNNs) in mobile devices and embedded systems, like field programmable gate arrays, is hampered by requirements for memory and computational power. In this paper we propose a method that employs a non-uniform fixed-point quantization and a virtual bit shift (VBS) to improve the accuracy of the quantization of the DNN weights. We evaluate our method in a speech enhancement application, where a fully connected DNN is used to predict the clean speech spectrum from the input noisy speech spectrum. A DNN is optimized, its memory requirement is calculated, and its performance is evaluated using the short-time objective intelligibility (STOI) metric. The application of the low-bit quantization leads to a 50% reduction of the DNN memory requirement while the STOI performance drops only by 2.7%.

AB - Effective employment of deep neural networks (DNNs) in mobile devices and embedded systems, like field programmable gate arrays, is hampered by requirements for memory and computational power. In this paper we propose a method that employs a non-uniform fixed-point quantization and a virtual bit shift (VBS) to improve the accuracy of the quantization of the DNN weights. We evaluate our method in a speech enhancement application, where a fully connected DNN is used to predict the clean speech spectrum from the input noisy speech spectrum. A DNN is optimized, its memory requirement is calculated, and its performance is evaluated using the short-time objective intelligibility (STOI) metric. The application of the low-bit quantization leads to a 50% reduction of the DNN memory requirement while the STOI performance drops only by 2.7%.

KW - Quantization (signal)

KW - Neural networks

KW - Memory management

KW - Speech enhancement

KW - Logic gates

KW - Table lookup

KW - Field programmable gate arrays

KW - neural network quantization

KW - memory footprint reduction

KW - FPGA

KW - hardware accelerators

U2 - 10.23919/Eusipco47968.2020.9287739

DO - 10.23919/Eusipco47968.2020.9287739

M3 - Conference contribution

T3 - European Signal Processing Conference

SP - 466

EP - 470

BT - 2020 28th European Signal Processing Conference (EUSIPCO)

PB - IEEE

T2 - European Signal Processing Conference

Y2 - 18 January 2021 through 22 January 2021

ER -

Memory Requirement Reduction of Deep Neural Networks for Field Programmable Gate Arrays Using Low-Bit Quantization of Parameters

Abstract

Publication series

Conference

Keywords

Publication forum classification

Access to Document

Fingerprint

Cite this