Variational Neural Networks implementation in Pytorch and JAX

Illia Oleksiienko; Dat Thanh Tran; Alexandros Iosifidis

doi:10.1016/j.simpa.2022.100431

Variational Neural Networks implementation in Pytorch and JAX

Illia Oleksiienko^*
, Dat Thanh Tran
, Alexandros Iosifidis

^*Corresponding author for this work

Computing Sciences

Research output: Contribution to journal › Article › Scientific › peer-review

3 Citations (Scopus)

18 Downloads (Pure)

Abstract

Bayesian Neural Networks consider a distribution over the network's weights, which provides a tool to estimate the uncertainty of a neural network by sampling different models for each input. Variational Neural Networks (VNNs) consider a probability distribution over each layer's outputs and generate parameters for it with the corresponding sub-layers. We provide two Python implementations of VNNs with PyTorch and JAX machine learning libraries that ensure reproducibility of the experimental results and allow implementing uncertainty estimation methods easily in other projects.

Original language	English
Article number	100431
Journal	Software Impacts
Volume	14
DOIs	https://doi.org/10.1016/j.simpa.2022.100431
Publication status	Published - Nov 2022
Publication type	A1 Journal article-refereed

Funding

This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 871449 (OpenDR). This publication reflects the authors’ views only. The European Commission is not responsible for any use that may be made of the information it contains. This work has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 871449 (OpenDR). This publication reflects the authors’ views only. The European Commission is not responsible for any use that may be made of the information it contains.

Keywords

Bayesian deep learning
Bayesian Neural Networks
JAX
PyTorch
Uncertainty estimation

Publication forum classification

Publication forum level 0

ASJC Scopus subject areas

Software

Access to Document

10.1016/j.simpa.2022.100431Licence: CC BY

1-s2.0-S2665963822001154-mainFinal published version, 386 KBLicence: CC BY

https://urn.fi/URN:NBN:fi:tuni-202212129074Licence: CC BY

Cite this

@article{c8ee14865f3c48549fc57cb9841a223e,

title = "Variational Neural Networks implementation in Pytorch and JAX",

abstract = "Bayesian Neural Networks consider a distribution over the network's weights, which provides a tool to estimate the uncertainty of a neural network by sampling different models for each input. Variational Neural Networks (VNNs) consider a probability distribution over each layer's outputs and generate parameters for it with the corresponding sub-layers. We provide two Python implementations of VNNs with PyTorch and JAX machine learning libraries that ensure reproducibility of the experimental results and allow implementing uncertainty estimation methods easily in other projects.",

keywords = "Bayesian deep learning, Bayesian Neural Networks, JAX, PyTorch, Uncertainty estimation",

author = "Illia Oleksiienko and Tran, \{Dat Thanh\} and Alexandros Iosifidis",

note = "Funding Information: This work has received funding from the European Union{\textquoteright}s Horizon 2020 research and innovation programme under grant agreement No 871449 (OpenDR). This publication reflects the authors{\textquoteright} views only. The European Commission is not responsible for any use that may be made of the information it contains. Funding Information: This work has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 871449 (OpenDR). This publication reflects the authors{\textquoteright} views only. The European Commission is not responsible for any use that may be made of the information it contains. Publisher Copyright: {\textcopyright} 2022 The Author(s)",

year = "2022",

month = nov,

doi = "10.1016/j.simpa.2022.100431",

language = "English",

volume = "14",

journal = "Software Impacts",

issn = "2665-9638",

publisher = "Elsevier",

}

TY - JOUR

T1 - Variational Neural Networks implementation in Pytorch and JAX

AU - Oleksiienko, Illia

AU - Tran, Dat Thanh

AU - Iosifidis, Alexandros

N1 - Funding Information: This work has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 871449 (OpenDR). This publication reflects the authors’ views only. The European Commission is not responsible for any use that may be made of the information it contains. Funding Information: This work has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 871449 (OpenDR). This publication reflects the authors’ views only. The European Commission is not responsible for any use that may be made of the information it contains. Publisher Copyright: © 2022 The Author(s)

PY - 2022/11

Y1 - 2022/11

N2 - Bayesian Neural Networks consider a distribution over the network's weights, which provides a tool to estimate the uncertainty of a neural network by sampling different models for each input. Variational Neural Networks (VNNs) consider a probability distribution over each layer's outputs and generate parameters for it with the corresponding sub-layers. We provide two Python implementations of VNNs with PyTorch and JAX machine learning libraries that ensure reproducibility of the experimental results and allow implementing uncertainty estimation methods easily in other projects.

AB - Bayesian Neural Networks consider a distribution over the network's weights, which provides a tool to estimate the uncertainty of a neural network by sampling different models for each input. Variational Neural Networks (VNNs) consider a probability distribution over each layer's outputs and generate parameters for it with the corresponding sub-layers. We provide two Python implementations of VNNs with PyTorch and JAX machine learning libraries that ensure reproducibility of the experimental results and allow implementing uncertainty estimation methods easily in other projects.

KW - Bayesian deep learning

KW - Bayesian Neural Networks

KW - JAX

KW - PyTorch

KW - Uncertainty estimation

U2 - 10.1016/j.simpa.2022.100431

DO - 10.1016/j.simpa.2022.100431

M3 - Article

AN - SCOPUS:85140775113

SN - 2665-9638

VL - 14

JO - Software Impacts

JF - Software Impacts

M1 - 100431

ER -

Variational Neural Networks implementation in Pytorch and JAX

Abstract

Funding

Keywords

Publication forum classification

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this