Paper repro: “Self-Normalizing Neural Networks”

Paper repro: “Self-Normalizing Neural Networks”

  • March 13, 2018
Table of Contents

Paper repro: “Self-Normalizing Neural Networks”

SNNs are really cool: their goal is to create neural networks in which, if the input of any layer is normally distributed, the output will automatically also be normally distributed. This is amazing because normalizing the output of layers is known to be a very efficient way to improve the performance of neural networks, but the current ways to do it (eg BatchNorm) basically involve weird hacks, while in SNN the normalization is an intrinsic part of the mathematics of the neural net.

Source: becominghuman.ai

Tags :
Share :
comments powered by Disqus

Related Posts

3 reasons to question the use of emotion-tracking AI in recruiting

3 reasons to question the use of emotion-tracking AI in recruiting

Read More
PyTorch – Internal Architecture Tour

PyTorch – Internal Architecture Tour

This post is a tour around the PyTorch codebase, it is meant to be a guide for the architectural design of PyTorch and its internals. My main goal is to provide something useful for those who are interested in understanding what happens beyond the user-facing API and show something new beyond what was already covered in other tutorials.

Read More
The Building Blocks of Interpretability

The Building Blocks of Interpretability

In 2015, our early attempts to visualize how neural networks understand images led to psychedelic images. Soon after, we open sourced our code as DeepDream and it grew into a small art movement producing all sorts of amazing things. But we also continued the original line of research behind DeepDream, trying to address one of the most exciting questions in Deep Learning: how do neural networks do what they do?

Read More