Mike Erlihson

Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere

Data can be represented as a vector in a continuous space, but often it is concentrated in a narrow area. Can it be improved?

DeepNightLearners

Improving Self-supervised Learning with Automated Unsupervised Outlier Arbitration

Augmenting images training data to add variability doesn't always end well since the augmentation changes the semantic content of the image. Here's how to solve it.

Computer Vision

Perceiver: General Perception with Iterative Attention

To overcome Transformers' squared complexity (w.r.t input length), the Perceiver article here offers a novel method to learn the QKV matrices. Check it out!

DeepNightLearners

Discriminator Rejection Sampling

A summary of a method to improve GAN generated images by exploiting accumulated "information" from the Discriminator during the GAN training process.

DeepNightLearners

Sharpness-Aware Minimization for Efficiently Improving Generalization

This very relevant article suggests an unusual method to improve neural network regularization abilities. I believe that this method has a great potential to enter the standard neural network training toolkit. I was also impressed by the detailed comparison to other methods.

DeepNightLearners

Representation learning via invariant causal mechanisms

This article suggests a method to construct augmentation resilient and styling invariant image representation in lower dimension (embeddings).

DeepNightLearners

Rethinking Attention with Performers

The article suggests a method to lower the Transformer's complexity to a linear order and proves all the arguments also in a rigorous form. The article is not easy to read, but luckily, to understand the main idea, the first 5-6 pages are more than enough.

DeepNightLearners

A causal view of compositional zero-shot recognition

One of the main challenges in zero-shot learning is enabling compositional generalization to the mode. This review is part of a series of reviews in Machine & Deep Learning that are originally published in Hebrew, aiming to make it accessible in a plain language under the name #DeepNightLearners.

DeepNightLearners

Supermasks in Superposition

The article suggests a brilliant training method, called SupSup, of a large neural network, which allows it to perform several different tasks.

Subscribe to Keep Current