Attention is all you need; Attentional Neural Network Models

Łukasz Kaiser

In the last few years Łukasz has worked on machine learning using neural networks as part of the Google Brain team. With his team he has built the TensorFlow machine learning system and he used it for research. He demonstrated that neural networks can learn complex discrete algorithms, such as long multiplication, just from examples. He also worked on natural language processing and constructed state-of-the-art NLP systems for many tasks including translation and summarization.

Before his time at Google he was a tenured researcher in Paris working on logic and automata theory. He used satisfiability solvers and other symbolic methods to build game playing systems and for program synthesis. In his theoretical work he proved theorems on descriptive complexity, game theory and automata that resolved long-standing open problems.