Attention is all you need; Attentional Neural Network Models
Łukasz Kaiser - Research Scientist at Google Brain - talks about attentional neural network models and the quick developments that have been made in this recent field. In his talk, he explains how such models "look at the past and generate the next word of the output" and how to train them.
Event on digital.makerfaire.eu
In the last few years Łukasz has worked on machine learning using neural networks as part of the Google Brain team. With his team he has built the TensorFlow machine learning system and he used it for research. He demonstrated that neural networks can learn complex discrete algorithms, such as long multiplication, just from examples. He also worked on natural language processing and constructed state-of-the-art NLP systems for many tasks including translation and summarization.
Before his time at Google he was a tenured researcher in Paris working on logic and automata theory. He used satisfiability solvers and other symbolic methods to build game playing systems and for program synthesis. In his theoretical work he proved theorems on descriptive complexity, game theory and automata that resolved long-standing open problems.