zlacker

Also in that article he says

"In fact, I’d go as far as to say that

    The concept of attention is the most interesting recent architectural innovation in neural networks."

when the initial attention paper was less than a year old, and two years before the transformer paper.