Jakob Uszkoreit, one of the authors of the influential 2017 paper “Attention Is All You Need,” reveals insights about the development of transformer technology in an interview with Ars Technica’s Benj Edwards. The paper, written by eight Google researchers, introduced the transformer architecture that now powers most modern AI systems including ChatGPT and various audio and image synthesis models. Uszkoreit explains that while his team at Google recognized the technology’s potential, they didn’t anticipate its dramatic impact on products like ChatGPT.
The researcher attributes ChatGPT’s delayed emergence partly to Google’s conservative approach to product development at the time. According to Uszkoreit, Google had developed similar language model capabilities around 2017 but chose not to release them publicly. He has since left Google to found Inceptive, a company focusing on applying deep learning to biochemistry for developing programmable medicines through RNA sequences.