During the last few years of NLP progress, we have seen the increase in performance proven by transformer models as they grow larger and larger.