If you encounter this issue, this is because the latest code base is not package to pypi for some reason. So i use a rather hacky way to resolve this. If you're running on your computer Clone https://github.com/huggingface/transformers and locate your installed python package for "transformer". It should be somewhere usr/lib/python/dist-packages. Replace all code in the dist-package transfomer folder with your clone folder. Yes, replace all as there are alot of dependencies. :( Using colab. This is almost the same, clone the repository and start upload to the destination folder. Loads of work and hope you have fun. To be honest, i didn't get it to do those hard core training. It just stall after a few runs.