BERT for dummies



I will keep this short. BERT is a pre-trained model (trained on wikipedia) that means we skip all those time consuming stuff of tokenizing and encoding those models. Everything here is a ripped off without modification from Google Colab example (please refer to link below for all the codes)

This example is a classification problem and it will classify if a sentence is grammatically correct or not.

Sample used :- COLA - a dataset for benchmarking grammar.

Summary of tasks before getting lost in the crazy codes maze.


- Load all the necessary library - which includes hugging face's BERT and pytorch

- Load this COLA dataset and transform it into a correct format

- Load BERT model

- Predict and Evaluate

The results is true or false.


Done :)


Reference link

https://colab.research.google.com/drive/1ywsvwO6thOVOrfagjjfuxEf6xVRxbUNO


Comments

Post a Comment

Popular posts from this blog

The specified initialization vector (IV) does not match the block size for this algorithm

Image
If you're getting  exception when trying to assigned key to a symmetric key and getting error message below :- The specified initialization vector (IV) does not match the block size for this algorithm And your code probably look like something below :- Then you need to go into debug mode and start looking into the supported size for IV as shown in diagram below :- As you can see, IV is 16 bytes, so you need to provide 16 bytes. The same goes for Key field too, if you're adding anything to it. As long as you provide a valid key size, then we're good and you're code will be ready.
Read more