recall vs percision

Precision 

Precision measure how good the model is at getting things right. For example, if a spam filter has a precision of 0.9, then it means that 90% of the emails that it predicts as spam are actually spam. 

 Precision = TP / (TP + FP) 

In Precision, we want straight up, how many did the model get right, how many did the model get wrong - that will give us how precise it is.  

Spam filtering: A spam filter with high precision will have few false positives, which means that it will not send many ham emails to the spam folder. So spam filter with high precision may also have some false negatives, which means that some spam emails will not be caught.

Recall Recall measures how good the model is at avoiding false negatives. For example, if a spam filter has a recall of 0.8, then it means that 80% of the spam emails are actually predicted as spam. 

 Recall = TP / (TP + FN) 

In Recall, we want to know this model predicted correctly and what is predicted badly when the result is actually true - that needed a review / revisit / recall. 

We don't want recall for medical system

Medical diagnosis: A medical diagnosis system with high recall will catch most of the sick patients, but it may also incorrectly classify some healthy patients as sick. A medical diagnosis system with high precision will correctly classify most of the healthy patients, but it may also miss some sick patients.

Kappa and ROC

Kappa and ROC curve are both metrics used to evaluate the performance of a binary classifier. However, they measure different things and are used in different situations.

Kappa measures the agreement between two raters or two classification methods, taking into account the chance agreement. It is a scalar metric, meaning that it is a single number that summarizes the performance of the classifier.

ROC curve (Receiver Operating Characteristic curve) plots the true positive rate (TPR) against the false positive rate (FPR) at different thresholds. It is a curve that shows the tradeoff between TPR and FPR as the threshold is varied.