Model Evaluation Framework

Evaluation Metrics

Primary Metrics

Accuracy:

• Overall correctness across all dialects
• Target: >85% for production
• Formula: (Correct Predictions) / (Total Predictions)

Macro F1-Score:

• Average F1 across all dialect classes
• Treats all dialects equally (good for imbalanced data)
• Target: >0.82