Beyond Fine-Tuning: Evaluating Prompt Engineering and Prompt Tuning for Roman Urdu Hate Speech Detection
| Source: arXiv AI
Tags: Roman Urdu, Hate Speech Detection, Prompt Tuning, Mistral-7B, PURUT, Low-Resource Languages
A study comparing prompt engineering and prompt tuning for Roman Urdu hate speech detection finds a mixed prompt configuration reaches up to 0.87 F1, well above zero-shot baselines.
Details
Roman Urdu is a low-resource language with non-standard spelling and frequent code-mixing, which makes hate speech detection hard. The authors compare zero-shot and few-shot prompting, and manual versus mixed prompt-based adaptation, on the PURUT dataset of about 72,771 social media comments using GPT-3.5, LLaMA-3, Mistral-7B and multilingual BERT. Few-shot prompting consistently beat zero-shot, with Mistral-7B at 0.67 F1 versus 0.56 for zero-shot models. A mixed prompt configuration reached up to 0.87 F1. For moderation teams working in low-resource languages, prompt tuning looks like a cheaper alternative to full fine-tuning.