The Web4/LS-W4-T5-SM-Emotions model is a specialized, fine-tuned version of the google/flan-t5-small base model. It is an encoder-decoder model, a type of architecture particularly well-suited for text-to-text tasks. This model takes a given text input and, through its internal processing, generates a single word representing the primary emotion expressed in the text. With 77 million parameters, it is a relatively lightweight model, making it efficient for inference.
This model is designed for emotion classification and is intended primarily for research and educational purposes. Its core function is to analyze short texts, such as social media posts, comments, or short sentences, and classify them into one of six predefined emotion categories:
joy
sadness
anger
love
fear
surprise
The model outputs one of these six words as its prediction. This capability can be useful for applications like sentiment analysis, emotion-aware chatbots, or large-scale analysis of emotional trends in text data.
The model was fine-tuned on the dair-ai/emotion dataset, a collection of 20,000 English social media messages. The dataset was split into:
Training set: 16,000 examples
Validation set: 2,000 examples
Test set: 2,000 examples
A critical aspect of the training data is its imbalance. The "joy" and "anger" categories are significantly more frequent than the other four. This imbalance may introduce a bias in the model, causing it to over-predict these two classes and potentially underperform on the less frequent emotions.
The fine-tuning process utilized the following key hyperparameters:
learning_rate: 5×10−5
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: AdamW (torch fused)
lr_scheduler_type: linear
num_epochs: 3
The training was conducted using the PyTorch and Hugging Face Transformers frameworks on an NVIDIA T4 GPU.
This model can be easily integrated into a Python environment using the Hugging Face transformers library. Below is a code example demonstrating how to use the pipeline for quick inference.
Huggingface: https://huggingface.co/Web4/LS-W4-T5-SM-Emotions