How MLY Identifies SEI, EEI, ELI and Alert Topics in Comments

MLY uses machine learning models to detect and categorize topics within comments, including the SEI, EEI, ELI, and Alert models.

Model Training

The model is trained using comments that have been manually annotated for each topic. These annotations teach the model to recognize linguistic and contextual patterns associated with different topics.

Performance Evaluation

After training, the model’s precision and recall are evaluated using a separate test dataset that was not part of the training phase. This ensures the performance results are unbiased and reflect how well the model generalizes to new data.

Threshold Optimization

For each topic, an optimal probability threshold is determined. This threshold maximizes the model’s ability to correctly identify that topic (balancing precision and recall). Each topic has its own tailored threshold.

Prediction Process

When a client submits a comment, the model predicts the probability that the comment belongs to each topic. These probabilities are compared to the respective optimized thresholds. If the probability for an topic exceeds its threshold, that topic is assigned to the comment.

Example Alert Topic - Toxic Environment