Classify chatbot replies in a way you can reproduce
A tool that automatically sorts chatbot replies into business categories, and does it in a way you can trust and reproduce.
Role
Machine Learning Engineer
Organisation
Japeto
Industry
Conversational AI · Analytics
Focus
ML · Pipeline · Analytics
/ Problem
Chatbot conversations are full of signals about what customers need, but tagging every reply by hand is slow and inconsistent. The task was to classify them automatically, and to make the results transparent enough to actually rely on.
/ Solution
A reproducible pipeline runs as clear stages: validate the data, split, build features, train, and serve. It compares 20 configurations across three algorithms, two feature types, and different label schemes, then shows accuracy, F1, cross-validation, and confusion matrices side by side, so the best model is chosen on evidence rather than one headline number.
The subtle danger here is data leakage: if replies from one conversation land in both the train and test sets, the model looks brilliant then fails in the real world. I made the splits session-aware, fixed the seed, and scored the test set once. The strongest configuration reached ~83% accuracy and 0.81 macro-F1 on honest splits. A score means nothing until you have proven the model was never allowed to cheat.