Projects / Model Evaluation

Sentiment Models Across Amazon Categories

Group of 5 Natural Language Processing (MSIN0221) UCL MSc Business Analytics

Do sentiment models reliably identify negative Amazon reviews across product categories? Aggregate accuracy says yes. Category-level evaluation tells a different story: a model that handles emotional fashion reviews can stumble on technical electronics reviews, and the average hides it.

2 stages
graded proposal (15%), then full execution
2 families
classical (TF-IDF + LR) vs transformer (BERT)
Per-category
performance breakdowns, not aggregate accuracy
18 tasks
planned and owned across a team of 5

Design

Sentiment benchmarks usually report one aggregate number, but review language differs sharply by domain: electronics reviews are technical, fashion reviews are emotional. A model scoring 90% overall can be failing badly inside one category, and a business acting on its outputs would never know. The method: stratify the evaluation by product category, then run topic modelling on the reviews the models got wrong.

Amazon ReviewsAll_Beauty + cross-category sample, cleaned and balanced
Two model familiesTF-IDF + logistic regression vs BERT-family transformers
Stratified evaluationperformance broken down per category, not pooled
Topic modellingon misclassified negatives → category-specific pain points and CX recommendations
Subgroup evaluation design: the same discipline used to check whether a fraud model fails on a specific customer segment.

Execution as a Team of Five

The project ran in two graded stages: a ~3,000-word proposal (research gap, literature from lexicon methods to RoBERTa, methodology), then full execution as the module's final report and recorded presentation. Eighteen tasks were planned, owned and tracked across the team, from dataset ingestion and VADER baselines through BERT fine-tuning to the final write-up.

Project Gantt chart and task ownership table
The delivery plan from the graded proposal: 18 tasks phased across the term, each with named owners, from data ingestion to submission.

My Contribution

Across the two stages I worked on the research-gap framing and methodology, the data preparation pipeline, the comparative model evaluation with category-level breakdowns, the topic-modelling analysis of negative reviews, and co-presented the recorded final presentation.