NLP Systems From Scratch

Individual Natural Language Processing (MSIN0221) UCL MSc Business Analytics

Four working NLP systems built from first principles in a single graded Jupyter notebook, on the Stanford Sentiment Treebank and literary corpora. No frameworks doing the thinking: the point was to show how these methods work underneath the library calls.

systems built from scratch in one pipeline

3 ways

to classify sentiment, compared head-to-head

2 corpora

Stanford Sentiment Treebank + literary texts

100

marks, graded individual technical assignment

Sentiment: Classical vs Neural vs LLM

A logistic-regression classifier comparing hand-engineered features against word embeddings, then a PyTorch MLP with hyperparameter tuning, then zero-shot and few-shot LLM classification over the same data. Three approaches compared head-to-head: where the simple model holds up, where the neural network earns its complexity, and where the LLM wins or fails.

Corpus Analysis and Retrieval

A document-retrieval system over literary corpora: text pre-processing, Named Entity Recognition, TF-IDF vectorisation and cosine-similarity ranking, with dimensionality reduction to visualise how the documents cluster.

Word cloud of the Moby Dick corpus — Corpus exploration before modelling: token frequency structure of Moby Dick after cleaning and stop-word removal.

Top 20 bigrams in the cleaned corpus, the raw material n-gram language models are built on.

PCA projection of Sherlock Holmes stories — 2D projection of TF-IDF document vectors for the Sherlock Holmes stories: the four novels separate cleanly from the short-story cluster.

Language Models

Generative n-gram language models written from scratch: 1-gram, 2-gram and beyond. Building them by hand shows exactly why longer context helps and where sparse counts make the models fall apart. A final open-ended methodological extension completed the graded notebook.