Review Analyzer

Three ways to learn a line

Every review becomes a vector of 13,000 zeros and ones — one per word in the training vocabulary. A linear classifier scores it as θ·x + θ₀ and predicts the sign. The three algorithms differ only in how they move θ:

• Perceptron — on a mistake, θ ← θ + y·x. Simple, converges if the data is separable, and it is: 99 % training accuracy but only ~80 % on new reviews.
• Average perceptron — same updates, but the answer is the average of every θ seen during training. Cheap insurance against the last few updates overfitting.
• Pegasos — stochastic sub-gradient descent on the SVM objective: shrink θ by (1 − ηλ) every step and add η·y·x when the margin is below 1, with η = 1/√t. The regulariser λ trades training accuracy for margin.

Because the model is linear, it is completely explainable: the Analyze tab colours every word by its weight, and the Words tab lists the biggest ones.

What the course found

The MITx project tunes T and λ on the validation set and reports test accuracy around 80–82 % for all three methods; stopword removal and count features don't help much, bigrams a little. Press Sweep T and λ on the Train tab to reproduce that table in your browser.

Implementation notes

• docs/js/sentiment.js ports project1.py one-to-one (same tokeniser, same update rules, same 1e-4 / 1e-5 numerical guards, same fixed training order from 4000.txt).
• Features are sparse maps, Pegasos applies its shrink lazily through a global scale, and the average perceptron uses the lazy-averaging trick — so a full 25-epoch run of all three takes about a second.

Data: 5,000 Amazon fine-food reviews from the MITx 6.86x project (4,000 train / 500 validation / 500 test).