The tasting table

Five models, then and now

Every table in the 2021 report, rerun with the same estimators and settings. Blue dots are the report's numbers; tomato dots are the 2026 rebuild on public Food.com data with the report's exact class counts.

Summary

The lineup

Gaussian Naive Bayes
350 Doc2Vec features
2021 report
0.6255 · F1 0.6281Table 4
2026, 5-fold CV
0.5422 · F1 0.5456−8.3 pts against the report
2026 hold-out
0.5357
Logistic Regression
350 Doc2Vec features
2021 report
0.7201implied by Figure 2
2026, 5-fold CV
0.6780 · F1 0.6703−4.2 pts against the report
2026 hold-out
0.6815
SVM (RBF, one-vs-rest)
350 Doc2Vec features
2021 report
0.7302 · F1 0.7175Table 4
2026, 5-fold CV
0.7128 · F1 0.6944−1.7 pts against the report
2026 hold-out
0.7117
Random Forest
Steps bag of words
2021 report
0.7885 · F1 0.7852Table 4
2026, 5-fold CV
0.7661 · F1 0.7573−2.2 pts against the report
2026 hold-out
0.7637
Stacked Naive Bayes
GNB + LR + SVM labels
2021 report
0.7297implied by Figure 5
2026, 5-fold CV
0.7103 · F1 0.7019−1.9 pts against the report
2026 hold-out
0.7094
Stacked NB, out-of-fold
The fix the report proposed, not run in 2021
2026 hold-out
0.7094

Accuracy and weighted F1 for each model in the 2021 report and the 2026 rebuild
Model	Features	2021 report	2026 rebuild, 5-fold CV	2026 hold-out
Gaussian Naive Bayes	350 Doc2Vec features	0.6255 · F1 0.6281Table 4	0.5422 · F1 0.5456−8.3 pts against the report	0.535710,000 unseen recipes
Logistic Regression	350 Doc2Vec features	0.7201implied by Figure 2	0.6780 · F1 0.6703−4.2 pts against the report	0.681510,000 unseen recipes
SVM (RBF, one-vs-rest)	350 Doc2Vec features	0.7302 · F1 0.7175Table 4	0.7128 · F1 0.6944−1.7 pts against the report	0.711710,000 unseen recipes
Random Forest	Steps bag of words	0.7885 · F1 0.7852Table 4	0.7661 · F1 0.7573−2.2 pts against the report	0.763710,000 unseen recipes
Stacked Naive Bayes	GNB + LR + SVM labels	0.7297implied by Figure 5	0.7103 · F1 0.7019−1.9 pts against the report	0.709410,000 unseen recipes
Stacked NB, out-of-fold	The fix the report proposed	not run in 2021	hold-out only	0.7094

The report printed no table entry for Logistic Regression or the stack, so their 2021 accuracy is worked out from the confusion matrices and the class counts.

Scorecard

Which findings survive the rerun

The report's main findings, each checked against the report's own numbers and against the rebuild. 10 of 11 hold in the rebuild.

Reproduced: Random Forest beats the SVM, then the stack, Logistic Regression and Gaussian Naive Bayes
RF 0.7661 > SVM 0.7128 > stack 0.7103 > LR 0.6780 > GNB 0.5422
Reproduced: Five Doc2Vec files (350 features) beat all six (450)
0.6840 against 0.6826
Reproduced: RBF is the best kernel, ahead of polynomial, linear and sigmoid
rbf 0.7145 > poly 0.6936 > linear 0.6840 > sigmoid 0.4714
Reproduced: With C = 1, accuracy rises and then falls with gamma, peaking at 0.01
γ 0.001: 0.6897, γ 0.01: 0.7096, γ 0.1: 0.7056
Not reproduced: With gamma = 0.01, accuracy rises and then falls with C, peaking at C = 1
Not in the rebuild: accuracy keeps rising with C (0.6909, 0.7096, 0.7125 for C = 0.1, 1, 10)
Reproduced: Random Forest accuracy levels off at about 200 trees
0.7661 at 200 trees, +0.05 points at best beyond
Reproduced: The method steps are the most telling text for the forest
steps 0.7691 > name 0.6800 > ingredients 0.6695
The report states this in words, without a table.
Reproduced: Stacking does not beat its best member, the SVM
stack 0.7103 against SVM 0.7128
Reproduced: Slow recipes are the hardest class for every model
slow recall GNB 35%, LR 17%, SVM 0%, RF 22%, stack 16%
Reproduced: Apart from Gaussian Naive Bayes, models mostly call slow recipes medium
slow recipes called medium: LR 55%, SVM 64%, RF 54%, stack 49%
Reproduced: Gaussian Naive Bayes is the weakest model but recalls the most slow recipes
GNB slow recall 35%, next best 22%

Each verdict is computed from the rerun's results when the site is built, and the test suite pins it, so the copy on this site cannot claim more than the data shows.

Step 1 · Doc2Vec files

Fewer features beat more

Staff supplied six Doc2Vec files: 50 and 100 dimensions for the name, the ingredients and the steps. With a linear SVM, the report found five files (350 features) slightly ahead of all six (450).

Linear SVM accuracy by feature set

2021 report
2026 rebuild

Setting2021 → 2026

All six files450 features0.6965 → 0.6826

Drop name 100-d350 featuresn/a → 0.6763

Drop ingredients 100-d350 features0.6989 → 0.6840

Drop steps 100-d350 featuresn/a → 0.6767

The report does not say which five files it kept, so its 350-feature score sits on the set the rebuild found best. Rebuild scores use the tuning split.

The rebuild agrees: dropping one 100-dimension file (drop ingredients 100-d) scores 0.6840 against 0.6826 for all six.

Step 2 · Kernels

RBF wins the kernel bake-off

On the 350 chosen features, the report compared four SVM kernels with default C and gamma.

SVM accuracy by kernel (Table 3.4.1)

2021 report
2026 rebuild

Setting2021 → 2026

Linear0.6989 → 0.6840

Polynomial0.7113 → 0.6936

RBF0.7368 → 0.7145

Sigmoid0.5482 → 0.4714

One-vs-rest SVC, C = 1, gamma = 'scale'. Rebuild scores use the tuning split.

Step 3 · C and gamma

A plus-shaped search

The report varied C with gamma at 0.01, then gamma with C at 1. In the report both curves rise and then fall, and C = 1, gamma = 0.01 came out on top. The rebuild's gamma curve does the same, but its C curve does not: accuracy keeps rising with C (0.6909, 0.7096, 0.7125 for C = 0.1, 1, 10).

RBF SVM accuracy (Table 3.4.2)

2021 report
2026 rebuild

RBF SVM accuracy for each C and gamma pair tried, 2021 report and 2026 rebuild.
C by gamma	γ = 0.001	γ = 0.01	γ = 0.1
C = 0.1	not tried	2021 0.70532026 0.6909	not tried
C = 1	2021 0.70512026 0.6897	2021 0.73562026 0.7096report's pick	2021 0.59742026 0.7056
C = 10	not tried	2021 0.71952026 0.7125rebuild's best	not tried

The report ran C = 1, gamma = 0.01 twice (0.7356 and 0.7362); the cell shows the first run. The solid outline marks the report's pick, the dashed one the rebuild's best. On the rebuilt data C = 10, gamma = 0.01 edges ahead (0.7125 against 0.7096), but the final SVM and the stack keep the report's C = 1, gamma = 0.01, so every result further down reruns the report's actual models.

Step 4 · Random Forest

More trees, until about 200

A Random Forest on the bag of words of the method steps was the report's best model. Accuracy climbs with the number of trees and then levels off, so the report settled on 200.

Accuracy by number of trees (Table 3.5.1)

2021 report
2026 rebuild

Show the numbers

Trees	2021 report	2026 rebuild
1	0.6381	0.6138
10	0.7415	0.7253
50	0.7767	0.7581
100	0.7823	0.7643
200	0.7885	0.7661
300	0.7876	0.7663
400	0.7878	0.7661
500	0.7870	0.7666

Tuning split, 200 trees

Which bag of words?

Random Forest accuracy on each CountVectorizer matrix
Text	Words	Accuracy
name	11,088	0.6800
ingredients	1,862	0.6695
steps	20,846	0.7691

The report found the steps matrix strongest, and so does the rebuild.

Rebuild peak: 500 trees at 76.7%. Mean gap to the report across the eight tree counts: 2.0 points. The report's choice of 200 trees scores 0.7661 in the rebuild.

Error analysis

Where each model goes wrong

True-normalised confusion matrices: each row shows where recipes of that true class ended up. The diagonal is recall.

2021 report

GNB, 2021 report. Rows are the true class, columns the predicted class.
true ↓ / pred →	Quick	Medium	Slow
Quick	0.70	0.26	0.04
Medium	0.35	0.58	0.07
Slow	0.35	0.26	0.39

Gaussian Naive Bayes, read off the report's figure.

Quick recall 70%Medium recall 58%Slow recall 39%

2026 rebuild

GNB, 2026 rebuild. Rows are the true class, columns the predicted class.
true ↓ / pred →	Quick	Medium	Slow
Quick	0.73	0.20	0.08
Medium	0.45	0.40	0.15
Slow	0.48	0.17	0.35

Gaussian Naive Bayes, pooled out-of-fold predictions over five folds.

Quick recall 73%Medium recall 40%Slow recall 35%

The pattern from 2021 holds: slow is the hardest class for every model, and apart from Gaussian Naive Bayes they call most slow recipes medium. Gaussian Naive Bayes is the odd one out. It is the weakest overall but catches the most slow recipes, which is exactly why the report hoped it would add something to the stack. See whether it did.

Honest differences

Why the numbers do not match exactly

Different recipes. The staff sample of Majumder et al.'s crawl is not public, so the rebuild uses a public-domain crawl of the same website, sampled to the report's exact class counts (17,705 / 20,246 / 2,049).
Rebuilt labels. The staff cut-offs were lost. The rebuild uses under 35 minutes, 35 minutes to 4 hours, and over 4 hours, the round numbers closest to the report's class shares.
Regenerated features. Doc2Vec vectors were retrained with gensim, so they are not the staff's vectors.
Tuning protocol. SVM sweeps and the RF matrix comparison use fold 1 of the same split (32,000 train / 8,000 validation). The final models use five-fold cross-validation like the report.
Stacking details. The report does not name its Naive Bayes variant; the rebuild uses categorical Naive Bayes over the three predicted labels.