Core algorithms: regression, trees, k-nearest neighbours and k-means

Screening questions rarely ask you to code an algorithm. They ask which one fits a problem, what its main setting controls, or why it produced a strange result. Supervised algorithms learn from labelled examples; unsupervised ones find structure without labels.

Predicting a number: linear regression

Linear regression fits a straight line (or a flat surface with several inputs) that minimises the squared gap between predicted and actual values. The output is a number. Example 1: predicting delivery time from distance and package weight. Each coefficient says how much the prediction changes per unit of that input, holding the others fixed. It struggles when the relationship curves or when a few extreme values pull the line.

Predicting a class: logistic regression, trees, k-NN

Logistic regression is a classifier despite the name. It produces a probability between 0 and 1, and a threshold (usually 0.5) turns that into a yes or no. Example 2: will this customer churn? A probability of 0.7 becomes "yes". Lowering the threshold catches more churners at the cost of more false alarms, which is why it is tied to precision and recall.

Decision trees split the data with a sequence of questions such as "is age under 30?" until each leaf is mostly one class. They need no scaling, handle mixed data and are easy to explain, but a deep tree memorises the training set. Limiting depth or minimum leaf size controls that. A random forest averages many trees trained on random subsets and is harder to overfit.

k-nearest neighbours stores the training data and classifies a new point by the majority vote of its k closest neighbours. Example 3: with k = 3 and the three nearest points labelled yes, yes, no, the prediction is yes. Small k is noisy; large k smooths over real boundaries. Distances depend on scale, so features must be standardised first.

Finding groups: k-means

k-means is unsupervised. You choose k, it places k centres, assigns each point to the nearest centre, moves each centre to the mean of its points, and repeats until nothing changes. Example 4: segmenting customers by spend and visit frequency into k = 4 groups for a marketing team. There are no correct labels to score against; the elbow method (plotting total within-cluster distance against k) helps choose k.

Common mistakes and a practice habit

• Calling logistic regression a regression algorithm in a "which is a classifier" question.
• Using k-NN or k-means without scaling, so income in thousands dominates age in years.
• Growing a tree with no depth limit and reporting training accuracy.

For each algorithm, write one line: what it outputs, its main setting, and one way it fails. Then take any public dataset and run a baseline and one alternative, comparing on a held-out set, and explain the difference in two sentences.