Machine learning approaches practice questions.

A city transport authority wants a model that predicts how many minutes a bus will be delayed, using weather, time of day and route length. Which machine learning approach fits this task?

• k-means clustering of the recorded routes
• Linear regression on the recorded delays
• Logistic regression on the recorded delays
• A decision tree that outputs 'late' or 'on time'

Answer: Linear regression on the recorded delays

The target is a continuous quantity measured in minutes, so a regression model is needed. Logistic regression and the tree both return a class, and k-means has no target variable at all.

During one iteration of the k-means algorithm, what happens once every data point has been assigned to a cluster?

• Each centroid moves to the mean position of the points assigned to it
• Each point is given the label of its nearest labelled neighbour
• The strength of every connection is adjusted using the output error
• The data set is split on the attribute that gives the highest information gain

Answer: Each centroid moves to the mean position of the points assigned to it

k-means alternates two steps: assign each point to the nearest centroid, then recompute each centroid as the mean of its members, repeating until the assignments stop changing. The other options describe k-NN, a neural network and a decision tree.

A k-nearest neighbours classifier uses k = 1, so a single mislabelled training point decides the prediction near it. What is the most likely effect of raising k to 15?

• Training takes much longer because the model must be refitted
• The boundary smooths out and single odd points matter less
• The classifier now returns a continuous value instead of a class
• The stored training examples are no longer needed at prediction time

Answer: The boundary smooths out and single odd points matter less

With k = 1 one neighbour decides; with k = 15 the majority of a larger neighbourhood decides, which lowers the variance of the prediction. k-NN stores all examples and fits nothing, so its training cost does not change.

A decision tree is grown until every leaf holds examples of one class only. It then scores 100 % on the training set and 61 % on the test set. Which action is most likely to improve its accuracy on new, unseen data?

• Grow the tree deeper so that more attributes are tested
• Add the test examples to the training set and grow again
• Prune branches that rest on very few examples
• Report the training score instead, as it is the higher figure

Answer: Prune branches that rest on very few examples

A perfectly pure tree has memorised the training set, and the gap between the two scores is overfitting. Pruning removes the low-support branches that encode noise; growing deeper makes the gap worse.

What distinguishes a random forest from a single decision tree?

• Many trees are built on random samples and their votes combined
• One tree is built, but its branches are visited in random order
• The tree is rebuilt from the start whenever a new example arrives
• The attributes are ranked at random instead of by information gain

Answer: Many trees are built on random samples and their votes combined

Each tree in a forest sees a different random sample of examples and attributes, so their individual errors are largely independent and the majority vote cancels much of the variance. A single tree keeps all of its own variance.

A logistic regression model decides whether an email is spam. What does it produce for a new email before any threshold is applied?

• A cluster number between 1 and k
• A value between 0 and 1, read as a probability
• The distance to the nearest training example
• An unbounded numeric value measured in the same units as the input

Answer: A value between 0 and 1, read as a probability

Logistic regression squeezes its weighted sum into the range 0 to 1, and that value is then compared with a threshold such as 0.5 to give a class. Linear regression is the model whose output is unbounded.