Algorithms practice questions.

A news publisher gives its headline-testing system a single target: the proportion of readers who click. Within months headlines are more emotive and articles are more often abandoned after a few seconds. The clearest reading of this outcome is that

• the measured target is only a proxy for the goal it replaced
• the system processed personal data without a lawful basis under the GDPR
• the training data contained too few headlines for the system to learn from
• the newsroom's journalists had been replaced entirely by a generative model

Answer: the measured target is only a proxy for the goal it replaced

Clicks stand in for reader value, so optimising the proxy can degrade the thing it was supposed to represent. The decision that matters is the choice of objective, which is made by people rather than by the algorithm.

A bank reports that its fraud model is 99.5 per cent accurate. Fraud occurs in 0.4 per cent of transactions, so a system that approved everything without looking would be 99.6 per cent accurate. What does the comparison show?

• Accuracy is uninformative where one outcome is rare, so the two kinds of error have to be counted separately
• The model must have been trained on a dataset in which fraud was far more common than it is in the bank's real traffic
• The model is performing worse than chance on the transactions it is asked to judge
• Accuracy cannot be calculated at all for a system that takes decisions about money

Answer: Accuracy is uninformative where one outcome is rare, so the two kinds of error have to be counted separately

With a rare outcome, almost every case is negative, so a headline accuracy figure is dominated by the easy majority and hides both the frauds missed and the customers wrongly blocked. Asking which errors were counted is the first question to put to any published performance claim.

A benefits office states that no decision is taken by automated means alone, because a caseworker confirms each of the 600 recommendations that reach the screen during a shift. The caseworker sees no reasons, holds no authority to depart from the recommendation and has no time to open the file. Why does the safeguard fail?

• Because the number of recommendations handled in one shift exceeds the maximum that data protection law permits a single caseworker to review
• Because the recommendations are produced by a learned model rather than by a published rule
• Because the office has never told applicants that a model is involved in the process
• Because a review without reasons, time or authority to differ is a signature rather than a decision

Answer: Because a review without reasons, time or authority to differ is a signature rather than a decision

A human in the loop is a safeguard only where the human can actually reach a different conclusion, which requires reasons, time and standing to overrule. Rubber-stamping converts a legal requirement into a formality while leaving the outcome fully automated.

In August 2020, England's exam regulator Ofqual replaced cancelled A-level exams with grades from a standardisation model that adjusted teachers' predictions using each school's results from previous years. Able students at schools with weak past results were downgraded. This outcome followed mainly because

• teachers at those schools had predicted lower grades than elsewhere
• each grade was tied to the past results of the student's school
• the model had been trained on too small a sample of schools
• a software fault assigned grades to the wrong candidates

Answer: each grade was tied to the past results of the student's school

The model treated a school's history as a constraint on this year's cohort, so an individual's grade partly encoded where they studied — a design decision, not a bug or a small sample. The public backlash led Ofqual to withdraw the model within days and use teacher grades instead.

A university team reports that its student dropout-prediction model 'is 97 per cent accurate'. The figure was obtained by running the finished model over the same 8 000 student records that were used to fit it. What is the specific fault in this evidence?

• Eight thousand records is too small a sample for an accuracy figure to mean anything.
• It was scored on the records it was fitted to, not on unseen ones.
• Accuracy should be reported with a margin of error rather than as a single number.
• The model was fitted to records about students rather than about the courses themselves.

Answer: It was scored on the records it was fitted to, not on unseen ones.

A model can score almost perfectly on the data it was fitted to simply by memorising them; only records the model has never seen say anything about future students. The distinction between fitting and generalising is invisible in the headline number, which is why the number has to be read together with the method.

A recruitment model is retrained with the applicant's sex removed from its inputs. Which of the remaining fields is most likely to act as a proxy for the removed variable?

• The number of years that have passed since the applicant's first qualification.
• The number of employees at the organisation where the applicant last worked.
• The clubs and societies the applicant lists on their CV.
• The month in which the applicant says they could start the job.

Answer: The clubs and societies the applicant lists on their CV.

Deleting a variable does not delete the information if another field still carries it, and the names of clubs and societies are strongly sex-linked. Reuters reported in 2018 that Amazon had scrapped an experimental recruiting tool that downgraded CVs containing the word 'women's'. Time since qualifying is a proxy for age, not for sex.