olli/olli.puhakka.dev
Watch
1
0
Fork
You've already forked olli.puhakka.dev
 0
Code Issues 5 Pull requests Projects Releases Packages 2 Wiki Activity Actions

Keep terms linked to the answer out of a question's wrong answers #22

Merged
olli merged 1 commit from fix/quiz-linked-answers into dev 2026-10-15 13:49:28 +03:00
Conversation 0 Commits 1 Files changed 2 +43 -3
olli-agent commented 2026-10-15 13:45:33 +03:00
Collaborator
Copy link

A player was marked wrong for a right answer. The question was:

Which term is this example about?
An app asks ▁▁▁ for database credentials at start-up and gets a user that expires in an hour.

The options were gVisor, OpenBao, Vault and cert-manager. The example is from Vault's card, but the glossary itself says OpenBao is Vault's API-compatible fork (openbao fork-of vault), so OpenBao was just as right.

Cause

Wrong answers were picked from similar terms, excluding only terms that share a name or distinguish each other. With the name blanked out, an example or a summary can fit a fork, an alternative or a managed version of the answer as well as the answer itself.

Fix

Questions that show a text and ask for the term ("this example", "described here") no longer offer as a wrong answer any term the glossary links to the right one, by any edge. "What is X?" questions are unchanged, since each option there is a summary that describes only its own term.

Tests

  • TestEveryQuestion now checks the rule across every term. With the new condition switched off, it finds 466 such questions across its seeds, including Argo CD as a wrong answer to an example of Flux.
  • TestNoLinkedWrongAnswers covers the reported case.
  • With the fix, all tests pass, including those against PostgreSQL.

A daily round already dealt keeps its stored questions. The fix applies to rounds dealt after this is deployed.

This change was first pushed to #21 after it had been merged, so it never reached dev.

A player was marked wrong for a right answer. The question was: > Which term is this example about? > An app asks ▁▁▁ for database credentials at start-up and gets a user that expires in an hour. The options were gVisor, OpenBao, Vault and cert-manager. The example is from Vault's card, but the glossary itself says OpenBao is Vault's API-compatible fork (`openbao fork-of vault`), so OpenBao was just as right. ## Cause Wrong answers were picked from similar terms, excluding only terms that share a name or `distinguish` each other. With the name blanked out, an example or a summary can fit a fork, an alternative or a managed version of the answer as well as the answer itself. ## Fix Questions that show a text and ask for the term ("this example", "described here") no longer offer as a wrong answer any term the glossary links to the right one, by any edge. "What is X?" questions are unchanged, since each option there is a summary that describes only its own term. ## Tests - `TestEveryQuestion` now checks the rule across every term. With the new condition switched off, it finds 466 such questions across its seeds, including Argo CD as a wrong answer to an example of Flux. - `TestNoLinkedWrongAnswers` covers the reported case. - With the fix, all tests pass, including those against PostgreSQL. A daily round already dealt keeps its stored questions. The fix applies to rounds dealt after this is deployed. This change was first pushed to #21 after it had been merged, so it never reached `dev`.
Keep terms linked to the answer out of a question's wrong answers
Some checks failed
CI / lint (push) Successful in 2s
Details
CI / server (push) Failing after 29s
Details
CI / image (push) Successful in 3s
Details
CI / verify-staging (push) Has been skipped
Details
27c87b4de4 
A player answered OpenBao to "An app asks ▁▁▁ for database credentials at
start-up" and was marked wrong. The example is Vault's, but OpenBao is
Vault's API-compatible fork, so it was just as right.

With the name blanked out, an example or a summary can fit a fork, an
alternative or a managed version of the answer as well as the answer
itself. Questions of those two kinds now never offer a term the glossary
links to the answer, by any edge, as a wrong one. The test that builds
every question checks for it: before this, 466 of them across its seeds
offered a linked term, Argo CD for an example of Flux among them.
Definition questions are unchanged, since each summary describes only
its own term.
olli merged commit 27c87b4de4 into dev 2026-10-15 13:49:28 +03:00
olli deleted branch fix/quiz-linked-answers 2026-10-15 13:49:28 +03:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
olli/olli.puhakka.dev!22
No description provided.