Searchplex ID Retrieval terminology registry

Artifact Dataset

Qrels

Canonical ID https://id.searchplex.net/qrels/

A structured set or file of query-document relevance judgments used to evaluate retrieval runs.

Also called

Scope

Qrels is TREC-style terminology for query relevance judgments. A qrels file usually records relevance labels for query-item pairs and is consumed by evaluation tools such as trec_eval. Qrels are not evaluation metrics and are not the same as the full test collection.

Related

Not equivalent to

  • Qrels is not nDCG. Qrels provide labels used to compute metrics such as nDCG; nDCG is the metric, not the judgment set.
  • Qrels is not Relevance. Qrels are recorded labels about relevance; they are not the relevance relationship itself.
  • Qrels is not Relevance Judgment. A qrels file contains many relevance judgments; an individual relevance judgment is one assessment.
  • Qrels is not Test Collection. Qrels are one component of a test collection, alongside resources such as a corpus and topics or queries.

Learn more

ID