Searchplex ID Retrieval terminology registry

Artifact Software

ir_datasets

Canonical ID https://id.searchplex.net/ir-datasets/

A Python package and catalog that provides a common interface for accessing, documenting, and working with many information-retrieval datasets and benchmarks.

Also called

  • ir-datasets

Scope

ir_datasets helps acquire and iterate over IR dataset components such as documents, queries, and relevance judgments. It is not itself a dataset, benchmark suite, qrels file, or evaluation metric.

Observed terms

  • IR datasets catalog academic
  • IR dataset loader industry

Not equivalent to

  • ir_datasets is not BEIR. ir_datasets provides access to BEIR and many other datasets; BEIR is a benchmark suite.
  • ir_datasets is not Qrels. ir_datasets can expose qrels; qrels are relevance-judgment files, not the software package.
  • ir_datasets is not Test Collection. ir_datasets is software and catalog infrastructure for working with test collections; a test collection is an evaluation resource.

Learn more

ID