Forschungskolloquium des IWS: Validating Simulations of User Query Variants

Vortrag, 07. Juni 2022

Der (zu gründende) Forschungsschwerpunkt Knowledge Discovery am Institut für Informationswissenschaft freut sich, das durch die Corona-Pandemie ausgesetzte Forschungskolloquium des IWS wieder anbieten zu können. Es findet als hybride Veranstaltung sowohl vor Ort als auch in Zoom statt.

Auf einen Blick

Timo Breuer: Validating Simulations of User Query Variants



  • 07. Juni 2022
  • 15.15 Uhr bis 17.30 Uhr
  • in meinen Kalender übertragen


Campus Südstadt, Claudiusstr. 1, Raum 158
oder per Zoom: Meeting ID: 83506753468, Kennwort 167256


Forschungskolloquium Informationswissenschaft


Timo Breuer Kontakt


keine Anmeldung notwendig

Weitere Informationen

System-oriented IR evaluations are limited to rather abstract understandings of real user behavior. As a solution, simulating user interactions provides a cost-efficient way to support system-oriented experiments with more realistic directives when no interaction logs are available. While there are several user models for simulated clicks or result list interactions, very few attempts have been made towards query simulations, and it has not been investigated if these can reproduce properties of real queries. In this work, we validate simulated user query variants with the help of TREC test collections in reference to real user queries that were made for the corresponding topics. Besides, we introduce a simple yet effective method that gives better reproductions of real queries than the established methods. Our evaluation framework validates the simulations regarding the retrieval performance, reproducibility of topic score distributions, shared task utility, effort and effect, and query term similarity when compared with real user query variants. While the retrieval effectiveness and statistical properties of the topic score distributions as well as economic aspects are close to that of real queries, it is still challenging to simulate exact term matches and later query reformulations.




Sven Wöhrle