AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challenge

Motivation

  • Early detection of glaucoma can avoid visual impairment,  which could be facilitated through screening.
  • Artificial intelligence (AI) could increase the cost-effectiveness of glaucoma screening, by reducing the need for manual labor.
  • AI approaches for glaucoma detection from color fundus photography (CFP) have been proposed and promising at-the-lab performances have been reported. However, large performance drops often occur when AI solutions are applied in real-world settings. Unexpected out-of-distribution data and bad quality images are major causes for this performance drop.

Aim

  • The development of solutions for glaucoma screening from CFP that are robust to real-world scenarios.

Scope

  • The participants will be provided with gradable images to develop a model for glaucoma screening. In the test set, there are ungradable images as well, encouraging the participants to develop methods that have robustness mechanisms that can detect out-of-distribution samples. The methods cannot be trained with ungradable images, since this type of data is not present in the train set and the use of external data is prohibited.
  • Evaluation is two-fold: glaucoma screening performance and robustness.

Prizes

๐Ÿฅ‡   1st place  3000 EUR in cash  AWS credits to the value of 5000 USD.
๐Ÿฅˆ 2nd place : 2000 EUR in cash + AWS credits to the value of 3000 USD.
๐Ÿฅ‰ 3rd place : 1000 EUR in cash + AWS credits to the value of 2000 USD.

Important dates

  • 1st of December 2021: Release of training data, Preliminary Test Phase 1 opens including a Docker container example for algorithms supported by grand-challenge.org
  • 1st of February 2022: Preliminary Test Phase 2 opens
  • 8th of February 2022: Final test phase opens
  • 1st 3rd 4th of March 2022: Deadline submission test results (closing of all test phases) and manuscript (see guidelines here). Note: After the winners were announced, all phases did open again for post-challenge submissions and will stay open indefinitely!
  • 21st of March 2022: Declaration of the leaderboard
  • 28th/29th/30th/31st of March 2022: Presentations at ISBI 2022

Organizers

  • Coen de Vente (Quantitative Healthcare Analysis (QurAI) Group, Informatics Institute, Universiteit van Amsterdam, Amsterdam, Noord-Holland, Netherlands; Department of Biomedical Engineering and Physics, Amsterdam UMC Locatie AMC, Amsterdam, Noord-Holland, Netherlands; Diagnostic Image Analysis Group (DIAG), Department of Radiology and Nuclear Medicine, Radboudumc, Nijmegen, Gelderland, Netherlands)
  • Koenraad A. Vermeer (Rotterdam Ophthalmic Institute, Rotterdam Eye Hospital, Rotterdam, Netherlands)
  • Nicolas Jaccard (Project Orbis International Inc., New York, United States)
  • Bram van Ginneken (Diagnostic Image Analysis Group (DIAG), Department of Radiology and Nuclear Medicine, Radboudumc, Nijmegen, Gelderland, Netherlands)
  • Hans G. Lemij (Rotterdam Ophthalmic Institute, Rotterdam Eye Hospital, Rotterdam, Netherlands)
  • Clara I. Sรกโ€Žnchez (Quantitative Healthcare Analysis (QurAI) Group, Informatics Institute, Universiteit van Amsterdam, Amsterdam, Noord-Holland, Netherlands; Department of Biomedical Engineering and Physics, Amsterdam UMC Locatie AMC, Amsterdam, Noord-Holland, Netherlands)

Final leaderboard at ISBI 2022

#

Authors

First author affilation

Final score

PDF

Algorithm

Code

1st

He Wang et al.

Peking Union Medical College Hospital

๐Ÿ‡จ๐Ÿ‡ณ

1.8

๐Ÿ“„*

๐Ÿ’พ

 

2nd

Firas Khader et al.

RWTH Aachen University Hospital

๐Ÿ‡ฉ๐Ÿ‡ช๐Ÿ‡จ๐Ÿ‡ญ

3

๐Ÿ“„*

๐Ÿ’พ

 

2nd

Temirgali et al.

CMC Technologies

๐Ÿ‡ฐ๐Ÿ‡ฟ

3

๐Ÿ“„

๐Ÿ’พ

 

4th

Tien-Dung Le

KBC

๐Ÿ‡ง๐Ÿ‡ช

6.5

๐Ÿ“„

๐Ÿ’พ 

 

5th

Adrian Galdran et al.

Universitat Pompeu Fabra

๐Ÿ‡ช๐Ÿ‡ธ๐Ÿ‡ฆ๐Ÿ‡บ

7

๐Ÿ“„

๐Ÿ’พ

 

6th

Densen Puthussery et al.

Founding Minds Software

๐Ÿ‡ฎ๐Ÿ‡ณ

7.3

๐Ÿ“„

๐Ÿ’พ

 

7th

Zekang Yang et al.

Institute of Computing Technology

๐Ÿ‡จ๐Ÿ‡ณ

7.8

๐Ÿ“„

๐Ÿ’พ 

 

8th

Satoshi Kondo et al.

Muroran Institute of Technology

๐Ÿ‡ฏ๐Ÿ‡ต

8

๐Ÿ“„

๐Ÿ’พ 

 

9th

Edward Wang et al.

University of Western Ontario

๐Ÿ‡จ๐Ÿ‡ฆ

8.8

๐Ÿ“„

๐Ÿ’พ 

๐Ÿง‘โ€๐Ÿ’ป

10th

Jรณnathan Heras et al.

University of La Rioja

๐Ÿ‡ช๐Ÿ‡ธ

9

๐Ÿ“„

๐Ÿ’พ

๐Ÿง‘โ€๐Ÿ’ป

11th

Teresa Araรบjo et al.

Medical University of Vienna

๐Ÿ‡ฆ๐Ÿ‡น

10.5

๐Ÿ“„*

๐Ÿ’พ 

 

11th

Mustafa Arikan

University College London, London

๐Ÿ‡ฌ๐Ÿ‡ง

10.5

๐Ÿ“„

๐Ÿ’พ 

 

13th

Yeong Chan Lee et al.

Samsung Medical Center

๐Ÿ‡ฐ๐Ÿ‡ท

11.3

๐Ÿ“„

๐Ÿ’พ

 

13th

Abdul Qayyum et al.

Universitรฉ de Bourgogne

๐Ÿ‡ซ๐Ÿ‡ท๐Ÿ‡ช๐Ÿ‡ธ๐Ÿ‡ฆ๐Ÿ‡บ

11.3

๐Ÿ“„

๐Ÿ’พ

 

15th

Wei Tang et al.

University of Groningen

๐Ÿ‡ณ๐Ÿ‡ฑ๐Ÿ‡จ๐Ÿ‡ณ

14.5

๐Ÿ“„

๐Ÿ’พ

 

*Also published in ISBI Challenge Proceedings: https://ieeexplore.ieee.org/xpl/conhome/9854512/proceeding

AIROGS session at ISBI 2022


Challenge paper and how to cite

See the challenge summary paper here and how to cite the challenge.

This challenge is sponsored by AWS.