AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challenge¶

Motivation¶

• Early detection of glaucoma can avoid visual impairment,  which could be facilitated through screening.
• Artificial intelligence (AI) could increase the cost-effectiveness of glaucoma screening, by reducing the need for manual labor.
• AI approaches for glaucoma detection from color fundus photography (CFP) have been proposed and promising at-the-lab performances have been reported. However, large performance drops often occur when AI solutions are applied in real-world settings. Unexpected out-of-distribution data and bad quality images are major causes for this performance drop.

Aim¶

• The development of solutions for glaucoma screening from CFP that are robust to real-world scenarios.

Scope¶

• The participants will be provided with gradable images to develop a model for glaucoma screening. In the test set, there are ungradable images as well, encouraging the participants to develop methods that have robustness mechanisms that can detect out-of-distribution samples. The methods cannot be trained with ungradable images, since this type of data is not present in the train set and the use of external data is prohibited.
• Evaluation is two-fold: glaucoma screening performance and robustness.

Prizes¶

Important dates¶

• 1st of December 2021: Release of training data, Preliminary Test Phase 1 opens including a Docker container example for algorithms supported by grand-challenge.org
• 1st of February 2022: Preliminary Test Phase 2 opens
• 8th of February 2022: Final test phase opens
• 1st 3rd 4th of March 2022: Deadline submission test results (closing of all test phases) and manuscript (see guidelines here). Note: After the winners were announced, all phases did open again for post-challenge submissions and will stay open indefinitely!
• 21st of March 2022: Declaration of the leaderboard
• 28th/29th/30th/31st of March 2022: Presentations at ISBI 2022

Organizers¶

• Coen de Vente (Quantitative Healthcare Analysis (QurAI) Group, Informatics Institute, Universiteit van Amsterdam, Amsterdam, Noord-Holland, Netherlands; Department of Biomedical Engineering and Physics, Amsterdam UMC Locatie AMC, Amsterdam, Noord-Holland, Netherlands; Diagnostic Image Analysis Group (DIAG), Department of Radiology and Nuclear Medicine, Radboudumc, Nijmegen, Gelderland, Netherlands)
• Koenraad A. Vermeer (Rotterdam Ophthalmic Institute, Rotterdam Eye Hospital, Rotterdam, Netherlands)
• Nicolas Jaccard (Project Orbis International Inc., New York, United States)
• Bram van Ginneken (Diagnostic Image Analysis Group (DIAG), Department of Radiology and Nuclear Medicine, Radboudumc, Nijmegen, Gelderland, Netherlands)
• Hans G. Lemij (Rotterdam Ophthalmic Institute, Rotterdam Eye Hospital, Rotterdam, Netherlands)
• Clara I. Sá‎nchez (Quantitative Healthcare Analysis (QurAI) Group, Informatics Institute, Universiteit van Amsterdam, Amsterdam, Noord-Holland, Netherlands; Department of Biomedical Engineering and Physics, Amsterdam UMC Locatie AMC, Amsterdam, Noord-Holland, Netherlands)

Final leaderboard at ISBI 2022¶

*Also published in ISBI Challenge Proceedings: https://ieeexplore.ieee.org/xpl/conhome/9854512/proceeding

AIROGS session at ISBI 2022¶

Challenge paper and how to cite¶

See the challenge summary paper here and how to cite the challenge.

This challenge is sponsored by AWS.