The PI-CAI challenge has officially concluded! So what now? ¶
Can I Submit my AI Algorithms for Tuning/Benchmarking? 👩💻👨💻¶
While the Open Development Phase – Tuning Leaderboard will remain live for continuous submissions (click here for instructions on this), interested teams can only make submissions to the Open Development Phase – Testing Leaderboard upon request to the organizers with a brief supporting document (including contact details and the scope of their project). This step is necessary to preserve the integrity of the hidden testing cohort (by preventing repeated submissions and overfitting). It also ensures traceability and verification of all post-challenge solutions that claim to match/outperform prior submissions on the testing dataset. Meanwhile, the Closed Testing Phase – Final Ranking Leaderboard will be closed, as it only represents the ranking of AI algorithms which were also trained on the private training datasets. ¶
PI-CAI will not be a one-time event. Future iterations may explore the effects of additional information (e.g. using full prostate mpMRI, instead of bpMRI sequences only; or using longitudinal data for active surveillance) and more rigorous testing (e.g. larger number of external testing data centers) on AI performance and generalization. ¶
