Submission of Inference Containers to the Open Development Phase 👩‍💻👨‍💻

Grand Challenge Algorithm Template

Algorithm submissions to the Open Development Phase of the PI-CAI challenge, are to be made in the form of inference containers. Inference containers are equivalent to Grand Challenge (GC) algorithms. They are Docker containers that encapsulate trained AI algorithms (architecture + model weights) and all of the components needed to load a given case (e.g. prostate bpMRI + clinical information), generate predictions (e.g. csPCa detection map + patient-level likelihood score), and store the corresponding output file(s) for evaluation.
The following code repositories contain such example submissions for the Open Development Phase. Each repo includes an ensemble of baseline AI models that was trained using fully supervised learning with 1295 human expert-annotated cases, or using semi-supervised learning with 1500 human expert or AI-annotated cases (Bosma et al., 2022). All training cases belong to the Public Training and Development Dataset, and all models were trained via 5-fold cross-validation using the exact same splits. You can use any of these repos as a template for your submission to the Open Development Phase:
Before implementing your own algorithm using any of these templates, we recommend that you first upload a GC algorithm based on an unaltered template. In the following sections, we will provide a walkthrough of how to do this.

Installing Docker on Local System 🐳

You must have Docker installed and running on your system for the following steps to work. If you are using Windows, we recommend installing Windows Subsystem for Linux (WSL). For more details, please watch the official tutorial by Microsoft for installing WSL 2 with GPU support.

Building, Testing and Exporting Algorithm Container on Local System

If you're using Linux/macOS, please use the shell scripts (.sh files) stated in all following commands. If you're using Windows, please use the batch scripts (.bat files) stated in all following commands.
Clone one of the template repos (see above) to your local system, with Git LFS initialized (e.g. this can be done directly using GitHub Desktop). Build and test your inference Docker container by running build.bat/, followed by test.bat/ Please note, that testing also runs a build, so running the build command separately is not necessary (if you are certain that everything is set up correctly). Testing will build the container and run it on images provided in the ./test/ folder. It will then check the output files (csPCa detection map + patient-level likelihood score) produced by your algorithm, against those pre-computed and present in ./test/. If all tests have completed successfully, then you're good to go. Export your container using export.bat/ Once complete, your container image should be ready as a single .tar.gz file.
For more details, we highly recommend completing this general tutorial on creating GC algorithms:

Uploading Algorithm Container to Grand Challenge

Navigate to Submit > Open Development Phase - Validation and Tuning and complete the form "on this page":

Once complete, click the "Save" button. At this point, your GC algorithm has been created and you are on its homepage. Now, click the "Containers" tab on the left panel, and then the "Upload a Container" button. Select your container image (.tar.gz file) for upload. After the upload is complete, click "Save". It typically takes 20-60 minutes till your container has been activated (depending on the size of your container). After its status is "Active", test out your container with a sample, unprocessed training case from the Public Training and Development Dataset. You can also use the sample case provided in the 'test' directory of the baseline algorithm template (see above).
To learn more about GC input/output interfaces, visit:

Create Containers for Your Own AI Algorithm

Following these same steps, you can easily encapsulate your own AI algorithm in an inference container by altering one of our provided baseline algorithm templates. You can implement your own solution by editing the functions in ./ Any additional imported packages should be added to ./requirements.txt, and any additional files and folders should be explicitly copied through commands in ./Dockerfile. To update your algorithm, you can simply test and export your new Docker container, after which you can update the container image for the GC algorithm that you created on with the new one. Please note, that your container will not have access to the internet when they are executed on, so all necessary model weights and resources must be encapsulated in your container image apriori. You can test whether this is true locally using the --network=none option of docker run.
If something doesn't work for you, please feel free to add a post regarding the same in the forum.

Submission to Open Development Phase - Validation and Tuning Leaderboard

Once you have your trained AI model uploaded as a fully-functional GC algorithm, you're now ready to make submissions to the Open Development Phase of the challenge! Navigate to the "Open Development Phase - Tuning Submission" page, select your algorithm, and click "Save" to submit. If there are no errors and evaluation has completed successfully, your score will be up on the leaderboard (typically in less than 24 hours).
⚠️ Please double-check all rules to make sure that your submission is compliant. Invalid submissions will be removed and teams repeatedly violating any/multiple rules will be disqualified. Also, please try-out your uploaded GC algorithm with a sample case before making a full submission to the leaderboard.

Submission to Open Development Phase - Testing Leaderboard

Once you have gone through your whole AI model development + tuning cycle, and are now certain that you have your final algorithm ready and trained for independent testing, please complete the following steps:
  • Make one final submission to the Open Development Phase - Tuning Leaderboard following all the steps stated above (only if you haven't already). This is to ensure that your final model is bug-free and performs just as expected.
  • Optionally, you can prepare a document (in PDF format; minimum 1 page; no restrictions on template or maximum length), briefly covering the model design and training strategy of your AI algorithm. Please include the names, designations and contact details of your team members in this document. You can upload this document using the "Supplementary File" field in the submission form, or using the "Publication" field to share a URL to the same. If you're submitting an AI algorithm that has already been presented in one of your team's prior publications, then you can directly share a URL to that publication. Please note, this step is mandatory if you want to qualify as one of the top 5 teams of the challenge and move on to the Closed Testing Phase. If you have been selected as one of the top 5 algorithms and have not provided this document in your submission, we will contact you for the same later down the line.
  • Please send us an e-mail at and, with your team name, list of team members, and a URL to your submitted algorithm (here is an example). We will use this e-mail thread to share updates regarding your submission and address issues (if any). Additionally, if you make it to the top 5, the designated team will receive the prizes listed here.
Results of all submissions (that are error-free and compliant with all participation rules) will be pre-computed in a hidden setting, and presented during our oral presentation at RSNA 2022. After this presentation, the results will also be made publicly available on the leaderboard.

We look forward to your submissions to the PI-CAI challenge. All the best! 👍