Activity
Ratings Progression
Challenge Categories
Challenges Entered
Behavioral Representation Learning from Animal Poses.
Latest submissions
See Allgraded | 179081 | ||
graded | 173187 | ||
graded | 173085 |
Latest submissions
See Allgraded | 180080 | ||
graded | 180079 |
A benchmark for image-based food recognition
Latest submissions
See Allfailed | 177330 | ||
graded | 177329 | ||
graded | 177328 |
What data should you label to get the most value for your money?
Latest submissions
See Allgraded | 177046 | ||
failed | 176542 | ||
failed | 176404 |
Improving the HTR output of Greek papyri and Byzantine manuscripts
Latest submissions
See Allgraded | 188253 | ||
graded | 188252 | ||
graded | 188221 |
ASCII-rendered single-player dungeon crawl game
Latest submissions
See Allsubmitted | 158196 | ||
failed | 158192 | ||
failed | 158191 |
Training sample-efficient agents in Minecraft
Latest submissions
See Allgraded | 158774 | ||
graded | 147099 | ||
graded | 141760 |
Airborne Object Tracking Challenge
Latest submissions
See Allgraded | 150618 | ||
graded | 148372 | ||
graded | 145797 |
5 Puzzles 21 Days. Can you solve it all?
Latest submissions
Machine Learning for detection of early onset of Alzheimers
Latest submissions
Sample Efficient Reinforcement Learning in Minecraft
Latest submissions
See Allgraded | 150880 | ||
failed | 149894 | ||
failed | 149002 |
Measure sample efficiency and generalization in reinforcement learning using procedurally generated environments
Latest submissions
5 Puzzles 21 Days. Can you solve it all?
Latest submissions
Self-driving RL on DeepRacer cars - From simulation to real world
Latest submissions
Robustness and teamwork in a massively multiagent environment
Latest submissions
3D Seismic Image Interpretation by Machine Learning
Latest submissions
See Allgraded | 82746 | ||
graded | 82738 | ||
graded | 82737 |
5 Puzzles 21 Days. Can you solve it all?
Latest submissions
Latest submissions
See Allgraded | 149881 | ||
submitted | 149879 | ||
failed | 144887 |
Play in a realistic insurance market, compete for profit!
Latest submissions
See Allgraded | 113613 | ||
graded | 110174 | ||
graded | 110172 |
5 Puzzles 21 Days. Can you solve it all?
Latest submissions
5 Puzzles 21 Days. Can you solve it all?
Latest submissions
Multi-Agent Reinforcement Learning on Trains
Latest submissions
See Allfailed | 84106 | ||
failed | 84105 | ||
failed | 84104 |
A dataset and open-ended challenge for music recommendation research
Latest submissions
A benchmark for image-based food recognition
Latest submissions
See Allgraded | 59791 | ||
failed | 59371 | ||
failed | 31084 |
Latest submissions
See Allgraded | 139559 | ||
graded | 139558 | ||
graded | 139557 |
Sample-efficient reinforcement learning in Minecraft
Latest submissions
See Allgraded | 120070 | ||
graded | 117123 | ||
graded | 113473 |
Latest submissions
See Allgraded | 125603 | ||
graded | 125602 | ||
graded | 125601 |
5 Puzzles, 3 Weeks. Can you solve them all? π
Latest submissions
Multi-agent RL in game environment. Train your Derklings, creatures with a neural network brain, to fight for you!
Latest submissions
Predicting smell of molecular compounds
Latest submissions
See Allfailed | 114224 | ||
failed | 114223 | ||
failed | 114222 |
Classify images of snake species from around the world
Latest submissions
See Allgraded | 60335 | ||
graded | 60334 |
5 Puzzles 21 Days. Can you solve it all?
Latest submissions
Latest submissions
Robots that learn to interact with the environment autonomously
Latest submissions
See Allgraded | 99115 | ||
graded | 99114 | ||
failed | 99109 |
5 Puzzles, 3 Weeks | Can you solve them all?
Latest submissions
See Allgraded | 115551 | ||
graded | 115537 | ||
graded | 115536 |
5 PROBLEMS 3 WEEKS. CAN YOU SOLVE THEM ALL?
Latest submissions
Latest submissions
Grouping/Sorting players into their respective teams
Latest submissions
5 Problems 15 Days. Can you solve it all?
Latest submissions
Disentanglement: from simulation to real-world
Latest submissions
Sample-efficient reinforcement learning in Minecraft
Latest submissions
Multi Agent Reinforcement Learning on Trains.
Latest submissions
Latest submissions
See Allgraded | 191700 | ||
graded | 191699 | ||
graded | 191698 |
Latest submissions
Visual SLAM in challenging environments
Latest submissions
Latest submissions
Latest submissions
Project 2: Road extraction from satellite images
Latest submissions
Project 2: build our own recommender system, and test its performance
Latest submissions
Project 2: build our own text classifier system, and test its performance.
Latest submissions
Help improve humanitarian crisis response through better NLP modeling
Latest submissions
See Allfailed | 32245 |
Reincarnation of personal data entities in unstructured data sets
Latest submissions
Robots that learn to interact with the environment autonomously
Latest submissions
Immitation Learning for Autonomous Driving
Latest submissions
A new benchmark for Artificial Intelligence (AI) research in Reinforcement Learning
Latest submissions
Latest submissions
Predict if users will skip or listen to the music they're streamed
Latest submissions
Spot the Boson
Latest submissions
PlantVillage is built on the premise that all knowledge that helps people grow food should be openly accessible to anyone on the planet.
Latest submissions
Latest submissions
Latest submissions
Latest submissions
Latest submissions
Latest submissions
Predict if users will skip or listen to the music they're streamed
Latest submissions
Recognizing bird sounds in monophone soundscapes
Latest submissions
Latest submissions
Latest submissions
Visual SLAM in challenging environments
Latest submissions
Latest submissions
Latest submissions
DA Final Project challenges for Monsoon 2020
Latest submissions
ACL-BioNLP Shared Task
Latest submissions
Latest submissions
Latest submissions
Latest submissions
Latest submissions
Latest submissions
Identify which side has least players
Latest submissions
Predict viewer reactions from a large-scale video dataset!
Latest submissions
Latest submissions
Detect Multi-Animal Behaviors from a large, hand-labeled dataset.
Latest submissions
See Allfailed | 125036 |
Reinforcement Learning, IIT-M, assignment 1
Latest submissions
Latest submissions
Latest submissions
Can you dock a spacecraft to ISS ?
Latest submissions
Latest submissions
Latest submissions
Latest submissions
Image-based plant identification at global scale
Latest submissions
Latest submissions
Localization, SLAM, Place Recognition, Visual Navigation, Loop Closure Detection
Latest submissions
Localization, SLAM, Place Recognition
Latest submissions
Round 2 - Active | Claim AWS Credits by beating the baseline
Latest submissions
See Allgraded | 179081 |
A Challenge on Continual Learning using Real-World Imagery
Latest submissions
Latest submissions
Learning From Human-Feedback
Latest submissions
Participant | Rating |
---|---|
![]() |
0 |
![]() |
0 |
![]() |
0 |
![]() |
0 |
![]() |
0 |
![]() |
|
|
|
![]() |
0 |
![]() |
0 |
|
|
![]() |
0 |
|
221 |
![]() |
265 |
![]() |
221 |
![]() |
0 |
Participant | Rating |
---|---|
|
|
![]() |
0 |
![]() |
0 |
![]() |
0 |
![]() |
0 |
|
|
![]() |
|
![]() |
0 |
![]() |
0 |
![]() |
0 |
![]() |
265 |
![]() |
0 |
|
221 |
![]() |
0 |
![]() |
272 |
![]() |
103 |
![]() |
|
![]() |
0 |
![]() |
0 |
![]() |
-39 |
![]() |
0 |
![]() |
0 |
![]() |
190 |
![]() |
0 |
![]() |
|
![]() |
|
![]() |
229 |
![]() |
221 |
![]() |
-
AIcrowd Novartis DSAI ChallengeView
-
AIcrowd AIcrowd Blitz - May 2020View
-
food-shivam Food Recognition ChallengeView
-
AIcrowd FlatlandView
-
shivam AI for Good - AI Blitz #3View
-
AIcrowdHQ Insurance pricing gameView
-
shivam Seismic Facies Identification ChallengeView
-
AIcrowd NeurIPS 2021: MineRL Diamond CompetitionView
ESCI Challenge for Improving Product Search

Predicition submission
About 21 hours agoHi @wxl121228, yes the prediction submissions are closed right now in favour of code submissions round.
More info about the code submission round:

π Code Submission Round Launched π
About 21 hours agoHi @amiruddin_nagri, the default leaderboard is βCode Submissionsβ which starter recently.
You can select βPrediction Submissionsβ to view the previous submissions and their relevant scores.

π Code Submission Round Launched π
YesterdayHi @wufanyou,
The absolute file paths are available as parameters of predict
function.
(to get rid of any confusion participants may face with directory structures)
We have data
, models
, etc folders available as examples for the getting started section and a better local development experience. You are free to upload your models and relevant files in folder/structure of your choice. The only constraint is that the entrypoint to your code evaluations will be run.py
.
The idea to provide product catalogue by default sounds good.
Please let us check internally and we will update on the request soon.

π Code Submission Round Launched π
YesterdayHi @wufanyou,
Thanks for your questions.
The submission process is as follows
The public test set phase is available for debugging purposes.
You can run your code end to end on the public split and the logs for this phase will be available, etc. The time taken in image building and public split stages, etc would not be counted towards your 30 minutes timeout (and have their own timeouts to avoid misuse).
The private split run however have 30 minutes purely reserved for your end to end run on the private dataset.
Yes, the default docker image is nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04
, however you are not restricted to use that and can choose to use requirements.txt
, environment.yml
to even custom images & Dockerfile
.
Please note, in case you are using your custom image, you need to make sure that they have relevant Nvidia drivers installed to make use of the GPU.
A full detail about the runtime is available in starter kitβs description as well as on Discourse here.
Please check more details in the starter kitβs README and let us know if you still have any follow up questions.

π Code Submission Round Launched π
Yesterday
Are there any way to modify the team nameοΌ
2 days agoHi @wufanyou,
Unfortunately you canβt modify the team name directly right now.
Please drop a message with your previous & new team name.
We will get it changed shortly!

Is there a limit on the total number of submissions when merging teams?
4 days agoHi @heya5,
This challenge has daily submissions limit of 5 submissions/day/task/team.
While I understand the reason behind submission * total_day
condition for team merging on other platforms.
We do not have this check currently for this challenge.
Please note, that, to tackle above issue, we generally have team freeze deadline after which new team members or teams canβt be created. It is on July 1, 2022 for this challenge.
Best,
Shivam

π Dataset Update: `v0.3` of the dataset has been released π
9 days agoHi @trenusch, please verify example_id
column in your submission.
It looks like you are using indices [0....n]
instead of the the example_id
which are required for a valid submission, due to which your submissions are failing.
You can check sample submission for the correct example_id
s as well.

π Dataset Update: `v0.3` of the dataset has been released π
9 days agoHi @trenusch,
The new test dataset files are already available on the resources page along with this announcement.
The new submissions since the release have the scores calculated on the new dataset (along with older submissions re-evaluated automatically).
We will start accepting the code submissions from 21st of June, 2022, until then you can continue submitting your predictions on the
v0.2
test sets. However, the evaluators will only consider the subset of the test set which are present in thev0.3
of the test sets.
Please let us know in case you are still facing some issue in accessing or submitting solution on new Task 3 dataset.

π Dataset Update: `v0.3` of the dataset has been released π
11 days agoYes, you can use it for training.
Spotify Million Playlist Dataset Challenge


Which is the deadline?
About 1 month agoHi, there is no deadline.
Itβs an always running benchmark.
HTREC 2022

No way to load SequenceTagger
19 days agoHi @Bonum, here is an working example [should work on Colab] on how to use SequenceTagger
:
https://www.aicrowd.com/showcase/sequencetagger-example
Yup, the files are missing in GitHub repository, while available in HF, so I have just merged them both to make it work.
All the best !

No way to load SequenceTagger
19 days agoHi @Bonum, the notebook has following example:
from transformers import AutoTokenizer, AutoModel
tokeniser = AutoTokenizer.from_pretrained("pranaydeeps/Ancient-Greek-BERT")
model = AutoModel.from_pretrained("pranaydeeps/Ancient-Greek-BERT")
These models are hosted on HF due to which will work directly.
While you are trying to use SuperPeitho-FLAIR-v2/final-model.pt
which isnβt, as the author told in the README.
We provide a pre-trained POS Tagging model in the directory SuperPeitho-FLAIR-v2
That means, you need to clone the main repository and use the model file directly.
$> git clone git@github.com:pranaydeeps/Ancient-Greek-BERT.git
$> cd Ancient-Greek-BERT
[...]
## run your python code along with above load command
All the best !
Food Recognition Benchmark 2022

π‘ Solutions are Public
30 days ago
Local Run Produces Different AP
About 2 months agoHi all,
Thanks for opening this discussion, and it is indeed an important one.
I am happy that we all can brainstorm and improve the pipeline for all the participants together.
The scores calculated right now weβre using the complete annotation files as @Mykola_Lavreniuk suspected, while the predictions made by your codebases get added for the calculation purposes on the go. The intention has always been to provide real-time feedback to you to show the approximate scores as soon as possible.
Issue #1: Score gradually increases
The flaw
We run COCOEval using complete annotations BUT with the subset of images (predictions generated ones), due to which the score starts with a much lower value and grows to your final scores.
Future update
We will be making an update in the pipeline based on this thread to create a subset of annotations file dynamically, calculate the scores with that for live reporting.
Issue #2: Validation scores donβt make sense
The flaw
We were using only 100 images in the validation phase because it worked as an integration test. We missed providing a proper scores table for the validation phase that you can use to compare local v/s online performance β which is our bad. The scores get calculated with annotation files for the complete validation subset, but only 100 images are present in the validation phase (as in Issue #1), causing much lower scores.
Future update
We will continue use a tiny subset to limit the compute used in validation phase and believe the score on 100 images should be reliable enough to compare local v/s online scoring. The list of these ~100 images will be shared in the resources section in future so you can reproduce the scores locally. (meanwhile the current scores were quite low, example 0.09AP while full evaluation for a lot more, it was happening due to Issue #1)

End of Round 2β±οΈ
About 2 months agoRound 2 of Food Recognition Benchmark has ended !!
This round saw a lot more activity and a lot more rank swaps on the leaderboard.
Thanks to you all for making it a success.
While a few of submissions are being re-evaluated at the moment, the public leaderboard looks like this right now:
The number of participants who crossed the official baseline has increased from 12 to 15.
At the same time, participants having score >= 0.200 AP is increased from 6 to 11 as well!
We canβt wait to complete the due diligence before making the winnersβ announcement.
Stay tuned for it!
Why Food Recognition Benchmark matters?
Itβs a good time to re-share about exciting updates on how the community collaboration has worked so far and what itβs enabling! [Blog Post]
The Food Recognition Benchmark is not a competition, but an ongoing collaboration with a community of inspiring individuals like yourself, to deliver the future of diet tracking in research cohorts around the world.
What can you do meanwhile?
Start some interesting discussions with other challenge participantsβ here on the Forums or on Discord Chat.
Share what worked and what didnβt during the competition. And maybe some interesting approaches you may want to try out in future?
Share any feedbacks you have for future Rounds either in this thread of here.
Guess the winners in this thread

ImageCLEF 2022 Caption - Caption Prediction

Notebooks
-
SequenceTagger Example It uses SuperPeitho-FLAIR-v2 provided in Ancient Greek BERTshivamΒ· 19 days ago
-
π Food Recognition Benchmark: Data Exploration & Baseline Dataset exploration and `detectron2` baseline training codeshivamΒ· 6 months ago
-
AOT Dataset walkthrough using helper scripts Demo of dataset helper scripts and walkthrough to the training datasetshivamΒ· About 1 year ago
-
Evaluate your predictions locally (validation flights) Demo on using metrics codebase for generating scores locallyshivamΒ· About 1 year ago
-
File name to frame metadata exploration Quick demo on how you can get metadata information of given frame image pathshivamΒ· About 1 year ago
-
BaseLine EfficientNet B0 224 Example of training data download via cli, with official baseline by Snakes OrganisersshivamΒ· Over 1 year ago
-
Download dataset in Colab/Notebook via CLI This notebook contain example on how to download dataset via notebook/colab using aicrowd-clishivamΒ· Over 1 year ago
-
Getting started with TIMSER (baseline notebook) Baseline containing code for submitting submissions for Time Series Prediction (AI Blitz 5)shivamΒ· Over 1 year ago
Notebooks
-
π Food Recognition Benchmark: Data Exploration & Baseline Dataset exploration and `detectron2` baseline training codeshivamΒ· 6 months ago
-
MMdetection training and submissions (Quick, Active) MMdetection training and evaluation code, getting 0.11 score on the leaderboardjerome_patelΒ· 5 months ago
-
Detectron2 training and submissions (Quick, Active) v2 of baseline, now supports submissions via Colab for both quick and active participationjerome_patelΒ· 5 months ago
-
EDA, FE, HPO - All you need (LB: 0.640) Detailed EDA, FE with Class Balancing, Hyper-Parameter Optimization of XGBoost using Optunajyot_makadiyaΒ· About 1 year ago
-
Evaluate your predictions locally (validation flights) Demo on using metrics codebase for generating scores locallyshivamΒ· About 1 year ago
-
AOT Dataset walkthrough using helper scripts Demo of dataset helper scripts and walkthrough to the training datasetshivamΒ· About 1 year ago
Is there a limit on the total number of submissions when merging teams?
About 20 hours agoHi @good-good-study, yes, it is 5 submissions/task/day.
We have just resetted the submission limit due to issue with code submissions earlier which should solve the wrong counting. Please let us know incase you still face any issues.