Location
Badges
Activity
Ratings Progression
Challenge Categories
Challenges Entered
Evaluate Natural Conversations
Latest submissions
Specialize and Bargain in Brave New Worlds
Latest submissions
See Allgraded | 244935 |
Small Object Detection and Classification
Latest submissions
See Allgraded | 240474 | ||
failed | 240454 | ||
failed | 240438 |
Identify user photos in the marketplace
Latest submissions
See Allfailed | 210395 | ||
failed | 210392 | ||
failed | 210264 |
Specialize and Bargain in Brave New Worlds
Latest submissions
Predicting smell of molecular compounds
Latest submissions
Latest submissions
Participant | Rating |
---|
Participant | Rating |
---|
-
collision_course NeurIPS 2022 - The Neural MMO ChallengeView
-
pho MosquitoAlert Challenge 2023View
MosquitoAlert Challenge 2023
Submission failed during Inference
6 months ago@dipam the submission codes are
#238089 #238088 #238030 #238029
Submission failed during Inference
6 months agoHi @dipam, can you have a quick look at this? We really are stuck since Mosquitoalert Validation runs through but Mosquitoalert Prediction does not, and we do not have any informative error logs.
π¬ Feedback & Suggestions
8 months agoThank you for organizing this competition!
I would like to provide feedback on the ground truth bounding boxes.
I compared the ground truth bounding boxes of the training data against the bounding boxes of a pre-trained model called Owl-ViT, and below are some issues which, as I am convinced, adversely effect the quality of the competition:
Bounding Box Quality
In some cases, the quality of the ground truth bounding boxes are bad, leading to low IoU between the ground truth and Owl-ViT, although Owl-ViT arguably leads to better bounding boxes. In those cases the IoU between the two boxes would even be below the required threshold for scoring.
Multiple Objects in One Image
In case of multiple objects, sometimes the ground truth bounding box is around the smaller object. I argue that one should either split those images in two such that no image contains more than one object, or have the ground truth around the largest object.
Bounding Boxes Around Multiple Objects
From my point of view, each ground truth bounding box should be around one object only.
Since the challenge metric involves thresholding the IoU between the predicted and (arguably bad) ground truth bounding boxes, I fear the scope of the challenge might change from trying to locate mosquitos as best as possible to reproducing the ground-truth bounding boxes as best as possible to achieve a good challenge score.
I therefore suggest to update the ground truth bounding boxes, or adjust the challenge metric as to not include the IoU.
Submission failed during Inference
6 months agoIn the validation phase, we have inference times of around 1s. So I assume the reason is that there are higher-res images in the test data, could that be?