Same submissions with different weights failing

@dipam , could you please pay attention that earlier this problem solved just by resubmitting, now I have faced similar situation, that my submits (without any changes) failed is some cases, in other cases submission just stuck for 10hours or even more than 24 hours.

image1448×729 78.5 KB

View post 1

My solution for the challenge

We have tried TRT before inference in our server. We also use reranking on GPU, but maybe this was longer…
even with VIT-H + reranking our solution was almost 10min, cause in some cases it failed and in some cases run successfully, depending on hardware.

View post 0

My solution for the challenge

@bartosz_ludwiczuk, Congratulations on achieving second place! I’m looking forward to reading about your winning solution.

View post 0

My solution for the challenge

We attempted to incorporate multiple external datasets into our experiments, spending considerable time trying to train our VIT-H model jointly with Product10k and other datasets, as well as training on other datasets and fine-tuning on Product10k. Surprisingly, despite our efforts, our current leaderboard score was achieved only by using the Product10k dataset; all other datasets resulted in a decrease in our score.

To improve our results, we utilized re-ranking for postprocessing, which gave us a marginal improvement of approximately 0.01%. Additionally, we experimented with convnext and VIT-G models, which boosted our local score by about 0.03%. However, even with the use of TensorRT, our models were unable to pass inference in 10 minutes.

View post 0

📥 Guidelines For Using External Dataset

we have tried:
rp2k
JD_Products_10K
Shopee
Aliproducts
DeepFashion_CTS
DeepFashion2
Fashion_200K
Stanford_Products

right now we are using only Products_10K, and models from OpenClip.

View post 0

Previous successful submissions failing

still waiting same, more than 23 hours

View post 1

Product Matching: Inference failed

@dipam, thank you for the quick response

View post 0

Product Matching: Inference failed

@dipam , have you changed some settings of the server for inference?
Previously I have faced near 1 failed submitting per day, just rerunning helps.
however today I have changed only NN weights files and thats all, 1 get 1 submit ok, and 4 other weights with same size, same model all same just other epochs - failed.
As to me it is very strange…
Could you pls check it?

#214502

#214483

#214469

If it is timeout, how it could be if other weights are ok, or just resubmitting sometimes helps?

View post 0

Is everybody using same Products 10k dataset

yes, they are strongly correlated

View post 0

Product Matching: Inference failed

@dipam, could you please check
#212567
#212566
It is failed in step “Build Packages And Env”, however I have changed only NN params.
As to me it is very strange…

View post 1

Is everybody using same Products 10k dataset

I don’t know about other competitors (could only assumed), but we use only this dataset, and it is sufficient.
So working on model, training pipeline, etc. provide the possibility to get LB=0.62114.

View post 7

Round1 and Round2

@dipam , thank you for clarification, now for me it looks understandable!

View post 0

Round1 and Round2

@dipam, could you please provide further details about the current situation? It appears that there is some confusion regarding the status of rounds 1 and 2. Additionally, updating the dataset at this stage of the competition seems unusual.

To clarify, it seems that round 1 has been completed, but there was no mention of it in the initial description. It is unclear what this means for the competition as a whole.

Furthermore, updating the dataset at this point in the competition seems questionable, especially if the previous dataset was used for calculating the leaderboard score. It is uncertain whether a new dataset will be used for the leaderboard calculation, and if so, how this will affect the competition.

View post 0

End of Round 2⏱️

Thanks to Aicrowd for hosting this yearly competition and benchmark. It was a lot of fun working on it, exploring and learning models for instance segmentation for solving the task on food recognition.
Thanks for my teammates for this work.
Thanks @shivam for a lot of helping us with aicrowd infrastructure, @gaurav_singhal for your paper on your previous year best approach and unbelievable race on score other participants for good time within this competition!
Also, congratulations to @gaurav_singhal and @Camaro!!!

View post 2

Extension of the round?

I understood, so the problem was with countdown. Thank you for clarification

View post 1

Extension of the round?

Hi, @shivam
Did the competition has been extended?
In the morning timer was 3 days left, now it is 5 days left and I could not find any announcement of it?

View post 0

Getting less mAP on MMdetection

If you need more info about difference between 1 and 2 stage models, you could read about RPN (region propose network) for example, how it works.
And also, for example about SOLO (1 stage model).
You could try solo in mmdet library as is and see if it helps you to boost your score.

View post 2

Getting less mAP on MMdetection

Hi, @saidinesh_pola,
There is a big secret here in data processing before training (as for me). And it is not so big difference between detectron2 and mmdetection.
I assumed that it is not big secret that
#1 team uses mmdet,
#2 team uses mmdet,
#3 team uses mmdet,
#4 team uses detectron2,
(you could observe it due to the errors in some of their submissions).
So, as for me mmdet is better due to large variety of different models, but it is not the main reason here.
For a long time I could not find the idea with data processing and tried to train data as is. And found that only 1 stage models could get good score, however all models with 2 and more stages are not good here (like mask-rcnn, HTC, detectors and so on.)
When I understood the difference between 1 and 2 stage models, I quickly find the data processing idea.
I think difference between your score and other teams from top is dataprocessing step, not the model, parameters or sophisticated augmentation.
I’d recommend you to try out the 1 stage model and see the results…

View post 1

Local Run Produces Different AP

Yes, I have same results. I assumed the evaluation are going in several machines or at least gpus, so they have not wait until all the images will be processed and calc mAP after it. Organizers probably decided to evaluate the score during the inference, and I assume when you get for example 10% of images processed other 90% of answers setted as 0.
Thus, your score is continuous improving within the evaluation more and more photos (less zeros remains).

View post 1

MMdetection sumission

Yes, in this round val dataset has been copied to train dataset. So, you should just remove images that are in val dataset from the train dataset by yourself, and you will get same workflow as in round 1.
Hope it will help you.

View post 3