Key Updates: Local End-to-End Evaluation Script (Docker), Temporal Track Dependency Fix, and Dockerfile Optimizations

Dear, @aicrowd_team!
I’m writing to kindly inquire about the expected timeline for when the private leaderboard will be officially available.
Thank you very much for your great work.

Best regards,
kcy4

View post 0

💬 Feedback & Suggestions

1. It’s for [Temporal Alignment Track]
2. I didn’t understand the meaning of “even if three systems from the same participant”. The competition is in phase 2(stage) now. Do you mean that the best system(submission) for both phase 1 and phase 2 will be used for the final? There was warm up stage, but we can’t see the stage(warm up) result now.

View post 0

💬 Feedback & Suggestions

Dear, @aicrowd_team.
Thank you for this great competition.
I have two questions.

1. What is the exact eval score?

• According to the leaderboard, the eval metric seems like it is av_aglign, which is the same as TA for #280900, but is the same as AV_ALIGN for #280380 for me. Which one is the correct one?

2. Can we choose two submissions at the end of the competition?

Best regards

View post 0

Key Updates: Local End-to-End Evaluation Script (Docker), Temporal Track Dependency Fix, and Dockerfile Optimizations

@aicrowd_team : I checked the fix. Thank you.

View post 0

Key Updates: Local End-to-End Evaluation Script (Docker), Temporal Track Dependency Fix, and Dockerfile Optimizations

@samtech: How about using ssh?

View post 0

Key Updates: Local End-to-End Evaluation Script (Docker), Temporal Track Dependency Fix, and Dockerfile Optimizations

Dear, @aicrowd_team.
I have a question.
Is it okay to ask you to review the competition score(or metric) for the ‘Temporal Alignment Track’.

P.S. Thank you for the great competition. This competition is really interesting in terms of its subject and difficulty for me.

View post 0

Key Updates: Local End-to-End Evaluation Script (Docker), Temporal Track Dependency Fix, and Dockerfile Optimizations

@aicrowd_team : Thank you for the speed fix.

View post 0

Key Updates: Local End-to-End Evaluation Script (Docker), Temporal Track Dependency Fix, and Dockerfile Optimizations

@aicrowd_team
Dear, aicrowd_team.
I think there is a bug in an evaluation score after this update.
Before the update, the scores of spatial_svgc_err and spatial_av were not the same.
However, after it, the scores are getting the same.
Could you check this bug, please?
Thank you very much.

Bug picture

스크린샷 2024-12-28 오전 8.28.34448×936 20.6 KB

View post 0

About the Temporal Alignment Track category

@aicrowd_team : Thank you for the update. I confirmed that there are two caption files(flist_train.csv & flist_val.csv).

View post 0

About the Temporal Alignment Track category

I downloaded the data a few minutes ago. But I still don’t see any caption data. Please, can you check this again?

View post 0

Service Announcement: Delays in GPU Node Provisioning

Hi, @mohanty
Thank you for your great work and announcement of this isssue.
Do we have to wait until you notify us of which we can submit again?

View post 0

Seems like EVERYONE's submission is getting 'FAILED'

Dear, hosters.
Are you monitering the competition’s leaderboard status?
It seems like everyone’s submission is getting failed since yesterday evening.
Would you review this problem and give us feedback?

Thank you.

p.s. Error message before error: Conv Ai Validation: Preparing the cluster for you

View post 1

Local evaluation was successful, but submission FAILED

I solved this problem by submitting the previous version which was successful. I didn’t find out the reason why error occurred.

View post 1

Local evaluation was successful, but submission FAILED

Dear, hosters!
I have one question.
I have submitted after local_evaluation was successful, but the submission failed.
Submission information is following.
submission id: #249018
submission hash: 9589f09230f521d133d20948e97c5e7b734444bb
Status in which error occurred: Conv Ai Validation
Status message before error: ImagePullBackOff: Back-off pulling image “aicrowd/submission:249018”
Error message: ErrimagePull: pc error: code = Unknown desc = failed to register layer: Error processing tar file (exit status 1): unexpected EOF
Would you review this problem, please?

Thank you
kcy4

View post 0

How to know my submission was successful and what is the right way to make a successful submission?

Dear @dipam,
Thank you for your answer.
I checked the logs and understood your clear guide.

View post 0

How to know my submission was successful and what is the right way to make a successful submission?

Dear, hosters!
Thank you for the great competition.
I have two questions.
Is there any method to know whether my submission is successful or not, and what is the correct method to make a successful submission?
I have submitted a dummy model which returns an empty image without any touch. But, I cannot see whether the submission is successful or not. I am asking you this because there are some mis-guide or misspellings.

1. According to “How to write your own model?”, you recommend to create a directory named ‘agents.’
   However, you say that we have to edit user_config.py in a directory of ‘agent.’
2. According to “How to write your own model?”, agent/user_config.py will be used for the evaluations. But according to “Submission Entrypoint”, models/user_config.py is used for the evaluations.
3. According to “How to write your own model?”, the function generate_responses is called for the evaluations. However according to ‘models/controlnet_model.py’, generate_design was implemented. There is no generate_responses function.

Because of these, I am confused about the right method for the evaluations.
Can you make it more clear?

Thank you,
Seongcheol Kim

View post 0