Activity
Ratings Progression
Challenge Categories
Challenges Entered
Disentanglement: from simulation to real-world
Latest submissions
Sample-efficient reinforcement learning in Minecraft
Latest submissions
A new benchmark for Artificial Intelligence (AI) research in Reinforcement Learning
Latest submissions
See Allgraded | 8950 | ||
graded | 8947 | ||
graded | 8894 |
Participant | Rating |
---|
Participant | Rating |
---|
Unity Obstacle Tower Challenge
End time / date is wrong
Over 4 years agoIn the rules/competition overview it states that the Round 2 end date is: 11:59 pm, Monday, July 15th
However, on the competition ticker, it states Jul 15th, 7:59 UTC which would be 11:59 pm, Sunday, July 14th. - it should be Jul 16th, 7:59 am UTC
Submissions are stuck
Over 4 years agoI submitted earlier today and they have been stuck in image build mode for 3hrs:
https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/145
and
https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/146
Why I was blocked from uploading new models for 3 days
Over 4 years ago@Miffyli - which platform are you running on? I can not get it working in MacOS and from what I read, docker does not support --network=host on Mac
Why I was blocked from uploading new models for 3 days
Over 4 years agoitβs when I try and run it local per these steps: https://github.com/Unity-Technologies/obstacle-tower-challenge#run-docker-image
it used to work, but not since the update of aicrowd-repo2docker
- @arthurj, does it work for you guys on Mac?
Why I was blocked from uploading new models for 3 days
Over 4 years agoThanks @mohanty! One problem Iβm still having is running the test seeds locally (100-105) - when I invoke the environment docker it ends. Iβm trying on MacOS
Why I was blocked from uploading new models for 3 days
Over 4 years agoIβve spent countless hours over the past 3 days trying to figure out why I could not upload/evaluate a new model nor reproduce the problem locally.
Basically, there is a breaking change whereby existing code will no longer run server-side - It would have saved me hours had there been a better error and/or someplace to look for notifications (I donβt think it helps to have 2 unity repros with issue trackers as well as the ai-crowd message board)
- June 13th, was my last successful upload of a model.
- On July 5th I tried to upload a new model - the only change to my code base the addition of the model and a reference to that model.
- When
aicrowd-bot
posted the log: it just said this:
2019-07-06T07:13:55.1056876Z root
2019-07-06T07:13:55.129380334Z Traceback (most recent call last):
2019-07-06T07:13:55.12943787Z File "run_evaluation.py", line 7, in <module>
2019-07-06T07:13:55.129471922Z import gym
2019-07-06T07:13:55.129478407Z ModuleNotFoundError: No module named 'gym'
- I learned that we can now do debug submits: Announcement: Debug your submissions however, all it gave was the same log
- I tried to reproduce locally, however, the
build.sh
script was giving me the error:AttributeError: /srv/conda/bin/python: undefined symbol: archive_errno
- I thought maybe a conda or pip package update may have broken something so I manually tied each one to the valid version from June 13th
- I thought there may be some local issue with my docker, so I cleaned, deleted, reset
- I ran
pip install --upgrade aicrowd-repo2docker
and saw that it updated. This solved my local issue and was able to reproduce the server side error. - Given that
aicrowd-repo2docker
had been updated, i thought to look at the commit logs and found that this https://github.com/Unity-Technologies/obstacle-tower-challenge/commit/99c68faf2ed0f01ee8bc3e411bbdd4e85484a733 removedsource activate base
from run.sh
Note: I still can not test locally - the agent code runs, but the environment docker immediately drops out. I also have to manually delete the docker image to force it to rebuilt (this was not the case prior to the aicrowd-repo2docker
upgrade
Is the new v2.2 used for scoring?
Almost 5 years agoLooks good! It would be great to have the videos fixed so we can see @unixpickle 's agent through level 16!!
Is the new v2.2 used for scoring?
Almost 5 years agooh strange - it recorded an average floor of 6, then removed it when it restarted training
Is the new v2.2 used for scoring?
Almost 5 years agoIt looks like the video worked and it updated the leaderboard but then it started scoring again.
Is the new v2.2 used for scoring?
Almost 5 years ago@mohanty - it failed - it looks like it gets stuck when recording the video and then times out
Is the new v2.2 used for scoring?
Almost 5 years agoI tried again - this time i get βUnable to orchestrate submission, please contact Administrators.β
https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/124
Is the new v2.2 used for scoring?
Almost 5 years ago@mohanty - Have you been able to update v2.2? I resubmitted my agent but got errors:
https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/123
UnityTimeOutException in evaluation
Almost 5 years ago@mohanty - would you post the logs on my submission here - https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/121 - Iβm not sure if it is the same problem. thanks
Possible bug with tower-seed in env 2.0
Almost 5 years agocould it be related to this issue: https://github.com/Unity-Technologies/obstacle-tower-env/issues/91#issuecomment-494218846
they pushed a fix for this
Has the submission process changed for round 2?
Almost 5 years agoI tried re-submitting my agent from round #1 to see how it did on round #2 - but get the following error
did something change? (all i did was push a text change and re-tag my repro)
The following containers terminated prematurely. : agent
Please contact administrators, or refer to the execution logs.
https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/119
Multiple dead submissions
Almost 5 years agoHi @mohanty - thanks for pointing this out - sorry I didnβt see this until this evening and so had continued to push but will back off per you points.
I think itβs great that you extended the timeline - it gives us the time to take more risks with experimental approaches.
Multiple dead submissions
Almost 5 years agoOK - thanks! as a rule, should I try and limit the number of concurrent submissions?
I know statistically that my current agent can score 10 on each seed so if I submit it enough times then at some point it should score that.
Multiple dead submissions
Almost 5 years ago@mohanty - I think I may have overloaded the system last night - I was trying to trigger multiple submissions so that I have them all queue up. Two show as Failed, Eight show as ImageBuildStarted after 15 hrs.
Iβm not sure if there is anything I can do on my end
(Win10) ModuleNotFoundError: No module named 'pwd'
About 5 years agoI also tried aicrowd-repo2docker .
and received the same error
Submissions are stuck
Over 4 years agoBoth submissions failed after 9hrs with the message
Task exceeded maximum timeout value (32400 seconds)
I tried submitting again, it has not gone into image_build mode - https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/147
I looks like submissions are totally down