Seattle, US
Activity
Ratings Progression
Challenge Categories
Challenges Entered
Completed
Max Planck Institute for Intelligent Systems
Disentanglement: from simulation to real-world
Latest submissions
MineRL Labs - Carnegie Mellon University
Sample-efficient reinforcement learning in Minecraft
Latest submissions
Unity Technologies
A new benchmark for Artificial Intelligence (AI) research in Reinforcement Learning
Latest submissions
See All| graded | 8950 | ||
| graded | 8947 | ||
| graded | 8894 |
| Participant | Rating |
|---|
| Participant | Rating |
|---|
Unity Obstacle Tower Challenge
End time / date is wrong
Almost 3 years agoIn the rules/competition overview it states that the Round 2 end date is: 11:59 pm, Monday, July 15th
However, on the competition ticker, it states Jul 15th, 7:59 UTC which would be 11:59 pm, Sunday, July 14th. - it should be Jul 16th, 7:59 am UTC
Submissions are stuck
Almost 3 years agoI submitted earlier today and they have been stuck in image build mode for 3hrs:
https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/145
and
https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/146
Why I was blocked from uploading new models for 3 days
Almost 3 years ago@Miffyli - which platform are you running on? I can not get it working in MacOS and from what I read, docker does not support --network=host on Mac
Why I was blocked from uploading new models for 3 days
Almost 3 years agoit’s when I try and run it local per these steps: https://github.com/Unity-Technologies/obstacle-tower-challenge#run-docker-image
it used to work, but not since the update of aicrowd-repo2docker - @arthurj, does it work for you guys on Mac?
Why I was blocked from uploading new models for 3 days
Almost 3 years agoThanks @mohanty! One problem I’m still having is running the test seeds locally (100-105) - when I invoke the environment docker it ends. I’m trying on MacOS
Why I was blocked from uploading new models for 3 days
Almost 3 years agoI’ve spent countless hours over the past 3 days trying to figure out why I could not upload/evaluate a new model nor reproduce the problem locally.
Basically, there is a breaking change whereby existing code will no longer run server-side - It would have saved me hours had there been a better error and/or someplace to look for notifications (I don’t think it helps to have 2 unity repros with issue trackers as well as the ai-crowd message board)
- June 13th, was my last successful upload of a model.
- On July 5th I tried to upload a new model - the only change to my code base the addition of the model and a reference to that model.
- When
aicrowd-botposted the log: it just said this:
2019-07-06T07:13:55.1056876Z root
2019-07-06T07:13:55.129380334Z Traceback (most recent call last):
2019-07-06T07:13:55.12943787Z File "run_evaluation.py", line 7, in <module>
2019-07-06T07:13:55.129471922Z import gym
2019-07-06T07:13:55.129478407Z ModuleNotFoundError: No module named 'gym'
- I learned that we can now do debug submits: Announcement: Debug your submissions however, all it gave was the same log
- I tried to reproduce locally, however, the
build.shscript was giving me the error:AttributeError: /srv/conda/bin/python: undefined symbol: archive_errno - I thought maybe a conda or pip package update may have broken something so I manually tied each one to the valid version from June 13th
- I thought there may be some local issue with my docker, so I cleaned, deleted, reset
- I ran
pip install --upgrade aicrowd-repo2dockerand saw that it updated. This solved my local issue and was able to reproduce the server side error. - Given that
aicrowd-repo2dockerhad been updated, i thought to look at the commit logs and found that this https://github.com/Unity-Technologies/obstacle-tower-challenge/commit/99c68faf2ed0f01ee8bc3e411bbdd4e85484a733 removedsource activate basefrom run.sh
Note: I still can not test locally - the agent code runs, but the environment docker immediately drops out. I also have to manually delete the docker image to force it to rebuilt (this was not the case prior to the aicrowd-repo2docker upgrade
Is the new v2.2 used for scoring?
About 3 years agoLooks good! It would be great to have the videos fixed so we can see @unixpickle 's agent through level 16!!
Is the new v2.2 used for scoring?
About 3 years agooh strange - it recorded an average floor of 6, then removed it when it restarted training
Is the new v2.2 used for scoring?
About 3 years agoIt looks like the video worked and it updated the leaderboard but then it started scoring again.
Is the new v2.2 used for scoring?
About 3 years ago@mohanty - it failed - it looks like it gets stuck when recording the video and then times out
Is the new v2.2 used for scoring?
About 3 years agoI tried again - this time i get ‘Unable to orchestrate submission, please contact Administrators.’
https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/124
Is the new v2.2 used for scoring?
About 3 years ago@mohanty - Have you been able to update v2.2? I resubmitted my agent but got errors:
https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/123
UnityTimeOutException in evaluation
About 3 years ago@mohanty - would you post the logs on my submission here - https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/121 - I’m not sure if it is the same problem. thanks
Possible bug with tower-seed in env 2.0
About 3 years agocould it be related to this issue: https://github.com/Unity-Technologies/obstacle-tower-env/issues/91#issuecomment-494218846
they pushed a fix for this
Has the submission process changed for round 2?
About 3 years agoI tried re-submitting my agent from round #1 to see how it did on round #2 - but get the following error
did something change? (all i did was push a text change and re-tag my repro)
The following containers terminated prematurely. : agent
Please contact administrators, or refer to the execution logs.
https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/119
Multiple dead submissions
About 3 years agoHi @mohanty - thanks for pointing this out - sorry I didn’t see this until this evening and so had continued to push but will back off per you points.
I think it’s great that you extended the timeline - it gives us the time to take more risks with experimental approaches.
Multiple dead submissions
Over 3 years agoOK - thanks! as a rule, should I try and limit the number of concurrent submissions?
I know statistically that my current agent can score 10 on each seed so if I submit it enough times then at some point it should score that.
Multiple dead submissions
Over 3 years ago@mohanty - I think I may have overloaded the system last night - I was trying to trigger multiple submissions so that I have them all queue up. Two show as Failed, Eight show as ImageBuildStarted after 15 hrs.
I’m not sure if there is anything I can do on my end
(Win10) ModuleNotFoundError: No module named 'pwd'
Over 3 years agoI also tried aicrowd-repo2docker . and received the same error
Submissions are stuck
Almost 3 years agoBoth submissions failed after 9hrs with the message
Task exceeded maximum timeout value (32400 seconds)I tried submitting again, it has not gone into image_build mode - https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/147
I looks like submissions are totally down