Loading
0 Follower
0 Following
joe_booth
Joe Booth

Organization

Vidya Gamer, LLC

Location

Seattle, US

Badges

2
1
1

Connect

Activity

Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
Jan
Feb
Mar
Mon
Wed
Fri

Ratings Progression

Loading...

Challenge Categories

Loading...

Challenges Entered

Latest submissions

No submissions made in this challenge.

Sample-efficient reinforcement learning in Minecraft

Latest submissions

No submissions made in this challenge.

A new benchmark for Artificial Intelligence (AI) research in Reinforcement Learning

Latest submissions

See All
graded 8950
graded 8947
graded 8894
Participant Rating
Participant Rating

Unity Obstacle Tower Challenge

Submissions are stuck

Over 4 years ago

Both submissions failed after 9hrs with the message Task exceeded maximum timeout value (32400 seconds)

I tried submitting again, it has not gone into image_build mode - https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/147

I looks like submissions are totally down

End time / date is wrong

Over 4 years ago

In the rules/competition overview it states that the Round 2 end date is: 11:59 pm, Monday, July 15th

However, on the competition ticker, it states Jul 15th, 7:59 UTC which would be 11:59 pm, Sunday, July 14th. - it should be Jul 16th, 7:59 am UTC

Why I was blocked from uploading new models for 3 days

Over 4 years ago

@Miffyli - which platform are you running on? I can not get it working in MacOS and from what I read, docker does not support --network=host on Mac

Why I was blocked from uploading new models for 3 days

Over 4 years ago

it’s when I try and run it local per these steps: https://github.com/Unity-Technologies/obstacle-tower-challenge#run-docker-image

it used to work, but not since the update of aicrowd-repo2docker - @arthurj, does it work for you guys on Mac?

Why I was blocked from uploading new models for 3 days

Over 4 years ago

Thanks @mohanty! One problem I’m still having is running the test seeds locally (100-105) - when I invoke the environment docker it ends. I’m trying on MacOS

Why I was blocked from uploading new models for 3 days

Over 4 years ago

I’ve spent countless hours over the past 3 days trying to figure out why I could not upload/evaluate a new model nor reproduce the problem locally.

Basically, there is a breaking change whereby existing code will no longer run server-side - It would have saved me hours had there been a better error and/or someplace to look for notifications (I don’t think it helps to have 2 unity repros with issue trackers as well as the ai-crowd message board)

  • June 13th, was my last successful upload of a model.
  • On July 5th I tried to upload a new model - the only change to my code base the addition of the model and a reference to that model.
  • When aicrowd-bot posted the log: it just said this:
2019-07-06T07:13:55.1056876Z root
2019-07-06T07:13:55.129380334Z Traceback (most recent call last):
2019-07-06T07:13:55.12943787Z   File "run_evaluation.py", line 7, in <module>
2019-07-06T07:13:55.129471922Z     import gym
2019-07-06T07:13:55.129478407Z ModuleNotFoundError: No module named 'gym'
  • I learned that we can now do debug submits: Announcement: Debug your submissions however, all it gave was the same log
  • I tried to reproduce locally, however, the build.sh script was giving me the error: AttributeError: /srv/conda/bin/python: undefined symbol: archive_errno
  • I thought maybe a conda or pip package update may have broken something so I manually tied each one to the valid version from June 13th
  • I thought there may be some local issue with my docker, so I cleaned, deleted, reset
  • I ran pip install --upgrade aicrowd-repo2docker and saw that it updated. This solved my local issue and was able to reproduce the server side error.
  • Given that aicrowd-repo2docker had been updated, i thought to look at the commit logs and found that this https://github.com/Unity-Technologies/obstacle-tower-challenge/commit/99c68faf2ed0f01ee8bc3e411bbdd4e85484a733 removed source activate base from run.sh

Note: I still can not test locally - the agent code runs, but the environment docker immediately drops out. I also have to manually delete the docker image to force it to rebuilt (this was not the case prior to the aicrowd-repo2docker upgrade

Is the new v2.2 used for scoring?

Almost 5 years ago

Looks good! It would be great to have the videos fixed so we can see @unixpickle 's agent through level 16!!

Is the new v2.2 used for scoring?

Almost 5 years ago

oh strange - it recorded an average floor of 6, then removed it when it restarted training

Is the new v2.2 used for scoring?

Almost 5 years ago

It looks like the video worked and it updated the leaderboard but then it started scoring again.

Is the new v2.2 used for scoring?

Almost 5 years ago

@mohanty - it failed - it looks like it gets stuck when recording the video and then times out

Is the new v2.2 used for scoring?

Almost 5 years ago

I tried again - this time i get β€˜Unable to orchestrate submission, please contact Administrators.’

https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/124

Is the new v2.2 used for scoring?

Almost 5 years ago

@mohanty - Have you been able to update v2.2? I resubmitted my agent but got errors:

https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/123

UnityTimeOutException in evaluation

Almost 5 years ago

@mohanty - would you post the logs on my submission here - https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/121 - I’m not sure if it is the same problem. thanks

Has the submission process changed for round 2?

Almost 5 years ago

I tried re-submitting my agent from round #1 to see how it did on round #2 - but get the following error

did something change? (all i did was push a text change and re-tag my repro)

   The following containers terminated prematurely. : agent
    Please contact administrators, or refer to the execution logs.

https://gitlab.aicrowd.com/joe_booth/obstacle-tower-challenge/issues/119

Multiple dead submissions

Almost 5 years ago

Hi @mohanty - thanks for pointing this out - sorry I didn’t see this until this evening and so had continued to push but will back off per you points.

I think it’s great that you extended the timeline - it gives us the time to take more risks with experimental approaches.

Multiple dead submissions

Almost 5 years ago

OK - thanks! as a rule, should I try and limit the number of concurrent submissions?

I know statistically that my current agent can score 10 on each seed so if I submit it enough times then at some point it should score that.

Multiple dead submissions

Almost 5 years ago

@mohanty - I think I may have overloaded the system last night - I was trying to trigger multiple submissions so that I have them all queue up. Two show as Failed, Eight show as ImageBuildStarted after 15 hrs.

I’m not sure if there is anything I can do on my end

(Win10) ModuleNotFoundError: No module named 'pwd'

About 5 years ago

I also tried aicrowd-repo2docker . and received the same error

joe_booth has not provided any information yet.