AIcrowd | keyu_li | Participants

0 Follower

0 Following

keyu_li

Activity

Apr

May

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Jan

Feb

Mar

Apr

Mon

Wed

Fri

Ratings Progression

Challenge Categories

Challenges Entered

Completed

NeurIPS 2020: Procgen Competition

OpenAI

Measure sample efficiency and generalization in reinforcement learning using procedurally generated environments

Latest submissions

No submissions made in this challenge.

Completed

NeurIPS 2020: MineRL Competition

MineRL Labs - Carnegie Mellon University

Sample-efficient reinforcement learning in Minecraft

Latest submissions

See All

graded	85647	Wed, 30 Sep 2020 14:46:48
graded	85312	Tue, 29 Sep 2020 14:38:06
graded	84968	Mon, 28 Sep 2020 13:46:22

Participant	Rating

Participant	Rating

CU-SF NeurIPS 2020: Procgen Competition
View
CU-SF NeurIPS 2020: MineRL Competition
View

NeurIPS 2019 : MineRL Competition

Evaluation state changes to "error" after successfully completing many episodes

Over 3 years ago

Hello, our submission met an unexpected error after being evaluated for about 19 hours and having received a list of scores. The agent had successfully finished 50+ episodes with rewards, but somehow the evaluation state changed to “error” and the submission became “failed”. We have carefully inspected the logs, but it shows no error during the evaluation, so we really want to know the reasons.
Here is the end of the logs:

2020-09-18T12:25:48.915896553Z 2020-09-18 12:25:48 aicrowd-neurips-2020-minerl-challenge-83001-9136 minerl.env.malmo.instance.1716ad[1348289] DEBUG [12:25:48] [Server thread/INFO]: [STDOUT]: Warning: managed 11/20 ticks this second.
2020-09-18T12:25:49.95209292Z 2020-09-18 12:25:49 aicrowd-neurips-2020-minerl-challenge-83001-9136 minerl.env.malmo.instance.1716ad[1348289] DEBUG [12:25:49] [Server thread/INFO]: [STDOUT]: Warning: managed 12/20 ticks this second.
2020-09-18T12:25:50.163278504Z 2020-09-18 12:25:50 aicrowd-neurips-2020-minerl-challenge-83001-9136 minerl.env.malmo.instance.1716ad[1348289] DEBUG [12:25:50] [Client thread/INFO]: [CHAT] §c§l0…
2020-09-18T12:25:50.173719115Z 2020-09-18 12:25:50 aicrowd-neurips-2020-minerl-challenge-83001-9136 minerl.env.malmo.instance.1716ad[1348289] DEBUG [12:25:50] [Server thread/INFO]: [STDOUT]: SERVER request state: DORMANT
2020-09-18T12:25:50.179293551Z 2020-09-18 12:25:50 aicrowd-neurips-2020-minerl-challenge-83001-9136 minerl.env.malmo.instance.1716ad[1348289] DEBUG [12:25:50] [Server thread/INFO]: [STDOUT]: SERVER enter state: CLEAN_UP
2020-09-18T12:25:50.185303103Z 2020-09-18 12:25:50 aicrowd-neurips-2020-minerl-challenge-83001-9136 minerl.env.malmo.instance.1716ad[1348289] DEBUG [12:25:50] [Server thread/INFO]: [STDOUT]: SERVER request state: CLEAN_UP
2020-09-18T12:25:50.293747746Z 2020-09-18 12:25:50 aicrowd-neurips-2020-minerl-challenge-83001-9136 minerl.env.malmo.instance.1716ad[1348289] DEBUG [12:25:50] [Server thread/INFO]: [STDOUT]: SERVER enter state: DORMANT
2020-09-18T12:25:53.173893457Z 2020-09-18 12:25:53 aicrowd-neurips-2020-minerl-challenge-83001-9136 root[1348289] INFO Inference End…
2020-09-18T12:25:53.174399012Z 2020-09-18 12:25:53 aicrowd-neurips-2020-minerl-challenge-83001-9136 crowdai_api.events[1348289] DEBUG Registering crowdAI API Event : CROWDAI_EVENT_INFO inference_ended {‘event_type’: ‘minerl_challenge:inference_ended’} # with_oracle? : False
2020-09-18T12:25:53.175598194Z None
2020-09-18T12:25:53.175620833Z [0] Starting evaluator.
2020-09-18T12:25:53.175627878Z [1] Starting evaluator.
2020-09-18T12:25:53.175660057Z current total_reward in this episode is (some score)
2020-09-18T12:25:53.175667197Z [0] Episode complete
2020-09-18T12:25:53.175673348Z [0] Episode complete
… (omited many lines) …
2020-09-18T12:25:53.176192674Z [49] Episode complete
2020-09-18T12:25:53.80812693Z Ending testing phase

P.S., we use pytorch 1.6.0 and CPU mode in this submission. May I ask which version of CUDA do you use on the server? or can the server support Pytorch 1.6.0 in GPU mode? Thanks a lot!