Loading
0 Follower
0 Following
jon_grantham

Location

US

Badges

2
1
1

Connect

Activity

Dec
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
Mon
Wed
Fri

Ratings Progression

Loading...

Challenge Categories

Loading...

Challenges Entered

Multi-Agent Dynamics & Mixed-Motive Cooperation

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

See All
graded 197404
failed 197402
graded 197401

ASCII-rendered single-player dungeon crawl game

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.
Participant Rating
Participant Rating

NeurIPS 2021 - The NetHack Challenge

🧞 Requesting Feedback and Suggestions

About 2 years ago

I believe the finals will use 4096 runs.

I really like the stable distribution idea – maybe next year? One of the things I have considered doing is writing a function to normalize our local scores to an even role distribution, but it has never moved to the top of my priority list.

Are the specs for the machine that the evaluations are run on available anywhere?

About 2 years ago

Thank you. That helps a lot.

When you say, “a single env should roughly give a throughput of 1500-2000 steps/second” – is that without considering time spent in the agent? How much of that is network latency? I.e., if we have an agent capable of performing faster than that, does all of the agent compute time get hidden under the latency?

Are the specs for the machine that the evaluations are run on available anywhere?

About 2 years ago

It is hard to optimize the speed without knowing this. In particular, code changes that have given a 2x speedup on different local machines do not seem to have impacted the submission run time.

jon_grantham has not provided any information yet.