Activity
Ratings Progression
Challenge Categories
Challenges Entered
Multi-Agent Dynamics & Mixed-Motive Cooperation
Latest submissions
Using AI For Buildingβs Energy Management
Latest submissions
Specialize and Bargain in Brave New Worlds
Latest submissions
See Allgraded | 197404 | ||
failed | 197402 | ||
graded | 197401 |
ASCII-rendered single-player dungeon crawl game
Latest submissions
Multi-Agent RL for Trains
Latest submissions
Participant | Rating |
---|
Participant | Rating |
---|
NeurIPS 2021 - The NetHack Challenge
π§ Requesting Feedback and Suggestions
About 3 years agoI believe the finals will use 4096 runs.
I really like the stable distribution idea β maybe next year? One of the things I have considered doing is writing a function to normalize our local scores to an even role distribution, but it has never moved to the top of my priority list.
Are the specs for the machine that the evaluations are run on available anywhere?
About 3 years agoThank you. That helps a lot.
When you say, βa single env should roughly give a throughput of 1500-2000 steps/secondβ β is that without considering time spent in the agent? How much of that is network latency? I.e., if we have an agent capable of performing faster than that, does all of the agent compute time get hidden under the latency?
Are the specs for the machine that the evaluations are run on available anywhere?
About 3 years agoIt is hard to optimize the speed without knowing this. In particular, code changes that have given a 2x speedup on different local machines do not seem to have impacted the submission run time.
Select your submissions for Test phase of the Nethack Challenge
Almost 3 years ago17 October, perhaps?