Submission issues

Hello @dipam_chakraborty

The GPU memory usage is not machine dependant. But it does vary a lot based on the version of tensorflow/pytorch used.

local runs are taking up only 14.3 GB (including the evaluation worker)

In that case, you can try replicating your local software stack during the evaluations (pip freeze / conda export)? Please let me know if I can help you with that.

View post 0

Submission issues

Hello @dipam_chakraborty

Every element shown on the graph is from a different node. Ideally, you would have 8 elements on the graph, 4 for training and 4 for rollouts. Sometimes, you end with more elements because we use preemptive nodes, and every new node also shows up there. Sometimes, for the failed submissions, you might see less than 8 elements as the jobs exited before the metrics could be scraped for that job.

View post 0

Submission issues

Hello @dipam_chakraborty

We re-queued the submission for evaluation.

View post 0

Rllib custom env

Hello @dipam_chakraborty

Thanks for sharing your thoughts. This is the reason why we want the participants to use ProcgenEnvWrapper. The evaluation env_config shared in the other post will be forced at the base env used in your wrapper. So, it is not possible to override the config that we set using a wrapper.

View post 0

Rllib custom env

Hello @tim_whitaker @anton_makiievskyi

We are passing a rollout flag in env_config. During the training phase we set this to rollout = False and during the rollouts, this will be set to rollout = True.

View post 1

Different training steps shown in gitlab

Hello @kaixin

This looks like a bug in plotting on the issue page. The complete plots and the corresponding logs (for 8M steps) are available on the submission dashboard. You can access the dashboard by following this link

View post 0

Custom Preprocessors

Hello @smileroom

You can place the code for your preprocessor at https://github.com/AIcrowd/neurips2020-procgen-starter-kit/blob/master/preprocessors/ and add the newly added custom preprocessor to the list of available preprocessors at https://github.com/AIcrowd/neurips2020-procgen-starter-kit/blob/master/preprocessors/__init__.py.

Then you should add it to the experiment YAML file, for example,

I hope the last part is what you were looking for.

View post 0

Change in daily submission limits for round 1

Hello,

We are decreasing the daily submission quota to 2 submissions per participant due to reduced cluster capacity.

View post 0

How to install external libraries?

Hello @dipam_chakraborty

I don’t think this was a network issue. The submission would fail before the start or would get marked failed after passing all the stages are the only possibilities we have with the network bug. I’m posting the relevant logs on the issue page.

View post 0

Customized trainer

Hello @kaixin

Did you have a look at https://github.com/AIcrowd/neurips2020-procgen-starter-kit/tree/master/algorithms/random_policy?

If you are looking for an even low-level implementation, this should be a good starting point, https://github.com/AIcrowd/neurips2020-procgen-starter-kit/blob/master/algorithms/custom_random_agent/custom_random_agent.py. If you plan to follow this one, please make sure that you are able to train your agent locally with the configuration provided in FAQ: Round 1 evaluations configuration.

View post 0

Question about 'num_outputs' in models

Hello @khrho_af

num_outputs comes from the environment’s action space. It is generally is equal to the dimension of the action space.

Please share your pain points so that we can improve the starter kit. Please feel free to post as many questions as you want. I’m sure that the community will be glad to help! This will also help others who might be searching for answers to similar questions.

View post 2

How to install external libraries?

Hello @dipam_chakraborty

I believe this is about submission #75240 and not #74986. We are still facing few network issues on our end and the evaluation didn’t start. However, we are re-queueing such failed submissions on our end and working on fixing the root cause of this.

Sorry for the inconvinience.

View post 0

I have question about max reward of env

Hello @minbeom

The max reward that is returned during the training is not the max reward of the environment. For every iteration we collect mean, min and max reward values of the finished episodes during that iteration. For example if your single iteration has 4 episodes, the rewards array for your iteration is, say, [0, 6, 20, 10], then

mean_reward =  6
min_reward = 0
max_reward = 20

Please note that these min, mean and max rewards are for the episodes of an iteration. The graph you shared plots these values for every iteration.

View post 0

AWS instance setup

Adding to @shivam’s response, we use the following values during the evaluation

RAY_MEMORY_LIMIT: "60129542144"
RAY_CPUS: "8"
RAY_STORE_MEMORY: "30000000000"

View post 1

Submission failed after training and rollouts completed

Hello @dipam_chakraborty

Apologies for this. We are having some network issues on the evaluation cluster, and hence few submissions are getting marked as failed inpsite of succeeding. Submission #74986 is one of them. We marked such submissions as succeeded and are working to fix the root issue causing this.

View post 0

FAQ: Round 1 evaluations configuration

Hello @xiaocheng_tang

Yes, you are free to pass additional parameters/flags in env_config. The only requirement is that the base env used by your gym wrapper should be ProcgenEnvWrapper provided in the starter kit.

The discussion from Rllib custom env might be useful to clear things up.

View post 0

Getting Rmax from environment

Hello @jurgisp

I do not think there is a way to get the max reward value from the gym environments. [refer]

The Rmin and Rmax for the publicly available environments are available at

For the private env used for round 1 (caterpillar), the min and max rewards are [R_min, R_max] = [8.25, 24] .

Please note that Rmin is the score for an agent that has no access to the observations and not the minimum possible score.

View post 2

FAQ: Debugging the submissions

View post 0

Round 1 is open for submissions 🚀

Hello @jurgisp

This was a config issue on our end. The issue is fixed and we re-queued your submission.

View post 1

The error of have not qualified for this round.?

Hello all,

The issue is fixed, and you should be able to make submissions now. We requeued the failed submissions. In case you made multiple submissions to get the submission to work and want to cancel a submission (so that they don’t count towards the submission quota), please share the submission IDs that you want us to cancel.

View post 1

The error of have not qualified for this round.?

Hello @RDL_lms

We do not have any minimum requirements to participate in round 1. This seems to have happened due to a config issue on our end. Can you try submitting again?

View post 0

Round 1 is open for submissions 🚀

Hello @Paseul

You can put any value in env_name. This will be overridden by relavant value during evaluation.

View post 0

Round 1 is open for submissions 🚀

Hello all!

Thank you for your participation and enthusiasm during the warm-up round! We are accepting submissions for round 1.

Changes for Round 1

Hardware available for evaluations

The evaluations will run on the following hardware:

Evaluation configuration

The configuration used during evaluations is available at FAQ: Round 1 evaluations configuration.

Environments

This round will run on three public (coinrun, bigfish, miner) and one private environment.

Scoring

The final score will be an average of mean normalized rewards for public environments and the private environment

Score = \frac{1}{6}*R_{coinrun} + \frac{1}{6}*R_{bigfish} + \frac{1}{6}*R_{miner} + \frac{1}{2}*R_{privateEnv}

R_{env} = Mean normalized reward for env.

Prizes

We are super excited to share that AWS is the official sponsor of this competition. Apart from sponsoring all the compute for this competition, AWS has been generous to extend the following prizes:

• $10,000 in AWS credits for the TOP 50 participants of the warm-up round. (you will be receiving a mail shortly!)
• TOP 3 teams of the final round will each get $1000 (cash) and $3000 in AWS credits!

For the next round

Because significant computation is required to train and evaluate agents in the final round, only the top 50 submissions from Round 1 will be eligible to submit solutions for Round 2.

View post 6

FAQ: Round 1 evaluations configuration

Helo @jurgisp

The above configuration will be used during both training and rollouts.

View post 1

Help..submission failed --- "Unknown grader_id"

Hello @dynmi

Nothing is wrong with your submission. The warm-up round is closed for new submissions and hence the error. We will send out an email to the participants when the next round starts, and you should be able to submit then.

View post 0

Same marks on the testing video

Hello @victor_le

What values are allowed to be modified? The values listed in env_config in impala-baseline.yaml ?

We will set all the env config params (except rand_seed) to the default values during evaluations, i.e. none of them is supposed to be changed by the participants.

Is the grader open-source?

We can’t opensource the grader at the moment. But you should be able to replicate the evaluation setup with the values mentioned in FAQ: Round 1 evaluations configuration.

View post 1

FAQ: Round 1 evaluations configuration

The following values will be set during the evaluations. Any changes that you make to these parameters will be dropped and replaced with the default values during the evaluations.

stop:
  timesteps_total: 8000000
  time_total_s: 7200

checkpoint_freq: 25
checkpoint_at_end: True

env_config:
  env_name: <accordingly>
  num_levels: 0
  start_level: 0
  paint_vel_info: False
  use_generated_assets: False
  distribution_mode: easy
  center_agent: True
  use_sequential_levels: False
  use_backgrounds: True
  restrict_themes: False
  use_monochrome_assets: False

# We use this to generate the videos during training
evaluation_interval: 25
evaluation_num_workers: 1
evaluation_num_episodes: 3
evaluation_config:
  num_envs_per_worker: 1
  env_config:
    render_mode: rgb_array

During the rollouts, we will also pass a rand_seed to the procgen env.

View post 5

Warmup round extension

Hello @maraoz @edan_meyer

Apologies for this. We got the compute sponsorship approved, and procurement is still in progress. We are working with them and will be launching next round as soon as it is complete.

Once the round is launched, we will send out an email to all the participants of the competition.

View post 4

Resource limits for submissions

Hello @maraoz

The evaluations right now are running with 16 vCPUs, 56 GB RAM.

View post 1

FAQ: Debugging the submissions

Hello @maraoz

Yes, it should be num_gpus + (num_workers+1)*num_gpus_per_worker <= 1. Thanks for pointing it out. Updated the post with the right variable name

View post 0

Rllib custom env

Hello @Mckiev

I’m not sure if I understood that right. We will use the same env for training and rollouts. The requirements from our side are

• The base env you use should be the env returned by ProcgenEnvWrapper rather than the one you get from gym.make.
• The wrapper that you use should extend gym.Wrapper class (in case you are writing one on your own).

Right way to use custom wrappers:

registry.register_env(
    "my_custom_env",
    lambda config: MyWrapper(ProcgenEnvWrapper(config))
)

Wrong way to use custom wrappers:

registry.register_env(
    "my_custom_env",
    lambda config: MyWrapper(gym.make("procgen:procgen-coinrun-v0", **config))
)

During the evaluation (both training and rollouts), we will use the env with your custom wrapper (if any).

If you have a more complex use case (like you need to pass some custom env variables but they should not be passed to the base env),

def create_my_custom_env(config):
    my_var = config.pop("my_var")
    env = ProcgenEnvWrapper(config)
    env = MyWrapper(env, my_var)
    return env

registry.register_env(
    "my_custom_env", create_my_custom_env
)

I hope this covers what you wanted to know.

View post 1

Rllib custom env

Hello @bob_wei

Yes, the base env should be the ProcgenEnvWrapper provided in the starter kit. You can use any gym wrapper on top of this. If you use the env from gym.make instead of ProcgenEnvWrapper, the rollouts will fail.

View post 0

Rllib custom env

Hello @bob_wei @Mckiev

We added support for using wrappers. Please give it a try. https://github.com/AIcrowd/neurips2020-procgen-starter-kit/tree/master/envs

View post 2

2 hours training time limit

Hello @tim_whitaker

The time_total_s: 7200 is already in place during evaluations. We also have a limit on max parallel evaluations, based on the resources available, to avoid long waits amid evaluations. We also have a hard timeout with 2 hours + buffer period to close the stuck evaluations. We will check if there are any issues with the scheduling time and adjust the buffer time accordingly.

I believe that you are referring to submission #70205. The GitLab updates are failing for that submission, but the evaluation is still in progress for that one. We will check why the updates on the GitLab issue page stopped.

View post 1

Same marks on the testing video

Hello @lars12llt @karolisram

Yes, these values are not supposed to be changed. We will override these values from the next grader update.

View post 0

Impala Pytorch Baseline Bug(s)

Hello @mtrazzi

You need to set "docker_build": true in your aicrowd.json file. Without this, we will use the default image and there will not be any pip installs from your requirements.txt.

View post 1

Is anyone experiencing the same warnings/errors?

Hello @mtrazzi

First of all, thanks for your effort in writing down these warnings/errors here. I’m sure that a lot of participants would have similar questions.

Dashboard crashes with error while attempting to bind on address ('::1', 8265, 0, 0): cannot assign requested address . I solved this by adding webui_host='127.0.0.1' in ray_init in train.py (cf. stackoverflow) on google colab , not sure i need to do the same for aicrowd submission (which would mean touching to train.py ).

You can ignore the dashboard error. You see that error because the port on which the dashboard is trying to bind is not available for it.

ls: cannot access '/outputs/ray-results/procgen-ppo/*/' this seems to be in how variables are set in run.sh. Don’t know why they would want to access ray-results early on.

We run on the evaluations on preemptive instances, which means that the node on which the evaluation is running on can shut down any time. Before starting the training, we check for any existing checkpoints and resume from that point. I understand that this is causing some confusion. We will hide these outputs in the next grader update.

given NumPy array is not writeable ( solved by downgrading to torch 1.3.1 locally, but still unclear how to downgrade when submitting , cf. discussion)

You can update the requirements.txt or edit your Dockerfile accordingly. Please make sure that you set "docker_build": true in your aicrowd.json file. If this is not set to true, we will not trigger a docker build and will use a default image.

[Errno 2] No such file or directory: 'merged-videos/training.mp4' : seems to be on aicrowd side, but maybe we need to change how we log videos? see this example or this PR.

This is not related to the issue you were mentioning. We generate a few videos for every few iterations and upload them at regular time intervals during training. These videos are shown on the GitLab issue page as well as the submission dashboard. This error basically means that we tried uploading the video bu,t there was no video. This typically happens when throughput is very low or, the rendering doesn’t work. If you are able to see a video on aicrowd website and the dashboard, rendering the video is not the problem. It looks like we missed some error handling here (though it will not effect your evaluation in any way). We will fix this in the next grader update.

WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 67108864 bytes available. This may slow down performance! ... you may need to pass an argument with the flag '--shm-size' to 'docker run'. : this seems to be on aicrowd server side, but maybe we need to change the docker file or do clever things?

Yes, docker by default allocates very less shared memory. Typically, I do not expect this to have a drastic impact on performance. In terms of throughput, we were getting similar results on a physical machine and on the evaluation pipeline. But if you want us to increase this, please reach out to us and we will be glad to look into it.

View post 1

Impala Pytorch Baseline Bug(s)

Hello @bob_wei

Can you try using pytorch 1.3.x version?

View post 2

Rllib custom env

Hello @bob_wei

You can use a custom preprocessor for this.

View post 1

Selecting seeds during training

Hello @Leckofunny

This should be possible. We do support callbacks so approach 2 mentioned in definitely doable. For the first approach, did you try passing it as a custom algorithm? According to this line,

You should be able to pass the train function in the approach 1 as a custom algorithm. Can you give it a try? If that doesn’t work, you should be able to extend the existing PPOTrainer class as custom algorithm.

For the re-init part, can you try running env.close() followed by env.__init__() with parameters from the current env? I’m not sure if this is really the right way. I’ll get back in case I find a better solution.

View post 0

Training Error?

Hello @Paseul1

As mentioned in this comment, there was a problem with your interpreter line (shebang) in run.sh. If this line is not provided, the script is run with the default shell, which happens to be /bin/sh in the evaluation environment. It worked locally for you because your default shell might be bash, zsh, or equivalent. Please let me know if you are referring to a different submission.

View post 0

How to save rollout video / render?

Hello @mtrazzi @xiaocheng_tang

This example should work.

#!/usr/bin/env python

import gym
import gym.wrappers

env = gym.make("procgen:procgen-coinrun-v0", render_mode="rgb_array")
env.metadata["render.modes"] = ["human", "rgb_array"]
env = gym.wrappers.Monitor(env=env, directory="./videos", force=True)

episodes = 10
_ = env.reset()

done = False
while episodes > 0:
    _, _, done, _ = env.step(env.action_space.sample())
    if done:
        _ = env.reset()
        episodes -= 1

View post 3

How to save rollout video / render?

Hello @xiaocheng_tang

Do you see any warning that says something like render() returned None when you use the Monitor wrapper?

View post 0

How to save rollout video / render?

Hello @xiaocheng_tang

Can you try the same with gym3==0.3.2 and procgen==0.10.3?

View post 0

How to save rollout video / render?

Hello @mtrazzi

For procgen==0.10.x, you need to pass render_mode="rgb_array" as a config option to the environment for the videos.

PS: Note that passing render_mode="rgb_array" will have a performance impact and is not recommended to be used during training.

View post 1

How to save rollout video / render?

Hello @mtrazzi

Can you share the procgen version that you are using?

View post 0

Don't receive "graded"

Hello @liziniu

Your submission #69444 is still in evaluation and is not stuck. There are a lot of submissions and the hence most of them are in queue.

You can check the gitlab issue to know in what stage the evaluation is in. If you see something like “Preparing the cluster for you ” it means that the submission is waiting for the cluster resources to be available. At this point we can evaluate 4 submissions in parallel. You can also check the last updated timestamp on the gitlab issue.

image697×192 20.6 KB

View post 0

FAQ: Regarding rllib based approach for submissions

Hello @alexander_ermolov

We explained how the random agent code provided in the starter kit works in FAQ: Implementing a custom random agent

View post 0

FAQ: Regarding rllib based approach for submissions

Hello @lars12llt

The CustomRandomAgent class is registered as custom/CustomRandomAgent. So you need to use custom/CustomRandomAgent or update it’s name.

View post 1

FAQ: Implementing a custom random agent

In this post, we will try to demystify the random agent code provided in the starter kit. The most common question we heard from the participants was the difference between the models and algorithms directories.

Our idea was to put all the models inside the models’ directory. These are the RL policy networks that you will build using convolutional layers, dense layers, and so on. The algorithms that govern the optimization of the RL policies go into the algorithms directory.

Implementing a custom random agent

Now, we will see what the code in the CustomRandomAgent class does [refer]. Our random agent does not learn anything. It returns random actions and collects rewards. First, we need to create the environment.

Now that we have the env ready, let’s randomly sample actions and run it till the episode finishes.

When training the agent, we want to run this in a loop for rollouts_per_iteration number of times.

Now, let’s collect the rewards and return a dict containing training stats for a given iteration.

That’s it! You can find the complete code for this agent at https://github.com/AIcrowd/neurips2020-procgen-starter-kit/blob/f8b943bffaf2c86a4c78043fcb0f1253ab1b42ba/algorithms/custom_random_agent/custom_random_agent.py

Now, how does rllib know that there is this custom agent that we want to use? We have a custom registry for this. First, list your python class as a custom algorithm here,

This will register the random agent class with the name custom/CustomRandomAgent. Now we need to add this to our experiments YAML file.

procgen-example:
  env: "procgen_env_wrapper"
  run: "custom/CustomRandomAgent"

So how does rllib know that it has to use the algorithms? We register all the custom algorithms and models in the train.py file!

Other resources on algorithms

Implement a custom loss function while leaving everything else as is.

https://docs.ray.io/en/master/rllib-concepts.html

More examples

View post 3

FAQ: Debugging the submissions

How to know why my submission failed?

When things go south, we try our best to provide you with the most relevant message on the GitLab issue page. They look somewhat like these.

Well, “Training failed” is not of much use. No worries! We got you covered. You can click on the Dashboard link on the issues page.

Scroll a bit down. You should find a pane that displays the logs emitted by your training code.

logs in the dashboard1271×568 121 KB

Note: We do not provide the logs for rollouts on the dashboard to avoid data leaks. We will provide the relevant logs for the rollouts upon tagging us.

Common errors faced

Error says I’m requesting x/1.0 GPUs where x > 1

Make sure that num_gpus + (num_workers+1)*num_gpus_per_worker is always <= 1.

My submission often times out

A low throughput can be due to various reasons. Checking the following parameters is a good starting point,

Adjust the rollout workers and number of gym environments each worker should sample from. These values should be good initials.

num_workers: 6
num_envs_per_worker: 20

Make sure your training worker uses GPU.

num_gpus: 0.6

Make sure that your rollout workers use a GPU.

num_gpus_per_worker: 0.05

Note: rllib does not allocate the specified amount of GPU memory to the workers. For example, having num_gpus: 0.5 does not mean that half of the GPU memory is allocated to the training process. These parameters are very useful in a scenario where one has multiple GPUs. These parameters will be used by rllib to figure out which worker goes on which GPU. Since the evaluations run on a single GPU, setting num_gpus and num_gpus_per_worker to a nominal non-zero positive value should suffice. For more information on the precise tuning of these parameters, refer.

Figuring out the right values for num_gpus and num_gpus_per_worker

You can run nvidia-smi on your machine when you start the training locally. It should report how much memory each of the workers is taking on your GPU. You can expect them to take more or less the same amount of memory during the evaluation. For example, say you are using num_workers: 6 for traning locally. The output for nvidia-smi should look similar to this.

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.64.00    Driver Version: 440.64.00    CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 208...  Off  | 00000000:03:00.0 Off |                  N/A |
| 43%   36C    P2   205W / 250W |   6648MiB / 11178MiB |     90%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0     16802      C   ray::RolloutWorker.sample()                819MiB   |
|    0     16803      C   ray::PPO.train()                           5010MiB  |
|    0     16808      C   ray::RolloutWorker.sample()                819MiB   |
|    0     16811      C   ray::RolloutWorker.sample()                819MiB   |
|    0     16813      C   ray::RolloutWorker.sample()                819MiB   |
|    0     16831      C   ray::RolloutWorker.sample()                819MiB   |
|    0     16834      C   ray::RolloutWorker.sample()                819MiB   |
+-----------------------------------------------------------------------------+

From this output, I know that a single rollout worker is taking around 819 MB and the trainer is taking around 5010 MB of GPU memory. The evaluations run on Tesla P100 which has 16 GB memory. So, I would set num_workers to 12. The GPU usage during the evaluation should roughly be

5010 MB + 819 MB *(12+1) = 15.3 GB

Note: The above values are dummy values. Please do not use these values when making a submission.

Run your code locally to avoid wasting your submission quota

We assume that you made necessary changes in run.sh.

Make sure that your training phase runs fine

./run.sh --train

Make sure that the rollouts work.

./run.sh --rollout

If you are using a "docker_build": true without modifying the dockerfile but to install py packages from requirements.txt,

• Create a virtual environment using conda / virtualenv / python3 -m venv
• Activate your new environment.
• Run pip instal -r requirements.txt.
• Run ./run.sh --train.
• Run ./run.sh --rollout.

In case you are using a completely new docker image, please build on top of the Dockerfile provided on the starter kit. You are free to choose a different base image, however, you need to make sure that all the packages that we initially were installing are still available. To avoid using failures in the docker build step, we recommend that you try running it before making a submission.

docker build .

This might take quite a while the frist time you run it. But it will be blazing fast from next time!

Not able to figure out what went wrong? Just tag @jyotish / @shivam on your issues page. We will help you!

View post 1

How can I rewrite my local algorithm as a custom algotithm in submission?

Hello @lars12llt

Welcome to the AIcrowd community!

1, I believe your question is how you can assign a GPU for the evaluation. All the evaluations will be run on a Tesla P100 (16 GB).

2, Ideally, we expect the model/network related code to stay in models directory while the training algorithm related code (like custom policy functions, etc.,) to go into algorithms directory. For the wrapper part, can you have a look at the custom preprocessors in rllib? https://docs.ray.io/en/master/rllib-models.html#custom-preprocessors

For the parallel sampling, you don’t need to do it, rllib does it for you. The num_workers parameter in the experiment config file (.yaml file) controls the number of rollout workers that are spawned. You can set the number of envs each worker needs to run using num_envs_per_worker.

3, Yes, we expect you to set the config file to set hyper-parameters. For why we want you to do it that way, please refer FAQ: Regarding rllib based approach for submissions

Some reference that might help:

• How custom algorithm (trainer) interacts with the rollout workers: https://docs.ray.io/en/master/rllib-training.html
• How models, preprocessors, envs interact: https://docs.ray.io/en/master/rllib-models.html#custom-preprocessors

View post 2

Submission/Gitlab Issue -> Git Tag

Hello @laxatives

You can view the submission details by clicking the “View” button of the respective submissions. The “Code” button takes you to the code that was submitted for a particular submission. You can view all the submission you made at https://www.aicrowd.com/challenges/neurips-2020-procgen-competition/submissions?my_submissions=true

You can compare two versions of your repository by visiting https://gitlab.aicrowd.com/username/repository/compare. In your case, this will be https://gitlab.aicrowd.com/laxatives/neurips-2020-procgen-starter-kit/compare

View post 0

Some question about vCPU, Help!

Hello @DRL_AGI

The evaluations happen on Google Cloud Platform. The evaluations run on an n1-standard-16 VM which has 16 vCPUs and 60 GB RAM [refer]. The vCPUs are based on Intel Skylake architecture and have a base clock speed of 2.0 GHz [refer].

View post 0

GPU utilization

Hello @shogoakiyama

Can you try setting

num_workers: 6 # Number of rollout workers to run
num_envs_per_worker: 20 # Number of envs to run per rollout worker
num_gpus: 0.6 # Fraction of GPU used by trainer
num_gpus_per_worker: 0.05 # Fraction of GPU used by rollout worker

Please make sure that num_gpus + num_gpus_per_worker*num_workers <= 1. Setting num_gpus to 0.5 doesn’t mean that half of the GPU memory is available to the trainer process. rllib doesn’t allocate GPUs but schedules the workers based on these values. They exist to make it easier to scale the training process to multiple GPUs. Since we use a single GPU during the evaluation, setting these values to some non-zero value should suffice.

View post 0

Unusually large tensor in the starter code

Hello @the_raven_chaser

The impala baseline provided in the starter kit takes close to 15.8 GB of GPU memory. As a starting point, you can try setting num_workers: 1 in the experiment YAML file and see if it works. You can also try running nvidia-smi to check how much memory is being utilized by the trainer and the rollout worker. Based on that, you can try increasing num_workers to a higher number.

View post 1

Multi-Task Challenge?

Hello @Leckofunny

Yes, the agent will be trained on unknown environments as well. The training and rollouts are done on the server-side. You only need to submit the code. You will be given access to the training logs once you make a submission.

View post 0

High average reward but low score

Hello @quang_tran

Something went wrong with the rollouts for two of your submissions. We are looking into it.

We are using these for R_min and R_max values

The rollouts will be run for 1000 episodes with at most 1000 steps per episode. You can check the configuration used during rollouts here, Several questions about the competition

View post 1

Experiment Yaml Spec

Hello @laxatives

Did you try using custom_options? https://github.com/AIcrowd/neurips2020-procgen-starter-kit/blob/master/experiments/procgen-starter-example.yaml#L158

Some resources that might be helpful,

Custom models using tensorflow: https://ray.readthedocs.io/en/stable/rllib-models.html#custom-models-tensorflow

You can find some example implementations at https://github.com/ray-project/ray/tree/master/rllib/examples

View post 0

Evaluation failed for custom environment

Hello @justin_yuan

The Dockerfile seems to be missing for this particular submission and hence the error. You can check the code that was submitted by clicking the “Code” button for the corresponding submission on https://www.aicrowd.com/challenges/neurips-2020-procgen-competition/submissions?my_submissions=true

View post 0

Several questions about the competition

Hello @tim_whitaker

Yes, any changes to train.py , rollout.py and envs/procgen_env_wrapper.py will be dropped during the evaluation. The custom algorithms (trainer class related ones) and the custom models will be used even during the rollouts. If the rollouts work locally without altering the provided rollout.py as expected, the same should work during evaluations as well.

View post 0

What are the evaluation environments?

Hello @quang_tran

The warm-up round will only use the coinrun environment.

View post 0

Number of levels in round 1?

Hello @quang_tran

This is probably what you were looking for? Several questions about the competition

View post 0

Time out Error, Help!

Hello @mj1

Can you recheck the parameters that you were using? You can check your submissions here, https://www.aicrowd.com/challenges/neurips-2020-procgen-competition/submissions?my_submissions=true

You can click on the “code” button to view the state of the code at the time of your submissions.

View post 0

Insufficient cluster resources to launch trial

Hello @CireNeikual

For why we want to use rllib, please refer FAQ: Regarding rllib based approach for submissions

Updating the requirements.txt won’t be enough. You also need to set "docker_build": true in your aicrowd.json file. More on using custom images can be found here, https://github.com/aicrowd/neurips2020-procgen-starter-kit#submission-environment-configuration

Please make sure to include the mlflow pip package in your custom image. We can’t post the evaluation updates on the gitlab issues page without this.

View post 0

Change experiement yaml file

Hello @comet

You can use your own wrapper for the training phase. But we will use the procgen_env_wrapper that was provided in the starter for the rollouts.

You can use your own experiment yaml file. You need to make sure to make necessary changes in run.sh (https://github.com/AIcrowd/neurips2020-procgen-starter-kit/blob/master/run.sh#L8) as well.

View post 0

Help..how to deal with Submission failed : iiid?

Hello @alexander_ermolov

Few submissions have failed yesterday due to a bug. We queued the effected submissions for re-evaluation.

View post 0

Submission available resources

Hello @iamhatesz

The evaluations are now running with 16 vCPUs.

View post 1

Impala Pytorch Baseline Bug(s)

Hello @gregory_eales,

The “fails but then continues” is because we re-evaluated your submission as it failed the first time due to an internal glitch.

Following is the throughput vs ray worker configuration on an evaluation node (with 1 P100 and 8 vCPUs) for impala baseline (tensorflow version).

View post 3

Help..how to deal with Submission failed : iiid?

Hello @breezeyuner, @laxatives

Thanks for reporting this. This was a bug on our end. We are restarting the evaluations for the effected submissions.

View post 0

Several questions about the competition

Hello @the_raven_chaser

You can change the environment configuration as you like and we will use the same during the training phase. During the rollouts, we will force the following configuration

{
    "num_levels": 0,
    "start_level": 0,
    "use_sequential_levels": false,
    "distribution_mode": "easy",
    "use_generated_assets": false
}

As long as your code uses rllib, things should work. You can add more environment variables or arguments to python train.py line in run.sh but we will still use the train.py wrapper that we provided to trigger the training. So any changes you make to train.py will be dropped.

This discussion should give you more context on why we want to enforce the use of a framework, FAQ: Regarding rllib based approach for submissions

All the best for the competition!

View post 0

Submission available resources

Hello @iamhatesz

Yes, we plan to extend it to 16 vCPUs in a few days.

View post 1

Submission available resources

Hello @iamhatesz

At the moment, the evaluations are running on an 8 vCPU , 1 GPU (P100) node. Of the 8 vCPUs, one is reserved for the evaluation worker.

View post 0

TypeError: cannot pickle 'property' object

Hello @gregory_eales, can you downgrade to python 3.7/3.6 and try again? There is a known issue with cloudpickle and python 3.8.

View post 4

Evaluation Limits

Hello @tim_whitaker

We saw the post and are discussing this with the organizers.

View post 0

Problem with rollouts on docker build

Hello @tim_whitaker

Is it possible to share a few details like

• How long did the rollouts take locally to complete?
• What hardware were the rollouts run on?
• For how many episodes and max_steps were the rollouts run for?

If you feel that this is something happening only during the evaluation, we can try to run your submission manually and see where things are going wrong.

Sorry for the late response.

View post 0

RE: Several questions about the competition

Hello @tim_whitaker

Yes, any changes to train.py, rollout.py and envs/procgen_env_wrapper.py will be dropped during the evaluation.

View post 0

Getting INTERNAL SERVER ERROR

Hello,

We are re-evaluating the failed submissions. They should be updated in a while.

View post 0

Submitting Solution: Push Error and No code with current Tag

@devsahu, Can you try running the following in your repository’s directory and let us know if it works? In case it fails again, can you post the error that you get?

git add --all
git commit -m "<Your commit message>"
./submission.sh <solution tag>

View post 0

Submitting Solution: Push Error and No code with current Tag

Hi @devsahu,

Sorry for the confusion. Can you run git log and git status commands while inside your repository’s directory and post their outputs?

View post 0

Submitting Solution: Push Error and No code with current Tag

Hi @devsahu

Can you post the outputs of git log and git status?

View post 0