About the submission content

Same Problem, we can wait for the official response…

View post 0

Bug in submission?

When I use the bash ./utility/docker_evaluation_locally.sh --verbose, the following information is printed, I don’t how to remove the \r from the shell script.

(base) ubuntu@ip-172-31-88-94:~/tmp/pycharm_project_22/competition_data$ ./utility/docker_evaluation_locally.sh --verbose
Note: Gathering environment variables from environ.sh
Building docker image, for skipping docker image build use “–no-build”
usage: aicrowd-repo2docker [-h] [–config CONFIG] [–json-logs]
[–image-name IMAGE_NAME] [–ref REF] [–debug]
[–no-build]
[–build-memory-limit BUILD_MEMORY_LIMIT]
[–no-run] [–publish PORTS] [–publish-all]
[–no-clean] [–push] [–volume VOLUMES]
[–user-id USER_ID] [–user-name USER_NAME]
[–env ENVIRONMENT] [–editable]
[–target-repo-dir TARGET_REPO_DIR]
[–appendix APPENDIX] [–subdir SUBDIR] [–version]
[–cache-from CACHE_FROM]
repo …
aicrowd-repo2docker: error: argument --image-name: ‘aicrowd/neurips2019-minerl-challenge\r:agent\r’ is not a valid docker image name. Image namemust start with an alphanumeric character andcan then use _ . or - in addition to alphanumeric.
To run your submission with nvidia drivers locally, use “–nvidia” with this script
–verbose

View post 0

Suggestions about the competition startkit

I think the competition startkit is not very clear, the upper bound of the submission time is 20, we don’t have enough chances to debug it.

1. Your team can write a baseline (DQN, PPO) using startkit.
2. Tell more details about the code format we should follow.

View post 0

Problems about the competition startkit

The following is all of the logs, it prints the following information, then it is blocked. :

root@94d903ce99a4:/home/user/competition# xvfb-run -s “-ac -screen 0 1280x1024x24” ./new/evaluation_locally.sh --verbose
Verifying (and downloading) MineRL dataset…
If you do not want to use the data:
run the local evaluation scripts with --no-data
If you want to use your existing download of the data:
make sure your MINERL_DATA_ROOT is set.

Data directory is data
Data verified! A+!
Not starting broadcast server for localhost.
NS running on localhost:9090 (127.0.0.1)
Warning: HMAC key not set. Anyone can connect to this server!
URI = PYRO:Pyro.NameServer@localhost:9090
WARNING:ray.rllib.utils.compression:lz4 not available, disabling sample compression. This will significantly impact RLlib performance. To install lz4, run pip install lz4.
2019-09-02 13:02:43,092 WARNING worker.py:1337 – WARNING: Not updating worker name since setproctitle is not installed. Install this with pip install setproctitle (or ray[debug]) to enable monitoring of worker processes.
2019-09-02 13:02:43,093 INFO node.py:498 – Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-09-02_13-02-43_092903_72280/logs.
2019-09-02 13:02:43,210 WARNING services.py:763 – Redis failed to start, retrying now.
2019-09-02 13:02:43,329 INFO services.py:409 – Waiting for redis server at 127.0.0.1:23248 to respond…
2019-09-02 13:02:43,448 INFO services.py:409 – Waiting for redis server at 127.0.0.1:12994 to respond…
2019-09-02 13:02:43,451 INFO services.py:806 – Starting Redis shard with 10.0 GB max memory.
2019-09-02 13:02:43,477 INFO node.py:512 – Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-09-02_13-02-43_092903_72280/logs.
2019-09-02 13:02:43,477 WARNING services.py:1298 – Warning: Capping object memory store to 20.0GB. To increase this further, specify object_store_memory when calling ray.init() or ray start.
2019-09-02 13:02:43,477 WARNING services.py:1323 – WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 67108864 bytes available. This may slow down performance! You may be able to free up space by deleting files in /dev/shm or terminating any running plasma_store_server processes. If you are inside a Docker container, you may need to pass an argument with the flag ‘–shm-size’ to ‘docker run’.
2019-09-02 13:02:43,478 INFO services.py:1446 – Starting the Plasma object store with 20.0 GB memory using /tmp.
2019-09-02 13:02:43,654 WARNING logger.py:139 – Couldn’t import TensorFlow - disabling TensorBoard logging.
2019-09-02 13:02:43,657 WARNING logger.py:233 – Could not instantiate <class ‘ray.tune.logger.TFLogger’> - skipping.
WARNING:ray.rllib.utils.compression:lz4 not available, disabling sample compression. This will significantly impact RLlib performance. To install lz4, run pip install lz4.
INFO:minerl.env.malmo.instance.84a92a:Starting Minecraft process: [’/tmp/tmp8___bykl/Minecraft/launchClient.sh’, ‘-port’, ‘9081’, ‘-env’, ‘-runDir’, ‘/tmp/tmp8___bykl/Minecraft/run’]
INFO:minerl.env.malmo.instance.84a92a:Starting process watcher for process 77277 @ localhost:9081
Exception in thread Thread-1:
Traceback (most recent call last):
File “/root/anaconda3/lib/python3.6/threading.py”, line 916, in _bootstrap_inner
self.run()
File “/root/anaconda3/lib/python3.6/threading.py”, line 864, in run
self._target(*self._args, **self._kwargs)
File “/root/anaconda3/lib/python3.6/site-packages/minerl/env/malmo.py”, line 896, in keep_alive_pyro
InstanceManager.add_keep_alive(os.getpid(), callback)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 275, in getattr
self._pyroGetMetadata()
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 615, in _pyroGetMetadata
self.__pyroCreateConnection()
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 588, in __pyroCreateConnection
uri = _resolve(self._pyroUri, self._pyroHmacKey)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 1911, in _resolve
return nameserver.lookup(uri.object)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 185, in call
return self.__send(self.__name, args, kwargs)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 476, in _pyroInvoke
raise data # if you see this in your traceback, you should probably inspect the remote traceback as well
Pyro4.errors.NamingError: unknown name: minerl.instance_manager

Traceback (most recent call last):
File “run.py”, line 2, in
import train
File “/home/user/competition/train.py”, line 14, in
from envs import diamond_env_creator
File “/home/user/competition/envs.py”, line 10, in
from hack import minerl
File “/home/user/competition/hack.py”, line 74, in
minerl.env.malmo.InstanceManager.get_instance = get_instance
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 291, in setattr
self._pyroGetMetadata()
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 615, in _pyroGetMetadata
self.__pyroCreateConnection()
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 588, in __pyroCreateConnection
uri = _resolve(self._pyroUri, self._pyroHmacKey)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 1911, in _resolve
return nameserver.lookup(uri.object)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 185, in call
return self.__send(self.__name, args, kwargs)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 476, in _pyroInvoke
raise data # if you see this in your traceback, you should probably inspect the remote traceback as well
Pyro4.errors.NamingError: unknown name: minerl.instance_manager
±-- This exception occured remotely (Pyro) - Remote traceback:
| Traceback (most recent call last):
| File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/naming.py”, line 91, in lookup
| uri, metadata = self.storage[name]
| KeyError: ‘minerl.instance_manager’
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 1421, in handleRequest
| data = method(*vargs, **kwargs) # this is the actual method call to the Pyro object
| File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/naming.py”, line 98, in lookup
| raise NamingError("unknown name: " + name)
| Pyro4.errors.NamingError: unknown name: minerl.instance_manager
±-- End of remote traceback
./new/evaluation_locally.sh: line 43: kill: (72357) - No such process
root@94d903ce99a4:/home/user/competition# INFO:minerl.env.malmo:Minecraft process psutil.Process(pid=77279, status=‘terminated’) terminated with exit code None
INFO:minerl.env.malmo:Minecraft process psutil.Process(pid=77277, status=‘terminated’) terminated with exit code None
^C
root@94d903ce99a4:/home/user/competition# ^C
root@94d903ce99a4:/home/user/competition# ps aux xvfb-run -s “-ac -screen 0 1280x1024x24” ./new/evaluation_locally.sh --verbose
error: unsupported option (BSD syntax)

Usage:
ps [options]

Try ‘ps --help <simple|list|output|threads|misc|all>’
or ‘ps --help <s|l|o|t|m|a>’
for additional help text.

For more details see ps(1).
root@94d903ce99a4:/home/user/competition# xvfb-run -s “-ac -screen 0 1280x1024x24” ./new/evaluation_locally.sh --verbose ps aux
Verifying (and downloading) MineRL dataset…
If you do not want to use the data:
run the local evaluation scripts with --no-data
If you want to use your existing download of the data:
make sure your MINERL_DATA_ROOT is set.

Data directory is data
Data verified! A+!
Not starting broadcast server for localhost.
NS running on localhost:9090 (127.0.0.1)
Warning: HMAC key not set. Anyone can connect to this server!
URI = PYRO:Pyro.NameServer@localhost:9090
WARNING:ray.rllib.utils.compression:lz4 not available, disabling sample compression. This will significantly impact RLlib performance. To install lz4, run pip install lz4.
2019-09-02 13:17:51,657 WARNING worker.py:1337 – WARNING: Not updating worker name since setproctitle is not installed. Install this with pip install setproctitle (or ray[debug]) to enable monitoring of worker processes.
2019-09-02 13:17:51,658 INFO node.py:498 – Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-09-02_13-17-51_658243_77480/logs.
2019-09-02 13:17:51,775 INFO services.py:409 – Waiting for redis server at 127.0.0.1:44695 to respond…
2019-09-02 13:17:51,893 INFO services.py:409 – Waiting for redis server at 127.0.0.1:18180 to respond…
2019-09-02 13:17:51,895 INFO services.py:806 – Starting Redis shard with 10.0 GB max memory.
2019-09-02 13:17:51,930 INFO node.py:512 – Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-09-02_13-17-51_658243_77480/logs.
2019-09-02 13:17:51,933 WARNING services.py:1298 – Warning: Capping object memory store to 20.0GB. To increase this further, specify object_store_memory when calling ray.init() or ray start.
2019-09-02 13:17:51,933 WARNING services.py:1323 – WARNING: The object store is using /tmp instead of /dev/shm because /dev/shm has only 67108864 bytes available. This may slow down performance! You may be able to free up space by deleting files in /dev/shm or terminating any running plasma_store_server processes. If you are inside a Docker container, you may need to pass an argument with the flag ‘–shm-size’ to ‘docker run’.
2019-09-02 13:17:51,933 INFO services.py:1446 – Starting the Plasma object store with 20.0 GB memory using /tmp.
2019-09-02 13:17:52,183 WARNING logger.py:139 – Couldn’t import TensorFlow - disabling TensorBoard logging.
2019-09-02 13:17:52,185 WARNING logger.py:233 – Could not instantiate <class ‘ray.tune.logger.TFLogger’> - skipping.
WARNING:ray.rllib.utils.compression:lz4 not available, disabling sample compression. This will significantly impact RLlib performance. To install lz4, run pip install lz4.
INFO:minerl.env.malmo.instance.b4f065:Starting Minecraft process: [’/tmp/tmp7trapicr/Minecraft/launchClient.sh’, ‘-port’, ‘9081’, ‘-env’, ‘-runDir’, ‘/tmp/tmp7trapicr/Minecraft/run’]
INFO:minerl.env.malmo.instance.b4f065:Starting process watcher for process 82476 @ localhost:9081
Exception in thread Thread-1:
Traceback (most recent call last):
File “/root/anaconda3/lib/python3.6/threading.py”, line 916, in _bootstrap_inner
self.run()
File “/root/anaconda3/lib/python3.6/threading.py”, line 864, in run
self._target(*self._args, **self._kwargs)
File “/root/anaconda3/lib/python3.6/site-packages/minerl/env/malmo.py”, line 896, in keep_alive_pyro
InstanceManager.add_keep_alive(os.getpid(), callback)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 275, in getattr
self._pyroGetMetadata()
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 615, in _pyroGetMetadata
self.__pyroCreateConnection()
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 588, in __pyroCreateConnection
uri = _resolve(self._pyroUri, self._pyroHmacKey)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 1911, in _resolve
return nameserver.lookup(uri.object)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 185, in call
return self.__send(self.__name, args, kwargs)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 476, in _pyroInvoke
raise data # if you see this in your traceback, you should probably inspect the remote traceback as well
Pyro4.errors.NamingError: unknown name: minerl.instance_manager

Traceback (most recent call last):
File “run.py”, line 2, in
import train
File “/home/user/competition/train.py”, line 14, in
from envs import diamond_env_creator
File “/home/user/competition/envs.py”, line 10, in
from hack import minerl
File “/home/user/competition/hack.py”, line 74, in
minerl.env.malmo.InstanceManager.get_instance = get_instance
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 291, in setattr
self._pyroGetMetadata()
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 615, in _pyroGetMetadata
self.__pyroCreateConnection()
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 588, in __pyroCreateConnection
uri = _resolve(self._pyroUri, self._pyroHmacKey)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 1911, in _resolve
return nameserver.lookup(uri.object)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 185, in call
return self.__send(self.__name, args, kwargs)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 476, in _pyroInvoke
raise data # if you see this in your traceback, you should probably inspect the remote traceback as well
Pyro4.errors.NamingError: unknown name: minerl.instance_manager
±-- This exception occured remotely (Pyro) - Remote traceback:
| Traceback (most recent call last):
| File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/naming.py”, line 91, in lookup
| uri, metadata = self.storage[name]
| KeyError: ‘minerl.instance_manager’
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 1421, in handleRequest
| data = method(*vargs, **kwargs) # this is the actual method call to the Pyro object
| File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/naming.py”, line 98, in lookup
| raise NamingError("unknown name: " + name)
| Pyro4.errors.NamingError: unknown name: minerl.instance_manager
±-- End of remote traceback
./new/evaluation_locally.sh: line 43: kill: (77557) - No such process
./new/evaluation_locally.sh: line 44: 77479 Terminated pyro4-ns
root@94d903ce99a4:/home/user/competition# INFO:minerl.env.malmo:Minecraft process psutil.Process(pid=82478, status=‘terminated’) terminated with exit code None
INFO:minerl.env.malmo:Minecraft process psutil.Process(pid=82476, status=‘terminated’) terminated with exit code None
@shivam

View post 0

Problems about the competition startkit

It can run in a docker envinment when I use xvfb-run -s "-ac -screen 0 1280x1024x24" python train.py ' But I get the following error when I use the script in competiton startkit repo, I use the bashxvfb-run -s “-ac -screen 0 1280x1024x24” ./utility/evaluate_locally.sh`, it will get the following error:

Traceback (most recent call last):
File “/root/anaconda3/lib/python3.6/threading.py”, line 916, in _bootstrap_inner
self.run()
File “/root/anaconda3/lib/python3.6/threading.py”, line 864, in run
self._target(*self._args, **self._kwargs)
File “/root/anaconda3/lib/python3.6/site-packages/minerl/env/malmo.py”, line 896, in keep_alive_pyro
InstanceManager.add_keep_alive(os.getpid(), callback)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 275, in getattr
self._pyroGetMetadata()
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 615, in _pyroGetMetadata
self.__pyroCreateConnection()
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 588, in __pyroCreateConnection
uri = _resolve(self._pyroUri, self._pyroHmacKey)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 1911, in _resolve
return nameserver.lookup(uri.object)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 185, in call
return self.__send(self.__name, args, kwargs)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 476, in _pyroInvoke
raise data # if you see this in your traceback, you should probably inspect the remote traceback as well
Pyro4.errors.NamingError: unknown name: minerl.instance_manager

Traceback (most recent call last):
File “run.py”, line 2, in
import train
File “/home/user/competition/train.py”, line 14, in
from envs import diamond_env_creator
File “/home/user/competition/envs.py”, line 10, in
from hack import minerl
File “/home/user/competition/hack.py”, line 74, in
minerl.env.malmo.InstanceManager.get_instance = get_instance
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 291, in setattr
self._pyroGetMetadata()
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 615, in _pyroGetMetadata
self.__pyroCreateConnection()
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 588, in __pyroCreateConnection
uri = _resolve(self._pyroUri, self._pyroHmacKey)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 1911, in _resolve
return nameserver.lookup(uri.object)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 185, in call
return self.__send(self.__name, args, kwargs)
File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 476, in _pyroInvoke
raise data # if you see this in your traceback, you should probably inspect the remote traceback as well
Pyro4.errors.NamingError: unknown name: minerl.instance_manager
±-- This exception occured remotely (Pyro) - Remote traceback:
| Traceback (most recent call last):
| File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/naming.py”, line 91, in lookup
| uri, metadata = self.storage[name]
| KeyError: ‘minerl.instance_manager’
|
| During handling of the above exception, another exception occurred:
|
| Traceback (most recent call last):
| File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/core.py”, line 1421, in handleRequest
| data = method(*vargs, **kwargs) # this is the actual method call to the Pyro object
| File “/root/anaconda3/lib/python3.6/site-packages/Pyro4/naming.py”, line 98, in lookup
| raise NamingError("unknown name: " + name)
| Pyro4.errors.NamingError: unknown name: minerl.instance_manager

View post 0

What should i do when submitting 20 times, can I change a account?

Submission often goes wrong, we have to submit many times to debug it.

View post 0