anssi 232

Done is never set to True

There was a bug in MineRL 0.3.1 where episode timeouts were not set. Upgrading to 0.3.3 (or soon-to-be-shared 0.3.4) should fix the issue.

View post 1

Preprocessing rule

@william_guss @BrandonHoughton

I know you guys are busy updating MineRL, but could you also shine light on the questions above (in this specific context)?

View post 1

Preprocessing rule

Quoting W. Guss from Discord on a very similar question:

We want to make it ILLEGAL for participants to make solutions like this that USE game knowledge.
Beyond stacking actions, you could imagine people using “movement detectors” to reverse engineer the action obfuscation, and then hard-code sequences/options (similar to scripted crafting).

Even with a rule against “hard coding” like above, I could just give these extracted action sequences to a policy as options for it to execute, but crucially I am using knowledge about minecraft to produce those options, rather than learning them from data with a “blind eye” to those options existing based on the rules
I think data augmentation in the style of CURL or image flipping etc, doesn’t encode game knowledge, but still it’s hard to write down a rule like this precisely

If your solution wouldn’t work on other domains that you knew nothing about (minus hyper parameters), you’re probably doing something wrong. e.g. let’s say we randomly trained your solution on an RTS like Starcraft, if it’s not general enough to work at all in that setting, that’s bad and we don’t want people to develop solutions which overfit to minecraft .

So if you “extract” options by knowing that there is a fixed sequence of items in the game or whatever, that’s overfitting to minecraft and not the intention of the competition .

For all intents and purposes, the environment should just be a black-box to your algorithm with Pixel + 64 dimensional vector observations and 64 dimensional acton vector

Thats not to say, you can’t do techniques like “automatic option extraction”, it’s just that those technqieus should be generic

So if that means e.g. training an auto-encoder on all of the (state,action) pairs and then doing $k-means$ to get clusters of actions, and then training a neural network for each option. That’s good!

View post 0

Competition metric seems to favor sample efficiency over generalization gap

Following is from the point of view of the competition. I agree it might be difficult to disentangle benefits of the method in sample efficiency and generalization with this limit.

From practical point of view, it levels the playing ground (somewhat) when the amount of training data is restricted. In Unity Obstacle Tower challenge, for example, one team trained for billions of timesteps, which was simply out of the reach of competitors without access to +50-core, multi-gpu setup. They can still use that hardware for faster evaluation of ideas/models/hyperparams though.

This also encourages for ideas other than “just brute-force it” with compute. It is easier to try out ideas when you know the limit of how long you should train

View post 1

About the rule on pre-trained model

I would assume no pretrained models are allowed, and this is enforced by removing “large” files.

If so, it is too relaxed. Our models are well below 10MB, regardless of the network architecture, and even if models were larger you could prune them to 90% of the original size and still retain most of the performance.

View post 2

When will the results of round 1 be announced?

I do not have an answer for that, but organizers are posting clips of top agents on Discord, most recent from roughly 6h ago. Linking them here so non-Discordians can also see them:

View post 1

Partially rendered env in MineRLObtainDiamondDense-v0

My wild guess is this has something to do with rendering of chunks and/or world-generator generating chunks. Similar things happen if you travel too fast to one direction (you reach “edge of the world”, but it will appear eventually). Not sure why previously rendered chunks would disappear like that.

View post 0

Equip item failed

Sorry for double-posting, but just for clarity:

Tested on Ubuntu 18.04 (Python 3.6) and Windows 10 (Python 3.7) with MineRL 0.2.4:

• Create ObtainDiamond-v0 environment, control by manually creating actions
• Open chat (press “T”) and give some items /give @a wooden_pickaxe 1 and /give @a stone_pickaxe 1.
• Wooden pickaxe is equiped (auto-equip the first slot).
• Using equip="stone_pickaxe" or equip="air" does not work, nor do the integer versions (3 and 1, respectively)

View post 1

How to use furnace to "nearbySmelt" coal?

Tested this on Windows 10 one more time, still no luck :(.

Random thought: Perhaps it works but it creates charcoal, and since charcoal != coal it won’t show up in the inventory list? They are very distinct items in Minecraft, but I am not sure if Malmo or your code abstracts this one out.

View post 0

How to use furnace to "nearbySmelt" coal?

I tested it like this (Ubuntu 18.04, Python 3.6, MineRL 0.2.4):

• Create env and control agent by feeding user input
• Focus on game, press “T” and command yourself necessary items (/give @a furnace 1, \give @a log 64 and /give @a planks 64)
• Place furnace, face it and try to use nearbySmelt=2 (="coal"). Nothing should happen (no crafting/smelting).

View post 0

Repeated reward for logs and furnace

Tried this as well with no luck (tried “wooden_pickaxe”, “none”, “air” and “stone_pickaxe”)

View post 0

How to use furnace to "nearbySmelt" coal?

Ah my bad. Indeed you should be able to burn logs into charcoal in default Minecraft but indeed the action nearbySmelt[coal] does not do anything after quick testing. It is either bugged or refers to something else (remnant of traditional smelting action?).

This has also been discussed here: How is the "reward" on leaderboard page computed? (Edit: Oh, it was you who reported this ^^’)

View post 0

Equip item failed

+1 on the equip command not working (can not unequip items nor equip anything). In fact it did not work in version 0.2.3 for us either.

View post 0

How to use furnace to "nearbySmelt" coal?

MineRL uses Malmo’s functions to make such smelting easier, essentially turning it into crafting. You only need to have iron ore and coal in your inventory, and once you do nearbySmelt iron you should get an iron ingot immediately. This was discussed on some other thread here on same channels.

View post 0

Block become much small when looking from top

That depends on the field-of-view of the player. You seem to be playing with the default 70 FOV (the lower picture), where as the recordings seem to be done in maximum 110 FOV (upper picture). This is based on quick round in vanilla Minecraft.

View post 0

Main hand observations

I believe maxDamage refers to how damaged the tool can get. I.e. If damage = maxDamage, then tool is broken and can not be used (as per Minecraft mechanics).

As for the none and air type: I am puzzled by this as well. Does one of them of never happen (they mean the same thing), or does none mean the player is not able to hit anything (“doesn’t even have hands”)?

View post 2

Different results between training, debug and evaluation

Could it be you overfit to the 100 seeds we have on shared binary of OT environment? AFAIK the evaluation uses seeds outside these 100.

We had something similar: Training model further did not make results more reliable in evaluation.

View post 0

Why I was blocked from uploading new models for 3 days

I use Linux (Ubuntu 16.04), and I dunno it works for other platforms.

View post 1

Why I was blocked from uploading new models for 3 days

Try running the environment without docker with same environment variables and port argument as in the examples. This works for me when I test my submission image.

View post 0

Is the evaluation seed truly random?

As I have understood it, the seeds are fixed (e.g. here Juliani tests one of the seeds). The wording on main page about random five seeds probably refers to five randomly preselected seeds.

View post 0

UnityTimeOutException in evaluation

We only updated one of the model files after a successful submission. Worker ID is set to zero and environment filename to None.

Looks like that submission (29) did eventually manage to run through. Did you happen to re-queue the submission or did it work out on itself? Edit: Looks like you re-queued it.

Edit2: Looks like that submission (29) is now stuck and does not finish correctly ^^’

Edit3: Looks like we figured it out: When testing locally outside docker images, the OT game ended up stuck with empty screen (skybox) and our code also waited for the OT environment. We first had to launch our agent code and wait until it tries to connect to OT environment (“Start training by pressing the Play…”), after which we launched OT env and the evaluation started successfully. The solution for submission was to modify agent code to first create OT environment before everything else, including larger imports (TF, Torch, etc), and then proceed with creating/loading agents.

View post 1

UnityTimeOutException in evaluation

@mohanty: That’s odd o: . I have another debug run going on right now, and it seems to be stuck. So far 500 seconds waited and no luck: https://gitlab.aicrowd.com/Miffyli/obstacletower-2019/issues/29.

But I will keep trying every now and then, if it would eventually work out. Thanks for the help!

View post 0

UnityTimeOutException in evaluation

Looks like I am running into same issue as described above: Previously submitted code suddenly is not working, and debugging gives me the timeout error. timeout_wait=300 did not solve the issue, but I will try with longer values. Here is one of the failed submissions: https://gitlab.aicrowd.com/Miffyli/obstacletower-2019/issues/25 .

Edit: timeout_wait=900 did not help either, still getting same error :/. https://gitlab.aicrowd.com/Miffyli/obstacletower-2019/issues/26

View post 0

Submitting in debug mode sets leaderboard score to 0

Yes. Also note that debug runs are done with different seeds than final evaluation.

More information here: Announcement: Debug your submissions

View post 1

Architectures of Round 1 winners

Apart from using improved Dopamine Rainbow, here is a short list of nuggets of information other people have shared:

• unixpickle’s blog post on hierarchical RL: https://blog.aqnichol.com/2019/04/03/prierarchy-implicit-hierarchies/
• Ross Wightman’s different version of Rainbow (got better results than the Dopamine one): https://github.com/rwightman/obstacle-tower-pytorch-rainbow
• Joe Booth got level 10 with “pure model-free RL” on standard-ish consumer hardware: https://twitter.com/iAmVidyaGamer/status/1122325568949633024

View post 4

Vector Observation contents

Looking at how ObstacleTowerEnv handles this observation, the vector tells number of keys as one-hot encoding (or with 1s up till number of keys, e.g. two keys would be [1,1,1,0,0,0]). So [1,0,0,0,0,0] means zero keys.

View post 0

Vector Observation contents

Apologies if this sounds bit naive, but make sure you are importing the correct Python libraries. E.g. do you have obstacle_tower_env.py file in the directory from which you run the experiments?

You can find the imported file with the following:

In [1]: import obstacle_tower_env                                                                                                                           

In [2]: obstacle_tower_env.__file__                                                                                                                         
Out[2]: '/home/USERNAME/.local/lib/python3.5/site-packages/obstacle_tower_env.py'

View post 0

V2.0 performance drop

If it is any help, we do the reset like this:

config = {"total-floors": 25}
env.reset(config)

Granted, we have not tried changing configs mid-run.

On the topic of configs: Some of the variables did not seem to have an effect of game (visual-theme and allowed-rooms), but this is only based on very brief experimentation so might be just a brainfart in our end.

View post 0

V2.0 performance drop

I haven’t measured FPS of the v2.0 env yet, but noticed two things:

• v2.0 is prettier: It has glows and fancier key models/textures. Perhaps this affects the performance little bit?
• reset() takes much longer now with default config. Set total-floors reset parameter to 25 or lower to speed up reset considerably.

View post 1

Vector Observation contents

Judging by this function in obstacle_tower_env, the first six elements are for the number of keys (Perhaps one-hot? Not sure). Seventh is the time remaining and eight is the floor number, as in v1.3.

View post 0

Question about the “Round 1 of Unity’s Obstacle Tower Challenge” email

@TruthMaker

Tax stuff: Ah yes, sounds reasonable. Would be bit of a mess if one of the Round 2 top3 would not be able to get the prize (assuming they want it).

On 4b): All of our team members received the email asking for ID and signature on the rules, and team lead got email with different greeting (“Dear Team Leader…”). I imagine they need everyone to sign the rules.

View post 0

Question about the “Round 1 of Unity’s Obstacle Tower Challenge” email

I second the post by @ipv6 , especially the parts (1,3) and (4):

(1,3) GCP credit might be tax-able in our country, so indeed it’d be nice to move to Round 2 without the GCP credits (and without the tax form).
(4) We in our do not have address in our standard IDs, although with more time we might be able to get such IDs.

View post 0