Loading

IJCAI 2022 - The Neural MMO Challenge

Neural MMO Environment Tutorial

A tutorial notebook on Neural MMO Environment, observation and actions

hanmo_chen

Tutorial on NMMO's observation and action space

This is a tutorial to show the observation space and action space of Neural MMO's environment.

More information is available at https://neuralmmo.github.io/.

Installation

See https://gitlab.aicrowd.com/neural-mmo/ijcai2022-nmmo-starter-kit

# Create a new conda env
conda create -n nmmo python=3.9
conda activate nmmo


# install ijcai competition setup and nmmo
pip install git+http://gitlab.aicrowd.com/henryz/ijcai2022nmmo.git

After installation, we can establish a demo environment to show how NMMO works.

Environment Initialization

In [1]:
!pip install -q --ignore-requires-python openskill==0.2.0a0
!pip install -q git+http://gitlab.aicrowd.com/henryz/ijcai2022nmmo.git
Collecting openskill==0.2.0a0
  Downloading openskill-0.2.0a0-py3-none-any.whl (41 kB)
     |████████████████████████████████| 41 kB 19 kB/s 
Collecting scipy<2.0.0,>=1.7.3
  Downloading scipy-1.8.0.tar.gz (38.3 MB)
     |████████████████████████████████| 38.3 MB 1.3 MB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
WARNING: Discarding https://files.pythonhosted.org/packages/b4/a2/4faa34bf0cdbefd5c706625f1234987795f368eb4e97bde9d6f46860843e/scipy-1.8.0.tar.gz#sha256=31d4f2d6b724bc9a98e527b5849b8a7e589bf1ea630c33aa563eda912c9ff0bd (from https://pypi.org/simple/scipy/) (requires-python:>=3.8,<3.11). Command errored out with exit status 1: /usr/bin/python3 /usr/local/lib/python3.7/dist-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmplmkg02pe Check the logs for full command output.
  Downloading scipy-1.7.3-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (38.1 MB)
     |████████████████████████████████| 38.1 MB 280 kB/s 
Requirement already satisfied: numpy<1.23.0,>=1.16.5 in /usr/local/lib/python3.7/dist-packages (from scipy<2.0.0,>=1.7.3->openskill==0.2.0a0) (1.21.5)
Installing collected packages: scipy, openskill
  Attempting uninstall: scipy
    Found existing installation: scipy 1.4.1
    Uninstalling scipy-1.4.1:
      Successfully uninstalled scipy-1.4.1
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.
Successfully installed openskill-0.2.0a0 scipy-1.7.3
     |████████████████████████████████| 266 kB 4.9 MB/s 
     |████████████████████████████████| 58 kB 4.8 MB/s 
     |████████████████████████████████| 50 kB 4.7 MB/s 
     |████████████████████████████████| 87 kB 5.8 MB/s 
     |████████████████████████████████| 389 kB 38.6 MB/s 
     |████████████████████████████████| 3.1 MB 46.1 MB/s 
     |████████████████████████████████| 134 kB 60.6 MB/s 
     |████████████████████████████████| 3.3 MB 36.2 MB/s 
     |████████████████████████████████| 75 kB 3.7 MB/s 
     |████████████████████████████████| 1.2 MB 55.4 MB/s 
     |████████████████████████████████| 756 kB 67.9 MB/s 
     |████████████████████████████████| 99 kB 8.1 MB/s 
     |████████████████████████████████| 297 kB 45.8 MB/s 
     |████████████████████████████████| 3.6 MB 37.5 MB/s 
     |████████████████████████████████| 251 kB 57.6 MB/s 
     |████████████████████████████████| 74 kB 3.1 MB/s 
     |████████████████████████████████| 51 kB 473 kB/s 
     |████████████████████████████████| 626 kB 40.4 MB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
  Building wheel for ijcai2022nmmo (setup.py) ... done
  Building wheel for nmmo (setup.py) ... done
  Building wheel for fire (setup.py) ... done
  Building wheel for setproctitle (setup.py) ... done
  Building wheel for Twisted (setup.py) ... done
  Building wheel for vec-noise (setup.py) ... done
  Building wheel for pettingzoo (setup.py) ... done
  Building wheel for gym (PEP 517) ... done
  Building wheel for py-cpuinfo (setup.py) ... done
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.
In [2]:
import nmmo
from ijcai2022nmmo import CompetitionConfig

env = nmmo.Env(CompetitionConfig())
  2%|▎         | 1/40 [00:00<00:04,  8.72it/s]
Generating 40 maps
100%|██████████| 40/40 [00:03<00:00, 11.95it/s]

The env setting for this competition:

  • Map of 128 * 128 grids
  • 128 players (16 teams * 8 players/team)
  • All players spawn concurrently when game starts

If you want to see the detailed configuration:

In [3]:
def printConfig(config):
    for attr in dir(config):
        if not attr.startswith('__'):
            print('{}: {}'.format(attr,getattr(config,attr)))
            
printConfig(env.config)
AGENTS: [<class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>]
AGENT_LOADER: <class 'nmmo.core.config.TeamLoader'>
BASE_HEALTH: 10
COMBAT_DEFENSE_WEIGHT: 0.3
COMBAT_DICE_SIDES: 20
COMBAT_FREEZE_TIME: 3
COMBAT_MAGE_REACH: 4
COMBAT_MELEE_REACH: 1
COMBAT_RANGE_REACH: 3
Combat: True
EMULATE_CONST_HORIZON: False
EMULATE_CONST_NENT: False
EMULATE_FLAT_ATN: False
EMULATE_FLAT_OBS: False
FORCE_MAP_GENERATION: False
GENERATE_MAP_PREVIEWS: False
HORIZON: 1024
MAP_GENERATOR: <class 'nmmo.core.terrain.MapGenerator'>
MAP_PREVIEW_DOWNSCALE: 16
NENT: 128
NMAPS: 40
NMOB: 128
NPC: True
NPC_LEVEL_MAX: 30
NPC_LEVEL_MIN: 1
NPC_LEVEL_SPREAD: 5
NPC_SPAWN_AGGRESSIVE: 0.8
NPC_SPAWN_ATTEMPTS: 25
NPC_SPAWN_NEUTRAL: 0.5
NPC_SPAWN_PASSIVE: 0.0
NPOP: 16
NSTIM: 7
NTILE: 6
N_AGENT_OBS: 100
PATH_CWD: /content
PATH_MAPS: maps
PATH_MAP_SUFFIX: map{}/map.npy
PATH_RESOURCE: /usr/local/lib/python3.7/dist-packages/nmmo/resource
PATH_ROOT: /usr/local/lib/python3.7/dist-packages/nmmo
PATH_TILE: /usr/local/lib/python3.7/dist-packages/nmmo/resource/{}.png
PLAYER_SPAWN_ATTEMPTS: 2
PROGRESSION_BASE_RESOURCE: 10
PROGRESSION_BASE_XP_SCALE: 10
PROGRESSION_COMBAT_XP_SCALE: 4
PROGRESSION_CONSTITUTION_XP_SCALE: 2
Progression: True
RENDER: False
RESOURCE_BASE_RESOURCE: 10
RESOURCE_FOREST_CAPACITY: 1
RESOURCE_FOREST_RESPAWN: 0.025
RESOURCE_HARVEST_RESTORE_FRACTION: 1.0
RESOURCE_HEALTH_REGEN_THRESHOLD: 0.5
RESOURCE_HEALTH_RESTORE_FRACTION: 0.1
Resource: True
SAVE_REPLAY: False
SPAWN: <bound method Config.SPAWN_CONCURRENT of <ijcai2022nmmo.config.CompetitionConfig object at 0x7f1ea4406150>>
SPAWN_CONCURRENT: <bound method Config.SPAWN_CONCURRENT of <ijcai2022nmmo.config.CompetitionConfig object at 0x7f1ea4406150>>
SPAWN_CONTINUOUS: <bound method Config.SPAWN_CONTINUOUS of <ijcai2022nmmo.config.CompetitionConfig object at 0x7f1ea4406150>>
TASKS: [Task(condition=<function player_kills at 0x7f1e8dbb1170>, target=1, reward=4.0), Task(condition=<function player_kills at 0x7f1e8dbb1170>, target=3, reward=6.0), Task(condition=<function player_kills at 0x7f1e8dbb1170>, target=6, reward=11.0), Task(condition=<function equipment at 0x7f1e8dbb10e0>, target=1, reward=4.0), Task(condition=<function equipment at 0x7f1e8dbb10e0>, target=10, reward=6.0), Task(condition=<function equipment at 0x7f1e8dbb10e0>, target=20, reward=11.0), Task(condition=<function exploration at 0x7f1e8dbb1560>, target=32, reward=4.0), Task(condition=<function exploration at 0x7f1e8dbb1560>, target=64, reward=6.0), Task(condition=<function exploration at 0x7f1e8dbb1560>, target=127, reward=11.0), Task(condition=<function foraging at 0x7f1e8dbb15f0>, target=20, reward=4.0), Task(condition=<function foraging at 0x7f1e8dbb15f0>, target=35, reward=6.0), Task(condition=<function foraging at 0x7f1e8dbb15f0>, target=50, reward=11.0)]
TEAM_SIZE: 8
TERRAIN_BORDER: 16
TERRAIN_CENTER: 128
TERRAIN_FLIP_SEED: False
TERRAIN_FOREST: 0.85
TERRAIN_FREQUENCY: -3
TERRAIN_FREQUENCY_OFFSET: 7
TERRAIN_GRASS: 0.7
TERRAIN_LAVA: 0.0
TERRAIN_LOG_INTERPOLATE_MAX: 0
TERRAIN_LOG_INTERPOLATE_MIN: -2
TERRAIN_SIZE: 160
TERRAIN_TILES_PER_OCTAVE: 8
TERRAIN_WATER: 0.3
WINDOW: 15
data: {'NPOP': 16, 'NENT': 128, 'SPAWN': <property object at 0x7f1e8db0cb30>, 'AGENT_LOADER': <class 'nmmo.core.config.TeamLoader'>, 'AGENTS': [<class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>, <class 'nmmo.core.agent.Agent'>], 'TASKS': [Task(condition=<function player_kills at 0x7f1e8dbb1170>, target=1, reward=4.0), Task(condition=<function player_kills at 0x7f1e8dbb1170>, target=3, reward=6.0), Task(condition=<function player_kills at 0x7f1e8dbb1170>, target=6, reward=11.0), Task(condition=<function equipment at 0x7f1e8dbb10e0>, target=1, reward=4.0), Task(condition=<function equipment at 0x7f1e8dbb10e0>, target=10, reward=6.0), Task(condition=<function equipment at 0x7f1e8dbb10e0>, target=20, reward=11.0), Task(condition=<function exploration at 0x7f1e8dbb1560>, target=32, reward=4.0), Task(condition=<function exploration at 0x7f1e8dbb1560>, target=64, reward=6.0), Task(condition=<function exploration at 0x7f1e8dbb1560>, target=127, reward=11.0), Task(condition=<function foraging at 0x7f1e8dbb15f0>, target=20, reward=4.0), Task(condition=<function foraging at 0x7f1e8dbb15f0>, target=35, reward=6.0), Task(condition=<function foraging at 0x7f1e8dbb15f0>, target=50, reward=11.0)], 'NMAPS': 40, 'PATH_MAPS': 'maps'}
game_system_enabled: <bound method Config.game_system_enabled of <ijcai2022nmmo.config.CompetitionConfig object at 0x7f1ea4406150>>
override: <bound method Template.override of <ijcai2022nmmo.config.CompetitionConfig object at 0x7f1ea4406150>>
print: <bound method Template.print of <ijcai2022nmmo.config.CompetitionConfig object at 0x7f1ea4406150>>
set: <bound method Template.set of <ijcai2022nmmo.config.CompetitionConfig object at 0x7f1ea4406150>>

Observation Space

You can get the observation space via env.observation_space(agent) but the output is really verbose.

In [4]:
env.observation_space(agent=1)
Out[4]:
Dict(Entity:Dict(Continuous:Box(-1048576.0, 1048576.0, (100, 13), float32), Discrete:Box(0, 4096, (100, 4), int32), N:Box(0, 100, (1,), int32)), Tile:Dict(Continuous:Box(-1048576.0, 1048576.0, (225, 4), float32), Discrete:Box(0, 4096, (225, 3), int32), N:Box(0, 225, (1,), int32)))

It is hard to understand, so the obs will be explained in detail in the following chapter.

In [5]:
obs = env.reset()
print(obs.keys())
/usr/local/lib/python3.7/dist-packages/nmmo/core/config.py:216: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  lows   = (left+np.zeros(per_side, dtype=np.int)).tolist()
dict_keys([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128])

The env returns obs in a dict and the dict keys are the agent's id from 1 to 128.

And let's take a look at the obs single agent receives:

In [6]:
obs[1]
Out[6]:
{'Entity': {'Continuous': array([[ 1.,  1.,  0., ..., 10., 10.,  0.],
         [ 1., 31.,  0., ..., 10., 10.,  0.],
         [ 1., 32.,  0., ..., 10., 10.,  0.],
         ...,
         [ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  0.],
         [ 0.,  0.,  0., ...,  0.,  0.,  0.]], dtype=float32),
  'Discrete': array([[  1,   5, 165, 199],
         [  1,   8, 159, 197],
         [  1,   8, 163, 197],
         [  1,   5, 165, 203],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0],
         [  0,   0,   0,   0]], dtype=int32),
  'N': array([4], dtype=int32)},
 'Tile': {'Continuous': array([[  0.,   0., 137.,  11.],
         [  0.,   0., 137.,  12.],
         [  0.,   0., 137.,  13.],
         [  0.,   0., 137.,  14.],
         [  0.,   0., 137.,  15.],
         [  0.,   2., 137.,  16.],
         [  0.,   2., 137.,  17.],
         [  0.,   1., 137.,  18.],
         [  0.,   2., 137.,  19.],
         [  0.,   2., 137.,  20.],
         [  0.,   2., 137.,  21.],
         [  0.,   2., 137.,  22.],
         [  0.,   2., 137.,  23.],
         [  0.,   1., 137.,  24.],
         [  0.,   1., 137.,  25.],
         [  0.,   0., 138.,  11.],
         [  0.,   0., 138.,  12.],
         [  0.,   0., 138.,  13.],
         [  0.,   0., 138.,  14.],
         [  0.,   0., 138.,  15.],
         [  1.,   2., 138.,  16.],
         [  0.,   4., 138.,  17.],
         [  0.,   2., 138.,  18.],
         [  0.,   2., 138.,  19.],
         [  0.,   2., 138.,  20.],
         [  0.,   2., 138.,  21.],
         [  0.,   2., 138.,  22.],
         [  0.,   2., 138.,  23.],
         [  0.,   1., 138.,  24.],
         [  0.,   1., 138.,  25.],
         [  0.,   0., 139.,  11.],
         [  0.,   0., 139.,  12.],
         [  0.,   0., 139.,  13.],
         [  0.,   0., 139.,  14.],
         [  0.,   0., 139.,  15.],
         [  0.,   2., 139.,  16.],
         [  0.,   4., 139.,  17.],
         [  0.,   4., 139.,  18.],
         [  0.,   2., 139.,  19.],
         [  0.,   1., 139.,  20.],
         [  0.,   1., 139.,  21.],
         [  0.,   1., 139.,  22.],
         [  0.,   1., 139.,  23.],
         [  0.,   1., 139.,  24.],
         [  0.,   1., 139.,  25.],
         [  0.,   0., 140.,  11.],
         [  0.,   0., 140.,  12.],
         [  0.,   0., 140.,  13.],
         [  0.,   0., 140.,  14.],
         [  0.,   0., 140.,  15.],
         [  0.,   2., 140.,  16.],
         [  0.,   4., 140.,  17.],
         [  0.,   4., 140.,  18.],
         [  0.,   2., 140.,  19.],
         [  0.,   1., 140.,  20.],
         [  0.,   1., 140.,  21.],
         [  0.,   1., 140.,  22.],
         [  0.,   1., 140.,  23.],
         [  0.,   1., 140.,  24.],
         [  0.,   1., 140.,  25.],
         [  0.,   0., 141.,  11.],
         [  0.,   0., 141.,  12.],
         [  0.,   0., 141.,  13.],
         [  0.,   0., 141.,  14.],
         [  0.,   0., 141.,  15.],
         [  0.,   2., 141.,  16.],
         [  0.,   4., 141.,  17.],
         [  0.,   4., 141.,  18.],
         [  0.,   2., 141.,  19.],
         [  0.,   2., 141.,  20.],
         [  0.,   1., 141.,  21.],
         [  0.,   1., 141.,  22.],
         [  0.,   1., 141.,  23.],
         [  0.,   1., 141.,  24.],
         [  0.,   1., 141.,  25.],
         [  0.,   0., 142.,  11.],
         [  0.,   0., 142.,  12.],
         [  0.,   0., 142.,  13.],
         [  0.,   0., 142.,  14.],
         [  0.,   0., 142.,  15.],
         [  1.,   2., 142.,  16.],
         [  0.,   4., 142.,  17.],
         [  0.,   5., 142.,  18.],
         [  0.,   5., 142.,  19.],
         [  0.,   5., 142.,  20.],
         [  0.,   4., 142.,  21.],
         [  0.,   2., 142.,  22.],
         [  0.,   1., 142.,  23.],
         [  0.,   1., 142.,  24.],
         [  0.,   1., 142.,  25.],
         [  0.,   0., 143.,  11.],
         [  0.,   0., 143.,  12.],
         [  0.,   0., 143.,  13.],
         [  0.,   0., 143.,  14.],
         [  0.,   0., 143.,  15.],
         [  0.,   2., 143.,  16.],
         [  0.,   4., 143.,  17.],
         [  0.,   4., 143.,  18.],
         [  0.,   4., 143.,  19.],
         [  0.,   4., 143.,  20.],
         [  0.,   4., 143.,  21.],
         [  0.,   4., 143.,  22.],
         [  0.,   2., 143.,  23.],
         [  0.,   4., 143.,  24.],
         [  0.,   4., 143.,  25.],
         [  0.,   0., 144.,  11.],
         [  0.,   0., 144.,  12.],
         [  0.,   0., 144.,  13.],
         [  0.,   0., 144.,  14.],
         [  0.,   0., 144.,  15.],
         [  0.,   2., 144.,  16.],
         [  0.,   2., 144.,  17.],
         [  1.,   2., 144.,  18.],
         [  0.,   2., 144.,  19.],
         [  0.,   2., 144.,  20.],
         [  0.,   2., 144.,  21.],
         [  1.,   2., 144.,  22.],
         [  0.,   2., 144.,  23.],
         [  0.,   2., 144.,  24.],
         [  0.,   2., 144.,  25.],
         [  0.,   0., 145.,  11.],
         [  0.,   0., 145.,  12.],
         [  0.,   0., 145.,  13.],
         [  0.,   0., 145.,  14.],
         [  0.,   0., 145.,  15.],
         [  0.,   0., 145.,  16.],
         [  0.,   0., 145.,  17.],
         [  0.,   0., 145.,  18.],
         [  0.,   0., 145.,  19.],
         [  0.,   0., 145.,  20.],
         [  0.,   0., 145.,  21.],
         [  0.,   0., 145.,  22.],
         [  0.,   0., 145.,  23.],
         [  0.,   0., 145.,  24.],
         [  0.,   0., 145.,  25.],
         [  0.,   0., 146.,  11.],
         [  0.,   0., 146.,  12.],
         [  0.,   0., 146.,  13.],
         [  0.,   0., 146.,  14.],
         [  0.,   0., 146.,  15.],
         [  0.,   0., 146.,  16.],
         [  0.,   0., 146.,  17.],
         [  0.,   0., 146.,  18.],
         [  0.,   0., 146.,  19.],
         [  0.,   0., 146.,  20.],
         [  0.,   0., 146.,  21.],
         [  0.,   0., 146.,  22.],
         [  0.,   0., 146.,  23.],
         [  0.,   0., 146.,  24.],
         [  0.,   0., 146.,  25.],
         [  0.,   0., 147.,  11.],
         [  0.,   0., 147.,  12.],
         [  0.,   0., 147.,  13.],
         [  0.,   0., 147.,  14.],
         [  0.,   0., 147.,  15.],
         [  0.,   0., 147.,  16.],
         [  0.,   0., 147.,  17.],
         [  0.,   0., 147.,  18.],
         [  0.,   0., 147.,  19.],
         [  0.,   0., 147.,  20.],
         [  0.,   0., 147.,  21.],
         [  0.,   0., 147.,  22.],
         [  0.,   0., 147.,  23.],
         [  0.,   0., 147.,  24.],
         [  0.,   0., 147.,  25.],
         [  0.,   0., 148.,  11.],
         [  0.,   0., 148.,  12.],
         [  0.,   0., 148.,  13.],
         [  0.,   0., 148.,  14.],
         [  0.,   0., 148.,  15.],
         [  0.,   0., 148.,  16.],
         [  0.,   0., 148.,  17.],
         [  0.,   0., 148.,  18.],
         [  0.,   0., 148.,  19.],
         [  0.,   0., 148.,  20.],
         [  0.,   0., 148.,  21.],
         [  0.,   0., 148.,  22.],
         [  0.,   0., 148.,  23.],
         [  0.,   0., 148.,  24.],
         [  0.,   0., 148.,  25.],
         [  0.,   0., 149.,  11.],
         [  0.,   0., 149.,  12.],
         [  0.,   0., 149.,  13.],
         [  0.,   0., 149.,  14.],
         [  0.,   0., 149.,  15.],
         [  0.,   0., 149.,  16.],
         [  0.,   0., 149.,  17.],
         [  0.,   0., 149.,  18.],
         [  0.,   0., 149.,  19.],
         [  0.,   0., 149.,  20.],
         [  0.,   0., 149.,  21.],
         [  0.,   0., 149.,  22.],
         [  0.,   0., 149.,  23.],
         [  0.,   0., 149.,  24.],
         [  0.,   0., 149.,  25.],
         [  0.,   0., 150.,  11.],
         [  0.,   0., 150.,  12.],
         [  0.,   0., 150.,  13.],
         [  0.,   0., 150.,  14.],
         [  0.,   0., 150.,  15.],
         [  0.,   0., 150.,  16.],
         [  0.,   0., 150.,  17.],
         [  0.,   0., 150.,  18.],
         [  0.,   0., 150.,  19.],
         [  0.,   0., 150.,  20.],
         [  0.,   0., 150.,  21.],
         [  0.,   0., 150.,  22.],
         [  0.,   0., 150.,  23.],
         [  0.,   0., 150.,  24.],
         [  0.,   0., 150.,  25.],
         [  0.,   0., 151.,  11.],
         [  0.,   0., 151.,  12.],
         [  0.,   0., 151.,  13.],
         [  0.,   0., 151.,  14.],
         [  0.,   0., 151.,  15.],
         [  0.,   0., 151.,  16.],
         [  0.,   0., 151.,  17.],
         [  0.,   0., 151.,  18.],
         [  0.,   0., 151.,  19.],
         [  0.,   0., 151.,  20.],
         [  0.,   0., 151.,  21.],
         [  0.,   0., 151.,  22.],
         [  0.,   0., 151.,  23.],
         [  0.,   0., 151.,  24.],
         [  0.,   0., 151.,  25.]], dtype=float32),
  'Discrete': array([[  0, 144, 178],
         [  0, 144, 179],
         [  0, 144, 180],
         [  0, 144, 181],
         [  0, 144, 182],
         [  2, 144, 183],
         [  2, 144, 184],
         [  1, 144, 185],
         [  2, 144, 186],
         [  2, 144, 187],
         [  2, 144, 188],
         [  2, 144, 189],
         [  2, 144, 190],
         [  1, 144, 191],
         [  1, 144, 192],
         [  0, 145, 178],
         [  0, 145, 179],
         [  0, 145, 180],
         [  0, 145, 181],
         [  0, 145, 182],
         [  2, 145, 183],
         [  4, 145, 184],
         [  2, 145, 185],
         [  2, 145, 186],
         [  2, 145, 187],
         [  2, 145, 188],
         [  2, 145, 189],
         [  2, 145, 190],
         [  1, 145, 191],
         [  1, 145, 192],
         [  0, 146, 178],
         [  0, 146, 179],
         [  0, 146, 180],
         [  0, 146, 181],
         [  0, 146, 182],
         [  2, 146, 183],
         [  4, 146, 184],
         [  4, 146, 185],
         [  2, 146, 186],
         [  1, 146, 187],
         [  1, 146, 188],
         [  1, 146, 189],
         [  1, 146, 190],
         [  1, 146, 191],
         [  1, 146, 192],
         [  0, 147, 178],
         [  0, 147, 179],
         [  0, 147, 180],
         [  0, 147, 181],
         [  0, 147, 182],
         [  2, 147, 183],
         [  4, 147, 184],
         [  4, 147, 185],
         [  2, 147, 186],
         [  1, 147, 187],
         [  1, 147, 188],
         [  1, 147, 189],
         [  1, 147, 190],
         [  1, 147, 191],
         [  1, 147, 192],
         [  0, 148, 178],
         [  0, 148, 179],
         [  0, 148, 180],
         [  0, 148, 181],
         [  0, 148, 182],
         [  2, 148, 183],
         [  4, 148, 184],
         [  4, 148, 185],
         [  2, 148, 186],
         [  2, 148, 187],
         [  1, 148, 188],
         [  1, 148, 189],
         [  1, 148, 190],
         [  1, 148, 191],
         [  1, 148, 192],
         [  0, 149, 178],
         [  0, 149, 179],
         [  0, 149, 180],
         [  0, 149, 181],
         [  0, 149, 182],
         [  2, 149, 183],
         [  4, 149, 184],
         [  5, 149, 185],
         [  5, 149, 186],
         [  5, 149, 187],
         [  4, 149, 188],
         [  2, 149, 189],
         [  1, 149, 190],
         [  1, 149, 191],
         [  1, 149, 192],
         [  0, 150, 178],
         [  0, 150, 179],
         [  0, 150, 180],
         [  0, 150, 181],
         [  0, 150, 182],
         [  2, 150, 183],
         [  4, 150, 184],
         [  4, 150, 185],
         [  4, 150, 186],
         [  4, 150, 187],
         [  4, 150, 188],
         [  4, 150, 189],
         [  2, 150, 190],
         [  4, 150, 191],
         [  4, 150, 192],
         [  0, 151, 178],
         [  0, 151, 179],
         [  0, 151, 180],
         [  0, 151, 181],
         [  0, 151, 182],
         [  2, 151, 183],
         [  2, 151, 184],
         [  2, 151, 185],
         [  2, 151, 186],
         [  2, 151, 187],
         [  2, 151, 188],
         [  2, 151, 189],
         [  2, 151, 190],
         [  2, 151, 191],
         [  2, 151, 192],
         [  0, 152, 178],
         [  0, 152, 179],
         [  0, 152, 180],
         [  0, 152, 181],
         [  0, 152, 182],
         [  0, 152, 183],
         [  0, 152, 184],
         [  0, 152, 185],
         [  0, 152, 186],
         [  0, 152, 187],
         [  0, 152, 188],
         [  0, 152, 189],
         [  0, 152, 190],
         [  0, 152, 191],
         [  0, 152, 192],
         [  0, 153, 178],
         [  0, 153, 179],
         [  0, 153, 180],
         [  0, 153, 181],
         [  0, 153, 182],
         [  0, 153, 183],
         [  0, 153, 184],
         [  0, 153, 185],
         [  0, 153, 186],
         [  0, 153, 187],
         [  0, 153, 188],
         [  0, 153, 189],
         [  0, 153, 190],
         [  0, 153, 191],
         [  0, 153, 192],
         [  0, 154, 178],
         [  0, 154, 179],
         [  0, 154, 180],
         [  0, 154, 181],
         [  0, 154, 182],
         [  0, 154, 183],
         [  0, 154, 184],
         [  0, 154, 185],
         [  0, 154, 186],
         [  0, 154, 187],
         [  0, 154, 188],
         [  0, 154, 189],
         [  0, 154, 190],
         [  0, 154, 191],
         [  0, 154, 192],
         [  0, 155, 178],
         [  0, 155, 179],
         [  0, 155, 180],
         [  0, 155, 181],
         [  0, 155, 182],
         [  0, 155, 183],
         [  0, 155, 184],
         [  0, 155, 185],
         [  0, 155, 186],
         [  0, 155, 187],
         [  0, 155, 188],
         [  0, 155, 189],
         [  0, 155, 190],
         [  0, 155, 191],
         [  0, 155, 192],
         [  0, 156, 178],
         [  0, 156, 179],
         [  0, 156, 180],
         [  0, 156, 181],
         [  0, 156, 182],
         [  0, 156, 183],
         [  0, 156, 184],
         [  0, 156, 185],
         [  0, 156, 186],
         [  0, 156, 187],
         [  0, 156, 188],
         [  0, 156, 189],
         [  0, 156, 190],
         [  0, 156, 191],
         [  0, 156, 192],
         [  0, 157, 178],
         [  0, 157, 179],
         [  0, 157, 180],
         [  0, 157, 181],
         [  0, 157, 182],
         [  0, 157, 183],
         [  0, 157, 184],
         [  0, 157, 185],
         [  0, 157, 186],
         [  0, 157, 187],
         [  0, 157, 188],
         [  0, 157, 189],
         [  0, 157, 190],
         [  0, 157, 191],
         [  0, 157, 192],
         [  0, 158, 178],
         [  0, 158, 179],
         [  0, 158, 180],
         [  0, 158, 181],
         [  0, 158, 182],
         [  0, 158, 183],
         [  0, 158, 184],
         [  0, 158, 185],
         [  0, 158, 186],
         [  0, 158, 187],
         [  0, 158, 188],
         [  0, 158, 189],
         [  0, 158, 190],
         [  0, 158, 191],
         [  0, 158, 192]], dtype=int32),
  'N': array([225], dtype=int32)}}
In [7]:
obs[1].keys()
Out[7]:
dict_keys(['Entity', 'Tile'])

The agent's obs consists of two parts:

  • Entity: the information of yourself, other players and npcs.
  • Tile: the information of local map with 15x15 size.

Entity Information

In [ ]:
obs[1]['Entity'].keys()
Out[ ]:
dict_keys(['Continuous', 'Discrete', 'N'])
In [ ]:
obs[1]['Entity']['Continuous'].shape
Out[ ]:
(100, 13)
In [ ]:
obs[1]['Entity']['Discrete'].shape
Out[ ]:
(100, 4)
In [ ]:
obs[1]['Entity']['N']
Out[ ]:
array([4], dtype=int32)

The entity information is a dictionary with the following keys

  • Continuous: the continuous features, a 2d ndarray with shape 100*13.
    • The first dimention 100 is the max number of agents that can be observed, and is controlled by config.N_AGENT_OBS.
    • The second dimention 13 is the number of feature columns and the meaning of each column will be explained in detail.
  • Discrete: the discrete features, a 2d ndarray with shape 100*4.
    • The first dimention 100 is the max number of agents that can be observed, and is controlled by config.N_AGENT_OBS.
    • The second dimention 4 is the number of feature columns.
    • Notice that the discrete information is duplicate of (a part of) the continuous information, which means you can simply drop the discrete information.
  • N: the number of agents observed (including yourself) in current vision.

Tile Information

In [ ]:
obs[1]['Tile'].keys()
Out[ ]:
dict_keys(['Continuous', 'Discrete'])
In [ ]:
obs[1]['Tile']['Continuous'].shape
Out[ ]:
(225, 4)
In [ ]:
obs[1]['Tile']['Discrete'].shape
Out[ ]:
(225, 3)

The tile information is a dictionary with the following keys

  • Continuous: the continuous features, a 2d ndarray with shape 225*4.
    • The first dimention 225 is the number of tiles within agent's vision, which is controlled by config.NSTIM. When config.NSTIM=7 by default, the number of tiles is (2*7+1)^2 = 225.
    • The second dimention 4 is the number of feature columns and the meaning of each column will be explained in detail.
  • Discrete: the discrete features, a 2d ndarray with shape 225*3.
    • The first dimention 225 is the number of tiles within agent's vision, which is controlled by config.NSTIM.
    • The second dimention 3 is the number of feature columns.
    • Notice that the discrete information is also duplicate of (a part of) the continuous information, which means you can simply drop the discrete information.

Feature Columns

Information Type Index Feature Description
Entity Continuous(100*13) 0 Mask whether this row contains information, 1 for useful, 0 for null.
1 Entity_ID ID of this entity; for players, ID>0, for NPCs, ID<0
2 Attacker_ID the ID of last agent that attacks this entity
3 Level the level of this entity
4 Population the population this entity belongs to, can be used to identify teammates and opponents; for players, population>0; for NPCs, population<0
5 Row_index the row index of this entity
6 Column_index the column index of this entity
7 Damage the damage this entity has been received
8 Timealive The time this entity has been alive
9 Food current food this entity has
10 Water current water this entity has
11 Health current health of this entity
12 Is_freezed whether this entity is freezed right now, 1 for freezed, 0 for not.
Discrete(100*4) 0 Mask whether this row contains information, 1 for useful, 0 for null.
1 Population the population this entity belongs to, can be used to identify teammates and opponents; for players, population>0; for NPCs, population<0
2 Row_index the row index of this entity
3 Column_index the column index of this entity
N / N_agents_observed the number of agents within this entity's vision right now.
Tile Continuous(225*4) 0 N_entity the current number of entities on this tile
1 Type the type of this tile, 0 for lava, 1 for water, 2 for grass, 3 for scrub, 4 for forest, 5 for stone
2 Row_index the row index of this tile
3 Column_index the column index of this tile
Discrete(225*3) 0 Type the type of this tile, 0 for lava, 1 for water, 2 for grass, 3 for scrub, 4 for forest, 5 for stone
1 Row_index the row index of this tile
2 Column_index the column index of this tile

Action Space

In [ ]:
from nmmo.io import action
act_space = env.action_space(agent=0)
print("Action space:")
print("*"*2,action.Attack,": ",act_space[action.Attack])
print("-"*8,action.Style,": ",act_space[action.Attack][action.Style])
print("-"*8,action.Target,": ",act_space[action.Attack][action.Target])
print("*"*2,action.Move,": ",act_space[action.Move])
print("-"*8,action.Direction,act_space[action.Move][action.Direction])
Action space:
** <class 'nmmo.io.action.Attack'> :  Dict(<class 'nmmo.io.action.Style'>:Discrete(3), <class 'nmmo.io.action.Target'>:Discrete(100))
-------- <class 'nmmo.io.action.Style'> :  Discrete(3)
-------- <class 'nmmo.io.action.Target'> :  Discrete(100)
** <class 'nmmo.io.action.Move'> :  Dict(<class 'nmmo.io.action.Direction'>:Discrete(4))
-------- <class 'nmmo.io.action.Direction'> Discrete(4)

As shown above, the action space is presented as a nested dictionary. And the keys are the classes from nmmo.io.action.

The agent can perform two actions at the same timestep:

  1. Attack: you can attack an entity (include npc and other players) within your vision.The action can be empty.(If you don't send this action, the agent will not attack anyone.)

    • Target: Choose the target the agent attack.
    • Style: Choose the style the agent use to attack.
  2. Move: you can move in the four directions. The action can be empty.(If you don't send this action, the agent will stay.)

    • Direction: Choose the direction the agent moves.

And you should return actions like this:

from nmmo.io import action
actions = {
  1: {action.Attack:{
        action.Style: 0,
        action.Target: 3
        },
        action.Move: {
          action.Direction: 1
        }
      },
  2: {action.Attack:{
        action.Style: 2,
        action.Target: 4
        },
        action.Move: {
          action.Direction: 3
        }
      },
  ...
}

Team-Based Env

With game setting of the competition, it would be better to make the input and output based on team. Therefore, users can focus on training teams, and do not need to care about how to seperate players into teams manully.

The TeamBasedEnv is almost the same as the nmmo.Env execpt for the reset() and step() methods which handle the team-player mapping automatically.

TeamBasedEnv is used in evaluation, and it may be used in training too.

from ijcai2022nmmo import CompetitionConfig, TeamBasedEnv

env = TeamBasedEnv(CompetitionConfig())

# input
actions_by_team = {
    # actions of team0
    0: {
        0: action_of_player0_in_team0,
        1: action_of_player1_in_team0,
        ...
        7: action_of_player7_in_team0,
    },
    # actions of team1
    1: {
        0: action_of_player0_in_team1,
        ...
    },
    ...
    # actions of team15
    15: {
        ...
        7: action_of_player7_in_team15
    },
}

(   
    observations_by_team, 
    rewards_by_team, 
    dones_by_team, 
    infos_by_team,
) = env.step(actions_by_team)

print(observations_by_team)

...

{
    # observations of team0
    0: {
        0: obs_of_player0_in_team0,
        1: obs_of_player1_in_team0,
        ...
        7: obs_of_player7_in_team0,
    },
    # observations of team1
    1: {
        0: obs_of_player0_in_team1,
        ...
    },
    ...
    # observations of team15
    15: {
        ...
        7: obs_of_player7_in_team15
    },
}

Replay

Before getting started, you need download a NMMO client from this page and run it locally. Here is an example showing how to save and load replay.

Note: This code block won't run in colab. Please run it on your local machine.

In [ ]:
import nmmo
from ijcai2022nmmo import CompetitionConfig, TeamBasedEnv, scripted


class Config(CompetitionConfig):
    SAVE_REPLAY = "demo"


def save_replay():
    """Demo for saving replay"""
    config = Config()
    env = TeamBasedEnv(config=config)
    scripted_ai = scripted.CombatTeam(None, config)
    obs = env.reset()
    t, horizon = 0, 32
    while True:
        env.render()
        decision = {}
        for team_id, o in obs.items():
            decision[team_id] = scripted_ai.act(o)
        env.step(decision)
        t += 1
        if t >= horizon:
            break
    env.terminal()


def load_replay():
    """Demo for loading replay"""
    replay = nmmo.Replay.load(Config.SAVE_REPLAY + ".replay")
    replay.render()


if __name__ == "__main__":
    save_replay()
    # load_replay()

Comments

drzhuzhe
Almost 2 years ago

whye value in Continue and Discrete obsevation space is different ?

tboulet
Almost 2 years ago

Thank you for this notebook !

You must login before you can post a comment.

Execute