Location
Badges
Activity
Ratings Progression
Challenge Categories
Challenges Entered
Multi-Agent Reinforcement Learning on Trains
Latest submissions
See Allfailed | 85972 | ||
failed | 83770 | ||
failed | 83759 |
Multi Agent Reinforcement Learning on Trains.
Latest submissions
See Allgraded | 32808 | ||
graded | 32782 | ||
graded | 32738 |
Multi-Agent Reinforcement Learning on Trains
Latest submissions
Participant | Rating |
---|---|
student | 271 |
microset | 0 |
Participant | Rating |
---|
Flatland Challenge
Malfunction data generator does not work
Almost 5 years agoHi @mlerik!
Please, check the behavior of the last agent in my new simulation. It seems that new_malfunction
is not shown correctly on step number 5. I see that this problem occurs on the first steps only (but all agents have entered the environment).
import time
import numpy as np
from flatland.envs.observations import TreeObsForRailEnv, GlobalObsForRailEnv
from flatland.envs.predictions import ShortestPathPredictorForRailEnv
from flatland.envs.rail_env import RailEnv
from flatland.envs.rail_generators import sparse_rail_generator
from flatland.envs.schedule_generators import sparse_schedule_generator
from flatland.utils.rendertools import RenderTool, AgentRenderVariant
np.random.seed(1)
stochastic_data = {'prop_malfunction': 0.5, # Percentage of defective agents
'malfunction_rate': 30, # Rate of malfunction occurence
'min_duration': 3, # Minimal duration of malfunction
'max_duration': 10 # Max duration of malfunction
}
TreeObservation = TreeObsForRailEnv(max_depth=2, predictor=ShortestPathPredictorForRailEnv())
speed_ration_map = {1.: 0.25, # Fast passenger train
1. / 2.: 0.25, # Fast freight train
1. / 3.: 0.25, # Slow commuter train
1. / 4.: 0.25} # Slow freight train
env = RailEnv(width=60,
height=60,
rail_generator=sparse_rail_generator(max_num_cities=12,
# Number of cities in map (where train stations are)
seed=14, # Random seed
grid_mode=False,
max_rails_between_cities=2,
max_rails_in_city=6,
),
schedule_generator=sparse_schedule_generator(speed_ration_map),
number_of_agents=5,
stochastic_data=stochastic_data, # Malfunction data generator
obs_builder_object=GlobalObsForRailEnv(),
remove_agents_at_target=True
)
obs = env.reset()
MY_ACTION = []
MY_ACTION.append({0: 4, 1: 2, 2: 4, 3: 4, 4: 2})
MY_ACTION.append({0: 2, 1: 2, 2: 2, 3: 2, 4: 2})
MY_ACTION.append({0: 2, 1: 2, 2: 2, 3: 2, 4: 2})
MY_ACTION.append({0: 2, 1: 2, 2: 2, 3: 2, 4: 2})
MY_ACTION.append({0: 2, 1: 2, 2: 3, 3: 2, 4: 4})
MY_ACTION.append({0: 2, 1: 2, 2: 1, 3: 2, 4: 2})
MY_ACTION.append({1: 4, 4: 4})
for step in range(7):
print("========== step number ", step, " ==========", sep = "")
action_dict = MY_ACTION[step]
print("my action ", action_dict)
for ind in range(env.get_num_agents()):
print(env.agents[ind].malfunction_data)
next_obs, all_rewards, done, _ = env.step(action_dict)
Malfunction data generator does not work
Almost 5 years agoWell, I have already downloaded and tested my code with this branch.
Firstly, I want to say that this branch uses gym utils (so, I had to download them to continue) - I saw an isssue with this problem.
Secondly, I send you a report with malfunction information. I did not explore it deeply, but found that there are some mistakes. As you can see, malfunction length do not update on the first steps.
Start episodeβ¦
0.9744243621826172
5
========== step number 0 ==========
0 [True, True, True, True, True] [2, 4, 0, 1, 3]
My action: {0: 4, 1: 4, 2: 4, 3: 4, 4: 4}
agent num 0 {βmalfunctionβ: 4, βmalfunction_rateβ: 30, βnext_malfunctionβ: 56, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 6, βmalfunction_rateβ: 30, βnext_malfunctionβ: 19, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 11, βmalfunction_rateβ: 30, βnext_malfunctionβ: 55, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 1, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 6, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 1 ==========
My action: {0: 4, 1: 4, 2: 4, 3: 4, 4: 4}
agent num 0 {βmalfunctionβ: 4, βmalfunction_rateβ: 30, βnext_malfunctionβ: 56, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 6, βmalfunction_rateβ: 30, βnext_malfunctionβ: 19, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 11, βmalfunction_rateβ: 30, βnext_malfunctionβ: 55, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 1, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 6, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 2 ==========
My action: {0: 4, 1: 4, 2: 4, 3: 4, 4: 4}
agent num 0 {βmalfunctionβ: 4, βmalfunction_rateβ: 30, βnext_malfunctionβ: 56, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 6, βmalfunction_rateβ: 30, βnext_malfunctionβ: 19, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 11, βmalfunction_rateβ: 30, βnext_malfunctionβ: 55, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 1, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 6, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 3 ==========
My action: {0: 4, 1: 4, 2: 4, 3: 4, 4: 4}
agent num 0 {βmalfunctionβ: 4, βmalfunction_rateβ: 30, βnext_malfunctionβ: 56, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 6, βmalfunction_rateβ: 30, βnext_malfunctionβ: 19, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 11, βmalfunction_rateβ: 30, βnext_malfunctionβ: 55, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 1, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 6, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 4 ==========
My action: {0: 2, 1: 4, 2: 4, 3: 4, 4: 4}
agent num 0 {βmalfunctionβ: 3, βmalfunction_rateβ: 30, βnext_malfunctionβ: 55, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 6, βmalfunction_rateβ: 30, βnext_malfunctionβ: 19, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 11, βmalfunction_rateβ: 30, βnext_malfunctionβ: 55, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 1, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 6, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 5 ==========
My action: {0: 2, 1: 4, 2: 4, 3: 4, 4: 4}
agent num 0 {βmalfunctionβ: 2, βmalfunction_rateβ: 30, βnext_malfunctionβ: 54, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 6, βmalfunction_rateβ: 30, βnext_malfunctionβ: 19, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 11, βmalfunction_rateβ: 30, βnext_malfunctionβ: 55, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 1, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 6, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 6 ==========
My action: {0: 2, 1: 2, 2: 4, 3: 4, 4: 4}
agent num 0 {βmalfunctionβ: 1, βmalfunction_rateβ: 30, βnext_malfunctionβ: 53, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 5, βmalfunction_rateβ: 30, βnext_malfunctionβ: 18, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 11, βmalfunction_rateβ: 30, βnext_malfunctionβ: 55, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 1, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 6, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 7 ==========
My action: {0: 2, 1: 2, 2: 4, 3: 2, 4: 2}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 52, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 4, βmalfunction_rateβ: 30, βnext_malfunctionβ: 17, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 11, βmalfunction_rateβ: 30, βnext_malfunctionβ: 55, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 6, βmalfunction_rateβ: 30, βnext_malfunctionβ: 0, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 6, βmalfunction_rateβ: 30, βnext_malfunctionβ: 5, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 8 ==========
My action: {0: 2, 1: 2, 2: 4, 3: 2, 4: 2}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 51, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 3, βmalfunction_rateβ: 30, βnext_malfunctionβ: 16, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 11, βmalfunction_rateβ: 30, βnext_malfunctionβ: 55, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 5, βmalfunction_rateβ: 30, βnext_malfunctionβ: 0, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 5, βmalfunction_rateβ: 30, βnext_malfunctionβ: 4, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 9 ==========
My action: {0: 2, 1: 2, 2: 4, 3: 2, 4: 1}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 50, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 2, βmalfunction_rateβ: 30, βnext_malfunctionβ: 15, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 11, βmalfunction_rateβ: 30, βnext_malfunctionβ: 55, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 4, βmalfunction_rateβ: 30, βnext_malfunctionβ: 0, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 4, βmalfunction_rateβ: 30, βnext_malfunctionβ: 3, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 10 ==========
My action: {0: 3, 1: 2, 2: 4, 3: 2, 4: 3}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 49, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 1, βmalfunction_rateβ: 30, βnext_malfunctionβ: 14, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 11, βmalfunction_rateβ: 30, βnext_malfunctionβ: 55, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 3, βmalfunction_rateβ: 30, βnext_malfunctionβ: 0, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 3, βmalfunction_rateβ: 30, βnext_malfunctionβ: 2, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 11 ==========
My action: {0: 3, 1: 2, 2: 2, 3: 2, 4: 1}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 48, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 13, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 10, βmalfunction_rateβ: 30, βnext_malfunctionβ: 54, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 2, βmalfunction_rateβ: 30, βnext_malfunctionβ: 0, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 2, βmalfunction_rateβ: 30, βnext_malfunctionβ: 1, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 12 ==========
My action: {0: 1, 1: 2, 2: 2, 3: 2, 4: 3}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 47, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 12, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 9, βmalfunction_rateβ: 30, βnext_malfunctionβ: 53, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 1, βmalfunction_rateβ: 30, βnext_malfunctionβ: 0, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 1, βmalfunction_rateβ: 30, βnext_malfunctionβ: 0, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 13 ==========
My action: {0: 4, 1: 2, 2: 2, 3: 4, 4: 2}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 46, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 11, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 8, βmalfunction_rateβ: 30, βnext_malfunctionβ: 52, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 0, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 0, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}========== step number 14 ==========
My action: {0: 4, 1: 4, 2: 4, 3: 4, 4: 4}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 45, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 10, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 51, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 5, βmalfunction_rateβ: 30, βnext_malfunctionβ: 29, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 9, βmalfunction_rateβ: 30, βnext_malfunctionβ: 100, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: True}========== step number 15 ==========
0 [False, False, False, False, False] [2, 4, 0, 1, 3]
1 [False, False, False, False, False] [3, 4, 1, 0, 2]
2 [False, False, False, False, False] [0, 3, 1, 2, 4]
3 [False, False, False, False, False] [2, 4, 1, 0, 3]
4 [False, False, False, False, False] [3, 2, 1, 4, 0]
5 [False, False, False, False, False] [3, 2, 1, 0, 4]
6 [False, False, False, False, False] [0, 3, 2, 4, 1]
7 [False, False, False, False, False] [2, 4, 0, 1, 3]
8 [False, False, False, False, False] [1, 0, 3, 4, 2]
9 [False, False, False, False, False] [2, 4, 3, 0, 1]
My action: {2: 4, 3: 4, 4: 4}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 44, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 9, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 6, βmalfunction_rateβ: 30, βnext_malfunctionβ: 50, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 4, βmalfunction_rateβ: 30, βnext_malfunctionβ: 28, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 8, βmalfunction_rateβ: 30, βnext_malfunctionβ: 99, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: True}========== step number 16 ==========
My action: {2: 4, 3: 4, 4: 4}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 43, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 8, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 5, βmalfunction_rateβ: 30, βnext_malfunctionβ: 49, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 3, βmalfunction_rateβ: 30, βnext_malfunctionβ: 27, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 7, βmalfunction_rateβ: 30, βnext_malfunctionβ: 98, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: True}========== step number 17 ==========
My action: {2: 4, 3: 4, 4: 4}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 42, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 7, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 4, βmalfunction_rateβ: 30, βnext_malfunctionβ: 48, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 2, βmalfunction_rateβ: 30, βnext_malfunctionβ: 26, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 6, βmalfunction_rateβ: 30, βnext_malfunctionβ: 97, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: True}========== step number 18 ==========
My action: {2: 4, 3: 4, 4: 4}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 41, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 6, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 3, βmalfunction_rateβ: 30, βnext_malfunctionβ: 47, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 1, βmalfunction_rateβ: 30, βnext_malfunctionβ: 25, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 5, βmalfunction_rateβ: 30, βnext_malfunctionβ: 96, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: True}========== step number 19 ==========
My action: {2: 4, 4: 4}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 40, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 5, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 2, βmalfunction_rateβ: 30, βnext_malfunctionβ: 46, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 24, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 4, βmalfunction_rateβ: 30, βnext_malfunctionβ: 95, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: True}========== step number 20 ==========
My action: {2: 4, 4: 4}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 39, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 4, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 1, βmalfunction_rateβ: 30, βnext_malfunctionβ: 45, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 23, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 3, βmalfunction_rateβ: 30, βnext_malfunctionβ: 94, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: True}========== step number 21 ==========
My action: {4: 4}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 38, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 3, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 44, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 22, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 2, βmalfunction_rateβ: 30, βnext_malfunctionβ: 93, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: True}========== step number 22 ==========
My action: {4: 4}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 37, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 2, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 43, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 21, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 1, βmalfunction_rateβ: 30, βnext_malfunctionβ: 92, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: True}========== step number 23 ==========
My action: {}
agent num 0 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 36, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 1 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 1, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 2 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 42, βnr_malfunctionsβ: 1, βmoving_before_malfunctionβ: False}
agent num 3 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 20, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: False}
agent num 4 {βmalfunctionβ: 0, βmalfunction_rateβ: 30, βnext_malfunctionβ: 91, βnr_malfunctionsβ: 2, βmoving_before_malfunctionβ: True}Episode: Steps 23 Score = -80.0
Malfunction data generator does not work
Almost 5 years agoHi mlerik
Just on every stepβ¦
Note that it is the latest master branch. If you want to test something else, I have some time right now.
Malfunction data generator does not work
Almost 5 years agoHi everyone!
I would like to point out that malfunction data generator in latest FLATland version (for example in flatland_2_0_example.py) does not work at all. I tried to set the maximum percentage of defective agents parameter but didn`t get a single occurence.
Please add this feature again. Thank you!
Rewards function calculation bug
About 5 years agoWell, actually I wanted to clarify the process of calculating the final score of our submissions.
I guess that you use this reward function to calculate optimality of solution in Round 1, as long as I found this line in run.py script:
if done['__all__']: print("Reward : ", sum(list(all_rewards.values())))
So, if you still use this approach (and you definitely use it in flatland_2_0_example.py), the score function is calculated incorrectly.
For example, I can order some agents not to move and enter the environment. In this case, they have no impact on total penalty, so the final score reduces (which of course is incorrect) - I can describe this with more details, if you want.
Thus, there is a bug in score calculation, which can be fixed by changing the default rewards function or making anything else.
Sorry for any of misunderstandings.
Rewards function calculation bug
About 5 years agoHi FLATland team!
I want to report a bug in rewards function (latest commit - 05.10).
I found that the system do not count any of score penalty (reward) of an agent if it has not started moving and formally does not exist on a map. It means that agent can wait in it`s start point for some steps for free.
Please, add rewards calculation for non-spawned agents like they are waiting.
Suggestion/bug fix of Flatland 2.0
About 5 years agoHi flatland team!
Just a few questions about current version of Flatland.
- I found that sometimes
agent.malfunction_data['next_malfunction']
parameter can be below zero. It happens when the next malfunction for a single agent occurs before current ended (I can send a simulation code, if you want). So, a flatland user have to write one extra line to find out anext_malfunction
value. Personally I use this code:
next_malfunction = max(self.env.agents[ind].malfunction_data['next_malfunction'], self.env.agents[ind].malfunction_data['malfunction'] + 1)
Please, fix your next_malfunction
output with something like that to avoid any of misunderstandings.
-
And one more question. Do you want to allow the agent to wait (with STOP_MOVING status) when its
position_fraction
is upper than zero - I mean when the agent started to move between two cells? I ask this as long as right now agents are unable to do that, which looks not so logical. -
What values of
malfunction_rate
andmax_duration
will you set in simulations (sorry, if I ask this too early).
P. S. Thank you for past bug fixs. Now, it is possible to make full solutions for round 2. You`ve done a great job!
Reporting Bugs in Flatland
About 5 years agoHi everyone!
I would like to report a bug of Flatland 2.0, connected with stochastic events.
As you can see, a stochastic event occurs with my agent on the first step. Then my agent waits for several steps, sending to the system β4β (or nothing).
However, when this stochastic event ends, something throws the agent out of railway system (or out of a map in my example). You can see that position of the agent is (-1, 2).
Note that this problem occurs in many different cases when an agent waits during stochastic event is ending.
Simulate the bug with following code (just copy to jupyter):
import time
from flatland.envs.observations import TreeObsForRailEnv
from flatland.envs.predictions import ShortestPathPredictorForRailEnv
from flatland.envs.rail_env import RailEnv
from flatland.envs.rail_generators import sparse_rail_generator
from flatland.envs.schedule_generators import sparse_schedule_generator
from flatland.utils.rendertools import RenderTool
stochastic_data = {'prop_malfunction': 1., # Percentage of defective agents
'malfunction_rate': 70, # Rate of malfunction occurence
'min_duration': 2, # Minimal duration of malfunction
'max_duration': 5 # Max duration of malfunction
}
speed_ration_map = {1.: 1., # Fast passenger train
1. / 2.: 0., # Fast freight train
1. / 3.: 0., # Slow commuter train
1. / 4.: 0.} # Slow freight train
env = RailEnv(width=25,
height=30,
rail_generator=sparse_rail_generator(num_cities=5, # Number of cities in map (where train stations are)
num_intersections=4, # Number of intersections (no start / target)
num_trainstations=25, # Number of possible start/targets on map
min_node_dist=6, # Minimal distance of nodes
node_radius=3, # Proximity of stations to city center
num_neighb=3, # Number of connections to other cities/intersections
seed=215545, # Random seed
grid_mode=True,
enhance_intersection=False
),
schedule_generator=sparse_schedule_generator(speed_ration_map),
number_of_agents=1,
stochastic_data=stochastic_data, # Malfunction data generator
)
env_renderer = RenderTool(env)
env_renderer.render_env(show=True, frames=False, show_observations=False)
_action = dict()
for step in range(4):
#_action[0] = 4
obs, all_rewards, done, info = env.step(_action)
print(info["malfunction"][0])
print(env.agents[0].position)
env_renderer.render_env(show=True, frames=False, show_observations=False)
time.sleep(0.1)
print("Here position of the agent is invalid")
Thank you for fixing this bug. Hope, it won`t be difficult.
[ANNOUNCEMENT] Start Round 2
Almost 5 years agoHi!
It seems that the Example link is unavailable. Could you please fix that.