Loading
Feedback

mlerik 109

Name

Erik Nygren

Organization

SBB

Location

CH

Badges

0
0
2

Connect

LinkedIn

Activity

Sep
Oct
Nov
Dec
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Mon
Wed
Fri

Ratings Progression

Loading...

Challenge Categories

Loading...

Challenges Entered

Multi Agent Reinforcement Learning on Trains.

Latest submissions

See All
failed 22806
failed 22682
failed 22681
Gold 0
Silver 0
Bronze 2
Trustable
May 16, 2020
Newtonian
May 16, 2020

Badges

  • Has filled their profile page
    May 16, 2020

  • May 16, 2020

  • May 16, 2020

  • May 16, 2020

  • May 16, 2020

  • May 16, 2020

  • May 16, 2020

  • May 16, 2020

  • May 16, 2020

  • May 16, 2020

  • May 16, 2020

  • May 16, 2020

  • May 16, 2020

  • May 16, 2020

  • May 16, 2020

  • May 16, 2020
  • Kudos! You've won a bronze badge in this challenge. Keep up the great work!
    Challenge: Flatland Challenge
    May 16, 2020
Participant Rating
Participant Rating
mlerik has not joined any teams yet...

Flatland Challenge

Competition has ended

9 months ago

Hi @mugurelionut

We are happy to hear that you enjoyed the competition and that there are still many unexplored ideas.

Yes we would like to make a formal publication. And yes the solutions will be co-authors as well. As soon as we have started the process we will reach out to all participants to be included in the publication.

In the coming days we will reach out to all top participants on the leaderboard to discuss the next steps.

Hope this helps for now.

Best regards,
Erik

Publishing the Solutions

9 months ago

Hi @mugurelionut

All submissions on the leaderboard are valid as they passed the official submission process.

Now we are only checking, that there are no licencing issues or cheats in the code. The order on the leaderboard will not change, except if we have to discualify solutions for cheating.

Hope this clarifies your questions.

Best regards,
Erik

Competition has ended

9 months ago

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

The Flatland Competition has finsihed!!!

:partying_face: :champagne: :smiley: :beers:

!!Thank you for all the great submissions and contributions!!

:partying_face: :champagne: :smiley: :beers:

Dear Particpants

After alot of hard work and many bugfixings, submissions and amazing solutions the Flatland Challenge has come to an end. We would like to thank all of you for your great contributions, inputs, enthusiasm, time and energy you spent on this challenge.

We are currently looking into the leaderboard solutions and will contact and announce the winners of the competition as soon as possible.

What’s next?
After reviewing all solutions, selected participants will be invited to attend the AMLD2020 where results from this Challenge will be presented. Furthermore we will prepare a publication containing some of the solutions provided through the challenge (we will reach out to participants in the coming days).

At the AMLD2020 we will also give a larger update about Flatland and what is coming next.

Thank you again for your great contributions and stay tuned…

Best regards,
The Flatland Team

Publishing the Solutions

9 months ago

Hi @fabianpieroth

We are currently evaluation the top submissions of the challenge. We are planning on publishing the results and the challenge together with the top participants.

As soon as we have veryfied the solutions to be valid of the participants we will reach out to them and set upt to publishing process together with them.

At the applied machine learning days we will also present the results together with the invited participants.

So stay tuned for further updates on this.

Best regards,
The Flatland Team

Is this an error in the local evaluation envs?

9 months ago

Hi @compscifan2019

This indeed looks like a bug. You don not need to handle these kinds of situations. Did you ever observe an agent being placed on such a rail segment? We will look into this bug and fix it.

Thanks for bringing this to our attention.

Best regards,
The Flatland Team

Further questions regarding submissions

10 months ago

Hi @vitaly_bondar

The mean percentage of agents done is calculated as you expected. Number of arrived trains divided by total number of trains and then we take the mean over all episodes.

If we would calculate the mean over all episodes ones we would bias the mean towards the results of the larger envs. In the current setting the bias is towards smaller envs where few agents have more influence on the mean of agents finished.

Hope this answers your question.

best regards,
Erik

Fraction of done-agents

10 months ago

Hi @eli_meirom

The time step is per Episode. This means this is the total number of environment steps you can call before the enviromnent terminates.

Currently we don’t put stricter schedules on the individual trains, thus each individual train can run whenever it is best suited within the total time window.

Hope this answers your questions.

Have fun with the Challenge! :slight_smile:

The Flatland Team

Flatland documentation / code inconsistency

10 months ago

Hi @eli_meirom

Thank you for pointing this out. We had just realized this at the current week. A workaround is to check out an earlier version of the Example. But we will also update the example to be compatible with the earlier version.

Sorry for the caused inconvenience…

[ANNOUNCEMENT] Submission wokring for Round 2

10 months ago

Hi @mugurelionut

Looking at the generated files. We use the default value of 20 agents for the files used for submission scoring. I will however still check to update this to be more precise in the future. Does this help you with your submissions?

Best regards,
Erik

[ANNOUNCEMENT] Submission wokring for Round 2

10 months ago

Hi @mugurelionut

Sorry for the delay. I will look into this now. Will let you know if I can give a fix to this as the levels are stored in pickle files currently and don’t contain the information about number of cities. Maybe we will have to regenerate the files with this updated information. will let you know as soon as I have fix.

Best regards,
Erik

Spawning of agents seems to work diffrent in the second step

10 months ago

This will highly depend on the ordering of the trains.
This i a design choice and there would be other solutions to this problem. In the case of flatland, trains can drive behind each other as long as the index of the train in front is lower than the index of the train behind it. For example.

Train 2 can follow train 1 directly. But train 1 CANNOT follow train 2 directly.

Does this help?

Which log are you refereing to? I’m happy to take a look at it.

Best regards

Erik

Spawning of agents seems to work diffrent in the second step

10 months ago

Hi @Lemutar

The check is actually done exactly the same way in all cases. We do not differentiate between spawning and movement in the environment.

Do you have an example where this differs between spawning and movement in the environment? If so, this would be very helpful for finding the bug and fixing it.

Thanks

Erik

Spawning of agents seems to work diffrent in the second step

10 months ago

hi @Lemutar

Looking at the example you provided it seem that agent 1 want to enter an occupied cell.
This has to do with the way the env.step executes the commands. It does take agent actions and executes them serially by increasing agent ID.

This means that agent 1 tries to enter cell (6,9), which is still occupied by agaent two who has not yet moved to cell (6,10). Because moving in to an occupied cell is an illegal action, the env does not execute the action and agent 1 stays outside the env and ready to depart.

Hope this clarifies your question.

best regards,
Erik

Training Related Questions

11 months ago

Additional information:

You can use git-lfs to store large model files

Training Related Questions

11 months ago

Hi @leventkent

Welcome to the Challenge :steam_locomotive::railway_car::railway_car::railway_car::railway_car::confetti_ball::tada:

Here the answers to your questions:

  • Given the limited time of each submission we recommend that you submit already trained models.
  • In the current round there are actually only 250 envs that you have to solve. They all have different maps and different schedules for individual agents.
  • They have to reside within the repository.

Hope this helps you get started with the challenge, have fun.

The Flatland Team

Tree Observation and Normalisation

11 months ago

For the stock observation builder the reasoning is the following:

  • Negative Values indicate no path possible
  • Values smaller than inf indicate something has been observed
  • Values equal to infinity means it was not observed here

Then with the normalization we set a certain range of values that we care about. If they are larger than e ceraint value we don’t differentiate between the anymore and thus all of them collaps to 1. The reasoning here is that the exact values for objects far away are irrelevant.

We highly suggest you use your own custom observation builder and normalization function. The stock functions are just meant as examples for you to build upon.

Hope this clarifies your questions.

Have fun with the Challenge :slight_smile:

Best regards,
Erik

[ANNOUNCEMENT] Submission wokring for Round 2

11 months ago

Hi @fabianpieroth

The agents done=True is necessary for training to indicate that the episode terminated and thus your ML-Approach is not expecting a next observation anymore.

The returned reward should be equal to the current state of the environment. Thus if not all agents have reached their target the reward is equal to the step reward of each agent. If you need a more negative reward for agents not terminating their task in time you could do reward shaping using the env information. env.agent.status will tell you whether or not the agent has finshed it’s individual task.

Looking at the code I see that if you continue the enviromnent beyond the time that it terminated it will return the positive reward to all agents. This is a bug on our side and we will fix this.

Hope this clarifies your question.

Best regards,
Erik

[ANNOUNCEMENT] Submission wokring for Round 2

11 months ago

Hi @mugurelionut

The actual formula that you mentioned is correct. There is a problem when loading from files as we currently don’t store the numbers of cities in the pickle file. thus it is impossible for you to currently compute the appropriate max_time_steps for the pickle file without the associated generator parameters.
I will open an issue about this on gitlab and adress it in the coming days. Sorry for the caused inconvenience.

Best regards,
Erik

Computation budget

11 months ago

Hi @mugurelionut

Thank you for pointing this out to us. We are sorry for the confusion that we caused and the lack of clear communication with the changes with malfunctions.

I try to shed some light on our thought process to explain the current malfunction behavior.

Agents malfunctioning before they enter the environment is equivalent to delayed departures of trains, which are quite common in real life and thus we need to model them in flatland as well. Updating the malfunciton variables before the trains enter the environment is done to simplify the task. If we were to update the malfunciton data only at the entrance of the train into the environment there would be no room for alternative solutions. By giving the controller access to delayed departures we hoped to go allow for more flexibility.

In the old malfunction model this was not necessary as we already gave information about when the next malfunction would occur, and thus planning was easier in advance of the events.

This environment is growing with important imput from the community such as yours. We apologize for the inconvenience that some of our changes cause along the way. But we truly believe that with all your great feedback and effort we have come quite far with Flatland.

We hope you all enjoy this challenging task as much as we do and help us develop the necessary tools to make public transport even better in the future.

Best regards,

The Flatland Team

Computation budget

11 months ago

Hi @mugurelionut

Your observation about the malfunction rate was correct. However, we introduced a different method to generate malfunctions from malfunction_generators. Thus just like the next_malfunction we don’t use malfunction_rate anymore for malfunctions.

In this new version, any agent can break so there is no point in leaving agents behind anymore.

For fairness all submissions in the leaderboard will be re-evaluated with these changes. This is also the reason why some variable are still present to keep code from crashing.

Yes when you update to the newest version of Flatland the new malfunction generators are in use. The FAQ has also been update to include these changes as well as explain the range of parameters and how to generate test files locally.

Your last assumption is wrong. They are not disjoint, the rate of malfunctions is low, but it can happen that multiple malfunctions occur at the same time. The malfunctions are indepentendt Poisson processes.

We are aware that this is a challenging task to solve, but that is unfortunately the reality of railway traffic where such problems have to be solved in real-time and where we can’t forsee when the next disturbance appears and how many we have at the same time. Thus the objective is to keep traffic as stable as possible even in presence of disturbances.

Hope this clarifies your issues.

Best regards,

Erik

Computation budget

11 months ago

Hi @mugurelionut

We are sorry for the inconvenience caused by the removal of next_malfunction. This was a leftover from early implementations of malfunctions and way mainly introduced to avoid divergence of environments on the service side from the client side. In the old implementation this value could be used for cheating as well, as you could ommit any agent with next malfunction value and only enter working agents into the environment.

In order to generate more realistic problems, where malfunctions arise stochasticly and you cannot predict them, we removed the next_malfunction parameter alltogether. We still allow for the malfunction duration to be read, this is also realistic as we usually have malfunction duration predictions.

Sorry for the confusion on the number of agents. Currently the upper limit is set to 200. This should remain the same for the whole challenge and only be updated for future challenges.

You can use the envs provided as explained in the starter kit for testing. I will also update the baselines repository to make use of the new parameters and put a good parameter set in the FAQ. Hope this helps.

Best regards,
Erik

Computation budget

11 months ago

Hi @Lbliek

Thank you for your question. Please consult the FAQ section for more information about this.

Best regards,

Erik

[ANNOUNCEMENT] Start Round 2

11 months ago

Hi @mugurelionut

This seems odd. Are you using the latest version of Flatland? SInce the latest version we have the default set to DONE_REMOVED. This is also the behavior that we have for the submissions.

If it will still not work please set remove_agents_at_target=True in the env constructor both in the client.py and service.py file.

But I think updating to Flatland version 2.1.8 should solve your problem.

Hope this helps,

The Flatland Team

Descriptions of the reward function

11 months ago

Hi @ryznefil

Yes this is correct. Given the time limit of the episodes it is hard to finish with all agents in time. Thus if you are able to manage traffic as good as possible by finishing the most agents your solution is considered better.

As in round 1 we expet many participants to achieve similar or the same score. Therefor we rank the same scores according to how fast the agents reach their target by looking at the normalized reward.

Hope this helps.

Best regards,

Erik

Descriptions of the reward function

11 months ago

Hi @ghabs

Sorry for the very late response. The information can be founde in the document here. This however is not very easy to find and understand thus in simple words, scores are currently computed as follows:

  1. Mean number of agents done, in other words how many agents reached their target in time.
  2. Mean reward is just the mean of the cummulated reward.
  3. If multiple participants have the same number of done agents we compute a β€œnomralized” reward as followes:
normalized_reward =cumulative_reward / (self.env._max_episode_steps +self.env.get_num_agents()

We are currently updating the FAQ section so the information will be available there soon.

Best regards,

The Flatland Team

[ANNOUNCEMENT] Submission wokring for Round 2

11 months ago

Dear @mugurelionut

Thank you for reaching out, hope this helps clarify the issue:

  1. Envs are currently never larger than (height, width) = (150, 150).
  2. Yes agents can only make decisions on cell entry. Once they have decided and have moved beyond the enty point there is no turning back. If they however chose to stop at cell entry they will again be allowed to chose an action.

Best regards,

The Flatland Team

[ANNOUNCEMENT] Submission wokring for Round 2

11 months ago

Hi @jasako

Yes it has changed, we now allow for much more time:

max_time_steps = int(4 * 2 * (env.width + env.height + 20))

This you can find in the run.py file in the starter kit

Have fun with the challenge :slight_smile:

Best regards,

The Flatland GTeam

[ANNOUNCEMENT] Submission wokring for Round 2

11 months ago

Hi @lcaubert

YES! :grinning::confetti_ball::tada:

Just use the startet kit linked above and read the official documenation to get started.

Best of luck and have fun

Best regards,

The Flatland Team

[ANNOUNCEMENT] Start Round 2

11 months ago

Thanks @mugurelionut for your update.

Yes we do make the malfunction update at the beginning of the step see here.

I will update the issue accoringly and discuss this in the team if we want to change the malfunction behavior or just make documentation clearer on this point.

Best regards,
Erik

[ANNOUNCEMENT] Submission wokring for Round 2

11 months ago

Hi @beibei

We are happy the train is rolling again :steam_locomotive::railway_car::railway_car::railway_car::railway_car: .

  1. The current parameter set are only with environments with max 200 agents. We introduced this due to some performance issues we were experiencing. If this changes in future we will let you know and also re-evaluate all previous submissions on the new env parameters.
  2. There again is an upper limit on the time allowed to run. It is currently set to 12h and a time-out if nothing happens on the server for 15 minutes.

Even though performance has increased alot with the latest fixes we are still working on some further improvements. If we achieve the desired performance ther might be slight updates to time limit as well as number of agents. We will communicate this transparently when something changes.

Best regards,

Erik

[ANNOUNCEMENT] Start Round 2

11 months ago

Hi @mugurelionut

Thanks for reporting this. I’m opening an issue on this an looking into it. Yes before the malfunction we should still perform the step.

Best regards,
Erik

[ANNOUNCEMENT] Submission wokring for Round 2

11 months ago

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Dear Flatlanders,

We have resolved the performance issues that kept submissions from working properly!

:tada::confetti_ball: You can now submit your solutions using the starter kit.:tada::confetti_ball:

Be sure to download the latest version from PyPi by running pip install -U flatland-rl.

Have fun with the challenge and feel free to reach out or share your thoughts here in the Forums.

The Flatland Team

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

[ANNOUNCEMENT] Start Round 2

11 months ago

Hi @mugurelionut

You are right in assuming that case 1 should occur. If you experienced case two, this would be a bug. It would be great if you could report this bug here: Issue Tracker

And follow these Guidlines.

Best regards,

The Flatland Team

[ANNOUNCEMENT] Start Round 2

11 months ago

Hi @mugurelionut

We are happy to hear that you like the improved version of Flatland. To answer your questions in short:

  1. Yes you can access all the information of the env and don’t have to reconstruct it from the observation. This was intended as we want participants to build their own observation builders and come up with clever ways to utilize the environment data.

  2. Yes we allow accessing this information. We did not want to make to big of a leap from earlier rounds.

  3. Yes next malfunction. Is the duration after the agent was repaired again. Sorry for the unclear naming and documentation. We will work on this.

  4. This was actually intended but I do see the problem which arises from this. We are discussing how we want to address this issue to satisfy our requirements without introducing confusing behavior. We will update everyone on this issue if we change the behavior.

Hope these answers are helpful.

We are thankful for your in depth feedback and we will update our documentation to better clarify the adressed details.

Best regards,

The Flatland Team

[ANNOUNCEMENT] Start Round 2

12 months ago

:warning::warning::warning:ATTENTION :warning::warning::warning:

Please don’t forget to update your Flatland version to the newest version before submitting. Older versions of Flatland will lead to divergences between the client and the server.

Best regards,

The Flatland Team

Can we still submit for round 1?

12 months ago

Thank you @compscifan2019 for your detailed bug report.

We saw this error with an earlier submission as well but now updated everythin. Can you reproduce this error still with the updated Round 2 submission or did it only fail with the old environments?

Best regards,
Erik

Can we still submit for round 1?

12 months ago

Hi @algomia

Yes, we did quite some enhancements to the behavior of the environment. I think you stand an equal chance to find a good solution to the problem as participants who submitted to round 1. There will also be an update to the baseline in the coming days introducing some new conecpts that can be used to tackle the problem. So stay tuned for an announcement.

Did you try to test your submission locally as explained in the starter-kit?

We are also working on a submission testing script that will let you generate similar environments as used in the submission scoring locally.

Best regards,

Erik

Can we still submit for round 1?

12 months ago

Hi @yizh9896

This was actually planned but we ran into some technical issues to run both rounds at the same time.
We would prefer to keep Rouind 2 open, as this is the most important round of the challenge.

Is there a reason for you to still test your code on round 1 instances? We are happy to support you and can test your code locally if you desire.

I will also update this information on the landing page.

Don’t hesistate to reach out if you have any further concerns or inputs.

Best regards,
Erik

[ANNOUNCEMENT] Start Round 2

12 months ago

merge was probably not fully done yet. Should work now.

[ANNOUNCEMENT] Start Round 2

12 months ago

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Dear Flatlanders :wink:

We are excited to announce that Round 2 is open for submissions!! :slight_smile::confetti_ball::tada::jack_o_lantern:

There are quite some changes included in this new round and few features were already announced with the release of Flatland 2.0 some of the main new features include:

  • Agents start outside the environment and have to actively enter to be on time
  • Agents leave the environment when their target is reached
  • Networks are much sparser an less paths are possible for each agent
  • Stochastic events stop agents and cause disruptions in the traffic network
  • Agents travel at different speeds

The baselines repository has been updated to incorporate these new changes and highlights how training can be implemented.

To make your submissions head over to the updated starter kit.

There is also a new Example file introducing the concepts of Flatland 2.1..

We are currently actively working to update all the documentation to the new changes so keep checking back or reaching out to us if there are any uncertainties.

We wish you all lots of fun with this new challenging round.

The Flatland Team

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Malfunction data generator does not work

12 months ago

Thanks @vetrov_andrew for reporting this bug.

Turns out there was a next malfunction time set to 0 in this case. I fixed this issue and also updated when the counter goes down for next malfunction. WIll run a few tests and push the updated version soon.

In a few hours there will be an announcement on the next round and the next version of Flatland and an example to highlight all the changes. Would be great if you can do an early review of that file as soon as it is online as you already have quite some experience with the new changes.

Best regards,
Erik

Is it possible to attempt only round 2?

12 months ago

Hi @sai_kalyan_siddanath

YES!!! :slight_smile: Round 1 examples are a good way to get to know the environment and experiment with different observatin builders and control algorithms.

We are now ready to release and start Round 2, so feel free to join in and help us solve this complex challenge.

Best regards,
The Flatland Team

Malfunction data generator does not work

12 months ago

Hi @vetrov_andrew

Thank you for the feedback. We are lookig into the issue with gym, which is needed to unify the seeding process across multiple instances.

The malfunction only starts counting when an agent actively enters the environment. This is a design choice. But we are happy to discuss benefits and drawbacks of this implementation.

Best regards,
Erik

Malfunction data generator does not work

12 months ago

Branch will be deleted when pipeline succeeds, then you can head over to master again.

Thanks to all participants for actively pointing out bugs and helping us evolve Flatland further.

Best regards,
The Flatland Team

Malfunction data generator does not work

12 months ago

Hi @vetrov_andrew

The current state and next release is on this branch.

It would be great to get some feedback if your code runs with this or if there are any issues.

Best regards
Erik

Malfunction data generator does not work

12 months ago

Out of curiosity what do you get if you print this at every step:

print(env.agents[a].malfunction_data)

where a is the agent number.

Malfunction data generator does not work

12 months ago

Hi @vetrov_andrew

This is only temporary. The fix is on it’s way just waiting for the test pipeline to pass and then we will merge. Would be great if you can then run a test again. But currently it does work on the feature branch with the bugfixes.

We had issues with diverging instances of different environments that needed to be addressed.

Best regards,
The Flatland Team

Error in observations.py

12 months ago

Hi @marcoliver_gewaltig

Thank you for pointing this bug out. It is already fixed and pending to be merged.

Best regards,
The Flatland Team

Rewards function calculation bug

12 months ago

Thanks @vetrov_andrew

For clarifying you bug report. I agree with your observation and will adjust this to the upcoming schedules.
This means any agent will be punished for leaving with a delay. By setting all starting times to 0 we basically introduce the negative reward for not entering.

I’m happy you brought this to our attention.

Best regards,
Erik

Rewards function calculation bug

12 months ago

Hi @vetrov_andrew

Thank you for pointing this out. We actually realized this ourselfes when training the agents.
Our suggestions would be to penalize the agents yourself for waiting. In future releases we will have more complex schedules where it is ok for agents to wait before entering the environment.

We suggest the following two solutions:

  1. Penalize waiting agents by looking at the info returned by the environment if env.agents[a].status == 0: all_rewards[a] -= 1
  2. Use other algorithms to decide what agent should enter when and only us RL for agents in the environment.

Does this help? Otherwise we can discuss further if this needs to be implemented and push it in the next update.

Best regards,

Erik

[ANNOUNCEMENT] Round 2 Update

12 months ago

Hi @RomanChernenko

We ran into issues when testing the implementation of the new features in Flatland 2.0 and felt that quality was insufficient for submissions. We have made great progress in the last few weeks and are expecting to start this weekend by releasing the major Flatland Update.

So keep your eyes open for the next announcement with detailed information about the update and the 2nd Round of the Challenge.

Best regards,

The Flatland Team

Reward function and reward information

12 months ago

Thank you @marcoliver_gewaltig

This are great inputs. I opened an issue here. Will get to this as soon as we have some capacity.

Best regards,
The Flatland Team

Reward function and reward information

12 months ago

Hi @marcoliver_gewaltig

Thank you for your input. The internat reward is mostly used for submission scoring. We encourage participants to shape their own reward to improve the behavior of their agents.
Do you have any suggestions how this could be simplified for participants? Are there variables / metrics the environment should provide in order to facilitat custom reward functions?

Best regards,
Erik

The train goes off the rails

12 months ago

Hi @xcilog

I think this is just a rendering issue. Looking at the observation rendering in the image you posted i think the render did not redraw the new rails.

Can you please try to do env_renderer.reset() where env_renderer is the renderer you defined.

Does this resolve your issue?

Best regards
Erik

Format of observation data in Flatland 2.0

12 months ago

Hi @mugurelionut

Thank you for pointing this out. I will open an Issue about it and fix this as soon as possible. Sorry for the inconvenience this caused.

Best regards,
Erik

[ANNOUNCEMENT] Round 2 Update

12 months ago

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Hi All

We would like to give a brief update about the Flatland challenge and the current work in progress.

First of all, thank you all for reporting bugs and putting in requests for new features. We are working hard to squash the last bugs and prepare the launch of the next round.

Unfortunately, there were some critical bugs in the new level generators which forced us to delay the launch of the second round.

The changes that are coming in this next round are still the same as posted here.
The only Difference will be that we are updating the way the sparse_rail_generator() works. This means it will create more realistic railway infrastructures.

Given the progress in the last few days we expect to have squashed all critical bugs and fixed the level generator in the coming week.

We are constantly updating the docs to reflect all the changes in the repository, we apologize for any inconsistencies or out-of-date information in the documentation. Please feel free to support us by contribution directly to the documentation (Don’t forget about the community contribution prize that will be awarded :slight_smile: ).

Thank you for your patience and great contributions to this project.

Best regards,

The Flatland Team

Your contributions to documentations and code are always helpful and you can submit any changes and documentations as pull requests to the main repository.

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Format of observation data in Flatland 2.0

12 months ago

hi @mugurelionut

Yes we did do some major update to the vanilla observations in Flatland 2.0. We are lagging a bit behind with our documentation as the team is currently very busy trying to squash the last few bugs so we can start the next round.

You can find a short documentation here

class GlobalObsForRailEnv(ObservationBuilder):
    """
    Gives a global observation of the entire rail environment.
    The observation is composed of the following elements:

        - transition map array with dimensions (env.height, env.width, 16),\
          assuming 16 bits encoding of transitions.

        - Two 2D arrays (map_height, map_width, 2) containing respectively the position of the given agent\
         target and the positions of the other agents targets.

        - A 3D array (map_height, map_width, 4) wtih
            - first channel containing the agents position and direction
            - second channel containing the other agents positions and diretions
            - third channel containing agent malfunctions
            - fourth channel containing agent fractional speeds
    """

Does this help? You can test your code using the current sparse_rail_generator but we are currently working on a fix and are hoping to push the updated version in the coming days.

Best regards,

The Flatland Team

Running `example/simple_example_1/2/3.py` render issue on Linux

About 1 year ago

Thanks @xzhaoma

We will look into this.

Best regards,

The Flatland Team

Suggestion/bug fix of Flatland 2.0

About 1 year ago

Hi @vetrov_andrew

Thanks for these great inputs.

  1. We will look into this and fix it such that next_malfunction is alwas greater or equal to zero.
  2. Currently we don’t want to allow agents to stop in the middle of the cell. (Positional fraction largen than 0). There is some real railway specific considerations such as reserved blocks that are similar to this behavior. But more importantly we disabled this to simplify the use of machine learning algorithms with the environment. If we allow stop actions in the middle of cells. then the controller needs to make much more observations and not only at cell changes. (Not set in stone and could be updated if the need arises)
  3. We are working on the exact values for malfunction_rate and max_duration. They will most likely be rare and not too long. Duration should be of a lenght that alternative routes are a meaningful strategy to improve punctuality. Malfunction rate should be kept low such that agents only break a few times.

We are working hard to make the final release happen this week.

Best regards,

The Flatland team

Training on GPU

About 1 year ago

Hi @maria_schoenholzer

If you are training on the vanilla implementation the speed up will be minor as the NN is very small in size. It uses the TreeObsForRailEnv which collects information along the possible routes of each agent and stores them in a tree. Thus no CNN are needed and the speed up for training the DDQN agent is minor.

In my experience the loading from CPU to GPU memory (Because flatland runs on the CPU) takes up to much time for learning speed up to be noticable. Using other approaches like A3C or using global observations and CNN’s would probably benefit from using the GPU.

Hope this helps.

Best regards,
Erik

First round submission with Flatland 1.0 or Flatland 2.0?

About 1 year ago

Hi @beibei

My suggestion would be to try a submission of the old format if you like to familiarize yourself with the submission process and have already developed your model.

We are working hard on pushing the necessary bug fixes to start round 2 submissions.

@beibei and @xzhaoma as stated before we would be very happy for contributions on the documentation and the getting started. We believe this is a very important part of the project and we believe that participants like you who have gotten started by yourself would be the best fit to write the introduction and getting started.

To coordinate the efforts please open issues in the baseline repos to let other participants know what you are working on. Then submit pull requests with you changes. Look at the Flatland main repository contribution guidelines here.

If you have any further question regarding contributions don’t hesitate to ask.

Best regards,

The Flatland Team

Baseline module import issue

About 1 year ago

Hi @xzhaoma

We really want this to be a community project where ideas grow together and we improve by ways that would be impossible if only a small team works on the project.

We would love to get your insights for many different topics. Therefore, I believe it is great if you contribute to the documentation directly through PR.

Best regards,
Erik

Future Malfunction Information

About 1 year ago

Hi @jonsafari

Thank you for your comment. In the current version of Flatland we do still provide this information to participants. It does indeed simplify the re-scheduling a bit as you already know in advance when you will need to re-schedule.

The task of real-time re-scheduling still remains however. At every new malfunction a random new time will be sampled and the novel situation needs to be analyzed and an optimal solution needs to be found.

The stochastic events in the current implementation are introduced to simulate realistic railway operations where a plan is made in advance and minor (or sometimes major) disturbances force you to re-schedule your traffic to keep traffic moving. Their effect is mostly to disturbe initial plans that can be generated out of the initial state of the environment. The time won by having a short view into the future (knowing in advance when the next malfunction happens) does simplifly the task at hand a little bit but still requires controlers to come up with new plans in a quick manner.

The current development of Flatland aims at making the challenge both accessible to Deep Learning / Machine Learning enthusiasts and Operations Researchers. Thus we formulate the tasks as a combination of planning and re-scheduling (short-time, real time planning).

I hope this clarifies our intentions. We are open for further inputs and suggestions on how we can improve Flatland.

Best regards,
Erik

Baseline module import issue

About 1 year ago

Thanks @xzhaoma

Merged your changes into the baselines.

Torch learning fails on latest baseline/flatland repo

About 1 year ago

Hi @xzhaoma

Thanks for reporting the bugs and running the tests. It seems odd that training would not result in higher percentages than 50%. The task with only one agent and no stochastic events should be solvable just like in earlier versions of Flatland. I will look into this and check that there is no bug in the tree observation or the observation normalization.

We will soon push a fix to the bugs in Flatland 2.0 which will again allow you to install through pip.

Best regards,

The Flatland Team

[ANNOUNCEMENT] Release Flatland 2.0

About 1 year ago

@ryznefil

Thanks for pointing this out. I updated the links now :slight_smile:

[ANNOUNCEMENT] Release Flatland 2.0

About 1 year ago

hi @jonsafari

Yes you can still make the round 1 task by setting the stochastic probability to 0 and not setting any specific velocity (This will result in speed of all trains to be 1.)

You can also still use the complex_rail_generator and complex_schedule_generator which will produce Round 1 tasks.

Best regards,
Erik

Torch learning fails on latest baseline/flatland repo

About 1 year ago

HI @marcoliver_gewaltig

It seems that the generator made a map with an illegal transistion map. I will look into this as well. Could you provide any futher information about what file you ran? and what parameters you used?

Best regards,
Erik

Torch learning fails on latest baseline/flatland repo

About 1 year ago

@xzhaoma

The second bug you reported is due to instabilities in the level generator. There were no Trainstations and thus no agents created in that environment.
We are working hard on fixing instabilities in the new level generator. Will push an update this week.

Best regards,
Erik

Baseline module import issue

About 1 year ago

HI @xzhaoma

Please send your fixes as PR. We are very glad for any community contributions to bring this challenge forward.

Best regards,
Erik

Torch learning fails on latest baseline/flatland repo

About 1 year ago

Hi @xzhaoma and @marcoliver_gewaltig

Thank you for brining these bugs to our attention. I have unfortunately been offline and haven’t had time to look into it. Our team is working on fixing it and we will provide an update asap.

We will also improve the documentation to make the challenge more accessible. We believe the problem at hand is very interesting and hope that many of you enjoy the challenging task.

Best regards

The Flatland Team

[ANNOUNCEMENT] Release Flatland 2.0

About 1 year ago

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Flatland 2.0 Release

The new version of Flatland Rail Environment is finally here. You can install it through

pip install -U flatland-rl

There are many changes under the hood so make sure to familiarize yourself with the new repository before updating your code and starting a first submission.

Whats New

Flatland is evolving to reflect trasportation systems challenges more realisticly. Therefore, two major constraints from real-world transportation systems where added and the railway network layout was moved closer to real network layouts. Here are the major changes:

  • Multi-Speed: Agents now travel at different speeds. For the scoring we will only be using 4 different speed levels. Speeds are represented in fractions and indication what fraction of a cell an agent can move at any time step. For example an agent with speed 1/2 needs 2 time steps to move to the next cell.
    • Fast Passenger Train: Speed 1
    • Slow Commuter Train: Speed 1/2
    • Fast Freight Train: Speed 1/3
    • Slow Freight Train: Speed 1/4
  • Stochastic Events: There are stochastic events introduced through a poisson point process which stop agents at random intervalls for varying durations. These stops simulate different incidents in railway networks such as delays and malfunctions. They are one of the main reasons why re-scheduling needs to be done in real time to keep traffic flowing without generating major delays.
  • Sparse Railway Network: The number of available ressources (cells with rails) has been reduced in this version with the new sparse_rail_generator and sparse_schedule_generator. This reflects the density of traffic on railway networks and increases the difficulty of planning ahead.
  • Schedule/Rail Generators: We updated the way levels are generated and seperated the tasks into generating the rail (rail_generator) and the schedule/tasks for agents (schedule_generator). You can find all relevant information here.

What this means for you

The changes will affect the way your controller has to interact with the environment. Make sure the take the following into account:

  • Your controller needs to monitor the state of the network during the roll out of each episode.
  • Agents can only make decisions upon cell entry (Fraction position smaller than epsilon), thus your controller only needs to decide for agents who are entering a new cell. What agents need to take an action in the next steps are indicated in the info returned by the step() function. More info here.
  • If you use RL agents make sure the include the correct states and actions. This is not handled automatically by the env as we want to provide full information. See here
  • Agents that are broken will signal this and provide information about malfunction duration (also in the info returned by step()). These agents resume travel once they are fixed automatically.
  • Important: The new more complex levels and the introduction of multi-speed and stochasticity mean that we can no longer guarantee feasibility of each instance. Your aim is still to get as many trains as possible to their final destination.

Submission scoring

We are currently working on improvements of the sparse_rail_generator() in order to finalize the scoring function for the next round. This means that submissions will start soon and there will be another announcement and an update on the Flatland landing page.

Submission format will be very similar to Round 1, you can assume the following constraints for the submission scoring:

  • Stochastic Events: Only a small fraction of agents will be affected by malfunctions (~10%)
  • Level dimension: The level dimensions will be very simlilar to Round 1. (x_dim, y_dim) <= (200,200)
  • Multi Speed: The different speeds will be as stated above.

Have fun with this new Flatland version!

The Flatland Team

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Baseline module import issue

About 1 year ago

:girl::adult::blonde_man::man::bearded_person::man_red_haired::man_curly_haired::man_white_haired::man_bald::woman::blonde_woman::woman_red_haired::woman_curly_haired::woman_white_haired::woman_bald::older_adult::older_man::older_woman::girl::adult::blonde_man::man::bearded_person::man_red_haired::man_curly_haired::man_white_haired::man_bald::woman::blonde_woman::woman_red_haired::woman_curly_haired::woman_white_haired::woman_bald::older_adult::older_man:

Friendly reminder: There is a community prize (:moneybag:) awarded to great community contributions such as documentations to get started, new observation builders, predictors, sharing of ideas and in general just helping the community advance with this research question…

:girl::adult::blonde_man::man::bearded_person::man_red_haired::man_curly_haired::man_white_haired::man_bald::woman::blonde_woman::woman_red_haired::woman_curly_haired::woman_white_haired::woman_bald::older_adult::older_man::older_woman::girl::adult::blonde_man::man::bearded_person::man_red_haired::man_curly_haired::man_white_haired::man_bald::woman::blonde_woman::woman_red_haired::woman_curly_haired::woman_white_haired::woman_bald::older_adult::older_man:

Baseline module import issue

About 1 year ago

Thank you @xzhaoma for your valuable feedback.

We have set up some testing for the package and are constantly trying to improve it such that everything works just fine.

We are very happy for valuable feedback like yours such that we improve our repository and make the challenge accessible to as many enthusiasts as possible.

If you like you are also very welcome to contribute directly to the baselines repo and help others get an easier start.

All the best,

The Flatland Team

Baseline module import issue

About 1 year ago

Hi @xzhaoma

Thank you for your feedback. We have now merged the beta version into the master branch. There will be an announcement this weekend and a new release of Flatland 2.0.

I will update the baselines repo as soon as possible.

Do you have any recommendation for the get-started part`? We also believe that this is a very important part and are happy for any helpful input.

Best regards,
Erik

Baseline module import issue

About 1 year ago

ATTENTION: The stock observation builders and predictors are not yet ready to be used. Be sure to use your custom obersvations and adjust them to the new Flatland properties such as stochastic events and different speeds.

These functions will be updated in the next few days.

Baseline module import issue

About 1 year ago

Hi @xzhaoma

The modules you are trying to import are only present in the beta version of Flatland. You have to clone the git repository and build it from source.

Hope this helps.

Best regards,

Erik

PS: If you are using the new beta repository you can also try out sparse_rail_generator and sparse_schedule_generator to be ready for Round 2 of the challenge.

Communication among agents

About 1 year ago

Hi @giulia_cantini

Currently you have to implement the communication between agents outside of the flatland env. If there is a need for support of communication channels within Flatland we are happy to look into thi.

If this is something you need please open ans issue. You are also very welcome to help us implement this feature and thus contribute to the community.

The Flatland Team

[ANNOUNCEMENT] Flatland 2.0 *Beta*

About 1 year ago

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Hi All Flatland enthusiasts

We are intensively working on Flatland 2.0 and all the upcoming changes for Round 2. We want to do all this work as transparent as possible to facilitate interactions and usage of the environment.

Therefore we wrote a brief introduction to Flatland 2.0 where you can test your code on this BETA version. We encourage you to report any bugs or suggestions in the issue tracker. This helps us improve the repository and give you a great challenge to solve.

The introduction to Flatland 2.0 beta can be found here.

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Test case 532 evaluation problem

About 1 year ago

Hi @ryznefil

In order to allow for a fair competition we do not release any of the test environment used to score a submission.

We did however internally run a test on the said test levels to see if they are feasible. Our own algorithms are aible to solve those specific instances. We will also have a look to see why your submission failes at this specific level and provide you with the appropriate debugging infromation.

Hope this helps to even improve your great solution further :slight_smile: .

Best regards

The Flatland Team

[ANNOUNCEMENT] Flatland 2.0 Pre-Information

About 1 year ago

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Hi All

We are working intensively on the next version of Flatland and assure you that it will be a fun challenge.

In order to mimic real world challenges, faced by real world railway companies, we involved experts from operations within SBB to improve the next version of Flatland.

We are currently implementing the last features and are hoping to release the next version of Flatland soon and thereby start Round 2.

To simplify the use of your current code for the next round and prepare any necessary adjustements we would like to announce the preliminary standings of the development of Flatland 2.0. You can expect to see the following changes in the upcoming release:

  • Much sparser railway network: Realistic railway networks where complex network nodes are connected through single railway connections increase the difficulty of optimal solutions.
  • Environment width and height will stay in the same order of magnitude (no to minor changes)
  • Number of agents in the environment will stay the same. But the limited railway network will lead to a higher density of Trains.
  • Different Speed Profiles: Each agent will move at a different speed (approx 4 speed profiles). Each agent can only stand still or move at its max speed. The fastest agent moves one cell per time step, all slower agents only move fractions of a cell. Agents can only chose an action when entering a new cell, the intermediate steps are executed automatically and an action is only required at the transition to the next cell.
  • Stochasticity during episodes: A malfunction model has been implemented which overrides agent actions with a certain probability (Poisson Procces). Malfunctions always lead to agents stopping. Frequency and duration of malfunctions are currently being investigated to provide a feasible level of complexity.

We will also update the stock observation builders* and predictors to incorporate these changes. This should help you getting started with your code.

We are also working on a FAQ Page where all questions from the discussion forum will be collected and answered.

As soon as we have finalized the changes in Flatland 2.0 we will be notified and the Challenge Page will be update with the final details.

In the meantime don’t hesitate to reach out to us with any requests or questions, we are here to help.

The Flatland Team

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Here is a sneak peak at early build of Flatland 2.0 (Gif to follow soon :wink: ):

[ANNOUNCEMENT] Updated Rules and Clarification

About 1 year ago

Hi @kalinja6

Thank you for your message. We agree that more information would be great for the whole community.
We have had many discussion with the people involved in planning and scheduling here at SBB in order to formulate the challenge as close to real life scenarios as possible.

We are currently working on these modifications and I will update the landing page, add a FAQ and make an Announcement here in the forums tomorrow about round two and what the new enhanced difficulties will be.

Thank you for your patience and your participation in this challenge.

Feel free to reach out to us if there is any further questions or suggestions.

Best regards,

The Flatland Team

Baseline installation

About 1 year ago

To view the performance of your agents. I suggest you use multi_agent_inference.py for your multi agent example (just load the correct network file from training) or uste render_agent_behavior.py for single agent behavior. If this does not work, don’t hesitate to reach out. We are here to help.

best regards,
Erik

Baseline installation

About 1 year ago

Hi @xzhaoma

Are you doing the single agent training or the multi agent training? Thanks alot for your PR. It’s merged into master now

[Introduction] Getting started with Flatland

About 1 year ago

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Hi All

I wanted to give a short introduction into the challenge to help newcomers have a start and evolve their algorithms from there.

To install Flatland follow there instructions here.

Once you have installed Flatland you can start playing around with the environment to get to know it and start developing your own algorithms.

The aim of the Flatland challenge is to find a controller that manages the traffic on any railway network as efficiently as possible, without creating any conflicts. You can achieve this by a number of different approaches.

  • Optimization algorithms can be used to find optimal solutions for any given situation. However, you need to be aware that in Round 2 there will be stochastic events happening during an episode so you need to be able to re-schedule very quickly.
  • You can use machine learning approaches (Reinforcement Learning is best suited here). For this we have written two basic introductions.

These introduction will give you a general feel on how to use predictors and observation builders and how to implement your own first DQN Controller.

Once you have understood what the observation builder and predictors do and you get a first feeling of why your agent can’t solve all the instances it is up to your creativity to improve on observation builders and predictors by building you custom ones.

Brief statement about the two:

  • Predictor: The predictor is used because railway re-scheduling is not a local problem to be solved but rather needs some short term planning. To avoid conflicts you need to be able to anticipate the conflicts in advance. The stock predictor of flatland predicts that every train will take it’s shortest path at any time. This however will not be true, so it would be better to come up with ways to better predict, or learn to predict the future positions of the trains.

-Observation Builders: Flatland comes with three Stock Observation Builders that help you get an idea which features are relevant for a controller to solve the task. Most good performing controllers currently use the Tree Observation or modifications of it. Playing around with this observation you will quickly realize its limitations and that you need to enhance your Custom Observation Builder to include any relevant features to solve the task efficiently.

We suggest that you also try the Editor to debug your model and create problems for training your agents.

If you have any further question on how to get started don’t hesitate to reach out to the team. And we encourage you to have open discussions here in the forum about your ideas and problems.

The Flatland Team

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

DDQN model submission failed

About 1 year ago

I actually did my submission using the cpu only installatino of pytorch. Then it should work out of the box without any changes.

DDQN model submission failed

About 1 year ago

Hi @Paseul

I’m sorry to hear that your submission did not work. Using pytorch-cpu should work. I made my submission this way. I propose the following steps to resolve the issues:

  1. If you are running on a CPU-only machine, please use torch.load with map_location='cpu' to map your storages to the CPU`
  2. if this does not work. Uninstall pytorch in your conda environment. Then run the following command in your conda env conda install -c pytorch pytorch-cpu. Once this is done you have to update your environment.yml as explained in the starter kit.

Please let me know if this resolves your issues.

Best regards
Erik

Baseline installation

About 1 year ago

Hi @xzhaoma

Thanks for brining this to my attention and the proposed solution.
Will update the rendering to the new version.

Best regards,
Erik

Baseline installation

About 1 year ago

Hi @xzhaoma

Thank you for your reply. I updated the training introduction. Will also see if we can simplify this further by automatically installing the dependencies.

Hope this clarifies it.

Best regards,
Erik

Baseline installation

About 1 year ago

Hi @xzhaoma

The baseline Repo does not need to be installed. Once you have installed Fltaland-RL by following the quick start instructions you can just clone the baseline repo to a desired location and run the code from that folder.

Hope this helps and have fun with the challenge :steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::grinning:

Best regards,
Erik

Simple DQN training navigation example

About 1 year ago

Hi @xzhaoma

Happy to hear that you joined the challenge. The introduction i wrote is using pytorch for the deep learning agent (DQN). If you are using conda you can install it by running this command in your conda environment: conda install -c pytorch pytorch-cpu. Note that this however is only the cpu version, which is sufficient for the introduction. If you need GPU support follow the instruction on the torch homepage.

If you are not using a conda setup yet I suggest you set it up following the instructions here: Flatland-Challenge-Starter-Kit.

Hope this helps. Otherwise don’t hesitate to ask if you get stuck.

Best regards,
Erik

Additional programming language

About 1 year ago

Hi @ryznefil

Sorry for the delayed response. This is of course possible as long as you can call the code from within run.py .

Best regards,

Flatland Team

Unable to run train_experiment.py in baselines/RLLib_training

About 1 year ago

Hi @ai-team-flatland-net

Yes this has changed. Since the split-tree is just a utility function it was moved out from the tree observation. Just be sure that you update all the variables such as num_features_per_node so that the split tree does split the different values correctly . Check out how it can be used in the baselines repo:

Hope this helps resolving your issues.

Best regards,
Erik

[ANNOUNCEMENT] Updated Rules and Clarification

About 1 year ago

Hi @mugurelionut

This was an expected outcome and one of the reasons we chose to do the challenge setup with these distinct rounds. We highly encourage the use of both classical OR algorithms as well as novel RL solutions and thus set round 0 to be well suited for OR algorithms, whereas round 2 with introduced stochasticity is better suited for learning algorithms.

Round 1 instances can be solved using classical planning algorithms, because there is no stochasticity involved in the enivornment and the complexity is rather low due to the speed of all trains being equal. This round serves as a kind of benchmark for RL approaches to see if they can achieve similar results as classical approaches.

Round 2 will differ from round 1 in that there are stochastic events occuring during an episode, so pre-planned trajectories will not be feasible anymore. Also the introduction of different speed profiles for different agents will enhance complexity.

I understand that tuning the optimality for round 1 is not the most interesting challenge. I therefore encourage you to prepare your code for stochastic events such as:

  • blocked cells forcing agents to reroute
  • delayed departures causing agents to start later than expected
  • unexecuted actions causing agents to replan their actions

We are still aiming for mid August release and will update you on more details about how stochastic events will be represented and with what probability they will occur.

We also encourage you to discuss among other participants how you solved the task and share your code publicly (there is also a community prize being awarded for contributions to the whole community).

If you have code that can be integrated into the FLATLAND to help everybody obtain better results don’t hesitate to contact us or open an issue on gitlab.

Best regards
Erik

[Discussion] Planning vs Re-scheduling

About 1 year ago

Planning vs Re-Scheduling

I would like to start a discussion about the differences between planning and re-scheduling and how they can be used to tackle the challenge at hand.

Planning

In railway (and in other transportation or logistics systems), this is done before the actual operation day and serves as a good starting point for operations. Finding an optimal plan that minimizes travel times, delays and costs can be done using many state-of-the-art OR algorithms (scales of real world problems are often the problem…).

For daily operations however, many unforeseen events occur and operations often deviate from the optimal plan. Thus another very important aspect of a good schedule/plan is robustness. In contrast to generate optimal schedules, the generation of stable/robust schedules is much more complex. Stability of a schedule can only be determined when the capabilities of re-scheduling are known, meaning how fast and reliable can we re-adapt our schedule.

Re-scheduling

Due to the many unforeseen events occurring in daily train operations, re-scheduling becomes essential to keep traffic moving efficiently. With re-scheduling we react to stochastic events that cause disturbances in the traffic network and make the initial schedule/plan infeasible.
Because re-scheduling cannot be pre-processed before the day of operations like the planning phase, there are harder constraints on computational time.
Re-scheduling algorithms need to handle uncertainties and come up with new plans for the transportation network, or at least the disturbed parts of it, in order to minimize delays occurring in the system. Because traffic is still running while the re-scheduled plan is being computed each found solution is only valid for a very brief time frame before it is outdated due to a changed state of the network.

Flatland Challenge and Re-scheduling

The FLATLAND challenge is set up to test different hypotheses around planning and scheduling. Many of you have probably realized that Round 0 and Round 1 can be solved by pre-computing optimal schedules and then just replay them in the environment. These are very good solutions and are one important step in the process of re-scheduling.
For Round 2 however, there will be a higher complexity due to the introduction of differential speeds as well as the occurrence of stochastic events during operations. Specially the introduction of stochasticity means that you’re algorithms need to observe the state of the environment constantly or at least when deviations from the expected environment state occur. In the cases where these changes occur, you will need to engage with the environment to resolve the issues. This is very similar to real world railway traffic management. Thus, your contributions to this problem might impact the way traffic management systems work in the future.

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Further questions regarding submissions

About 1 year ago

Hi @mugurelionut

You are correct with your assumption about the max number of steps allowed in Round 1.

The rail environment will terminate a episode at step = 1.5 (width+height).

ATTENTION: If you plan to use the max number of steps per episode in your code, be sure to make it variable as this is likely to change for the more complex Round 2.

[ANNOUNCEMENT] Updated Rules and Clarification

About 1 year ago

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

ANNOUNCEMENT

We have updated the rules to clarify how scoring and final ranking will be done in this challenge.

tl;dr : Prizes will be awarded according to ROUND 2 Leaderboard only

Why the different rounds?

We use the different rounds to introduce the complex problem of vehicle re-scheduling in a step by step manner.

Round 0 (Beta):
This round was intended to find remaining bugs in the flatland environment as well as give a simple introduction to the repository.

Round 1 Avoiding Conflicts:
In contrast to other means of transportation, a railway network is prone to dead-locks, where trains are unable to resolve the conflict. This round is intended for participants to figure out how to detect such dead-locks in a timely manner and avoid them.

Round 2 Optimizing Traffic:
The complexity of this round is far superior to the previous rounds. Introducing mixed speed profiles for different trains in the network leads to new traffic management problems. In addition to avoiding dead-locks the ordering of trains on a given route as well as the chosen routes will impact the punctuality of the trains. This round is very close to the real world problem faced in daily operations by any railway company. Given the Complexity of this last round in contrast to the previous rounds it outweighs them in the final scoring. Therefore only this round is considered for the final scoring.

We clarified this in the updated rules as there where two conflicting statements in the previous version of the rules.

We wish you a lot of fun with the challenge and encourage you to openly discuss ideas on how to tackle this real world problem with novel approaches.

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Further questions regarding submissions

About 1 year ago

Hi @RomanChernenko and @mugurelionut

Thanks for your replies. There will be an announcement later today and an update to the rules to clarify how submission scoring works. And how prizes are awarded.

best,
Erik

Further questions regarding submissions

About 1 year ago

Hi @RomanChernenko

Thanks for your questions here are the answers:

  1. Each submission has 4 CPU and 16GB of RAM, currently no GPU (@mohanty please update if we have changes here )
  2. The score is mean percentage of agents who solved their own objective (arrived at their destination). E.g if half the agents arrive at their target in time the score is 0.5. Because there might be several submissions with the same score we also compute the mean reward which is just the mean reward over all agents and all episodes. (reward = -1 for each step, reward = 0 for agent at its target, reward = 1 for all agents if all agents reach target)
  3. The scores will not be combined. Only scores from round 2 will be considered for the prizes.
  4. For round 1 the max number of trains is set to 50 at a size of 100x100.

Further questions regarding submissions

About 1 year ago

Hi @ryznefil

I’m happy to answer your questions:

  1. Yes you can use any algorithm you would like to use. Keep in mind that computational complexity will vastly increase in round 2 as we allow for different train speeds.
  2. Currently you have 8 hour computational time limit to solve 1000 environments. Also if there is no action performed by your controller for 15 minutes the submission scoring will be aborted. (@mohanty anything more to add here?) (Update from @mohanty : you will have access to 3 cores of CPU and 8 GB of RAM. No GPUs are available at this point of time. )
  3. Yes you can modify you run.py script . For example: You can pre-compute all the action you want to do (e.g. as a list) and your controller just provides the appropriate actions to the environment at each step. The environment was built with reinforcement learning in mind and thus the action at each step is needed. Therefore OR approaches need to do a little hacking to change their results into lists of actions.

[ANNOUNCEMENT] Submissions Open

About 1 year ago

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Hi All

We are now all set for your submissions.
Please follow the instructions in the starter-kit to get your submission started.

ATTENTION for Windows users: Please use WSL to avoid windows dependencies. We will update you as soon as we have another solution ready.

The Flatland Team :star_struck:

:steam_locomotive::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car::railway_car:

Is teams allowed?

About 1 year ago

Hi @Iewtun

It is very difficult to detect if participants are exchanging information privately. If we detect that participants are using multiple accounts to bypass the submission limit, this is considered malicious activity and will lead to disqualification.

We therefore encourage you to openly exchange on the forums as well as share open source code such as observation builders and predictors for the challenge.

It is important to note that we will also be awarding the best community contribution. This is in the form of new observations, better predictors, or anything that helps the community evolve and achieve better results.

I hope this clarifies your question.

Best regards
Erik

Is teams allowed?

About 1 year ago

Hi Lewis

  1. There is no maximum Team size for this competition. Read the rules to understand everything about the prizes as these will not be adapted to team size. (e.g. Travel grants not granted to all team members, prize awarded to team lead --> Distribution is up to your team leader)
  2. You can share your data outside of your team. We encourage to do so openly to foster collaboration and great advancements in this field of research. As long as there is no licencing or ip violation this is fine.
  3. You can merge your teams. There needs to be defined a new team lead and your old team submissions will be removed and not considered in the ranking. Get in touch with us if you want to merge a team.

Hope this answers all your questions.

Good luck and have fun :slight_smile:

Baselines show improved performance

About 1 year ago

Hi All

I’m still working on the Baselines Repo, but the agents start showing more complex behaviour.

Here I use a dueling double DQN agent with the tree based observation and the shortest path predictor as explained in the baselines repo.

Any suggestions how we can improve the observation or predictor to get even better performance? Happy to discuss ideas

Will the worlds our submissions will be evaluated against be known or unknown to participants

About 1 year ago

Hi @tim_b_herbstrith

The agents will be evaluated on a β€œsecret” test set, of different env dimensions and number of agents. This is done to ensure that the solutions generalize well. We will however, release a set of generated β€œLevels” that are similar to the once used for evalution.
This will happen soon so stay tuned.

ANNOUNCEMENT: Beta Prolonged

About 1 year ago

ANNOUNCEMENT: We increased the duration of the beta round for one week.

Round 1: Avoid conflicts will start on Monday 22 July with the release of the next version of Flatland.

Stay tuned and have fun exploring Flatland. Please report any bugs you find here.

The Flatland Team

Error in baseline

About 1 year ago

Hi Mrugankakarte

It should now be fixed in the baselines repo. If it is not working yet then you have to clone the git repo of flatland and install it. We will push a new version of flatland soon. All fixes will be in there.

best regards,
Erik

Error in baseline

About 1 year ago

Hi mrugankakarte

In training_navigation.py we only use num_features_per_node to set the dimension size for the agent. I will also fix the attribute error. I saw it was implemented wrongly in the observation builder.

best regards

Erik

Questions about cell details and targeted size of the world

About 1 year ago

Hi Marc-Oliver

Thanks for the great questions. Here are some short answers:

  1. In this challenge all trains have the same departure time. All agents depart when the first env.step() is executed. This will only change in future challenges and versions of flatland
  2. Stations can be in any cell. They are often chosen in dead-end cells for feasibility. If an agent enters its station it cannot move anymore. Thus station in the middle of lines will block the line once the agent reaches the station.
  3. A cell capacity is equal to one. This means there can never be more than one train at a given cell!
  4. For scoring the submissions there will be a max_size and a max_nr_agents. Current estimates will be a max_size = 100x100 with max_nr_agent=100. This can however still change as we are running tests now.

Hope this infromation helps.

Best regards,
Erik

Can trains change their direction?

About 1 year ago

Hi Marc-Oliver

The agent cannot change the direction of movement in any cell. The absolute direction/orientation of the agent changes at curves, switches. The only cell type where the agent changes direction of movement relative to the rail is at the dead-end cell. If the agent moves forward when facing the dead-end it will be turned around 180Β° and exits the cell where it entered the cell.

Hope this helps

best regards
Erik

Simple DQN training navigation example

About 1 year ago

Hi All

If you like to see a simple example on how to get started with training on flatland.
Have a look at the simple training_navigation.py file in the baselines repository.

I wrote a simple introduction markdown. Feel free to comment if something is not clear.

Erik*

Is teams allowed?

About 1 year ago

Hi Vitaly

Nice to have you on our challenge :slight_smile:.

Yes teams are allowed, please see ressources page for details.

Looking forward to your submissions.
Erik

Unable to install Flatland

About 1 year ago

Hi,

We have had some issues with Cairo and are trying to resolve this issue. We will get back to you as soon as possible with a fix.

Please try to install cairo seperatly under this Link

The widget is only used if you want to draw your own levels in Flatland using a Jupyter notebook.

You can install the widget through the instructions here :Jupyter Canvas Widget

I did the developer install on windows 10 and it worked. You need to run the commands in with administrative rights in the console.

Let me know if this worked. Happy to help otherwise.

Erik

Mutli Agent Setup

About 1 year ago

Response by @mohanty :

Hi Panayiotis,

Thanks for your interest in the library and the competition, and for raising for valid points.

I agree, that the goal is to encourage solutions that are de-centralized, scalable. But I have mixed feellings about leveraging only local information available to the agents, when in almost all the cases, the agents can have access to global information if their is a demonstrated use for the same.

The good point about local obserrvations (small grid window around the agent) is that its clearly scalable, but the downside is that there might not be enough information for agents to do long term planning in larger grids (especially when we havent introduced the ideas of communication etc). The good point about tree based observation is that it makes much easier to train the agents by providing much denser information to the agents, but the downside is, I personally feel that this is pushing us in the direction of handcrafting-features and in turn, also subconsciously biasing the direction of thoughts for people doing research with the library : one of the reasons why I have always been a very vocal internally against using the tree observations. On the other hand, Erik is the person who came up with the first implementation and successful experiments using the same, and is quite a supporter of the tree based observations : which work !!

The global observations are a different story though !!

The only reason I love global observations is that they are -generalizable-, and at the same time (atleast theoretically) provide all the information needed to enable complex emergent coordination behavious among agents.

For smaller grids, ofcourse, having access to global observations makes the problem trivial, and in many cases, with even old school agent based approaches, you might solve the task at hand. But as soon as we start scaling to grid worlds of say order of 10,000x10,000, then IMHO, global observations are the most promising observations to train generalizable agents : because thousands of agents with small local observationos will not have enough information. And the tree based observations will start becoming restrictive expensive to compute ! But if we have a generalisable solution of how to digest the global observation, then that could actually scale.

One idea (Which we havent yet experimented with), was imagining layers of convolutions which convert the global observation into much dense (and lower dimensional) representations which can then eventually be used for your policy explorations, then an approach like that could easily scale to much larger grids !!

Hence, we agreed to let the global observation stay with the hope that there are enough incentives (And information) available to participants who might want to explore generalisable solutions when we increase the grid sizes to much higher values.

Mutli Agent Setup

About 1 year ago

Question from Panayiotis :

Given that the goal of the Flatland challenge is to develop multi-agent reinforcement learning techniques, it stands to reason to encourage solutions that are decentralized, scalable, and utilize only local information provided to each train. Reading the description, though, it seems that this is not the case. I would argue that solutions which only utilize β€œtree” or β€œlocal” observations should be favoured over solutions which use β€œglobal” observation.

Clarifications on Tree Observation

About 1 year ago

Thank you for your feedback. I will try to clarify to comments on the observation builder to make it more understandable.

The distance is calculated from the current agents position. Thus, it tells the agent how far away the observed target of the other agent is from the current location of the observing agent.

The current node is the one being constructed in the tree observation. It is the next branching point along the agents path. I will update the comment soon and also implement an example on how to use it.

Also I will make the baselines Repo publicly available so you can see how to use the observation in a training setup.

Clarifications on Tree Observation

About 1 year ago

I need some guidance from you regarding the observation part.

I’m trying to understand how the observation is constructed from the β€œ flatland.observation.get(handle) ”, but I have difficulties understanding it from the comments. I’m not sure if I can find more details/examples somewhere else.

Some update is needed to the comments, due to the changes from 5 features to 8 features.

Moreover, I think a little bit clarification in the terminology would be really helpful for the readers and the challenge participants

For Example #2: β€œ distance to a target of another agent is detected between the previous node and the current one .” Γ  is the calculated value the distance to another agent’s target? What does exactly the previous node mean?

Is the Current node where the agent is located, or it is the node that we are getting the features for?

Observations for Flatland

About 1 year ago

Thank you for your feedback. I will try to clarify to comments on the observation builder to make it more understandable.

The distance is calculated from the current agents position. Thus, it tells the agent how far away the observed target of the other agent is from the current location of the observing agent.

The current node is the one being constructed in the tree observation. It is the next branching point along the agents path. I will update the comment soon and also implement an example on how to use it.

Also I will make the baselines Repo publicly available so you can see how to use the observation in a training setup.

Observations for Flatland

About 1 year ago

Asked by Farahd:

I need some guidance from you regarding the observation part.

I’m trying to understand how the observation is constructed from the β€œ flatland.observation.get(handle) ”, but I have difficulties understanding it from the comments. I’m not sure if I can find more details/examples somewhere else.

Some update is needed to the comments, due to the changes from 5 features to 8 features.

Moreover, I think a little bit clarification in the terminology would be really helpful for the readers and the challenge participants

For Example #2: β€œ distance to a target of another agent is detected between the previous node and the current one .” Γ  is the calculated value the distance to another agent’s target? What does exactly the previous node mean?

Is the Current node where the agent is located, or it is the node that we are getting the features for?

Reporting Bugs in Flatland

About 1 year ago

Thanks for your interest in the library and the competition, and for raising for valid points.

I agree, that the goal is to encourage solutions that are de-centralized, scalable. But I have mixed feellings about leveraging only local information available to the agents, when in almost all the cases, the agents can have access to global information if their is a demonstrated use for the same.

The good point about local obserrvations (small grid window around the agent) is that its clearly scalable, but the downside is that there might not be enough information for agents to do long term planning in larger grids (especially when we havent introduced the ideas of communication etc). The good point about tree based observation is that it makes much easier to train the agents by providing much denser information to the agents, but the downside is, I personally feel that this is pushing us in the direction of handcrafting-features and in turn, also subconsciously biasing the direction of thoughts for people doing research with the library : one of the reasons why I have always been a very vocal internally against using the tree observations. On the other hand, Erik is the person who came up with the first implementation and successful experiments using the same, and is quite a supporter of the tree based observations : which work !!

The global observations are a different story though !!

The only reason I love global observations is that they are -generalizable-, and at the same time (atleast theoretically) provide all the information needed to enable complex emergent coordination behavious among agents.

For smaller grids, ofcourse, having access to global observations makes the problem trivial, and in many cases, with even old school agent based approaches, you might solve the task at hand. But as soon as we start scaling to grid worlds of say order of 10,000x10,000, then IMHO, global observations are the most promising observations to train generalizable agents : because thousands of agents with small local observationos will not have enough information. And the tree based observations will start becoming restrictive expensive to compute ! But if we have a generalisable solution of how to digest the global observation, then that could actually scale.

One idea (Which we havent yet experimented with), was imagining layers of convolutions which convert the global observation into much dense (and lower dimensional) representations which can then eventually be used for your policy explorations, then an approach like that could easily scale to much larger grids !!

Hence, we agreed to let the global observation stay with the hope that there are enough incentives (And information) available to participants who might want to explore generalisable solutions when we increase the grid sizes to much higher values.

Reporting Bugs in Flatland

About 1 year ago

Given that the goal of the Flatland challenge is to develop multi-agent reinforcement learning techniques, it stands to reason to encourage solutions that are decentralized, scalable, and utilize only local information provided to each train. Reading the description, though, it seems that this is not the case. I would argue that solutions which only utilize β€œtree” or β€œlocal” observations should be favoured over solutions which use β€œglobal” observation.

Reporting Bugs in Flatland

About 1 year ago

Have you found any BUG in the Flatland environment code or have a request for an essential FEATURE? Please let us know so we can fix and enhance Flatland to become a great multi-agent environment.

Observations for Flatland

About 1 year ago

Feel free to check out our baseline training code to get an understanding of how to use the different observations: Baselines

Observations for Flatland

About 1 year ago

One of the main difficulties for this multi-agent problem is to define a suitable observation with allows for great scalability of the algorithms as well as generalizability across many problem instances.

We have provided you with three basic observations: Global, Local Grid and Tree Observation. And we encourage you to build on these or invet novel observations that help solve this problem.

This Thread is used to discuss different aspects, benefits or drawbacks of the provided observation and help us collaboratively come up with better solutions.

Here are the base observations provided by us: Observation builders

mlerik has not provided any information yet.