Activity
Ratings Progression
Challenge Categories
Challenges Entered
Multi-Agent Reinforcement Learning on Trains
Latest submissions
See Allfailed | 93962 | ||
graded | 89220 | ||
graded | 89210 |
Classify images of snake species from around the world
Latest submissions
See Allgraded | 9963 | ||
failed | 9876 | ||
failed | 9775 |
Multi Agent Reinforcement Learning on Trains.
Latest submissions
See Allfailed | 32805 | ||
failed | 32778 | ||
failed | 32758 |
Latest submissions
See Allgraded | 60255 | ||
failed | 60250 |
Latest submissions
See Allfailed | 60273 | ||
graded | 60271 |
Badges
-
Editor
-
TrustableHas filled their profile page
-
Anniversary
-
First Mention
-
Welcome
-
Enthusiast
-
First Emoji
-
First Onebox
-
Basic User
Participant | Rating |
---|---|
![]() |
89 |
Participant | Rating |
---|---|
![]() |
89 |
-
inside-job FlatlandView
Flatland

Current status of imitation agent in baseline repository
3 months agoThe imitation trainer works. We have generated results for them. You could do a training and simultaneous evaluation using the script
train.py -ief baselines/custom_imitation_learning_rllib_tree_obs/ppo_imitation_tree_obs.yaml --eager --trace
(drop -e
flag if you don’t want to do evaluation)
The only thing is the OR expert solution uses was for an older flatland version where the malfunction rate was different. So if you are training with malfunctions, you can workaround it by doing the below changes in the flatland source code
change below line in method malfunction_from_file in the file flatland.envs.malfunction_generators.py
mean_malfunction_rate = 1/oMPD.malfunction_rate
The documentation here https://flatland.aicrowd.com/research/baselines/imitation_learning.html is a bit old , we will update it soon.
You can refer to this Google Colab notebook also which has the details along with the results https://colab.research.google.com/drive/1oK8yaTSVYH4Av_NwmhEC9ZNBS_Wwhi18#scrollTo=P_IMrdL27Ii7
Let me know if you are facing any issues.

RLLib Baselines on Colab!
3 months agoWe have taken the repo from https://gitlab.aicrowd.com/flatland/neurips2020-flatland-baselines
and made it into a simple colab notebook
All training scripts are also provided, so one can modify the configs and do runs of their own. Evaluation is also run and the script to calculate scores on an independent test set is also provided.

Using a trained agent in RLlib
4 months agoYour approach seems correct in principle … not sure why the trainer cannot restore from checkpoint. You could compare with the example provided.

Using a trained agent in RLlib
4 months agoyou can refer to the rollout.py
script in the AIcrowd baselines for flatland
And the corresponding script
Note that this runs small environments with a custom seed. You will have to change the environment logic for your purpose.


Expert demonstrations for Imitation Learning: Recreating Malfunctions
4 months agoThe flatland-rl version has been updated to 2.2.2. (Upgrade it using the command pip install -U flatland-rl
). Can you check if the malfunctions are replicable with the same seed? Let us know if you are facing any issues.

Expert demonstrations for Imitation Learning: Recreating Malfunctions
4 months agoIt does it slightly different from the MARWIL/Apex-DQfD versions in that it runs every episode alternatively via IL and RL (the ratio is defaulted to 50% ratio but it can be changed and also decayed over time by changing the configs).

Expert demonstrations for Imitation Learning: Recreating Malfunctions
4 months agoYou could also try our online RL Solution which does not require any of these intermediate steps like generating experiences. It runs everything on the fly…
You can find a pure IL and IL + PPO solution here
We haven’t documented it but we will do it soon. It uses the last year’s 2nd place solution from CkUA. Unfortunately, it was from an earlier flatland version, so as of now you have to change the malfunction behaviour as per previous versions as follows
change below line in method malfunction_from_file in the file flatland.envs.malfunction_generators.py
mean_malfunction_rate = 1/oMPD.malfunction_rate

Expert demonstrations for Imitation Learning: Recreating Malfunctions
4 months agoAre you using the same flatland versions for both creation and loading environments? The solution for creating the experiences in the AICrowd baselines for MARWIL and APE-X DQfD were mostly used in environments without malfunctions and they used the seed value of 1001 (https://flatland.aicrowd.com/research/baselines/imitation_learning.html).
Flatland Challenge

Solution Codes and Approaches
About 1 year agoI haven’t submitted it yet. But I can share the results of few envs from the local evaluation
Evaluation Number : 1
Reward : -43.83333333333334
====================================================================================================
Evaluation Number : 1
Current Env Path : ./test-envs/Test_5/Level_1.pkl
Env Creation Time : 0.42258167266845703
Number of Steps : 1120
Mean/Std of Time taken by Controller : 0.01895513470683779 0.0018288652394336735
Mean/Std of Time per Step : 0.1480530451451029 0.00961317459096298
Evaluation Number : 2
Reward : -43.41666666666668
====================================================================================================
Evaluation Number : 2
Current Env Path : ./test-envs/Test_3/Level_0.pkl
Env Creation Time : 0.29577183723449707
Number of Steps : 960
Mean/Std of Time taken by Controller : 0.02316151708364487 0.002471594250438426
Mean/Std of Time per Step : 0.14834324022134146 0.01293447105111391
Evaluation Number : 3
Reward : -52.00000000000002
====================================================================================================
Evaluation Number : 3
Current Env Path : ./test-envs/Test_6/Level_0.pkl
Env Creation Time : 1.5495717525482178
Number of Steps : 1760
Mean/Std of Time taken by Controller : 0.02436668398705396 0.0039495335690074304
Mean/Std of Time per Step : 0.19943869560956956 0.024274925544436107

Solution Codes and Approaches
About 1 year agoI have added another code file with a different approach that does not use a model.
The code can be found in the local Github location
For a simple demonstration of how we solve a dense railway network, simply run the file
MultipleAgentNavigationObsConflict.py.
This file does not use any additional packages other than the ones required for flatland and can be run with the latest flatland-rl version 2.1.10

Solution Codes and Approaches
About 1 year agoI have put up some code here
This includes actorcritictrainer.py file which implements an actor critic approach and ESStrategyTraining.py which implements an evolutionary strategy approach.
The results seem to be similar to the Duelling Double DQN approach. I have saved sample results and pre-trained weights.
This has been done using stock observations.
Adding to Erik’s comments, my observations are
- These models do not show improvement even after training for longer periods and show comparable performance, suggesting that we need to do better feature engineering.
As of now, I next plan to do some visualizations and add documentation to the code to better.
Any comments/suggestions are most welcome.

Submission Errors Flatland
Over 1 year agoI am getting an error on evaluation
https://gitlab.aicrowd.com/nilabha/flatland-challenge-starter-kit/issues/6
Can yo help with the logs

Submission Errors Flatland
Over 1 year agoI am not able to see logs but I can see comments on the issue though only the first line.
2019-08-03T15:15:13.396985671Z Traceback (most recent call last):…

Submission Errors Flatland
Over 1 year agoI am getting an error in evaluation
https://gitlab.aicrowd.com/nilabha/flatland-challenge-starter-kit/issues/3
Somehow I cannot see any error logs though debug=True
I have tested this in the local environment (using redis server etc…) and it is working.
Can you please help with the error logs.
Thanks,
Nilabha
Snake Species Identification Challenge

Submission Errors
Over 1 year agoThanks kongas
I used the below code to remove the images
filter_func = lambda x: str(x) not in lsRemove
test_img = (ImageList.from_folder(path).filter_by_func(filter_func))
Though there is another error…
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/33
@mohanty
Has the competition ended or will it restart again. Would have liked to get a score as my validation results were good.
Is it possible to get the error logs?

Evaluation stuck [Edit : Evaluation took a long time]
Over 1 year agoall submissions seems to be queued for a long time.
Is there some problem?
Thanks

Submission Errors
Over 1 year ago@mohanty
I have put a workaround to find the images which fail loading and delete them and later add these probability which are all equal to 1/45
However I still get an error
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/32
Can you please help with the error?
Thanks,
Nilabha

Submission Errors
Over 1 year agoCan you please help with the error logs. I keep getting an evaluation failure.
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/31
I have tried reducing the batch size also in case of any memory issue. However the error still seems to be there.
From trial and error I notice that the error happens in the code line learn.tta function from the fast ai library.
Thanks,
Nilabha

Submission Errors
Over 1 year ago@mohanty
I have been using trial and error to see which line of code the submission is failing, unfortunately I have crossed the limit of submission
Could you provide the error logs to debug the issue?
I have isolated the issue to few lines of code but it would be helpful to know what is the error.
Thanks,
Nilabha

Submission Errors
Over 1 year ago@mohanty
I have set num_workers=0
I still get the error.
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/24
Could you share the error logs?
Thanks,
Nilabha

Submission Errors
Over 1 year agoI am feeling guilty having to ask you everytime
I wouldn’t mind if all my logs are made public to be honest.
But on another note would the deadlines of this competition be extended. Due to less time left in the competition, I haven’t done much to replicate this submission process locally (I think there are some instructions in the link https://github.com/stanfordnmbl/neurips2019-learning-to-move-starter-kit). As of now I just do bash run.sh to test that the code is working in my local.

Submission Errors
Over 1 year ago@mohanty
I have redone with a fresh copy from repo and updating the files.
Sorry for asking you again but I still get the error in my latest submission http://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/22
Do you know what is the error?
Thanks,
Nilabha

Submission Errors
Over 1 year ago@mohanty
run.sh it seems already has executable permission, hence there is no change in my local files on doing chmod +x run.sh
permissions are -rwxrwxrwx in my local
I have looked at the permissions for the other files and they are the same.
Do you think there is some other error?
Thanks,
Nilabha

Submission Errors
Over 1 year agoThanks. Its working now. The build was also failing due to the environmental.yaml checked using LFS.
Unfortunately the evaluation is failing now after a successful build. Can you kindly help me with the error logs?
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/18

Submission Errors
Over 1 year agoI am trying to submit but I am getting below error
Submission failed : Malformed JSON provided in aicrowd.json
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/15
I don’t see any issues with this file.
Do you know what is the problem?
Thanks,
Nilabha

Submission Errors
Over 1 year agoI have uploaded another version of environment.yml file but the image build fails.
Can you please provide me with the error logs?
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/10
Thanks,
Nilabha

Submission Errors
Over 1 year agoHi Mohanty
Thanks for the reply. I submitted some new submissions after committing but somehow they are not getting submitted for evaluation.
For example:
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/commits/submission-v3.2
Can you let me know what is the problem?
Thanks,
Nilabha
Thanks,
Nilabha

Submission Errors
Over 1 year ago@mohanty @shivam @arjun_nemani
I can’t quite find any logs available for my failed submission.
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/3
Request you to kindly share the logs
I see many submissions of other participants also failing. In case of an issue in submission do we just message the organisers to get the logs or is there any other way.

Cannot upload trained model to gitlab.aicrowd
Over 1 year agoIts working now after I redid the whole thing and was able to push a submission. However I see the image built successfully but failed after that.I am not able to see any logs. Where can I see the error / log information of why it failed.
[aicrowd-bot @aicrowd-bot added [Failed] label after [image_built_successfully]

Cannot upload trained model to gitlab.aicrowd
Over 1 year agoI am getting an error pack exceeds maximum size
git push aicrowd submission-v0.3
Locking support detected on remote “aicrowd”. Consider enabling it with:
$ git config lfs.https://gitlab.aicrowd.com/*/info/lfs.locksverify true
Uploading LFS objects: 100% (12/12), 146 MB | 3.4 MB/s, done
Counting objects: 71, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (69/69), done.
remote: fatal: pack exceeds maximum allowed size
error: pack-objects died of signal 13MiB | 1.62 MiB/s
error: failed to push some refs to 'git@gitlab.aicrowd.com:.git’
I get same error with git and git-lfs
Any idea what is the issue? Size including weights is less than 200 MB

Issue in Test Images
Over 1 year agoI notice that there are 44 images in the test set of 17331 are not readable. Few example are 1c56622f067257a814254b3a0d66a504.jpg , 2b8060650fbc5c4d04c831e37b6a2af3.jpg
Do we ignore these images? Will these be ignored in the submissions?
Following code gets the list of images:
lsFiles has the full list of paths for the images
from PIL import Image
lsRemove = []
for sFiles in tqdm(lsFiles):
try:
Image.open(sFiles)
except:
lsRemove.append(sFiles)

Submission Error Log for Snake Identification Competition
Over 1 year agoHi Shivam,
Let me know if you are able to share the logs or where I can find them.
I am keen to try couple more submissions.
Thanks,
Nilabha

Submission Error Log for Snake Identification Competition
Over 1 year agoHi Shivam
I cant quite find any logs in my gitlab issue.
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/3
I see many submissions of other participants also failing. In case of an issue in submission do we just message you to get the logs or is there any other way.
Thanks,
Nilabha

Error/ Evaluation log for Snake Species Competition Submission
Over 1 year agoThis is the link to my submission
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/3

Error/ Evaluation log for Snake Species Competition Submission
Over 1 year agoHi,
I submitted my model for submission in the snake species competition but it has label as error. Can you send me the error logs or is there any way I can see the error logs?

Submission Error Log for Snake Identification Competition
Over 1 year agoHi Shivam
I pushed my model to the gitlab rep for submission. However I see the image built successfully but failed after that.
[aicrowd-bot @aicrowd-bot added [Failed] label after [image_built_successfully]
I noticed this thread for the obstacle tower
where it says “debug”: true
I am interested in teh error log for
https://gitlab.aicrowd.com/nilabha/snake-species-identification-challenge/issues/3
Do we know how we can get the error logs or you will be able to provide it.
Thanks,
Nilabha
-
Accelerate the learning – increase agents behavior at higher speed! New changes related to flatland
Byadrian_egli23 days ago -
RLLib Baselines on Colab! This Colab notebook allows you to train a full Flatland agent using the provided PPO baseline.
Bynilabha23 days ago
Current status of imitation agent in baseline repository
3 months agoThe above script was for doing PPO and IL alternately…
If you want a pure IL, you can try
train.py -ef baselines/custom_imitation_learning_rllib_tree_obs/pure_imitation_tree_obs.yaml --eager --trace