Loading
2 Follower
0 Following
dipam
Dipam Chakraborty

Organization

ML Engineer at AIcrowd

Location

Kolkata, IN

Badges

7
4
3

Connect

Activity

Oct
Nov
Dec
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Mon
Wed
Fri

Ratings Progression

Loading...

Challenge Categories

Loading...

Challenges Entered

Latest submissions

No submissions made in this challenge.

Latest submissions

See All
graded 199053
graded 199034
failed 199033

Interactive embodied agents for Human-AI collaboration

Latest submissions

See All
graded 198521
failed 198520
failed 197794

What data should you label to get the most value for your money?

Latest submissions

No submissions made in this challenge.

A benchmark for image-based food recognition

Latest submissions

No submissions made in this challenge.

Behavioral Representation Learning from Animal Poses.

Latest submissions

See All
graded 198630
graded 197504
graded 197503

Airborne Object Tracking Challenge

Latest submissions

No submissions made in this challenge.

ASCII-rendered single-player dungeon crawl game

Latest submissions

See All
graded 158823
failed 158209
failed 158208

Latest submissions

No submissions made in this challenge.

Latest submissions

See All
graded 152892
graded 152891
failed 152884

Machine Learning for detection of early onset of Alzheimers

Latest submissions

No submissions made in this challenge.

Measure sample efficiency and generalization in reinforcement learning using procedurally generated environments

Latest submissions

No submissions made in this challenge.

Self-driving RL on DeepRacer cars - From simulation to real world

Latest submissions

See All
graded 165209
failed 165208
failed 165206

Robustness and teamwork in a massively multiagent environment

Latest submissions

No submissions made in this challenge.

5 Puzzles 21 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

Multi-Agent Reinforcement Learning on Trains

Latest submissions

No submissions made in this challenge.

Latest submissions

See All
graded 143804
graded 125756
graded 125751

5 Problems 15 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

Learn to Recognise New Behaviors from limited training examples.

Latest submissions

See All
graded 125756
graded 125589

Reinforcement Learning, IIT-M, assignment 1

Latest submissions

See All
graded 125767
submitted 125747
graded 125006

IIT-M, Reinforcement Learning, DP, Taxi Problem

Latest submissions

See All
graded 125767
graded 125006
graded 124921

Latest submissions

See All
graded 128400
submitted 128365

Latest submissions

See All
failed 131869
graded 130090
graded 128401

Latest submissions

See All
failed 131869
graded 130090
graded 128401

Latest submissions

See All
graded 135842
graded 130545

Latest submissions

No submissions made in this challenge.

Round 1 - Completed

Latest submissions

No submissions made in this challenge.

Identify Words from silent video inputs.

Latest submissions

No submissions made in this challenge.

Round 2 - Active | Claim AWS Credits by beating the baseline

Latest submissions

See All
graded 198630
graded 182252
graded 178951

Round 2 - Active | Claim AWS Credits by beating the baseline

Latest submissions

See All
graded 197504
graded 197503
graded 182254

Use an RL agent to build a structure with natural language inputs

Latest submissions

See All
graded 198521
failed 198520
failed 197794

Language assisted Human - AI Collaboration

Latest submissions

See All
graded 196399
graded 196379
failed 196363
Participant Rating
nachiket_dev_me18b017 0
cadabullos 0
Participant Rating

NeurIPS 2022: CityLearn Challenge

Turning off rendering?

About 9 hours ago

Hi @mansur

Unfortunately they cannot be turned off from user side at the moment. I agree they do take a lot of time. However that time is not counted for your agent’s throughput. It only means you get your scores a bit slower.

Evaluation Failed (Exit code 1) inference failed

About 10 hours ago

@pokuk76 , Failed runs only on private runs is commonly getting caused from hardcoded values such as number of buildings or observation size. You can check if your local evaluation works if you change the parameters in the schema.

If you’re still unable to fix the issue please let me know your submission id.

Removal of logs from Phase 2 runs

5 days ago

@jack-etheredge

Added comment with log here

Removal of logs from Phase 2 runs

5 days ago

Let me know the submission link I’ll help. Also does it pass for render but fail for the main run or fail for all of them?

Phase 2 episode count and timeout update

18 days ago

Thanks a lot for the suggestions @tymur_prorochenko :smiley:

Indeed, we do have a private leaderboard planned for Phase 3.

Originally we had planned to hide the scores of the 7 buildings that will be added in Phase 3 completely. However I also like your idea of hiding last 4 months, we’ll discuss this internally.

Stay tuned for updates on Phase 3.

Phase 2 episode count and timeout update

18 days ago

Hi everyone,

Since there is currently no stochasticity in the environment, we are reducing the number of episodes for the online evaluation to 1. If your agent has any randomness, this will increase the variance of your scores. We suggest to seed your agents such that your score is consistent.

The timeout for agents is now updated to 30 minutes. Your agent has to complete 1 episode in 30 minutes.

Also, as a participant pointed out, it is possible to use the information of the first episode to overfit a solution to the subsequent episodes. This is not in the spirit of the competition, and we request you to setup your agents such that they can run episodes independently.

Starting from phase 3, the parallel run setup will be changing, but the episodes will remain 1.

Removal of logs from Phase 2 runs

18 days ago

Hi everyone,

One of the participants rightly pointed that it was possible to get the schema information of the private buildings by printing the data and causing an error in the submission to get the logs.

In light of this, we have disabled the logs for the private runs, you will still be able to see logs for the rendering run.

It is against the spirit of the competition to try and obtain private data. We hope that no one has used this exploit, and will closely monitor the logs.

If you face any issues due to the removal of the logs, please let us know.

When is the deadline for adding team members?

18 days ago

Hi @hyejin

The team freeze deadline is 24th October 00:00 UTC, which is 1 week before the end of the competition.

We keep this to avoid too many changes in the leaderboard due to team joins at the last moment.

Can we use GPU for online evaluation?

18 days ago

Hi @zheng_shun

No, we’re not providing GPU access for the Citylearn Challenge. Are you facing issues running any particular model or due to low throughput?

We have tried to set the time limits on evaluation such that its enough, but are open to increasing it in case its needed.

How many times of the episode will the submitted agent been tested for?

About 1 month ago

Hi @blanck_smith

Making a small correction to what @kingsley_nweye mentioned. The online evaluator runs 5 episodes in 2 parallel runs. So a total of 10 episodes are run and averaged.

Editting register_reset function

About 2 months ago

Hi @semih_tasbas

Yes you can change register_reset as long as it takes the same inputs and returns the valid set of actions.

NeurIPS 2022 IGLU Challenge - NLP Task

Is it ok to directly wrap the gym env in `create_single_env` in local_evaluation.py when evaluating the model in aicrowd?

About 2 months ago

@CH_do

The score comes from a private set of tasks. Please check if you model is overfitting to the public tasks.

Is it ok to directly wrap the gym env in `create_single_env` in local_evaluation.py when evaluating the model in aicrowd?

About 2 months ago

@CH_do

done will be True after the env resets now. Thanks again for checking this.

Done parameter in local evaluation

About 2 months ago

Thanks @CH_do for pointing out a bug in the evaluation loop which prevented done being passed to the agent after the env was reset.

This is fixed now. Please merge the changes from the latest starter kit.

Also, note that this means the data passed to an agent when done=True will be new observation from env.reset(). The evaluator on the server will follow the same pattern as local_evaluation.

Is it ok to directly wrap the gym env in `create_single_env` in local_evaluation.py when evaluating the model in aicrowd?

About 2 months ago

Oh, this is a bug. Thanks for pointing this out, I’ll fix it asap.

Is it ok to directly wrap the gym env in `create_single_env` in local_evaluation.py when evaluating the model in aicrowd?

About 2 months ago

Hi @CH_do

The done parameter is the same as what the env outputs. You can use that to detect resets.

NLP Local evaluation fix

About 2 months ago

Hi everyone,

Thanks to the feedback of a participant on the IGLU Contest slack channel. We fixed bug in the evaluation loop. We made the change to the local evaluation as well as the actual evaluator.

Please merge changes to the latest local_evaluation.py file from

Is it ok to directly wrap the gym env in `create_single_env` in local_evaluation.py when evaluating the model in aicrowd?

About 2 months ago

Hi @CH_do

Unfortunately we do not support this. Any changes in local_evaluation are not used for the actual evaluation.

Please add all wrapper related logic to the agent class you’re submitting in agents/user_config.py

Is the `agentPos` and `grid` available during evaluation in aicrowd?

About 2 months ago

Hi @CH_do

agentPos and grid are not available during evaluation. We’ve also removed it from the local evaluation now to match the evaluator. Please pull from the starter kit again to sync to the changes.

dipam has not provided any information yet.

Notebooks

Create Notebook