Loading
7 Follower
3 Following
jyotish
Jyotish

Organization

AIcrowd

Location

IN

Badges

5
3
3

Activity

Dec
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
Mon
Wed
Fri

Ratings Progression

Loading...

Challenge Categories

Loading...

Challenges Entered

ASCII-rendered single-player dungeon crawl game

Latest submissions

See All
graded 155140
graded 147319

Self-driving RL on DeepRacer cars - From simulation to real world

Latest submissions

No submissions made in this challenge.

Machine Learning for detection of early onset of Alzheimers

Latest submissions

No submissions made in this challenge.

Measure sample efficiency and generalization in reinforcement learning using procedurally generated environments

Latest submissions

See All
submitted 90059
graded 83575
failed 81249

Sample Efficient Reinforcement Learning in Minecraft

Latest submissions

No submissions made in this challenge.

Airborne Object Tracking Challenge

Latest submissions

No submissions made in this challenge.

Latest submissions

See All
submitted 167156
graded 167122

Robustness and teamwork in a massively multiagent environment

Latest submissions

No submissions made in this challenge.

3D Seismic Image Interpretation by Machine Learning

Latest submissions

See All
failed 99353

5 Puzzles 21 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Play in a realistic insurance market, compete for profit!

Latest submissions

See All
graded 125874
graded 121934
failed 116909

5 Puzzles 21 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

Multi-Agent Reinforcement Learning on Trains

Latest submissions

No submissions made in this challenge.

A dataset and open-ended challenge for music recommendation research

Latest submissions

No submissions made in this challenge.

A benchmark for image-based food recognition

Latest submissions

See All
graded 114994
graded 114972
failed 114971

Latest submissions

No submissions made in this challenge.

Sample-efficient reinforcement learning in Minecraft

Latest submissions

No submissions made in this challenge.

Latest submissions

See All
failed 124981
failed 124727
failed 124726

5 Puzzles, 3 Weeks. Can you solve them all? πŸ˜‰

Latest submissions

No submissions made in this challenge.

Multi-agent RL in game environment. Train your Derklings, creatures with a neural network brain, to fight for you!

Latest submissions

No submissions made in this challenge.

Predicting smell of molecular compounds

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

5 Problems 21 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

5 Puzzles 21 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

5 Puzzles, 3 Weeks | Can you solve them all?

Latest submissions

No submissions made in this challenge.

5 PROBLEMS 3 WEEKS. CAN YOU SOLVE THEM ALL?

Latest submissions

No submissions made in this challenge.

Grouping/Sorting players into their respective teams

Latest submissions

No submissions made in this challenge.

Dog breed classification

Latest submissions

No submissions made in this challenge.

5 Problems 15 Days. Can you solve it all?

Latest submissions

See All
failed 71051
failed 71041

Sample-efficient reinforcement learning in Minecraft

Latest submissions

No submissions made in this challenge.

Multi Agent Reinforcement Learning on Trains.

Latest submissions

No submissions made in this challenge.

Recognise Handwritten Digits

Latest submissions

See All
graded 165274
graded 165272
failed 165271

Crowdsourced Map Land Cover Prediction

Latest submissions

See All
graded 60315
graded 60314

Latest submissions

No submissions made in this challenge.

5 Problems 15 Days. Can you solve it all?

Latest submissions

No submissions made in this challenge.

Identifying relevant concepts in a large corpus of medical images

Latest submissions

No submissions made in this challenge.

5 PROBLEMS 3 WEEKS. CAN YOU SOLVE THEM ALL?

Latest submissions

See All
failed 77264

Real Time Mask Detection

Latest submissions

See All
graded 67702
graded 67701
graded 67600

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Solve the jigsaw and finish the picture!

Latest submissions

No submissions made in this challenge.

Predicting wine quality

Latest submissions

No submissions made in this challenge.

Predict whether an individual will be back to prison

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Predict viewer reactions from a large-scale video dataset!

Latest submissions

See All
graded 124097

Reinforcement Learning, IIT-M, assignment 1

Latest submissions

No submissions made in this challenge.

Latest submissions

See All
failed 156316

5 puzzles and 1 week to solve them!

Latest submissions

No submissions made in this challenge.

Latest submissions

See All
graded 128368

Latest submissions

No submissions made in this challenge.

Multi-Agent Reinforcement Learning on Trains

Latest submissions

No submissions made in this challenge.

Latest submissions

No submissions made in this challenge.

Train your RL agents

Latest submissions

See All
graded 165868
failed 162152
Participant Rating
vrv
saketha_ramanujam 0
branden_murray 0
jansi_rani_s_v 0
lars12llt 0
will_kwan 0
BhaviD 0
Participant Rating
shivam
aicrowd-bot
vrv

RL Project 2021-56807e

Hit a connection error, exiting silently

10 days ago

Hello @tahir_javed_cs20d407

Can you give us a few submission IDs so that we can debug this further on our end?

Hit a connection error, exiting silently

10 days ago

Hello @tahir_javed_cs20d407

This happens when the evaluation times out (2 hours is the current timeout) or the evaluation takes up too much of memory.

Why is the evaluation failing at evaluation rollout phase?

11 days ago

Hello @utsav_dey_cs20s009

The issue should be fixed now. We are re-evaluating the effected submissions.

Evaluation failing at last stage

11 days ago

Hello @tahir_javed_cs20d407

The issue should be fixed now. We are re-evaluating the effected submissions.

Evaluation failing at last stage

11 days ago

Hello @tahir_javed_cs20d407

Thanks for pointing this out. We are looking into the issue and will soon fix it.

NeurIPS 2021 - The NetHack Challenge

Are the specs for the machine that the evaluations are run on available anywhere?

3 months ago

Hello @jon_grantham

is that without considering time spent in the agent?

Yes, it is without considering the time spent in the agent.

How much of that is network latency? I.e., if we have an agent capable of performing faster than that, does all of the agent compute time get hidden under the latency?

This includes the network latency, processing delay and everything that is needed to send a request and get the response. Any latency/compute time on the agent will be added to this.

For example, if you have something like

import aicrowd_gym
from tqdm import trange

env = aicrowd_gym.make("NetHackChallenge-v0")
env.reset()
for _ in trange(1000000):
    _, _, done, _ = env.step(1)
    if done:
        env.reset()

This should give you a throughput of 1500-2000 iterations per second during the evaluation.

Are the specs for the machine that the evaluations are run on available anywhere?

3 months ago

Hello @jon_grantham

The evaluations run on AWS EC2 instances. The resources available are as follows

GPU enabled flag in aicrowd.json vCPUs Memory GPU AWS instance type
true 4 16 GB NVIDIA T4 g4dn.xlarge
false 4 16 GB - m5.xlarge

Note:
During the evaluations, you will receive a proxy NetHack env object instead of the actual environment. This proxy object talks to the actual NetHack env over the network and returns the values as needed. We do this to prevent participants from tampering with the env. This also adds an overhead. Based on our benchmarks, a single env should roughly give a throughput of 1500-2000 steps/second. Using something like a batched env increases the throughput.

Hope this helps and please feel free to reach out to us for any help.

Iterating over Dict spaces in aicrowd-gym

4 months ago

Hello @anssi

Thanks for sharing this with us.

  • Allow access to observation/action space variables that exist in normal Gym (e.g. spaces for Dict obs)

We will soon update the evaluator to allow access to a few more attributes. At the moment the following attributes/methods can be accessed.

On action space:

(
    "contains",
    "dtype",
    "sample",
    "shape",
    "n",
)

On observation space:

(
    "bounded_above",
    "bounded_below",
    "contains",
    "dtype",
    "high",
    "is_bounded",
    "low",
    "sample",
    "shape",
    "n",
)


  • This is perhaps result of bad programming from my end, but make sure variables that do not exist in the environment return appropriate exceptions. For example, I did hasattr("something", env). something was not a variable in the environment, but with aicrowd-gym it was set to None. This caused hasattr to return True and subsequently things failing.

I believe this happens with gym.Env objects as well. For example, if I do

import gym
env = gym.make("CartPolve-v0")
setattr(env, "something", "some value")

it doesn’t return any error. If possible, can you share with a simple example the expected behaviour?

Note: During the evaluation, the env object you have access to is a proxy object and not the actual gym.Env instance. So setting attributes from your code will not set any attributes on the actual env object. If you have any specific use case for doing this, please feel free to reach out to us and we can try to accommodate your use case.

Edit: Sorry, misread the last part. We will try to get the read access to all the env attributes in our future releases.

Problems with starter kit setup

5 months ago

Hello @dylan_miller

It looks like one of your apt repositories is giving the error. Kitware also made cmake installable using pip. You can run the following to install the latest version of cmake from kitware.

pip install -U cmake

Reference: https://pypi.org/project/cmake/

Music Demixing Challenge ISMIR 2021

Can I use GPU for prediction?

5 months ago

Hello @woosung_choi @Li_Xue_Han

Sorry, my bad. GPUs are not available for this competition. Apologies again for the confusion.

Can I use GPU for prediction?

5 months ago

Hello @Li_Xue_Han

Edit: GPUs are not available for this competition

You can use GPUs for inference. Please refer How to enable GPU for your submission? for more information on using GPUs for your submission.

"pod deleted during operation" during generating predictions

6 months ago

Hello @Piteryo

The evaluation ran out of memory. Can you check how much RAM is being used when you run this locally?

Here is a screenshot of the RAM usage for reference.

Failed at "Build Image" with no logs

6 months ago

Hello @kim_min_seok

Thanks for notifying us about this. This was an issue on our end and it is fixed now. We queued the effected submissions for re-evaluation.

The Neural-MMO Challenge

Airborne Object Tracking Challenge

Discussion on SiamMOT baseline

6 months ago

Hello @octo

Replied on the thread. Let us know if you need any further help on this :slight_smile:

Getting error in siam-mot baseline submission "pack exceeded maximum allowed size"

6 months ago

Hello @paulids

You need to use Git LFS to push the large files in your repository (your saved models, etc.,). You can find more information on how to use LFS here

ADDI Alzheimers Detection Challenge

AssertionError: Incorrect prediction directory structure. View the submission for more details

6 months ago

Hello @SenticLab.UAIC

It looks like # Prediction phase heading is missing in your notebook. Please add this heading after your training code block has ended. We use these to figure out which part of the notebook needs to be executed during evaluation.

For more information please refer this discussion.

Dataset: transport endpoint not connected

6 months ago

Hello @e_t

Can you please drop a mail to servicedesk@aridhia.com regarding this?

Submission Error : 'DATASET_PATH' not recognised as an internal or external command

6 months ago

Hello @jigsaw

Can you share any output that you might have got after running the command? Do you see something like

[NbConvertApp] Executing notebook with kernel: python

when you run the command?

Submission Error : 'DATASET_PATH' not recognised as an internal or external command

6 months ago

Hello @jigsaw

Can you try this?

jyotish has not provided any information yet.

Notebooks

Create Notebook