
Location
Badges
Activity
Challenge Categories
Challenges Entered
Train LLMs to Play Chess
Latest submissions
Build an LLM agent for five real-world games
Latest submissions
See All| failed | 309573 | ||
| failed | 309501 | ||
| failed | 309496 |
A dataset and open-ended challenge for music recommendation research
Latest submissions
Latest submissions
| Participant | Rating |
|---|
| Participant | Rating |
|---|
Orak Game Agent Challenge 2025 Forum
Duplicate failed submissions, requesting for a final submission slot
27 days agoI am just curious.
Our code should be the same with what we submitted for the evaluation test? (with the best score)
Our code will be re-executed for the hidden test-case.
Then, we don’t need to actually submit for evaluation.
Leather board score is a reference, which is not the final score.
Deadline Extended to Feb 8 + Instructions for Final Submission Package
About 1 month agoFor Track 2 (who might use commercialized APIs without fine-tuned),
can we skip to submit Model artifacts and documentation, Reproducibility artifacts?
calling OpenAI API is all
Missing processed_map directory causes remote evaluation failure
About 1 month agoI think something wrong in the recent submission server.
-
“Confirming if the
processed_mapfiles should be part of the official starter-kit or if they should be uploaded manually by participants.”
→ I don’t think so, it worked automatically in the previous version.
And there is also module load error in the recent start-kit version:
ModuleNotFoundError: No module named ‘evaluation_utils.live_export’ - Orak Game Agent Challenge - AIcrowd Forum
No one success the submission in one-week.
Pokemon, does the high-level-tools work?
About 1 month agothe recent startkits have that use_tool function.
look here.
ModuleNotFoundError: No module named 'evaluation_utils.live_export'
About 1 month agoIn the recent starterkit,
- evaluation_utils/live_export.py file is missing.
File “C:\MyProjectFolders\orak-2025-starter-kit-master_new\evaluation_utils\game_server_launcher.py”, line 10, in from evaluation_utils.live_export import live_export_enabled
ModuleNotFoundError: No module named ‘evaluation_utils.live_export’
Pokemon, does the high-level-tools work?
About 1 month agoI was using old code.
The recent code in git have that function.
Pokemon, does the high-level-tools work?
About 1 month agoThere is high-level tools in poketmon based on the doc.
(1) Does it work? (i failed…, though simple command worked such as A,B button)
-
return ‘use_tool(move_to, (x=7 , y=1))’
(2) Using the high-level tool is counted as 1 step?
if the high-level tool is counted as a 1 step, we have to use it to reduce the steps. (200 steps is very limited). Then, I can’t continue to work on pokemon without the tools.
Anyone can successfully use the high-level tools in poketmon?
-
move_to(x, y): Finds and executes a path to the specified map coordinates (x,y). -
interact_with_object(name): Interacts with a specified object in the environment. -
warp_with_warp_point(x, y): Uses a specified warp point to move to a different location. -
overworld map transition(direction): Transitions to an adjacent map in the given direction. -
continue_dialog(): Advances the current dialogue. -
select_move_in_battle(move_name): Selects and uses a specific move in a Pokemon battle. -
switch_pkmn_in_battle(pokemon_name): Switches to a different Pokemon in the player’s party during a battle. -
run_away(): Attempts to flee from a wild Pokemon battle. -
use_item_in_battle(item_name): Uses a specified item during a Pokemon battle.
Example:
use_tool(select_move_in_battle, (move_name="Thunder Shock"))
Star2 Crash Issue
About 1 month ago- STAR2 MAX_STEPS issue
MAX_STEPS value doesn’t work in Star2 in original code.
I have a game log that played 2 hours.
there were 2 reasons
(1) ‘MULTI-ATTACK’ cannot destroy all of enemy’s building (some of theirs are built in the corner)
(2) MAX_STEPS (ex., 1000 steps) didn’t work.
in star_craft_env.py.
def check_process(self, reset=False):
ctx = getattr(self, "_mp_ctx", multiprocessing.get_context())
if self.p is not None:
if self.p.is_alive():
#if not self.game_over.value: # should be changed like below. after MAX_STEPS doesn't mean it is game_over
if not self.game_over.value and not reset: # Check if the game is over
return # Return ONLY if game is running AND we are NOT resetting (MAX_STEP)
# KILL PROCESS LOGIC (depends on the OS)
self.p.terminate()
To re-make this bug,
in common.py
change star_craft value to low, it is not gonna finished after that max_step.
MAX_STEPS = {
“twenty_fourty_eight”: 1000,
“super_mario”: 100,
“pokemon_red”: 200,
“star_craft”: 50,
}
Star2 Crash Issue
About 1 month agoThanks for your response.
I’ve identified several issues, and I’ll describe them starting with the most critical one.
(1) Crash
This is a critical crash that causes the simulation to fail and therefore must be fixed.
After applying the fix, building actions can still fail.
**(2) Multi-RETREAT **
When enemy is going back to their base (after fight), my units automatically follow them to the enemy main base.
In this situation, it appears that the retreat target position is updated to the combat destination (i.e., the enemy main base), causing the retreat behavior to effectively function the same as an attack command.
As a result, there is currently no reliable way to make units retreat.
(3) Minimum distance for building placement
There seems to be a minimum distance constraint when placing buildings such as pylons.
On extremely small maps like FLAT64, this can result in pylons being placed very close to the enemy main base.
(4) Replay file (minor)
Each execution runs three episodes. However, the replay files use the same filename and overwrite each other, so only one replay file remains.
If replays are intended to be saved per episode, the filename logic would need to be modified to generate a separate replay file for each episode.
While it is true that participants are expected to account for these constraints, this is quite unexpected situation when playing startcraft.
and I understand that some participants have already developed under this environment, so too late to
update the code.
Pokemon map is broken?
About 2 months agoso… did you mean that
the current evaluation server has the map loading issue (such as /? /?).
Then, the score in the leather board was achevied without the map information…?
I haven’t tried poketmon for this contest because of the map loading problem.
I will try your solution.
Thanks in advance.
Star2 Crash Issue
About 2 months ago
This map is not good.
My units can’t go attack because of the ROCK obstacle. (They stucked there almost a few minutes after ‘MULTI-ATTACK’)
Star2 Crash Issue
About 2 months agoThis is very critical…
When they just skip to build an important building such as PYLON.
It’s very confusing should I consider, this will be happened in real-test too, so should I try to over-building.
If someone wants to re-make this bug, just keep building ‘BUILD PYLON’, after three of them it just stop building.
Obs_str position misalignment: Mario's coordinates gradually drift from the actual position
About 2 months agoPossible Mario simulation bug? - Orak Game Agent Challenge - AIcrowd Forum
The same issue (gradual misalignment) with me.
I think this is kinda of bug(?) from the simulator.
I don’t know how to fix it.
When I see the screenshots, the mario is gradually going forward but the position_x of text is the same.
So, I just think that to get 1.0 point, we have to pass the monters through upstrair-blocks (diffrent y_axis with the mosters)
Star2 Crash Issue
About 2 months agoso, defending with cannons is not gonna work.
many fails happen when try to build cannons.
(couldn’t find best_position)
Star2 Crash Issue
About 2 months agoThere is a crash issue in StarCraft 2.
While executing a strategy of defending with Photon Cannons followed by an attack, the simulation would irregularly crash and terminate.
Upon debugging, the problem was traced to building placement. According to bots.py, a building can only be constructed once best_position is determined. When the surrounding area search repeatedly fails to find a suitable location, best_position remains None, causing a crash when attempting to build.
In the bots.py file, within the function:
python
async def handle_action_build_building(self, building_type: UnitTypeId, building_limits=None):
There is this commented section:
python
# if not best_position:
# print(f"Still no suitable position found for {building_type}. Aborting.")
# return
Fix: Uncomment this section so that if best_position cannot be found, the function simply skips the build:
python
if not best_position:
print(f"Still no suitable position found for {building_type}. Aborting.")
return
This change prevents the crash by gracefully handling cases where no valid building location is found, allowing the bot to continue running instead of terminating the simulation.
Impossible to run Star2 in Linux (no map file)
About 2 months ago(1)
there is no the map file in the links. Ancient Cistern LIE
Maps - AI Arena Wiki
I’ve downloaded all the file, but there isn’t.
Also, some links are broken in the url.
it’s difficult to find the instructions in the link.
Maybe, there is youtube link to apply the patch.
They used SC2 map editor? and updated config file?
it’s quite difficult to understand what they are doing for me.
It would be easy to simply share the map files for this contest for the linux users I believe.
Impossible to run Star2 in Linux (no map file)
About 2 months ago-
I spent a few hours trying to run the Star2 simulation on Linux, but I couldn’t get it to work. The biggest issue was that the “Ancient Cistern LIE” file could not be found even in the whole internet.
-
So, I reconfigured the ORAK simulation on Windows instead. There was a gRPC-related issue, but it was resolved by modifying
game_server_launcher.pyas shown below:
cmd = [
sys.executable, # fixed
game_server_script,
]
env = os.environ.copy() -
On Windows, I renamed the “Ancient Cistern LIE” file to “Ancient Cistern LE” and downloaded the file into /Maps/. When running the simulation, I could clearly see the game launch and play screen, which makes prompt debugging much easier.
On Linux, I was not able to see the StarCraft play screen during execution.




Deadline Extended to Feb 8 + Instructions for Final Submission Package
27 days agothere is no @aicrowd in the gitlab.