Badges
Activity
Ratings Progression
Challenge Categories
Challenges Entered
Completed
Meta
Improve RAG with Real-World Benchmarks
Latest submissions
Amazon Search
Revolutionise E-Commerce with LLM!
Latest submissions
See All| graded | 252932 | ||
| failed | 252924 | ||
| failed | 252729 |
Shopping Session Dataset
Latest submissions
AIcrowd
Amazon KDD Cup 2023
Latest submissions
| Participant | Rating |
|---|
| Participant | Rating |
|---|
-
softmaxgogogo Amazon KDD Cup 2024: Multi-Task Online Shopping Challenge for LLMsView
-
AbsolutelyCrazyElephantSurgeon Meta Comprehensive RAG Benchmark: KDD Cup 2024View
Amazon KDD Cup 2024: Multi-Task Online Shopping Ch
I wonder if predictive timeouts are common
7 months agoI need to admit that using Zero-shot CoT is an unwise behavior and your advice is helpful. To minimize the requirements.txt sounds make sense, but I notice that in the docs/hardware-and-system-config.md you recommend to use pip3 freeze >> requirements.txt to export all the packages, which can be confusing. And we did worry that not specifying a version would cause compatibility issues at that time 
.
I wonder if predictive timeouts are common
7 months agorequirements.txt We follow the instruction to use pip freeze > requirements.txt ,It’s painful to check for availability one by one 
.
I wonder if predictive timeouts are common
7 months agoWhat’s more, I can understand the failure submissions will increase the burden on the platform, but I have to say this is unfair. We make many failure submissions because the package version cannot be found, and we have to modify the requirements.txt over and over again. But to be honesty, each version of the requirements.txt can install the packages in out devices perfectly. It seems that we have no any other way to test our solutions before submission, but only using the system for debugging over and over again. I think this is depressing .
I wonder if predictive timeouts are common
7 months agoMy team submitted it several times and failed without exception. The message tells me Timeout while making a prediction, but I’m pretty sure on my computer they predicted it very quickly. I tried quantifying the model to 4 bits, but I still got a timeout error. If no offense is taken, I would like to get a specific reason for the predicted timeouts.
No submission slots
7 months agoI strongly agree with that. To be honesty, my team spend most of time dealing with environmental problems because there always has some packages that we can install but the platform cannot and tell us the version cannot be found. There was only one “successful” submission, which is killed in validation and raise AssertionError: Timeout while making a prediction. We try to enhance the quantization and submit again however there is no submission slots. I think this is depressing because we do well in our computer but we spend too much time on things that don’t work.
About Mistral-7b submission on track1
7 months agoWe use Mistral-7B and meet the same mistake.We have tried quantifying the model to int4, which generally reduces memory consumption and speeds up Inferencing, but it doesn’t seem to work.