Historical dialogue : based on gold references or actual past responses?

4 months ago

Hi!

Seeking clarification on Taks 1 and the historical dialogue.

From the Task1 readme.md , we read that

For each batch of conversations, the first set of turns will be provided to your model. After the response is receieved the further turns of the same conversation will be provided. Each conversation will have exactly 7 turns.

Now, looking at the sample discussion:

My understanding is that the model is first provided with message from Anne (persona A).

“Oh , what a long queue ! I need to arrive at the seminar in 30 minutes but there 's not much hope now … you seem to be in a rush too , are you alright ?”

From the sample notes, it is clear at this point that the gold reference is invisible. My understanding is that the gold reference is an appropriate response.

Gold Reference:

“Oh hi ! I am OK but I 'm in a bit of a rush too actually . I have a meeting in an hour and I wanted to arrive early so that I could rehearse my presentation again . I 'm David by the way . What kind of seminar are you attending ?”

Let's say my model outputs this response to Anne's first message (instead of the gold ref):

“Hi, I understand the frustration of being in a long queue, but it’s important to stay patient. I’m actually in a bit of a rush as well.”

Ultimately, for performance evaluation, my model response is compared to the Gold Reference.

I seek clarification on the following turns.

The model will be provided with the historical dialogue.
Will the historical dialogue be constructed using:

• both Anne’s question and the previous Gold References?
  or
• both Anne’s question and my model actual previous responses?

Thanks!

🚨 Final Leaderboard (for real) + Community Contribution Prize

Almost 3 years ago

I noticed user @aio (which was ranked #1) has disappeared from the leaderboard.
Did something happen?

#gossip

Final leaderboard not visible

Almost 3 years ago

Go to sleep, results will get there faster

Final Leaderboard results are out

Almost 3 years ago

I remember that message.

The meme would have been EPIC.

Final Leaderboard results are out

Almost 3 years ago

Before I get in any trouble, let me confirm that I didn’t see anything back then. There was a message saying that the results were not available.

Final Leaderboard results are out

Almost 3 years ago

Jokes aside, back in May, I noticed something odd in the leaderboard.

When it was loading, if you had a crappy internet connection, you could notice the “Final Results” button but then it was disappearing within milli-seconds.

Something like this:

Back then, I looked more into this, digging within the html code and found the url for the Final leaderboard.

Note that the current leaderboard has been fixed and you won’t be able to find the url.

 

Went and look into my earlier notes to retrieve the url.

Found it… Anyone interested?
Keep this a secret, we don’t want the staff to know lol.

Final Leaderboard results are out

Almost 3 years ago

good one

however, it might had pushed the staff because they are out just now!!!

 

EDIT: I see plenty of folks hanging on discourse… only a couple of minutes and already 10 hits on the link, lol!

Final leaderboard not visible

Almost 3 years ago

Choose your own adventure

A)

B)

C)

Final leaderboard not visible

Almost 3 years ago

Drumroll

Motivation from the submission history

Almost 3 years ago

 

One last time!

Motivation from the submission history

Almost 3 years ago

From today’s townhall, this is my motivation for the finals.

Printing as many bingo cards as possible!

 

Aridha workspace query

Almost 3 years ago

Isn’t there a Start button on your workbench menu?

If not, you might need @nrad 's help.

He will need your workspace name. “dc-mar21…”

Submission done -- Inference time is too high!

Almost 3 years ago

Well, it looks like they all got pumped through.

Motivation from the submission history

Almost 3 years ago

Unique graph. Thanks for sharing.
Sharp spike? I have difficulties seeing it, lol.
Exciting finals to come!

Perhaps , the only way to break the
Resistance is to use a special
Outlawed approach…
But can it really be called…
Innovation?
Not so sure about that…
Good luck to all !

any acrostic fans ?

Help with saving and loading model

Almost 3 years ago

How did you define your AICROWD_ASSETS_DIR ?

It looks like it has been defined as the same your AICROWD_DATASET_PATH.

Python + R notebook

Almost 3 years ago

You need to install R.

You can run the script provided in that post ^.

Once done, you should be able to see “R kernel” available in Jupyter.

Python + R notebook

Almost 3 years ago

Thanks for the chuckle!

Don’t you have enough python notebooks at your disposal? Hahaha

 

On a side note, I’d love for the staff to share the submission breakdown by language.
(My guess is 95% Python, 5% R)

Add a 4th class?

Almost 3 years ago

@siddharth , thank for sharing.

Last week, I had performed the same experiment locally with similar results.

 

Iteratively;

• removed X number of outliers from the Normal diagnosis
• adjusted weights to reflect less Normal counts
• trained
• validated against a hold-out

 

Here’s a chart with 20 runs, removing from ~300 to ~3000 records (out of ~32k records).

 

 

Much effort for just …

 
 

I still believe this logic has merit…
Maybe I’m just bad at removing the appropriate records.
I encourage anyone to try it and share their experience.

Stop Linux VM

Almost 3 years ago

Pinging @nrad .
He will need your Workbench id… . dc_mar21…

Feature engineering

Almost 3 years ago

Have you ever heard of:

• Eccentricity
• Semi-latus rectum

 

In my notebook, I hinted about some features with regards to the ellipse such as :

ellipse_area = 3.1451926 * df$double_major * df$double_minor / 2 # surface area of the clock

 

Have a look at this super rich wikipedia page:

 

Disclaimer: You might find some super useful features in there, or perhaps nothing, but hey, at least you will learn new stuff!