🏆 Final Results & Next Steps

Thank you organizers, and congrats to the winners.

• I agree with @ningjia - the statement that “Final rankings are based on the average team rank across both phases” not only contradicts previous communication, but I also believe it is unfair that this statement was not communicated to us until after the competition ended.
• I believe it is unfair that @ningjia was quietly removed from fourth place on the “final” rankings, as seen by the post’s revision history. These rankings are evidently not final and are subject to change without notice.

On a related note, the private leaderboard for the Competition Phase is not visible to the public. – Edit: it is now visible

Thank you again.

View post 0

Final Round Test Set Released 🧨🚀

Just checking, will the final private scores be calculated using our “best” submission (according to public leaderboard) or our “last” submission during this phase?
Edit: Or will the best submission according to the private leaderboard will be used?
Without knowing which submission is used on the private leaderboard, we can’t be certain of which model to document in our Solution Documentation.

View post 2

Clarification on Future Data and Back Cast

Because this clarification is unlisted, it is unclear whether this is a rule we should follow - should we just follow the competition overview rules which state that we are to back-cast and all the data is considered historic? Thank you

View post 0

The Private Test Set

Hello everyone, I hope you are enjoying this competition as much as I am.

I have one concern with the Private Test Set that I would like to kindly bring to everyone’s attention. We learned in the live Q&A that this set contains only one site: Site F.

Considering that the goal of the competition is to design a model that is generalizable to diverse buildings and climates, is it fair to decide the winners using scores on solely one site? Considering that a team could tune a model for site F and argue in their documentation that the model would excel for various test sites, how can we all ensure that the winning models would also win on a diverse test set?

Thank you,
Ryan

photo11847×1036 301 KB

photo21173×660 93.7 KB

View post 2

📹 Townhall Recording & Q&A with Challenge Organisers | How to use digital twin data to predict demand response capacity

Thank you very much Sneha for responding. On the topic of the competition phase ending, I would like to kindly highlight that in the video at time 26:00 Emily says the competition phase “should end a week before the 19th of October,” and I’m wondering if that is true or if we can trust the overview page that states it ends on the 19th.

View post 0

📹 Townhall Recording & Q&A with Challenge Organisers | How to use digital twin data to predict demand response capacity

Hi @liberifatali in the video at time 21:20, the slide shows that Site F is the private test set and Site E is the public test set. This also implies that our submitted predictions for Site D Capacity are never evaluated. My concern is that the final winning teams will be determined solely by Site F when the goal of the competition is to design a model that works for any site, and I’m asking the organizers if this is a valid concern.

About the back-cast question:
At time 38:30 a question arises that @jack_vandyke and I believe can be interpreted as “are we allowed to use data from timestamps > T when we are predicting for timestamp T?” At time 39:27 Matt answers no to this question, implying that we’re supposed to only use data from timestamps <=T. As the competition overview does not mention this rule, we are awaiting clarification on this topic from the organizers.

View post 4

📹 Townhall Recording & Q&A with Challenge Organisers | How to use digital twin data to predict demand response capacity

Thank you for sharing the fact that Site F is the Private Test Set.

I have one concern with this. We understand that our model should be generalizable to any site. With that in mind, is it fair for the final ranking to depend solely on performance at a single site?

The competition overview says “the private test set will be used for the final ranking.” I fear that it may be possible to design a model that does very well on Site F but is not generalizable to other sites. If there will be additional factors considered when determining the final winners, we would appreciate learning more about them.

Thank you,
Ryan

View post 2

📹 Townhall Recording & Q&A with Challenge Organisers | How to use digital twin data to predict demand response capacity

Hello and thank you for this insightful presentation.
Thank you for clarifying that the Competition Phase ends earlier than October 19.
Is the correct date and time October 15 23:59 UTC as seen in the image that I have attached?

View post 0