Similar bot, way lower skill/rank?


#1

Hey Gents,

I´m wondering why it takes so long for my bot v14 to climb up the ladder (currently rank ~400). It is very similar to a previous version v9 (rank ~200). Inbetween those two bots i had some bugs which dropped me to rank ~1800.

Is that probably the reason why is takes so long to get back to rank ~200?

happy haliting!
n340win


#2

Yes, the first reason is that mu (rank mean) changes less the more games you play. The second reason is that mu is very volatile while you play less then 200-300 games. So your v9 bot maybe is overvalued after a series of victories.


#3

It feels like there is something wrong with scoring.


#4

While the rating calculation is presumably correct, it seems to me the match rate starts dropping off before the bot has reached a stable rating. Especially, what particular bots you keep getting matched against matters, which I’m presuming is a function of your current mean estimate mu - so there may be the effect of your bot getting stuck for an annoyingly long while on a particular rating, before shaking out of it. I’m saying this while being ignorant of the theoretical foundations of rating system and its implementation here though.


#5

ya this looks like it. its slowly crawling up the ladder. winning ~8/10 games… would it make sense to change the matchmaking to overcome something like that?


#6

You can try to reupload your bot. Your will lose your ladder position, but your mu will stay as is. As a result you will return to stable position quicker. I did it with my bot 1 or 2 times when strange rating drops occured.


#7

How to game the system.


#8

Resubmitting an identical bot results in lower ranking. Can anybody reproduce that?


#9

Nah I’d rather just improve my bot than investigating fine nuances in the rating system.


#10

+1 to resubmitting. I submitted 3 broken bots yesterday, which made my ranking fall to -8 (I was actually last at one point!). On submitting a better bot, it climbed a lot for the first few games, then levelled out. Resubmitting that a couple of times fixed things - otherwise you’ll have to wait a loooong time for your rating to climb back up.


#11

Here is my bot submitted 4 times

BOT VERSION RATING RANK GAMES RETIRED ON

Current (22) 61.28 223 83 Still Playing
21 62.29 197 134 02/11/18 17:12
20 61.32 213 40 02/11/18 09:28
19 63.6 157 523 02/11/18 08:50

#12

So, resubmitting doesn’t generally do anything to your ranking (your changes there look like noise). The time you need to resubmit is if your current bot is much better than your previous bot. That’s because it’s very slow for a bot to get more than about 15 ranking points better than your previous submissions, however good it is.

It’s arguable that’s a problem with the rating system, though not one that affects very many submissions.