Similar bot, way lower skill/rank?


#1

Hey Gents,

I´m wondering why it takes so long for my bot v14 to climb up the ladder (currently rank ~400). It is very similar to a previous version v9 (rank ~200). Inbetween those two bots i had some bugs which dropped me to rank ~1800.

Is that probably the reason why is takes so long to get back to rank ~200?

happy haliting!
n340win


#2

Yes, the first reason is that mu (rank mean) changes less the more games you play. The second reason is that mu is very volatile while you play less then 200-300 games. So your v9 bot maybe is overvalued after a series of victories.


#3

It feels like there is something wrong with scoring.


#4

While the rating calculation is presumably correct, it seems to me the match rate starts dropping off before the bot has reached a stable rating. Especially, what particular bots you keep getting matched against matters, which I’m presuming is a function of your current mean estimate mu - so there may be the effect of your bot getting stuck for an annoyingly long while on a particular rating, before shaking out of it. I’m saying this while being ignorant of the theoretical foundations of rating system and its implementation here though.


#5

ya this looks like it. its slowly crawling up the ladder. winning ~8/10 games… would it make sense to change the matchmaking to overcome something like that?


#6

You can try to reupload your bot. Your will lose your ladder position, but your mu will stay as is. As a result you will return to stable position quicker. I did it with my bot 1 or 2 times when strange rating drops occured.


#7

How to game the system.


#8

Resubmitting an identical bot results in lower ranking. Can anybody reproduce that?


#9

Nah I’d rather just improve my bot than investigating fine nuances in the rating system.


#10

+1 to resubmitting. I submitted 3 broken bots yesterday, which made my ranking fall to -8 (I was actually last at one point!). On submitting a better bot, it climbed a lot for the first few games, then levelled out. Resubmitting that a couple of times fixed things - otherwise you’ll have to wait a loooong time for your rating to climb back up.


#11

Here is my bot submitted 4 times

BOT VERSION RATING RANK GAMES RETIRED ON

Current (22) 61.28 223 83 Still Playing
21 62.29 197 134 02/11/18 17:12
20 61.32 213 40 02/11/18 09:28
19 63.6 157 523 02/11/18 08:50

#12

So, resubmitting doesn’t generally do anything to your ranking (your changes there look like noise). The time you need to resubmit is if your current bot is much better than your previous bot. That’s because it’s very slow for a bot to get more than about 15 ranking points better than your previous submissions, however good it is.

It’s arguable that’s a problem with the rating system, though not one that affects very many submissions.


#13

I think there really is a problem with matchmaking which results in a very slow convergence. I am more than 48hrs after submission of my last bot which is just a tad better than the previous one but it needs over 48hrs to get the correct mu rating. Here is the bot journey:

I am also getting paired with the same player in 13 out of 15 most recent games.


#14

Being paired with the same player lots of times like that usually means they’ve resubmitted their bot very recently, and your bot is judged to be about the right rank to test it against. They’re getting a flood of matches where their opponents are selected to suit them, and you’re involved in those matches.


#15

This clearly proves matchmaking is not working properly. The range of players to play against should be a bit wider. The matchmaker suffers from confirmation bias and tries really hard to keep your mu unchanged. This is why it takes so long to establish proper rank. Maybe somebody with better understanding of trueskill algorithm could chime in, but I don’t think trueskill claim to fame is correct rankings after 600 games(72 hours).