"Silph Research Group Survey: Go Battle League Ratings"

#PokemonGO: We'd like to inform you that the Silph Research Group is conducting a survey on the Go Battle League ratings and how they are calculated. Based on our experience with a study we conducted within the group (see below), we set up an updated survey for the whole community to take part in.https://ift.tt/39Ujkrg you just hit rank 7 or are about to hit it very soon, please fill out the form or save it for later and help us disassemble the formula behind the ratings.Your help is very much appreciated.Pre-season surveySeveral weeks ago, an internal survey was circulated within the Silph Research Group in order to investigate the factors involved in the initial rating for GO Battle League.We asked for the following features from trainers that had reached Rank 7:Rank 7 RatingApproximate win rate when reaching rank 7XPAce Trainer Badge CountGreat League Veteran Badge CountUltra League Veteran Badge CountMaster League Veteran Badge CountHero Badge CountDate when reaching rank 7Wins in first 5 battles in GBLWins in first 10 battles in GBLFrom this, we inferred several other features, for exampleLevelBadge Ranks (None, Bronze, Silver, Gold)Highest Badge Rank (and 2nd highest, 3rd, 4th, 5th)Highest League Badge Rank (and 2nd, 3rd)Based on other work done in the community, we were specifically testing the hypothesis that one or more of the badges displayed on the GO Battle League interface are used in determining the initial ranking. The data set ended up being too small to make solid conclusions (n=89), so this post is a combination call for data, as well as some preliminary findings.Before we get started, note that we conducted the survey days after many people reached rank 7, so several features had to be recollected from memory, reducing the quality of the data. In case a feature was left out by a researcher, we filled it with the average of the dataset.FindingsLinear regression was the primary tool used to analyze this data set. Since the number of data points is fairly low, we employed leave-2-out cross-validation for model selection purposes.Starting from the simplest model which only contains the bias term (i.e. fitting a constant to the data), we successively added more features while making sure the validation error doesn't go up.For error measurement, mean squared error was employed.In the end, the best model we could find uses the features Win Rate, Level, sqrt(XP), and Wins in first 5 battles (on top of the bias term). Here are the results: We sorted all 89 trainers by their ranking (orange) for better readability. Their rating prediction is given in blue. The less vertical difference between a pair of orange and blue points, the better.The full formula readsRating = -7840 + 1134*Winrate + 227.1*Level + 0.08089*sqrt(XP) + 111.5*(Wins in first 5 battles)A graphic comparing the observed vs. predicted ratings can be found hereDiscussionSanity check: Except for the bias term, every weight is positive, which is a good sign. The included features are all "positive" ones (in a sense that the higher the value of that feature, the better we expect the rating to be).It definitely strikes as odd that both experience and level are included as separate features. We also would've expected badges to play a role, especially because they're shown in the GBL screen. While we were in the position to choose between including XP and any kind of badges into the model, XP provided the most merit. Especially when noticing that sqrt(XP) is even better. We believe that we missed a feature that scales similar to sqrt(XP).Ratings above 2800 are mostly underpredicted, while ratings around 2k-2.8k are mostly overpredicted, further hinting at a feature we might've missed. via /r/TheSilphRoad https://ift.tt/3d3V3ki