"Why we should probably hold off on questions about magicarp shiny probabilities for now..."

#PokemonGO: [Warning - this gets a bit nerdy] So, we've seen a lot of questions along the lines of "has the probability of catching a shiny magicarp changed since the water event." This is a really hard question to address since the probabilities of catching a shiny are so low in the first place. For example, say we want to know whether the probability of catching a shiny during the water event was higher than in the later versions of the original game, about 1/4000 (I hear). And, for the sake of illustration, assume that Niantic set the probability to 1/400.In this case, by the standards of the field, we'd want to gather enough magicarp so that if the true probability is 1/400, we'd be able to show that the probability is higher than 1/4000 with 80% probability (the standard in the field, using a 5% probability of a false rejection of the hypothesis). In this case, we’d need to catch 5106 magicarp just to be 80% certain that we could say that the probability of catching a shiny is higher than 1/4000 (feel free to check my math here). Ok, this is tough, but with a group of dedicated researchers (or a bot) this should be possible.Now, lets say we know that during the water event, the shiny probability was 1/400. Here I want to emphasize that we really have very little certainty that this was the actual number! But, let’s say that this was true, and that Niantic lowered the rate to 1/500. In this case, we’d need to catch 62,537 magicarp to have an 80% chance of being able to say that the probability of catching a shiny is less than 1/400!!!Note that if the true number is 1/400 and it halved to 1/800 it would be significantly easier to identify this decrease (search for a binomial power calculator if you want to play around with the numbers). However, again, we don’t really know what the number was for the water event — it could easily have been as low as 1/1000, which mean a change to 1/2000 would take a sample of 15,658 to identify a decrease with 80% probability.In short, we don’t really know if the probabilities of finding a shiny has decreased from the water event, and don’t really know what the probability was in the first place — the sample size on the anecdotal evidence is too low, and aggregating reports won’t give us random sample. But, I bet someone is doing some research on this…(cue SR research report on shinies?)Happy Travels! via /r/TheSilphRoad http://ift.tt/2pC2pTP