"How does biased reporting effect the shiny rate - more as you might think"

#PokemonGO: A little bit over a week ago /u/JurianPec did a great analysis of the shiny rate of Sableye. You can find the article here: http://ift.tt/2hbQpc5 solid statistical work has been done (bootstrap), even a discussion of the effect of Go Plus data. But in my view the elephant in the room wasn't addressed - neither in the article nor in the comment section. What effect would biased reporting have on the outcome?Trolled data has been mentioned and corrected (5% of data got removed because of trolling actions and the reporting of data got closed) - but biased reporting is much more likely - and more difficult to guard against. So let me define what I mean by biased reporting: A biased report would be a SilphRoad Reader who submitted (true and correct) data. But he only submitted the data because of a special motivation - in this case - because he/she did catch a shiny. Having caught that shiny might just have offered the extra bit of motivation to report his/her data/First let us start with the original data:No PlusWith PlusTotal14379222365642188058444There has been some revised data - so while I calculate a rate of 1 out of 263 later data is closer to 1 out of 250 or 256. But to be honest - this is the least of our concerns.Now just for a moment assume that x% of data being submitted because a shiny was caught (biased reporting). I don't know how big x is. The only data point I have is >20% false data from a survey about gen2 starters hatching. I wrote about it here: http://ift.tt/2zdFHIX made a table for x (as I don't know the exact value), the number of shiny bias reported and a corrected rate for shinies. So how do I calculate the corrected rate? I take the total number of shinies (222) and take away the biased number. I then take the total reported (5844) and calculate 5844*(100%-x%) as the total of unbiased reported data. There is one additional assumption here - it assumes that biased reports on average report the same total as unbiased in each report. Nothing I can do against that without additional data.Now let us look at the data we get:x%Biased Shiny% Shiny BiasedCorrected Rate0%00%2631%188%2832%3516%3073%5324%3354%7132%3715%8840%4156%10648%4737%12456%5528%14164%6659%15972%84210%17780%115611%19487%186812%21295%504213%229103%-6825First at all - let us look at the last entry. My model predicts at 13% biased reporting a negative drop rate. Off course this makes no sense. But it is useful as it gives an upper limit of where the biased reporting could be.But here is the point: How can we be sure that biased reporting is a lot smaller as 5%. Even at 2% we already have a shiny drop rate of smaller 1 out of 300. The drop rate difference between Go Plus uses and non Go Plus users is <10%. This is irrelevant if we can't ensure that we can't ensure that biased reporting happens or that steps are taken to minimize the effect of biased reporting.Edit: There was a comment when someone admitted to deliberately report grinded non-shinies. So let us look how this effects the data. Again I look at x being the number of repondents who report just to spite the data because they are unlucky. I assume they have grinded an average of 300 non shiny.x%Biased non shiny% biased non-shinyCorrected Rate0%002631%52959%2392%1059018%2163%1588527%1924%2118036%1685%2647545%1446%3177054%1207%3706563%968%4236072%729%4765582%4910%5295091%2511%58245100%1So reporting 300 non shiny just beause you got unlucky has an even greater effect as reporting 1 shiny (assuming 32 total caught). But this is on a equal number basis. I doubt there are a lot of reports with >300 non-shiny rerorted. I might have to ask the author of the original work.The issue is - both has a significant effect and only safe guarding against such reports and avoiding them will lead to accurate results. Cutting down on biased reports yields more for the quality as getting more reports.And no - I don't expect both to even out - but it will lesson the impact. via /r/TheSilphRoad http://ift.tt/2zxcNEu