"The Death of New EX Raid Eligible Gyms: Changes to Niantic’s EX Eligibility Detection System since July 2018."

#PokemonGO: AbstractNiantic’s systems are not automatically checking whether new POIs are EX eligible, and are not automatically applying 'EX Raid Eligible' tags to POIs when they are being converted to gyms. This has reduced the number of new EX eligible gyms. No new POIs created after August 2018 have been eligible, and no gyms at all (either newly created or converted from old stops) since 28th November have been eligible. One exception to this rule seems to be gyms which have received edits, but there is still very little information available about this.Based on the collected data, in order to be an EX eligible gym, the gym needs to fit the following criteria:The centre of the level 20 s2 cell needs to fall inside an eligible OSM area (e.g. park), with the OSM data from ~July 2016. Also, the areas need to be marked as ways, not relations.The gym needs to be outside zones that block spawns (e.g. natural=wetland, amenity=school, natural=water, etc.)The gym needs to not be bound by a closed road (e.g. roundabout) or footpath. (way[highway](if:is_closed());)(New) The POI which is now the gym needs to have been created before 4th August 2018 (± 15 days). (Either as a gym or as a Pokestop.)(New) The POI needed to be a gym before the 28th November 2018.If a gym meets these criteria then it will display “EX RAID GYM” when viewed and will be able to hand out EX raid passes. Gyms not meeting these criteria will not. Portal location edits (and possibly other types of edits) may be able to override rule #4 and/or rule #5 (T.B.D.).IntroductionHi, I’m /u/Tntnnbltn. You may remember me from posts such as the original research into EX eligibility criteria for gyms and the Photo Score model for influencing what POIs turn into gyms. I say this because by I hopefully should have a reputation for (a) having a good understanding of Niantic’s eligibility systems, and (b) being methodical in my data collection and analysis.I’ve created over 200 portals of my own since reaching Level 10 in Ingress earlier this year, and through my own work and coordination with others, I have created over 100 new gyms. Since the discovery of the Photo Score model, I have been influencing which POIs become gyms in order to prioritise gyms that are EX eligible and accessible for large groups of people.My original EX eligibility overpass-turbo query was good, but not perfect. There were always a few gyms which should have been EX eligible but never gave out passes. Since the release of EX tags, we have since been able to show that there are exclusion areas that block EX passes, such as closed highway junctions and OSM tags that blocks spawns (e.g. amenity=school) and this explains why some gyms never became EX eligible.But over the last 6 months, something else has changed. Time and time again, gyms that should be EX eligible according to all known criteria simply haven’t been, with no other explanation. It is something I have noticed through my own gym creation. It is something that has also been discussed in various comment threads on thesilphroad, but without much attention from the wider public. (Even I was not fully aware of this until I started doing some digging a few days ago, and I tend to keep on top of all of the OSM/Ingress topics).And so, I present to you my findings: The death of new EX eligible gyms over the last 6 months.MethodologyI compiled a list of all gyms I had created from April 2018 to present and ran them through osmcoverer. This provided a list of 58 gyms that should be eligible based on being in park areas. Gyms were then inspected to determine whether they have an EX Eligible tag as of January 2019, revealing 31 Eligible gyms and 27 Ineligible gyms. Gyms on the list that did NOT have an EX Eligible tag were checked for tags known to block EX status (http://bit.ly/2B51suv). Exclusion areas explained the lack of eligibility for three gyms (one due to natural=water, one due to being in a park surrounded by a closed-loop residential road, and one due to having a footpath around the playground that formed a closed loop). These three gyms were removed from list, leaving 24 gyms with unexplained ineligibility.I collected dates from Ingress approval emails that corresponded to each gym. These dates are listed in the results table below. The gyms typically went live 1-2 days after the email, however there were multiple time periods in August-September where Ingress portal syncs stopped for up to a week. The decision to use Ingress approval dates rather than the dates that gyms went live was because the email dates were more consistent and easier to find information on, especially during times when portal syncs were not consistent.Each gym was classified as either ‘New’ (if the gym spawned from the very last POI to be added to that cell) or ‘Converted’ (if the gym spawned from an existing Pokestop). For gyms that were converted, the date of creation for the POI that was converted was recorded if known.I searched thesilphroad for terms related to ‘ex eligibility new gyms’ which turned up a number of relevant threads. I reached out to some people by PM to collect more data for my post. The data I received from PMs and through reading the comments in the Reddit thread were consistent with my findings, and some data from others was included in my results table if particularly relevant for showing trends or cut-off dates. I didn't personally check the EX eligibility of gyms sent to me by /u/crusA2009 and /u/newacountz as the rest of the post had been written by that stage, but none of their reported gyms were outliers with other data already collected.ResultsNew POI gymsThis first table shows the eligibility of gyms created from new POIs. The date is the date of the Ingress approval with most gyms going live 0-3 days later. Most of the gym information is from myself unless otherwise indicated.DateTitleEligibleSource2018-05-26Laughton Park PlaygroundYes2018-06-08Queens GardensYes2018-06-19Ken Cantwell ParkYes2018-06-21Millbridge Bridge MuralYes2018-06-24Splash of ColourYes2018-06-26Nolan Avenue Reserve PlaygroundYes2018-06-30Water Tap MuralYes2018-07-06Picton Ecoscape Trail EntranceYes2018-07-08Pinaster Reserve PlaygroundYes2018-07-09Rock Around the CrocYes2018-07-18Harwood Close PlaygroundYes2018-08-01(INGRESS/POGO SYNC STOPS FOR 6 DAYS)2018-08-19Horizon Park Shelter/Picnic AreaNo/u/Chris-Ben-Wadin2018-08-28Nazareth Borough Park Small PavilionNo/u/WormWar12018-09-07Pepsi Ball FieldNo/u/Beezwax20172018-09-09Bindoon Loop Reserve PlaygroundNo2018-09-11Nolan Avenue Reserve Basketball HoopNo2018-09-13(EX TAGS DATAMINED FROM APK)2018-09-20(EX TAGS GO LIVE)2018-09-28Pointless PitstopNo2018-10-07Spielplatz WulfsbreedeNo/u/crusA20092018-11-05Spielplatz KirchdornbergNo/u/crusA20092018-11-10South West Phoenix Football ClubNo2018-11-13Spiellandschaft in BrakeNo/u/crusA20092018-11-19Spielburg aus HolzNo/u/crusA20092018-11-28(EX TAGS UPDATED/FIXED)2018-12-15Merrill Basketball CourtNo2018-12-24Cadoux Promenade Reserve PlaygroundNo2018-12-25Redfox Crescent Reserve Basketball HoopNo2019-01-06Wielki Gościniec Litewski - OkuniewNo/u/newacountz2019-01-07Eaton Recreation CentreNo2019-01-14Esperance Football ClubNo2019-01-16The MarconNo2019-01-16Esperance Netball AssociationNo2019-01-17Ports Football ClubNo2019-01-17Chantilly Park PlaygroundNo2019-01-17Victoria Street Park PlaygroundNo2019-01-19Moorman Park Small ShelterNo/u/Beezwax20172019-01-19Wavering Baseball FieldNo/u/Beezwax20172019-01-19Haven Place Reserve PlaygroundNo2019-01-19Kleiner SpielplatzNo/u/crusA20092019-01-20The CastleNo2019-01-21Bolzplatz im StadtwäldchenNo/u/crusA2009Sometime between 18th July 2018 and 19th August 2018, new POIs stopped being able to be classified as EX eligible. This pre-dates the release of the EX eligible tag system in September, but may be related. For example, the OSM visual map update in late January 2018 used data from mid-December 2017, so it is possible that something was done in late July or early August to prepare for EX tags in September.I included in the table the week that Ingress/PoGo sync stopped in early August. This was not the only time period when the sync stopped (there were at least another two periods in September at least) but given the time period it may be relevant. There were jokes at the time that the button-pusher was on holidays, but it may have been an intentional takedown while they were changing or updating something related to the POI database. It may be that other users reading this can provide more information which will help restrict the time period when new submissions stopped being eligible.Converted POI gymsThe following table shows the date of gyms created by converting an existing POI (i.e. a Pokestop) to form a gym. The left column shows the date that the final POI was added to the cell to cause the gym conversion. On the right is the date when the gym itself was added as a POI. If no date is listed, then the POI is ‘old’ (created prior to July 2018).The ‘Eligible’ tag refers to the current tag displaying on the gyms. A number of these gyms did not have the tag when they were first created and picked up either EX eligible tags at a later date. This will be discussed in more detail below.Date (last POI)TitleEligibleDate (POI that became gym)Source2018-04-27Dalyellup Wentworth PlaygroundYes-2018-05-26Horseshoe Lake PagodaYes-2018-05-30Bunbury Football ClubYes-2018-06-08Bunbury SoccerdromeYes-2018-06-26Clifton ParkYes-2018-06-29Robinson Park GosnellsYes-2018-07-01Len Ferguson ParkYes-2018-07-04Central Park Avenue ReserveYes-2018-07-05Sanctuary Waters GazeboYes-2018-07-05Piara Water PlaygroundYes-2018-07-24Armstrong Park Sign Warton RoadYes-2018-07-27Working With The EnvironmentYes-2018-08-01(INGRESS/POGO SYNC STOPS FOR 6 DAYS)2018-08-04Lofthouse ParkYes2018-06-202018-08-28Church of St. MarkYes-2018-09-04INRI Jesus am KreuzYes2018-07-20/u/crusA20092018-09-09Atelier Farbenspiel (altes Bauernhaus)Yes-/u/crusA20092018-09-13Ranford Road ParklandsYes-2018-09-13(EX TAGS DATAMINED FROM APK)2018-09-20(EX TAGS GO LIVE)2018-09-21Badmington SignYes-2018-10-14Setlers HallYes-2018-10-17Melaleuca Park PlaygroundNo2018-10-132018-10-29West ShelterYes2018-07-08/u/Chris-Ben-Wadin2018-10-29Sindhi Park SignYes-2018-11-06Bunbury Archery - Club RoomsYes-2018-11-07Field #1Yes-/u/Chris-Ben-Wadin2018-11-08Frank Buswell ForeshoreYes-2018-11-15Horizon Park PlaygroundNo2018-09-02/u/Chris-Ben-Wadin2018-11-28(EX TAGS UPDATED/FIXED)2018-11-29Plac Zabaw FalenicaNo-/u/newacountz2018-12-05Kletterparadies SchlosshofstraßeNo2018-09-14/u/crusA20092018-12-06Chatter BoxNo-2018-12-07Siłownia Plenerowa Wisełki/OsterwyNo2018-06-16/u/newacountz2018-12-08Lofthouse Park PlaygroundNo2018-06-172018-12-10Ryde St PlaygroundNo-2018-12-10Amherst PlaygroundNo-2018-12-11WaldspielplatzNo2018-11-24/u/crusA20092018-12-14Spielplatz SaarstraßeNo2018-08-24/u/crusA20092018-12-24Siłownia Plenerowa Ul. ŻwanowieckaNo2018-12-07/u/newacountz2018-12-27Szkółka RoślinNo2018-06-18/u/newacountz2018-12-29Kuhl ParkNo-2018-12-29Outdoor GymNo-2018-12-30Abenteuerspielplatz Campus Quartier SüdNo2018-08-09/u/crusA20092019-01-02Yilgarn Way Adventure PlaygroundNo-2019-01-13Dreiecks StatueNo-/u/crusA20092019-01-15RutschenturmNo2018-12-27/u/crusA20092019-01-16Bayonne ParkNo2018-12-112019-01-17Blaues SpielgerüstNo2018-01-17/u/crusA2009When EX raid tags were originally launched, there were initially some issues. Some gyms which should have received tags did not, including gyms which had previously handed out EX raid passes and a number of sponsored gyms. On 28th November 2018, Niantic ‘fixed’ something which resulted in these gyms getting their tags back (thread #1, thread #2). It was also noticed at this time that some, but not all, newly-created gyms received EX raid tags for the first time (thread). The various comments in the thread seems to indicate that it was only gyms converted from older POIs which picked up tags, and this data support this hypothesis.The EX Eligible column shows the current tag for each gym. I don’t have data for whether each gym had an EX tag when it launched on September 20th, or whether they only picked up tags on November 28th, but /u/crusA2009 recalls that ‘INRI Jesus am Kreuz’ had an EX tag initially whereas ‘Atelier Farbenspiel (altes Bauernhaus)’ only gained an EX tag on 28th November. These gyms were made only 5 days apart, so I suspect that all of the eligible gyms on this table that were approved between ~9th September 2018 and 28th November 2018 only picked up their tags when the 28th November refresh occurred.There are some gyms of key importance in the period of September to November which are important to note: Melaleuca Park Playground (POI created 13th October 2018, converted to gym 17th October 2018) and Horizon Park Playground (POI created 2nd September 2018, converted to gym 15th November 2018). Both were gyms prior to the refresh, but did not pick up tags. Both had the initial POI approved after the July/August cut-off, so it appears that this lines up with the data from the ‘new gyms’ table. ‘INRI Jesus am Kreuz’ did pick up the tag and that POI was initially created 20th July 2018, so that restricts the date cut-off for ‘new POI eligibility’ to be somewhere between the 20th July 2018 and 19th August 2018 (once again, this is the date of the Ingress portal being approved; the sync to PoGo is typically 0-3 days later).For gyms converted after the 28th November, no gyms have become eligible, regardless of when the POI was created. This is demonstrated by ‘Plac Zabaw Falenica’ which was converted on 29th November from an old Pokestop but is not eligible. Meanwhile, /u/GuldfiskenHolm posted that their gym was created half an hour before the EX refresh and it gained an EX tag (link).Explaining the DataThe following is some hypothesising of Niantic’s internal processes based on the collection of data observed…Late July/Early August (possibly 1st Aug to 6th Aug): Niantic runs their entire POI database through an algorithm to determine whether each POI is in an EX eligible area. The POI database includes gyms and Pokestops. This is a static list, and the EX eligibility of new POIs is not added when new POIs are approved.Early September: Niantic uses the data from late July/early August to apply EX tags to all POIs which are currently gyms. POIs which didn’t exist when the list was created (e.g. ‘Horizon Park Shelter/Picnic Area’) do not get tags. Due to some reason (intentional or unintentional) not all gyms receive tags when they should.September to November: New gyms are created, but the process used in early September is also not automatic so no new gyms during this time period pick up EX tags.28th November 2018: Niantic corrects the issue of missing EX tags by reapplying the July/August eligibility list to all currently existing gyms. All gyms created between September and November from old POIs that were part of the original July/August list pick up their EX tags (explaining the eligibility for nine gyms on the list created between September and November). But any POIs created after the July/August cutoff were not on that original list so do not pick up EX tags (this explains the lack of eligibility for 10 new gyms and 2 converted gyms which existed before 28th November but were created after mid-August).Post-28th November: Whatever process was used by Niantic on 28th November has not been used again. This explains the lack of eligibility of 29 gyms from the results table, both from new POIs and old POIs.The Exception… Portal EditsOn 7th January 2019, /u/H2OPsy posted a topic titled ‘Portal Edit forced ex raid tag’. They had a gym which was not in a park area and was not eligible, but the portal was slightly misplaced. The OP submitted an edit request, which was approved in early January. When the portal sync occurred, not only did the gym moved but it also picked up an EX tag.This indicates that Niantic’s system is automatically re-checking EX eligibility and re-applying EX Eligible tags when portals are moved, but not when POIs are created and/or converted into gyms. I have no idea why this difference would exist. Possibly it is so EX raid passes can be cancelled if a gym is moved to a non-eligible area, and this is just a side effect?We do not have data about whether title and/or description edits would achieve the same effect.I was on the fence about whether to mention this at all, as I worry that it will lead to an inundation of useless OPR edits. I am at 12,900 portals analysed through OPR, and I can testify that no OPR reviewers want a deluge of useless portal edits. It clogs up the system and is frustrating trying to tell apart two close-together location edits on a grainy satellite map with no Streetview.For those people think that portal edits will solve their EX eligibility problems, my experience with edits is that they have very rarely even come back. Excluding the portals I submitted yesterday, I have received responses to 263/266 portal submissions. Meanwhile, I have received responses to 6/260 portal edits, with edits dating back to 2017 still not processed. Edits only get received by local OPR reviewers (unlike portal submissions which can get received by people across the entire country) so different areas will have different response times depending on the number of local reviewers. If your area is anything like mine, it will be faster for Niantic to fix their system than to rely on edits to refresh EX eligibility.Questions Still To Be AnsweredWhat was the exact cut-off date for new POIs? So far we know it was somewhere between 20th July 2018 and 19th August 2018, but with additional crowd-sourced data we may be able to refine this date range.Can title and description edits also cause gyms to become EX eligible, or is it only location edits? It could be something inherent to the edit process, or something only invoked when locations change. Also, is it important that the gym in this case moved from a non-EX area to an EX area? Would the same result have happened if the movement stayed within an EX area?Will Niantic update the EX eligible gyms? If so, will the next update make a new version of the July/August list (i.e. all new POIs to date could be eligible) or will it involve doing the same thing they did in November by using the existing July/August list (i.e. pre-August POIs which have been converted to gyms could be eligible but post-August POIs cannot).In the past (pre-July 2018), were new gyms in EX eligible areas always EX eligible when they were created, or did they remain ineligible until Niantic manually refreshed their database? Perhaps a system similar to this has always been in operation but we didn't know because there were no visible EX eligible tags. Perhaps this is why people thought they needed to try to 'activate' new EX gyms before they would hand out the first pass.Closing ThoughtsMy Message to Portal SubmittersHopefully this resolves ongoing questions which I have been seeing about EX eligibility of new gyms. Basically, anything which is being created now-ish won’t be eligible, and outside of portal edits there isn’t anything you can do to make it eligible.If you have a POI approved which will make a new gym and there are two EX eligible candidates – one old and one new – it might be better to make the older POI become the gym. Neither will be EX eligible right away, but if Niantic does something identical to the process run on 28th November 2018 then we might see another wave of old-POI gyms gaining EX tags while post-August POIs continue to miss out.Portal edits are only available through Scanner Redacted at the moment, so if you are not an existing Ingress player you cannot submit edits. If there are edits which improve the quality of the POI then that would benefit the POI database, but trying to move a bunch of gyms by a few inches is going to cause frustration. I don’t know for sure whether title and description edits can also force gym to gain EX eligible status (it does other things like update the portal photo so it might), so maybe correcting a typo in a title or adding a good description could be better ways of inducing edits. If the POI is legitimately in the wrong place, a location edit is valid but consider adding a correctly positioned and correctly oriented photosphere to help reviewers see which location is correct, especially if not clear from the available Streetview or satellite imagery.My Message to NianticDear Niantic devs, I hope that this information actually reaches you (it seems that some issues only seem to get addressed when they blow up on Reddit).The issues outlined above with respect to EX eligibility have a negative impact on players, and in particular is most detrimental to players in rural areas. People in rural areas likely had less areas mapped on OSM in 2016, have less POIs available to turn into gyms, and have less ability to travel to EX eligible gyms. Being able to add new gyms into EX eligible parks was the one hope that a lot of rural communities probably had to potentially gain their first EX gym, but given their rural nature it would take longer for players in those areas to reach Level 10 in Ingress and be able to make portal submissions, and they may not have been able to do this before the cutoff for new POIs in August 2018.It is also going to be a growing area for complaint if/when Pokestop submissions are released to more countries, as more and more gyms will be created which won’t be able to be EX eligible.The following changes would immensely benefit the community:The EX eligibility code is (still) based on OSM data from July 2016. This should be updated periodically, preferably along with the nest data and visual map data so that all three bits of data are in sync with each other.New POIs should be checked for EX eligibility when they are created or converted into gyms, and all eligible gyms should gain their EX eligible tag and status automatically if they meet the criteria to be an eligible gym.If changes/fixes are made, then greater communication and openness with the community about these changes is needed, e.g. through the Developer Insights or some other platform, so that we know the issues have been resolved and what to expect in the future. Often it feels like Niantic is trying to be the 'magician behind the curtain', but researchers in the community have been able to determine how various features in the game are related to things such as openstreetmap.org data and s2 cells. We are ready to hear developer insights that communicate to a similar level of detail to what we already know rather than generic support copy-and-paste statements, or no statements at all. via /r/TheSilphRoad http://bit.ly/2DD3548