"Saving The Nest Atlas: Our Expenses, "Enterprise Level" Usage, And Open Source"

#PokemonGO: Twelve months ago, when the Silph Road was a brand new community, we Executives gave periodic 'Quarterly Reports' to our travelers - updating all on the Road in regards to our network's growth, milestones, goals, and challenges.Well, today it's time to give an update in the same fashion as our early reports, travelers.The Toughest Hurdle The Silph Road Team Has FacedI'd like to update everyone on something that took place roughly a week ago.The Global Nest Atlas has been a wildly successful community effort. From our humble beginnings (back when you couldn't even see multiple species on the same map shudder) to our current state where within 24 hours of a migration over 10,000 nests are verified in a single day, the project has seen enormous, consistent growth. In fact, we have more travelers using the Atlas now than at any prior point in time.But there's a downside to our network growing as large as it has, travelers.Basemap tiles are expensive for web apps. "Tiles" are the square images that stitch together to make the map. Here's an example tile. Every time you use the atlas you are served dozens of tiles - and so is every other traveler using the Atlas.The Global Nest Atlas was built on Google Maps first. But we quickly exceeded the 25k "map views" per day quota limit. At that point we reached out to MapBox and described the nature of our community and how our project needed heavier usage, but wasn't a profit-oriented site that could afford their 'enterprise' tier.MapBox said their enterprise plans started at $25,000/yr but that they would be able to be flexible for a unique situation like ours. You see, very few sites need basemaps at the volume we do. And the sites that do are all larger companies that can afford super-expensive pricing tiers. MapBox also gave us a few months free to migrate over and get stable - which was awesome. We began earmarking $2-$2.5k/mo. of the ad-revenue budget for basemap tiles in preparation for when our free months expired.But then disaster struck.When MapBox finally gave us a quote - it was over 7x higher than the last numbers we'd discussed: $180,000/yr. So instead of the $2-2.5k/mo for basemap tiles, they wanted $15k/mo. And they had already charged us $15k for a month at that rate. (This bill grew to $18k+ within 3 days.)Needless to say, travelers, the Atlas generates only a small fraction of that revenue, and we still have large infrastructure costs for the rest of the web app (unrelated to the basemaps).We were pretty shocked.At this point, Google Maps Enterprise plans were too expensive, MapBox was too expensive, and our conversations with Esri were looking like they'd be far too expensive as well.No one serves basemap tiles in a cost-effective way for community projects like this. So, at this point, we had only two paths forward: shut down the web-based Atlas or ditch all the map companies and serve the damn thing ourselves.So What's Happening?Fortunately, thanks to the amazing community of volunteers at OpenStreetMap, it's possible (but challenging) to download the entire planet, render the tiles, and serve them yourself like your own Google Maps. It isn't easy, and it takes Dev Ops expertise. Fortunately, our team has Marco Ceppi - who's been working tirelessly with us on this. The man has almost single-handedly saved the Silph Road's Atlas.After sprinting a for over a week, we finally scaled out our own basemap tile server last night. So, this is why the map looks a bit more 'beige' today. The colors are different, as they're not from MapBox anymore. They're from us! (Based on OSM data in a giant database.)This comes with strings. We need to continually update the map or it will become outdated. We plan to do this weekly - the current map is about 1 week stale. (This takes a lot of processing horsepower) We also need to render all the imagery in Retina, not just standard definition. And of course, we need to build and maintain more moving pieces to be able to serve the millions of basemap tiles seamlessly - and that has been very challenging. It's also expensive in its own right - but nothing NEAR as expensive as the enterprise plans we're ditching to do it.For a full technical writeup, check out the Silph.io blog Marco updates about the process. It's a good read.So Are We Out of the Woods?We're close, travelers.Last night we turned off MapBox. (Thank goodness - they were charging us like $1k/day by the end)But we still have some kinks to work out. Stay tuned, the map will keep getting better over time, but please be patient as we work to get retina tiles rendered and the colors tweaked for improved readability and aesthetics.Any Other Cool Feature Coming?Yes! The long awaited 'vetting' features the Atlas has needed are in the works. These have taken backseat to our more pressing financial perils, but now that things are stabilizing, we will be able to roll these out sooner than later. :) Stay tuned for some major improvements to the Atlas' capabilities as a whole.Do You Need Funding?For the first time, we're stuck with a bill we can't quite pay, travelers. We've been pretty chapped at MapBox, to be completely honest. I personally feel significantly bait-n-switched by their sales team. To go from a $25k/yr ballpark quote to $180k/yr without any conversation is just bad business. I love Leaflet and the vector tiles they're pioneering, but this experience went from a great one to one that has left a very, very bad taste in our team's mouth.The bill they're serving us is not in our current capacity to pay.We've never asked our travelers for PayPal donations; we've never set up a Patreon. But as sad as it makes us, we could use a little help making this transition financially. If you've gotten value from the Silph Road team this year and are in a position to give, we would be forever grateful if you would help us make the transition off of enterprise pricing at MapBox and onto our open source solution. We've set up a GoFundMe page for any who wish to help.No worries if you can't or aren't interested in contributing, travelers. The Silph Road is a community built around a game - not a political cause or social injustice. We don't expect it of anyone. But many have asked how to contribute over the months, and we have typically just pointed towards our Gear Shop. This time, we would welcome the help. We don't need ongoing contributions - just help with our last MapBox bill to be rid of that yoke. Moving forward, our operations will be within our ad revenues and our budget will balance without help.Parting WordsWe've never faced an emergency of this scale on the Road, travelers, but even in our most desperate state, I have to say I'm blown away by the wonderful folks who've stepped up to help the Silph Road become home in the Pokemon GO community. Marco Ceppi (who we all owe so much) has donated more than time to this (and past) emergencies. And the good folks who send us error reports or kind words via the subreddit, our email, or Twitter, you make the tough days worth it.Now, head back out there, travelers. These 5k eggs won't walk themselves.- Executive Dronpes - via /r/TheSilphRoad http://ift.tt/2i3IqMS