Have Prime Network issues seen during last First Saturday and the Field Test been resolved?
During both the September First Saturday as well as the Field Test events, a lot of people saw network performance issues in Prime. During the events, Prime behaved as if there was a very slow internet connection although other apps (including Redacted) saw very fast connections.
Many people suggested that this was a problem on the Prime server end with load balancing or flood protection when there are a lot of simultaneous connections from the same set of IP addresses.
So far we have not seen an acknowledgement of the issue by Niantic. Has this issue been addressed in the meantime? Can we expect Prime to work on the first Prime-Only First Saturday in ten days? (And then even more importantly during the anomaly a week later)?
Can we please get some confirmation that Niantic is aware of this issue? @NianticBrian
References:
Comments
In less than a week, all the servers running redacted can be rededicated to running Prime.
You are assuming things like you are a niantic employee.
Are you?
If not, stop saying nonsenses please.
It would surprise me if they had 2 different server networks, both apps talking to one set of servers appears to make far more logistical and financial sense to me.
Well, they have not been resolved ☹️
FS ruined. Umbra anomalies doomed.
Game experience beyond poor so far.
RIP Ingress.
You can add October for server side slowness too. It was horrible trying to do anything.
As a result of the NIA Field Tests, we scaled servers up to meet demand. Today, during early-morning hours (PDT) we saw increased latency and issues signing in to Ingress. We scaled our servers up and latency returned to normal within several hours. Please note that this was a server-side issue.
For the Oct 12 anomalies, we're using past anomaly data and current Events tab registrations as a baseline, with some additional buffer.
We're now discussing next steps to address First Saturday participants who were unable to participate today during this window. Thank you.
Didn't you have previous FS data to predict clusters load? This answer seems to be not so professional as it could be. We can only pray about upcoming anomaly atm.
Game server is down on FS timing some agent didn't achieve 5000 AP what can be done now.
I hope Niantic are aware that *IF* the scanner fails on 12 October anomaly day, this time around I don't think Ingress players will take it in stride anymore. Agents have taken a lot, and put up with a lot, because it was a free game.
There is so much anipathy at this stage, that I'm talking PoGo like class action lawsuits to get not only money refunded for the ticket, but also for air fares, lodgings etc.
May I, with limited knowledge on server side errors and latency, suggest the buffer to be put up to maximum capacity on anomaly day?
The people from Fev Games have urged everyone to take at least the group photo so they can validate the event so they can salvage as much as they can
I'll get my popcorn ready while watching intel for fun on the said day. /s
Many agents simply didn't drive to any FS as the drive was too far for at the time, a non working scanner. I heard that from multiple agents across many different chats...
I feel bad for the FS participants. I myself had driven a few hours from home to take down a BAF & put up some big fields of my own after our FS finished. Thankfully I at least got the BAF down, but used a lot of fuel for stuff all. Didn’t even get much AP gain as I needed to flip the portal to try reuse it.
It's a bloody disgrace!
Due to the lack of Playability I now have only 2 completed missions of a 6 mission (first Saturday) banner. And with upcoming mission day the image will be screwed because it is not like I am able to travel to that town fast and hopefully finish it.
We also had some first attendees to the fs as well as their first banner. Probably last on both.
A nice show of how to send people away screaming from a great game.
The FS I was at, Bradford UK was fine for its first hour 12pm-1pm (GMT+1)? Then was unplayable for the second hour. I planned to do 48 missions but just about managed 12, which in the grand scheme of things, isn’t the end of the world. I could always go back.
With regards to the usual double AP, awarded for the full two hours. To make this right, Niantic could make the window an hour and quadruple your AP to put people’s AP roughly where it should be, or, and probably significantly easily done - make November’s FS award triple AP? We had a few newbies to FS and I really hope that these technical faults don’t put them off future events.
Finally, I caved and bought the FS bundle consisting of R1s and and hand full of CHS, I bought this minutes before the faults and was not able to use these items. I probably deployed 10 of those R1s max. As I wasn’t able to use my purchase for its intended purpose, the event items during the event, I am pursuing a refund. Yes - I could still use those items but not in the timeframe of double AP and in the FS playzone where the high turnover of portals would net even more AP. I’d encourage anyone else in a similar situation to also contact support. I’ve been passed along to someone else and am currently awaiting a decision.
With computers and software, things happen, for Ingress to work, there are many moving components that have to work in harmony. I’m glad Niantic has held their hands up straight away, and, with great optimism, look forward to how they intend to make good on their issue.
in September, with looming redacted removal I attended a FS In Scarborough, UK. Poor data assumed is what caused the black screen, switched to redacted and had a decent day. Yesterday, October FS no redacted and only 25-30 of game play. Prime was shocking. Black screens, actions not being recorded. Mission progress details missing. We paid for train journeys, food and drinks, all on the expectations we would play ingress. Next week we are heading to an anomaly where you have introduced pay to play scheme. After taking our money directly, you better make sure we are able to play.... you are using up the social tank attributed to ingress. Let’s be clear, FS success yesterday was due to a social groups, not ingress.
NIA.... your XM bar is looking empty....
For the Oct 12 anomalies, we're using past anomaly data and current Events tab registrations as a baseline, with some additional buffer.
Since this is a purely Prime related issue, I presume you're running on new servers for Prime? Perhaps that buffer needs to be bigger than you'd normally expect, given that the last two events were dangerously close to crashing, even though they didn't have significantly increased participation compared to previous events.
Why don’t the servers automatically scale when load and latency conditions demand? With suitable thresholds and alerts to Nia operators to intervene if things look dodgy.
Why wasn’t this predicted by Nia after the Field Test debacle, when exactly the same problems of Prime not coping with concurrent local and global demand were reported via direct individual support chats and through parallel event feedback channels?
Why could Redacted cope on that same Field Test Event day at the same time that Prime was melting down?
Why can Pokemon Go’s Community Days cope with the huge concurrent demand and Prime evidently can’t? Given That all three games famously share the same technology platform.
Why do Passcode releases regularly, predictably get into a similar state of overload and breakdown?
What is the likelihood that the decryption and Passcode challenge at the global anomalies in a week will hit scaling problems and cause intense frustration?
Currently I’m putting ‘show-stopping problems at the anomalies’, at 99%. Please prove me wrong.
@NianticBrian
Thank you for the technical insights, this kind of communication is appreciated.
But my (and many others) big concern is, that you plan to scale anomaly servers based on previous anomaly data.
I assume, you people at Nia scaled the servers during yesterday's FS based on previous FS data .. and it turned out to be not enough. So whatever your load prediction was, it was obviously off the reality. And then it took multiple hours to add additional resources, so to us it doesn't even look like Niantic was prepared that something like this could happen and your confidence level was very high, that your existing capacities could handle the FS gameplay.
Please excuse that this leads to some doubts regarding the upcoming anomalies.
So what ist Niantic doing differently from its previously failed attempts to scale the network to a capacity which can handle this event?
Prime didn't really work in Myriad Helsinki for me, or at least the Redacted fared much better. Prime was just weirdly unresponsive. However, network and server issues have always been part of my anomaly experience. I'm not putting blame on Prime if stuff goes wrong.
On the other hand, taking money for the mere participation is a thorny issue. For me at least, but it seems that for many others too. There's an implicit promise that Niantic has made, and their track record is far from stellar. It's kinda like "fool me once" situation, and they already pulled a bait-and-switch with the late communication.
Why can Pokemon Go’s Community Days cope with the huge concurrent demand
Well, it seems you little know about pogo cd and sz, they often can't too! =)
I'm sure, should the upcoming Anomaly and Mission Day fail like this, it will be pretty hard for you, Niantic, to ever repair the additional damage done to the already pissed community.
We scaled our servers up and latency returned to normal within several hours.
@NianticBrian
... why does it take several hours? Cloud technology allows to scale up in seconds or minutes. Isn't Ingress running in the (Google) Cloud?
I thought because of the big crowd i can't even deploy a resonator to the portal during our IFS, It made us stop because of the server laggy-ness......
As an active tester during fieldtests in Lelystad the Netherlands i was happy that i could switch to redacted, a lot of players who couldn’t had to wait until prime was playable.
A bigger problem was that some players where able to use prime and others not during the playwindow how will that affect the anomaly outcome if one team is able to have enough players that can actualy play the game and even win that round?
Think of it as a teamsport where half of the team is sitting down to tie their shoelaces and the opposite team can play without any hassle and score...
The Prime scanner has been slow all week - ever since Redacted was shut down.
Specifically, in my experience, it has responded slow to actions like hacking. Throughout the week it has taken many seconds, and in some cases dozens of seconds, to get hack output.
This indicates Niantic should have known about server capacity issues for several days.
My general idea is that Redacted was easier on server load, didn't need much resources.
Prime needs more but nobody uses it. So it's pretty well balanced... Until some event crops up and servers just go down on the demand that Redacted never needed.
I'm sure of one thing: if Niantic doesn't turn off some useless prime requests, no anomaly or FS will work.
Let's remove the pics from comm, Niantic... And everything else.
This new scanner is a complete garbage...we want Readacted back!
I tested both apps by doing the same action.
start app, wait to see portal, simple hack, wait until animation is passed to see what gear you get, close app.
there is a big difference in data usage, other agents say that cache of prime was not loaded but i did this at home with only one portal in range and prior to that i had prime running on wifi to load the cache.