How CAL false positives probably mean the end of large ops
In another thread someone suggested that you only had to worry if you thought that players would be "very likely hindered" by CAL. I was going to reply in the thread but this seems like a big enough issue to make it a top-level discussion
Imagine you are planning an op that is large enough to cover, oh, let's say 20% of the continental United States. How many agents do you think you need for that? Clearly the answer is a lot, and let's just say 100 agents, although that's probably low. A fair number of the portals that players will be going to will be remote, rarely-accessed portals, some holding long links. Finding enough agents in the right places might mean reaching out to a semi-retired agent and sweet talking them into opening the scanner and taking down a blocking link.
So, we have 100 agents for an op, each going to a different portal, and each of them will either be taking down blockers or throwing links, often from CAL-enabled portals. Your op doesn't succeed unless each and every one of your agents is successful at their task, and because of this you only need a low false-positive rate for your spectacular op to fail.
What happens to your 100-agent op if there's a 1% false positive rate? It will fail more than 63% of the time! What about a 2% false-positive rate? Your op has a nearly 87% chance of failure.
50 agents? Nearly 40% chance of failure at 1% FP, nearly 64% at 2% FP rate.
200 agents? Nearly 87% chance of failure at 1% FP, more than 98% at 2% FP rate. More than 63% at 0.5% FP rate.
Even with a small, 20-person op there's more than 18% chance of failure at 1% FP, and about 33% at 2% FP rate.
Let's look at this from a planning perspective. You have an idea for a crazy large op so you map it all out and figure out your personnel needs just like before. But what now? Fewer people are playing than before so recruiting is harder and you might have to pull agents out of retirement. Your op is a lot more work than it would have been a few years ago.
In addition to all of the other challenges you need to think about how to work around CAL. If you just sent the same number of people as before your op will probably fail. What now? Double up on people, but where are you going to get them all from a smaller pool of active agents? Don't forget that people will be reluctant to go out if they think Niantic has a good chance of screwing it up for them.
RIP (most) large ops.
Comments
1% is absurdly huge, that's 1 in 100, at that point the system would be considered a failure. Assuming all agents play in the previous week I would say it's more likely to be 1 in 10,000, or 1 in 100,000.
Now players can see that how unreasonably dissatisfied some players are, while previously frequently urging for "come on Niantic do something to improve anti-cheating while I have some good ideas as I'm a senior software engineer", after Niantic really did something. Something that has not proven to be a failure with as much as 1% false positives yet. Maybe some players just care more about showing their cleverness than anti-spoofing.
Seems the author of this thread really enjoy establishing scarecrows to target at. Then I would say, now that 1% is raised from your pure imagination, why not jumping to a conclusion from nothing and intentionally assume that the false positive rate is 50%? 60%? That would help develop your exaggerated and illusive theory and successfully get players frightened about the CAL system when they actually don't need to. Not to mention the fact that underlying infrastructure of CAL system has already been developed several years ago and has already been used on a small scale before the announcement of CAL warnings.
Amused to see players don't even know previous "link ban" (which means they hasn't really done much OP) jump to criticize the (not that completely new) CAL system because "it may prevent players from doing OP". More "hey Niantic you did things wrong again" propaganda and drama wrapped in the form of "technical analysis", please. Popcorns in hand.
"RIP (most) large ops"? RIP my popcorns.
I think the biggest problem with the whole CAL system is that Niantic didn't give enough of an explanation. I fully understand that this is very likely done so cheaters can't figure a way around it, but in turn, it has created a lot of uncertainty in the community. For example, what does "regularly playing" mean? Is it once per day, per week, or even more then just the daily hack?
The portals which are CAL locked are also kind of a mystery. "Known strategic Portals" is fair enough, those are most likely portals that have already been spoofed once and got restored by Niantic, as well as other known portals. But how is the "dynamic list of Portals" determined, and how often is the list refreshed? Are they based on portal density plus activity, which would most of the time indicate portals that are more out of the way, is it portals that have beem held by an agent for X days or have X amount of MU or a long link hanging from them? And if I go check this portal for CAL, how can I know that once the OP starts the next day, the portal hasn't been locked.
I 100% do believe that this is an amazing addition, and assuming Niantic gives a tiny bit more information about it, it would calm a lot of agents that are currently in arms about it. It would be even better if there was a way to "announce" an OP to Niantic with agent names so they can get cleared before an OP, not just after they ran into a CAL portal during which they'd have to wait for support. I personally do understand the worries about the system, however I still think this is a good idea.
Many people that play in the large fielding style of gameplay know what hard anchors get spoofed on a regular basis. Just make sure the agent throwing the link is legit. This isn't new.
I don't think a 1% false positive rate is high for Niantic, but we'll see. CAL-enabled portals don't get a lot of traffic pretty much by definition, so we probably won't know for a few months how often people are getting blocked incorrectly. We've already seen one reasonably-credible report on here, though.
As for my personal bona fides: I've been doing large fields and strategic defense type stuff since 2014. The largest one that I've designed and thrown myself was in the western US and was around 9.2M MU per layer but I've worked on ops that were significantly larger. And yes, I know that anchors get spoofed on a regular basis... one of the most recent ones that I've thrown was being spoofed down before I even got back into cell service, although Niantic seems to have shut the spoofer down after they took out two layers.. An anchor got spoofed down a few days later, so two spoofs on one field. Yay? I've also researched and reported a lot of spoofers and had strategic durables restored on several occasions. I've also been part of large ops that failed because of Niantic's errors rather than anything that was under player control.
For large ops it's reasonably common to pull in casual players and semi-retired ones because they are conveniently-located for blockers. You know them personally and can vouch for their legitimacy but will Niantic's algorithms think that they're legit when they have barely played for the last year and then show up out of the blue to take down a long blocker?
My point remains: One false positive is all it takes to cause an op to fail.
You have no proof that semi-retired players are going to trigger CAL warnings more easily as you said. That's your wild guess. A system if designed well won't work in this way. CAL system is supposed to determine "how different a specific player is playing from normal players" while he/she was playing, not "how heavily a specific is playing on a regular basis". The last thing that Niantic would do is to punish anyone who don't play often, considering the fact that there are not many new players incoming and Ingress game calls for players to not be shutdown. Moreover, Niantic is much more reluctant and conservative to make players restricted (even if some behaviors of them are suspicious but not too absurd) than anti-spoofing advocates like me.
"Technical analysis" built on wild guess is nothing but drama. But I've grabbed more popcorns so feel free to do more.
I've seen the unable to link protection kick in, for others and for myself. It also appears to be 'catchy'.
Group of us at a remote portal trying to do very long links, one agent could link, then no longer able to.
Dropped keys to another agent, they could get one link in, then no more. And so on. Lucky there was a whole bunch of us and we completed the op. My biggest gripe was my wifes longest link score now beats mine! :)
This is the link ban and "underlying infrastructure of CAL system" I mentioned. Such restrictions and protection already exist before the formal announcement of CAL (while the author of this thread don't know about it)
With the formal launch of CAL system, it seems that players can now do tests by for example recharging the portal. They don't have to do actual links before they could determine whether shortly after they could do the strategic links as planned or not. If they receive CAL warnings then they could do an appeal which is supposed to be reviewed within probably 24 hours. The explicit lifting of CAL restrictions with appeals by Niantic customer support makes OP planning and execution a little bit safer.
In this case, it's possible that the formal launch of CAL system actually adds some transparency to this blackbox and is making the job of doing OPs to some extent more convenient, instead of more difficult. What do you think?
In summary, the point you are trying to make sounds like:
One of your team members was playing Civilization VI too happily last night that he failed to wake up and arrive at planned strategic portal. For unknown reasons your whole team is counting on him for that portal with no rehearsals and no backups and no backup plans and thus your OP fails. All blame Niantic. Bad Niantic. How dare Niantic not making your OP succeed? The entire world is supposed to lead a path to your OP's success. If the plan is not executed as expected then there must be something wrong with Ingress system, not the fact that you are counting on a single player and the chance that no accidental things would happen during your OP upon every single participant.
Rather than appreciating Niantic's minor step forward in anti-spoofing and talking about how to make CAL system more effective, helpful and accurate, simply deny, deny and deny the system with your own assumption and hypothetical situations not supported by data. Make a conclusion ("CAL system will 100% fail") first and then assume a ratio that is much higher than accidental events (to make readers think something accidental and of extremely rare occurrence would definitely happen for sure and affect a lot of players) which we didn't see in the past whole week, and to which your conclusion could ascribe. That's very constructive.
Glad that Niantic rarely listens to players otherwise they won't be able to develop anything. Whatever they do, there are players denying that. This time they are even denying a system that has been running for years with not MANY complaints emerged! Oh god, is there anything that they won't deny?
In Niantic Wayfarer, agents can see their reviewer score, but don't know how it is calculated.
Could such an approach work for CAL scores?
It's a non-working system. Let me explain with a specific example. This is how the profile of a malicious violator looks like.
This intruder "flies" throughout the area and is an active player. This player observes the timing and so on. How are you going to punish the cheater in this case? Especially if the player is familiar with the leadership of the faction of his country and the complaints that were repeatedly filed against this player were rejected and the player continues to fly as if nothing had happened? We have indestructible representatives of the resistance who can do absolutely anything! These are active agents who actively trade by installing bots to protect their portals, they trade by using cheats to destroy portals in 1 second, and much more. And this has been happening for years! Niantic can't ban such an agent!
What does this have to do with protecting specific portals.... move that to the spoof thread.....
@LuoboTiX The author of this discussion is well aware of the existing mechanisms, and the false positives mentioned by @SSSputnik have been all too common over the years. That's why I don't think a false positive rate of 1% is a high estimate at all.
And you think, will Nia can protect the portal from such a cheater?
Need more information on the portals in question, but this isn't the place, take it to the spoofing thread ....
How to stop worrying about CAL with not worrying about what if situations...
The game is definitely past its prime (sorry, had to) with playerbase so let's see how this system plays out before what ifs.
They won't and they definitely shouldn't. We didn't know anything about CAL until a topic was made so who knows how long this was tested. I would rather be in the dark about methods against spoofing.
While "all too common" is an additional "assumption", I won't call for you to make efforts to prove it with data, though. You don't have to. But if you would like to use it to make points for other assumptions you still need numbers and examples.
Or, let's just concentrate on the complaints. If there are indeed 1% false positives as you firmly assumed, then we will see piles of examples here on the forum within 1 month. I bet that it won't happen. Others are welcomed to place your bet.
CAL system cannot stop pro, well-prepared spoofers. Such spoofer may have accumulated high CAL scores.
However this system from what we could see can prevent malicious access to strategic portals by not that pro and well-prepared amateur spoofers. For example, someone saw a large blue field appearing remotely and would like to take it down with no planning in advance. He trained a new account or login into his backpack L4 account, virtually flys to that portal. The moment he tries to apply Jarvis he is going to be blocked by CAL system. This is a common scenario during XMA battles and amateur spoofers are much more than pro spoofers.
CAL system's not helping stop pro, well-prepared spoofers does not mean the system is not working. It's just working to some extent. An anti-cheating system working to some extent at the cost of generating not many false positives is better than no system at all.
@LuoboTiX "All too common" is based on years of experience with ops and working regionally. Since you're engaging with this discussion presumably you have a similar level of experience with tactical and strategic planning and operations and you' have the same level of information that I do.
@LuoboTiX , so we have this CAL system which is a sort of anti-cheating system and take care of amateur spoofers. Nonetheless, the CAL system CANNOT STOP professional spoofers. So, we would just have to settle for an amateurish or mediocre anti-cheating system because it is better than none at all? The legit players of this game deserve more than a mediocre anti-cheating system, by improving the way Niantic handles the spoofing and multi-user "bot" complaints, deliver a better and more sophisticated anti-cheating system than the CAL.
Appreciating CAL system has no consequence on "settle for it". I support your pursuing a better, competent system that can identify professional spoofers while a minor step forward on anti-spoofing is still a step forward.
OK - my final thoughts (for now :)
1) Let's give the new system a go. We can always whine when a big op fails coz of it. (We're good at that).
2) Pro spoofers may be able to gain a high CAL, but get banned once they get reported for a strategic portal takedown. Would they make the effort over and over for this? (Especially given we can now reinstate portals and links very quickly compared to past times).
Why not be more transparent about this? Let agents see their (and others) CAL score on the profile, and make it clear what type of CAL access level is required on Intel.
This way you could plan around it; somewhat. In any case I think it's good they are actually doing something...
i dont think they want go out public with that to prevent abuse.
As practice has shown, pro spoofers did not beat the support portal, but crossed the portals of the bed :))
It does not help
By the way, the ability to link under the fields will really, really help them in thisthis 😀
I have raised this before in other threads.
Any new open-loop anti cheat methods will fail.
And as much as people dislike scanning wayspots.
This is currently the ONLY closed-loop method to prevent cheating from spoofing.
It will however, not prevent multi accounting. That needs a different mechanism to prevent multiple accounting. I have played another ARG where there is an effective multi accounting prevention system, it will have to be tailored to each of Niantic's games.
I have outlined for potential big ops how this can work with limited to BGAN service only.
1. Scan and upload a portal, that is in good cell comms and is as close to the actual portal that you are visiting.
2. Scan, but upload later, the strategic portal.
2a. If it is at least possible for one person in your party to upload on the spot, this is better, because this enables an in game mesh of said portal.
3. Because you scanned the portal mesh, it can be verified that you are there, even without uploading the mesh on the spot.
4. The more players that upload scans of the strategic portal, the more accurate the on site verification will be. Again if in low signal or on BGAN, upload those scans later, and generally as soon as you are back in good cell comms. As the time stamp of that scan you had done will coincide with the activity at said portal, proving with what is effectively a 3rd Factor Location Authentication.
@Jo0Lz Showing people their score would be an incredibly effective method for teaching people how to work around the blocks. Bad actors would just spin up tons of accounts and try different things to see how their score changed.
@Grogyan You're still overlooking the fact that not everyone has a scanning-capable phone. Also, I understand that most people typically scan the ground or something rather than producing a useful scan of the actual portal. How would Niantic instantly verify that I had scanned the actual (rarely-visited) portal? If I'm on the top of a mountain at 2 a.m. to take down a portal (and I have been) and there's not enough light for me to get a good scan what happens?
@Hosette I have addressed the fact that some people have got devices that are not able to scan.
The short answer is something that they are not going to like, but is still necessary.
Your device is too old, buy a new one.
I have also proven that those on a limited budget do not even have to buy the latest and greatest mobile on the market. A 2year old device is more than adequate.
<quote>If I'm on the top of a mountain at 2 a.m. to take down a portal (and I have been) and there's not enough light for me to get a good scan what happens?</quote>
On the one hand it is extremely unsafe to climb up a mountain at such a late hour. Though I personally have been in areas late night that are unsafe, and have to tread super carefully.
I raised the inability to scan after dark with @NianticBrian sometime ago, to be able to turn on the flash of your mobile. If this isn't enough for a scan, then use a torch.
Again, reiterating, with offline ML of the wayspot to detect the correct object is the right one, the more scans uploaded (if in low comms areas, upload later) will enable a really good chance that an after dark scan will also be valid.
@Grogyan You fail to understand how "authentication" works.
Authentication requires that the entity doing the authenticating (Niantic, in this case) know the correct answer to the questions being asked. In this case, the entire problem is that Niantic does not know the correct answers:
As it stands, your scheme is easy to break: Nefarious actors can already fake their location coordinates. They can fake scans of a portal merely by faking their location coordinates, building whatever (plausible) object they'd like of a portal in a convenient location of their choosing, and scanning that object. And they can submit enough of these scans to overwhelm those of actual, legitimate Agents.
And that's ignoring the fact that real world locations aren't static. Just because the first 100 scans of, say, a bench show just a plain bench, doesn't mean that when the 101st scan comes along, some artist hasn't decided to weld a six foot tall chicken in full plate mail to the bench. How's Niantic to know that the chicken isn't now a legitimate part of the portal?
And that's all ignoring the fact that CAL looks designed to stop nefarious actors before they act. The best your scheme can do is to catch nefarious actors after they act.