Agents uploading fake scans - how can we support fair play?
I'm aware that the scout medals can be somewhat of a controversial point. While some people enjoy contributing to Niantic and improving the overall quality of portals, other people feel that farming out scans to players is free labor and refuse to do it (even though not even Googlemaps cars can reach certain portals, it's beyond me how people think it's Niantic's job to scan).
That being said, everyone is free to scan if they agree and ignore the medal completely if they disagree.
I happen to belong to the first group of people - I enjoy contributing and uploading pictures (even though the photo sync bug's still not working, I'm stubbornly sticking at it), so I take my time to scan portals properly and claim the controller to work on my medals. This sometimes means some work, as some portals are hard to scan, difficult to find, displaced or straight-up non-existent. Other times, there are bugs that ruin your scans, and you have to start from scratch. It's time-consuming - but after all, there's merit there in the medal.
My issue is with agents who systematically upload fake scans. I've seen an agent upload scans from displaced portals (even portals that irl are a 5-minute walk away, no joke). I've also seen impossible numbers of scans on scanner profiles. I've seen an agent drive under a bridge that's a portal and upload the scan later (how on Earth were they able to scan anything usable from a moving car on a bypass under a bridge?). So here are examples of agents who want to scan everything regardless of what they're really scanning for the sake of getting points towards the medal.
I've also seen an agent stand right in front of a portal, jab their arm in and out with their phone pointing at the ground, stop, tap their phone, walk up to another portal waving the phone at the ground, stop, tap their phone wave it some more at the ground... A little while later, they claimed the scout controller with several scans. There were perfectly scannable portals in front of this person, but they chose to scan the ground. In this case I assume the argument is "I don't want to do free work for Niantic" and yet, they scan.
These are the instances I've seen personally - I'm sure there's much more... And worse.
I feel this sabotages not only the work that honest scanners do, but also the whole point as to why Niantic implemented portal scanning for future features. I think that if certain agents disagree with portal scanning, they simply shouldn't scan and that's that, ignore the medal entirely. It's disheartening to see dishonest agents make a fool out of those who do put in the work! Their medals are based on lies and are getting equal merit to the ones achieved honestly.
How could this be controlled? Is there any way to comb out the cheaters? Does anyone know if there are any plans to?
I don't think there is a way really. You could check new scans against existing ones, and if they differ too much, they get rejected. Problem with that is that one, if the first scan is fake, Niantic can't check it against anything and later legit scans might be rejected, and two, if you scan something big like a building, you'd have to start where the first scan was so the system can verify the new scans.
Another way could be to wait until there are a few scans, then compare them and throw out any outliers with sanctions against the agent if too many of their scans are thrown out. That again however relies on the quality of every scan and the amount of them.
I think awarding scout Scanning a lot, in terms of proving location.
Also as many have pointed out that some are just not doing proper scans, Pokémon GO too.
So to award players for scanning, could be good if the scan they do do, are as similar to one they or someone else did.
Though it would be a really big undertaking for the RWP team to enable to have that as an option in Ingress and Pokémon GO.
The most basic way, I imagine, is to have a physical person reviewing submitted scans to ensure they match up with the object in the Portal photograph before awarding AP or Scout Controller designation. Most Portals have one or more photographs, which can be used for comparison.
The fact that scanning works on the Honor System of just assuming that the submission is valid (unlike Portal submissions, edits, photos, and removals) means that it is far easier to game. Having someone actually review the scan and accept/reject it before the player is credited would undoubtedly cut down on scanning abuse, but is probably incredibly labor-intensive.
Given that scans generally vary in length, and even legitimate submissions can accidentally include a few frames of the sidewalk, I always thought it would have made more sense to award credit based on the number of frames submitted in a single scan that get accepted. Have the first batches of Portal scans be verified by Niantic for accuracy, then have the algorithm reject future frames that deviate wildly from the others and give credit for frames that match previously accepted frames.
The fact that the current system is so easily gamed makes me wonder if Niantic figured that, even if most of the scans they receive are poor, it's still cheaper than being more selective about scan submissions.
Niantic changed the scanning guidelines shortly after they implemented scanning and the new medals (the amount of poorly made scans must have been huge) - light-sensitivity, length (5 seconds was the minimum at first)... And in the latest update it tells us if we're too slow (even though there's a bug there too, I could wave my phone frantically and it'd still say I was too slow). I realise they're still fine-tuning it to nip poorly-made scans in the bud, but I agree there has to be some sort of monitoring.
@BERN1ESANDERS you make a good point, all the other aspects in terms of portal quality is kept a close eye on collectively on Wayfarer. A portal scan is no different from a portal picture, other than that it's a new experimental feature. But scans award medals and photos don't, so accepting scans as valid from the get-go doesn't make much sense.
I agree with @InvestigateXM 's idea of sanctioning agents who upload too many fake scans. Of course there would have to be a solid base of previous scans to cross-check with, but @BERN1ESANDERS 's idea of an algorithm that could bunch together similar scans and single out outliers sounds promising (although I have no idea how this would be done and/or if there are enough resources on Niantic's part, unless there were some way to "Wayfarerize" scans).
What the sanctions could be, I have no idea. A soft-ban on scans? A three-strike policy like in portal nomination/edit abuse? Is it worth mentioning @NianticBrian ?
Good thoughts! For some background, my line of thinking is based off the demos we've seen so far provided by Niantic and 6D.ai, which they acquired in March of last year to accelerate their crowdsource AR scanning and modeling tech. It's some pretty cool stuff, as shown from a demo from 2018. I always figured part of the process of scanning was stitching together images, location, and accelerometer data to recognize the angles of an object and create a mesh of the object. There must be some way to compare a) the GPS location of the shot, b) the way the camera was angled when the shot was taken, and c) the image itself to tell where the frames fit on the overall mesh. If the location is far away, and the camera is facing straight down the entire time, and the image can't be matched to any existing images of the Portal, then there may be a good justification to reject the frames and not award any AP or medal credit. If the ratio of accepted to rejected gets too low, then perhaps they could take the Wayfarer approach and stop awarding badge credit until the player can consistently get scans approved.
What does everyone think? Should there be more penalties for poor quality scans? What would be a reasonable penalty for not uploading a valid scan?
I completely agree and made a report on this back in December, where NianticBrian said he was working on a fix.
In my case, the people "scanning" the portals are on the same level as spoofers. Unrealistic travel times and no proof of actually being at the portal (take a look at their scans for instance and you won't see the portals, just the inside of a car or train). This is especially evident when it's after sunset and it would be too dark to conduct scans ordinarily anyway.
I've also seen impossible numbers of scans on scanner profiles.
What is an impossible number of scans? I've got double Onyx Scout and Onyx Scout Controller, but that was done through a lot of hard work and commitment. There are a few others with both Onyx too.
Everything else you say is true, I assume people doing fake scans are just trying to get the numbers for the badges and/or the AP, but they don't appreciate the exploration aspect of scanning. It's sad really.
I am still secretly hoping NianticBrian is forced to watch every single scan for imposing Scout Controller on us (with a ridiculous comparison to Pioneer), but I know that will never happen lol
Should there be more penalties for poor quality scans?
Unless someone is using massive numbers of low-quality scans to maintain Scout Controller status on a portal, thus preventing others from gaining that status, this all feels to me like a solution in search of a problem. Someone else's low-quality scans have no real impact on me or the game. Given that, and the fact that there will always be people looking to see what they can get away with, I think I'd rather flip the question around:
What can Niantic do to incentivize high quality scans?
You make a very good point there - after all, they're faking something in-game to their benefit, although technically they do have to be on-site to even make a fake (spoofed?) scan. It IS cheating however, in my view. And making fake scans on purpose to "stick it to Nia" (in their minds)?
All they're doing is devaluing a medal that's hard work to get, therefore undermining all the legit work done by agents. I have to congratulate you on your onyx and double onyx @oscarc1, it's a lot of hard work and time invested. Very impressive! I can't imagine getting 12k scout controllers done, ever! In reference to your question, what I'm referring to with "impossible numbers of scans" was based on the info on the agent profile I saw, within context, and also having seen scanning abuse in their behaviour. By no means did I mean it as a stand-alone sign of abuse. Apologies if it sounded that way.
I like your thinking @BERN1ESANDERS - if we can wait for agreements and photos to be reviewed to get AP, I'm sure we can wait for scan points. What you propose would definitely support fair play! Would love to see that implemented, even if it means outsourcing the bulk of the review back to us, after a prior phone angle/frame cross-check filter.
In terms of sanctions, I'd go with what I said earlier - softbans and strikes, comm/wayfarer-style.
Good point-- there's generally minimal in-game impact of poor scans on other players, though I'd counter that the costs of storing and sifting through these poor scans probably has some impact on the development of quality AR models for future features. It's tough to quantify that, though; for all I know, Niantic really doesn't care either way. It's funny to imagine someone who's taken thousands of bad scans of their couch Portal one day logging in to discover that Ingress now has a high-quality model of their lap and desk.
What can Niantic do to incentivize high quality scans?
I'm of the belief that Niantic can't truly incentivize high-quality scans until there's a physical in-game measure of the gameplay contributions that players can make by submitting high-quality scans. Some players don't care about the quality of scans if their scans get thrown into what is essentially an information black hole. If Niantic has both a way to easily quantify improvements to the models, as well as a way to discern poor scans from beneficial ones, then they could better reflect that in the app and provide other incentives accordingly. I've pondered a hypothetical "Portal Sequencing" feature where Portals are given a broad tier reflecting the progress of their AR modelling with associated rewards. A Tier 0 Sequenced Portal has few or no scans, a Tier 1 Sequenced Portal is mostly mapped but has a rough and potentially incomplete mesh, and a Tier 2 Sequenced Portal has a high-quality and fleshed-out model that is likely to be among the first models included in future AR applications. Perhaps lower Tiers could provide more AP and Scout Controller points for (approved) scans, while higher Tiers would be eligible for AR inclusion or some other in-game rewards.
Just one idea of many, I'm sure. It's an interesting topic and I'm excited to see where it goes once Niantic's vision for AR becomes more mature.
I have scanned several portals for science, need to see some results (3d mesh, "dead drops" mentioned by Brian, or location improvements, etc.) to stay motivated and upload more.
However, I don't think it's "abuse" to scan the ground - at least, not until Niantic says so. Must be easy to filter out scans facing down. And also, bad scans are data, too. One of the big problems with scans - people don't want to play with camera out near playgrounds and busy locations. So IF Niantic plans to make a feature or new game with camera, they can get the "proper" and "ground" scans ratio, and see if/where people are ready for such gameplay.
When I had a glance at the title of this post I thought it's exposing somebody exploiting the scans to remove valid portals by submitting scans made somewhere else, which is imparing the gaming environment, because as far as I know this is the only place where Niantic would do take portal scans into considerations. Instead it seems that you are complaining about people who do not do scans seriously or fumble the portal scanning on purpose to get more portal scan counts in their profile because you are an earnest scan uploader. Yes it's not good but in fact it's a too trivial thing from my perspective.
I won't be surprised if Niantic does not care about that or on the same level as you expected. In this case I'd recommend you being not caring too much about that too, as they are not imparing the gaming environment directly. If you read my post then you would realize that Niantic even does not really care about delivering due punishment to some spoofers who did hurt not only our feelings but also impair the gaming environment to a great extent. Just like what many players would reply on any complaints: If you think this is a serious issue, please report it to Niantic Support. (and I assure you that they would not reply to you something else besides standard automatic replies which means you won't know whether they see it as a problem or not)
Niantic did not pay us to help them improve the quality of the Ingress game itself and the entire gaming environments. We paid users, as volunteers and vigilantes, proactively helped on that with passion. See what we got? Automatic standard replies. Closing of our posts. Removing of our comments. Strange players flocking to the forum to deconstruct and devalue your efforts and precious time spent on contribution to the game by emphasizing speech like "how can you say that they are fake, maybe they are just normal scans but with poor quality", "prove ab.use", "do not defame other players", "go to Niantic Support, not here", "If Niantic Support replied to you then you should not complain because they 100% took everything into consideration" and by performing Disagree attacks. None of them means we are treated with full respect. Why bothering then?
Request for help to reward honest, good players when bad, cheating players are not banned universally and fairly is also surreal. It's because the portal scannings currently have no actual impact on the game besides reviewing invalid portal reports and no rewards that cheaters do not get interested and participate into this "portal scans game". If players can get quite much bonus by uploading scans and if players become convinced about the fact that even if they upload bad portal scans or probably the fake ones as you mentioned they would still not get warnings or probations, then the server which store portal scans would be flooded instantly by really poor or even artificial scans. At that time you would feel extremely sad because the value of your work on portal scannings would be diluted to none, your badge or medal acquired by uploading good portal scans would be extremely easily acquired by some other ingenuine accounts in a short time and you would come to this forum again to seek help desperately. This is the current situation about spoofing and the reason why I did not vote for adding bonus rewards for (good) portal scans because it would just be exploited and won't be stopped even though you do care about that a lot and somebody tells you that you would get help. You are building a castle on the sand and ultimately you would know that I'm right.
As someone who used to scan a lot and enjoyed it for the longest time until it got insanely labour intensive, time consuming, and the iOS bug made it more of a chore than anything, I completely agree with the sentiment here.
I have done scans in 5 different cities in total, and while doing so, I have submitted couple hundreds of portal edits (for titles, descriptions, location, as well as flagged stuff that no longer exist for removal). Unfortunately, what was fun and part of exploration became one of the most annoying aspects of the game for me. Thanks to the iOS scanning bug, I was left with two options: either I focus on just scanning, or attempt to multitask (i.e. unique scan, unique capture, unique hack) and most likely lose the scan during upload due the game crashing repeatedly with the increased number of tasks. Since scanning was a priority then, I decided to go with task one to get one thing off my list but lost a significant number of unique captures opportunities.
As for the actual scans, for portals where I could no longer find the POI, I did “environmental scans” based on Brian’s suggestion. I also did “environmental scans” for portals that were tiny survey markers fixed on the ground. Admittedly, I tried scanning two bridge portals in total in a moving car to see what could be a resolution to an issue like that. Based on my test, I feel like one thing that can be done to avoid people scanning while moving in a vehicle/train is if the scan gives an error message saying the user is no longer in portal range and that they need to remain in the range to actually scan the portal. As for the quality assurance part, I honestly doubt it could be forced because scanning is an optional feature to begin with and I don’t know much about the backend stuff that goes into building an AR module to know if the ground/feet scan data provided by those abusing it could even be used for their modelling. Having said that, I completely agree with @Toxoplasmolly: Niantic needs to do more to incentivize higher quality scans because if there is one thing I learned is that the minimal AP and the ridiculous requirements for scout controller is counter productive for Niantic and their goals with AR at the end of the day.
Lastly, not that this matters anymore because we thankfully have unique markers now but this is an (initial) example of how I was tracking portals that I had already scanned (I can’t seem to find the later screenshot or the ones I had for the 4 other cities on my phone).
>Based on my test, I feel like one thing that can be done to avoid people scanning while moving in a vehicle/train is if the scan gives an error message saying the user is no longer in portal range and that they need to remain in the range to actually scan the portal.
That could also backfire for portals that are misplaced, aka where you have to activate the scan window and then walk to the portal, and for portals that are big, like building, where you have to take multiple steps back to get it in frame or where you have to walk to get all sides. What could be a compromise is a speed lock, if you move too fast, assume you're on a bicycle or in a car and stop the scan.
My question is, why do you consider such scans "fake" or "abusive" in the first place? Brian openly said somewhere that they appreciate such scans of surroundings of a portal too!🤔 And the floor is still a part of surroundings.
As for scans of the inside of a car or train... That's sad of course but what could you do to filter that out?
P.S. I personally don't scan right after the release of Scout Controller. I don't want to do labor, I'll rather play the game.
To answer my own question, I'll repeat a comment of mine from the early days of Portal Scanning, since it still applies today:
What would be neat(!!) is seeing our contribution to a portal’s 3-D point cloud visualization update in near real-time: Record video. <Magic.> View visualization before and after the recording, maybe with suggestions for which angles and views require more data, or with suggestions as to why the recording did or did not help significantly.
I understand what you say but bear in mind it's also trivial to the fake scan uploaders. They don't think it's cheating or has any impact, just as you do, so it's game. They simply don't care precisely because there are no consequences to their abuse. It's human condition to not learn or do things right until there's some "punishment" or negative consequence to their actions. That's the root of it. But for those who do take their time to scan, who do care about portal network quality, it's really disheartening - and I still believe there should be some way to support fair play. Maybe it's not a medal you particularly care about, but some players do - which makes it not so trivial. And after all, the post was to generate a discussion and see if there were any like-minded agents out there who have any ideas or proposals as to what to do about this, not to complain. Constructive proposals, that sort of thing. Keeping quiet about it is worse, I think. I wouldn't be so quick to dismiss something someone else cares about because it doesn't affect you.
And who knows if these fake scans will ultimately be used to check whether the portal "exists" or not... I'm sure there's been more than one malicious scan in case these are used as proof for portals to be taken off the network in the future.
@InvestigateXM yes, a speedlock! I would've thought that was basic - the game doesn't even let you recharge portals if you go faster than X km/h, doesn't it already affect scanning? I haven't tried doing it from a moving vehicle, so I have no idea.
@Naruwasher I see what you mean - I aso capture, visit, scan and take a photo to update. It's a lot to take for the game, not sure if it has to do with our phone RAM too (no clue, just off the top of my head). I've also reported duplicate portals (but to no avail). It would be interesting to see how AR technology will affect portals in the long run. apparently the iOs scanning bug will be fixed in the next update (fingers crossed, iOS user here too)!
Well a scan is obviously fake if the portal is right in front of them but they choose scan solely the ground in front of their feet. Or if the portal is displaced and nowhere in sight/out of range. Or if they're scanning a table lamp as their couch portal instead of the real thing. That sort of thing. But yes, I totally get your point, it's difficult to assess what a fake scan is if it's just the data with no other context whatsoever if agents or someone in Nia were to review the scans, let alone an algorithm. I guess there'd have to be a human filter then, they'd have to judge from the kind of portal scanned (is it something easy to scan? Or something that's harder like a huge building or a small plaque that doesn't allow for much scanning?) and derive the quality of the scan from there. But again, I'm no expert in tech or AR!
Well a scan is obviously fake if the portal is right in front of them but they choose scan solely the ground in front of their feet.
Not really. Look here:
Q: “Portal Scans?” More like "Environment Scans." No one is scanning only the Portal.
A: The surrounding area within 40m of a Portal can be useful mapping data to help place the Portal into the AR map. An “Environment Scan” may be a clearer name, and it means we can better anchor Portals in the right place and position in its environment, complementing lat/long information about the Portal. Agents scanning the Portal and also its environment or surrounding area are providing high-quality Portal Scans that improve the Portal Network.
So does this mean you think it's okay to scan only the pavement instead of the actual portal? I don't think @NianticBrian meant the pavement you're standing on as "environmental scans". It's a given, but it's certainly a loophole.
I agree with you. See what I predicted previously? "how can you say that they are fake" has come. Some players on this forum has a complete system of verbal tricks to make players, who would like to request for Niantic's help to make things correct, upset and exhausted.
The idea behind "Fake scan" is obvious and we all know it and we know that it's not OK as common sense. It's not because "nobody is scanning only the Portal" that one is encouraged or allowed as per the rules to scan everything EXCEPT the portal on purpose. After all, it's called "Portal Scanning".
There is no need to define or redefine "fake scan" for someone who generally does not do portal scanning at all. We got your idea. If portal scanning was done is a way that intentionally avoids or 100% misses the structure of portals then it's of course fake scans. The problem lies in how to evaluate the extent that some players' avoiding the portals, not extending the basic concept of "Portal Scanning" so that it becomes "It's OK to scan everything EXCEPT the portal because it's called Environment Scanning" with the help of reference or explanations somewhere else.
Fake scans are not OK, that's all and should lead to no questions.
What about adopting a system of statistical sampling of submissions leading to a defined quantity of checks by Niantic. Then have a a card system - yellow for a warning, then red if you submit nonsense. For red your scanning count goes back to zero.
Sounds like another job for machine learning.
This implies that they actually want to police it. Adding a level of "Don't submit scans" is irrelevant if they're able to identify that it's a bad scan. They can just discard that one and hope your next one is better.
It's not in their interests to stop people from scanning, if they're able to manage the quality, which would be required to do the policing.
**** scanning :)
Just my 2 cents here...
Why are we worried about fake scans? To date, it doesn't affect the game board at all. There is no improvement to the gameboard as a result of scanning.
We don't have any visibility of what is submitted, so why worry about it? Niantic is the one that wants the data, and they are the only ones that can decide whether a scan is good quality, bad quality or unusable altogether.
All you can do as a player is to possibly educate your community to submit quality valid scans. But you can't enforce it.
If Niantic decides on review that someone is submitting subpar scans, I'm sure they will decide on how to deal with it.
Why worry? Well, do you remember the uproar when Niantic introduced Second Sundays as a way to increment your Mission Day medal? "Oh noez, other people are devaluing my hard-earned MD medal by doing it on Second Sunday!" And this led to a new (unnecessary in my view) Second Sunday medal.
So this is basically "Oh noez, other people are devaluing my hard-earned scout medal by sitting in a pub and scanning the floor." (And no, I don't think an indoor floor scan even counts as "environment" - and yes, I have heard an agent say they would be doing this to get points for EOS Imprint.)
The amount of gatekeeping in what amounts to "Just play and you'll have the easy Onyx badges" when levelling, is absurd.
This is a bit like comparing apples to oranges, in that they are both fruit.
Scanning has always had its own badges, with its own set of criteria. Niantic never tried to co-opt scanning to further another vaguely-similar pre-existing badge.
While I understand the thought behind Second Sunday, using the MD badge for it was not right. Mission Days are well-established events with a clear purpose and set expectations. Co-opting that badge for a generic "do any old 6 missions once a month" was not the way to go, and yes, it did cheapen the badge.
"It cheapens the badge" is exactly what the OP is complaining about here.
But low-quality scans have been a "feature" since day one, over two years ago. There is nothing to cheapen because it was always cheap.