Non-Agreement Postmortem?
I would love it if there was a way to see how nominations were voted when we get it wrong. Even if it were a sampling (1 in 5 or 10), I think that reviewers could learn a lot and vote more accurately according to the guidelines.
My agreement rate has been slipping in the last few weeks. I assume some of it is lag, but as I'm below 70% agreement, I'd like to know what I'm missing.
1
Comments
% of Agreements compared to total reviewed is not a good determining factor.
I live in an area with a low turnover rate. So example at one point I had only had 12,000 Agreements to 25,000 reviews which looks terrible but after a couple months of sitting that went up to around 20,000/25,000.
If they just showed a simple approved or rejected and maybe the highest reject reason on past reviews that might help. Too much detail would allow players to figure out the system analytics and more accurately game the system which is a very bad thing.
You get 80% agreements?
When I got access, I reviewed a lot in a short amount of time. I basically reviewed more and got more agreements in a month than agents around me who've been playing for years.
It's super slow. I think it was around 25% agreements but I've never fallen from green. After getting disillusioned with the process, I have only reviewed about 200 in 2 months and slowly but surely it is ticking up. Up to 44% now, but you have to wonder how many still need a verdict and how many you got "wrong".
Absolutely I wonder how many total I’ve gotten wrong. There is no way to know.
The agreements vs total reviews is an irrelevant number unless you basically stop reviewing forever. You don't get agreements until enough votes have been placed to approve/reject a portal. Portals you've reviewed could sit in queue for months before that happens. Over time. It's easy to have a high agreement rate but still have thousands more reviewed than agreed upon.