Reputation Activity Score

[Mick]

What up gamers,

Today we have a new, separate suggestion that I would like to bring up to the community.

Right now one of the main issues with reputation totals is that they are not reflective of a user’s actual reputability, so it has been a common suggestion for us to completely remove reputation totals. In my opinion, this is not an adequate solution, and instead, we should consider trying to change our current reputation totals into a number that takes many more factors into consideration. This number will be considered your ‘activity score’. It is worth noting that this number is never going to be able to be perfect and should not be used by itself. We heavily recommend users perform their own due diligence and always look through a person’s posts, reputation, and other parts of their profile to get a better idea of who you are dealing with.

What I would like to propose today is an idea that Kram and I have been considering and preparing for a while now, and it is for us to remove the current reputation totals to replace it with a new number called ‘Activity Score’. The activity score of a user is calculated using an algorithm that takes into account many aspects of their contribution to the site, both positive and negative. With this score, we would be keeping the exact method of calculation completely unknown, including most of the specific metrics and how they are weighted. Some metrics include feedback, posts, account age, as well as negative factors such as warning points. This system does consider banned and legacy reputation, but it simply contributes a relatively lower amount compared to feedback with proof. We will still add the ability to add proof to your legacy feedback to make it worth more and we highly encourage you to do so.

Although we will not be sharing specifics about how the system calculates scores, below are some scores to show how this change would influence a variety of users on our platform. Please note that these scores are not finalised and are only approximate for now until we can gauge community interest to see if this is an idea worth developing further.

Spoiler: User scores

Zyger: 298
Doge: 297
LonelyTree: 205
Asa: 202
Glove: 183
Chearful: 173
ThaMango: 165
Ivain: 156
Ghast: 129
Choo:129
User: 109
null: 90
Dann: 77
JayMC: 61
Rika: 41
2pac:-2

Along with this, we would still continue to grey out legacy and banned reputation, no longer accept reports on this feedback, add a notice about legacy reputation, and send out messages to all users with feedback they need to upload to migrate their legacy feedback.

This idea is pretty different to anything else we have considered in the past and I am unsure of what you all will think of this, so please provide your input below and I look forward to considering it all .

Thanks everyone,

 

[Kram]

Honestly, I think this is a good route to take, since all these factors do play an important role in reputation. It really helps balance things out and makes it harder to abuse. However, I think it can be polished up a bit, though I know it still is being fully thought-out still. I also hope we are updated on the progress and are told about how it works more, without revealing any private requirements. But my conclusion on this is that I support and agree with the implementation of such a system.



1. I'm not sure how this could be done, but there needs to be a better way to factor in posts. I believe posts should not have a big impact on the score, and I say this as someone with 10k+ posts. Some people post-farm a ton and this shouldn't result in such a big boost to their score. And the person in question posts crap on every new thread, most of the time things which barely contribute to the discussion and just look like they were made for the sake of posting. Potential ideas to combat this: Cap it off at a certain amount of posts (perhaps cap it at a certain amounts of post per year), use a formula based on quantity of posts + reaction : post ratio (a formula so that someone with 1 post but 100 reactions isn't turbo-boosted), perhaps length of the post - these are just some quick ideas, I'm sure there is a good solution for this.

2. Account age - This is a good factor, BUT it can be greatly devalued by people who steal MCM logins of old accounts and use those accounts to make themselves seem more reputable. When I was a mod I had discussed a way to combat this problem because back then it was VERY prevalent, I had to use the ACP every other day to discover more and more people using stolen MCM accounts and by the looks of it, people still do this. The solution I had proposed was to make it so that users which haven't logged in in over 6 months have their accounts "soft-locked" where they get a message upon login saying to check their email for a link to continue using their account. This should greatly help, as though many people reuse passwords, much fewer tend to reuse them on their emails.

We can't say too much on how it's being calculated, but, posts are calculated in a way where post farming doesn't really help you in the grand scheme of things. We took some users from notable members in the top messages section to make sure the score they get is somewhat representative of them to make sure it doesn't open it to heavy abuse. The way it's calculated right now I'm pretty happy with personally and it shouldn't be able to be abused to a strong extent, and if we do see that happening, we will tweak that factor more based on the activity we see.
Same with account age, there's a way we are considering that to also prevent it from being abused in that manner.

 

[sylin]

You should use your head too.

Why release a potentially exploitable, vague system? What if someone figures out the factors through slow long-term debugging and decides to abuse them and not share them with anyone?

There can only be so many factors anyway, and most of them are probably already obvious (get an avatar picture, login daily, post a bunch of threads and suggestions, profile comments, get good reactions, upgrade your account, whatever, etc.). What the fuck else could be a factor. There's barely anything to do on the forums.

I have looked a lot of your posts and you seem like a very well informed and well written person, but I think that some of the replies are being overly critical of this system.

This system is not being claimed to be perfect, rather it is being implemented as a step-up from the reputation "number" which is inconclusive and easily exploitable (I would argue more exploitable: someone with 30 $2 looks 30x more reputable than someone who had to one transaction totaling $60 (that $60 transaction shows the user is arguably more reputable since it is easier to scam with single large transactions)).

Also I don't understand your point saying there is nothing to measure, there are a multitude of possible factors: Join date (someone who just joined may have less attachment to the site/possibility of ban evading), Post number (activity/commitment to site means they are less likely to scam), reaction number (though I don't think it should/would be weighed heavily this could be a means of showing you aren't posting useless bullshit), recent activity/daily login (what you were doing in 2016 may not be as reflective of what you are doing now), warning points (a gauge of how you well follow/respect rules, though perhaps it may not need to be a heavy factor), shoutbox engagement (shows commitment to site, though I don't think this should/would be a heavy factor), upgraded account/amount contributed to site (shows you are committed to site and less likely to abandon what you've spent on it) number of positive feedback/rep, and transaction totals. These are all possible factors and perhaps there are more. I think it would be much more difficult to try and exploit a system with multiple facets than one with one.

Again, this system is only trying to improve the inconclusive, easily exploitable reputation number. If someone is going to transact with someone it should be clear that you would review their reputation and do your own research, this number is not being implemented as a replacement to that.

I do agree that the system is pretty vague; however, that defeats the argument that it is easily exploitable does it not?
If/when the system is implemented users will have a general sense of the number and I doubt that anyone trying to 'debug' to exploit the system will not just as easily if not more easily be able to do the same with the reputation number since there's no need to debug anything to know how to exploit it.

 

[alice]

it is being implemented as a step-up from the reputation "number" which is inconclusive and easily exploitable

I would argue that "Activity Score" is far more inconclusive and far more exploitable.

Whereas in the current system the only way to gather reputation is through successful market deals, you could just raise your activity score by doing a variety of things like post farming, upgrading your account, being a longtime member, chatting in the shoutbox, baiting for agree reactions, etc. Most active users already do all of the above without ever touching the market or trading anything. Scammers especially would go through that effort to build up a trusted profile since there would be no monetary investment involved to build up their credibility to those who would mainly solely rely on an emphasized Activity Score metric.

(I would argue more exploitable: someone with 30 $2 looks 30x more reputable than someone who had to one transaction totaling $60 (that $60 transaction shows the user is arguably more reputable since it is easier to scam with single large transactions)).

This is debatable.

I'd trust someone that has 10 successful $50 transactions more than the person that has 2 successful $250 transactions for multiple reasons. The key factor here is to just research the person you're dealing with and figure out for yourself whether you think they're credible enough or not, as you've said, which I'm sure most already do because no one wants to lose money.

Also I don't understand your point saying there is nothing to measure

I never said this. I also mentioned a bunch of things that could contribute toward someone's activity score. Most of the same things you said, actually.

However, keep in mind, all of the mentioned things ONLY measure someone's forum activity. Not necessarily their credibility as a trader. After all, it is called "Activity Score" for a reason.

The problem here is that you are trying to replace the standard reputation system, a system that everyone is familiar with, including both new users and even offsite users with a system that only active McMarket users would be familiar with. Many online marketplace forums all use the same reputation system plugins, and they all call it the same term -- reputation. Not "Forum Score" or whatever which is obviously vague and sounds like it could be obtained easily.

Am I for seeing more metrics like Activity Score on users' profiles? Sure. Why not? It would be cool. It would be even cooler if it was dynamic and was a number that could go up and down depending on how active the user is, although while mostly staying unrelated to and only slightly affected by reputation.

Am I for completely replacing the reputation system with an activity score system? Absolutely not. As I said, many new users and offsite users would be unfamiliar with such a system. You don't want to deviate too far from the standard of what a typical online marketplace forums does. And you definitely don't want to measure someone's reputation via metrics outside of successful trade transactions.

Rather than replacing the reputation system, it should simply complement it. "Activity Score" might sound better if it was renamed to "Trust Level" (a lot better, IMO) but it's still vague. I'd rather look at someone's join date and reputation myself than rely on some vague system to do it for me. Of course, you will argue that I could still do that on the new system, but I'm arguing that we shouldn't place as much emphasis on something like activity score and make it seem like it's a solid measure of a user's trust level.

Again, this system is only trying to improve the inconclusive, easily exploitable reputation number.

Activity Score would be even more exploitable.

I do agree that the system is pretty vague; however, that defeats the argument that it is easily exploitable does it not?

No, not at all. As I said, there are only several things that you could realistically measure that would contribute to someone's activity score, and we've both probably already mentioned the core ones on a whim. It wouldn't take that long at all for an active member to figure out how to boost their activity score by trying out random things over the weeks and just doing normal member things like posting and gathering positive reactions.

We need to stop focusing so much on endlessly revamping the damned reputation system over and over and focus on other things and things to complement it like Trust Scans, forum badges, more overall forum security, revamping postbits and giving reputation more merit and attention, other metrics, removing scammer breeding grounds such as the loans & investment subforums, etc.

• A Trust Scan could show MANY things, such as whether a user is on a proxy or on a VPN (less trustworthy, potential sign of a hijacked account), has a Last Login location or IP address range from the creation IP address, whether the user has multiple accounts, whether the user has an open scam report, and so on.

• What do I mean by forum badges? Aside from aesthetics, many forum badges can also be informative. Some forums give users a 2FA badge for having 2FA enabled on their accounts. These are always the signs of a more serious trader. Maybe a phone-verified badge or ID verified badge or whatever, depending on how serious you want to get.

• Also, for some reason, and I did report it (got denied of course), but threads like this are allowed to exist https://www.mc-market.org/threads/520710/ where users (mostly security unaware children, obviously, who else would want 50 cents(?)) just post their PayPal e-mail. Many of these users are unaware of database leak websites. You could literally hijack half of the damn McM accounts in this thread with little to no effort and use them to scam out. From what I know, McMarket has no login verification so you could just login someone else's account with a completely different IP address without having to verify it via e-mail. It's actually ridiculous, but McM staff is just as oblivious when it comes to security so of course they won't encourage users to practice proper security habits by trashing threads like this.

Going to end it here. I could write so much more but I don't want to make it too long.

 

[sylin]

Am I for completely replacing the reputation system with an activity score system? Absolutely not. As I said, many new users and offsite users would be unfamiliar with such a system. You don't want to deviate too far from the standard of what a typical online marketplace forums does. And you definitely don't want to measure someone's reputation via metrics outside of successful trade transactions.

This score is replacing the reputation total on users' postbits, not uprooting the reputation system itself. I am only talking about the number on the postbit.

Rather than replacing the reputation system, it should simply complement it. "Activity Score" might sound better if it was renamed to "Trust Level"

Yes I was trying to think of a few other names (I couldn't get very far, had some ideas like 'Prestige' or 'Credit Score' however the first sounds corny and the second sounds like loaning and neither incorporate all the facets of this system) I think "Trust Level" is better than 'Activity Score' for sure.

This gets at what I think is most worrisome about the system, what the term "Activity Score" suggests that I think you're trying to get at--is if this score is done wrong--where it doesn't weigh monetary transactions enough above post number, reactions, etc. (less direct factors to trust in the marketplace since they take "no monetary investment").
If done right I think this would be a definite improvement over the old system for this reason. It would weigh important variables including the transaction amounts (I hope), and whether the reputation is legacy or not (i.e. old/provides proof), that are left out with only showing the reputation count. This would make it not only harder to exploit, but also much more accurate of their 'trust'.

Though you may not agree with me I think post number and other factors should exist (as they do have relevance in showing commitment to the site, tendency to following rules, etc.); however, they should not have monumental impact and should have a limit as Doge suggested.

Of course, you will argue that I could still do that on the new system, but I'm arguing that we shouldn't place as much emphasis on something like activity score and make it seem like it's a solid measure of a user's trust level.

Understandable, I just think that the reputation count is not a viable indicator especially with the recent removal of legacy or banned users' feedback in that score I think it is just not well rounded.

I may not have quoted all of your response but I believe I responded to most of it.

• A Trust Scan could show MANY things, such as whether a user is on a proxy or on a VPN (less trustworthy, potential sign of a hijacked account), has a Last Login location or IP address range from the creation IP address, whether the user has multiple accounts, whether the user has an open scam report, and so on.

• What do I mean by forum badges? Aside from aesthetics, many forum badges can also be informative. Some forums give users a 2FA badge for having 2FA enabled on their accounts. These are always the signs of a more serious trader. Maybe a phone-verified badge or ID verified badge or whatever, depending on how serious you want to get.

• Also, for some reason, and I did report it (got denied of course), but threads like this are allowed to exist https://www.mc-market.org/threads/520710/ where users (mostly security unaware children, obviously, who else would want 50 cents(?)) just post their PayPal e-mail. Many of these users are unaware of database leak websites. You could literally hijack half of the damn McM accounts in this thread with little to no effort and use them to scam out. From what I know, McMarket has no login verification so you could just login someone else's account with a completely different IP address without having to verify it via e-mail. It's actually ridiculous, but McM staff is just as oblivious when it comes to security so of course they won't encourage users to practice proper security habits by trashing threads like this.

I totally agree with this, that post was very sketchy and a 2FA badge would be useful.

 

[casket]

Use your head... What's the best thing that could happen if the factors were released... You're happy? Well the worst thing is that people would exploit it nearly immediately. You don't need to know the ins and outs of it because this suggestion comes down to the concept itself of it.

Nice out of context quote thank you very much, Much cool

And not taking it out of my whole spew and took it out of the TL;DR for maddy

 

[Maddy]

We can't say too much on how it's being calculated, but, posts are calculated in a way where post farming doesn't really help you in the grand scheme of things. We took some users from notable members in the top messages section to make sure the score they get is somewhat representative of them to make sure it doesn't open it to heavy abuse. The way it's calculated right now I'm pretty happy with personally and it shouldn't be able to be abused to a strong extent, and if we do see that happening, we will tweak that factor more based on the activity we see.
Same with account age, there's a way we are considering that to also prevent it from being abused in that manner.

top shoutbox users!11!

 

[Lotus]

2. Account age - This is a good factor, BUT it can be greatly devalued by people who steal MCM logins of old accounts and use those accounts to make themselves seem more reputable. When I was a mod I had discussed a way to combat this problem because back then it was VERY prevalent, I had to use the ACP every other day to discover more and more people using stolen MCM accounts and by the looks of it, people still do this. The solution I had proposed was to make it so that users which haven't logged in in over 6 months have their accounts "soft-locked" where they get a message upon login saying to check their email for a link to continue using their account. This should greatly help, as though many people reuse passwords, much fewer tend to reuse them on their emails.

I agree that account jacking is a problem since credential-stuffing is a huge issue, however, I believe this suggestion and that problem are two separate issues.

Assuming it’s a perfect world where accounts aren’t being stolen, I think account age is a great factor.

As for the account security issue, Mick/Jayson need to genuinely prioritise a mitigation implementation for that, as well as figure out solutions. I banned hundreds of 2015/2016/2017 stolen accounts when I moderated and I honestly am unsure who on the team is still on the lookout for them (and I doubt the newer staff are aware of such issues and how to combat them), because the site does have an active person/group that is compromising accounts. Doge says it was prevalent back then, but it still is prevalent now. Me and Jayson did try out some minor solutions, but we never implemented anything big.

Many companies deal with this and they take a wide array of different approaches, i.e. if there is a different IP login all of a sudden after 6 months, they’ll lock users out with a verification.
But there can be other more indirect approaches, such as offering 2FA setup on the registration page, which would encourage more new users to be aware of such a feature and therefore be more likely to activate it.

The team should spend a night discussing stop gap solutions for security. It’s not a low priority task, but a necessity.

Off topic, but where else am I supposed to say this.

 

[MarkElf]

I really like this suggestion. I hope resources play some role in the score, ideally though reviews. I haven't looked too deeply Into this thread but will your scored diminish if you don't do a deal after a period of time?

 

[alice]

I agree that account jacking is a problem since credential-stuffing is a huge issue, however, I believe this suggestion and that problem are two separate issues.

Assuming it’s a perfect world where accounts aren’t being stolen, I think account age is a great factor.

As for the account security issue, Mick/Jayson need to genuinely prioritise a mitigation implementation for that, as well as figure out solutions. I banned hundreds of 2015/2016/2017 stolen accounts when I moderated and I honestly am unsure who on the team is still on the lookout for them (and I doubt the newer staff are aware of such issues and how to combat them), because the site does have an active person/group that is compromising accounts. Doge says it was prevalent back then, but it still is prevalent now. Me and Jayson did try out some minor solutions, but we never implemented anything big.

Many companies deal with this and they take a wide array of different approaches, i.e. if there is a different IP login all of a sudden after 6 months, they’ll lock users out with a verification.
But there can be other more indirect approaches, such as offering 2FA setup on the registration page, which would encourage more new users to be aware of such a feature and therefore be more likely to activate it.

The team should spend a night discussing stop gap solutions for security. It’s not a low priority task, but a necessity.

Off topic, but where else am I supposed to say this.

Just require login verification for different IP address login. Should stop 99.9% accounts from getting stolen. Very easy solution.

And get rid of fucking threads like https://www.mc-market.org/threads/520710/

You shouldn't be allowed to post personal information such as your PP e-mail. I reported it and it got denied for some reason.

 

[Poket]

Youtube uses algorithms why don't you leave mcm Mick, just go to Youtube great algaritm retard

 

[Lotus]

Just require login verification for different IP address login. Should stop 99.9% accounts from getting stolen. Very easy solution.

You need to have some sort of time buffer where the user hasn’t logged in for a long time before reverifying when IP changes occur. If you have a dynamic IP at home, or use mobile data regularly to access the site, or travel etc. It can be infuriating for people having to reverify all the time.

It’s a balance between convenience and security.

 

[puzeh]

I'm not entirely sure about this suggestion or maybe I just don't understand the concept too well. I've joined in 2016, have done 70+ deals, never had a scam report open on me, or gained any negative reputation. But since I replied to a friend's thread with a joking "vouch" and got a warning for it, would that factor into my score? Personally, I don't look at users' posts as a method of determining if it's safe to deal with them. All I care about (and what I think most do) is feedback. I get that people can do feedback for others for being helpful, providing info, or other services other than selling a product, but for people who exclusively gain rep by doing deals it's more important to me. I'd maybe be more on the fence or maybe I could understand it better if the method of calculating this score wasn't so secretive. I get that it has to be hidden to prevent boosting, but if people catch on to something unimportant that boosts this score it could be worse than "rep4rep" has ever been.

 

[Kram]

I'm not entirely sure about this suggestion or maybe I just don't understand the concept too well. I've joined in 2016, have done 70+ deals, never had a scam report open on me, or gained any negative reputation. But since I replied to a friend's thread with a joking "vouch" and got a warning for it, would that factor into my score? Personally, I don't look at users' posts as a method of determining if it's safe to deal with them. All I care about (and what I think most do) is feedback. I get that people can do feedback for others for being helpful, providing info, or other services other than selling a product, but for people who exclusively gain rep by doing deals it's more important to me. I'd maybe be more on the fence or maybe I could understand it better if the method of calculating this score wasn't so secretive. I get that it has to be hidden to prevent boosting, but if people catch on to something unimportant that boosts this score it could be worse than "rep4rep" has ever been.

In your specific case, it wouldn’t as your warning has expired. If it was still active, it would count but it’s a very small rule and how it would effect you would be representative of that. Rep 4 rep is hard to fix because it’s hard to know when it’s happening. This score, it’s easier to fix the abuse because we just need to tweak the factor that’s being abused.

 

[Poket]

In your specific case, it wouldn’t as your warning has expired. If it was still active, it would count but it’s a very small rule and how it would effect you would be representative of that. Rep 4 rep is hard to fix because it’s hard to know when it’s happening. This score, it’s easier to fix the abuse because we just need to tweak the factor that’s being abused.

Why would rep be counted with warnings, I don’t care if the buyer or seller does dumb shit lol?? The only thing that rep should include is monetary deals, because that is the only thing that actually matters. Rep isn’t how nice and active you are, that doesn’t make more likely to buy from you in compared to the dude with multiple deals with real money.

 

[Jack]

Sounds like a great idea. By removing user interaction from the system (i.e. leaving feedback or reporting feedback) it solves many the issues of the reputation system. I doubt it will be perfect though - no system would be perfect - I imagine there will be a few instances of equally reputable users with drastically different scores, but at least the algorithm will prevent people from trying to game the system.

Although, you should probably come up with a better name ("User Score" comes to mind?). "Activity" makes it sound like site activity. Surely someone will X amount of posts that's been registered for 1 year should be just as trustworthy (if not more) than someone with 3X posts only having been registered for 1 month (the latter user would be "more active" by definition, but I don't think people would generally think of them as being more reputable).

 

[Ally]

Sounds like a great idea. By removing user interaction from the system (i.e. leaving feedback or reporting feedback) it solves many the issues of the reputation system. I doubt it will be perfect though - no system would be perfect - I imagine there will be a few instances of equally reputable users with drastically different scores, but at least the algorithm will prevent people from trying to game the system.

Although, you should probably come up with a better name ("User Score" comes to mind?). "Activity" makes it sound like site activity. Surely someone will X amount of posts that's been registered for 1 year should be just as trustworthy (if not more) than someone with 3X posts only having been registered for 1 month (the latter user would be "more active" by definition, but I don't think people would generally think of them as being more reputable).

How about... TrustScore™ .

 

[User]

Honestly, I feel like this would be a good site addition, but kept separate from reputation. There are tons and tons of users with very few posts/reactions (or high warning points) that are considered highly reputable. It is good for determining how much they've contributed to the site, yes, but not necessarily how trustworthy they are in terms of their deal-based reputation.

 

[alice]

Honestly, I feel like this would be a good site addition, but kept separate from reputation. There are tons and tons of users with very few posts/reactions (or high warning points) that are considered highly reputable. It is good for determining how much they've contributed to the site, yes, but not necessarily how trustworthy they are in terms of their deal-based reputation.

Pretty much. The most serious traders won't ever post or use their account for anything personal.