Marketplace 'Review Trolls?'
Permalink
Wondering if there is any recourse to what I'm going to call 'Review Trolls' for lack of a better term?
I recently received a review for an addon of mine with a low rating. Of course I realize that not everyone is going to like my particular addon and I am fine with that, but I would expect some kind of helpful/descriptive feedback accompanying a low rating and/or some kind of communications with the developer leading up to said low rating.
This particular review only states 'nothing more to add...' with a subject line of 'Disappointed'. I received no 'Pre-Sale Questions', there is nothing in the 'Questions & Discussion' section, no 'Support' requests were submitted and not a single PM was received from the reviewer in question. I even contacted the reviewer via PM to try get some feedback as justification for his review so that I could try to address his concerns, but have gotten no response.
And so again, I am wondering if there is any recourse for valueless reviews submitted by 'Review Trolls?'
Thanks.
I recently received a review for an addon of mine with a low rating. Of course I realize that not everyone is going to like my particular addon and I am fine with that, but I would expect some kind of helpful/descriptive feedback accompanying a low rating and/or some kind of communications with the developer leading up to said low rating.
This particular review only states 'nothing more to add...' with a subject line of 'Disappointed'. I received no 'Pre-Sale Questions', there is nothing in the 'Questions & Discussion' section, no 'Support' requests were submitted and not a single PM was received from the reviewer in question. I even contacted the reviewer via PM to try get some feedback as justification for his review so that I could try to address his concerns, but have gotten no response.
And so again, I am wondering if there is any recourse for valueless reviews submitted by 'Review Trolls?'
Thanks.
its kinda hard to tell who is trolling ya know, try contacting the user directly i guess,
He did say he had tried that.
I agree that this is a bit of problem. With the new site design you can now add your own reply to reviews at least. I'd suggest doing that.
I agree that this is a bit of problem. With the new site design you can now add your own reply to reviews at least. I'd suggest doing that.
Ya I noticed that and am going to, but that doesn't help with the aggregated rating for an addon that only has 3 reviews :).
Ideally it would eventually get buried, but I'm not expecting masses of reviews to help in that department ....
Ideally it would eventually get buried, but I'm not expecting masses of reviews to help in that department ....
Only people who "purchase" an add-on/theme are allowed to add a review, so I'm not sure there is/should be much more to be done about this. If you look at how reviews are done on other sites, the c5 marketplace is no worse and in some cases better (e.g. on Amazon you don't even need to have purchased something to write a "review").
Fair enough, just thought I would check before commenting on this particular review.
Thanks for the responses!
Thanks for the responses!
While it would be nice for the user to ask for support or a question before leaving a bad review, and it would be even more nice for them to leave a reason why, we can't start removing bad reviews just because they're bad. Maybe if the person is really found out to be a troll/spammer then I suppose that would be acceptable.
The way I personally look at reviews when deciding whether to purchase something is if there is substance behind them (positive or negative). It's really easy to fake a review, but it's harder to put your thoughts behind it if you're faking it.
The way I personally look at reviews when deciding whether to purchase something is if there is substance behind them (positive or negative). It's really easy to fake a review, but it's harder to put your thoughts behind it if you're faking it.
yeah as you might imagine this comes up around the office a lot. It kills us that ecommerce has three stars because someone has decided its a security hole that you can't delete orders, or is comparing it to magento when we clearly say its not for that scale of problem.
We've discussed adding a little weight to recent reviews. So if your add-on starts out rough and gets better over time, that might help.. of course, that's not always going to improve things... We've also thought about adding "was this helpful" to reviews and using that.. perhaps having a poster rating as well and factoring that in.. (so if you've only ever posted one review and its negative, or all your reviews are negative, that carries less weight than a more balanced reviewer..)
push comes to shove however, this is all a lot of annoying work that creates a system people can call inaccurate.
there is now a message you can add to the thank you order for your add-on. It will show up in the email, and again after 7 days in a follow up email that is part of a push we're adding now.
I suggest you all make special offers for your add-ons at 10% off and then tell people in that message : "Hey give us a review and we'll give you a URL for 10% off any add-on of mine" or something like that..
That way people who are happy with your stuff are motivated to say so, because its my belief that reviews as a whole tend to attract negativity.
We've discussed adding a little weight to recent reviews. So if your add-on starts out rough and gets better over time, that might help.. of course, that's not always going to improve things... We've also thought about adding "was this helpful" to reviews and using that.. perhaps having a poster rating as well and factoring that in.. (so if you've only ever posted one review and its negative, or all your reviews are negative, that carries less weight than a more balanced reviewer..)
push comes to shove however, this is all a lot of annoying work that creates a system people can call inaccurate.
there is now a message you can add to the thank you order for your add-on. It will show up in the email, and again after 7 days in a follow up email that is part of a push we're adding now.
I suggest you all make special offers for your add-ons at 10% off and then tell people in that message : "Hey give us a review and we'll give you a URL for 10% off any add-on of mine" or something like that..
That way people who are happy with your stuff are motivated to say so, because its my belief that reviews as a whole tend to attract negativity.
I have a suggestion...
How about displaying in graphical format (pie or stacked bar chart) the number of ratings RELATIVE TO the total number of sales for that particular add-on. This would provide a bit more context. I mean, if 100 copies of an add-on have sold, and there's a single 2-star rating by a disgruntled customer, that says something very different to a prospective buyer than just seeing the single 2-star rating alone. You wouldn't need (or want) to indicate the actual quantities, but rather just show the proportions graphically. This could be done easily with Google chart services (which I think the survey block already uses).
Additionally, the dev should have the option of turning this graphical display on or off per add-on when editing that add-on.
Just a thought,
-Steve
How about displaying in graphical format (pie or stacked bar chart) the number of ratings RELATIVE TO the total number of sales for that particular add-on. This would provide a bit more context. I mean, if 100 copies of an add-on have sold, and there's a single 2-star rating by a disgruntled customer, that says something very different to a prospective buyer than just seeing the single 2-star rating alone. You wouldn't need (or want) to indicate the actual quantities, but rather just show the proportions graphically. This could be done easily with Google chart services (which I think the survey block already uses).
Additionally, the dev should have the option of turning this graphical display on or off per add-on when editing that add-on.
Just a thought,
-Steve
i like the essence of this idea, i wonder how it could be made to effectively work.
You dont want to really create relative ranking, there's very little in common with something as complex as ecommerce and as simple as superfish.... but yeah i see what you're saying theres a truth to that approach i find appealing if it works.
Best wishes,
(Pecked out on a mobile device...)
http://about.me/frz
You dont want to really create relative ranking, there's very little in common with something as complex as ecommerce and as simple as superfish.... but yeah i see what you're saying theres a truth to that approach i find appealing if it works.
Best wishes,
(Pecked out on a mobile device...)
http://about.me/frz
> You dont want to really create relative ranking...
Just to clarify, I'm in no way suggesting that add-ons be ranked relative to one another. You can't compare apples and oranges. All I'm suggesting is that prospective purchasers be allowed to "visualize" the relative number of purchasers of that particular add-on who left feedback. For an example, see the following...
http://chart.apis.google.com/chart?chs=300x225&cht=p&chco=6...
I suspect that the overwhelming majority of add-on purchasers who are satisfied with their purchase never bother to leave feedback. What I'm suggesting might make that a bit more obvious to prospective buyers.
But again, the dev should be able to turn this on or off at will.
-Steve
Just to clarify, I'm in no way suggesting that add-ons be ranked relative to one another. You can't compare apples and oranges. All I'm suggesting is that prospective purchasers be allowed to "visualize" the relative number of purchasers of that particular add-on who left feedback. For an example, see the following...
http://chart.apis.google.com/chart?chs=300x225&cht=p&chco=6...
I suspect that the overwhelming majority of add-on purchasers who are satisfied with their purchase never bother to leave feedback. What I'm suggesting might make that a bit more obvious to prospective buyers.
But again, the dev should be able to turn this on or off at will.
-Steve
This is an interesting idea. I wasn't sure about it at first, but upon seeing the pie chart it makes sense. An alternative to the pie chart that might be simpler for people to grasp would be to have a graphic (for example a single star) that was scaled according the the reviews. An add-on that had no reviews would have a medium size. Four and five star reviews would make it bigger, one and two star reviews smaller. The amount it was scaled would be proportional to the star ratings, their counts, and the over number of purchases. Another option would be to give all products a full size star to start and then scale them smaller with more negative reviews (quantity and quality), and perhaps time.
very interesting. I wish that this could be communicated at the
marketplace list view in some meaningful way however...
best wishes
Franz Maruna
CEO - concrete5.org
http://about.me/frz
marketplace list view in some meaningful way however...
best wishes
Franz Maruna
CEO - concrete5.org
http://about.me/frz
I think we're missing the forest for the trees here: the problem is that there are simply too few reviews for any given addon for it to make any statistical sense.
So I think the problem that needs to be addressed isn't how to best display the 2 or 3 reviews, but rather how to increase the number of people reviewing. This is admittedly a much more difficult task.
One idea off the top of my head is build in a rating system on the "Add Functionality" page of the dashboard -- unrated packages from the marketplace could have the star-rating control right there in the list of installed packages. Maybe a little message at the top urging them to give a rating? Nothing too obtrusive (and certainly nothing in the front-end editing interface).
-Jordan
So I think the problem that needs to be addressed isn't how to best display the 2 or 3 reviews, but rather how to increase the number of people reviewing. This is admittedly a much more difficult task.
One idea off the top of my head is build in a rating system on the "Add Functionality" page of the dashboard -- unrated packages from the marketplace could have the star-rating control right there in the list of installed packages. Maybe a little message at the top urging them to give a rating? Nothing too obtrusive (and certainly nothing in the front-end editing interface).
-Jordan
I think one of the reasons there aren't many reviews is you have to write a review- if it was just stars I'd bet a lot more people would use them, especially if they were linked in the dashboard page as jordan suggests. (like itunes has write a review and then rating.)
good idea, we will work reviews into the next version of the core from
your Dashboard.
best wishes
Franz Maruna
CEO - concrete5.org
http://about.me/frz
your Dashboard.
best wishes
Franz Maruna
CEO - concrete5.org
http://about.me/frz
> the problem is that there are simply too few reviews for any given addon for
> it to make any statistical sense.
Yes, good point, Jordan.
> One idea off the top of my head is build in a rating system on the "Add Functionality"
> page of the dashboard -- unrated packages from the marketplace could have
> the star-rating control right there in the list of installed packages.
That's not a bad idea, but I have to say that I'm not convinced of the value of the 5 star ratings. I'd like to suggest a much simpler approach - either favorable or unfavorable. IOW, the options would be thumbs up or down, and if thumbs down, they'd have to provide at least SOME supportive commentary - i.e. couldn't submit a negative rating without a comment.
The bottom line is that MOST people simply don't take the time to leave any kind of feedback at all, and in my experience, that generally means they are happy with the purchase. As soon as you "nuance" the rating with various levels, that almost demands that the user explain their rating, and very few want to take the time to explain their rationale. Simply clicking one of two icons - favorable or unfavorable - simplifies the feedback process. MOST people will click a "thumbs up" icon if they are satisfied with the product and be done with it. I, as a developer, know what I want to know - that my product met or exceeded their expectations, and it took the customer only a fraction of a second to express their satisfaction.
Basically, all I'm interested in is whether the customer is happy with the purchase or not. And if not, why not. It's that simple.
-Steve
> it to make any statistical sense.
Yes, good point, Jordan.
> One idea off the top of my head is build in a rating system on the "Add Functionality"
> page of the dashboard -- unrated packages from the marketplace could have
> the star-rating control right there in the list of installed packages.
That's not a bad idea, but I have to say that I'm not convinced of the value of the 5 star ratings. I'd like to suggest a much simpler approach - either favorable or unfavorable. IOW, the options would be thumbs up or down, and if thumbs down, they'd have to provide at least SOME supportive commentary - i.e. couldn't submit a negative rating without a comment.
The bottom line is that MOST people simply don't take the time to leave any kind of feedback at all, and in my experience, that generally means they are happy with the purchase. As soon as you "nuance" the rating with various levels, that almost demands that the user explain their rating, and very few want to take the time to explain their rationale. Simply clicking one of two icons - favorable or unfavorable - simplifies the feedback process. MOST people will click a "thumbs up" icon if they are satisfied with the product and be done with it. I, as a developer, know what I want to know - that my product met or exceeded their expectations, and it took the customer only a fraction of a second to express their satisfaction.
Basically, all I'm interested in is whether the customer is happy with the purchase or not. And if not, why not. It's that simple.
-Steve
There's a lot of strength to that argument.
The 5 stars thing also creates a weird and inaccurate comparison
issue. Is this image gallery better than that one because it has 4 vs
3 stars? That assumes the same person is comparing both in a
randomized setting, which simply isn't what's going on.
best wishes
Franz Maruna
CEO - concrete5.org
http://about.me/frz
The 5 stars thing also creates a weird and inaccurate comparison
issue. Is this image gallery better than that one because it has 4 vs
3 stars? That assumes the same person is comparing both in a
randomized setting, which simply isn't what's going on.
best wishes
Franz Maruna
CEO - concrete5.org
http://about.me/frz
I think I lot of the problem here could be fixed simply by just adding a couple of lines of text for reviewer on that interface, suggesting that reviewers who have encountered a technical issue or who are looking for some particular feature try using the support forum first to give the developer an opportunely to address their concerns, and otherwise, asking reviewers to elaborate a bit.
We actually just added this in the push: when you first click the "add review" button you get a notice that reviews last forever, and you really shouldn't review something negatively unless you've already opened a support ticket.
nice, thanks
I understand what you're talking about here -- I recently got a *really* strange (bad) review for an addon. Never got contacted about it. I responded to the review and PM'ed the poster saying I can help them fix it or even give them a refund, but they never responded. I couldn't understand why someone who spent money and hated it wouldn't even ask for a refund?! (Perhaps they had email notifications turned off in their profile? This was the case for me the first year or so of being a C5 community member!)
That being said, I'd like to play a little devil's advocate here and suggest that since these are paying customers, there is no motivation to be a troll for troll's sake -- it might be that the system is confusing or that things were presented to them in a way that made them think they were getting one thing when in fact they were getting something else.
Sounds like a lot some of the recent modifications by Andy and Franz will help address these issues, which is awesome.
Also, I wish there was some way to send an EMAIL (not a PM, because users might have notifications turned off) to disgruntled customers. But I understand there's privacy concerns here. At the very least would be nice to know if the user in question even has email notifications turned on or off for PM's -- at least we'd know that they're never going to hear anything we say.
-Jordan
That being said, I'd like to play a little devil's advocate here and suggest that since these are paying customers, there is no motivation to be a troll for troll's sake -- it might be that the system is confusing or that things were presented to them in a way that made them think they were getting one thing when in fact they were getting something else.
Sounds like a lot some of the recent modifications by Andy and Franz will help address these issues, which is awesome.
Also, I wish there was some way to send an EMAIL (not a PM, because users might have notifications turned off) to disgruntled customers. But I understand there's privacy concerns here. At the very least would be nice to know if the user in question even has email notifications turned on or off for PM's -- at least we'd know that they're never going to hear anything we say.
-Jordan
