Research Papers: D3 Methods

Mitigating Online Product Rating Biases Through the Discovery of Optimistic, Pessimistic, and Realistic Reviewers

[+] Author and Article Information
Sunghoon Lim

Industrial and Manufacturing Engineering,
The Pennsylvania State University,
University Park, PA 16802
e-mail: slim@psu.edu

Conrad S. Tucker

Engineering Design and Industrial and
Manufacturing Engineering,
The Pennsylvania State University,
University Park, PA 16802
e-mail: ctucker4@psu.edu

1Corresponding author.

Contributed by the Design Automation Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received March 31, 2017; final manuscript received August 2, 2017; published online October 2, 2017. Assoc. Editor: Yan Wang.

J. Mech. Des 139(11), 111409 (Oct 02, 2017) (11 pages) Paper No: MD-17-1247; doi: 10.1115/1.4037612 History: Received March 31, 2017; Revised August 02, 2017

The authors of this work present a model that reduces product rating biases that are a result of varying degrees of customers' optimism/pessimism. Recently, large-scale customer reviews and numerical product ratings have served as substantial criteria for new customers who make their purchasing decisions through electronic word-of-mouth. However, due to differences among reviewers' rating criteria, customer ratings are often biased. For example, a three-star rating can be considered low for an optimistic reviewer. On the other hand, the same three-star rating can be considered high for a pessimistic reviewer. Many existing studies of online customer reviews overlook the significance of reviewers' rating histories and tendencies. Considering reviewers' rating histories and tendencies is significant for identifying unbiased customer ratings and true product quality, because each reviewer has different criteria for buying and rating products. The proposed customer rating analysis model adjusts product ratings in order to provide customers with more objective and accurate feedback. The authors propose an unsupervised model aimed at mitigating customer ratings based on rating histories and tendencies, instead of human-labeled training data. A case study involving real-world customer rating data from an electronic commerce company is used to validate the method.

Copyright © 2017 by ASME
Your Session has timed out. Please sign back in to continue.


Jiang, Y. , Shang, J. , and Liu, Y. , 2010, “ Maximizing Customer Satisfaction Through an Online Recommendation System: A Novel Associative Classification Model,” Decis. Support Syst., 48(3), pp. 470–479. [CrossRef]
Hu, N. , Bose, I. , Koh, N. S. , and Liu, L. , 2012, “ Manipulation of Online Reviews: An Analysis of Ratings, Readability, and Sentiments,” Decis. Support Syst., 52(3), pp. 674–684. [CrossRef]
Tuarob, S. , and Tucker, C. S. , 2015, “ Automated Discovery of Lead Users and Latent Product Features by Mining Large Scale Social Media Networks,” ASME J. Mech. Des., 137(7), p. 071402. [CrossRef]
Singh, A. S. , and Tucker, C. S. , 2015, “ Investigating the Heterogeneity of Product Feature Preferences Mined Using Online Product Data Streams,” ASME Paper No. DETC2015-47439.
Kang, S. , and Tucker, C. S. , 2016, “ Automated Mapping of Product Features Mined From Online Customer Reviews to Engineering Product Characteristics,” ASME Paper No. DETC2016-59772.
Lim, S. , and Tucker, C. S. , 2016, “ A Bayesian Sampling Method for Product Feature Extraction From Large-Scale Textual Data,” ASME J. Mech. Des., 138(6), p. 061403. [CrossRef]
Singh, A. S. , and Tucker, C. S. , 2017, “ A Machine Learning Approach to Product Review Disambiguation Based on Function, Form and Behavior Classification,” Decis. Support Syst., 97, pp. 81–91. [CrossRef]
Rose, S. , Hair, N. , and Clark, M. , 2011, “ Online Customer Experience: A Review of the Business-to-Consumer Online Purchase Context: Online Customer Experience,” Int. J. Manage. Rev., 13(1), pp. 24–39. [CrossRef]
Srinivasan, S. S. , Anderson, R. , and Ponnavolu, K. , 2002, “ Customer Loyalty in e-Commerce: An Exploration of Its Antecedents and Consequences,” J. Retailing, 78(1), pp. 41–50. [CrossRef]
Chevalier, J. A. , and Mayzlin, D. , 2006, “ The Effect of Word of Mouth on Sales: Online Book Reviews,” J. Mark. Res., 43(3), pp. 345–354. [CrossRef]
Chen, P.-Y. , Dhanasobhon, S. , and Smith, M. D. , 2008, “ All Reviews Are Not Created Equal: The Disaggregate Impact of Reviews and Reviewers at Amazon.com,” SSRN, Rochester, NY, accessed Aug. 22, 2017, https://ssrn.com/abstract=918083
Aral, S. , 2014, “ The Problem With Online Ratings,” MIT Sloan Manage. Rev., 55(2), pp. 47–52.
Dave, K. , Lawrence, S. , and Pennock, D. M. , 2003, “ Mining the Peanut Gallery: Opinion Extraction and Semantic Classification of Product Reviews,” 12th International Conference on World Wide Web (WWW), Budapest, Hungary, May 20–24, pp. 519–528.
Mudambi, S. M. , and Schuff, D. , 2010, “ What Makes a Helpful Review? A Study of Customer Reviews on Amazon.com,” MIS Q., 34(1), pp. 185–200.
Hu, N. , Pavlou, P. A. , and Zhang, J. J. , 2009, “ Why Do Online Product Reviews Have a J-Shaped Distribution? Overcoming Biases in Online Word-of-Mouth Communication,” Commun. ACM, 52(10), pp. 144–147. [CrossRef]
Lim, S. , Tucker, C. S. , and Kumara, S. , 2017, “ An Unsupervised Machine Learning Model for Discovering Latent Infectious Diseases Using Social Media Data,” J. Biomed. Inf., 66, pp. 82–94. [CrossRef]
Asur, S. , and Huberman, B. A. , 2010, “ Predicting the Future With Social Media,” IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT), Toronto, ON, Canada, Aug. 31–Sept. 3, pp. 492–499.
Zhan, J. , Loh, H. T. , and Liu, Y. , 2009, “ Gather Customer Concerns From Online Product Reviews—A Text Summarization Approach,” Expert Syst. Appl., 36(2), pp. 2107–2115. [CrossRef]
Menon, R. , Tong, L. H. , Sathiyakeerthi, S. , Brombacher, A. , and Leong, C. , 2004, “ The Needs and Benefits of Applying Textual Data Mining Within the Product Development Process,” Qual. Reliab. Eng. Int., 20(1), pp. 1–15. [CrossRef]
Rai, R. , 2012, “ Identifying Key Product Attributes and Their Importance Levels From Online Customer Reviews,” ASME Paper No. DETC2012-70493.
Wong, T.-L. , and Lam, W. , 2008, “ Learning to Extract and Summarize Hot Item Features From Multiple Auction Web Sites,” Knowl. Inf. Syst., 14(2), pp. 143–160. [CrossRef]
Wong, T.-L. , and Lam, W. , 2009, “ An Unsupervised Method for Joint Information Extraction and Feature Mining Across Different Web Sites,” Data Knowl. Eng., 68(1), pp. 107–125. [CrossRef]
Wu, F. , and Huberman, B. A. , 2010, “ Opinion Formation Under Costly Expression,” ACM Trans. Intell. Syst. Technol., 1(1), pp. 1–13. [CrossRef]
Tucker, C. , and Kim, H. , 2011, “ Predicting Emerging Product Design Trend by Mining Publicly Available Customer Review Data,” 18th International Conference on Engineering Design, Impacting Society Through Engineering Design, Lyngby/Copenhagen, Denmark, Aug. 15–18, pp. 43–52.
Liu, Y. , Jin, J. , Ji, P. , Harding, J. A. , and Fung, R. Y. , 2013, “ Identifying Helpful Online Reviews: A Product Designer's Perspective,” Comput.-Aided Des., 45(2), pp. 180–194. [CrossRef]
Ferguson, T. , Greene, M. , Repetti, F. , Lewis, K. , and Behdad, S. , 2015, “ Combining Anthropometric Data and Consumer Review Content to Inform Design for Human Variability,” ASME Paper No. DETC2015-47640.
Archak, N. , Ghose, A. , and Ipeirotis, P. G. , 2007, “ Show Me the Money!: Deriving the Pricing Power of Product Features by Mining Consumer Reviews,” 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Jose, CA, Aug. 12–15, pp. 56–65.
McGlohon, M. , Glance, N. S. , and Reiter, Z. , 2010, “ Star Quality: Aggregating Reviews to Rank Products and Merchants,” Fourth International AAAI Conference on Weblogs and Social Media (ICWSM), Washington, DC, May 23–26, pp. 114–121.
Qiu, L. , Pang, J. , and Lim, K. H. , 2012, “ Effects of Conflicting Aggregated Rating on eWOM Review Credibility and Diagnosticity: The Moderating Role of Review Valence,” Decis. Support Syst., 54(1), pp. 631–643. [CrossRef]
Mukherjee, A. , Venkataraman, V. , Liu, B. , and Glance, N. S. , 2013, “ What Yelp Fake Review Filter Might Be Doing?,” Seventh International AAAI Conference on Weblogs and Social Media (ICWSM), Cambridge, MA, July 8–11, pp. 409–418.
Lim, E.-P. , Nguyen, V.-A. , Jindal, N. , Liu, B. , and Lauw, H. W. , 2010, “ Detecting Product Review Spammers Using Rating Behaviors,” 19th ACM International Conference on Information and Knowledge Management (CIKM), Toronto, ON, Canada, Oct. 26–30, pp. 939–948.
Mukherjee, A. , Liu, B. , and Glance, N. , 2012, “ Spotting Fake Reviewer Groups in Consumer Reviews,” 21st International Conference on World Wide Web (WWW), Lyon, France, Apr. 16–20, pp. 191–200.
Willemsen, L. M. , Neijens, P. C. , Bronner, F. , and de Ridder, J. A. , 2011, “ ‘Highly Recommended!’ The Content Characteristics and Perceived Usefulness of Online Consumer Reviews,” J. Comput.-Mediated Commun., 17(1), pp. 19–38. [CrossRef]
Hu, N. , Pavlou, P. A. , and Zhang, J. , 2006, “ Can Online Reviews Reveal a Product's True Quality?: Empirical Findings and Analytical Modeling of Online Word-of-Mouth Communication,” Seventh ACM Conference on Electronic Commerce (EC), Ann Arbor, MI, June 11–15, pp. 324–330.
Fei, G. , Mukherjee, A. , Liu, B. , Hsu, M. , Castellanos, M. , and Ghosh, R. , 2013, “ Exploiting Burstiness in Reviews for Review Spammer Detection,” Seventh International AAAI Conference on Weblogs and Social Media (ICWSM), Cambridge, MA, July 8–12, pp. 175–184.
Zhang, J. Q. , Craciun, G. , and Shin, D. , 2010, “ When Does Electronic Word-of-Mouth Matter? A Study of Consumer Product Reviews,” J. Bus. Res., 63(12), pp. 1336–1341. [CrossRef]
Kousha, K. , and Thelwall, M. , 2016, “ Can Amazon.com Reviews Help to Assess the Wider Impacts of Books?,” J. Assoc. Inf. Sci. Technol., 67(3), pp. 566–581. [CrossRef]
Friedman, M. , and Kandel, A. , 1999, Introduction to Pattern Recognition: Statistical, Structural, Neural and Fuzzy Logic Approaches, Vol. 32, Imperial College Press, London. [CrossRef]
Popoviciu, T. , 1935, “ Sur les Équations Algébriques Ayant Toutes Leurs Racines Réelles,” Mathematica, 9, pp. 129–145.
Cormen, T. H. , Leiserson, C. E. , Rivest, R. L. , and Stein, C. , 2001, Introduction to Algorithms, Vol. 6, MIT Press, Cambridge, UK.
Fisher, R. A. , 1921, “ On the Probable Error of a Coefficient of Correlation Deduced From a Small Sample,” Metron, 1, pp. 3–32.
Bird, S. , 2006, “ NLTK: The Natural Language Toolkit,” COLING/ACL on Interactive Presentation Sessions, Sydney, Australia, July 17–18, pp. 69–72.
McAuley, J. , Targett, C. , Shi, Q. , and van den Hengel, A. , 2015, “ Image-Based Recommendations on Styles and Substitutes,” 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Santiago, Chile, Aug. 9–13, pp. 43–52.
McAuley, J. , Pandey, R. , and Leskovec, J. , 2015, “ Inferring Networks of Substitutable and Complementary Products,” 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sydney, Australia, Aug. 10–13, pp. 785–794.


Grahic Jump Location
Fig. 3

The distributions of Amazon.com's and the lab experiments' product star ratings

Grahic Jump Location
Fig. 2

An example of biased ratings, along with reviewer A's and reviewer B's product rating histories

Grahic Jump Location
Fig. 1

Classification of optimistic/pessimistic/realistic/unreliable reviewers based on customer rating histories and product sales rankings

Grahic Jump Location
Fig. 4

Overview of the proposed method

Grahic Jump Location
Fig. 5

Possible probability distributions of product ratings of optimistic, pessimistic, realistic, and unreliable reviewers

Grahic Jump Location
Fig. 6

An example of applying a minimum distance classifier to classify Ci

Grahic Jump Location
Fig. 7

The correlation coefficients of product sales rankings

Grahic Jump Location
Fig. 8

The correlation coefficients of reviewers' sentiment scores

Grahic Jump Location
Fig. 9

The distributions of the original star ratings and the adjusted star ratings for 739 products



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In