Research Papers: Design Theory and Methodology

Mass Collaboration Project Recommendation Within Open-Innovation Design Networks

[+] Author and Article Information
Zachary Ball

Mechanical and Aerospace Engineering,
University at Buffalo – SUNY,
Buffalo, NY 14260
e-mail: zlball@buffalo.edu

Kemper Lewis

Fellow ASME
Mechanical and Aerospace Engineering,
University at Buffalo – SUNY,
Buffalo, NY 14260
e-mail: kelewis@buffalo.edu

1Corresponding author.

Contributed by the Design Theory and Methodology Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received July 5, 2018; final manuscript received October 19, 2018; published online December 20, 2018. Assoc. Editor: Scarlett Miller.

J. Mech. Des 141(2), 021105 (Dec 20, 2018) (11 pages) Paper No: MD-18-1537; doi: 10.1115/1.4041858 History: Received July 05, 2018; Revised October 19, 2018

Mass collaboration within the design engineering process supports the inclusion of unique perspectives when working on complex problems. Increasing the number of individuals providing input and support into these perplexing challenges can increase innovation, decrease product development times, and provide solutions that truly encompass the needs of the market. One of the greatest challenges within mass collaboration engineering projects is the organization of individuals within these large design efforts. Understanding which projects would most effectively benefit from additional designers or contributors is paramount to supporting mass collaboration design networks. Within such networks, there exists a large number of contributors as well as a large pool of potential projects. Matching individuals with the projects that they can provide the greatest benefit to or building a team of individuals for newly developed projects requires the consideration of previous performance and an understanding of individual competencies and design abilities. This work presents a framework which recommends individual project placement based on individual abilities and the project requirements. With this work, a pool of individuals and potential projects are simulated, and the application of a hybrid recommender system is explored. To complement the simulation, an additional case study with empirical data is performed to study the potential applicability of the proposed framework. Overall, it was found that recommended team compositions greatly outperform the baseline team development, most notably as greater consideration is placed on collaborative recommendations.

Copyright © 2019 by ASME
Your Session has timed out. Please sign back in to continue.


Howe, J. , 2006, “ The Rise of Crowdsourcing,” accessed Dec. 2016, http://www.wired.com/wired/archive/14.06/crowds.html
Estellés-Arolas, E. , and González-Ladrón-de-Guevara, F. , 2012, “ Towards an Integrated Crowdsourcing Definition,” J. Inf. Sci., 38(2), pp. 189–200. [CrossRef]
Miller, B. N. , Albert, I. , Lam, S. K. , Konstan, J. A. , and Riedl, J. , 2003, “ MovieLens Unplugged: Experiences With an Occasionally Connected Recommender System,” Eighth International Conference on Intelligent User Interfaces, Miami, FL, Jan. 12–15, pp. 263–266. http://citeseerx.ist.psu.edu/viewdoc/download?doi=
Linden, G. , Smith, B. , and York, J. , 2003, “ Amazon.com Recommendations: Item-to-Item Collaborative Filtering,” IEEE Internet Comput., 7(1), pp. 76–80. [CrossRef]
Liu, J. , Dolan, P. , and Pedersen, E. R. , 2010, “ Personalized News Recommendation Based on Click Behavior,” 15th International Conference on Intelligent User Interfaces, Hong Kong, China, Feb. 7–10, pp. 31–40.
Aitamurto, T. , 2012, Crowdsourcing for Democracy: A New Era in Policy-Making, Parliament of Finland, Helsinki, Finland.
Aitamurto, T. , and Landemore, H. E. , 2015, “ Five Design Principles for Crowdsourced Policymaking: Assessing the Case of Crowdsourced Off-Road Traffic Law in Finland,” J. Soc. Media Organ., 2(1), pp. 1–19. http://www2.mitre.org/public/jsmo/pdfs/02-01-5-design-principles.pdf
Brabham, D. C. , 2009, “ Crowdsourcing the Public Participation Process for Planning Projects,” Plan. Theory, 8(3), pp. 242–262. [CrossRef]
Bentzien, J. , Bharadwaj, R. , and Thompson, D. C. , 2015, “ Crowdsourcing in Pharma: A Strategic Framework,” Drug Discov. Today, 20(7), pp. 874–883. [CrossRef] [PubMed]
Poetz, M. K. , and Schreier, M. , 2012, “ The Value of Crowdsourcing: Can Users Really Compete With Professionals in Generating New Product Ideas?,” J. Prod. Innov. Manage., 29(2), pp. 245–256. [CrossRef]
Koch, G. , Füller, J. , and Brunswicker, S. , 2011, “ Online Crowdsourcing in the Public Sector: How to Design Open Government Platforms,” Online Communities and Social Computing, Springer, Berlin, pp. 203–212.
Brabham, D. C. , 2013, “ Using Crowdsourcing in Government,” IBM Center for the Business of Government, Washington, DC, accessed Dec. 5, 2016, http://www.businessofgovernment.org/report/using-crowdsourcing-government
Schenk, E. , and Guittard, C. , 2009, “ Crowdsourcing: What Can Be Outsourced to the Crowd, and Why,” Workshop on Open Source Innovation, Strasbourg, France, p. 72.
Brabham, D. C. , 2008, “ Crowdsourcing as a Model for Problem Solving: An Introduction and Cases,” Converg. Int. J. Res. New Media Technol., 14(1), pp. 75–90. [CrossRef]
Howe, J. , 2008, Crowdsourcing: Why the Power of the Crowd is Driving the Future of Business, Three Rivers Press, New York.
Panchal, J. H. , Sha, Z. , and Kannan, K. N. , 2017, “ Understanding Design Decisions Under Competition Using Games With Information Acquisition and a Behavioral Experiment,” ASME J. Mech. Des., 139(9), p. 091402.
Sha, Z. , Kannan, K. N. , and Panchal, J. H. , 2015, “ Behavioral Experimentation and Game Theory in Engineering Systems Design,” ASME J. Mech. Des., 137(5), p. 051405.
Vincent, T. L. , 1983, “ Game Theory as a Design Tool,” J. Mech. Transm. Autom. Des., 105(2), pp. 165–170. [CrossRef]
Lewis, K. , and Mistree, F. , 1997, “ Modeling Interactions in Multidisciplinary Design: A Game Theoretic Approach,” AIAA J., 35(8), pp. 1387–1392. [CrossRef]
Takai, S. , 2010, “ A Game-Theoretic Model of Collaboration in Engineering Design,” ASME J. Mech. Des., 132(5), p. 051005.
Takai, S. , 2016, “ A Multidisciplinary Framework to Model Complex Team-Based Product Development,” ASME J. Mech. Des., 138(6), p. 061402.
Brabham, D. C. , 2010, “ Moving the Crowd at Threadless: Motivations for Participation in a Crowdsourcing Application,” Inf. Commun. Soc., 13(8), pp. 1122–1145. [CrossRef]
Ren, Y. , Bayrak, A. E. , and Papalambros, P. Y. , 2016, “ Ecoracer: Game-Based Optimal Electric Vehicle Design and Driver Control Using Human Players,” ASME J. Mech. Des., 138(6), p. 061407.
Burnap, A. , Ren, Y. , Gerth, R. , Papazoglou, G. , Gonzalez, R. , and Papalambros, P. Y. , 2015, “ When Crowdsourcing Fails: A Study of Expertise on Crowdsourced Design Evaluation,” ASME J. Mech. Des., 137(3), p. 031101.
Burnap, A. , Ren, Y. , Papalambros, P. Y. , Gonzalez, R. , and Gerth, R. , 2013, “ A Simulation Based Estimation of Crowd Ability and Its Influence on Crowdsourced Evaluation of Design Concepts,” ASME Paper No. DETC2013-13020.
Fuge, M. , and Agogino, A. , 2014, “ How Online Design Communities Evolve Over Time: The Birth and Growth of OpenIDEO,” ASME Paper No. DETC2014-35243.
Fuge, M. , Tee, K. , Agogino, A. , and Maton, N. , 2014, “ Analysis of Collaborative Design Networks: A Case Study of OpenIDEO,” ASME J. Comput. Inf. Sci. Eng., 14(2), p. 021009.
Ball, Z. , and Lewis, K. , 2017, “ The Design of the Crowd: Organizing Mass Collaboration Efforts,” ASME Paper No. DETC2017-68127.
Ball, Z. , and Lewis, K. , 2018, “ Project Recommendation for Mass Collaboration Design Networks,” ASME Paper No. DETC2018-85978.
Ball, Z. , and Lewis, K. , 2018, “ Observing Network Characteristics in Mass Collaboration Design Projects,” Des. Sci., 4, p. e4.
Ricci, F. , Rokach, L. , and Shapira, B. , 2011, “ Introduction to Recommender Systems Handbook,” Recommender Systems Handbook, Springer, Boston, MA, pp. 1–35.
Adomavicius, G. , and Tuzhilin, A. , 2005, “ Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions,” IEEE Trans. Knowl. Data Eng., 17(6), pp. 734–749. [CrossRef]
Baeza-Yates, R. , and Ribeiro-Neto, B. , 1999, Modern Information Retrieval, ACM Press, New York.
Salton, G. , 1989, “ Automatic Text Processing: The Transformation, Analysis, and Retrieval of Read,” Addison-Wesley, Reading, MA.
Pazzani, M. , and Billsus, D. , 1997, “ Learning and Revising User Profiles: The Identification of Interesting Web Sites,” Mach. Learn., 27(3), pp. 313–331. [CrossRef]
Lops, P. , De Gemmis, M. , and Semeraro, G. , 2011, “ Content-Based Recommender Systems: State of the Art and Trends,” Recommender Systems Handbook, Springer, Boston, MA, pp. 73–105.
Sarwar, B. , Karypis, G. , Konstan, J. , and Riedl, J. , 2001, “ Item-Based Collaborative Filtering Recommendation Algorithms,” Tenth International Conference on World Wide Web, Hong Kong, China, May 1–5, pp. 285–295.
Aggarwal, C. C. , Wolf, J. L. , Wu, K.-L. , and Yu, P. S. , 1999, “ Horting Hatches an Egg: A New Graph-Theoretic Approach to Collaborative Filtering,” Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, Aug. 15–18, pp. 201–212.
Pazzani, M. J. , 1999, “ A Framework for Collaborative, Content-Based and Demographic Filtering,” Artif. Intell. Rev., 13(5/6), pp. 393–408. [CrossRef]
Geiger, D. , and Schader, M. , 2014, “ Personalized Task Recommendation in Crowdsourcing Information Systems—Current State of the Art,” Decis. Support Syst., 65, pp. 3–16. [CrossRef]
Stankovic, M. , Jovanovic, J. , and Laublet, P. , 2011, “ Linked Data Metrics for Flexible Expert Search on the Open Web,” Extended Semantic Web Conference, Crete, Greece, May 29–June 2, pp. 108–123.
Dror, G. , Koren, Y. , Maarek, Y. , and Szpektor, I. , 2011, “ I Want to Answer; Who Has a Question?: Yahoo! Answers Recommender System,” 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, Aug. 21–24, pp. 1109–1117.
Chilton, L. B. , Horton, J. J. , Miller, R. C. , and Azenkot, S. , 2010, “ Task Search in a Human Computation Market,” ACM SIGKDD Workshop on Human Computation, Washington, DC, July 25, pp. 1–9.
Yuen, M.-C. , King, I. , and Leung, K.-S. , 2011, “ Task Matching in Crowdsourcing,” International Conference on Internet of Things and Fourth International Conference on Cyber, Physical and Social Computing (IThings/CPSCom), Dalian, China, Oct. 19–22, pp. 409–412.
Cosley, D. , Frankowski, D. , Terveen, L. , and Riedl, J. , 2007, “ SuggestBot: Using Intelligent Task Routing to Help People Find Work in Wikipedia,” 12th International Conference on Intelligent User Interfaces, Honolulu, HI, Jan. 28–31, pp. 32–41.
Castro-Herrera, C. , 2010, “ A Hybrid Recommender System for Finding Relevant Users in Open Source Forums,” Third International Workshop On Managing Requirements Knowledge (MARK), Sydney, Australia, Sept. 27, pp. 41–50.
Kao, W.-C. , Liu, D.-R. , and Wang, S.-W. , 2010, “ Expert Finding in Question-Answering Websites: A Novel Hybrid Approach,” ACM Symposium on Applied Computing, Sierre, Switzerland, Mar. 22–26, pp. 867–871.
Liu, X. , Croft, W. B. , and Koll, M. , 2005, “ Finding Experts in Community-Based Question-Answering Services,” 14th ACM International Conference on Information and Knowledge Management, Bremen, Germany, Oct. 31–Nov. 5, pp. 315–316.
Riahi, F. , Zolaktaf, Z. , Shafiei, M. , and Milios, E. , 2012, “ Finding Expert Users in Community Question Answering,” 21st International Conference on World Wide Web, Lyon, France, Apr. 16–20, pp. 791–798.
University at Buffalo School of Engineering and Applied Sciences, 2016, “ Course Flowsheets,” University at Buffalo, Buffalo, NY, accessed Dec. 5, 2016, https://www.eng.buffalo.edu/undergrad/advisement/flowsheets/
Pedersen, K. , Emblemsvag, J. , Bailey, R. , Allen, J. K. , and Mistree, F. , 2000, “ Validating Design Methods Research,” ASME Paper No. DETC2000/DTM-14579. https://www.researchgate.net/profile/Farrokh_Mistree/publication/238355807_The''Validation_Square''-Validating_Design_Methods/links/5616e3b808ae839f3c7d586c.pdf
Eppinger, S. D. , and Browning, T. R. , 2012, Design Structure Matrix Methods and Applications, MIT Press, Cambridge, MA.
Browning, T. R. , 2001, “ Applying the Design Structure Matrix to System Decomposition and Integration Problems: A Review and New Directions,” IEEE Trans. Eng. Manage., 48(3), pp. 292–306. [CrossRef]
Satzger, B. , Psaier, H. , Schall, D. , and Dustdar, S. , 2011, “ Stimulating Skill Evolution in Market-Based Crowdsourcing,” International Conference on Business Process Management, Clermont-Ferrand, France, Aug. 30–Sept. 2, Raleigh, NC, Apr. 26–30, pp. 66–82.
Horowitz, D. , and Kamvar, S. D. , 2010, “ The Anatomy of a Large-Scale Social Search Engine,” 19th International Conference on World Wide Web, pp. 431–440.
Burke, R. , and Ramezani, M. , 2011, “ Matching Recommendation Technologies and Domains,” Recommender Systems Handbook, Springer, Boston, MA, Raleigh, NC, Apr. 26–30, pp. 367–386.
Kramer, J. , Agogino, A. M. , and Roschuni, C. , 2016, “ Characterizing Competencies for Human-Centered Design,” ASME Paper No. DETC2016-60085.


Grahic Jump Location
Fig. 1

Flow chart of proposed recommendation system

Grahic Jump Location
Fig. 2

Default parameters over 500 projects with 95% confidence

Grahic Jump Location
Fig. 3

Default parameters over 2500 projects with 95% confidence

Grahic Jump Location
Fig. 4

Design Scores over 500 projects with 95% confidence

Grahic Jump Location
Fig. 5

Average design scores at varying weights with 95% confidence

Grahic Jump Location
Fig. 6

Surface plot showing the impacts of similarity and performance parameters

Grahic Jump Location
Fig. 7

Project completion frequency at varying degrees of similarity and performance

Grahic Jump Location
Fig. 8

Design scores over multiple projects with 95% confidence



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In