Research Papers: Design Automation

Sensitivity of Vehicle Market Share Predictions to Discrete Choice Model Specification

[+] Author and Article Information
C. Grace Haaf

Mechanical Engineering,
Carnegie Mellon University,
Pittsburgh, PA 15213
e-mail: grace.haaf@gmail.com

Jeremy J. Michalek

Mechanical Engineering,
Engineering and Public Policy,
Carnegie Mellon University,
Pittsburgh, PA 15213
e-mail: jmichalek@cmu.edu

W. Ross Morrow

Assistant Professor
Mechanical Engineering,
Iowa State University,
Ames, IA 50011
e-mail: wrmorrow@iastate.edu

Yimin Liu

Technical Expert
Ford Motor Company,
Dearborn, MI 48121
e-mail: Yliu59@ford.com

An electronic companion to this paper containing the appendices referenced herein can be found at http://repository.cmu.edu/meche/70/

We use the term “vehicle design” to refer to vehicle make-model.

This is distinct from the “class dummies only logit” which includes data for the entire market but uses only dummies representing each class as covariates.

A likelihood ratio test of the best logit and mixed logit models calculated on 2007 data suggests that there is sufficient evidence to reject the null hypothesis that the mixed logit model predicts significantly better at the α = 0.1 level.

We reject the null hypothesis that the coefficient is equal to zero at the α = 0.01 level for a two-sided t-test.

Contributed by the Design Automation Committee of ASME for publication in the JOURNAL OF MECHANICAL DESIGN. Manuscript received May 28, 2013; final manuscript received June 23, 2014; published online October 20, 2014. Assoc. Editor: Bernard Yannou.

J. Mech. Des 136(12), 121402 (Oct 20, 2014) (9 pages) Paper No: MD-13-1232; doi: 10.1115/1.4028282 History: Received May 28, 2013; Revised June 23, 2014

When design decisions are informed by consumer choice models, uncertainty in choice model predictions creates uncertainty for the designer. We investigate the variation and accuracy of market share predictions by characterizing fit and forecast accuracy of discrete choice models for the US light duty new vehicle market. Specifically, we estimate multinomial logit models for 9000 utility functions representative of a large literature in vehicle choice modeling using sales data for years 2004–2006. Each model predicts shares for the 2007 and 2010 markets, and we compare several quantitative measures of model fit and predictive accuracy. We find that (1) our accuracy measures are concordant: model specifications that perform well on one measure tend to also perform well on other measures for both fit and prediction. (2) Even the best discrete choice models exhibit substantial prediction error, stemming largely from limited model fit due to unobserved attributes. A naïve “static” model, assuming share for each vehicle design in the forecast year = share in the last available year, outperforms all 9000 attribute-based models when predicting the full market one year forward, but attribute-based models can predict better for four year forward forecasts or new vehicle designs. (3) Share predictions are sensitive to the presence of utility covariates but less sensitive to covariate form (e.g., miles per gallons versus gallons per mile), and nested and mixed logit specifications do not produce significantly more accurate forecasts. This suggests ambiguity in identifying a unique model form best for design. Furthermore, the models with best predictions do not necessarily have expected coefficient signs, and biased coefficients could misguide design efforts even when overall prediction accuracy for existing markets is maximized.

Copyright © 2014 by ASME
Your Session has timed out. Please sign back in to continue.


Hazelrigg, G. A., 1998, “A Framework for Decision-Based Engineering Design,”ASME J. Mech. Des., 120(4), pp. 653–658. [CrossRef]
Train, K. E., 2009, Discrete Choice Models With Simulation, Cambridge University, New York.
Wassenaar, H. J., and Chen, W., 2003, “An Approach to Decision-Based Design With Discrete Choice Analysis for Demand Modeling,” ASME J. Mech. Des., 125(3), pp. 490–497. [CrossRef]
Wassenaar, H. J., Chen, W., Cheng, J., and Sudjianto, A., 2005, “Enhancing Discrete Choice Demand Modeling for Decision-Based Design,” ASME J. Mech. Des., 127(4), pp. 514–523. [CrossRef]
Besharati, B., Luo, L., Azarm, S., and Kannan, P. K., 2006, “Multi-Objective Single Product Robust Optimization: An Integrated Design and Marketing Approach,” ASME J. Mech. Des., 128(4), pp. 884–892. [CrossRef]
Williams, N., Azarm, S., and Kannan, P. K., 2008, “Engineering Product Design Optimization for Retail Channel Acceptance,” ASME J. Mech. Des., 130(6), p. 061402. [CrossRef]
Shiau, C.-S. N., and Michalek, J. J., 2009, “Optimal Product Design Under Price Competition,” ASME J. Mech. Des., 131(7), p. 071003. [CrossRef]
Frischknecht, B. D., Whitefoot, K., and Papalambros, P. Y., 2010, “On the Suitability of Econometric Demand Models in Design for Market Systems,” ASME J. Mech. Des., 132(12), p. 121007. [CrossRef]
He, L., Chen, W., Hoyle, C., and Yannou, B., 2012, “Choice Modeling for Usage Context-Based Design,” ASME J. Mech. Des., 134(3), p. 031007. [CrossRef]
Morrow, W. R., Long, M., and MacDonald, E. F., 2012, “Models in Decision Based Design Optimization,” ASME DETC2012-71176. [CrossRef]
Whitefoot, K. S., and Skerlos, S. J., 2012, “Design Incentives to Increase Vehicle Size Created From the U.S. Footprint-Based Fuel Economy Standards,” Energy Policy, 41(1), pp. 402–411. [CrossRef]
Resende, C. B., Grace Heckmann, C., and Michalek, J. J., 2012, “Robust Design for Profit Maximization With Aversion to Downside Risk From Parametric Uncertainty in Consumer Choice Models,” ASME J. Mech. Des., 134(10), p. 100901. [CrossRef]
Payne, J. W., 1976, “Task Complexity and Contingent Processing in Decision Making: An Information Search,” Organ. Behav. Hum. Perform., 16(2), pp. 366–387. [CrossRef]
MacDonald, E. F., Gonzalez, R., and Papalambros, P. Y., 2009, “Preference Inconsistency in Multidisciplinary Design Decision Making,” ASME J. Mech. Des., 131(3), p. 031009. [CrossRef]
Morrow, W. R., Long, M., and MacDonald, E. F., 2014, “Market-System Design Optimization With Consider-Then-Choose Models,” ASME J. Mech. Des., 136(3). p. 031003. [CrossRef]
Knittel, C. R., and Metaxoglou, K., 2012, “Estimation of Random Coefficient Demand Models: Two Empiricists' Perspective,” Rev. Econ. Stat., 96(1), pp. 34–59. [CrossRef]
Allcott, H., and Wozny, N., 2012, “Gasoline Prices, Fuel Economy, and the Energy Paradox,” Rev. Econ. Stat. (to be published).
Dubé, J.-P., Fox, J. T., and Su, C.-L., 2012, “Improving the Numerical Performance of Static and Dynamic Aggregate Discrete Choice Random Coefficients Demand Estimation,” Econometrica, 80(5), pp. 2231–2267. [CrossRef]
Beresteanu, A., and Li, S., 2011, “Gasoline Prices, Government Support, and the Demand for Hybrid Vehicles in the United States,” Int. Econ. Rev. (Philadelphia), 52(1), pp. 161–182. [CrossRef]
Copeland, A., Dunn, W., and Hall, G., 2011, “Inventories and the Automobile Market,” RAND J. Econ., 42(1), pp. 121–149. [CrossRef]
Hoyle, C., Chen, W., Wang, N., and Gomez-Levi, G., 2011, “Understanding and Modelling Heterogeneity of Human Preferences for Engineering Design,” ASME J. Eng. Des., 22(8), pp. 583–601. [CrossRef]
Li, S., Liu, Y., and Zhang, J., 2011, “Lose Some, Save Some: Obesity, Automobile Demand, and Gasoline Consumption in the United States,” J. Environ. Econ. Manage., 61(1), pp. 52–66. [CrossRef]
Musti, S., and Kockelman, K. M., 2011, “Evolution of the Household Vehicle Fleet: Anticipating Fleet Composition, PHEV Adoption and GHG Emissions in Austin, Texas,” Transp. Res. Part A Policy Pract., 45(8), pp. 707–720. [CrossRef]
Zhang, T., Gensler, S., and Garcia, R., 2011, “A Study of the Diffusion of Alternative Fuel Vehicles: An Agent-Based Modeling Approach,” J. Prod. Innov. Manag., 28(2), pp. 152–168. [CrossRef]
Axsen, J., Mountain, D. C., and Jaccard, M., 2009, “Combining Stated and Revealed Choice Research to Simulate the Neighbor Effect: The Case of Hybrid-Electric Vehicles,” Resour. Energy Econ., 31(3), pp. 221–238. [CrossRef]
Dagsvik, J. K., and Liu, G., 2009, “A Framework fo Analyzing Rank-Ordered Data With Application to Automobile Demand,” Transp. Res. Part A Gen., 43(1), pp. 1–12. [CrossRef]
Hoyle, C., Chen, W., Ankenman, B., and Wang, N., 2009, “Optimal Experimental Design of Human Appraisals for Modeling Consumer Preferences in Engineering Design,” ASME J. Mech. Des., 131(7), p. 071008. [CrossRef]
Shiau, C.-S. N., and Michalek, J. J., 2009, “Should Designers Worry About Market Systems?,” ASME J. Mech. Des., 131(1), p. 011011. [CrossRef]
Vance, C., and Mehlin, M., 2009, Tax Policy and CO2 Emissions—An Econometric Analysis of the German Automobile Market, Rheinisch-Westfälisches Institut für Wirtschaftsforschung, Ruhr-Universität Bochum, Universität Dortmund, Universität Duisburg-Essen, Bochum, Dortmund, Duisburg, Essen, Germany.
Dasgupta, S., Siddarth, S., and Silva-Risso, J., 2007, “To Lease or to Buy? A Structural Model of a Consumer's Vehicle and Contract Choice Decisions,” J. Mark. Res., 44(3), pp. 490–502. [CrossRef]
Train, K., and Winston, C., 2007, “Vehicle Choice Behavior and the Declining Market Share of U.S. Automakers,” Int. Econ. Rev. (Philadelphia), 48(4), pp. 1469–1497. [CrossRef]
Kemper, L., 2006, “Multilevel Optimization for Enterprise-Driven Decision-Based Product Design,” Decision Making in Engineering, L. Kemper, W. Chen, and L. Schmidt, eds., ASME Press, NY, pp. 203–214.
Berry, S., Levinsohn, J., and Pakes, A., 2004, “Differentiated Products Demand Systems From a Combination of Micro and Macro Data: The New Car Market,” J. Polit. Econ., 112(1), pp. 68–105. [CrossRef]
Greene, D. L., Patterson, P. D., Singh, M., and Li, J., 2005, “Feebates, Rebates and Gas-Guzzler Taxes: A Study of Incentives for Increased Fuel Economy,” Energy Policy, 33(6), pp. 757–775. [CrossRef]
Michalek, J. J., Papalambros, P. Y., and Skerlos, S. J., 2004, “A Study of Fuel Efficiency and Emission Policy Impact on Optimal Vehicle Design Decisions,” ASME J. Mech. Des., 126(6), pp. 1062–1070. [CrossRef]
Petrin, A., 2002, “Quantifying the Benefits of New Products: The Case of the Minivan,” J. Polit. Econ., 110(4), pp. 705–729. [CrossRef]
Sudhir, K., 2001, “Competitive Pricing Behavior in the Auto Market: A Structural Analysis,” Mark. Sci., 20(3), pp. 42–60. [CrossRef]
Berry, S., Levinsohn, J., and Pakes, A., 1999, “Voluntary Export Restraints on Automobiles: Evaluating a Trade Policy,” Am. Econ. Rev., 89(3), pp. 400–430. [CrossRef]
Brownstone, D., and Train, K., 1999, “Forecasting New Product Penetration With Flexible Substitution Patterns,” J. Econ., 89(1–2), pp. 109–129. [CrossRef]
Dagsvik, J. K., Wennemo, T., Wetterwald, D. G., and Aaberge, R., 2002, “Potential Demand for Alternative Fuel Vehicles,” Transp. Res. Part B: Methodol., 36(4), pp. 361–384. [CrossRef]
McCarthy, P. S., 1996, “Market Price and Income Elasticities of New Vehicle Demands,” Rev. Econ. Stat., 78(3), pp. 543–547. [CrossRef]
Berry, S., Levinsohn, J., and Pakes, A., 1995, “Automobile Prices in Market Equilibrium,” Econometrica, 63(4), pp. 841–890. [CrossRef]
Boyd, J. H., and Mellman, R. E., 1980, “The Effect of Fuel Economy Standards on the US Automotive Market: A Hedonic Demand Analysis,” Transp. Res. Part A Gen., 14, pp. 367–378. [CrossRef]
Lave, C., and Train, K., 1979, “A Disaggregate Model of Auto-Type Choice,” Transp. Res. Part A Gen., 13(1), pp. 1–9. [CrossRef]
Feit, E. M., Beltramo, M. A., and Feinberg, F. M., 2010, “Reality Check: Combining Choice Experiments with Market Data to Estimate the Importance of Product Attributes,” Manage. Sci., 56(5), pp. 785–800. [CrossRef]
Bunch, D., Greene, D., Lipman, T., and Shaheen, S., 2011, Potential Design, Implementation, and Benefits of a Feebate Program for New Passenger Vehicles in California, University of California, Davis, Institute of Transportation Studies, Davis, CA.
U.S. Energy Information Administration, 2011, Annual Energy Outlook 2011 With Projections to 2035, U.S. Energy Information Administration, Washington, DC.
Gigerenzer, G., and Brighton, H., 2009, “Homo Heuristicus: Why Biased Minds Make Better Inferences,” Top. Cogn. Sci., 1(1), pp. 107–143. [CrossRef] [PubMed]
Train, K., 1979, “Consumers' Responses to Fuel-Efficient Vehicles: A Critical Review of Econometric Studies,” Transportation (Amsterdam), 8(3), pp. 237–258. [CrossRef]
Chen, C., 2001, “Design for the Environment: A Quality-Based Model for Green Product Development,” Manage. Sci., 47(2), pp. 250–263. [CrossRef]
Shiau, C.-S. N., Michalek, J. J., and Hendrickson, C. T., 2009, “A Structural Analysis of Vehicle Design Responses to Corporate Average Fuel Economy Policy,” Transp. Res. Part A: Policy Pract., 43(9–10), pp. 814–828. [CrossRef]
NevoA., 2000, “A Practitioner's Guide to Estimation of Random-Coefficients Logit Models of Demand,” J. Econ. Manage. Strategies, 9(4), pp. 513–548. [CrossRef]
Dubé, J., Fox, J., and Su, C., 2012, “Improving the Numerical Performance of Static and Dynamic Aggregate Discrete Choice Random Coefficients Demand Estimation,” Econometrica80(5), pp. 2231–2267. [CrossRef]
Skrainka, B., and Judd, K., 2011, “High Performance Quadrature Rules: How Numerical Integration Affects a Popular Model of Product Differentiation,” (published online). Available at: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1870703
Bunch, D. S., Bradley, M., and Golob, T. F., 1993, “Demand for Clean Fuel Vehicles in California A Discrete Choice Stated Preference Pilot Project,” Transp. Res. Part A: Policy Pract., 27A(3), pp. 237–253. [CrossRef]
Brownstone, D., Bunch, D. S., and Train, K., 2000, “Joint Mixed Logit Models of Stated and Revealed Preferences for Alternative-Fuel Vehicles,” Transp. Res. Part B: Methodol., 34(5), pp. 315–338. [CrossRef]
McFadden, D., and Train, K., 2000, “Mixed MNL Models for Discrete Response,” J. Appl. Econ., 15(5), pp. 447–470. [CrossRef]
Selby, B., and Kockelman, K., 2012, “Microsimulating Automobile Markets: Evolution of Vehicle Holdings and Vehicle Pricing Dynamics,” Transp. Res. Forum, 51(2), pp. 83–96.
Ward's Automotive Group, 2008, “Ward's Yearly Vehicle Specifications.” Available at: http://wardsauto.com/data-center
Polk, 2008, “IHS Polk Insight Yearly Volume Data.” Available at: http://blog.polk.com/blog/new-vehicle-sales
U.S. Energy Information Administration, 2013, “U.S. All Grades All Formulations Retail Gasoline Prices” (accessed: Aug. 1, 2013) http://tonto.eia.gov/dnav/pet/hist/LeafHandler.ashx?n=PET&s=EMM_EPM0_PTE_NUS_DPG&f=A
“Annual Average Indexex,” Bur. Labor Statitstics Data Online, (accessed: Aug. 1, 2013), http://www.bls.gov/cpi/cpi_dr.htm#2007
Wooldridge, J. M., 2006, Introductory Econometrics: A Modern Approach, Thomas Higher Education, Mason, OH.
Kullback, S., and Leibler, R. A., 1951, “On Information and Sufficiency,” Ann. Math. Stat., 22(1), pp. 79–86. [CrossRef]
Akaike, H., 1973, “Information Theory and an Extension of the Maximum Likelihood Principle,” Proceedings of the Second International Symposium on Information Theory, B. N.Petrov and F.Csaki, eds., Akademiai Kiado, Budapest, Hungary, pp. 267–281.
Schwarz, G., 1978, “Estimating the Dimension of a Model,” Ann. Stat., 6(2), pp. 461–464. [CrossRef]
Burnham, K. P., and Anderson, D. R., 2002, Model Selection and Multi-Model Inference: A Practical Information-Theoretic Approach, Springer, New York.
Claeskens, G., and Hjort, N., 2008, Model Selection and Model Averaging, Cambridge University Press, NY.
Fiebig, D. G., Keane, M. P., Louviere, J., and Wasi, N., 2009, “The Generalized Multinomial Logit Model: Accounting for Scale and Coefficient Heterogeneity,” Mark. Sci., 29(3), pp. 393–421. [CrossRef]
Heffner, R. R., Kurani, K. S., and Turrentine, T. S., 2007, “Symbolism in California's Early Market for Hybrid Electric Vehicles,” Transp. Res. Part D: Transp. Environ., 12(6), pp. 396–413. [CrossRef]
Axsen, J., and Kurani, K. S., 2011, “Interpersonal Influence in the Early Plug-in Hybrid Market: Observing Social Interactions With an Exploratory Multi-Method Approach,” Transp. Res. Part D: Transp. Environ., 16(2), pp. 150–159. [CrossRef]
Hauser, J., and Wernerfelt, B., 1990, “An Evaluation Cost Model of Consideration Sets,” J. Consum. Res., 16(4), pp. 393–408. [CrossRef]
Gilbride, T. J., and Allenby, G. M., 2004, “A Choice Model With Conjunctive, Disjunctive, and Compensatory Screening Rules,” Mark. Sci., 23(3), pp. 391–406. [CrossRef]
Lucas, R. E., 1976, “Econometric Policy Evaluation: A Critique,” Carnegie-Rochester Conference Series on Public Policy, 1, pp. 19–46. [CrossRef]


Grahic Jump Location
Fig. 1

CDF of error tolerance for the best logit model specifications as measured by likelihood/KL and AIC/BIC measures on 2004–2006 sales estimation data and 2007 sales prediction data compared to alternative models (a) full market and (b) new vehicle designs only





Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In