When design decisions are informed by consumer choice models, uncertainty in choice model predictions creates uncertainty for the designer. We investigate the variation and accuracy of market share predictions by characterizing fit and forecast accuracy of discrete choice models for the US light duty new vehicle market. Specifically, we estimate multinomial logit models for 9000 utility functions representative of a large literature in vehicle choice modeling using sales data for years 2004–2006. Each model predicts shares for the 2007 and 2010 markets, and we compare several quantitative measures of model fit and predictive accuracy. We find that (1) our accuracy measures are concordant: model specifications that perform well on one measure tend to also perform well on other measures for both fit and prediction. (2) Even the best discrete choice models exhibit substantial prediction error, stemming largely from limited model fit due to unobserved attributes. A naïve “static” model, assuming share for each vehicle design in the forecast year = share in the last available year, outperforms all 9000 attribute-based models when predicting the full market one year forward, but attribute-based models can predict better for four year forward forecasts or new vehicle designs. (3) Share predictions are sensitive to the presence of utility covariates but less sensitive to covariate form (e.g., miles per gallons versus gallons per mile), and nested and mixed logit specifications do not produce significantly more accurate forecasts. This suggests ambiguity in identifying a unique model form best for design. Furthermore, the models with best predictions do not necessarily have expected coefficient signs, and biased coefficients could misguide design efforts even when overall prediction accuracy for existing markets is maximized.

References

1.
Hazelrigg
,
G. A.
,
1998
, “
A Framework for Decision-Based Engineering Design
,”
ASME J. Mech. Des.
,
120
(
4
), pp.
653
658
.10.1115/1.2829328
2.
Train
,
K. E.
,
2009
,
Discrete Choice Models With Simulation
,
Cambridge University
,
New York
.
3.
Wassenaar
,
H. J.
, and
Chen
,
W.
,
2003
, “
An Approach to Decision-Based Design With Discrete Choice Analysis for Demand Modeling
,”
ASME J. Mech. Des.
,
125
(
3
), pp.
490
497
.10.1115/1.1587156
4.
Wassenaar
,
H. J.
,
Chen
,
W.
,
Cheng
,
J.
, and
Sudjianto
,
A.
,
2005
, “
Enhancing Discrete Choice Demand Modeling for Decision-Based Design
,”
ASME J. Mech. Des.
,
127
(
4
), pp.
514
523
.10.1115/1.1897408
5.
Besharati
,
B.
,
Luo
,
L.
,
Azarm
,
S.
, and
Kannan
,
P. K.
,
2006
, “
Multi-Objective Single Product Robust Optimization: An Integrated Design and Marketing Approach
,”
ASME J. Mech. Des.
,
128
(
4
), pp.
884
892
.10.1115/1.2202889
6.
Williams
,
N.
,
Azarm
,
S.
, and
Kannan
,
P. K.
,
2008
, “
Engineering Product Design Optimization for Retail Channel Acceptance
,”
ASME J. Mech. Des.
,
130
(
6
), p.
061402
.10.1115/1.2898874
7.
Shiau
,
C.-S. N.
, and
Michalek
,
J. J.
,
2009
, “
Optimal Product Design Under Price Competition
,”
ASME J. Mech. Des.
,
131
(
7
), p.
071003
.10.1115/1.3125886
8.
Frischknecht
,
B. D.
,
Whitefoot
,
K.
, and
Papalambros
,
P. Y.
,
2010
, “
On the Suitability of Econometric Demand Models in Design for Market Systems
,”
ASME J. Mech. Des.
,
132
(
12
), p.
121007
.10.1115/1.4002941
9.
He
,
L.
,
Chen
,
W.
,
Hoyle
,
C.
, and
Yannou
,
B.
,
2012
, “
Choice Modeling for Usage Context-Based Design
,”
ASME J. Mech. Des.
,
134
(
3
), p.
031007
.10.1115/1.4005860
10.
Morrow
,
W. R.
,
Long
,
M.
, and
MacDonald
,
E. F.
,
2012
, “
Models in Decision Based Design Optimization
,”
ASME
DETC2012-71176.10.1115/DETC2012-71176
11.
Whitefoot
,
K. S.
, and
Skerlos
,
S. J.
,
2012
, “
Design Incentives to Increase Vehicle Size Created From the U.S. Footprint-Based Fuel Economy Standards
,”
Energy Policy
,
41
(1), pp.
402
411
.10.1016/j.enpol.2011.10.062
12.
Resende
,
C. B.
,
Grace Heckmann
,
C.
, and
Michalek
,
J. J.
,
2012
, “
Robust Design for Profit Maximization With Aversion to Downside Risk From Parametric Uncertainty in Consumer Choice Models
,”
ASME J. Mech. Des.
,
134
(
10
), p.
100901
.10.1115/1.4007533
13.
Payne
,
J. W.
,
1976
, “
Task Complexity and Contingent Processing in Decision Making: An Information Search
,”
Organ. Behav. Hum. Perform.
,
16
(2)
, pp.
366
387
.10.1016/0030-5073(76)90022-2
14.
MacDonald
,
E. F.
,
Gonzalez
,
R.
, and
Papalambros
,
P. Y.
,
2009
, “
Preference Inconsistency in Multidisciplinary Design Decision Making
,”
ASME J. Mech. Des.
,
131
(
3
), p.
031009
.10.1115/1.3066526
15.
Morrow
,
W. R.
,
Long
,
M.
, and
MacDonald
,
E. F.
,
2014
, “
Market-System Design Optimization With Consider-Then-Choose Models
,”
ASME J. Mech. Des.
,
136
(
3
). p.
031003
.10.1115/1.4026094
16.
Knittel
,
C. R.
, and
Metaxoglou
,
K.
,
2012
, “
Estimation of Random Coefficient Demand Models: Two Empiricists' Perspective
,”
Rev. Econ. Stat.
,
96
(
1
), pp.
34
59
.10.1162/REST_a_00394
17.
Allcott
,
H.
, and
Wozny
,
N.
,
2012
, “
Gasoline Prices, Fuel Economy, and the Energy Paradox
,”
Rev. Econ. Stat.
(to be published).
18.
Dubé
,
J.-P.
,
Fox
,
J. T.
, and
Su
,
C.-L.
,
2012
, “
Improving the Numerical Performance of Static and Dynamic Aggregate Discrete Choice Random Coefficients Demand Estimation
,”
Econometrica
,
80
(
5
), pp.
2231
2267
.10.3982/ECTA8585
19.
Beresteanu
,
A.
, and
Li
,
S.
,
2011
, “
Gasoline Prices, Government Support, and the Demand for Hybrid Vehicles in the United States
,”
Int. Econ. Rev.
(Philadelphia),
52
(
1
), pp.
161
182
.10.1111/j.1468-2354.2010.00623.x
20.
Copeland
,
A.
,
Dunn
,
W.
, and
Hall
,
G.
,
2011
, “
Inventories and the Automobile Market
,”
RAND J. Econ.
,
42
(
1
), pp.
121
149
.10.1111/j.1756-2171.2010.00128.x
21.
Hoyle
,
C.
,
Chen
,
W.
,
Wang
,
N.
, and
Gomez-Levi
,
G.
,
2011
, “
Understanding and Modelling Heterogeneity of Human Preferences for Engineering Design
,”
ASME J. Eng. Des.
,
22
(
8
), pp.
583
601
.10.1080/09544821003604496
22.
Li
,
S.
,
Liu
,
Y.
, and
Zhang
,
J.
,
2011
, “
Lose Some, Save Some: Obesity, Automobile Demand, and Gasoline Consumption in the United States
,”
J. Environ. Econ. Manage.
,
61
(
1
), pp.
52
66
.10.1016/j.jeem.2010.10.001
23.
Musti
,
S.
, and
Kockelman
,
K. M.
,
2011
, “
Evolution of the Household Vehicle Fleet: Anticipating Fleet Composition, PHEV Adoption and GHG Emissions in Austin, Texas
,”
Transp. Res. Part A Policy Pract.
,
45
(
8
), pp.
707
720
.10.1016/j.tra.2011.04.011
24.
Zhang
,
T.
,
Gensler
,
S.
, and
Garcia
,
R.
,
2011
, “
A Study of the Diffusion of Alternative Fuel Vehicles: An Agent-Based Modeling Approach
,”
J. Prod. Innov. Manag.
,
28
(
2
), pp.
152
168
.10.1111/j.1540-5885.2011.00789.x
25.
Axsen
,
J.
,
Mountain
,
D. C.
, and
Jaccard
,
M.
,
2009
, “
Combining Stated and Revealed Choice Research to Simulate the Neighbor Effect: The Case of Hybrid-Electric Vehicles
,”
Resour. Energy Econ.
,
31
(
3
), pp.
221
238
.10.1016/j.reseneeco.2009.02.001
26.
Dagsvik
,
J. K.
, and
Liu
,
G.
,
2009
, “
A Framework fo Analyzing Rank-Ordered Data With Application to Automobile Demand
,”
Transp. Res. Part A Gen.
,
43
(1), pp.
1
12
.10.1016/j.tra.2008.06.005
27.
Hoyle
,
C.
,
Chen
,
W.
,
Ankenman
,
B.
, and
Wang
,
N.
,
2009
, “
Optimal Experimental Design of Human Appraisals for Modeling Consumer Preferences in Engineering Design
,”
ASME J. Mech. Des.
,
131
(
7
), p.
071008
.10.1115/1.3149845
28.
Shiau
,
C.-S. N.
, and
Michalek
,
J. J.
,
2009
, “
Should Designers Worry About Market Systems?
,”
ASME J. Mech. Des.
,
131
(
1
), p.
011011
.10.1115/1.3013848
29.
Vance
,
C.
, and
Mehlin
,
M.
,
2009
,
Tax Policy and CO2 Emissions—An Econometric Analysis of the German Automobile Market
, Rheinisch-Westfälisches Institut für Wirtschaftsforschung, Ruhr-Universität Bochum, Universität Dortmund, Universität Duisburg-Essen,
Bochum, Dortmund, Duisburg, Essen, Germany
.
30.
Dasgupta
,
S.
,
Siddarth
,
S.
, and
Silva-Risso
,
J.
,
2007
, “
To Lease or to Buy? A Structural Model of a Consumer's Vehicle and Contract Choice Decisions
,”
J. Mark. Res.
,
44
(3), pp. 490–502.10.1509/jmkr.44.3.490
31.
Train
,
K.
, and
Winston
,
C.
,
2007
, “
Vehicle Choice Behavior and the Declining Market Share of U.S. Automakers
,”
Int. Econ. Rev. (Philadelphia)
,
48
(
4
), pp.
1469
1497
.10.1111/j.1468-2354.2007.00471.x
32.
Kemper
,
L.
,
2006
, “
Multilevel Optimization for Enterprise-Driven Decision-Based Product Design
,” Decision Making in Engineering, L. Kemper, W. Chen, and L. Schmidt, eds., ASME Press, NY, pp.
203
214
.
33.
Berry
,
S.
,
Levinsohn
,
J.
, and
Pakes
,
A.
,
2004
, “
Differentiated Products Demand Systems From a Combination of Micro and Macro Data: The New Car Market
,”
J. Polit. Econ.
,
112
(
1
), pp.
68
105
.10.1086/379939
34.
Greene
,
D. L.
,
Patterson
,
P. D.
,
Singh
,
M.
, and
Li
,
J.
,
2005
, “
Feebates, Rebates and Gas-Guzzler Taxes: A Study of Incentives for Increased Fuel Economy
,”
Energy Policy
,
33
(
6
), pp.
757
775
.10.1016/j.enpol.2003.10.003
35.
Michalek
,
J. J.
,
Papalambros
,
P. Y.
, and
Skerlos
,
S. J.
,
2004
, “
A Study of Fuel Efficiency and Emission Policy Impact on Optimal Vehicle Design Decisions
,”
ASME J. Mech. Des.
,
126
(
6
), pp.
1062
1070
.10.1115/1.1804195
36.
Petrin
,
A.
,
2002
, “
Quantifying the Benefits of New Products: The Case of the Minivan
,”
J. Polit. Econ.
,
110
(
4
), pp.
705
729
.10.1086/340779
37.
Sudhir
,
K.
,
2001
, “
Competitive Pricing Behavior in the Auto Market: A Structural Analysis
,”
Mark. Sci.
,
20
(
3
), pp.
42
60
.10.1287/mksc.20.1.42.10196
38.
Berry
,
S.
,
Levinsohn
,
J.
, and
Pakes
,
A.
,
1999
, “
Voluntary Export Restraints on Automobiles: Evaluating a Trade Policy
,”
Am. Econ. Rev.
,
89
(
3
), pp.
400
430
.10.1257/aer.89.3.400
39.
Brownstone
,
D.
, and
Train
,
K.
,
1999
, “
Forecasting New Product Penetration With Flexible Substitution Patterns
,”
J. Econ.
,
89
(1–2)
, pp.
109
129
.10.1016/S0304-4076(98)00057-8
40.
Dagsvik
,
J. K.
,
Wennemo
,
T.
,
Wetterwald
,
D. G.
, and
Aaberge
,
R.
,
2002
, “Potential Demand for Alternative Fuel Vehicles,”
Transp. Res. Part B: Methodol.
,
36
(4), pp. 361–384.10.1016/S0965-8564(01)00013-1
41.
McCarthy
,
P. S.
,
1996
, “
Market Price and Income Elasticities of New Vehicle Demands
,”
Rev. Econ. Stat.
,
78
(
3
), pp.
543
547
.10.2307/2109802
42.
Berry
,
S.
,
Levinsohn
,
J.
, and
Pakes
,
A.
,
1995
, “
Automobile Prices in Market Equilibrium
,”
Econometrica
,
63
(
4
), pp.
841
890
.10.2307/2171802
43.
Boyd
,
J. H.
, and
Mellman
,
R. E.
,
1980
, “
The Effect of Fuel Economy Standards on the US Automotive Market: A Hedonic Demand Analysis
,”
Transp. Res. Part A Gen.
,
14
, pp.
367
378
.10.1016/0191-2607(80)90055-2
44.
Lave
,
C.
, and
Train
,
K.
,
1979
, “
A Disaggregate Model of Auto-Type Choice
,”
Transp. Res. Part A Gen.
,
13
(
1
), pp.
1
9
.10.1016/0191-2607(79)90081-5
45.
Feit
,
E. M.
,
Beltramo
,
M. A.
, and
Feinberg
,
F. M.
,
2010
, “
Reality Check: Combining Choice Experiments with Market Data to Estimate the Importance of Product Attributes
,”
Manage. Sci.
,
56
(
5
), pp.
785
800
.10.1287/mnsc.1090.1136
46.
Bunch
,
D.
,
Greene
,
D.
,
Lipman
,
T.
, and
Shaheen
,
S.
,
2011
, Potential Design, Implementation, and Benefits of a Feebate Program for New Passenger Vehicles in California, University of California, Davis, Institute of Transportation Studies, Davis, CA.
47.
U.S. Energy Information Administration
,
2011
,
Annual Energy Outlook 2011 With Projections to 2035
,
U.S. Energy Information Administration
,
Washington, DC
.
48.
Gigerenzer
,
G.
, and
Brighton
,
H.
,
2009
, “
Homo Heuristicus: Why Biased Minds Make Better Inferences
,”
Top. Cogn. Sci.
,
1
(
1
), pp.
107
143
.10.1111/j.1756-8765.2008.01006.x
49.
Train
,
K.
,
1979
, “
Consumers' Responses to Fuel-Efficient Vehicles: A Critical Review of Econometric Studies
,”
Transportation
(Amsterdam),
8
(3), pp.
237
258
.10.1007/BF00169990
50.
Chen
,
C.
,
2001
, “
Design for the Environment: A Quality-Based Model for Green Product Development
,”
Manage. Sci.
,
47
(
2
), pp.
250
263
.10.1287/mnsc.47.2.250.9841
51.
Shiau
,
C.-S. N.
,
Michalek
,
J. J.
, and
Hendrickson
,
C. T.
,
2009
, “
A Structural Analysis of Vehicle Design Responses to Corporate Average Fuel Economy Policy
,”
Transp. Res. Part A: Policy Pract.
,
43
(
9–10
), pp.
814
828
.10.1016/j.tra.2009.08.002
52.
Nevo
A.
,
2000
, “
A Practitioner's Guide to Estimation of Random-Coefficients Logit Models of Demand
,”
J. Econ. Manage. Strategies
,
9
(
4
), pp.
513
548
.10.1111/j.1430-9134.2000.00513.x
53.
Dubé
,
J.
,
Fox
,
J.
, and
Su
,
C.
,
2012
, “
Improving the Numerical Performance of Static and Dynamic Aggregate Discrete Choice Random Coefficients Demand Estimation
,”
Econometrica
80
(5), pp.
2231
2267
.10.3982/ECTA8585
54.
Skrainka
,
B.
, and
Judd
,
K.
,
2011
, “
High Performance Quadrature Rules: How Numerical Integration Affects a Popular Model of Product Differentiation
,” (published online). Available at: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1870703
55.
Bunch
,
D. S.
,
Bradley
,
M.
, and
Golob
,
T. F.
,
1993
, “
Demand for Clean Fuel Vehicles in California A Discrete Choice Stated Preference Pilot Project
,”
Transp. Res. Part A: Policy Pract.
,
27A
(
3
), pp.
237
253
.10.1016/0965-8564(93)90062-P
56.
Brownstone
,
D.
,
Bunch
,
D. S.
, and
Train
,
K.
,
2000
, “
Joint Mixed Logit Models of Stated and Revealed Preferences for Alternative-Fuel Vehicles
,”
Transp. Res. Part B: Methodol.
,
34
(
5
), pp.
315
338
.10.1016/S0191-2615(99)00031-4
57.
McFadden
,
D.
, and
Train
,
K.
,
2000
, “
Mixed MNL Models for Discrete Response
,”
J. Appl. Econ.
,
15
(5)
, pp.
447
470
.10.1002/1099-1255(200009/10)15:5<447::AID-JAE570>3.0.CO;2-1
58.
Selby
,
B.
, and
Kockelman
,
K.
,
2012
, “
Microsimulating Automobile Markets: Evolution of Vehicle Holdings and Vehicle Pricing Dynamics
,”
Transp. Res. Forum
,
51
(
2
), pp.
83
96
.
59.
Ward's Automotive Group
,
2008
, “
Ward's Yearly Vehicle Specifications
.” Available at: http://wardsauto.com/data-center
60.
Polk
,
2008
, “
IHS Polk Insight Yearly Volume Data
.” Available at: http://blog.polk.com/blog/new-vehicle-sales
61.
U.S. Energy Information Administration
,
2013
, “
U.S. All Grades All Formulations Retail Gasoline Prices
” (accessed: Aug. 1,
2013
) http://tonto.eia.gov/dnav/pet/hist/LeafHandler.ashx?n=PET&s=EMM_EPM0_PTE_NUS_DPG&f=A
62.
Annual Average Indexex
,” Bur. Labor Statitstics Data Online, (accessed: Aug. 1,
2013
), http://www.bls.gov/cpi/cpi_dr.htm#2007
63.
Wooldridge
,
J. M.
,
2006
,
Introductory Econometrics: A Modern Approach
,
Thomas Higher Education
,
Mason, OH
.
64.
Kullback
,
S.
, and
Leibler
,
R. A.
,
1951
, “
On Information and Sufficiency
,”
Ann. Math. Stat.
,
22
(
1
), pp.
79
86
.10.1214/aoms/1177729694
65.
Akaike
,
H.
,
1973
, “
Information Theory and an Extension of the Maximum Likelihood Principle
,”
Proceedings of the Second International Symposium on Information Theory
,
B. N.
Petrov
and
F.
Csaki
, eds.,
Akademiai Kiado, Budapest, Hungary
, pp.
267
281
.
66.
Schwarz
,
G.
,
1978
, “
Estimating the Dimension of a Model
,”
Ann. Stat.
,
6
(
2
), pp.
461
464
.10.1214/aos/1176344136
67.
Burnham
,
K. P.
, and
Anderson
,
D. R.
,
2002
,
Model Selection and Multi-Model Inference: A Practical Information-Theoretic Approach
,
Springer
,
New York
.
68.
Claeskens
,
G.
, and
Hjort
,
N.
,
2008
, Model Selection and Model Averaging, Cambridge University Press, NY.
69.
Fiebig
,
D. G.
,
Keane
,
M. P.
,
Louviere
,
J.
, and
Wasi
,
N.
,
2009
, “
The Generalized Multinomial Logit Model: Accounting for Scale and Coefficient Heterogeneity
,”
Mark. Sci.
,
29
(
3
), pp.
393
421
.10.1287/mksc.1090.0508
70.
Heffner
,
R. R.
,
Kurani
,
K. S.
, and
Turrentine
,
T. S.
,
2007
, “
Symbolism in California's Early Market for Hybrid Electric Vehicles
,”
Transp. Res. Part D: Transp. Environ.
,
12
(
6
), pp.
396
413
.10.1016/j.trd.2007.04.003
71.
Axsen
,
J.
, and
Kurani
,
K. S.
,
2011
, “
Interpersonal Influence in the Early Plug-in Hybrid Market: Observing Social Interactions With an Exploratory Multi-Method Approach
,”
Transp. Res. Part D: Transp. Environ.
,
16
(
2
), pp.
150
159
.10.1016/j.trd.2010.10.006
72.
Hauser
,
J.
, and
Wernerfelt
,
B.
,
1990
, “
An Evaluation Cost Model of Consideration Sets
,”
J. Consum. Res.
,
16
(
4
), pp.
393
408
.10.1086/209225
73.
Gilbride
,
T. J.
, and
Allenby
,
G. M.
,
2004
, “
A Choice Model With Conjunctive, Disjunctive, and Compensatory Screening Rules
,”
Mark. Sci.
,
23
(
3
), pp.
391
406
.10.1287/mksc.1030.0032
74.
Lucas
,
R. E.
,
1976
, “
Econometric Policy Evaluation: A Critique
,”
Carnegie-Rochester Conference Series on Public Policy
,
1
, pp.
19
46
.10.1016/S0167-2231(76)80003-6
You do not currently have access to this content.