Research Papers

Applied Tests of Design Skills—Part II: Visual Thinking

[+] Author and Article Information
Jami J. Shah

Mechanical and Aerospace Engineering,
Arizona State University,
Tempe, AZ 85258

Jay Woodward

Department of Educational Psychology,
Texas A&M University,
College Station, TX 77843

Steven M. Smith

Department of Psychology,
Texas A&M University,
College Station, TX 77843

Contributed by the Design Education Committee of ASME for publication in the Journal of Mechanical Design. Manuscript received May 9, 2012; final manuscript received April 10, 2013; published online xx xx, xxxx. Assoc. Editor: Janis Terpenny.

J. Mech. Des 135(7), 071004 (May 24, 2013) (11 pages) Paper No: MD-12-1246; doi: 10.1115/1.4024228 History: Received May 09, 2012; Revised April 10, 2013

A number of cognitive skills relevant to conceptual design have been previously identified: divergent thinking, visual thinking, spatial reasoning, qualitative reasoning, and problem formulation. A battery of standardized test has been developed for each of these skills. This is the second paper in a series of papers on testing individual skill level differences in engineers and engineering students. In the first paper, we reported on the theoretical and empirical basis for divergent thinking test, as well as, on test formulation, data collection, norming studies, and statistical validation of that test. This paper focuses similarly on the efforts related to the visual thinking and spatial reasoning in engineering context. We have decomposed visual thinking into six categories: visual comprehension including perceptual speed, visual memory (that is, the visual memory system), visual synthesis mental image manipulation/transformation, spatial reasoning, and graphical expression/elaboration. We discuss the theoretical basis of a comprehensive test for engineers, test composition, trial runs, and computation of reliability measures. The alpha version was given to a small set of subjects to determine clarity of the questions and gauge difficulty level. The beta version was used for norming and test validation from over 500 samples that included engineering students and a smaller number of practicing engineers. Construct validation was achieved through basing the construction of our instrument off other well-known measures of visual thinking, while content validity was assured through thoroughly sampling the domain of visual thinking and including a variety of items both pertinent and specific to the engineering design process. The factor analysis reveals that there are possibly two eigenvalues above 1.0, an indication that it is a stable and accurate instrument. We emphasize that these tests are not just dependent on native abilities, but on education and experience; design skills are teachable and learnable.

Copyright © 2013 by ASME
Your Session has timed out. Please sign back in to continue.


Shah, J., 2005, “Identification, Measurement & Development of Design Skills in Engineering Education,” International Conference on Engineering Design (ICED05), Melbourne, Australia, Aug.
Khorshidi, M., Woodward, J., and Shah, J., 2012, “Towards a Comprehensive Test of Qualitative Reasoning Skill in Design,” ASME Design Theory and Methodology Conference, DETC2012-70657, Chicago.
Shah, J., Millsap, R., Woodward, J., and Smith, S. M., 2012, “Applied Tests of Design Skills—Part I: Divergent Thinking,” ASME J. Mech. Des., V134, p. 021005. [CrossRef]
Anderson, J. R., 2005, Cognitive Psychology and Its Implications, Worth Publishers, New York.
Matlin, M. W., 2009, Cognition, 7th ed., John Wiley & Sons, Inc., Hoboken, NJ.
Schacter, D. L., Gilbert, D. T., and Wegner, D. M., 2009, Psychology, Worth Publishers, New York, NY.
Finke, R. A., 1996, “Imagery, Creativity, and Emergent Structure,” Conscious Cogn., 5, pp. 381–393. [CrossRef] [PubMed]
Finke, R. A., 1985, Theories Relating Mental Imagery to Perception,” Psychol. Bull., 98, pp. 236–259. [CrossRef] [PubMed]
Finke, R. A., Pinker, S., and Farah, M., 1989, “Reinterpreting Visual Patterns in Mental Imagery,” Cogn. Sci., 13, pp. 51–78. [CrossRef]
Ester, E. F., Serences, J. T., and Awh, E., 2009, “Spatially Global Representations in Human Primary Visual Cortex During Working Memory Maintenance,” J. Neurosci., 29(48), pp. 15258–15265. [CrossRef] [PubMed]
Kosslyn, S. M., Thompson, W. L., Gitelman, D. R., and Alpert, N. M., 1998, “Neural Systems That Encode Categorical vs. Coordinate Spatial Relations: PET Investigations,” Psychobiology, 26, pp. 333–347.
Baddeley, A. D., and Wilson, B. A., 2002, “Prose Recall and Amnesia: Implications for the Structure of Working Memory,” Neuropsychologia, 40, pp. 1737–1743. [CrossRef] [PubMed]
Brown, H. D., Kosslyn, S. M., and Dror, I. E., 1998, “Aging and Scanning of Imagined and Perceived Visual Images,” Exp. Aging Res., 24, pp. 181–194. [CrossRef] [PubMed]
Paivio, A., and Csapo, K., 1973, “Picture Superiority in Free Recall: Imagery or Dual Coding?,” Cogn. Psychol., 5(2), pp. 176–206. [CrossRef]
Unsworth, N., and Engle, R. W., 2008, “Speed and Accuracy of Accessing Information in Working Memory: An Individual Differences Investigation of Focus Switching,” J. Exp. Psychol.: Learn., Mem., Cognit., 34(3), pp. 616–630. [CrossRef]
Baddeley, A. D., 2007, Working Memory, Thought, and Action, Oxford University, Oxford, UK.
Brady, T. F., Konkle, T., and Alvarez, G. A., 2009, “Compression in Visual Working Memory: Using Statistical Regularities to Form More Efficient Memory Representations,” J. Exp. Psychol. Gen., 138(4), pp. 487–502. [CrossRef] [PubMed]
Shepard, R. N., and Metzler, J., 1971, “Mental Rotation of Three-Dimensional Objects,” Science, 171, pp. 701–703. [CrossRef] [PubMed]
Cooper, L., and Lang, J. M., 1996, “Imagery and Visual–Spatial Representations. Imagery and Visual–Spatial Representations,” Memory: Handbook of Perception and Cognition, E. L.Bjork and R. A.Bjork, eds., Academic Press, London, UK.
Howes, M. B., 2007, Human Memory: Structures and Images, Sage Publications, Inc., Thousand Oaks, CA.
De Beni, R., Pazzaglia, F., and Gardini, S., 2006, “The Role of Mental Rotation and Age in Spatial Perspective-Taking Tasks,” Appl. Cogn. Psychol., 20(6), pp. 807–821. [CrossRef]
Dror, I. E., and Kosslyn, S. M., 1994, “Mental Imagery and Aging,” Psychol. Aging, 9(1), pp. 90–102. [CrossRef]
Paivio, A., 1971, Imagery and Verbal Processes, Holt, Rinehart, and Winston, New York.
Bower, G. H., 1970, “Analysis of the Mnemonic Device,” Am. Sci., 58, pp. 496–501.
Bellezza, F. S., 1981, “Mnemonic Devices: Classification, Characteristics, and Criteria,” Rev. Educ. Res., 51(2), pp. 247–275. [CrossRef]
Chambers, D., and Reisberg, D., 1985, “Can Mental Images be Ambiguous?,” J. Exp. Psychol. Hum. Percept. Perform., 11(3), pp. 317–328. [CrossRef]
Reed, S., 1974, “Structural Descriptions and the Limitations of Visual Images,” Mem. Cognit., V2(2), pp. 329–336.
Tversky, B., and Hard, B. M., 2009, “Embodied and Disembodied Cognition: Spatial Perspective-Taking,” Cognition, 110, pp. 124–129. [CrossRef] [PubMed]
Tversky, B., 2005, “Functional Significance of Visuospatial Representations,” Handbook of Higher-Level Visuospatial Thinking, P.Shah and A.Miyake, eds., Cambridge University Press, New York, pp. 1–34.
Goel, V., 1995, Sketches of Thought, MIT, Cambridge, MA.
Goldschmidt, G., 1994, “On Visual Design Thinking: The Vis Kids of Architecture,” Des. Stud., 15(2), pp. 158–174. [CrossRef]
McKim, R., 1980, Experiences in Visual Thinking, Wadsworth, Inc., Belmont, CA.
Ullman, D., Wood, S., and Craig, D., 1990, “The Importance of Drawing in the Mechanical Design Process,” Comput. Graphics, V14(2), pp. 263–274. [CrossRef]
Goldschmidt, G., 1992, “Serial Sketching: Visual Problem Solving in Designing,” Cybern. Syst., 23(2), pp. 191–219. [CrossRef]
Larkin, J., and Simon, H., 1987, “Why a Diagram is (Sometimes) Worth Ten Thousand Words,” Cogn. Sci., 11, pp. 65–99. [CrossRef]
Goldschmidt, G., 2000, “The Backtalk of Self-Generated Sketches,” Des. Issues, 19(1), pp. 72–88. [CrossRef]
Suwa, M., and Tversky, B., 2001, “Constructive Perception in Design,” Computational and Cognitive Models of Creative Design, J. S.Gero and M. L.Maher, eds., University of Sydney, Sydney, Australia.
Beittel, K., 1972, Mind and Context in the Art of Drawing: An Empirical and Speculative Account, Holt Rinehart and Winston, NY.
McKoy, F., Vargas-Hernandez, N., Summers, J., and Shah, J., 2001, “Influence of Design Representation on Effectiveness of Idea Generation,” ASME DTM Conference, Pittsburgh, PA, Sept. 10–13.
Tversky, B., Agrawala, M., Heiser, J., Lee, P., Hanrahan, P., Phan, D., Stolte, C., and Daniele, M.-P., 2007, “Cognitive Design Principles for Generating Visualizations,” Applied Spatial Cognition: From Research to Cognitive Technology, G.Allen, ed., Erlbaum, Mahwah, NJ.
McGown, A., Green, G., and Rodgers, P., 1998, “Visible Ideas: Information Patterns of Conceptual Sketch Activity,” Des. Stud., 19(4), pp. 431–453. [CrossRef]
Verstijnen, I. M., Van Leeuwen, C., Goldschmidt, G., Hamel, R., and Hennessey, J. M., 1998, “Sketching and Creative Discovery,” Des. Stud., 19(4), pp. 519–546. [CrossRef]
Hegarty, M., and Kozhevnikov, M., 1999, “Types of Visual–Spatial Representations and Mathematical Problem Solving,” Educ. Psychol., 91, pp. 684–689. [CrossRef]
Kozhevnikov, M., Hegarty, M., and Mayer, R., 2002, “Revising the Visualizer-Verbalizer Dimension: Evidence for Two Types of Visualizers,” Cogn. Instruct., 20(1), pp. 47–77. [CrossRef]
Kozhevnikov, M., Hegarty, M., and Motes, R., 2007, “Spatial Visualization in Physics Problem Solving,” Cogn. Sci., V31, pp. 549–579. [CrossRef]
Presmeg, N., 1986, “Visualization in High School Mathematics,” Learn. Math., 63(3), pp. 42–46.
Reynolds, C. R., Pearson, N. A., and Voress, J. K., 2002, Developmental Test of Visual Perception—Adolescent and Adult, Pro-Ed, Austin, TX.
College Entrance Examination Board, 1939, CEEB Special Aptitude Test in Spatial Relations, New York.
Fay, E., and Quaiser-Pohl, C., 1999, Schnitte—Ein Test zur Erfassung des räumlichen Vorstellungsvermögens, Swets Test Services, Frankfurt, Germany.
Rey, A., 1941, “L'examen Psychologique dans les cas d'encephalopathie Traumatique,” Arch. de Psychol., 28, pp. 215–285.
Ekstrom, R. B., French, J. W., Harman, H. H., and Dermen, D., 1976, Kit of Factor-Referenced Cognitive Tests, Educational Testing Service, Princeton, NH.
Vandenberg, S. G., and Kuse, A. R., 1978, “Mental Rotation, a Group Test of Three-Dimensional Spatial Visualization,” Percept. Mot. Skills, 47, pp. 599–604. [CrossRef] [PubMed]
Guay, R. B., 1980, Purdue Spatial Visualization Test, Purdue Research Foundation, West Lafayette, IN.
Just, M. A., and Carpenter, P. A., 1985, “Cognitive Coordinate Systems: Accounts of Mental Rotation and Individual Differences in Spatial Ability,” Psychol. Rev., 92, pp. 137–172. [CrossRef] [PubMed]
Seddon, G. M., Eniaiyeju, P. A., and Chia, L. H. L., 1985, “The Factor Structure for Mental Rotations of Three-Dimensional Structures Represented in Diagrams,” Res. Sci. Technol. Educ., 3(1), pp. 29–42. [CrossRef]
Shah, J., Woodward, J., and Smith, S., 2011, “Applied Tests of Engineering Design Skills: Visual Thinking Characterization, Test Development and Validation,” ICED 2011, Copenhagen, Denmark.


Grahic Jump Location
Fig. 1

Illustrative examples for M4 and M5

Grahic Jump Location
Fig. 2

Example scoring basis and weights (Module M2)

Grahic Jump Location
Fig. 3

Example responses and VC scoring for Module 9

Grahic Jump Location
Fig. 4

Score ranges and norms




Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In