Fretting is a damaging phenomenon, generally observed when a mating pair is subjected to a small amplitude of oscillatory motion. The contact behavior in fretting is governed by a complex interaction between mechanical properties of mating pair, contact geometry, and loading conditions. In most of the practical applications, dissimilar materials are chosen for a contacting pair with one of the materials having superior material properties than other so as to replace the worn-out or unfit component during the maintenance. In the literature, many researchers have studied the effect of dissimilar materials on fretting behavior but mainly in the context of hardness. As experimental methodology has been adopted in these studies, the effect of dissimilar material properties has been reported in terms of global variables like wear volume or fretting fatigue life, but its influence on underlying local contact tractions could not be studied. In the present work, a two-dimensional finite element analysis has been carried out for a cylinder-on-plate configuration. The effect of dissimilar materials for the mating pair has been studied by modeling elastic–plastic behavior for combinations of three different materials, namely, SS 304, ASTM A302-B, and aluminum. The validation of the finite element model is carried out by comparing the results of elastic analysis with the analytical solutions available in the literature. The pertinent contact parameters in the context of fretting wear, namely, contact pressure, contact slip, and contact stresses are extracted. A frictional dissipation energy density-based approach is used for the qualitative comparison of the fretting damage for different cases and validated with the literature data.