The data-driven approach is emerging as a promising method for the topological design of the multiscale structure with greater efficiency. However, existing data-driven methods mostly focus on a single class of unit cells without considering multiple classes to accommodate spatially varying desired properties. The key challenge is the lack of inherent ordering or “distance” measure between different classes of unit cells in meeting a range of properties. To overcome this hurdle, we extend the newly developed latent-variable Gaussian process (LVGP) to creating multi-response LVGP (MRLVGP) for the unit cell libraries of metamaterials, taking both qualitative unit cell concepts and quantitative unit cell design variables as mixed-variable inputs. The MRLVGP embeds the mixed variables into a continuous design space based on their collective effect on the responses, providing substantial insights into the interplay between different geometrical classes and unit cell materials. With this model, we can easily obtain a continuous and differentiable transition between different unit cell concepts that can render gradient information for multiscale topology optimization. While the proposed approach has a broader impact on the concurrent topological and material design of engineered systems, we demonstrate its benefits through multiscale topology optimization with aperiodic unit cells. Design examples reveal that considering multiple unit cell types can lead to improved performance due to the consistent load-transferred paths for micro- and macrostructures.