Thermal management is critical for safety, performance, and durability of lithium-ion batteries that are ubiquitous in consumer electronics, electric vehicles (EVs), aerospace, and grid-scale energy storage. Toward mass adoption of EVs globally, lithium-ion batteries are increasingly used under extreme conditions including low temperatures, high temperatures, and fast charging. Furthermore, EV fires caused by battery thermal runaway have become a major hurdle to the wide adoption of EVs. These extreme conditions pose great challenges for thermal management and require unconventional strategies. The interactions between thermal, electrochemical, materials, and structural characteristics of batteries further complicate the challenges, but they also enable opportunities for developing innovative strategies of thermal management. In this review, the challenges for thermal management under extreme conditions are analyzed. Then, the progress is highlighted in two directions. One direction is improving battery thermal management systems based on the principles of heat transfer, which are generally external to Li-ion cells. The other direction is designing novel battery structures, which are generally internal of Li-ion cells such as smart batteries with embedded sensors and actuators. The latter approach could greatly simplify or even eliminate the need for battery thermal management under extreme conditions. New research integrating these two approaches is recommended.