Beitou, Taiwan is just north of Taipei, about 1 hour in the MRT from downtown (TWD 35, or just RM3.50). This area is known for its hot springs. Even the air stinks of sulphur sometimes.
You can opt to go visit the outdoor hot springs, which is normally full of local people, or you could enjoy one of the many spas at the area.
Most hotels here also provide hot spring water directly into your bathtub!
So, if you're too lazy to climb up the hill to an outdoor hot spring, or don't want to face the crowds there, you can still relax and enjoy the springs right in your own hotel room.