[samokan]

if you are taking the Shinkansen

Tokyo - Osaka : 2 hours (more or less ) ( shinkansen)
Osaka - Kyoto : 30 -40 minutes ( not shinkansen )
Osaka - Kobe : 20 -30 minutes ( not shinkansen)
Osaka - Beppu : 4 hours (shinkansen)

you don't need to go to Beppu to experience a real onsen , "Arima Onsen" in Kobe is considered one of the oldest Onsen in Japan. It's just 40 minutes away from Kobe. Just google Arima Onsen.

Another is the Shirahama Onsen in Wakayama. just a an hour or so from Osaka.

But if you still have some time, just go to beppu .. it is I think an interesting place to visit...