Yes, it should take about 25 minutes. Then you have a 5 minute walk along a very "colourful" stretch of Yonge St. Entirely safe, but, uh, shall we say, I guess seedy comes to mind. Intermingled with large stores like Guess and HMV, but definitely seedy.
You can point this out to your friends back home when they start in about how "clean" Toronto is!