Maybe it's because I'm a small town guy but I just go up to people and start talking to them. Hahaha!
One time, I was on the Yamanote-sen when a girl sat down beside me and pulled out an English textbook. Of course I took the opportunity to speak to her in English and she was pleasantly surprised. We talked for a bit, exchanged cell phone #'s and met a few times to have an English conversation over a coffee or beer.
I know that city people - especially in Tokyo aren't exactly receptive to having strangers approach them. However, if you have something in common... especially the English language, they'll be willing to put their guard down for a bit. After that, it's up to your charm and humor to win them over.
And no, I'm not into nanpa. In fact, I talked to a businessman at a Starbucks in Japan and we had a really good conversation. So much that he invited me to his favorite kyabakura (hostess bar) for some drinks afterwards. But that's a different story...
