You could be thinking of Asakusa, not Akihabara. I can guarantee Asakusa'd be heaving, too.
I say go for Meiji-Jingu anyway. It'll be heaving, but it'll be THE place to be, and if you don't live in Tokyo it could be a once-in-a-lifetime experience. I've no love of crowds (in fact I'm mildly demophobic), and even I'd give it a shot
