I would say that 99% of the time, it`s obvious when a non-native speaker is speaking Japanese. When there is someone who truly has no accent I am very impressed.
However, accent does not equal unintelligibility. Even with an accent, unless you are completely ignoring any of the rules of pronunciation, most everyone is going to understand you. But, yes, they`re going to know without looking that you`re not Japanese.
As far as I know, I don`t have an accent in Japanese. I can talk to people on the phone and over the intercom, then have them do a double take when we meet face to face... Occasionally even as far as asking in poor English for me to call the Japanese person they were just speaking to. (Happens quite a lot with utilities, deliveries, etc.) People tell me that I sound perfectly normal, but I consider direct proof with phone and intercom a lot more trustworthy.
I think a lot of it has to do with your age when you acquire the language more than actual study method or language skill. Obviously, that won`t HURT, but there are very skilled non-native Japanese speakers who have been in Japan a very very long time and still have a strong accent.
Personally, I would be most concerned about learning the basic rules of pronunciation and trying to imitate as closely as possible than worry deeply about accent. Accept it as a given, and if you do acquire a very natural accent consider yourself very lucky.