I would imagine people coming to Japan from China would have exposed that flaw
Also, I can vouch for Taiwanese speaking Japanese pretty well. My grandmother-in-law's Japanese is pretty durn good, considering she probably hasn't spoken it since WWII. She lives in Taiwan. That's how we communicate, since I don't speak Mandarin or Taiwanese. She learned it while Japan occupied Taiwan when she was a girl.