I was there during 1994/95 on the JET programme in Tsu in Mie-ken (about 50 miles south of Nagoya). Tsu = pop. 100,000 university town and prefectural capital of Mie. It had a fair number of 'ex-pats' and was a nice mix between city and country. I only stayed a year because:
1) They asked me if I wanted to do a second year after 3 months when the novelty was starting to wear off, the weather was at its coldest and I was getting a bit pissed off - if they had asked me towards the end of the year I might well have stayed longer, but by then they had already filled my post.
2) Language teaching (especially in my post the JET programme) was easy enough, a good lifestyle and paid well, but I decided that I need to get back to the UK to do some more studying and move into something other than TEFL. I didn't see much of a future in Japan for myself unless I really learnt Japanese up to a very good level or wanted to carry on teaching.
I have some very fond memories of people and places in Japan and would recommend working there for a year to anyone. If you get a choice of where to go then a smaller town/city is a nice compromise between being way out in the countryside (which can be isolating although good if you really want to totally immerse yourself and learn the language fast) and one of the big mega-cities which can be impersonal, expensive and where people are less interested in gaijin. I actually think that the Japanese countryside, the chilled-out side and the more traditional aspects of the culture (as opposed to the plastic/trash/pop sides) are the most worthwhile and rewarding, and these can be more accessible in smaller places and places where you can get to know people better. Having said that, it was always really fun to take a trip into Nagoya, Kyoto or Osaka for shopping (eg english language books), 'proper' western or indian food and for the nightclub/bar stuff.