A foreign resident who paid into the pension for 6 months or more and leaves Japan for good can ask for part of it back, within 2 years after leaving.
When you leave Japan for good.
Apply after you leave. About 20% tax is withheld from the 厚生年金 part, which you can get back by filing a return. There is a limit on the years counted, and after you receive it your contribution period is lost.
General explanation, for guidance. Rules and amounts change: confirm with the responsible office.