I think it is possible, but mostly you lose the accent before you actually lose the language. If you speak your mother tongue at a young age, your family and relatives should encourage it. I originally from Portugal, but I don't look Portuguese. I got my Dad's genes so I just look British white, so my family and relatives only speak to me in English but speak to everyone else in Portuguese. So when I went to Portugal a couple years back, my accent sounded so wrong, I literally had to learn the language from scratch.