Learning one of the two languages first doesn't impede you from learning the other language later. The question you should ask yourself is which one do you want to learn first? French seems like the logical choice at first as you already speak two Indo-European languages, but if your heart is set on Japanese you will find it just as easy to learn. The kanji characters can be easier to learn if you brake them down. For example, learn only 10 characters every second day, before you know it you will have learnt all the 2000 kanji.