What this means, building off of what the evidence I gave, is that this model is not learning the morphemes (smallest root meaning). This exact characteristic is part of why these words sound weird. It is the same problem as the one brought up by tasogare.
I am not totally happy with the results but have not had a chance to train my own