What you're describing is the behavior you get from any base model that has not been instruction-tuned. The article is clear that this model is not for "direct use". It needs tuning for a specific application.
>>coder5+(OP)
how does one fine tune it to follow instructions? I would have thought they have open source training set for these instruction-follow fine tunes?