I guess I just didn’t know that LLMs were set up his way. I figured they were fed massive hash tables of behaviour directly into their robot brains before a text prompt was even plugged in.
But yea, tested it myself and got the same result.
There are several ways to go about it, like (in order of effectiveness): train your model from scratch, combine a couple of existing models, finetune an existing model with extra data you want it to specialise on, or just slap a system prompt on it. You generally do the last step at any rate, so it’s existence here doesn’t proof the absence of any other steps. (on the other hand, given how readily it disregards these instructions, it does seem likely).