If people do enjoy talking with the person, I do not neccessarily see the point in arguing against the demotion? If they are especially knowledgeable in the lore or particularly sociable that's all good and nice, but those are things that do not require a GM role?
Having friendly GM's that help ...