There is no such thing as a ghost villager in vanilla minecraft. Are you sure it's not a pillager or evoker? Or are you played with some sort of mod? That would explain it. Im just a little confused.