That would be a very lame excuse, I mean unless they tell us that the Duke's amulet was made by a deity or a magical being and the others weren't, since Rose possibly being a descendant of the priestess of the natives who protected the island in the past had a special resistance to this type of magic so a special amulet is needed to be able to convert her sexual energy into magic or something like that, but so far we are more than halfway through the game and they haven't mentioned anything like that to us. :_D
and it is more than clear that the royal family of almystira are descendants of the natives of the island because one of the reasons we can intuit this is how the king had the knowledge to reach the hidden village of the natives if it is supposed to be hidden with magic, and the other reason I have to believe that, is the possible relationship that there was between king flavian and the priestess of the natives of the island, this because of the events that have been mentioned to us so far in the story, of course we cannot go into much depth because of the little information there is, but from how things happened in the past between the natives and king flavian, and the presence of the pacekeepers who are similar to the knights who protected the priestess, one can deduce that this is really how things were.