You need to wait until the fruits ripen and fall off (looking at the chest), then you can replace it. Ah I see, thanks! That wasn't clear at all while playing, I did see the action in battle, but it has no useful description.
