Experienced the same. Found a way to delete those folders with the illegal character. A folder is not allowed to have a period in its name. At least not in Windows.
As crazy as it sounds, i used WinRAR.
Open WinRAR, go to the folder with the broken folders in it. Select them all and then press Shift+Delete
Windows couldn't do it. WinRAR could.
(Copied and pasted from translator)
First of all, good morning, afternoon or evening.
I felt cheated when I couldn't get past the character creator.
To have known that, well, I used the cmd with elevated permissions to be able to delete the entire game folder along with that folder, and the other subfolders.