Intelligence I agree with. Charm is actually trivial to increase at the hospital once you can afford the surgeries. You can buy it up to 89 assuming you grind out 4 points via caress or the factory so you get it to a number ending in 4 or 9 before you get the surgeries. Starting with high charm does make the early game easier though if you want to sex up the girls during the first month before you can afford the surgeries.The very most important is intelligence and charm.
Everything else can be worked on.
Manipulate is harder to increase IMO, although once you start taking the classes and reading the books you can get more manipulate that way than you can charm. Personally, I boost charm to 89 and manipulate to 90 before I take any classes or read any books that affect either stat.