So doing a full chick run then you mean? Then yes you'll miss the restaraunt. Lilly can be got in the party even with a chick run.
But different strokes, I don't need to nail everything with 2 legs to have fun in a game. I play for the story, not for the sex. So mileage varies I guess as to what is enjoyable on which affinity playthrough. As for general interections, well yeah I guess, if you like acting like a dik with the associated dialogue then chick route will be less interesting,.
But ask yourself this, what if DPC does different things for diff routes? Can do all of Sages stuff with chick affinity, all you miss out on are a few RP's. How important they become (utterly pointless currently with exception of MJ picnic scene) remains to be seen. I have saves for a few characters with different affinities. Maybe Sages best content in the future will come from chick route as like many women in the world, after being treated like crap from a dik suddenly realises and appreciate the long-term stability of a chick.
Worth doing a chick run imo (or a couple for your fave LI's)...sooner or later the RP system for all the girls has to come into effect. Chasing Jill on dik affinity and the RP comes into play? Kiss that chance goodbye. There also certain scenes that can only be seen with chick affinity.