VA has the same "issue" than animation.
You can do it, but to do it the "right" way and you need a lot of resource. For animation you can do it alone, and as I know Runey do it himself. Yet a short animation need resource that you could spend otherwise on a lot of "standard" pics. For Harem Hotel it works and the (later) animation are well done, but look at the changelog and see how often it was written there that he has redone some of them.
So if you have, and you always have, limited resource, one question how to spend it.
Same goes for VA, you have to search & find the cast, pay them and spend a lot of time on it. Also to do this well, you possible need at least some semi-professional tools and they can cost quiet a little bit.
And even if all goes well, you have the equiment you needed and found the VAs, they could be scattered all over Europe, Asian and America. So you would need to fly them to a central location, host them and either go yourself there too or trust the staff of your chosen subcontractor.
TL;DR: VAs to do well is quiet difficulty, cost money and time and seldom worth the effort. Same is true for animation although to a lesser extent.
