The issue is that the glamor server is almost certainly a multiple dispatch server (it's designed to handle hundreds of clients on the internet). You'd have to bypass all of that in the client mode. Far from impossible, but it means breaking some of the encapsulation and kind of a mess. Also have to configure the client to listen to the socket instead of initiating the connection.
Also, people will accuse you of failing to learn the lessons of FTP or Xwindows, although they mostly don't apply in this case.