Typically remote access can only be done by you giving permission at that time [...]
This apply to, and only to, legit remote access.
Any software have its flaws, and this include OSes. Even Linux have had, and still have, ways to escalate the privileges and open the door for a remote access
without needing user consent. The fact that it's not something easy to do, that its knowledge that only handfuls have, and that it usually can't be done without leaving some trace that can be found
if you look for them, doesn't mean that it's impossible to do it, and to do it without the user be aware about it.
No software is safe, all, absolutely all, can present a threat for your security. In the mid 00's, OpenBSD, an OS 100% dedicated to paranoid computer security, have had its repository servers been corrupted and a version including a backdoor be available for few hours (from memory around 5 hours).
Therefore, the question isn't "is it safe", but "how much do I trust the person/entity who made that software". Knowing that even a 100% trust do not prevent the unthinkable to happen.
Then come the second question: how can I increase this level of trust?
And here there's many possibilities:
Wait a day or two before downloading from a public source, like here by example.
The more people have downloaded the software, the higher are the chance for someone to catch the malicious code that can hide inside. This while, for more institutional sources, the more time pass, the higher are the chance for the staff to notice that their network have been compromised.
Therefore, the longer you wait, the higher is the level of trust, keeping in mind that it will never reach 100%, not even 95%.
Launch every software you're doubtful about in a sandbox or virtual machine.
It will contain the risk. In case of compromised software, the damages will be restrained to the sandbox or VM, not reaching the computer they run inside. See this as an airlock, whatever happen inside will stay inside and not propagate outside.
Have two computers. One connected to Internet that you use to browse the web and download software, and one not connected to Internet, dedicated to the use of software you downloaded. To limits the burden you can connect the two together
but punctually and with a strongly enforced one way, one port, policy.
And those are only three of the possibilities. Which one to adopt depend on you.
