Pareidolia seems to be the one that broke Sensei in the first place, and plans on him becoming the Collector afterwards:
I don't think Pareidolia has much actual power without Sensei, hence the whole "you have to trust me" and "I'm the only one who knows how to use you" stuff:
Even in it's season, where it should be at it's strongest, it's still trying to convince Sensei to choose it:
Not to mention, threatening to take more from Sensei to get him back in action:
If Pareidolia actually could just do whatever it wanted with Sensei, it probably would. It's likely only "letting" him grieve, because it has to.
Like Pareidolia has said:
Pareidolia's actual power also seems to revolve around manipulation. It doesn't seem to have the same level of raw power as the others. At least not yet.