I use Facebook, and I have never unfollowed others even when their views are vastly different from mine. I try to game the algorithm a bit by using lists which show me certain people but more of their stuff because lists are small, so I can see more from the people on that list.
I too, use Feedly religiously.
The problem is you and I and others like us see ourselves as curators of our own content, not merely consumers. We are the curious, the techies, the ones who are willing to take the time to be open. Most of the world is not like us.