I used to, until I had the same problem. I started off removing the ones that were really inactive but now I only have two others besides my own favourited, and even then only because 1) so I can see if mine has new comments when I go to the blog list and 2) because the other two are my best friends.

Otherwise, no. I don't favourite anymore.

That's not to say I don't frequent certain blogs though - I just don't click the little heart.