I'm more than happy to leave if there's not at least 1 fish at my table, and even then I'll leave if the fish isn't that big a fish. I'm not bothered about the "challenge" of playing against good players because there's usually 2 of them on every table which gives me ample oppurtunities to learn how to beat them.

Having said that, it's good practise to learn to adjust to your image. Also, watch out for tilt, that might be a problem for you.