Only twice? It almost seems like the first match I play in any particular session I'm almost guaranteed to get disconnected. My connection is decent (speedwise) but one thing I've always been able to count on is it's stability. I've been playing with more randoms recently though...maybe that's it.
I should never be disconnecting really, unless connection speed isn't actually that much of a major factor in such matters.