I and my friends have been having issues with not being able to invite some people to a party when they're online. The party up screen shows them as being offline and we can't send the invite. The only way to fix it is to back out of multiplayer and reconnect.
The fact that one of our party members ended up in a completely separate battle, yet we could hear him and vice versa the entire time. In fact he played two elimination battles without us while still linked via headset while we played the deathmatch.
Seems the Party code in general might need some debugging.