Larian is looking into this, along with the publisher (though they reported that many of the cases they were aware of were due to the host playing in single player mode).
Natu, have you played any peer to peer co-op games, or just server based multiplayer?
XBox One has a multiplayer API which handles the network connections, so there shouldn't be any game specific issues in that regard. Peer to peer connections can have more trouble than server based, due to usually having both ends of the connection behind routers and ISPs, while server based connections have one end explicitly designed around multiplayer.