That could be the Xbox. There was a recent discussion on here about hardwired Xboxes temporarily killing network ports until the router and Xbox is rebooted. Try things tempoarily without the Xbox and see if things work better.
Do you have anything other then the switch connected to the other ports on the T3200? If so are any of those devices affected when you get this problem? If everything is connected to the switch then maybe try connecting a device (TV or computer or XBox) directly to the T3200 and see what happens. It sounds like a broadcast storm is somehow being created when the game playing ends (perhaps the game server flooding your network looking for the computer or XBox) - and the T3200 responds by killing the port the switch is connected to.