I can absolutely see how it's a distraction for some people and an annoyance for others. That's something that I figured would happen... I mean, there's a reason why other sites have not gone the route of allowing an "anything goes" section, at least not that I'm aware of.
However, I have grown quite fond of arguing with certain people, and to be honest, it's what kept me coming back for the past year or so. Maybe if you keep it, make it an additional login or something... just a thought.
Whatever you do with the Pit, I'm sure you will make an educated decision.