I'd keep the rubber band on Exploding Grievous. I cut the one on mine off (stupid) and now his torso doesn't stay with the legs.
I got another rubber band to put around him, and it's hidden well by the cape, so I like like it better this way.
I'm not being facetious at all when I say this, but I don't know why that never occured to me. I guess sometimes I over think things.