Using Gym Chalk to combat sweaty hands

Hi all,

I've been using gym chalk to help with my sweaty hand syndrome I get when playing against another player. I play on both a Madcatz Te2+ with a Hayabusa joystick and a Hori VLX Premium Hayabusa joystick. Is there any precautions I should take to prevent any damage(if there would even be any) to the joysticks at all. I noticed sometimes that some flecks get on the face of the case which I promptly wipe off. Is this a bad idea? Let me know what you think. Later!