Exactly like KLM said, the only reason your Karma would be decreased is if you did something that broke the rules. Or, a user decreased your Karma for being annoying or something (however, if a user decreases your Karma and they weren't supposed to, a Staff member will decrease theirs and you will come back). You should not be able to even look at the chat, let alone be there if you broke a rule and such.
Once your Karma comes back, only then may you have the privileges of everyone else - to chat, view the chat, increase and decrease Karma, etc. etc.
Sorry man, for this idea,
-1.