It depends on the ruber. Vulcanised rubber would have the most activated carbon in it...but you would not use it for much boat related stuff anyway...
latex rubber is safe, as is silicone "rubber" and any number of rubbers that have achieved the same stability as vulkcanised rubber, but through chemical processes instead of heat.
All in all, I think you are safe. really.