We place our trash in garbage bags and then, once they're full, throw them overboard. The trash usually floats for a while, as the air in the bags gives them some buoyancy. One time we had a whole string if 15 or so bags floating behind us row for as far as the eye could see of the Florida Keys! This was at dusk, with the sun illuminating each of the bulging white bags in a beautiful "S" on the horizon. Absolutely gorgeous sight.
One word of caution, however: if you choose to use this method, be sure to remove any papers with your boat's name or your name on them. Nobody wants a bunch of water cops telling you what you can and cannot do.