That's great that you're giving racing a try. It's a good way to improve sailing skills. It can be hard to do that in a vacuum abord your own boat, unless you have someone more experienced aboard. So you might want to consider signing on as crew aboard one of those winning boats to learn a few tricks of the trade.
As to your questions, I have to say I am somewhat puzzled at your suggestion that you don't use your topping lift? [I assume you are refering to the boom topping lift, not the spinnaker pole's?] Unless by that you mean that you don't adjust it after you have raised the mainsail and then completely eased it? As long as you are easing it after raising the main, you shouldn't touch it again until it's time to drop the mainsail. It is not normally used for sail trim.
As for the traveller: Very generally speaking, you are going to bring the traveller up toward the windward side of the boat for upwind sailing, and ease it down to leeward when sailing off the wind. Going upwind, you would never want to raise it so high to windward that the boom crosses over the fore-and-aft centerline of the deck to the windward side of the boat. Depending on the wind conditions, you may need to "play" the traveller quite a bit as you're sailing upwind. That would usually be in heavier air, when you would leave the mainsheet set at the appropriate trim and just drop the traveller down during puffs to spill some of the breeze.
Vang: It primarily helps to hold the boom down when the main is fully eased off the wind, because at those sheeting angles the mainsheet is no longer holding the boom down. As a very general rule, sail with a moderate amount of tension on it, rather than leaving it loose.
This could be a very long thread, but there are many helpful books on sailtrim that you might consider. I'm drawing a blank on titles but others might toss a few out.