If the lights you see have a constant bearing to you, you are on a collision course.
Let's try the following case. A large motor driven vessel (ship) is on a collision course with you. Let's try and simulate what we would see. Corrections welcome
In this specific case (an interesting one I would think

), the lights do not appear to move except as mentioned as a large ship's red and green side lights might appear to move away from each other horizontally (assuming they are positioned on the sides of a beamy ship) and masthead light(s) (2 if over 50 meters in length) appear higher and higher as the bow of the ship gets dangerously close. Before you've seen this motion, you've hopefully altered course appropriately according to the rules (and maybe the other guy has too if you are lucky and he knows the rules). You've altered your course to starboard.
After you've changed course according to the rules, his green light should go away and the red light should be visible along with the 2 mast head lights (over 50 meters). As the ship goes by, the 2 mast head lights would reveal themselves to be one behind the other, with the aft most light higher than the forward light. You'll observe these 2 lights, along with the red side light. Once he goes by even further, both the mast heads and red side light go away, and all you see is the stern light. Now you can really breath easy.
The speed at which all these changes occurs would be faster if your were closer.