I know many have asked why I care about PHRF for a "family boat". I probably should have explained that while I do want a competitive racer/cruiser for future beer can racing, I use the PHRF as a good litmus test for the turn of speed I'm looking for. Seems the 150's and below are where I'm feeling most comfortable.
Read about the base handicap for PHRF too. It is usually something like 500 seconds. A 170 vs 150 boat is only about 3% slower, not the 13% that you'd guess if you did ((170/150)*100)-100.
I have a boat that is pretty typical speed wise of 30' cruisers with base PHRF in the 170s, setup for cruising (dodger, anchor, fixed prop, etc) we were given a PHRF of 201. When cruising I find that we still pass a lot of boats that should have a base PHRF in the 130s, the difference between the two isn't that great and really comes down to paying attention to sail trim. I've also been passed by boats that have a higher PHRF than mine, but that are being better sailed.
There are a lot of ways that a boat with a base handicap of 170 could outperform a boat with a base handicap of 150. A folding prop, smaller dodger, not dragging a dinghy, and better sails would more than make up that difference.
Even if the true handicaps were 20 seconds apart that is a difference of 5 minutes on a 30 mile day.
I think it's worth paying some attention to PHRF if you want to avoid really slow boats, but I wouldn't worry about the difference of 20 seconds.