My intro to GPS navigation has been through a recent purchase of the Garmin GPS60. Although a little over your $100 limit, I have found this to be a very usable budget GPS, with quite a number of mariner related features. It is powered by battery, but has an input to hook into your boats power system via an optional extra connector.
I think suitable mounts can aslo be purchased if you didn't want it to be strictly hand held.
As camaraderie says, these cheaper options are no less accurate, but often limited in capability eg no inputs, outputs, chart plotting etc. GPS60 is an example of this, but I have found it suits me fine using paper charts.
Check it out for yourself . . .