On our first boat we had a compass, depth sounder, VHF, and paper charts. We were younger, and therefore less likely to have anything bad happen to us
but that said, IMHO sailing this way teaches you not to depend on the gadgets, and serves you well when the gadgets, electrical system, etc. breaks.
You can get along surprisingly well in the fog, following depth contours, sailing mark to mark, dead reckoning, using the compass, and blowing your fog horn.
Today we sail with chart plotters, radar, VHF, AIS, SSB, cell phone, backup handheld GPS, cell phone based chart plotter, IPAD based chart plotter, Wind, knot log, depth sounder, autopilot...yea and all the stuff is integrated. Sometimes I think we could just stay at home and let the boat sail itself, but I'm not giving that stuff back. It definitely settles the stomach when you hear a fog horn, and you see the barge and tow on the radar.
On the other hand, I can see the argument for keeping it simple too. I think you could put together a pretty safe system for low bucks. I wouldn't skip the built in VHF (for safety), hand held GPS's are cheap, depth sounder for safety, magnetic compass of course, and probably the most expensive item is radar if your are going into traffic in the fog. The back of your neck is your wind instrument, the paper charts are your chart plotter, you are your autopilot as long as you don't plan extended passages with minimal crew.