Thanks for the tips.
This evening I went to town and did some price shopping. Probably nobody cares about this but I will report anyway since I went to the trouble. I made up a list of things I normally eat, things that I have here at the house, and then I went into four stores and compared the prices. I have done that one other time but it has been some time ago and I wanted to check mostly on canned goods this time.
Went to a local dollar store, Kroger's, Martin's, and Walmart.
As if you couldn't guess, Walmart was consistently the least expensive. Walmart's "great value" brand was especially inexpensive, it was 10-35% less expensive than the off brands at the other stores, but even comparing popular brands like "Del Monte" they were still less expensive by a margin. Krogers was the most expensive here but that might not be the way it is everywhere of course. I didn't even bother going to the local Food Lion store because they are a lot more expensive than the others, I remember that from last time, they make money on their convenient location. Surprisingly the local dollar store which is the most convenient location didn't do half bad on prices, I will shop there again.
I suspect that Costco will beat all of those prices on things that they have.
I will check into the restaurant supply stores that is a great idea!
Edit, adding one thought to this. I think the obvious thing to do to save a few bucks on canned goods is just to buy a lot of stuff when it's on sale if it's something you eat. The prices for things that were marked down at most of the stores were significantly less expensive, definitely worth it to stock up.