This question gets asked so much here.
It is scary when you first start looking into this sport and figuring out where and how you fit in.
It sounds to me like you just want to get your feet wet. This is good. Too many people decide to plunge in head first, buy a 30,000$ boat, sail it a few times, sell it and that chapter of their life is over.
First things first--- Check your local area.
Are there any sailing associations or clubs nearby? (Rudder club, Sail Lazer, ect.) What about yacht clubs that do races where you could perhaps gain a spot as crew?
If you REALLY have none of these resources near you or you simply want your OWN boat then you go to step 2. Check out Boattrader, craigslist, boats.com ect. Search for boats like the Bethea or Flying Scot or Laser. These boats are all of the dinghy class but have the bigger boat set-ups of mains and jibs with cunninghams and boom-vangs ect. ect. These little boats are easily trailored and set=up. launched and sailed by one person with very minimal skill. On craigslist you should be able to find one of these boats for between 500-1000$.
Before you make a purchase buy a book like "Inspecting the Aging Sailboat," or something of the like. This will give you the knowledge to buy a boat that is not a lemon. (We are not taking big boats sailnet, dinghys here, not much to it for inspections! Lets not advise surveys on a laser mmm k?)
If the boats hull looks good, mast is straight and not cracked or corroded, and the sails are not ripped or torn you are pretty much good to go. Give a good look to the rudder attachment too.
It is a bit scary starting out, but once you get your feet wet you will quikcly see its not so bad, especially when starting on a dinghy. The real learning curve is when you start getting into REAL sailboats with electrical,plumbing, and engine systems to maintain and fix. THAT is where the real curve begins.
All you need is your points of sail and a fair breeze mate.