It's not the dumbest idea, you will gain a lot of experience very quickly. If the trip does become overwhelming, or something critical breaks on the boat, you can always stop at a marina. I can think of worse things than having to spend the winter on a boat in say, Baltimore.
I had a lot more sailing experience than you when I went down the icw. But I did it on a new to me boat. In retrospect, I really wish I had lived and cruised on it for a season at home, before setting off down the icw. It took a while for me to get familiar with all the systems on the boat (electrical, plumbing, etc.), and to get things set up the way I wanted for cruising. Cruising is more than just sailing. It's like sailing the house you live in.
If you do go, I would leave NOW! You want to be in the Chesapeake in October and on the icw in November. The weather is gorgeous on the Chesapeake in October, with the leaves changing and all. In November, conditions on the Chesapeake can get downright nasty. Many an ocean sailor has said the worst sailing conditions he has ever seen were on the Chesapeake.
If you go, I guarantee it will be one of the most exciting (and hopefully best) things you do in your life. Shameless self promotion: I wrote a book about my trip. It's available on amazon at Amazon.com: Captain's Blog with Comments: Scott Englehart: Books
Gemini Catamaran Split Decision