We have only motored in the ICW. Its very narrow in places and you have to watch your depth. But you can sail on the river down as far as Green Cove. I was born and raised in Jax., left town in '75 :-). But we have always wanted to sail on the river. I don't know what your experience level is, but as lake sailors we have a lot of new skills to practice and they are all available here. Check out this forum
Living Aboard Forums
Several of the folks are in Jax. There is a marinas thread but I understand that prices are down since they were last posted. If you drive around, you always see folks anchored out. And the city has some low or no cost (except during "events") facilities available for limited use.
As for cold - it snowed when I was in the 2nd grade!