No, there is no reason to furl a storm jib.
Your idea of hoisting it by itself will work.
A Solent stay is a better idea because you could then raise it from the cockpit after furling the jib. And with a downhaul which is very easy to rig you could also lower it from the cockpit.
Living aboard in Victoria Harbour