With today's materials, you can get lines that are much smaller in diameter which are rated for the job, but as you point out, there is a tradeoff between the hand and the diameter. If whoever is helping you just looks at the specifications, they may not consider this factor. I've had small diameter lines recommended to me by well meaning riggers, so I'm not sure you got advice that's any worse than advice I've received in the past
The good news about smaller/lighter lines is that in light air say for your jib, sometimes just the weight of the line is a factor in sail shape, but, if you cannot hand the line, it's not worth it IMHO.
Without doing the details, it's possible that you got lines that are likely not to break, but aren't user friendly. I'd consider particularly the diameter of the sheets, as you'll handle them the most. Consider swapping out the jib and main sheets, and maybe keep the rest?