I agree with some of what has been said. The three boats listed are fairly different in design. Not at all familiar with the Fast Passage 39, but the Valiant 40 has a very good reputation. The designer of that boat, Bob Perry is a somewhat frequent poster on this message board and if he happens upon this thread I'm sure he can offer some great advice.
I've owned and sailed a Downeaster 38 for the past 8 years. I'm not sure what build issues krisscross has experienced, but my experience and those of other owners that I've talked to, are that they are very well built boats. I've also been on several Westsails (which have a good build reputation) and in my opinion the Downeasters are very similar. They are solid layup on the hull, over an inch thick below the waterline. The bulkheads are all heavily tabbed in. I have heard of one hull that had blisters, but I don't think it's common. My hull has none.
Not saying they're perfect. Any Downeaster on the market now would be pretty long in the tooth, and will have associated age related issues. If it has the original wiring, it will likely need to be replaced. Same with the engine. The caprail will likely leak unless it's been recently rebedded and it would be a good idea to replace all of the standing rigging before any hard sailing. I think these are reasonable issues when you are buying a 32-40 year old boat.
The Downeaster does have a short, low aspect rig, hurting it's speed in light air. The schooner rig offers a bit more sail area and flexibility, but I've always thought that putting two sticks on a 38 foot boat was a bit much. An asymmetrical spinnaker helps greatly off the wind. I don't think that the Downeaster is particularly tender compared to other boats of similar design. It has less initial stability than most flat bottomed, beamy boats of today but that's to be expected from the design. Basically you'll heel to about 10 degrees in the first 10 or so knots, up wind. But it will heel only another 5 or degrees up until you are cracking on in 25 knots or so. Then you start reducing sail. I've only had my rail buried once, and that was in 40 kt gusts.
Performance wise, meh, if I was in a hurry to get there I would fly.

They don't point well compared to a sloop rigged fin keel boat, but it's also not nearly as bad as you hear people describing at the yacht club bar or on forums like this. (They can sail upwind, FFS, square riggers can sail upwind!) On the other hand, they do track very well. On a close to about a beam reach, they will sail themselves if your sails are balanced.
They have tones of storage aboard. I can literally have tools, boxes, spares, and random stuff stacked up almost to the overhead and it all has a place to get stowed when the work is done and it's time to go sailing. The lazarette could be it's own berth, I throw everything in there.
Maneuvering the boat under power in tight spaces is always a challenge. I enjoy the challenge but some might find it off putting. Backing up in any kind of wind is very difficult. The bow wants to blow down wind and this windage is almost always going to be more powerful than the rudder in reverse. This is definitely a liability for these boats but if you own one, you pick your battles and sometimes you will find yourself anchoring out rather than taking a slip because the T-dock or bulkhead is full.
Summary (IMO) for the Downeast:
Good:
Solid (in my experience) heavy duty build
Tracking ability
Comfortable motion in most sea states (subjective)
Roomy, plenty of storage
Looks, they are quite pretty, I think
Bad:
Light air boat speed
Pointing ability
Maneuverability (backing up in particular)
Age
Hope this helps. If you have any specific questions, let me know.
-Argyle