I don't think it makes the boat illegal to operate as ABYC is not a legal requirement though your insurance company would likely be unhappily. Many insurance policies require meeting ABYC for modifications and might ask for some systems to be brought up to the standard though some may not.
Sent from my ADR6425LVW using Tapatalk
See... 46 CRF 182.130 below. This applies to non-commercial small boats as well as commercial. Though inspection isn't required.
Title 46: Shipping
CHAPTER I: COAST GUARD, DEPARTMENT OF HOMELAND SECURITY (CONTINUED)
SUBCHAPTER T: SMALL PASSENGER VESSELS (UNDER 100 GROSS TONS)
PART 182: MACHINERY INSTALLATION
Subpart A: General Provisions
182.130 - Alternative standards.
As an alternative to complying with the provisions of this part, a vessel of not more than 19.8 meters (65 feet) in length, carrying not more than 12 passengers, and propelled by gasoline or diesel internal combustion engines, other than a High Speed Craft, may comply with ABYC H-2, ABYC H-22, ABYC H-24, ABYC H-25, ABYC H-32, ABYC H-33, ABYC P-1, and ABYC P-4 (all eight standards incorporated by reference, see 46 CFR 175.600) as specified in this part.
[USCG-2003-16630, 73 FR 65207, Oct. 31, 2008]