It's not normal for the A-4 to make a knocking noise. The only wierd noise you should notice is a noise like a crunching whine while in reverse. That said, It sounds to me like the accessory drive is causing the problem. You didn't mention the if the A-4 was an early or late modle. It could also be the transmission. As noted before use a long handeled screwdriver or a stehascope, and stick it to various places on the engine while it is running and making the noise.
Contact Myoyer Marine and get the A-4 bible to start. http://www.moyermarine.com/
Don is very helpful and is known as the A-4 Guru.