This has to be a fuel
system / throttle control problem.
When you said that you can put it in neutral and get it to rev up a little and then put it back in gear and it stays revved up whereas you can't put it in gear first and and then rev it up is the biggest piece of evidence that suggests there is some throttle / governor problem.
There's no need to check compression or starter
speed or anything. It's not getting fuel
! When starting it is taking a very long time to build up enough fuel
at cranking speeds to do anything.
I think the governor may be built in with the injector pump
, which is probably where all the problems are.