Resolution:
Jobs may develop a race condition during Task submission, between the openTaskSubmission call within the job.start() thread and a call to closeTaskSubmission(). This race condition happens if you do not override createTaskInputs in the job class. Sometimes the internal openTaskSubmission gets called after closeTaskSubmission so the job never completes because it is in the open task submission state.

The simple workaround is to override createTaskInputs as a no-op, and call openTaskSubmission() prior to starting the job.