Fix NPE in RemoteTaskRunner when some tasks in ZooKeeper but not in Overlord - #5511
Conversation
|
👍 We should call out in release notes that there will be a small per task increase in the amount of data stored in zookeeper. |
| zkWorker.getWorker(), | ||
| TaskLocation.unknown(), | ||
| runningTasks.get(taskId).getDataSource() | ||
| announcement.getTaskDataSource() |
There was a problem hiding this comment.
If the task has already been shutdown, will this make it start back up again?
There was a problem hiding this comment.
I guess there's already a "we'll add it if we don't know about it" thing immediately below, so the check must be elsewhere.
|
@jihoonson can you record in this PR where that assurance is made... prior shutdown tasks should be terminated instead of added to the active queue. I forget where it is and its not easy to find. |
|
@drcrallen if you mean When the |
@clintropolis thanks for the review. This also sounds good. |
|
@drcrallen did you have any more comments on this PR? |
…verlord (apache#5511) * Fix NPE in RemoteTaskRunner when some tasks in ZooKeeper but not in Overlord * revert unnecessary change
…verlord (apache#5511) * Fix NPE in RemoteTaskRunner when some tasks in ZooKeeper but not in Overlord * revert unnecessary change
Fixes #5506.
This change is