Repository navigation
#6336 causes overlord console page loading to be extreme slow #6726
Description
As discussed in #6336 (comment), I think we should revert #6336 first and find a better way to complete #6336. How do you think @FaxianZhao @jihoonson.
Activity
vogievetsky commentedon May 6, 2019
Is this still an issue in the latest code?
QiuMM commentedon May 9, 2019
Yes, it is. I have made some improvement in my druid code, I'll open a PR to patch this when I'm not busy.
FaxianZhao commentedon May 10, 2019
I suggest use the Fine-Grained Lock resolve this kind issues.
I think my previous draft code won't cause this, because I use
synchronized (lockResult.getTaskLock())instead oftoolbox.getTaskLockbox().doInCriticalSection, but it looks bad for committed code.I think we could push down lock action to the TaskLock level. Giant lock is used to approve TaskLock, and TaskLock is work for the real business. We could implement TaskLockType with ReadWriteLock, but I haven't go through the whole picture.
jihoonson commentedon May 10, 2019
@QiuMM thanks. I'm looking forward to your PR.
@FaxianZhao yeah, fine-grained locking would mitigate this issue. But, synchronizing
taskLockisn't enough to fix the issue of #6336, becausetaskLockis shared by the tasks of the same groupId but the tasks of different groupIds should be able to request segment allocation for the same dataSource and interval at the same time. Check out #7491.I think the best way to handle this is synchronizing segmentId allocation per dataSource per interval as @QiuMM did here.
github-actions commentedon Jul 30, 2023
This issue has been marked as stale due to 280 days of inactivity.
It will be closed in 4 weeks if no further activity occurs. If this issue is still
relevant, please simply write any comment. Even if closed, you can still revive the
issue at any time or discuss it on the dev@druid.apache.org list.
Thank you for your contributions.