Repository navigation
Resource Warnings with LocalCluster #2486
Description
Hi Everyone,
After starting a local cluster on my machine as shown here:
print(cluster)
client = Client()
print(client)
cluster.close()
I got so many Resource Warnings as shown below.
ResourceWarning:
unclosed file <_io.BufferedWriter name='/home/amal/PycharmProjects/dynamicparcels/dynpar/dask-worker-space/worker-4t43rjqe.dirlock'>
/usr/local/lib/python3.6/dist-packages/distributed/diskutils.py:161: ResourceWarning:
unclosed file <_io.BufferedWriter name='/home/amal/PycharmProjects/dynamicparcels/dynpar/dask-worker-space/worker-4fyxnn5d.dirlock'>
<Client: scheduler='tcp://127.0.0.1:43123' processes=8 cores=8>
distributed.scheduler - INFO - Clear task state
distributed.nanny - INFO - Closing Nanny at 'tcp://127.0.0.1:37009'
distributed.nanny - INFO - Closing Nanny at 'tcp://127.0.0.1:41811'
distributed.nanny - INFO - Closing Nanny at 'tcp://127.0.0.1:38581'
distributed.nanny - INFO - Closing Nanny at 'tcp://127.0.0.1:42381'
distributed.nanny - INFO - Closing Nanny at 'tcp://127.0.0.1:41167'
distributed.nanny - INFO - Closing Nanny at 'tcp://127.0.0.1:39899'
distributed.nanny - INFO - Closing Nanny at 'tcp://127.0.0.1:43027'
distributed.nanny - INFO - Closing Nanny at 'tcp://127.0.0.1:34385'
distributed.scheduler - INFO - Remove worker tcp://127.0.0.1:39323
distributed.core - INFO - Removing comms to tcp://127.0.0.1:39323
distributed.scheduler - INFO - Remove worker tcp://127.0.0.1:46047
distributed.core - INFO - Removing comms to tcp://127.0.0.1:46047
distributed.scheduler - INFO - Remove worker tcp://127.0.0.1:45833
distributed.core - INFO - Removing comms to tcp://127.0.0.1:45833
distributed.scheduler - INFO - Remove worker tcp://127.0.0.1:36641
distributed.core - INFO - Removing comms to tcp://127.0.0.1:36641
distributed.scheduler - INFO - Remove worker tcp://127.0.0.1:42175
distributed.core - INFO - Removing comms to tcp://127.0.0.1:42175
distributed.scheduler - INFO - Remove worker tcp://127.0.0.1:33033
distributed.core - INFO - Removing comms to tcp://127.0.0.1:33033
distributed.scheduler - INFO - Remove worker tcp://127.0.0.1:46337
distributed.core - INFO - Removing comms to tcp://127.0.0.1:46337
distributed.scheduler - INFO - Remove worker tcp://127.0.0.1:37449
distributed.core - INFO - Removing comms to tcp://127.0.0.1:37449
distributed.scheduler - INFO - Lost all workers
distributed.scheduler - INFO - Scheduler closing...
distributed.scheduler - INFO - Scheduler closing all comms
/usr/lib/python3.6/asyncio/base_events.py:511: ResourceWarning:
unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>
Do anyone have an idea how can I fix that please ?
Activity
mrocklin commentedon Jan 28, 2019
cc @pitrou any thoughts on this?
pitrou commentedon Jan 30, 2019
Not really. The symptoms are rather vague...
pl-marasco commentedon Feb 4, 2019
Hi @bamal ,
have you tried to clean completely the folder?
/home/amal/PycharmProjects/dynamicparcels/dynpar/dask-worker-space/
One possibility is that, since it looks like you're using pycharm, you still have running an underneath process or you stopped an unfinished process and one file is still locked.
Are you using a private machine with admin privileges or a corporate one with less privileges?
jonmoore commentedon Jul 13, 2021
I don't have a minimized reproducible case yet, but have also seen this warning occur repeatably when using instances of Client while running distributed 2021.02.0 under pytest 6.2, Python 3.8.8, Windows 7.
The warning is that an instance of asyncio.base_events.BaseEventLoop was not closed before being deleted by the garbage collector. In the cases I've investigated the loops were created by calls beginning with Client.init, running through distributed.utils.LoopRunner.start, which creates a daemon thread that calls tornado's BaseAsyncIOLoop.start-->asyncio.get_event_loop->(distributed)AnyThreadLoopEventPolicy.get_event_loop->, which creates BaseEventLoop. The loop then gets recorded in LoopRunner._all_loops. It looks like it is meant to be closed by a call to LoopRunner.stop.
If anyone knows off-hand if the current behavior is intentional or not, that would be helpful to know before digging further into this.
fjetter commentedon Jul 14, 2021
@jonmoore the
LoopRunnershould be closed if theClientis closed, seedistributed/distributed/client.py
Lines 1489 to 1490 in 67f57bc
I guess, for some reason, the client is not closed properly before the interpreter shuts down. We've been fixing a few stability related things over the past months and I would encourage you trying a more recent version to see if the problem persists.
jrbourbeau commentedon Nov 2, 2021
@jonmoore, following up on @fjetter's comment, does upgrading to the latest
distributedrelease help?zanieb commentedon Apr 13, 2022
See #6122 and mwilliamson/locket.py#14