Skip to content

Add TaskState class to worker.py #4097

Closed

Description

@mrocklin

Currently we record state in the Worker class with a variety of dictionaries that hold information like the function to be run, the type of the output, the status, the dependencies that we're waiting on, the number of bytes of the output, and so on. We might consider turning all of these collections into a single collection of a TaskState type.

We did this transformation in the Scheduler a while ago see scheduler.py::TaskState and that seems to have been a good choice. We should probably repeat it for the worker. Maybe something like the following (incomplete)

class TaskState:
    def __init__(self, key, run_spec):
        self.key = key
        self.dependencies = ()
        self.dependents = ()
        self.duration = None
        self.priority = None
        self.state = None
        self.dep_state = None
        self.who_has = None
        self.nbytes = None
        self.resource_restrictions = None
        self.actor = None
        self.exception = None
        self.traceback = None
        self.type = None

This has the advantage of being a good learning task

Activity

  1. mrocklin commented on Sep 8, 2020

    @mrocklin
    MemberAuthor

    @gforsyth if you wanted a warmup ...

  2. gforsyth commented on Sep 9, 2020

    @gforsyth
    Contributor

    Sounds good. I'll pick this up today.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions