Skip to main content

WorkerGroup

Configuration for a group of dask worker pods. Allows specifying the number of workers, a custom Docker image for the workers, and resource requests for the worker pods.

Attributes​

  • number_of_workers: Optional[int] = 1

    • Number of workers to use. Optional, defaults to 1.

  • image: Optional[str] = None

    • Custom image to use. If None, will use the same image the task was registered with. Optional, defaults to None. The image must have dask[distributed] installed. The provided image should have the same Python environment as the job runner/driver as well as the scheduler.

  • resources: Optional[Resources] = None

    • Resources to request for the worker pods. Optional, defaults to None.

Constructors​

  • Configuration for a group of dask worker pods

  • Parameters

    • number_of_workers: Optional[int]
      • Number of workers to use. Optional, defaults to 1.
    • image: Optional[str]
      • Custom image to use. If None, will use the same image the task was registered with. Optional, defaults to None. The image must have dask[distributed] installed. The provided image should have the same Python environment as the job runner/driver as well as the scheduler.
    • resources: Optional[Resources]
      • Resources to request for the worker pods. Optional, defaults to None.