Skip to main content

RayFunctionTask

This class is a plugin that transforms local Python code for execution within a Ray job. It handles Ray initialization, including setting up the working directory and runtime environment when running in a cluster. It also defines custom configuration for Ray jobs, specifying head and worker node configurations, autoscaling, and runtime environment details.

Attributes​

  • task_type: string = "ray"

    • Actual Plugin that transforms the local python code for execution within Ray job.

  • plugin_config: RayJobConfig

    • Actual Plugin that transforms the local python code for execution within Ray job.

Methods​

def pre()

  • Initializes Ray if it hasn't been already. It sets up the necessary parameters for initialization, including the address and runtime environment if running within a cluster. The runtime environment includes the current working directory and a list of files/directories to exclude during packaging.

  • Return Value: Dict[str, Any]

    • An empty dictionary, as the primary purpose is initialization side effects.
def custom_config(sctx: [SerializationContext](src_flyte_models_serializationcontext)) - > Optional[Dict[str, Any]]

  • Generates a custom configuration for a Ray job based on the provided plugin configuration. This includes setting up head and worker node specifications, runtime environment details (both base64 encoded and YAML), and other Ray job parameters. It constructs a RayJob object and returns its dictionary representation.

  • Parameters

  • Return Value: Optional[Dict[str, Any]]

    • A dictionary representing the Ray job configuration, or None if no configuration is generated.