TaskEnvironment

Environment class to define a new environment for a set of tasks. This class allows users to specify the Docker image, resources, environment variables, secrets, and dependencies for a task execution environment. It also supports caching and reusability policies for optimizing task execution.

Attributes

• cache: CacheRequest = "disable"

  • Cache policy for the environment.

• reusable: ReusePolicy | None = None

  • Reuse policy for the environment, if set, a python process may be reused for multiple tasks.

• plugin_config: Optional[Any] = None

  • None

Constructors

• Environment class to define a new environment for a set of tasks.

  Example usage:

  env = flyte.TaskEnvironment(name="my_env", image="my_image", resources=Resources(cpu="1", memory="1Gi"))
  
  @env.task
  async def my_task():
      pass

  :param name: Name of the environment :param image: Docker image to use for the environment. If set to "auto", will use the default image. :param resources: Resources to allocate for the environment. :param env_vars: Environment variables to set for the environment. :param secrets: Secrets to inject into the environment. :param depends_on: Environment dependencies to hint, so when you deploy the environment, the dependencies are also deployed. This is useful when you have a set of environments that depend on each other. :param cache: Cache policy for the environment. :param reusable: Reuse policy for the environment, if set, a python process may be reused for multiple tasks.

• Parameters

  • name: string
    • Name of the environment
  • image: string
    • Docker image to use for the environment. If set to "auto", will use the default image.
  • resources: Resources
    • Resources to allocate for the environment.
  • env_vars: Dict[str, str]
    • Environment variables to set for the environment.
  • secrets: SecretRequest
    • Secrets to inject into the environment.
  • depends_on: List[Environment]
    • Environment dependencies to hint, so when you deploy the environment, the dependencies are also deployed. This is useful when you have a set of environments that depend on each other.
  • cache: CacheRequest
    • Cache policy for the environment.
  • reusable: ReusePolicy | None
    • Reuse policy for the environment, if set, a python process may be reused for multiple tasks.

Methods

def clone_with(name: str, image: Optional[Union[str, [Image](src_flyte__image_image), Literal["auto"]]] = None, resources: Optional[[Resources](src_flyte__resources_resources)] = None, env_vars: Optional[Dict[str, str]] = None, secrets: Optional[SecretRequest] = None, depends_on: Optional[List[[Environment](src_flyte__environment_environment)]] = None, kwargs: Any) - > [TaskEnvironment](src_flyte__task_environment_taskenvironment)

• Clone the TaskEnvironment with new parameters.

Besides the base environment parameters, you can override kwargs like cache, reusable, etc.

:param name: The name of the environment. :param image: The image to use for the environment. :param resources: The resources to allocate for the environment. :param env_vars: The environment variables to set for the environment. :param secrets: The secrets to inject into the environment. :param depends_on: The environment dependencies to hint, so when you deploy the environment, the dependencies are also deployed. This is useful when you have a set of environments that depend on each other. :param kwargs: Additional parameters to override the environment (e.g., cache, reusable, plugin_config).

• Parameters

  • name: str
    • The name of the environment.
  • image: Optional[Union[str, Image, Literal["auto"]]]
    • The image to use for the environment.
  • resources: Optional[Resources]
    • The resources to allocate for the environment.
  • env_vars: Optional[Dict[str, str]]
    • The environment variables to set for the environment.
  • secrets: Optional[SecretRequest]
    • The secrets to inject into the environment.
  • depends_on: Optional[List[Environment]]
    • The environment dependencies to hint, so when you deploy the environment, the dependencies are also deployed. This is useful when you have a set of environments that depend on each other.
  • kwargs: Any
    • Additional parameters to override the environment (e.g., cache, reusable, plugin_config).

• Return Value: TaskEnvironment

  • A new TaskEnvironment instance with the specified overrides.

def task(_func: Optional[Any] = None, short_name: Optional[str] = None, cache: CacheRequest | None = None, retries: Union[int, [RetryStrategy](src_flyte__retry_retrystrategy)] = 0, timeout: Union[timedelta, int] = 0, docs: Optional[Documentation] = None, pod_template: Optional[Union[str, "V1PodTemplate"]] = None, report: bool = False, max_inline_io_bytes: int = MAX_INLINE_IO_BYTES) - > Union[[AsyncFunctionTaskTemplate](src_flyte__task_asyncfunctiontasktemplate), Callable[P, R]]

• Decorate a function to be a task.

:param _func: Optional The function to decorate. If not provided, the decorator will return a callable that accepts a function to be decorated. :param short_name: Optional A friendly name for the task (defaults to the function name) :param cache: Optional The cache policy for the task, defaults to auto, which will cache the results of the task. :param retries: Optional The number of retries for the task, defaults to 0, which means no retries. :param docs: Optional The documentation for the task, if not provided the function docstring will be used. :param timeout: Optional The timeout for the task. :param pod_template: Optional The pod template for the task, if not provided the default pod template will be used. :param report: Optional Whether to generate the html report for the task, defaults to False. :param max_inline_io_bytes: Maximum allowed size (in bytes) for all inputs and outputs passed directly to the task (e.g., primitives, strings, dicts). Does not apply to files, directories, or dataframes.

• Parameters

  • _func: Optional[Any]
    • Optional The function to decorate. If not provided, the decorator will return a callable that accepts a function to be decorated.
  • short_name: Optional[str]
    • Optional A friendly name for the task (defaults to the function name)
  • cache: CacheRequest | None
    • Optional The cache policy for the task, defaults to auto, which will cache the results of the task.
  • retries: Union[int, RetryStrategy]
    • Optional The number of retries for the task, defaults to 0, which means no retries.
  • timeout: Union[timedelta, int]
    • Optional The timeout for the task.
  • docs: Optional[Documentation]
    • Optional The documentation for the task, if not provided the function docstring will be used.
  • pod_template: Optional[Union[str, "V1PodTemplate"]]
    • Optional The pod template for the task, if not provided the default pod template will be used.
  • report: bool
    • Optional Whether to generate the html report for the task, defaults to False.
  • max_inline_io_bytes: int
    • Maximum allowed size (in bytes) for all inputs and outputs passed directly to the task (e.g., primitives, strings, dicts). Does not apply to files, directories, or dataframes.

• Return Value: Union[AsyncFunctionTaskTemplate, Callable[P, R]]

  • An AsyncFunctionTaskTemplate or a callable that accepts a function to be decorated.

def tasks()

• Get all tasks defined in the environment.

• Return Value: Dict[str, TaskTemplate]

  • A dictionary of all tasks in the environment, where keys are task names and values are TaskTemplate objects.

def add_task(task: TaskTemplate) - > TaskTemplate

• Add a task to the environment.

Useful when you want to add a task to an environment that is not defined using the task decorator.

:param task: The TaskTemplate to add to this environment.

• Parameters

  • task: TaskTemplate
    • The TaskTemplate to add to this environment.

• Return Value: TaskTemplate

  • The TaskTemplate that was added to the environment.