ReusePolicy

ReusePolicy can be used to configure a task to reuse the environment. This is useful when the environment creation is expensive and the runtime of the task is short. The environment will be reused for the next invocation of the task, even the python process maybe be reused by subsequent task invocations. A good mental model is to think of the environment as a container that is reused for multiple tasks, more like a long-running service.< br >< br >Caution: It is important to note that the environment is shared, so managing memory and resources is important.

Attributes

• replicas: Union[int, Tuple[int, int]] = 2

  • Either a single int representing number of replicas or a tuple of two ints representing the min and max.

• idle_ttl: Union[int, timedelta] = 30

  • The maximum idle duration for an environment, specified as either seconds (int) or a timedelta, after which all replicas in the environment are shutdown. If not set, the default is configured in the backend (can be as low as 90s). When a replica remains idle — meaning no tasks are running — for this duration, it will be automatically terminated, also referred to as environment idle timeout.

• concurrency: int = 1

  • The maximum number of tasks that can run concurrently in one instance of the environment. Concurrency of greater than 1 is only supported for async tasks.

• scaledown_ttl: Union[int, timedelta] = 30

  • The minimum time to wait before scaling down each replica, specified as either seconds (int) or a timedelta. This is useful to prevent rapid scaling down of replicas when tasks are running frequently. If not set, the default is configured in the backend.

Constructors

• Initializes the ReusePolicy with configuration for environment reuse.

Args: replicas: Either a single int representing the number of replicas or a tuple of two ints representing the min and max replicas. idle_ttl: The maximum idle duration for an environment in seconds or as a timedelta, after which all replicas are shut down. Defaults to 30 seconds. concurrency: The maximum number of tasks that can run concurrently in one instance of the environment. Defaults to 1. scaledown_ttl: The minimum time to wait before scaling down each replica in seconds or as a timedelta. Defaults to 30 seconds.

• Parameters

  • replicas: Union[int, Tuple[int, int]]
    • Either a single int representing number of replicas or a tuple of two ints representing the min and max.
  • idle_ttl: Union[int, timedelta]
    • The maximum idle duration for an environment, specified as either seconds (int) or a timedelta, after which all replicas in the environment are shutdown. If not set, the default is configured in the backend (can be as low as 90s). When a replica remains idle — meaning no tasks are running — for this duration, it will be automatically terminated, also referred to as environment idle timeout.
  • concurrency: int
    • The maximum number of tasks that can run concurrently in one instance of the environment. Concurrency of greater than 1 is only supported for async tasks.
  • scaledown_ttl: Union[int, timedelta]
    • The minimum time to wait before scaling down each replica, specified as either seconds (int) or a timedelta. This is useful to prevent rapid scaling down of replicas when tasks are running frequently. If not set, the default is configured in the backend.

Methods

• Performs post-initialization validation and conversion of parameters.

• Return Value: None

  • This method does not return any value.

• Returns the idle TTL as a timedelta. If idle_ttl is not set, returns the global default.

• Return Value: timedelta | None

  • The idle TTL as a timedelta object or None if not set.

• Returns the minimum number of replicas.

• Return Value: int

  • The minimum number of replicas.

• Returns the scaledown TTL as a timedelta. If scaledown_ttl is not set, returns None.

• Return Value: timedelta | None

  • The scaledown TTL as a timedelta object or None if not set.

• Returns the maximum number of replicas.

• Return Value: int

  • The maximum number of replicas.