aces.cluster_configs.clusters

Utilies to get cluster configuration files

Module Contents

Functions

list_packaged_clusters(→ list[str])	Return a list of cluster names that are available in the packaged set of
get_cluster_spec(→ dict[Any, Any])	Given a cluster name, obtain the appropriate SLURM configuration
get_dask_runner(→ prefect_dask.DaskTaskRunner)	Creates and returns a DaskTaskRunner configured to establised a SLURMCluster instance

aces.cluster_configs.clusters.list_packaged_clusters() → list[str]

Return a list of cluster names that are available in the packaged set of dask_jobqueue specification YAML files.

Returns:

list[str]: A list of preinstalled dask_jobqueue cluster specification files

aces.cluster_configs.clusters.get_cluster_spec(cluster: str) → dict[Any, Any]

Given a cluster name, obtain the appropriate SLURM configuration file appropriate for use with SLURMCluster.

This cluster spec is expected to be consistent with the cluster_class and cluster_kwargs parameters that are used by dask_jobqueue based specifications.

Args:

cluster (str): Name of cluster to look up for processing

Raises:

ValueError: Raised when cluster is not in KNOWN_CLUSTERS and has not corresponding YAML file.

Returns:

dict[Any, Any]: Dictionary of know options/parameters for dask_jobqueue.SLURMCluster

aces.cluster_configs.clusters.get_dask_runner(cluster: str = 'galaxy_small', extra_cluster_kwargs: dict = None) → prefect_dask.DaskTaskRunner

Creates and returns a DaskTaskRunner configured to establised a SLURMCluster instance to manage a set of dask-workers. The SLURMCluster is currently configured only for Galaxy.

Keyword Args:

cluster (str): The cluster name that will be used to search for a cluster specification file.

This could be the name of a known cluster, or the name of a yaml file installed amoung the cluster_configs directory of the aces module.

Returns:

DaskTaskRunner: A dask task runner capable of being used as a task_runner for a prefect flow