Configuring asyncmd

Configuring asyncmd#

Various functions can be used to configure asyncmd behavior during runtime. Most notably are probably the functions to limit resource use (i.e. number of concurrent SLURM jobs, number of open files, number of processes, etc.) and functions to influence the Trajectory CV value caching like setting the default cache type for all Trajectory or registering a h5py file (or group) for caching.

Show/print current configuration#

asyncmd.config.show_config() → None#

Print/show current configuration.

General resource usage#

asyncmd.config.set_max_process(num: int | None = None, max_num: int | None = None) → None#

Set the maximum number of concurrent python processes.

If num is None, default to os.cpu_count() / 4.

Parameters:

  • num (int, optional) – Number of processes, if None will default to 1/4 of the CPU count.

  • max_num (int, optional) – If given the number of processes can not exceed this number independent of the value of CPU count. Useful mostly for code that runs on multiple different machines (with different CPU counts) but still wants to avoid spawning hundreds of processes.

asyncmd.config.set_max_files_open(num: int | None = None, margin: int = 30) → None#

Set the maximum number of concurrently opened files.

By default use the systems soft resource limit.

Parameters:

  • num (int, optional) – Maximum number of open files, if None use systems (soft) resourcelimit, by default None

  • margin (int, optional) – Safe margin to keep, i.e. we will only ever open num - margin files, by default 30

Raises:

ValueError – If num <= margin.

SLURM settings and resource usage#

asyncmd.config.set_slurm_max_jobs(num: int | None) → None#

Set the maximum number of simultaneously submitted SLURM jobs.

Parameters:

num (int or None) – The maximum number of simultaneous SLURM jobs for this invocation of python/asyncmd. None means do not limit the maximum number of jobs.

asyncmd.config.set_slurm_settings(sinfo_executable: str | None = None, sacct_executable: str | None = None, sbatch_executable: str | None = None, scancel_executable: str | None = None, min_time_between_sacct_calls: int | None = None, num_fails_for_broken_node: int | None = None, success_to_fail_ratio: int | None = None, exclude_nodes: list[str] | None = None) → None#

Set single or multiple settings relevant for SLURM job control.

Call this function if you want to change e.g. the path/name of SLURM executables. This function only modifies thoose settings for which a value other than None is passed. See set_all_slurm_settings if you want to set/ modify all slurm settings and/or reset them to their defaults.

Parameters:

  • sinfo_executable (str, optional) – Name of path to the sinfo executable, by default None.

  • sacct_executable (str, optional) – Name or path to the sacct executable, by default None.

  • sbatch_executable (str, optional) – Name or path to the sbatch executable, by default None.

  • scancel_executable (str, optional) – Name or path to the scancel executable, by default None.

  • min_time_between_sacct_calls (int, optional) – Minimum time (in seconds) between subsequent sacct calls, by default None.

  • num_fails_for_broken_node (int, optional) – Number of failed jobs we need to observe per node before declaring it to be broken (and not submitting any more jobs to it), by default None.

  • success_to_fail_ratio (int, optional) – Number of successful jobs we need to observe per node to decrease the failed job counter by one, by default None.

  • exclude_nodes (list[str], optional) – List of nodes to exclude in job submissions, by default None, which results in no excluded nodes.

CV value caching#

asyncmd.config.set_trajectory_cache_type(cache_type: str, copy_content: bool = True, clear_old_cache: bool = False) → None#

Set the cache type for TrajectoryFunctionWrapper values.

By default the content of the current caches is copied to the new caches. To clear the old/previously set caches (after copying their values), pass clear_old_cache=True.

Parameters:

  • cache_type (str) – One of “h5py”, “npz”, “memory”.

  • copy_content (bool, optional) – Whether to copy the current cache content to the new cache, by default True

  • clear_old_cache (bool, optional) – Whether to clear the old/previously set cache, by default False.

Raises:

ValueError – Raised if cache_type is not one of the allowed values.

asyncmd.config.register_h5py_cache(h5py_group) → None#

Register a h5py file or group for CV value caching.

Note that this also sets the default cache type to “h5py”, i.e. it calls set_trajectory_cache_type() with cache_type="h5py".

Note that a h5py.File is just a slightly special h5py.Group, so you can pass either. asyncmd will use either the file or the group as the root of its own stored values. E.g. you will have h5py_group["asyncmd/TrajectoryFunctionValueCache"] always pointing to the cached trajectory values and if h5py_group is the top-level group (i.e. the file) you also have (file["/asyncmd/TrajectoryFunctionValueCache"] == h5py_group["asyncmd/TrajectoryFunctionValueCache"]).

Parameters:

h5py_group (h5py.Group or h5py.File) – The file or group to use for caching.

Contents