psij.launchers package

Subpackages

• psij.launchers.scripts package
  • Module contents

Submodules

psij.launchers.aprun module

A module for AprunLauncher.

class AprunLauncher(config=None)

Bases: MultipleLauncher

Launches a job using Cobalt’s aprun.

Parameters

config (Optional[JobExecutorConfig]) – An optional configuration.

psij.launchers.jsrun module

A module for JsrunLauncher.

class JsrunLauncher(config=None)

Bases: MultipleLauncher

Launches a job using LSF’s jsrun.

Parameters

config (Optional[JobExecutorConfig]) – An optional configuration.

psij.launchers.mpirun module

A module for MPILauncher.

class MPILauncher(config=None)

Bases: MultipleLauncher

Launches jobs using mpirun.

mpirun is a tool provided by MPI implementations, such as Open MPI.

Parameters

config (Optional[JobExecutorConfig]) – An optional configuration.

psij.launchers.multiple module

A module containing the MultipleLauncher.

class MultipleLauncher(script_path=PosixPath('/home/runner/work/psij-python/psij-python/src/psij/launchers/scripts/multi_launch.sh'), config=None)

Bases: ScriptBasedLauncher

A launcher that launches multiple identical copies of the executable.

The exit code of the job corresponds to the first non-zero exit code encountered in one of the executable copies or zero if all invocations of the executable succeed.

Parameters

• config (Optional[JobExecutorConfig]) – An optional configuration.

• script_path (Path) –

get_additional_args(job)

See get_additional_args().

Parameters

job (Job) –

Return type

List[str]

psij.launchers.script_based_launcher module

class ScriptBasedLauncher(script_path, config=None)

Bases: Launcher

A launcher that uses a script to start the job, possibly by wrapping it in other tools.

This launcher is an abstract base class for launchers that wrap the job in a script. The script must be a bash script and is invoked with the first four parameters as:

• the job ID

• a launcher log file, which is taken from the launcher_log_file configuration setting and defaults to /dev/null

• the pre- and post- launcher scripts, or empty strings if they are not specified

Additional positional arguments to the script can be specified by subclasses by overriding the get_additional_args() method.

The remaining arguments to the script are the job executable and arguments.

A simple script library is provided in scripts/launcher_lib.sh. Its use is optional and it is intended to be included at the beginning of a main launcher script using source $(dirname “$0”)/launcher_lib.sh. It does the following:

• sets ‘-e’ mode (exit on error)

• sets the variables _PSI_J_JOB_ID, _PSI_J_LOG_FILE, _PSI_J_PRE_LAUNCH, and _PSI_J_POST_LAUNCH from the first arguments, as specified above.

• saves the current stdout and stderr in descriptors 3 and 4, respectively

• redirects stdout and stderr to the log file, while prepending a timestamp and the job ID to each line

• defines the commands “pre_launch” and “post_launch”, which can be invoked by the main script.

When invoking the job executable (either directly or through a launch command), it is recommended that the stdout and stderr of the job process be redirected to descriptors 3 and 4, respectively, such that they can be captured by the entity invoking the launcher rather than ending up in a the launcher log file.

A successful completion of the launcher should be signalled by the launcher by printing the string “_PSI_J_LAUNCHER_DONE” to stdout. The launcher can then exit with the exit code returned by the launched command. This allows executor to distinguish between a non-zero exit code due to application failure or due to a premature launcher failure.

The actual launcher scripts, as well as the library, are deployed at run-time into the work directory, where submit scripts are also generated. This directory is meant to be accessible by both the node submitting the job as well as the node launching the job.

Parameters

• script_path (Path) – A path to a script that is invoked as described above.

• config (Optional[JobExecutorConfig]) – An optional configuration.

Return type

None

get_additional_args(job)

Returns any additional arguments, after first mandatory four, to be passed to the script.

Parameters

job (Job) – The job that is being launched.

Return type

List[str]

get_launch_command(job, log_file=None)

See get_launch_command().

Parameters

• job (Job) –

• log_file (Optional[str]) –

Return type

List[str]

psij.launchers.single module

A module containing the SingleLauncher.

class SingleLauncher(config=None)

Bases: ScriptBasedLauncher

A launcher that launches a single copy of the executable. This is the default launcher.

Parameters

config (Optional[JobExecutorConfig]) – An optional configuration.

psij.launchers.srun module

A module for MPILauncher.

class SrunLauncher(config=None)

Bases: MultipleLauncher

Launches a job using Slurm’s srun.

See the Slurm Workload Manager.

Parameters

config (Optional[JobExecutorConfig]) – An optional configuration.

Module contents

This package contains all launcher implementations.

Launcher implementations must be in this package in order for them to be registered automatically. Alternatively, you may register a launcher implementation using psij.job_launcher.Launcher.register_launcher().

class AprunLauncher(config=None)

Bases: MultipleLauncher

Launches a job using Cobalt’s aprun.

Parameters

config (Optional[JobExecutorConfig]) – An optional configuration.

class JsrunLauncher(config=None)

Bases: MultipleLauncher

Launches a job using LSF’s jsrun.

Parameters

config (Optional[JobExecutorConfig]) – An optional configuration.

class MPILauncher(config=None)

Bases: MultipleLauncher

Launches jobs using mpirun.

mpirun is a tool provided by MPI implementations, such as Open MPI.

Parameters

config (Optional[JobExecutorConfig]) – An optional configuration.

class MultipleLauncher(script_path=PosixPath('/home/runner/work/psij-python/psij-python/src/psij/launchers/scripts/multi_launch.sh'), config=None)

Bases: ScriptBasedLauncher

A launcher that launches multiple identical copies of the executable.

The exit code of the job corresponds to the first non-zero exit code encountered in one of the executable copies or zero if all invocations of the executable succeed.

Parameters

• config (Optional[JobExecutorConfig]) – An optional configuration.

• script_path (Path) –

get_additional_args(job)

See get_additional_args().

Parameters

job (Job) –

Return type

List[str]

class SingleLauncher(config=None)

Bases: ScriptBasedLauncher

A launcher that launches a single copy of the executable. This is the default launcher.

Parameters

config (Optional[JobExecutorConfig]) – An optional configuration.

class SrunLauncher(config=None)

Bases: MultipleLauncher

Launches a job using Slurm’s srun.

See the Slurm Workload Manager.

Parameters

config (Optional[JobExecutorConfig]) – An optional configuration.