Portable Submission Interface for Jobs

PSI/J is a portability layer across different HPC workload managers allowing workflow developers and users to create portable workflows with a standard API. PSI/J aims to focus effort by pooling into a single layer the collective knowledge obtained from these disparate efforts. PSI/J is composed of both a language-agnostic community-defined specification and the specification's language-specific implementations. The high-level goals of the specification are to be lightweight, user-space, minimally prescriptive, scalable via asynchronous operations, general, and extensible to different systems, schedulers, implementations.

The PSI/J specification is broken into three layers that each build on one another:

• The first layer forms the base of the specification and focuses on supporting the launching and monitoring of "local" jobs (i.e., the client is running on the same system or cluster as the jobs).
• The second layer builds on the first to provide support for "remote" jobs (i.e., the client must connect over the network, potentially with authentication, to launch and monitor jobs), managing multiple clusters simultaneously, and allowing file staging.
• The third layer builds on the previous two to support efficient execution of small jobs through the "nested", "pilot job", or "jobs-in-jobs" paradigm.