jobflow

Official Resources

• Homepage: https://materialsproject.github.io/jobflow/
• Documentation: https://materialsproject.github.io/jobflow/
• Source Repository: https://github.com/materialsproject/jobflow
• License: BSD 3-Clause License

Overview

Jobflow is a free, open-source library for writing computational workflows. It is intended to be the successor to the workflow primitives in FireWorks. Jobflow separates the definition of the workflow (Jobs and Flows) from the execution engine (JobStore). This allows workflows to be run locally (for development/debugging) or deployed to remote managers like FireWorks without changing the code.

Scientific domain: Workflow definition, computational pipelines
Target user community: Workflow developers, Atomate2 users

Capabilities (CRITICAL)

• Separation of Concerns: Decouples "what to run" from "how to run it".
• Composability: Easy to combine Jobs into Flows.
• Dynamic: Supports dynamic workflows (Outputs of one job determining the next job).
• Local Execution: Can run complex flows on a laptop without a database.
• Serialization: robust serialization of Python objects.

Sources: Jobflow documentation

Inputs & Outputs

• Input formats: Python Job objects
• Output data types: JSON/BSON (via Maggma stores)

Interfaces & Ecosystem

• atomate2: Built entirely on jobflow.
• FireWorks: Can act as a remote execution backend for jobflow.
• Maggma: Used for data storage.

Workflow and Usage

1. Define a job function:

   @job
   def add(a, b):
       return a + b
   

2. Create flow:

   j1 = add(1, 2)
   j2 = add(j1.output, 3)
   flow = Flow([j1, j2])
   

3. Run: run_locally(flow)

Performance Characteristics

• Optimized for flexibility and development speed
• Low overhead for local execution

Application Areas

• Developing new computational workflows
• Atomate2 development
• General python pipeline management

Community and Support

• Developed by Materials Project
• Part of the next-generation MP stack

Verification & Sources

Primary sources:

1. Homepage: https://materialsproject.github.io/jobflow/
2. GitHub: https://github.com/materialsproject/jobflow

Confidence: VERIFIED

Verification status: ✅ VERIFIED

• Website: ACTIVE
• Documentation: COMPREHENSIVE
• Source: OPEN (GitHub)
• Development: ACTIVE
• Applications: Workflow definition, dynamic flows