Workflow Processing

This document gives an overview of how Federated Workflow Processing in TREs is achieved in Five Safes TES, and provides guidance on running analysis using the Five Safes TES Tools.

Overview

Five Safes TES runs sequential workflows defined as GA4GH TES Tasks, federating them to multiple TREs where they are executed.

This Sequence Diagram  shows the happy path by which Workflow Processing occurs using Five Safes TES.

It shows the actors involved, and the interactions between different components from the initialisation of analysis, through federation, execution (shown in a single TRE), and egress, up to the outputs being made available.

Guides

The Five Safes TES Documentation has guides for submitting analysis workflows via the Submission Layer’s TES API:

Get an Access Token

This guide describes how to retrieve an access token for use with the API:

Analysis Quickstart

The Analysis Quickstart guide simply runs a “Hello World” workflow, with one step:

Calculate a federated mean

This end-to-end guide uses a sample researcher analysis script written in python.

The script makes a Five Safes TES Submission to get the partial summary statistics for a mean calculation on an OMOP dataset.

Once the summary statistics have been returned, the script then performs the aggregation process to calculate a mean across the federated datasets.

It is expected that each Target TRE has an OMOP Dataset configured for the Project; the guide includes details on how to bootstrap a sample dataset.