Distribute Task Execution (DTE)

Nx supports running commands across multiple machines. You can either set it up by hand (by using batching or binning) or use Nx Cloud.

Read the comparison of the two approaches.

Set up

To distribute your task execution, you need to (1) connect to Nx Cloud and (2) enable DTE in your CI workflow. Each of these steps can be enabled with a single command:

nx connect-to-nx-cloud

nx generate @nrwl/workspace:ci-workflow --ci=github

The --ci flag can be github, circleci or azure. For more details on setting up DTE, read this guide.

CI Execution Flow

There are two main parts to the CI set up:

  1. The main job that controls what is going to be executed
  2. The agent jobs that actually execute the tasks

The main job execution flow looks like this:

1# Coordinate the agents to run the tasks
2- npx nx-cloud start-ci-run
3# Run any commands you want here
4- nx affected --target=lint
5- nx affected --target=test
6- nx affected --target=build
7# Stop any run away agents
8- npx nx-cloud stop-all-agents
9

The agent job execution flow is very simple:

1# Wait for tasks to execute
2- npx nx-cloud start-agent
3

Illustrated Guide

For more details about how distributed task execution works, check out the illustrated guide by Nrwlian Nicole Oliver.

how does distributed task execution work in Nx Cloud?

Recipes

Reference