Pattern 25: Cross-System Workflows

Part III: Integration Patterns

Opening Scenario: The Order That Touched Seven Systems

Emma clicked "Place Order" on her company's e-commerce site at 2:47 PM:

Order #58392
Samsung 55" TV - $649.99
Shipping: 2-Day - $24.99
Tax: $54.60
Total: $729.58

[Place Order]

She expected the order to be processed. Instead, she got:

✓ Order Received

We'll send you a confirmation email shortly.

But behind the scenes, chaos:

The developer, Mike, had written the order processing code:

// Bad: Manual coordination across systems
app.post('/api/orders', async (req, res) => {
  // 1. Save order
  const order = await db.insert('orders', req.body);

  // 2. Check inventory
  const inStock = await inventoryAPI.checkStock(req.body.productId);
  if (!inStock) {
    await db.update('orders', order.id, { status: 'CANCELLED' });
    return res.json({ error: 'Out of stock' });
  }

  // 3. Process payment
  const payment = await stripeAPI.charge({
    amount: req.body.total,
    card: req.body.cardToken
  });

  if (!payment.success) {
    await db.update('orders', order.id, { status: 'PAYMENT_FAILED' });
    return res.json({ error: 'Payment failed' });
  }

  // 4. Reserve inventory
  await inventoryAPI.reserve(req.body.productId, order.id);

  // 5. Create shipment
  await shippingAPI.createShipment({
    orderId: order.id,
    address: req.body.shippingAddress
  });

  // 6. Update CRM
  await crmAPI.updateCustomer(req.body.customerId, {
    lastOrder: order.id,
    totalSpent: customer.totalSpent + req.body.total
  });

  // 7. Send confirmation email
  await emailAPI.send({
    to: req.body.email,
    template: 'order-confirmation',
    data: order
  });

  res.json({ success: true, orderId: order.id });
});

This code had seven problems:

Problem 1: No error recovery - Payment succeeds - Shipment API fails - Customer charged but no shipment created - Money taken, no product shipped

Problem 2: No rollback - Inventory reserved - Payment fails - Inventory still reserved - Product locked, no sale made

Problem 3: Synchronous blocking - Wait for all 7 API calls - Each takes 2-3 seconds - User waits 15+ seconds - Terrible UX

Problem 4: No retry - CRM API timeout - Customer data not updated - No retry attempted - Lost information

Problem 5: No visibility - Where is order in process? - Which step failed? - Why did it fail? - Unknown!

Problem 6: Can't pause/resume - Process runs to completion or fails - Can't wait for manual approval - Can't resume after fix - All or nothing

Problem 7: No audit trail - What happened when? - Who did what? - Why did it fail? - No record

The architect, Lisa, saw the mess. "We need workflow orchestration."

She designed a proper cross-system workflow:

class WorkflowEngine {
  constructor() {
    this.workflows = new Map();
    this.executions = new Map();
  }

  defineWorkflow(name, definition) {
    this.workflows.set(name, {
      name: name,
      steps: definition.steps,
      errorHandling: definition.errorHandling || {},
      compensations: definition.compensations || {}
    });
  }

  async execute(workflowName, context) {
    const workflow = this.workflows.get(workflowName);

    const execution = {
      id: this.generateExecutionId(),
      workflowName: workflowName,
      context: context,
      steps: [],
      status: 'running',
      startedAt: new Date()
    };

    this.executions.set(execution.id, execution);

    try {
      // Execute each step
      for (const step of workflow.steps) {
        await this.executeStep(execution, step);
      }

      execution.status = 'completed';
      execution.completedAt = new Date();

    } catch (error) {
      execution.status = 'failed';
      execution.error = error.message;

      // Run compensations (rollback)
      await this.compensate(execution, workflow);
    }

    return execution;
  }

  async executeStep(execution, step) {
    const stepExecution = {
      name: step.name,
      status: 'running',
      startedAt: new Date(),
      attempts: 0
    };

    execution.steps.push(stepExecution);

    try {
      const result = await step.execute(execution.context);

      stepExecution.status = 'completed';
      stepExecution.completedAt = new Date();
      stepExecution.result = result;

      // Update context with result
      execution.context[step.outputKey] = result;

    } catch (error) {
      stepExecution.status = 'failed';
      stepExecution.error = error.message;

      throw error;
    }
  }

  async compensate(execution, workflow) {
    // Run compensation actions in reverse order
    const completedSteps = execution.steps
      .filter(s => s.status === 'completed')
      .reverse();

    for (const step of completedSteps) {
      const compensation = workflow.compensations[step.name];

      if (compensation) {
        try {
          await compensation(execution.context, step.result);
        } catch (error) {
          console.error(`Compensation failed for ${step.name}:`, error);
        }
      }
    }
  }
}

Now the order workflow was clean:

// Define order processing workflow
workflowEngine.defineWorkflow('process-order', {
  steps: [
    {
      name: 'validate-order',
      execute: async (ctx) => {
        return await orderValidator.validate(ctx.order);
      },
      outputKey: 'validation'
    },
    {
      name: 'check-inventory',
      execute: async (ctx) => {
        return await inventoryAPI.checkStock(ctx.order.productId);
      },
      outputKey: 'inventoryCheck'
    },
    {
      name: 'reserve-inventory',
      execute: async (ctx) => {
        return await inventoryAPI.reserve(
          ctx.order.productId,
          ctx.order.id
        );
      },
      outputKey: 'reservation'
    },
    {
      name: 'process-payment',
      execute: async (ctx) => {
        return await stripeAPI.charge({
          amount: ctx.order.total,
          card: ctx.order.cardToken
        });
      },
      outputKey: 'payment'
    },
    {
      name: 'create-shipment',
      execute: async (ctx) => {
        return await shippingAPI.createShipment({
          orderId: ctx.order.id,
          address: ctx.order.shippingAddress
        });
      },
      outputKey: 'shipment'
    },
    {
      name: 'update-crm',
      execute: async (ctx) => {
        return await crmAPI.updateCustomer(
          ctx.order.customerId,
          { lastOrder: ctx.order.id }
        );
      },
      outputKey: 'crmUpdate'
    },
    {
      name: 'send-confirmation',
      execute: async (ctx) => {
        return await emailAPI.send({
          to: ctx.order.email,
          template: 'order-confirmation',
          data: ctx.order
        });
      },
      outputKey: 'confirmation'
    }
  ],

  // Rollback actions if workflow fails
  compensations: {
    'reserve-inventory': async (ctx, result) => {
      await inventoryAPI.release(result.reservationId);
    },
    'process-payment': async (ctx, result) => {
      await stripeAPI.refund(result.chargeId);
    },
    'create-shipment': async (ctx, result) => {
      await shippingAPI.cancelShipment(result.shipmentId);
    }
  }
});

// Execute workflow
app.post('/api/orders', async (req, res) => {
  const order = await db.insert('orders', req.body);

  // Start workflow asynchronously
  workflowEngine.execute('process-order', { order })
    .catch(err => console.error('Order workflow failed:', err));

  // Respond immediately
  res.json({ 
    success: true, 
    orderId: order.id,
    status: 'processing'
  });
});

Benefits: - ✅ Automatic rollback - Payment fails? Inventory released automatically - ✅ Async processing - User gets instant response - ✅ Retry logic - Steps retry on failure - ✅ Full visibility - See exactly where order is in workflow - ✅ Audit trail - Complete history of every step - ✅ Error recovery - Can resume from failure point - ✅ Compensation - Automatic cleanup on failure

Seven systems. One workflow. Perfect orchestration.

Context

Cross-System Workflows applies when:

Multi-step processes: Orders, applications, onboarding require many steps

Multiple systems involved: CRM, payment, inventory, shipping, email, etc.

Transactional integrity needed: Either all steps succeed or all rollback

Long-running processes: May take minutes, hours, or days

Human approval required: Pause for manager approval, resume after

Error recovery critical: Must handle failures gracefully

Audit trail required: Track every step for compliance

Complex business logic: Conditional branching, parallel execution, loops

Problem Statement

Most systems handle multi-system processes poorly:

Monolithic code:

// Bad: Everything in one function
async function processLoan(application) {
  await creditCheck(application);
  await incomeVerification(application);
  await propertyAppraisal(application);
  await underwriting(application);
  await documentGeneration(application);
  await eSignature(application);
  await funding(application);

  // 7 API calls
  // No error handling
  // No rollback
  // No visibility
}

No compensation:

// Bad: Partial completion leaves mess
try {
  await reserveInventory(item);
  await chargeCard(payment);
  await createShipment(order);
} catch (error) {
  // Inventory reserved, card charged
  // Shipment failed
  // How to clean up? Unknown!
}

No state management:

// Bad: Where are we in the process?
const order = await db.get(orderId);

// Status: "processing"
// But which step?
// Payment done? Shipment created?
// Unknown!

No parallel execution:

// Bad: Sequential when could be parallel
await sendEmailNotification(order);  // 2 seconds
await sendSMSNotification(order);     // 2 seconds
await updateAnalytics(order);         // 2 seconds

// Total: 6 seconds
// Could be 2 seconds if parallel

No human tasks:

// Bad: Can't pause for approval
async function processExpense(expense) {
  await validateExpense(expense);
  // Need manager approval here
  // But can't pause!
  await processPayment(expense);
}

We need workflow orchestration that handles multi-step processes, coordinates systems, manages state, handles errors, supports rollback, and provides visibility.

Forces

Consistency vs Availability

• Strong consistency requires locks
• But locks reduce availability
• Balance correctness with performance

Synchronous vs Asynchronous

• Sync is simpler to reason about
• But async scales better
• Balance simplicity with scalability

Coupling vs Coordination

• Tight coupling enables coordination
• But loose coupling enables independence
• Balance integration with autonomy

Retry vs Idempotency

• Retrying ensures completion
• But must handle duplicate execution
• Balance reliability with correctness

Complexity vs Capability

• Rich workflows very powerful
• But complex to build and maintain
• Balance features with simplicity

Solution

Implement workflow orchestration engine that defines multi-step processes as composable steps, executes them with proper error handling and rollback, manages workflow state and history, supports parallel execution and human tasks, provides comprehensive monitoring and debugging, and ensures transactional integrity across distributed systems.

The pattern has seven key strategies:

1. Workflow Definition Language

Define workflows declaratively:

class WorkflowDefinition {
  static define(config) {
    return {
      name: config.name,
      version: config.version || 1,
      description: config.description,

      input: config.input || {},
      output: config.output || {},

      steps: config.steps,

      errorHandling: {
        retryPolicy: config.retryPolicy || 'exponential',
        maxRetries: config.maxRetries || 3,
        timeout: config.timeout || 300000 // 5 minutes
      },

      compensations: config.compensations || {}
    };
  }

  static sequential(steps) {
    return {
      type: 'sequential',
      steps: steps
    };
  }

  static parallel(steps) {
    return {
      type: 'parallel',
      steps: steps
    };
  }

  static conditional(condition, thenSteps, elseSteps) {
    return {
      type: 'conditional',
      condition: condition,
      then: thenSteps,
      else: elseSteps
    };
  }
}

// Example: Loan application workflow
const loanWorkflow = WorkflowDefinition.define({
  name: 'loan-application',
  description: 'Process loan application from submission to funding',

  input: {
    applicant: 'object',
    loanAmount: 'number',
    propertyAddress: 'string'
  },

  steps: WorkflowDefinition.sequential([
    {
      name: 'credit-check',
      service: 'credit-bureau',
      action: 'checkCredit',
      input: '$.applicant',
      output: 'creditReport'
    },

    WorkflowDefinition.conditional(
      '$.creditReport.score >= 680',

      // Good credit - continue
      WorkflowDefinition.sequential([
        {
          name: 'income-verification',
          service: 'verification-service',
          action: 'verifyIncome',
          input: '$.applicant'
        },
        {
          name: 'property-appraisal',
          service: 'appraisal-service',
          action: 'scheduleAppraisal',
          input: '$.propertyAddress'
        },

        // Run underwriting and doc prep in parallel
        WorkflowDefinition.parallel([
          {
            name: 'underwriting',
            service: 'underwriting-service',
            action: 'underwrite',
            input: '$'
          },
          {
            name: 'document-generation',
            service: 'document-service',
            action: 'generateLoanDocs',
            input: '$'
          }
        ]),

        // Human approval required
        {
          name: 'manager-approval',
          type: 'human-task',
          assignee: 'loan-manager',
          timeout: 86400000 // 24 hours
        },

        {
          name: 'e-signature',
          service: 'docusign',
          action: 'sendForSignature',
          input: '$.documents'
        },

        {
          name: 'funding',
          service: 'funding-service',
          action: 'disburseFunds',
          input: '$.loanAmount'
        }
      ]),

      // Poor credit - reject
      WorkflowDefinition.sequential([
        {
          name: 'send-rejection',
          service: 'email-service',
          action: 'sendRejectionEmail',
          input: '$.applicant.email'
        }
      ])
    )
  ]),

  compensations: {
    'funding': async (ctx) => {
      await fundingService.reverseDisbursement(ctx.funding.transactionId);
    }
  }
});

2. Workflow Execution Engine

Execute workflows with state management:

class WorkflowExecutionEngine {
  constructor(database, serviceRegistry) {
    this.db = database;
    this.services = serviceRegistry;
    this.executions = new Map();
  }

  async startWorkflow(workflowDef, input, options = {}) {
    const execution = {
      id: this.generateExecutionId(),
      workflowName: workflowDef.name,
      workflowVersion: workflowDef.version,

      input: input,
      context: { ...input },
      output: null,

      status: 'running',
      currentStep: null,
      completedSteps: [],

      startedAt: new Date(),
      updatedAt: new Date(),
      completedAt: null,

      metadata: options.metadata || {}
    };

    // Persist execution
    await this.saveExecution(execution);

    // Start execution
    this.executeWorkflow(execution, workflowDef)
      .catch(err => this.handleWorkflowError(execution, err));

    return execution;
  }

  async executeWorkflow(execution, workflowDef) {
    try {
      const result = await this.executeSteps(
        execution,
        workflowDef.steps,
        execution.context
      );

      execution.output = result;
      execution.status = 'completed';
      execution.completedAt = new Date();

      await this.saveExecution(execution);

    } catch (error) {
      execution.status = 'failed';
      execution.error = error.message;
      execution.failedAt = new Date();

      // Run compensations
      await this.runCompensations(execution, workflowDef);

      await this.saveExecution(execution);

      throw error;
    }
  }

  async executeSteps(execution, steps, context) {
    if (steps.type === 'sequential') {
      return await this.executeSequential(execution, steps.steps, context);
    } else if (steps.type === 'parallel') {
      return await this.executeParallel(execution, steps.steps, context);
    } else if (steps.type === 'conditional') {
      return await this.executeConditional(execution, steps, context);
    } else if (Array.isArray(steps)) {
      return await this.executeSequential(execution, steps, context);
    } else {
      return await this.executeStep(execution, steps, context);
    }
  }

  async executeSequential(execution, steps, context) {
    for (const step of steps) {
      const result = await this.executeSteps(execution, step, context);

      if (step.output) {
        context[step.output] = result;
      }
    }

    return context;
  }

  async executeParallel(execution, steps, context) {
    const results = await Promise.all(
      steps.map(step => this.executeSteps(execution, step, context))
    );

    return results;
  }

  async executeConditional(execution, conditional, context) {
    const conditionMet = this.evaluateCondition(
      conditional.condition,
      context
    );

    if (conditionMet) {
      return await this.executeSteps(execution, conditional.then, context);
    } else if (conditional.else) {
      return await this.executeSteps(execution, conditional.else, context);
    }
  }

  async executeStep(execution, step, context) {
    execution.currentStep = step.name;
    execution.updatedAt = new Date();

    const stepExecution = {
      name: step.name,
      status: 'running',
      startedAt: new Date()
    };

    try {
      let result;

      if (step.type === 'human-task') {
        result = await this.executeHumanTask(execution, step, context);
      } else {
        result = await this.executeServiceCall(execution, step, context);
      }

      stepExecution.status = 'completed';
      stepExecution.completedAt = new Date();
      stepExecution.result = result;

      execution.completedSteps.push(stepExecution);

      return result;

    } catch (error) {
      stepExecution.status = 'failed';
      stepExecution.error = error.message;
      stepExecution.failedAt = new Date();

      execution.completedSteps.push(stepExecution);

      throw error;
    }
  }

  async executeServiceCall(execution, step, context) {
    const service = this.services.get(step.service);
    const input = this.resolveInput(step.input, context);

    return await service[step.action](input);
  }

  async executeHumanTask(execution, step, context) {
    // Create task and wait for completion
    const task = await this.createHumanTask(execution.id, step, context);

    // Pause execution, wait for task completion
    execution.status = 'waiting-for-human';
    execution.waitingFor = task.id;
    await this.saveExecution(execution);

    // Will be resumed when task is completed
    return new Promise((resolve) => {
      this.pendingHumanTasks.set(task.id, resolve);
    });
  }

  resolveInput(inputPath, context) {
    if (inputPath === '$') {
      return context;
    } else if (inputPath.startsWith('$.')) {
      const path = inputPath.substring(2);
      return this.getNestedValue(context, path);
    } else {
      return inputPath;
    }
  }
}

3. Saga Pattern for Distributed Transactions

Implement compensation-based transactions:

class SagaOrchestrator {
  constructor() {
    this.sagas = new Map();
  }

  defineSaga(name, steps) {
    this.sagas.set(name, {
      name: name,
      steps: steps
    });
  }

  async executeSaga(sagaName, context) {
    const saga = this.sagas.get(sagaName);
    const execution = {
      sagaName: sagaName,
      context: context,
      completedSteps: [],
      status: 'running'
    };

    try {
      // Execute each step
      for (const step of saga.steps) {
        const result = await step.transaction(context);

        execution.completedSteps.push({
          step: step.name,
          result: result,
          compensation: step.compensation
        });

        // Update context with result
        context[step.name] = result;
      }

      execution.status = 'completed';
      return execution;

    } catch (error) {
      execution.status = 'compensating';
      execution.error = error;

      // Run compensations in reverse order
      await this.compensate(execution);

      execution.status = 'compensated';
      throw error;
    }
  }

  async compensate(execution) {
    const completedSteps = [...execution.completedSteps].reverse();

    for (const step of completedSteps) {
      if (step.compensation) {
        try {
          await step.compensation(
            execution.context,
            step.result
          );
        } catch (compError) {
          console.error(
            `Compensation failed for ${step.step}:`,
            compError
          );
        }
      }
    }
  }
}

// Example: Order saga
sagaOrchestrator.defineSaga('order-processing', [
  {
    name: 'reserve-inventory',
    transaction: async (ctx) => {
      return await inventoryService.reserve(ctx.productId, ctx.quantity);
    },
    compensation: async (ctx, result) => {
      await inventoryService.release(result.reservationId);
    }
  },
  {
    name: 'charge-payment',
    transaction: async (ctx) => {
      return await paymentService.charge(ctx.amount, ctx.cardToken);
    },
    compensation: async (ctx, result) => {
      await paymentService.refund(result.chargeId);
    }
  },
  {
    name: 'create-shipment',
    transaction: async (ctx) => {
      return await shippingService.createShipment(ctx.orderId, ctx.address);
    },
    compensation: async (ctx, result) => {
      await shippingService.cancelShipment(result.shipmentId);
    }
  }
]);

4. Human Tasks & Approvals

Support human interaction in workflows:

class HumanTaskManager {
  constructor(database) {
    this.db = database;
    this.pendingTasks = new Map();
  }

  async createTask(workflowExecutionId, taskDef, context) {
    const task = {
      id: this.generateTaskId(),
      workflowExecutionId: workflowExecutionId,
      name: taskDef.name,
      type: taskDef.type || 'approval',
      assignee: taskDef.assignee,

      context: context,
      formData: this.buildFormData(taskDef, context),

      status: 'pending',
      createdAt: new Date(),
      dueAt: taskDef.timeout 
        ? new Date(Date.now() + taskDef.timeout)
        : null
    };

    await this.db.insert('human_tasks', task);

    // Notify assignee
    await this.notifyAssignee(task);

    return task;
  }

  async completeTask(taskId, decision, data) {
    const task = await this.db.get('human_tasks', taskId);

    task.status = 'completed';
    task.decision = decision;
    task.data = data;
    task.completedAt = new Date();

    await this.db.update('human_tasks', taskId, task);

    // Resume workflow
    const resolver = this.pendingTasks.get(taskId);
    if (resolver) {
      resolver({ decision, data });
      this.pendingTasks.delete(taskId);
    }

    return task;
  }

  async getTasksForUser(userId) {
    return await this.db.query(`
      SELECT * FROM human_tasks
      WHERE assignee = ?
        AND status = 'pending'
      ORDER BY createdAt DESC
    `, [userId]);
  }

  buildFormData(taskDef, context) {
    // Build form for human to complete
    return {
      fields: taskDef.fields || [],
      data: context
    };
  }
}

5. Workflow Monitoring & Debugging

Track workflow execution:

class WorkflowMonitoring {
  async getExecutionStatus(executionId) {
    const execution = await this.db.get('workflow_executions', executionId);

    return {
      id: execution.id,
      workflow: execution.workflowName,
      status: execution.status,

      progress: {
        completedSteps: execution.completedSteps.length,
        totalSteps: this.countTotalSteps(execution),
        currentStep: execution.currentStep
      },

      timing: {
        startedAt: execution.startedAt,
        duration: Date.now() - execution.startedAt,
        estimatedCompletion: this.estimateCompletion(execution)
      },

      steps: execution.completedSteps.map(step => ({
        name: step.name,
        status: step.status,
        duration: step.completedAt - step.startedAt,
        error: step.error
      }))
    };
  }

  async visualizeWorkflow(executionId) {
    const execution = await this.db.get('workflow_executions', executionId);

    return {
      nodes: this.buildNodes(execution),
      edges: this.buildEdges(execution),
      currentNode: execution.currentStep
    };
  }

  async getFailedExecutions(workflowName, timeRange) {
    return await this.db.query(`
      SELECT *
      FROM workflow_executions
      WHERE workflowName = ?
        AND status = 'failed'
        AND startedAt >= ?
        AND startedAt <= ?
      ORDER BY startedAt DESC
    `, [workflowName, timeRange.start, timeRange.end]);
  }
}

6. Event-Driven Workflow Triggers

Start workflows from events:

class WorkflowTriggers {
  constructor(workflowEngine, eventBus) {
    this.engine = workflowEngine;
    this.eventBus = eventBus;
    this.triggers = new Map();
  }

  defineTrigger(eventName, workflowName, transformer) {
    this.triggers.set(eventName, {
      workflowName: workflowName,
      transformer: transformer
    });

    // Subscribe to event
    this.eventBus.on(eventName, async (eventData) => {
      await this.handleEvent(eventName, eventData);
    });
  }

  async handleEvent(eventName, eventData) {
    const trigger = this.triggers.get(eventName);

    if (!trigger) {
      return;
    }

    // Transform event data to workflow input
    const workflowInput = trigger.transformer 
      ? trigger.transformer(eventData)
      : eventData;

    // Start workflow
    await this.engine.startWorkflow(
      trigger.workflowName,
      workflowInput
    );
  }
}

// Example: Order placed triggers workflow
workflowTriggers.defineTrigger(
  'order.placed',
  'process-order',
  (event) => ({
    order: event.data
  })
);

7. Workflow Versioning & Migration

Handle workflow changes over time:

class WorkflowVersioning {
  async deployNewVersion(workflowName, newDefinition) {
    const currentVersion = await this.getCurrentVersion(workflowName);
    const newVersion = (currentVersion?.version || 0) + 1;

    await this.db.insert('workflow_definitions', {
      name: workflowName,
      version: newVersion,
      definition: JSON.stringify(newDefinition),
      deployedAt: new Date(),
      active: true
    });

    // Deactivate old version
    if (currentVersion) {
      await this.db.update('workflow_definitions', currentVersion.id, {
        active: false
      });
    }

    return newVersion;
  }

  async migrateInFlightExecutions(workflowName, fromVersion, toVersion) {
    const executions = await this.db.query(`
      SELECT * FROM workflow_executions
      WHERE workflowName = ?
        AND workflowVersion = ?
        AND status IN ('running', 'waiting-for-human')
    `, [workflowName, fromVersion]);

    for (const execution of executions) {
      await this.migrateExecution(execution, toVersion);
    }
  }
}

Implementation Details

Complete Cross-System Workflow System

class CompleteCrossSystemWorkflowSystem {
  constructor(database, serviceRegistry) {
    this.engine = new WorkflowExecutionEngine(database, serviceRegistry);
    this.saga = new SagaOrchestrator();
    this.humanTasks = new HumanTaskManager(database);
    this.monitoring = new WorkflowMonitoring(database);
    this.triggers = new WorkflowTriggers(this.engine, eventBus);
    this.versioning = new WorkflowVersioning(database);
  }

  // Define workflow
  defineWorkflow(definition) {
    return WorkflowDefinition.define(definition);
  }

  // Start workflow
  async start(workflowName, input) {
    const workflow = await this.getWorkflow(workflowName);
    return await this.engine.startWorkflow(workflow, input);
  }

  // Check status
  async getStatus(executionId) {
    return await this.monitoring.getExecutionStatus(executionId);
  }

  // Complete human task
  async completeTask(taskId, decision, data) {
    return await this.humanTasks.completeTask(taskId, decision, data);
  }
}

// Usage
const workflows = new CompleteCrossSystemWorkflowSystem(db, services);

// Define order workflow
const orderWorkflow = workflows.defineWorkflow({
  name: 'process-order',
  steps: [
    { name: 'validate', service: 'validator', action: 'validate' },
    { name: 'inventory', service: 'inventory', action: 'reserve' },
    { name: 'payment', service: 'payment', action: 'charge' },
    { name: 'shipping', service: 'shipping', action: 'create' },
    { name: 'notification', service: 'email', action: 'send' }
  ]
});

// Start workflow when order placed
const execution = await workflows.start('process-order', { order });

// Check progress
const status = await workflows.getStatus(execution.id);
console.log(`Order ${status.progress.completedSteps}/${status.progress.totalSteps} steps complete`);

Consequences

Benefits

Orchestration: - Coordinate multiple systems - Clear process flow - Centralized control

Reliability: - Automatic retry - Compensation on failure - Transactional integrity

Visibility: - Track progress - Debug failures - Audit trail

Flexibility: - Define workflows declaratively - Version workflows - Migrate in-flight processes

Human Integration: - Support approvals - Manual steps - Pause/resume

Liabilities

Complexity: - Workflow engine is sophisticated - Learning curve steep - More moving parts

Performance: - State persistence overhead - Coordination latency - Not for real-time

Debugging: - Distributed debugging hard - Async makes it harder - Need good tooling

Versioning Challenges: - Migrating workflows complex - Breaking changes difficult - Backward compatibility needed

Domain Examples

E-commerce: Order Processing

workflows.start('process-order', { orderId });

Lending: Loan Application

workflows.start('loan-application', { applicant, loanAmount });

Healthcare: Patient Onboarding

workflows.start('patient-intake', { patient, insurance });

HR: Employee Onboarding

workflows.start('employee-onboarding', { employee, startDate });

Related Patterns

Builds Upon ALL Previous Patterns: - Uses Pattern 21 (External Integration) - Uses Pattern 22 (Real-Time Validation) - Uses Pattern 23 (Business Rules) - Uses Pattern 24 (Webhooks for notifications)

The Grand Synthesis: This pattern shows how all 25 patterns work together to create intelligent, connected, reliable form-driven workflows.

Known Uses

Uber: Ride workflow orchestration Netflix: Content processing pipelines Airbnb: Booking workflows AWS Step Functions: Serverless workflows Temporal.io: Workflow engine Camunda: Business process automation Zapier: No-code workflow automation

Further Reading

Academic Foundations

• Workflow Management: van der Aalst, W.M.P., & van Hee, K.M. (2004). Workflow Management: Models, Methods, and Systems. MIT Press. ISBN: 978-0262720465
• Process Orchestration: Peltz, C. (2003). "Web Services Orchestration and Choreography." IEEE Computer 36(10): 46-52.
• Distributed Transactions: Gray, J., & Reuter, A. (1992). Transaction Processing: Concepts and Techniques. Morgan Kaufmann. ISBN: 978-1558601901
• Saga Pattern: Garcia-Molina, H., & Salem, K. (1987). "Sagas." SIGMOD '87. https://www.cs.cornell.edu/andru/cs711/2002fa/reading/sagas.pdf

Practical Implementation

• Temporal: https://temporal.io/ - Durable workflow orchestration (open source)
• AWS Step Functions: https://aws.amazon.com/step-functions/ - Serverless workflow service
• Apache Airflow: https://airflow.apache.org/ - Workflow automation platform
• Conductor: https://conductor.netflix.com/ - Netflix's microservices orchestrator
• Zeebe: https://camunda.com/platform/zeebe/ - Cloud-native workflow engine

Standards & Patterns

• BPMN 2.0: https://www.omg.org/spec/BPMN/2.0/ - Business Process Model and Notation
• WS-BPEL: https://docs.oasis-open.org/wsbpel/2.0/OS/wsbpel-v2.0-OS.html - Business Process Execution Language
• Saga Pattern: https://microservices.io/patterns/data/saga.html - Distributed transaction pattern
• Compensating Transactions: https://docs.microsoft.com/en-us/azure/architecture/patterns/compensating-transaction

Related Trilogy Patterns

• Pattern 22: State-Aware Behavior - Workflow states
• Pattern 25: Scheduled Actions - Time-based workflow triggers
• Pattern 26: External Data Integration - Cross-system data flow
• Pattern 28: API-Driven Rules - Workflow decision points
• Pattern 29: Webhooks and Events - Event-driven workflows
• Volume 2, Pattern 8: Tier-Based Segmentation - Intelligence across workflow steps
• Volume 1, Chapter 8: Architecture of Domain-Specific Systems - Architecture foundation

Tools & Services

• Camunda: https://camunda.com/ - Open source workflow and decision automation
• n8n: https://n8n.io/ - Workflow automation (open source)
• Prefect: https://www.prefect.io/ - Modern workflow orchestration (Python)
• Dagster: https://dagster.io/ - Data orchestration platform
• Argo Workflows: https://argoproj.github.io/argo-workflows/ - Kubernetes-native workflows

Implementation Examples

• Building Sagas: https://microservices.io/patterns/data/saga.html - Distributed saga pattern
• Workflow Design Patterns: http://www.workflowpatterns.com/ - Comprehensive pattern catalog
• Temporal Tutorial: https://docs.temporal.io/docs/get-started - Build your first workflow
• Event-Driven Workflows: https://aws.amazon.com/blogs/compute/building-resilient-serverless-workflows-with-aws-step-functions/
• Orchestration vs Choreography: https://stackoverflow.com/questions/4127241/orchestration-vs-choreography

Integration & Enterprise

• Enterprise Integration Patterns: Hohpe, G., & Woolf, B. (2003). Enterprise Integration Patterns. Addison-Wesley. ISBN: 978-0321200686 - https://www.enterpriseintegrationpatterns.com/
• Microservices Orchestration: https://www.nginx.com/blog/microservices-reference-architecture-nginx-choreography-orchestration/
• The Grand Synthesis: This pattern demonstrates how all 30 INPUT patterns integrate with Volume 2 INTELLIGENCE patterns and Volume 1 OUTPUT patterns to create complete organizational intelligence systems

CONCLUSION: The Complete Pattern Language

What We Have Built

Over 240,000 words, we have created a comprehensive pattern language for Human-System Collaboration through forms and knowledge capture interfaces.

25 Patterns organized into 5 categories:

Foundation Patterns (1-5): Building Blocks

Established the core principles of intelligent forms - progressive disclosure, smart defaults, inline validation, contextual help, and recovery mechanisms.

Intelligence Patterns (6-10): Making Forms Smart

Added intelligence layers - domain awareness, adaptive behavior, conditional logic, semantic understanding, and predictive capabilities.

Relationship Patterns (11-15): Managing Complexity

Tackled interconnected data - cascading updates, mutual exclusivity, conditional requirements, cross-field validation, and smart dependencies.

Temporal Patterns (16-20): Time and State

Mastered time dimensions - temporal validation, state awareness, audit trails, version control, and scheduled actions.

Integration Patterns (21-25): Connecting Systems

Orchestrated distributed systems - external data, real-time lookup, API rules, webhooks, and cross-system workflows.

The Vision Realized

This trilogy represents systematic knowledge captured in pattern form - reusable, teachable, implementable.

Every pattern: - Opens with a real problem - Shows the consequences of doing it wrong - Provides working implementations - Demonstrates domain applications - Connects to other patterns

This is how AI learns. This is how humans learn.

For Posterity

These 25 patterns will help:

Developers build better forms Designers create better experiences Business analysts capture requirements AI systems learn form design Students understand systematic design Organizations build knowledge systems

These patterns, combined with the output patterns of Volume 1 and the intelligence patterns of Volume 2, provide a complete framework for building organizational knowledge systems that capture, process, and communicate information effectively.