Best practices for Workflows

You can refer to the best practices listed here when orchestrating your services using Workflows.

This is not an exhaustive list of recommendations and doesn't teach you the basics of how to use Workflows. This document assumes that you already have a general understanding of the overall Google Cloud landscape and of Workflows. For more information, see the Google Cloud Well-Architected Framework and the Workflows overview.

Select an optimal communication pattern

When designing a microservices architecture for deploying multiple services, you can select from the following communication patterns:

  • Direct service-to-service communication

  • Indirect event-driven communication (also known as choreography)

  • Automated configuration, coordination, and management (also known as orchestration)

Make sure to consider the benefits and drawbacks of each of the preceding options and select an optimal pattern for your use case. For example, direct service-to-service communication might be simpler to implement than other options but it tightly couples your services. In contrast, an event-driven architecture lets you loosely couple your services; however, monitoring and debugging might be more complicated. Finally, a central orchestrator like Workflows, while less flexible, lets you coordinate the communication between services without the tight coupling of direct service-to-service communication, or the intricacy of choreographed events.

You can also combine communication patterns. For example, in event-driven orchestration, closely-related services are managed in an orchestration that is triggered by an event. Similarly, you might design a system where one orchestration results in a Pub/Sub message to another orchestrated system.

General tips

Once you've decided to use Workflows as your service orchestrator, keep in mind the following helpful tips.

Avoid hardcoding URLs

You can support workflows that are portable across multiple environments and easier to maintain by avoiding hardcoded URLs. You can achieve this in the following ways:

  • Define URLs as runtime arguments.

    This can be helpful when your workflow is invoked through a client library or the API. (However, this won't work if your workflow is triggered by an event from Eventarc and the only argument that can be passed is the event payload.)

    Example

    main:
  params: [args]
  steps:
    - init:
        assign:
          - url1: ${args.urls.url1}
          - url2: ${args.urls.url2}

    When you run the workflow, you can specify the URLs. For example:

    gcloud workflows run multi-env --data='{"urls":{"url1": "URL_ONE", "url2": "URL_TWO"}}'

  • Use environment variables and create a workflow that is dynamically configured depending on the environment to which it is deployed. Or, create a workflow that can be reused as a template and configured according to separately maintained environment variables.

  • Use a substitution technique that lets you create a single workflow definition file, but deploy variants by using a tool that replaces placeholders in your workflow. For example, you can use Cloud Build to deploy a workflow and in the Cloud Build configuration file, add a step to replace placeholder URLs in the workflow.

    Example

    steps:
‐ id: 'replace-urls'
  name: 'gcr.io/cloud-builders/gcloud'
  entrypoint: bash
  args:
    - -c
    - |
      sed -i -e "s~REPLACE_url1~$_URL1~" workflow.yaml
      sed -i -e "s~REPLACE_url2~$_URL2~" workflow.yaml
‐ id: 'deploy-workflow'
  name: 'gcr.io/cloud-builders/gcloud'
  args: ['workflows', 'deploy', 'multi-env-$_ENV', '--source', 'workflow.yaml']

    You can then substitute variable values at build time. For example:

    gcloud builds submit --config cloudbuild.yaml \
    --substitutions=_ENV=staging,_URL1="URL_ONE",_URL2="URL_TWO"

    For more information, see Submit a build via CLI and API.

    Or, you can use Terraform to provision your infrastructure and define a configuration file that creates workflows for each environment by using input variables.

    Example

    variable "project_id" {
  type = string
}

variable "url1" {
  type = string
}

variable "url2" {
  type = string
}

locals {
  env = ["staging", "prod"]
}

# Define and deploy staging and production workflows
resource "google_workflows_workflow" "multi-env-workflows" {
  for_each = toset(local.env)

  name            = "multi-env-${each.key}"
  project         = var.project_id
  region          = "us-central1"
  source_contents = templatefile("${path.module}/workflow.yaml", { url1 : "${var.url1}-${each.key}", url2 : "${var.url2}-${each.key}" })
}

    When variables are declared in the root module of your configuration, they can be assigned values in a number of ways. For example

    terraform apply -var="project_id=PROJECT_ID" -var="url1=URL_ONE" -var="url2=URL_TWO"