Get started with workflows

In this tutorial, you'll create a workflow that indexes and searches through national parks data, demonstrating the core concepts and capabilities of workflows along the way.

• To use workflows, turn on the Elastic Workflows advanced setting (workflows:ui:enabled).
• You must have the appropriate subscription. Refer to the subscription page for Elastic Cloud and Elastic Stack/self-managed for the breakdown of available features and their associated subscription tiers.

Let's examine each part of our first workflow to understand how it works.

name: 🏔️ National Parks Demo
description: Creates an Elasticsearch index, loads sample national park data using bulk operations, searches for parks by category, and displays the results.
enabled: true
tags: ["demo", "getting-started"]
		

• name: A unique identifier for your workflow.
• description: Explains the workflow's purpose.
• enabled: Controls whether the workflow can be run.
• tags: Labels for organizing and finding workflows.

consts:
  indexName: national-parks-data
		

• consts: Defines reusable values that can be referenced throughout the workflow.
• Accessed using template syntax: {{ consts.indexName }}. This promotes consistency and makes the workflow easier to maintain.

triggers:
  - type: manual
		

• triggers: Defines how the workflow starts.
• manual: Specifies the trigger type. Manual triggers require explicit user action (clicking the Run icon ) to start a workflow.

- name: create_parks_index
  type: elasticsearch.indices.create
  with:
    index: "{{ consts.indexName }}"
    settings:
      number_of_shards: 1
      number_of_replicas: 0
    mappings:
      properties:
        name: { type: text }
        category: { type: keyword }
        description: { type: text }
		

• Step type: This is an action step that directly interacts with Elasticsearch.
• Step purpose: Establishes the data structure for our park information, ensuring fields are properly typed for searching and aggregation.
• Key elements:
  • Uses elasticsearch.indices.create, which is a built-in action that maps to the Elasticsearch Create Index API.
  • Defines mappings to control how data is indexed (text for full-text search, keyword for exact matching).
  • References the constant indexName for consistency.
  • Sets index settings for optimal performance in this demo.

- name: bulk_index_park_data
  type: elasticsearch.bulk
  with:
    index: "{{ consts.indexName }}"
    operations:
      - name: "Yellowstone National Park"
        category: "geothermal"
        description: "America's first national park, established in 1872..."
      - name: "Grand Canyon National Park"
        category: "canyon"
        description: "Home to the immense Grand Canyon..."
      # ... additional parks
		

• Step type: Another internal action step using Elasticsearch's bulk API.
• Step purpose: Efficiently loads multiple documents in a single operation, populating our index with sample data.
• Key elements:
  • The operations array contains the documents to index.
  • Each document becomes a searchable record in Elasticsearch.
  • Uses the field names defined in our mappings (name, category, description).
  • Each document becomes a searchable record with consistent field structure.
  • This step demonstrates how to handle batch operations in workflows.

- name: search_park_data
  type: elasticsearch.search
  with:
    index: "{{ consts.indexName }}"
    size: 5
    query:
      term:
        category: "canyon"
		

• Step type: Internal action step for querying Elasticsearch.
• Step purpose: Retrieves specific data based on criteria, demonstrating how workflows can make decisions based on data.
• Key elements:
  • Searches for parks with category "canyon" (will find Grand Canyon and Zion).
  • Results are automatically available to subsequent steps via steps.search_park_data.output.
  • Limits results to 5 documents for manageable output.
  • Shows how workflows can filter and process data dynamically.

- name: log_results
  type: console
  with:
    message: |-
      Found {{ steps.search_park_data.output.hits.total.value }} parks in category "canyon".
      Top results: {{ steps.search_park_data.output.hits.hits | json(2) }}
		

• Step type: A console step for output and debugging.
• Step purpose: Presents the results in a human-readable format, demonstrating how to access and format data from previous steps.
• Key elements:
  • Template variables access the search results: {{ steps.search_park_data.output }}.
  • The | json(2) filter formats JSON output with indentation.
  • Uses the exact step name search_park_data to reference previous step output.
  • Shows how data flows through the workflow and can be transformed.

This workflow introduces several fundamental concepts:

• Action steps: Built-in steps that interact with Elasticsearch and Kibana APIs.
• Data flow: How information moves from step to step using outputs and template variables.
• Constants: Reusable values that make workflows maintainable.
• Template syntax: The {{ }} notation for dynamic values.
• Step chaining: How each step builds on previous ones to create a complete process.