Skip to content

Workflow Variable Processing Documentation

WARNING: Never use {{ ... }} (Go template) or ${...} syntax for dynamic values in YAML workflow/config files. Only use dot notation (e.g., task_id.output_key, INPUT.var, TASKFLOW.process_id). The {{ ... }} syntax is only allowed inside prompt templates, not in YAML config.

1. Variable Sources and Syntax Rules

1.1 Core Variable Syntax Rules

  1. Never use ${} or {{}} syntax in YAML

    # INCORRECT
    config:
      message: "${task1.output}"
    # INCORRECT
    config:
      message: "{{ task1.output }}"
    
    # CORRECT
    config:
      message: "task1.output"
    

  2. Use proper template syntax only in prompts

    # CORRECT (in prompt template only)
    system_prompt: |
      Hello {{ .name }}, your score is {{ .score }}
    
    # INCORRECT (in YAML config)
    config:
      message: "Hello {{ .name }}"
    

1.2 Trigger Input Variables

  • Variables from webhook triggers are mapped using input_mapping in the workflow YAML
  • Accessed via INPUT. prefix in task configurations
  • Example from workflow_email_processing.yml:
    input_mapping:
      resource_id: "TRIGGER.payload.resourceId"
      threadId: "TRIGGER.payload.threadId"
      unipileAccountId: "TRIGGER.payload.unipileAccountId"
    
    # Usage in tasks
    config:
      resource_id: "INPUT.resource_id"
      thread_id: "INPUT.threadId"
    

1.3 Task Output Variables

  • Each task can produce output that becomes available to subsequent tasks
  • Task outputs are stored in the execution context
  • Access patterns:
  • Direct task output: task_id.output_key
  • Extraction tool output: task_id.tool_call_results.tool_name.field_name
  • Nested fields: task_id.output_key.nested_field

Example from workflow_email_processing.yml:

# Accessing extraction tool output
prompt_context:
  email_analysis_structured: "analyze_email.tool_call_results.extract_email_analysis_data"
  suggested_actions: "analyze_email.tool_call_results.extract_email_analysis_data.suggested_actions"

# Accessing nested fields
config:
  sender_email: "fetch_resource_content.resource.Content.sender.email"

1.4 Prompt Template Variables

  • Prompts define required variables in their templates
  • Variables must be provided in:
  • prompt_context in task configuration
  • default_variables in prompt template definition

Example from workflow_email_processing.yml:

# In prompts.yml
templates:
  mail_analysis:
    system_prompt: |
      Email Content: {{ .email_content }}
    default_variables:
      email_content: "<Contenu de l'e-mail XML ici>"

# In workflow.yml
tasks:
  - id: "analyze_email"
    config:
      prompt_context:
        email_content: "fetch_resource_content.resource.Content"

2. Variable Context Management

2.1 Execution Context

  • Implemented in internal/taskflow/context.go
  • Thread-safe map for variable storage
  • Key methods:
    Get(key string) (interface{}, bool)
    Set(key string, value interface{})
    Merge(data map[string]interface{})
    

2.2 Flow Context

  • Maintains state during workflow execution
  • Tracks current task ID
  • Handles variable scoping and lifecycle

3. Variable Resolution Process

3.1 Task Configuration Resolution

  1. Resolution Order:
  2. Task output variables (e.g., task_id.output_key)
  3. Input variables (e.g., INPUT.variable_name)
  4. Default values from prompt templates

  5. Path Resolution:

    # Simple path
    config:
      resource_id: "INPUT.resource_id"
    
    # Nested path
    config:
      sender_email: "fetch_resource_content.resource.Content.sender.email"
    
    # Array access
    config:
      first_action: "analyze_email.tool_call_results.extract_email_analysis_data.suggested_actions[0]"
    

3.2 Prompt Variable Resolution

  1. System checks required variables in prompt template
  2. Looks for variables in:
  3. Task's prompt_context
  4. Prompt's default_variables
  5. Validates all required variables are present

4. Extraction Tool Output Mapping

4.1 Tool Output Structure

When an extraction tool is used in a prompt, its output is stored with this structure:

task_id:
  tool_call_results:
    tool_name:
      field1: value1
      field2: value2

4.2 Common Patterns

  1. Basic Tool Output Access

    # Access a single field
    task_id.tool_call_results.tool_name.field_name
    
    # Example from workflow_email_processing.yml
    analyze_email.tool_call_results.extract_email_analysis_data.summary
    

  2. Nested Object Access

    # Access nested fields
    task_id.tool_call_results.tool_name.object_field.nested_field
    
    # Example
    generate_drafts.tool_call_results.generate_single_draft_content.to[0].identifier
    

  3. Array Access

    # Access array elements
    task_id.tool_call_results.tool_name.array_field[index]
    
    # Example
    analyze_email.tool_call_results.extract_email_analysis_data.suggested_actions[0]
    

4.3 Common Issues and Solutions

  1. Missing Tool Results Path

    # INCORRECT
    email_analysis: "analyze_email.parsed_content"
    
    # CORRECT
    email_analysis: "analyze_email.tool_call_results.extract_email_analysis_data"
    

  2. Incorrect Field Path

    # INCORRECT
    suggested_actions: "analyze_email.parsed_content.suggested_actions"
    
    # CORRECT
    suggested_actions: "analyze_email.tool_call_results.extract_email_analysis_data.suggested_actions"
    

  3. Missing Required Fields

    # Add missing required fields to prompt_context
    prompt_context:
      summary: "analyze_email.tool_call_results.extract_email_analysis_data.summary"
      # ... other fields
    

5. Best Practices

5.1 Variable Naming

  • Use consistent naming conventions
  • Prefix trigger inputs with INPUT.
  • Use dot notation for task outputs
  • Document variable paths in comments

5.2 Prompt Templates

  • Define all required variables
  • Provide default values when possible
  • Document variable requirements
  • Include error handling templates

5.3 Task Configuration

  • Validate variable references
  • Handle missing variables gracefully
  • Use proper error handling templates
  • Document variable dependencies

5.4 Error Handling

error_handling:
  on_error:
    - type: "retry_with_prompt"
      append_to_system_prompt: "retry_with_prompt"
      max_retries: 2

6. Common Validation Errors

  1. Missing Required Variables
  2. Error: prompt template requires variable '{{ .summary }}'
  3. Fix: Add missing variable to prompt_context

  4. Invalid Task Output References

  5. Error: references non-existent output 'resource.Content'
  6. Fix: Check task output structure and path

  7. Invalid Input References

  8. Error: references non-existent task 'INPUT'
  9. Fix: Verify input mapping in workflow definition

  10. Incorrect Template Syntax

  11. Error: invalid template syntax: ${variable}
  12. Fix: Use {{ .variable }} syntax

  13. Missing Error Handling

  14. Error: missing error handling template
  15. Fix: Add error handling configuration