Workflow Variable Processing Documentation¶

WARNING: Never use {{ ... }} (Go template) or ${...} syntax for dynamic values in YAML workflow/config files. Only use dot notation (e.g., task_id.output_key, INPUT.var, TASKFLOW.process_id). The {{ ... }} syntax is only allowed inside prompt templates, not in YAML config.

1. Variable Sources and Syntax Rules¶

1.1 Core Variable Syntax Rules¶

1. Never use ${} or {{}} syntax in YAML

   # INCORRECT
   config:
     message: "${task1.output}"
   # INCORRECT
   config:
     message: "{{ task1.output }}"
   
   # CORRECT
   config:
     message: "task1.output"
   

2. Use proper template syntax only in prompts

   # CORRECT (in prompt template only)
   system_prompt: |
     Hello {{ .name }}, your score is {{ .score }}
   
   # INCORRECT (in YAML config)
   config:
     message: "Hello {{ .name }}"
   

1.2 Trigger Input Variables¶

• Variables from webhook triggers are mapped using input_mapping in the workflow YAML
• Accessed via INPUT. prefix in task configurations
• Example from workflow_email_processing.yml:

  input_mapping:
    resource_id: "TRIGGER.payload.resourceId"
    threadId: "TRIGGER.payload.threadId"
    unipileAccountId: "TRIGGER.payload.unipileAccountId"
  
  # Usage in tasks
  config:
    resource_id: "INPUT.resource_id"
    thread_id: "INPUT.threadId"
  

1.3 Task Output Variables¶

• Each task can produce output that becomes available to subsequent tasks
• Task outputs are stored in the execution context
• Access patterns:
• Direct task output: task_id.output_key
• Extraction tool output: task_id.tool_call_results.tool_name.field_name
• Nested fields: task_id.output_key.nested_field

Example from workflow_email_processing.yml:

# Accessing extraction tool output
prompt_context:
  email_analysis_structured: "analyze_email.tool_call_results.extract_email_analysis_data"
  suggested_actions: "analyze_email.tool_call_results.extract_email_analysis_data.suggested_actions"

# Accessing nested fields
config:
  sender_email: "fetch_resource_content.resource.Content.sender.email"

1.4 Prompt Template Variables¶

• Prompts define required variables in their templates
• Variables must be provided in:
• prompt_context in task configuration
• default_variables in prompt template definition

Example from workflow_email_processing.yml:

# In prompts.yml
templates:
  mail_analysis:
    system_prompt: |
      Email Content: {{ .email_content }}
    default_variables:
      email_content: "<Contenu de l'e-mail XML ici>"

# In workflow.yml
tasks:
  - id: "analyze_email"
    config:
      prompt_context:
        email_content: "fetch_resource_content.resource.Content"

2. Variable Context Management¶

2.1 Execution Context¶

• Implemented in internal/taskflow/context.go
• Thread-safe map for variable storage
• Key methods:

  Get(key string) (interface{}, bool)
  Set(key string, value interface{})
  Merge(data map[string]interface{})
  

2.2 Flow Context¶

• Maintains state during workflow execution
• Tracks current task ID
• Handles variable scoping and lifecycle

3. Variable Resolution Process¶

3.1 Task Configuration Resolution¶

1. Resolution Order:
2. Task output variables (e.g., task_id.output_key)
3. Input variables (e.g., INPUT.variable_name)

4. Default values from prompt templates

5. Path Resolution:

   # Simple path
   config:
     resource_id: "INPUT.resource_id"
   
   # Nested path
   config:
     sender_email: "fetch_resource_content.resource.Content.sender.email"
   
   # Array access
   config:
     first_action: "analyze_email.tool_call_results.extract_email_analysis_data.suggested_actions[0]"
   

3.2 Prompt Variable Resolution¶

1. System checks required variables in prompt template
2. Looks for variables in:
3. Task's prompt_context
4. Prompt's default_variables
5. Validates all required variables are present

4. Extraction Tool Output Mapping¶

4.1 Tool Output Structure¶

When an extraction tool is used in a prompt, its output is stored with this structure:

task_id:
  tool_call_results:
    tool_name:
      field1: value1
      field2: value2

4.2 Common Patterns¶

1. Basic Tool Output Access

   # Access a single field
   task_id.tool_call_results.tool_name.field_name
   
   # Example from workflow_email_processing.yml
   analyze_email.tool_call_results.extract_email_analysis_data.summary
   

2. Nested Object Access

   # Access nested fields
   task_id.tool_call_results.tool_name.object_field.nested_field
   
   # Example
   generate_drafts.tool_call_results.generate_single_draft_content.to[0].identifier
   

3. Array Access

   # Access array elements
   task_id.tool_call_results.tool_name.array_field[index]
   
   # Example
   analyze_email.tool_call_results.extract_email_analysis_data.suggested_actions[0]
   

4.3 Common Issues and Solutions¶

1. Missing Tool Results Path

   # INCORRECT
   email_analysis: "analyze_email.parsed_content"
   
   # CORRECT
   email_analysis: "analyze_email.tool_call_results.extract_email_analysis_data"
   

2. Incorrect Field Path

   # INCORRECT
   suggested_actions: "analyze_email.parsed_content.suggested_actions"
   
   # CORRECT
   suggested_actions: "analyze_email.tool_call_results.extract_email_analysis_data.suggested_actions"
   

3. Missing Required Fields

   # Add missing required fields to prompt_context
   prompt_context:
     summary: "analyze_email.tool_call_results.extract_email_analysis_data.summary"
     # ... other fields
   

5. Best Practices¶

5.1 Variable Naming¶

• Use consistent naming conventions
• Prefix trigger inputs with INPUT.
• Use dot notation for task outputs
• Document variable paths in comments

5.2 Prompt Templates¶

• Define all required variables
• Provide default values when possible
• Document variable requirements
• Include error handling templates

5.3 Task Configuration¶

• Validate variable references
• Handle missing variables gracefully
• Use proper error handling templates
• Document variable dependencies

5.4 Error Handling¶

error_handling:
  on_error:
    - type: "retry_with_prompt"
      append_to_system_prompt: "retry_with_prompt"
      max_retries: 2

6. Common Validation Errors¶

1. Missing Required Variables
2. Error: prompt template requires variable '{{ .summary }}'

3. Fix: Add missing variable to prompt_context

4. Invalid Task Output References

5. Error: references non-existent output 'resource.Content'

6. Fix: Check task output structure and path

7. Invalid Input References

8. Error: references non-existent task 'INPUT'

9. Fix: Verify input mapping in workflow definition

10. Incorrect Template Syntax

11. Error: invalid template syntax: ${variable}

12. Fix: Use {{ .variable }} syntax

13. Missing Error Handling

14. Error: missing error handling template
15. Fix: Add error handling configuration