Building automations

This guide walks you through creating a simple login automation from scratch. By the end, you’ll understand how to define actions, use parameters, and run your first Optexity workflow.

What you’ll learn: - How to record browser interactions with the Optexity Recorder - How to understand and edit automation JSON/Python - How to use parameters for dynamic values - How to run your automation via the API

Prerequisites

Before you begin, ensure you have:

✅ Completed the Installation guide
✅ Optexity running in your local environment
✅ Your API key from the dashboard

Install the Recorder Extension

Install the Optexity Recorder extension from the Chrome Web Store. This extension captures your browser interactions and converts them into automation workflows.

Install the extension

Click “Add to Chrome” on the Chrome Web Store page

Pin the extension

Click the puzzle icon in Chrome and pin Optexity Recorder for easy access

Add your API key

Click the extension icon and enter your API key from the dashboard

What We’re Building

Let’s create an automation that logs into a website. This simple example demonstrates the core concepts you’ll use in every Optexity workflow:

Step	Action	Description
1	Click	Navigate to the login page by clicking “Sign in”
2	Type	Enter the email address
3	Type	Enter the password
4	Click	Submit the login form

Step 1: Record the Automation

The fastest way to create an automation is by recording your actions directly in the browser.

Navigate to the target website

Open Chrome and go to the website you want to automate (e.g., https://example.com)

Start capturing

Click the Optexity Recorder extension icon and hit Start Capture

Perform your actions

Interact with the website naturally, click buttons, fill in forms, navigate pages. The recorder captures every interaction.

Stop and save

When finished, click Complete Capture. The automation is automatically saved to your dashboard as a JSON file.

Recording Tips: - Perform actions slowly and deliberately for better accuracy - Avoid unnecessary scrolling or hovering - The recorder captures clicks, text input, and form selections

Step 2: Understand the Automation Structure

Once recorded, your automation is saved as JSON on the dashboard. Let’s break down the structure using a login example.

The Complete Automation

Here is a sample automation:

{
    "url": "https://example.com",
    "parameters": {
        "input_parameters": {
            "username": ["user@example.com"],
            "password": ["mypassword123"]
        },
        "generated_parameters": {}
    },
    "nodes": [
        {
            "interaction_action": {
                "click_element": {
                    "command": "get_by_role(\"link\", name=\"Sign in\")",
                    "prompt_instructions": "Click the Sign in link in the navigation bar"
                }
            },
            "end_sleep_time": 1.0
        },
        {
            "interaction_action": {
                "input_text": {
                    "command": "get_by_role(\"textbox\", name=\"Email\")",
                    "prompt_instructions": "Enter the email address in the login form",
                    "input_text": "{username[0]}"
                }
            },
            "end_sleep_time": 1.0
        },
        {
            "interaction_action": {
                "input_text": {
                    "command": "get_by_role(\"textbox\", name=\"Password\")",
                    "prompt_instructions": "Enter the password in the login form",
                    "input_text": "{password[0]}"
                }
            },
            "end_sleep_time": 1.0
        },
        {
            "interaction_action": {
                "click_element": {
                    "command": "get_by_role(\"button\", name=\"Sign In\")",
                    "prompt_instructions": "Click the Sign In button to submit the form"
                }
            },
            "end_sleep_time": 1.0
        }
    ]
}

Here’s what a login automation looks like in Python:

from optexity.schema.actions.interaction_action import (
    ClickElementAction,
    InputTextAction,
    InteractionAction,
)
from optexity.schema.automation import ActionNode, Automation, Parameters

login_automation = Automation(
    url="https://example.com",
    parameters=Parameters(
        input_parameters={
            "username": ["user@example.com"],
            "password": ["mypassword123"],
        },
        generated_parameters={},
    ),
    nodes=[
        # Step 1: Click the Sign in link
        ActionNode(
            interaction_action=InteractionAction(
                click_element=ClickElementAction(
                    command="""get_by_role("link", name="Sign in")""",
                    prompt_instructions="Click the Sign in link in the navigation bar",
                )
            )
        ),
        # Step 2: Enter the email
        ActionNode(
            interaction_action=InteractionAction(
                input_text=InputTextAction(
                    command="""get_by_role("textbox", name="Email")""",
                    input_text="{username[0]}",
                    prompt_instructions="Enter the email address in the login form",
                )
            )
        ),
        # Step 3: Enter the password
        ActionNode(
            interaction_action=InteractionAction(
                input_text=InputTextAction(
                    command="""get_by_role("textbox", name="Password")""",
                    input_text="{password[0]}",
                    prompt_instructions="Enter the password in the login form",
                )
            )
        ),
        # Step 4: Click Sign In button
        ActionNode(
            interaction_action=InteractionAction(
                click_element=ClickElementAction(
                    command="""get_by_role("button", name="Sign In")""",
                    prompt_instructions="Click the Sign In button to submit the form",
                )
            )
        ),
    ],
)

Breaking Down Each Component

Automation — The Container

The Automation object is the top-level container that holds your entire workflow:

Automation(
    url="https://example.com",      # Where the browser starts
    parameters=Parameters(...),      # Input and generated values
    nodes=[...],                     # Sequence of actions
)

Property	Purpose
`url`	The starting URL—where the browser navigates first
`parameters`	Container for variables used during execution
`nodes`	Ordered list of actions to execute sequentially

Parameters — Your Variables

Parameters define the data flowing through your automation. They’re divided into two types:Input Parameters — Values you provide before execution:

input_parameters={
    "username": ["user@example.com"],  # Access as {username[0]}
    "password": ["secret123"],          # Access as {password[0]}
}

Generated Parameters — Values extracted during execution:

generated_parameters={
    "order_id": [],      # Populated by extraction actions
    "confirmation": [],  # Available for later steps
}

Values are stored as lists of strings. Access them using {variable_name[index]} syntax, where index is typically 0 for single values.

ActionNode — Individual Steps

Each ActionNode represents a single atomic action. An ActionNode contains exactly one of these action types: | Action Type | Purpose | Example | |-------------|---------|---------| | interaction_action | Click, type, select, scroll, navigate | Clicking a button | | extraction_action | Extract data from the page | Scraping product prices | | assertion_action | Verify conditions | Check if logged in | | python_script_action | Run custom Python code | Data transformation | | fetch_2fa_action | Handle two-factor authentication | Get OTP from email |

python ActionNode(         interaction_action=InteractionAction( click_element=ClickElementAction(         command="""get_by_role("button", name="Submit")""", prompt_instructions="Click the submit         button", ) ), before_sleep_time=0.0, # Wait before action end_sleep_time=1.0, # Wait after         action )

Locators — Finding Elements

Optexity uses Playwright locators to find elements on the page. The command field accepts Playwright’s powerful locator syntax:

# By role (recommended — most resilient)
command="""get_by_role("button", name="Submit")"""

# By text content
command="""get_by_text("Welcome back")"""

# By label (great for form fields)
command="""get_by_label("Email address")"""

# By test ID
command="""get_by_test_id("login-button")"""

# By CSS selector
command="""locator("#email-input")"""

The prompt_instructions field provides a natural language fallback. If the locator fails, Optexity’s AI uses this description to find the element visually.

Step 3: Edit and Customize

After recording, you may want to customize your automation. Common edits include:

Parameterizing Values

Replace hardcoded values with parameters for flexibility:

# Before: Hardcoded email
input_text="user@example.com"

# After: Parameterized
input_text="{username[0]}"

Adding Descriptive Instructions

Improve the prompt_instructions for better AI fallback:

# Basic (less reliable)
prompt_instructions="Click the button"

# Descriptive (more reliable)
prompt_instructions="Click the blue 'Sign In' button at the bottom of the login form"

Adjusting Timing

Control execution speed with timing properties:

ActionNode(
    interaction_action=InteractionAction(...),
    before_sleep_time=2.0,  # Wait 2 seconds before action
    end_sleep_time=1.0,     # Wait 1 second after action
)

Step 4: Run Your Automation

Once your automation is defined, execute it through the inference API.

Using cURL

curl -X POST http://localhost:9000/inference \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint_name": "login-flow",
    "input_parameters": {
      "username": ["user@example.com"],
      "password": ["mypassword123"]
    },
    "unique_parameter_names": ["username"]
  }'

Understanding the Request

Field	Description
`endpoint_name`	The name of your automation (as saved on the dashboard)
`input_parameters`	Values to inject into your automation
`unique_parameter_names`	Parameters that uniquely identify this run (for deduplication)

Expected Response

A successful run returns:

{
    "status": "success",
    "run_id": "run_abc123",
    "output": {
        "generated_parameters": {},
        "extracted_data": []
    }
}

Common Patterns

Here are some patterns you’ll use frequently:

Clicking Elements

ActionNode(
    interaction_action=InteractionAction(
        click_element=ClickElementAction(
            command="""get_by_role("button", name="Continue")""",
            prompt_instructions="Click the Continue button",
        )
    )
)

Filling Form Fields

ActionNode(
    interaction_action=InteractionAction(
        input_text=InputTextAction(
            command="""get_by_label("Email")""",
            input_text="{email[0]}",
            prompt_instructions="Enter email in the form field",
        )
    )
)

Selecting Dropdowns

from optexity.schema.actions.interaction_action import SelectOptionAction

ActionNode(
    interaction_action=InteractionAction(
        select_option=SelectOptionAction(
            command="""get_by_label("Country")""",
            select_values=["United States"],
            prompt_instructions="Select country from dropdown",
        )
    )
)

Handling New Tabs

ActionNode(
    interaction_action=InteractionAction(
        click_element=ClickElementAction(
            command="""get_by_role("link", name="View Details")""",
            prompt_instructions="Click link that opens in new tab",
        )
    ),
    expect_new_tab=True,  # Waits for and switches to new tab
)

Troubleshooting

Element not found

Problem: The automation fails to find an element. Solutions: 1. Improve prompt_instructions with more visual details 2. Try a different locator strategy (role → label → text → CSS) 3. Add before_sleep_time to wait for the element to appear 4. Check if the element is inside an iframe

Action executed too fast

Problem: The page hasn’t loaded before the next action runs. Solution: Increase end_sleep_time on the previous action:

python ActionNode(             interaction_action=InteractionAction(...), end_sleep_time=3.0, # Wait 3 seconds after action             )

Variable not substituted

Problem: You see {username[0]} instead of the actual value. Solutions: 1. Ensure the variable is defined in input_parameters 2. Check the index is correct (starts at 0) 3. Verify the syntax: {variable_name[index]}

Next Steps

Now that you’ve built your first automation, explore these topics to level up:

Core Concepts

Deep dive into the automation model, nodes, and execution flow

Locators

Master element location strategies for reliable automations

Interaction Actions

Explore all available interaction types: clicks, inputs, navigation

Extraction Actions

Learn to capture data from web pages into your workflow

Quickstart

Action Types

Inference

Advanced

FAQs

Building automations

Prerequisites

Install the Recorder Extension

What We’re Building

Step 1: Record the Automation

Step 2: Understand the Automation Structure

The Complete Automation

Breaking Down Each Component

Step 3: Edit and Customize

Parameterizing Values

Adding Descriptive Instructions

Adjusting Timing

Step 4: Run Your Automation

Using cURL

Understanding the Request

Expected Response

Common Patterns

Clicking Elements

Filling Form Fields

Selecting Dropdowns

Handling New Tabs

Troubleshooting

Next Steps

Core Concepts

Locators

Interaction Actions

Extraction Actions

Quickstart

Building Automations

Action Types

Inference

Advanced

FAQs

​Prerequisites

​Install the Recorder Extension

​What We’re Building

​Step 1: Record the Automation

​Step 2: Understand the Automation Structure

​The Complete Automation

​Breaking Down Each Component

​Step 3: Edit and Customize

​Parameterizing Values

​Adding Descriptive Instructions

​Adjusting Timing

​Step 4: Run Your Automation

​Using cURL

​Understanding the Request

​Expected Response

​Common Patterns

​Clicking Elements

​Filling Form Fields

​Selecting Dropdowns

​Handling New Tabs

​Troubleshooting

​Next Steps

Core Concepts

Locators

Interaction Actions

Extraction Actions

Prerequisites

Install the Recorder Extension

What We’re Building

Step 1: Record the Automation

Step 2: Understand the Automation Structure

The Complete Automation

Breaking Down Each Component

Step 3: Edit and Customize

Parameterizing Values

Adding Descriptive Instructions

Adjusting Timing

Step 4: Run Your Automation

Using cURL

Understanding the Request

Expected Response

Common Patterns

Clicking Elements

Filling Form Fields

Selecting Dropdowns

Handling New Tabs

Troubleshooting

Next Steps