Interaction actions represent user interactions with the browser: clicking buttons, filling forms, navigating pages, and more.
Available Actions
| Action | Purpose |
|---|
click_element | Click buttons, links, elements |
input_text | Type into text fields |
select_option | Select from dropdowns |
check | Check/uncheck checkboxes |
upload_file | Upload files |
go_to_url | Navigate to a URL |
go_back | Go back in history |
close_tabs_until | Close tabs until condition met |
download_url_as_pdf | Save page as PDF |
key_press | Press keyboard keys |
agentic_task | AI agent for complex tasks |
close_overlay_popup | Dismiss popups/modals |
Common Properties
All element-targeting actions share these properties:
| Property | Type | Default | Description |
|---|
command | str | None | None | Playwright locator (e.g., get_by_role("button")) |
xpath | str | None | None | XPath selector (alternative to command) |
prompt_instructions | str | Required | Description for AI fallback |
skip_prompt | bool | False | Skip AI if locator fails (for optional elements) |
assert_locator_presence | bool | False | Skip action if element doesn’t exist |
max_tries | int | 10 | Maximum retry attempts |
max_timeout_seconds_per_try | float | 1.0 | Timeout per attempt |
Use command for deterministic element finding (fast, no LLM tokens). The AI uses prompt_instructions as fallback when locators fail.
Click Element
{
"interaction_action": {
"click_element": {
"command": "get_by_role(\"button\", name=\"Submit\")",
"prompt_instructions": "Click the submit button"
}
}
}
Click Properties
| Property | Type | Default | Description |
|---|
double_click | bool | False | Double-click instead of single |
expect_download | bool | False | Click triggers file download |
download_filename | str | None | Auto-generated | Filename for download |
Examples
Double-click:
{
"click_element": {
"command": "get_by_role(\"row\", name=\"Item 1\")",
"double_click": true
}
}
{
"click_element": {
"command": "get_by_role(\"button\", name=\"Export\")",
"expect_download": true,
"download_filename": "report.pdf"
}
}
Input Text
{
"interaction_action": {
"input_text": {
"command": "get_by_label(\"Email\")",
"input_text": "{email[0]}",
"prompt_instructions": "Enter the email address"
}
}
}
| Property | Type | Default | Description |
|---|
input_text | str | None | None | Text to enter (supports variables) |
fill_or_type | "fill" | "type" | "fill" | How to enter text |
is_slider | bool | False | Element is a slider |
press_enter | bool | False | Press Enter after input |
Fill vs Type
| Mode | Behavior | Use When |
|---|
fill | Sets value instantly | Standard form fields |
type | Types character by character | Autocomplete, search with suggestions |
{
"input_text": {
"command": "get_by_label(\"Search\")",
"input_text": "laptop",
"fill_or_type": "type"
}
}
{
"input_text": {
"command": "get_by_role(\"slider\", name=\"Price\")",
"input_text": "500",
"is_slider": true
}
}
If input_text references an empty variable, the action is skipped automatically.
Select Option
{
"interaction_action": {
"select_option": {
"command": "get_by_label(\"Country\")",
"select_values": ["United States"],
"prompt_instructions": "Select country"
}
}
}
Select Properties
| Property | Type | Default | Description |
|---|
select_values | list[str] | Required | Values to select |
expect_download | bool | False | Selection triggers download |
download_filename | str | None | Auto-generated | Filename for download |
Optexity supports fuzzy matching for select values—“United States” will match “UNITED STATES OF AMERICA”.
Check (Checkbox/Radio)
{
"interaction_action": {
"check": {
"command": "get_by_label(\"I agree to the terms\")",
"prompt_instructions": "Check the terms agreement"
}
}
}
Navigation Actions
Go to URL
| Property | Type | Default | Description |
|---|
url | str | Required | URL to navigate to |
new_tab | bool | False | Open in new tab |
{
"interaction_action": {
"go_to_url": {
"url": "https://example.com/dashboard",
"new_tab": true
}
}
}
Go Back
{
"interaction_action": {
"go_back": {}
}
}
Close Tabs Until
Close tabs until reaching a specific URL or tab index:
| Property | Type | Default | Description |
|---|
matching_url | str | None | None | Stop at this URL |
tab_index | int | None | None | Stop at this tab index (0-based) |
{
"interaction_action": {
"close_tabs_until": {
"matching_url": "https://example.com/dashboard"
}
}
}
File Operations
Upload File
{
"interaction_action": {
"upload_file": {
"command": "get_by_label(\"Upload Document\")",
"file_path": "/path/to/document.pdf",
"prompt_instructions": "Upload the document"
}
}
}
Download Page as PDF
Capture the current page or a specific URL as PDF:
| Property | Type | Default | Description |
|---|
download_filename | str | None | Auto-generated | Filename for PDF |
url | str | None | Current page | URL to download |
{
"interaction_action": {
"download_url_as_pdf": {
"download_filename": "page_snapshot.pdf"
}
}
}
Handling New Tabs
When an action opens a new tab, set expect_new_tab on the action node:
{
"type": "action_node",
"interaction_action": {
"click_element": {
"command": "get_by_role(\"button\", name=\"Open Details\")",
"prompt_instructions": "Click to open in new tab"
}
},
"expect_new_tab": true
}
Retry Configuration
For slow-loading elements, increase retry attempts:
{
"interaction_action": {
"max_tries": 15,
"max_timeout_seconds_per_try": 2.0,
"click_element": {
"command": "get_by_role(\"button\", name=\"Submit\")",
"prompt_instructions": "Click submit"
}
}
}
Increase max_tries rather than timeout per try. This finds elements faster when they appear while still allowing for slow pages.
