Web Automation

The Web Automation node provides browser-based automation powered by Playwright. It can navigate to web pages, interact with UI elements (click, type, select), extract data, take screenshots, and execute complex multi-step browser workflows. This is useful for automating web applications that do not have APIs, scraping data from web portals, and performing end-to-end UI testing.

note

The web automation engine also supports dedicated verification actions (assert page data, list ordering, media, layout, and link health) and web scraping actions (extract repeated rows, structured metadata, network responses, and multi-page crawls). These are documented for RPA scripts in Verification & Web Scraping Actions.

Actions

Action	Description
web:navigate	Navigate to a URL
web:click	Click an element
web:type	Type text into an input field
web:select	Select an option from a dropdown
web:getText	Extract text content from an element
web:getAttribute	Get an attribute value from an element
web:screenshot	Take a screenshot of the page or an element
web:waitForElement	Wait for an element to appear
web:evaluate	Execute JavaScript in the page context
web:scroll	Scroll the page or an element
web:hover	Hover over an element
web:pressKey	Press a keyboard key
web:uploadFile	Upload a file via a file input
web:switchFrame	Switch to an iframe context
web:closeBrowser	Close the browser session

---

web:navigate

Opens a URL in the browser.

Parameters

Parameter	Type	Required	Description
url	string	Yes	URL to navigate to
waitUntil	string	No	Wait condition: load, domcontentloaded, networkidle. Defaults to load
timeout	integer	No	Navigation timeout in milliseconds. Defaults to 30000

web:click

Clicks an element identified by a selector.

Parameters

Parameter	Type	Required	Description
selector	string	Yes	CSS selector or Playwright selector (e.g., text=Submit, #login-btn)
button	string	No	Mouse button: left, right, middle. Defaults to left
clickCount	integer	No	Number of clicks. Defaults to 1
timeout	integer	No	Element wait timeout in milliseconds

web:type

Types text into an input field. Clears the field first by default.

Parameters

Parameter	Type	Required	Description
selector	string	Yes	Input element selector
text	string	Yes	Text to type
clearFirst	boolean	No	Clear the field before typing. Defaults to true
delay	integer	No	Delay between keystrokes in milliseconds. Defaults to 0

web:select

Selects an option from a dropdown select element.

Parameters

Parameter	Type	Required	Description
selector	string	Yes	Select element selector
value	string	No	Option value attribute
label	string	No	Option visible text
index	integer	No	Option index (zero-based)

web:getText

Extracts the text content of an element.

Parameters

Parameter	Type	Required	Description
selector	string	Yes	Element selector

web:getAttribute

Gets the value of an HTML attribute from an element.

Parameters

Parameter	Type	Required	Description
selector	string	Yes	Element selector
attribute	string	Yes	Attribute name (e.g., href, data-id, class)

web:screenshot

Captures a screenshot of the page or a specific element.

Parameters

Parameter	Type	Required	Description
selector	string	No	Element selector for element-level screenshot. If omitted, captures the full page
fullPage	boolean	No	Capture the full scrollable page. Defaults to false
format	string	No	Image format: png or jpeg. Defaults to png

web:waitForElement

Waits for an element to appear in the DOM.

Parameters

Parameter	Type	Required	Description
selector	string	Yes	Element selector
state	string	No	Target state: visible, hidden, attached, detached. Defaults to visible
timeout	integer	No	Maximum wait time in milliseconds. Defaults to 30000

web:evaluate

Executes JavaScript code in the browser page context and returns the result.

Parameters

Parameter	Type	Required	Description
script	string	Yes	JavaScript code to execute

web:closeBrowser

Closes the browser session and releases resources. Should be called at the end of any web automation sequence.

Working with Iframes

Many web applications embed content inside iframes, such as payment forms, third-party widgets, or embedded editors. Use web:switchFrame to enter an iframe context before interacting with elements inside it.

How it works:

1. Use web:switchFrame with the iframe's selector (CSS or Playwright selector) to switch into that iframe context.
2. All subsequent actions (web:click, web:type, web:getText, etc.) operate within the iframe until you explicitly switch back.
3. To return to the main page frame, call web:switchFrame with no selector or with main as the selector value.

Parameters for web:switchFrame

Parameter	Type	Required	Description
selector	string	No	CSS or Playwright selector for the iframe element. Omit or use main to switch back to the main frame

Example scenario: A web application embeds a Stripe payment form inside an iframe. To fill in the card number:

Step 1: web:navigate → url: "https://app.example.com/checkout"
Step 2: web:waitForElement → selector: "iframe#payment-frame"
Step 3: web:switchFrame → selector: "iframe#payment-frame"
Step 4: web:type → selector: "input[name='cardnumber']", text: "4242424242424242"
Step 5: web:type → selector: "input[name='exp-date']", text: "12/28"
Step 6: web:switchFrame → selector: "main"   (back to main page)
Step 7: web:click → selector: "#place-order-btn"

tip

Always use web:waitForElement to confirm the iframe has loaded before switching into it. Iframes often load asynchronously and may not be immediately available.

Selectors: CSS vs Playwright vs XPath

The Web Automation node accepts three types of selectors. Choose the one that best fits your use case.

CSS Selectors

Standard CSS selectors are the most common choice. They work well for elements with IDs, classes, or specific attributes.

Selector	What it matches
#login-btn	Element with id login-btn
.submit-button	Element with class submit-button
input[name="email"]	Input element with name attribute email
div.form-group > input	Input that is a direct child of a div.form-group
table tr:nth-child(3) td:first-child	First cell of the third table row

Playwright Selectors

Playwright-specific selectors target elements by their visible text, ARIA role, or test ID. These are more resilient to markup changes.

Selector	What it matches
text=Submit	Element containing the exact text "Submit"
text=Submit Order	Element containing "Submit Order"
role=button[name="Login"]	Button with accessible name "Login"
data-testid=submit	Element with data-testid="submit" attribute
placeholder=Enter your email	Input with matching placeholder text

XPath Selectors

XPath selectors are useful for complex DOM traversal or when you need to match elements based on their content or position in the tree. Prefix with xpath=.

Selector	What it matches
xpath=//button[@id='submit']	Button with id submit
xpath=//div[contains(@class,'result')]	Div whose class contains result
xpath=//span[text()='Total']	Span with exact text "Total"
xpath=//table//tr[3]/td[1]	First cell in the third row of any table
xpath=//input[@type='text'][1]	First text input on the page

Which selector type should I use?

• CSS selectors for simple, stable cases where elements have clear IDs or classes.
• Playwright text selectors for buttons, links, and labels where visible text is the most reliable identifier.
• XPath selectors for complex DOM traversal, matching based on element text content, or navigating parent-child relationships where CSS falls short.

Cookie and Session Management

Browser sessions within the Web Automation node persist cookies and session state throughout a single web automation sequence (within a single job execution).

Key behaviors:

• Session persistence: Once you log in to a web application, all subsequent web:navigate steps in the same job execution retain the authenticated session. You do not need to log in again for each page.
• Isolation: Each job execution starts with a fresh browser session. There is no cookie or session leakage between separate job executions.
• Reading cookies: Use web:evaluate with a script like document.cookie to read the current cookies in the browser.
• Setting cookies: Use web:evaluate to set cookies via JavaScript, for example:

  document.cookie = "session_token=abc123; path=/; domain=.example.com"

• Local storage and session storage: Access these via web:evaluate using localStorage.getItem('key') or sessionStorage.setItem('key', 'value').

note

If you need to pass authentication tokens between a Web Automation sequence and other nodes (such as an API node), extract the token using web:evaluate and store it in a variable.

Screenshots and PDF Generation

Screenshots

The web:screenshot action captures images in PNG or JPEG format.

Full page vs viewport:

• By default, web:screenshot captures only the visible viewport.
• Set fullPage to true to capture the entire scrollable page, including content below the fold.

Element screenshots:

• Provide a selector parameter to capture just a single element. This is useful for capturing charts, tables, or specific sections of a page.

Output format:

• Screenshots are returned as base64-encoded strings in the step output.
• The base64 output can be passed to subsequent nodes. For example, attach it to an email using the Email node, or upload it to S3 using the AWS S3 node.

Format selection:

• Use png (default) for lossless quality, best for text-heavy content and UI screenshots.
• Use jpeg for smaller file sizes, suitable for photographic content.

PDF Generation

For PDF generation, use web:evaluate to invoke Playwright's PDF functionality via JavaScript:

await page.pdf({ path: 'output.pdf', format: 'A4', printBackground: true })

note

PDF generation is only supported when using the Chromium browser engine. Set the browser connection setting to chromium (the default).

Performance Best Practices

Follow these guidelines to build reliable and efficient web automations.

• Use waitForElement before interacting with elements. Always wait for the target element to be in the expected state (visible or attached) before clicking, typing, or reading. This prevents flaky failures caused by race conditions.

• Avoid hardcoded delays. Do not use a generic delay step to wait for page loading. Instead, use web:waitForElement on a specific element that indicates the page is ready. This is both faster and more reliable.

• Use networkidle for single-page applications. SPAs load data asynchronously after the initial page load. Set waitUntil to networkidle in web:navigate to wait until all network requests have settled.

• Always close the browser. Call web:closeBrowser as the final step in your web automation sequence. This releases memory and browser processes on the job executor host. Unclosed browsers consume resources until the job times out.

• Keep sequences focused. Long browser automation sequences with many steps are inherently fragile. If a workflow requires extensive browser interaction, consider breaking it into smaller, independent jobs that each handle a specific task.

• Use headless mode. The default headless mode (headless: true) is faster and uses less memory than headed mode. Only use headed mode for debugging.

• Consider viewport size. Responsive websites may render differently at different viewport sizes. If elements are missing or laid out unexpectedly, try setting the viewport connection setting to match the expected screen size (e.g., 1920x1080 for desktop layouts).

Common Problems

Problem	Likely Cause	Solution
Element not found	Selector is incorrect, element has not loaded yet, or element is inside an iframe	Verify the selector in browser DevTools. Add a web:waitForElement step before the interaction. Check if the element is inside an iframe and use web:switchFrame first.
Click does not work	Element is covered by an overlay, modal, or sticky header	Scroll the element into view using web:scroll before clicking. Dismiss any overlays first. Try using web:evaluate with element.click() as a fallback.
Page loads but content is missing	Content is loaded asynchronously after the initial page load	Use networkidle as the waitUntil option in web:navigate, or add a web:waitForElement step targeting the specific content you need.
Authentication popup appears	The site uses HTTP basic auth or a browser-level auth dialog	Pass credentials in the URL (e.g., https://user:pass@example.com), or use web:evaluate to handle the authentication programmatically.
Timeout errors	Page takes too long to load or element never appears	Increase the timeout parameter. Check that the page actually loads by taking a screenshot before the failing step. Verify network connectivity from the job executor host.
Screenshots are blank or incomplete	Page has not finished rendering when the screenshot is taken	Add a web:waitForElement step targeting a key element on the page before taking the screenshot. For pages with animations, add a short delay via web:evaluate using new Promise(r => setTimeout(r, 1000)).

Connection

The Web Automation node does not require a traditional connection. Browser sessions are managed by the Playwright runtime on the job executor host. The following global settings can be configured:

Setting	Description
browser	Browser engine: chromium, firefox, webkit. Defaults to chromium
headless	Run in headless mode. Defaults to true
viewport	Browser viewport size (e.g., { "width": 1280, "height": 720 })
userAgent	Custom user agent string
proxy	Proxy configuration for the browser

Output

navigate / click / type

getText

screenshot

evaluate