Liking cljdoc? Tell your friends :D

User Guide

Table of Contents

Introduction

Etaoin offers the Clojure community a simple way to script web browser interactions from Clojure and Babashka. It is a thin abstraction atop the W3C WebDriver protocol that also endeavors to resolve real-world nuances and implementation differences.

History

Ivan Grishaev (@igrishaev) created Etaoin and published its first release to Clojars in Feb of 2017. He and his band of faithful contributors grew Etaoin into a well respected goto-library for browser automation.

In May 2022, finding his time had gravitated more to back-end development, Ivan offered Etaoin for adoption to clj-commons. It is now currently under the loving care of @lread and @borkdude.

Interesting Alternatives

If Etaoin is not your cup of tea, you might also consider:

Clojure based:

Other:

Supported OSes & Browsers

Etaoin’s test suite covers the following OSes and browsers for both Clojure and Babashka:

OSChromeFirefoxSafariEdge

Linux (ubuntu)

yes

yes

-

-

macOS

yes

yes

yes

yes

Windows

yes

yes

-

yes

We did once test against PhantomJS, but since work has long ago stopped on this project, we have dropped testing

Installation

There are two steps to installation:

  1. Add the etaoin library as a dependency to your project

  2. Install the WebDriver for each web browser that you want to control with Etaoin

Add the Etaoin Library Dependency

For Clojure Users

Etaoin supports Clojure v1.9 and above.

Add the following into the :dependencies vector in your project.clj file:

   [etaoin "1.0.40"]

Or the following under :deps in your deps.edn file:

   etaoin/etaoin {:mvn/version "1.0.40"}

For Bababashka Users

We recommend the current release of babashka.

Add the following under :deps to your bb.edn file:

   etaoin/etaoin {:mvn/version "1.0.40"}

Installing the Browser WebDrivers

Etaoin controls web browsers via their WebDrivers. Each browser has its own WebDriver implementation that must be installed.

If it is not already installed, you will need to install the web browser too (Chrome, Firefox, Edge). This is usually via a download from its official site. Safari comes bundled with macOS.

WebDrivers and browsers are updated regularly to fix bugs. Use current versions.

Some ways to install WebDrivers:

  • Google Chrome Driver

  • Geckodriver for Firefox

  • Safari Driver

    • macOS only: Set up Safari options as the Webkit page instructs (scroll down to "Running the Example in Safari" section).

  • Microsoft Edge Driver

    • macos: (download manually)

    • Windows: scoop install edgedriver
      Edge and msedgedriver must match so you might need to specify the version: scoop install edgedriver@101.0.1210.0

    • Download: Official Microsoft download site

  • Phantom.js browser
    (obsolete, no longer tested)

Check your WebDriver installations launching by launching these commands. Each should start a process that includes its own local HTTP server. Use Ctrl-C to terminate.

chromedriver
geckodriver
safaridriver -p 0
msedgedriver
phantomjs --wd

You can optionally run the Etaoin test suite to verify your installation.

Some Etaoin API tests rely on ImageMagick. Install it prior to running test.

From a clone of the Etaoin GitHub repo

  • To check tools of interest to Etaoin:

    bb tools-versions

  • Run all tests:

    bb test all

  • For a smaller sanity test, you might want to run api tests against browsers you are particularly intested in. Example:

    bb test api --browser chrome

During the test run, browser windows will open and close in series. The tests use a local handcrafted HTML file to validate most interactions.

See Troubleshooting if you have problems - or reach out on Clojurians Slack #etaoin or GitHub issues.

Getting Started

The great news is that you can automate your browser directly from your Babashka or Clojure REPL. Let’s interact with Wikipedia:

(require '[etaoin.api :as e]
         '[etaoin.keys :as k])

;; Start WebDriver for Firefox
(def driver (e/firefox)) ;; a Firefox window should appear

;; let's perform a quick Wiki session

;; navigate to wikipedia
(e/go driver "https://en.wikipedia.org/")
;; wait for the search input to load
(e/wait-visible driver [{:id :simpleSearch} {:tag :input :name :search}])

;; search for something interesting
(e/fill driver {:tag :input :name :search} "Clojure programming language")
(e/fill driver {:tag :input :name :search} k/enter)
(e/wait-visible driver {:class :mw-search-results})

;; click on first match
(e/click driver [{:class :mw-search-results} {:class :mw-search-result-heading} {:tag :a}])
(e/wait-visible driver {:id :firstHeading})

;; check our new url location
(e/get-url driver)
;; => "https://en.wikipedia.org/wiki/Clojure"

;; and our new title
(e/get-title driver)
;; => "Clojure - Wikipedia"

;; does page have Clojure in it?
(e/has-text? driver "Clojure")
;; => true

;; navigate through history
(e/back driver)
(e/forward driver)
(e/refresh driver)
(e/get-title driver)
;; => "Clojure - Wikipedia"

;; let's explore the info box
;; What's its caption? Let's select it with a css query:
(e/get-element-text driver {:css "table.infobox caption"})
;; => "Clojure"

;; Ok,now let's try something trickier
;; Maybe we are interested what value the infobox holds for the Family row:
(let [wikitable (e/query driver {:css "table.infobox.vevent tbody"})
      row-els (e/children driver wikitable {:tag :tr})]
  (for [row row-els
        :let [header-col-text (e/with-http-error
                                (e/get-element-text-el driver
                                                       (e/child driver row {:tag :th})))]
        :when (= "Family" header-col-text)]
    (e/get-element-text-el driver (e/child driver row {:tag :td}))))
;; => ("Lisp")

;; Etaoin gives you many options, we can do the same-ish in one swoop in XPath:
(e/get-element-text driver "//table[@class='infobox vevent']/tbody/tr/th[text()='Family']/../td")
;; => "Lisp"

;; When we are done we quit, which stops the Firefox WebDriver
(e/quit driver) ;; the Firefox Window should close

Most api functions require the driver as the first argument. The doto macro can give your code a DSL feel. A portion of the above rewritten with doto:

(require '[etaoin.api :as e]
         '[etaoin.keys :as k])

(def driver (e/firefox))

(doto driver
  (e/go "https://en.wikipedia.org/")
  (e/wait-visible [{:id :simpleSearch} {:tag :input :name :search}])
  (e/fill {:tag :input :name :search} "Clojure programming language")
  (e/fill {:tag :input :name :search} k/enter)
  (e/wait-visible {:class :mw-search-results})
  (e/click [{:class :mw-search-results} {:class :mw-search-result-heading} {:tag :a}])
  (e/wait-visible {:id :firstHeading})
  (e/quit))

Playing Along in your REPL

We encourage you to try the examples in from this user guide in your REPL.

The Interwebs is constantly changing. This makes testing against live sites impractical. The code in this user guide has instead been tested to work against our little sample page.

Until we figure out something more clever, it might be easiest to clone the etaoin GitHub repository and run a REPL from there.

Unless otherwise directed, our examples throughout the rest of this guide will assume you’ve already executed the equivalent of:

(require '[etaoin.api :as e]
         '[etaoin.keys :as k]
         '[clojure.java.io :as io])

(def sample-page (-> "doc/user-guide-sample.html" io/file .toURI str))

(def driver (e/chrome)) ;; or replace chrome with your preference
(e/go driver sample-page)

More Getting Started

You can use fill-multi to shorten the code like so:

(e/fill driver :uname "username")
(e/fill driver :pw "pass")
(e/fill driver :text "some text")

;; let's get what we just set:
(mapv #(e/get-element-value driver %) [:uname :pw :text])
;; => ["username" "pass" "some text"]

into:

;; issue a browser refresh
(e/refresh driver)
(e/fill-multi driver {:uname "username2"
                      :pw "pass2"
                      :text "some text2"})

;; to get what we just set:
(mapv #(e/get-element-value driver %) [:uname :pw :text])
;; => ["username2" "pass2" "some text2"]

If any exception occurs during a browser session, the WebDriver process might hang until you kill it manually. To prevent that, we recommend the with-<browser> macros:

(e/with-firefox driver
  (doto driver
    (e/go "https://google.com")
    ;; ... your code here
    ))

This will ensure that the WebDriver process is closed regardless of what happens.

Unit Tests as Docs

The sections that follow describe how to use Etaoin in more depth.

In addition to these docs, the Etaoin api tests are also a good reference.

Creating and Quitting the Driver

Etaoin comes with many options to create a WebDriver instance.

As previously mentioned, we recommend the with-<browser> convention when you need proper cleanup.

Let’s say we want to create a chrome headless driver:

(require '[etaoin.api :as e])

;; at the base we have:
(def driver (e/boot-driver :chrome {:headless true}))
;; do stuff
(e/quit driver)

;; This can also be expressed as:
(def driver (e/chrome {:headless true}))
;; do stuff
(e/quit driver)

;; Or...
(def driver (e/chrome-headless))
;; do stuff
(e/quit driver)

The with-<browser> functions handle cleanup nicely:

(e/with-chrome {:headless true} driver
  (e/go driver "https://clojure.org"))

(e/with-chrome-headless driver
  (e/go driver "https://clojure.org"))

Replace chrome with firefox, edge or safari for other variants. See API docs for details.

See Driver Options for all options available when creating a driver.

Selecting Elements

Queries (aka selectors) are used to select the elements on the page that Etaoin will interact with.

;; let's start anew by refreshing the page:
(e/refresh driver)
;; select the element with an html attribute id of 'uname' and fill it with text:
(e/fill driver {:id "uname"} "Etaoin")
;; select the first element with an html button tag and click on it:
(e/click driver {:tag :button})

  • A query returns a unique element identifier typically meaningful only as a selector to other functions it is passed to.

  • Many functions accept a query directly. For example:

    ;; specifying query directly
(e/get-element-text driver {:tag :button})
;; => "Submit Form"
;; specifying the result of a query (notice the `-el` fn variant here)
(e/get-element-text-el driver (e/query driver {:tag :button}))
;; => "Submit Form"

An exception is thrown if a query does not find an element. Use exists? to check for element existence:

(e/exists? driver {:tag :button})
;; => true
(e/exists? driver {:id "wont-find-me"})
;; => false

Simple Queries, XPath, CSS

  • :active finds the current active element. The Google page, for example, automatically places the focus on the search input. So there is no need to click on it first:

    (e/go driver "https://google.com")
(e/fill driver :active "Let's search for something" k/enter)

  • any other keyword is translated to an html id attribute:

    (e/go driver sample-page)
(e/fill driver :uname "Etaoin" k/enter)
;; alternatively you can:
(e/fill driver {:id "uname"} "Etaoin Again" k/enter)

  • a string containing an XPath expression. (Be careful when writing XPath manually, see Troubleshooting.) Here we find an input tag with an attribute id of uname and an attribute name of username:

    (e/refresh driver)
(e/fill driver ".//input[@id='uname'][@name='username']" "XPath can be tricky")

;; let's check if that worked as expected:
(e/get-element-value driver :uname)
;; => "XPath can be tricky"

  • a map with either :xpath or :css key with a string in corresponding syntax:

    (e/refresh driver)
(e/fill driver {:xpath ".//input[@id='uname']"} "XPath selector")
(e/fill driver {:css "input#uname[name='username']"} " CSS selector")

;; And here's what we should see in username input field now:
(e/get-element-value driver :uname)
;; => "XPath selector CSS selector"

    This CSS selector reference may be of help.

Map Syntax Queries

A query can also be a map that represents an XPath expression as data. The rules are:

  • A :tag key represents a tag’s name. Defaults to *.

  • An :index key expands into the trailing XPath [x] clause. Useful when you need to select a third row from a table, for example.

  • Any non-special key represents an attribute and its value.

  • :fn/ is a prefix followed by a supported query function.

Examples:

  • find the first div tag

    (= (e/query driver {:tag :div})
   ;; equivalent via xpath:
   (e/query driver ".//div"))
;; => true

  • find the n-th (1-based) div tag

    (= (e/query driver {:tag :div :index 1})
   ;; equivalent via xpath:
   (e/query driver ".//div[1]"))
;; => true

  • find the tag a where the class attribute equals to active

    (= (e/query driver {:tag :a :class "active"})
   ;; equivalent xpath:
   (e/query driver ".//a[@class='active']"))

  • find a form by its attributes:

    (= (e/query driver {:tag :form :method :GET :class :formy})
   ;; equivalent in xpath:
   (e/query driver ".//form[@method=\"GET\"][@class='formy']"))

  • find a button by its text (exact match):

    (= (e/query driver {:tag :button :fn/text "Submit Form"})
   ;; equivalent in xpath:
   (e/query driver ".//button[text() = 'Submit Form']"))

  • find an nth element (p, div, whatever, it does not matter) with "blarg" text:

    (e/get-element-text driver {:fn/has-text "blarg" :index 3})
;; => "blarg in a p"

;; equivalent in xpath:
(e/get-element-text driver ".//*[contains(text(), 'blarg')][3]")
;; => "blarg in a p"

  • find an element that includes a class:

    (e/get-element-text driver {:tag :span :fn/has-class "class1"})
;; => "blarg in a span"

;; equivalent xpath:
(e/get-element-text driver ".//span[contains(@class, 'class1')]")
;; => "blarg in a span"

  • find an element that has the following domain in a href:

    (e/get-element-text driver {:tag :a :fn/link "clojure.org"})
;; => "link 3 (clojure.org)"

;; equivalent xpath:
(e/get-element-text driver ".//a[contains(@href, \"clojure.org\")]")
;; => "link 3 (clojure.org)"

  • find an element that includes all of the specified classes:

    (e/get-element-text driver {:fn/has-classes [:class2 :class3 :class5]})
;; => "blarg in a div"

;; equivalent in xpath:
(e/get-element-text driver ".//*[contains(@class, 'class2')][contains(@class, 'class3')][contains(@class, 'class5')]")
;; => "blarg in a div"

  • find explicitly enabled/disabled input widgets:

    ;; first enabled input
(= (e/query driver {:tag :input :fn/enabled true})
   ;; equivalent xpath:
   (e/query driver ".//input[@enabled=true()]"))
;; => true

;; first disabled input
(= (e/query driver {:tag :input :fn/disabled true})
   ;; equivalent xpath:
   (e/query driver ".//input[@disabled=true()]"))
;; => true

;; return a vector of all disabled inputs
(= (e/query-all driver {:tag :input :fn/disabled true})
   ;; equivalent xpath:
   (e/query-all driver ".//input[@disabled=true()]"))
;; => true

Vector Syntax Queries

A query can be a vector of any valid query expressions. For vector queries, every expression matches the output from the previous expression.

A simple, somewhat contrived, example:

(e/click driver [{:tag :html} {:tag :body} {:tag :button}])
;; our sample page shows form submits, did it work?
(e/get-element-text driver :submit-count)
;; => "1"

You may combine both XPath and CSS expressions

Reminder: the leading dot in an XPath expression means starting at the current node 
;; under the html tag (using map query syntax),
;;  under a div tag with a class that includes some-links (using css query),
;;   click on a tag that has
;;    a class attribute equal to active (using xpath syntax):
(e/click driver [{:tag :html} {:css "div.some-links"} ".//a[@class='active']"])
;; our sample page shows link clicks, did it work?
(e/get-element-text driver :clicked)
;; => "link 2 (active)"

Advanced Queries

Querying the nth Element Matched

Sometimes you may want to interact with the nth element of a query. Maybe you want to click on the second link within:

<ul>
    <li class="search-result">
        <a href="a">a</a>
    </li>
    <li class="search-result">
        <a href="b">b</a>
    </li>
    <li class="search-result">
        <a href="c">c</a>
    </li>
</ul>

You can use the :index like so:

(e/click driver [{:tag :li :class :search-result :index 2} {:tag :a}])
;; check click tracker from our sample page:
(e/get-element-text driver :clicked)
;; => "b"

or you can use the nth-child trick with the CSS expression like this:

;; start page anew
(e/refresh driver)
(e/click driver {:css "li.search-result:nth-child(2) a"})
(e/get-element-text driver :clicked)
;; => "b"

Finally it is also possible to obtain the nth element directly by using query-all:

;; start page anew
(e/refresh driver)
(e/click-el driver (nth (e/query-all driver {:css "li.search-result a"}) 1))
(e/get-element-text driver :clicked)
;; => "b"

Notice:

  • The use of click-el here. The query-all function returns an element, not a selector that can be passed to click directly

  • The nth offset of 1 instead of 2. Clojure’s nth is 0-based, and our search indexes are 1-based.

Querying a Tree

query-tree pipes selectors. Every selector queries elements from the previous one. The first selector finds elements from the root, subsquent selectors find elements downward from each of the previous found elements.

Given the following HTML:

<div id="query-tree-example">
  <div id="one">
    <a href="#">a1</a>
    <a href="#">a2</a>
    <a href="#">a3</a>
  </div>
  <div id="two">
    <a href="#">a4</a>
    <a href="#">a5</a>
    <a href="#">a6</a>
  </div>
  <div id="three">
    <a href="#">a7</a>
    <a href="#">a8</a>
    <a href="#">a9</a>
  </div>
</div>

The following query will find a vector of div tags, then return a set of all a tags under those div tags:

(->> (e/query-tree driver :query-tree-example {:tag :div} {:tag :a})
     (map #(e/get-element-text-el driver %))
     sort)
;; => ("a1" "a2" "a3" "a4" "a5" "a6" "a7" "a8" "a9")

Interacting with Queried Elements

To interact with elements found via a query or query-all function call you have to pass the query result to either click-el or fill-el (note the -el suffix):

(e/click-el driver (first (e/query-all driver {:tag :a})))

You can collect elements into a vector and arbitrarily interact with them at any time:

(e/refresh driver)
(def elements (e/query-all driver {:tag :input :type :text :fn/disabled false}))

(e/fill-el driver (first elements) "This is a test")
(e/fill-el driver (rand-nth elements) "I like tests!")

Interactions

Some basic interactions are covered under Selecting Elements, here we go into other types of interactions and more detail.

Emulate how a Real Person Might Type

Real people type slowly and make mistakes. To emulate these characteristics, you can use the fill-human function. The following options are enabled by default:

{:mistake-prob 0.1 ;; a real number from 0.1 to 0.9, the higher the number, the more typos will be made
 :pause-max    0.2} ;; max typing delay in seconds

which you can choose to override if you wish:

(e/refresh driver)
(e/fill-human driver :uname "soslowsobad"
              {:mistake-prob 0.5
               :pause-max 1})

;; or just use default options by omitting them
(e/fill-human driver :uname " typing human defaults")

For multiple inputs, use fill-human-multi

(e/refresh driver)
(e/fill-human-multi driver {:uname "login"
                            :pw "password"
                            :text "some text"}
                           {:mistake-prob 0.1
                            :pause-max 0.1})

Mouse Clicks

The click function triggers the left mouse click on an element found by a query term:

(e/click driver {:tag :button})

The click function uses only the first element found by the query, which sometimes leads to clicking on the wrong items. To ensure there is one and only one element found, use the click-single function. It acts the same but raises an exception when querying the page returns multiple elements:

(e/click-single driver {:tag :button :name "submit"})

Although double-clicking is rarely purposefully employed on web sites, some naive users might think it is the correct way to click on a button or link.

A double-click can be simulated with double-click function (Chrome, Phantom.js). It can be used, for example, to check your handling of disallowing multiple form submissions.

(e/double-click driver {:tag :button :name "submit"})

There are also "blind" clicking functions. They trigger mouse clicks on the current mouse position:

(e/left-click driver)
(e/middle-click driver)
(e/right-click driver)

Another set of functions do the same but move the mouse pointer to a specified element before clicking on them:

(e/left-click-on driver {:tag :a})
(e/middle-click-on driver {:tag :a})
(e/right-click-on driver {:tag :a})

A middle mouse click can open a link in a new background tab. The right click sometimes is used to imitate a context menu in web applications.

Keyboard Chords

There is an option to input a series of keys simultaneously. This useful to imitate holding a system key like Control, Shift or whatever when typing.

The namespace etaoin.keys includes key constants as well as a set of functions related to keyboard input.

(require '[etaoin.keys :as k])

A quick example of entering ordinary characters while holding Shift:

(e/refresh driver)
(e/wait 1) ;; maybe we need a sec for active element to focus
(e/fill-active driver (k/with-shift "caps is great"))
(e/get-element-value driver :active)
;; => "CAPS IS GREAT"

The main input gets populated with "CAPS IS GREAT". Now maybe you’d like to delete the last word. Assuming you are using Chrome, this is done by pressing backspace holding Alt. Let’s do that:

(e/fill-active driver (k/with-alt k/backspace))
(e/get-element-value driver :active)
;; => "CAPS IS "

Consider a more complex example which repeats real user behaviour. You’d like to delete everything from the input. First, you move the cursor to the very beginning of the input field. Then move it to the end holding shift so everything gets selected. Finally, you press delete to clear the selected text:

(e/fill-active driver k/home (k/with-shift k/end) k/delete)
(e/get-element-value driver :active)
;; => ""

There are also with-ctrl and with-command functions that act as you would expect.

These functions do not apply to the global browser’s shortcuts. For example, neither "Command + R" nor "Command + T" reload the page or open a new tab.

All the etaoin.keys/with-* functions are just wrappers upon the etaoin.keys/chord function that might be used for complex cases.

File Uploading

Clicking on a file input button opens an OS-specific dialog. You technically cannot interact with this dialog using the WebDriver protocol. Use the upload-file function to attach a local file to a file input widget. An exception will be thrown if the local file is not found.

;; open a web page that serves uploaded files
(e/go driver "http://nervgh.github.io/pages/angular-file-upload/examples/simple/")

;; bind element selector to variable; you may also specify an id, class, etc
(def file-input {:tag :input :type :file})

;; upload a file form your system to the first file input
(def my-file "env/test/resources/html/drag-n-drop/images/document.png")
(e/upload-file driver file-input my-file)

;; or pass a native Java File object:
(require '[clojure.java.io :as io])
(def my-file (io/file "env/test/resources/html/drag-n-drop/images/document.png"))
(e/upload-file driver file-input my-file)

When interacting with a remote WebDriver process, you’ll need to avoid the local file existence check by using remote-file like so:

(e/upload-file driver file-input (e/remote-file "/yes/i/really/do/exist.png"))

The remote file is assumed to exist where the WebDriver is running. The WebDriver will throw an error if it does not exist.

Scrolling

Etaoin includes functions to scroll the web page.

The most important one, scroll-query jumps the the first element found with the query term:

(e/go driver sample-page)
;; scroll to the 5th h2 heading
(e/scroll-query driver {:tag :h2} {:index 5})

;; and back up to first h1
(e/scroll-query driver {:tag :h1})

To jump to the absolute pixel positions, use scroll:

(e/scroll driver 100 600)
;; or pass a map with x and y keys
(e/scroll driver {:x 100 :y 600})

To scroll relatively by pixels, use scroll-by with offset values:

;; scroll right by 100 and down by 300
(e/scroll-by driver 100 300)
;; use map syntax to scroll left by 50 and up by 200
(e/scroll-by driver {:x -50 :y -200})

There are two convenience functions to scroll vertically to the top or bottom of the page:

(e/scroll-bottom driver) ;; you'll see the footer...
(e/scroll-top driver)    ;; ...and the header again

The following functions scroll the page in all directions:

(e/scroll driver [0 0])     ;; let's start at top left

(e/scroll-down driver 200)  ;; scrolls down by 200 pixels
(e/scroll-down driver)      ;; scrolls down by the default (100) number of pixels

(e/scroll-up driver 200)    ;; the same, but scrolls up...
(e/scroll-up driver)

(e/scroll-right driver 200) ;; ... and right
(e/scroll-right driver)

(e/scroll-left driver 200)  ;; ...left
(e/scroll-left driver)
All scroll actions are carried out via Javascript. Ensure your browser has it enabled.

Working with frames and iframes

You can only interact with items within an individual frame or iframe by first swithing to them.

Say you have an HTML layout like this:

<iframe id="frame1" src="...">
  <p id="in-frame1">In frame2 paragraph</p>
  <iframe id="frame2" src="...">
    <p id="in-frame2">In frame2 paragraph</p>
  </iframe>
</iframe>

Let’s explore switching to :frame1.

(e/go driver sample-page)
;; we start in the main page, we can't see inside frame1:
(e/exists? driver :in-frame1)
;; => false

;; switch context to frame with id of frame1:
(e/switch-frame driver :frame1)

;; now we can interact with elements in frame1:
(e/exists? driver :in-frame1)
;; => true
(e/get-element-text driver :in-frame1)
;; => "In frame1 paragraph"

;; switch back to top frame (the main page)
(e/switch-frame-top driver)

To reach nested frames, you can dig down like so:

;; switch to the first top-level iframe with the main page: frame1
(e/switch-frame-first driver)
;; downward to the first iframe with frame1: frame2
(e/switch-frame-first driver)
(e/get-element-text driver :in-frame2)
;; => "In frame2 paragraph"
;; back up to frame1
(e/switch-frame-parent driver)
;; back up to main page
(e/switch-frame-parent driver)

Use the with-frame macro to temporarily switch to a target frame, do some work, returning its last expression, while preserving your original frame context.

(e/with-frame driver {:id :frame1}
  (e/with-frame driver {:id :frame2}
    (e/get-element-text driver :in-frame2)))
;; => "In frame2 paragraph"

Executing Javascript

Use js-execute to evaluate a Javascript code in the browser:

(e/js-execute driver "alert('Hello from Etaoin!')")
(e/dismiss-alert driver)

Pass any additional parameters to the script with the arguments array-like object.

(e/js-execute driver "alert(arguments[2].foo)" 1 false {:foo "hello again!"})
(e/dismiss-alert driver)

We have passed 3 arguments:

  1. 1

  2. false

  3. {:foo "hello again!} which is automatically converted to JSON {"foo": "hello again!"}

The alert then presents the foo field of the 3rd (index 2) argument, which is "hello again!".

To return any data to Clojure, add return into your script:

(e/js-execute driver "return {foo: arguments[2].foo, bar: [1, 2, 3]}"
                     ;; same args as previous example:
                     1 false {:foo "hello again!"})
;; => {:bar [1 2 3], :foo "hello again!"}

Notice that the JSON has been automatically converted to edn.

Asynchronous Scripts

Use js-async to deal with scripts that rely on async strategies such as setTimeout. The WebDriver creates and passes a callback as the last argument to your script. To indicate that work is complete, you must call this callback.

Example:

(e/js-async
  driver
  "var args = arguments; // preserve the global args
  // WebDriver added the callback as the last arg, we grab it here
  var callback = args[args.length-1];
  setTimeout(function() {
    // We call the WebDriver callback passing with what we want it to return
    // In this case we pass we chose to return 42 from the arg we passed in
    callback(args[0].foo.bar.baz);
  },
  1000);"
  {:foo {:bar {:baz 42}}})
;; => 42

If you’d like to override the default script timeout, you can do so for the WebDriver session:

;; optionally save the current value for later restoration
(def orig-script-timeout (e/get-script-timeout driver))
(e/set-script-timeout driver 5) ;; in seconds
;; do some stuff
(e/set-script-timeout driver orig-script-timeout)

or for a block of code via with-script-timeout:

(e/with-script-timeout driver 30
  (e/js-async driver "var callback = arguments[arguments.length-1];
                      //some long operation here
                      callback('phew,done!');"))
;; => "phew,done!"

Wait Functions

The main difference between a program and a human being is that the first one operates very fast. A computer operates so fast, that sometimes a browser cannot render new HTML in time. After each action, you might consider including a wait-<something> function that polls a browser until the predicate evaluates to true. Or just (wait <seconds>) if you don’t care about optimization.

The with-wait macro might be helpful when you need to prepend each action with (wait n). For example, the following form:

(e/with-wait 1
  (e/refresh driver)
  (e/fill driver :uname "my username")
  (e/fill driver :text "some text"))

is executed something along the lines of:

(e/wait 1)
(e/refresh driver)
(e/wait 1)
(e/fill driver :uname "my username")
(e/wait 1)
(e/fill driver :text "some text")

and thus returns the result of the last form of the original body.

The (doto-wait n driver & body) acts like the standard doto but prepends each form with (wait n). The above example re-expressed with doto-wait:

(e/doto-wait 1 driver
  (e/refresh)
  (e/fill :uname "my username")
  (e/fill :text "some text"))

This is effectively the same as:

(doto driver
  (e/wait 1)
  (e/refresh)
  (e/wait 1)
  (e/fill :uname "my username")
  (e/wait 1)
  (e/fill :text "some text"))

In addition to with-wait and do-wait there are a number of waiting functions: wait-visible, wait-has-alert, wait-predicate, etc (see the full list in the API docs. They accept default timeout/interval values that can be redefined using the with-wait-timeout and with-wait-interval macros, respectively. They all throw if the wait timeout is exceeded.

(e/with-wait-timeout 15 ;; time in seconds
  (doto driver
    (e/refresh)
    (e/wait-visible {:id :last-section})
    (e/click {:tag :a})
    (e/wait-has-text :clicked "link 1")))

Wait text:

  • wait-has-text waits until an element has text anywhere inside it (including inner HTML).

    (e/click driver {:tag :a})
(e/wait-has-text driver :clicked "link 1")

  • wait-has-text-everywhere like wait-has-text but searches for text across the entire page

    (e/wait-has-text-everywhere driver "ipsum")

Load Strategy

When you navigate to a page, the driver waits until the whole page has been completely loaded. That’s fine in most cases but doesn’t reflect the way human beings interact with the Internet.

Change this default behavior with the :load-strategy option:

  • :normal (the default) wait for full page load (everything, include images, etc)

  • :none don’t wait at all

  • :eager wait for only DOM content to load

For example, the default :normal strategy:

(e/with-chrome driver
  (e/go driver sample-page)
  ;; by default you'll hang on this line until the page loads
  ;; (do-something)
)

Load strategy option of :none:

(e/with-chrome {:load-strategy :none} driver
  (e/go driver sample-page)
  ;; no pause, no waiting, acts immediately
  ;; (do-something)
)

The :eager option only works with Firefox at the moment.

Actions

Etaoin supports Webdriver Actions. They are described as "virtual input devices". They act as little device input scripts that run simultaneously.

Here, in raw form, we have an example of two actions. One controls the keyboard, the other the pointer (mouse).

;; a keyboard input
{:type    "key"
 :id      "some name"
 :actions [{:type "keyDown" :value "a"}
           {:type "keyUp" :value "a"}
           {:type "pause" :duration 100}]}
;; some pointer input
{:type       "pointer"
 :id         "UUID or some name"
 :parameters {:pointerType "mouse"}
 :actions    [{:type "pointerMove" :origin "pointer" :x 396 :y 323}
              ;; double click
              {:type "pointerDown" :duration 0 :button 0}
              {:type "pointerUp" :duration 0 :button 0}
              {:type "pointerDown" :duration 0 :button 0}
              {:type "pointerUp" :duration 0 :button 0}]}

You can create a map manually and send it to the perform-actions method:

(def keyboard-input {:type    "key"
                     :id      "some name"
                     :actions [{:type "keyDown" :value "e"}
                               {:type "keyUp" :value "e"}
                               {:type "keyDown" :value "t"}
                               {:type "keyUp" :value "t"}
                               ;; duration is in ms
                               {:type "pause" :duration 100}]})
;; refresh so that we'll be at the active input field
(e/refresh driver)
;; perform our keyboard input action
(e/perform-actions driver keyboard-input)

Or you might choose to use Etaoin’s action helpers. First you create the virtual input device:

(def keyboard (e/make-key-input))

and then fill it with the actions:

(-> keyboard
    (e/add-key-down k/shift-left)
    (e/add-key-down "a")
    (e/add-key-up "a")
    (e/add-key-up k/shift-left))

Here’s a slightly larger working annotated example:

;; virtual inputs run simultaneously so we'll create a little helper to generate n pauses
(defn add-pauses [input n]
  (->> (iterate e/add-pause input)
       (take (inc n))
       last))

(let [username (e/query driver :uname)
      submit-button (e/query driver {:tag :button})
      mouse (-> (e/make-mouse-input)
                ;; click on username
                (e/add-pointer-click-el
                  username k/mouse-left)
                ;; pause 10 clicks to allow keyboard action to enter username
                ;; (key up and down for each of keypress for etaoin)
                (add-pauses 10)
                ;; click on submit button
                (e/add-pointer-click-el
                  submit-button k/mouse-left))
      keyboard (-> (e/make-key-input)
                   ;; pause 2 ticks to allow mouse action to first click on username
                   ;; (move to username element + click on it)
                   (add-pauses 2)
                   (e/with-key-down k/shift-left
                     (e/add-key-press "e"))
                   (e/add-key-press "t")
                   (e/add-key-press "a")
                   (e/add-key-press "o")
                   (e/add-key-press "i")
                   (e/add-key-press "n")) ]
  (e/perform-actions driver keyboard mouse))

To clear the state of virtual input devices, release all currently pressed keys etc, use the release-actions method:

(e/release-actions driver)

Capturing Screenshots

Calling the screenshot function dumps the current visible page into a PNG image file on your disk. Specify any absolute or relative path. Specify a string:

(e/screenshot driver "target/etaoin-play/screens1/page.png")

or a File object:

(require '[clojure.java.io :as io])
(e/screenshot driver (io/file "target/etaoin-play/screens2/test.png"))

Screenshots for Specific Elements

With Firefox and Chrome, you can also capture a single element within a page, say a div, an input widget, or whatever. It doesn’t work with other browsers at this time.

(e/screenshot-element driver {:tag :form :class :formy} "target/etaoin-play/screens3/form-element.png")

Screenshots after each form

Use with-screenshots to take a screenshot to the specified directory after each form is executed in the code block. The file naming convention is <webdriver-name>-<milliseconds-since-1970>.png

(e/refresh driver)
(e/with-screenshots driver "target/etaoin-play/saved-screenshots"
  (e/fill driver :uname "et")
  (e/fill driver :uname "ao")
  (e/fill driver :uname "in"))

this is equivalent to something along the lines of:

(e/refresh driver)
(e/fill driver :uname "et")
(e/screenshot driver "target/etaoin-play/saved-screenshots/chrome-1.png")
(e/fill driver :uname "ao")
(e/screenshot driver "target/etaoin-play/saved-screenshots/chrome-2.png")
(e/fill driver :uname "in")
(e/screenshot driver "target/etaoin-play/saved-screenshots/chrome-3.png")

Print Page to PDF

Use print-page to print the current page to a PDF file:

(e/with-firefox-headless driver
  (e/go driver sample-page)
  (e/print-page driver "target/etaoin-play/printed.pdf"))

See API docs for details.

Peeking deeper

Sometimes it is useful to go a little deeper.

Invoking WebDriver Implementation Specific Features

The Etaoin API exposes an abstraction of the W3C WebDriver protocol. This is normally all you need, but sometimes you’ll want to invoke a WebDriver implementation feature that is not part of the WebDriver protocol.

Etaoin talks to the WebDriver process via its execute function. You can use this lower level function to send whatever you like to the WebDriver process.

As a real-world example, Chrome supports taking screenshots with transparent backgrounds.

Here we use Etaoin’s execute function to ask Chrome to do this:

(e/with-chrome driver
  ;; navigate to our sample page
  (e/go driver sample-page)
  ;; send the Chrome-specific request for a transparent background
  (e/execute {:driver driver
              :method :post
              :path [:session (:session driver) "chromium" "send_command_and_get_result"]
              :data {:cmd "Emulation.setDefaultBackgroundColorOverride"
                     :params {:color {:r 0 :g 0 :b 0 :a 0}}}})
  ;; and here we take an element screenshot as per normal
  (e/screenshot-element driver
                        {:tag :form}
                        (str "target/etaoin-play/saved-screenshots/form.png")))

Reading a Browser’s Console Logs

Function get-logs returns the browser’s console logs as a vector of maps. Each map has the following structure:

(e/js-execute driver "console.log('foo')")
(e/get-logs driver)
;; [{:level :info,
;;   :message "console-api 2:32 \"foo\"",
;;   :source :console-api,
;;   :timestamp 1654358994253,
;;   :datetime #inst "2022-06-04T16:09:54.253-00:00"}]

;; on the 2nd call, for chrome, we'll find the logs empty
(e/get-logs driver)
;; => []

Currently, logs are available in Chrome and Phantom.js only. The message text and the source type will vary by browser vendor. Chrome wipes the logs once they have been read. Phantom.js wipes the logs when the page location changes.

DevTools: Tracking HTTP Requests, XHR (Ajax)

You can trace events that come from the DevTools panel. This means that everything you see in the developer console now is available through the Etaoin API. This currently only works for Google Chrome.

To start a driver with devtools support enabled specify a :dev map.

(require '[etaoin.api :as e])

(e/with-chrome driver {:dev {}}
  ;; do some stuff
)

The value must not be a map (not nil). When :dev an empty map, the following defaults are used.

{:perf
 {:level :all
  :network? true
  :page? false
  :categories [:devtools.network]
  :interval 1000}}

We’ll work with a driver that enables everything:

(require '[etaoin.api :as e])

(def driver (e/chrome {:dev
                       {:perf
                        {:level :all
                         :network? true
                         :page? true
                         :interval 1000
                         :categories [:devtools
                                      :devtools.network
                                      :devtools.timeline]}}}))

Under the hood, Etaoin sets up a special perfLoggingPrefs dictionary inside the chromeOptions object.

Now that your browser is accumulating these events, you can read them using a special dev namespace.

The results will be different when you try this, but here’s what I experienced:

(require '[etaoin.dev :as dev])

(e/go driver "https://google.com")

(def reqs (dev/get-requests driver))

;; reqs is a vector of maps
(count reqs)
;; 23

;; what were the request types?
(frequencies (map :type reqs))
;; {:script 6,
;;  :other 2,
;;  :xhr 4,
;;  :image 5,
;;  :stylesheet 1,
;;  :ping 3,
;;  :document 1,
;;  :manifest 1}

;; Interesting, we've got Js requests, images, AJAX and other stuff
;; let's take a peek at the last image:
(last (filter #(= :image (:type %)) reqs))
;;    {:state 4,
;;     :id "14535.6",
;;     :type :image,
;;     :xhr? false,
;;     :url
;;     "https://www.google.com/images/searchbox/desktop_searchbox_sprites318_hr.webp",
;;     :with-data? nil,
;;     :request
;;     {:method :get,
;;      :headers
;;      {:Referer "https://www.google.com/?gws_rd=ssl",
;;       :sec-ch-ua-full-version-list
;;       "\" Not A;Brand\";v=\"99.0.0.0\", \"Chromium\";v=\"102.0.5005.61\", \"Google Chrome\";v=\"102.0.5005.61\"",
;;       :sec-ch-viewport-width "1200",
;;       :sec-ch-ua-platform-version "\"10.15.7\"",
;;       :sec-ch-ua
;;       "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"102\", \"Google Chrome\";v=\"102\"",
;;       :sec-ch-ua-platform "\"macOS\"",
;;       :sec-ch-ua-full-version "\"102.0.5005.61\"",
;;       :sec-ch-ua-wow64 "?0",
;;       :sec-ch-ua-model "",
;;       :sec-ch-ua-bitness "\"64\"",
;;       :sec-ch-ua-mobile "?0",
;;       :sec-ch-dpr "1",
;;       :sec-ch-ua-arch "\"x86\"",
;;       :User-Agent
;;       "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.61 Safari/537.36"}},
;;     :response
;;     {:status nil,
;;      :headers
;;      {:date "Sat, 04 Jun 2022 00:11:36 GMT",
;;       :x-xss-protection "0",
;;       :x-content-type-options "nosniff",
;;       :server "sffe",
;;       :cross-origin-opener-policy-report-only
;;       "same-origin; report-to=\"static-on-bigtable\"",
;;       :last-modified "Wed, 22 Apr 2020 22:00:00 GMT",
;;       :expires "Sat, 04 Jun 2022 00:11:36 GMT",
;;       :cache-control "private, max-age=31536000",
;;       :content-length "660",
;;       :report-to
;;       "{\"group\":\"static-on-bigtable\",\"max_age\":2592000,\"endpoints\":[{\"url\":\"https://csp.withgoogle.com/csp/report-to/static-on-bigtable\"}]}",
;;       :alt-svc
;;       "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"",
;;       :cross-origin-resource-policy "cross-origin",
;;       :content-type "image/webp",
;;       :accept-ranges "bytes"},
;;      :mime "image/webp",
;;      :remote-ip "142.251.41.68"},
;;     :done? true}
The details of these responses come from Chrome and are subject to changes to Chrome.

Since we’re mostly interested in AJAX requests, there is a function get-ajax that does the same but filters XHR requests:

;; refresh to fill the logs again
(e/go driver "https://google.com")
(e/wait 2) ;; give ajax requests a chance to finish

(last (dev/get-ajax driver))
;; {:state 4,
;;  :id "14535.59",
;;  :type :xhr,
;;  :xhr? true,
;;  :url
;;    "https://www.google.com/complete/search?q&cp=0&client=gws-wiz&xssi=t&hl=en-CA&authuser=0&psi=OtuaYq-xHNeMtQbkjo6gBg.1654315834852&nolsbt=1&dpr=1",
;;  :with-data? nil,
;;  :request
;;  {:method :get,
;;   :headers
;;   {:Referer "https://www.google.com/",
;;    :sec-ch-ua-full-version-list
;;    "\" Not A;Brand\";v=\"99.0.0.0\", \"Chromium\";v=\"102.0.5005.61\", \"Google Chrome\";v=\"102.0.5005.61\"",
;;    :sec-ch-viewport-width "1200",
;;    :sec-ch-ua-platform-version "\"10.15.7\"",
;;    :sec-ch-ua
;;    "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"102\", \"Google Chrome\";v=\"102\"",
;;    :sec-ch-ua-platform "\"macOS\"",
;;    :sec-ch-ua-full-version "\"102.0.5005.61\"",
;;    :sec-ch-ua-wow64 "?0",
;;    :sec-ch-ua-model "",
;;    :sec-ch-ua-bitness "\"64\"",
;;    :sec-ch-ua-mobile "?0",
;;    :sec-ch-dpr "1",
;;    :sec-ch-ua-arch "\"x86\"",
;;    :User-Agent
;;    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.61 Safari/537.36"}},
;;  :response
;;  {:status nil,
;;   :headers
;;   {:bfcache-opt-in "unload",
;;    :date "Sat, 04 Jun 2022 04:10:35 GMT",
;;    :content-disposition "attachment; filename=\"f.txt\"",
;;    :x-xss-protection "0",
;;    :server "gws",
;;    :expires "Sat, 04 Jun 2022 04:10:35 GMT",
;;    :accept-ch
;;    "Sec-CH-Viewport-Width, Sec-CH-Viewport-Height, Sec-CH-DPR, Sec-CH-UA-Platform, Sec-CH-UA-Platform-Version, Sec-CH-UA-Full-Version, Sec-CH-UA-Arch, Sec-CH-UA-Model, Sec-CH-UA-Bitness, Sec-CH-UA-Full-Version-List, Sec-CH-UA-WoW64",
;;    :cache-control "private, max-age=3600",
;;    :report-to
;;    "{\"group\":\"gws\",\"max_age\":2592000,\"endpoints\":[{\"url\":\"https://csp.withgoogle.com/csp/report-to/gws/cdt1\"}]}",
;;    :x-frame-options "SAMEORIGIN",
;;    :strict-transport-security "max-age=31536000",
;;    :content-security-policy
;;    "object-src 'none';base-uri 'self';script-src 'nonce-xM7BqmSpeu5Zd6usKOP4JA' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/cdt1",
;;    :alt-svc
;;    "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"",
;;    :content-type "application/json; charset=UTF-8",
;;    :cross-origin-opener-policy "same-origin-allow-popups; report-to=\"gws\"",
;;    :content-encoding "br"},
;;   :mime "application/json",
;;   :remote-ip "142.251.41.36"},
;;  :done? true};; => nil

A typical pattern of get-ajax usage is the following. You’d like to check if a certain request has been fired to the server. So you press a button, wait for a while, and then read the requests made by your browser.

Having a list of requests, you search for the one you need (e.g. by its URL) and then check its state. The :state field has got the same semantics of the XMLHttpRequest.readyState. It’s an integer from 1 to 4 with the same behavior.

To check if a request has been finished, done or failed, use these predicates:

;; fill the logs
(e/go driver "https://google.com")
(e/wait 2) ;; give ajax requests a chance to finish

(def reqs (dev/get-ajax driver))
;; you'd search for what you are interested in here
(def req (last reqs))

(dev/request-done? req)
;; => true

(dev/request-failed? req)
;; => nil

(dev/request-success? req)
;; => true

Note that request-done? doesn’t mean the request has succeeded. It only means its pipeline has reached a final step.

when you read dev logs, you consume them from an internal buffer that gets flushed. The second call to get-requests or get-ajax will return an empty list.

Perhaps you want to collect these logs. A function dev/get-performance-logs return a list of logs so you accumulate them in an atom or whatever:

;; setup a collector
(def logs (atom []))

;; make requests
(e/refresh driver)

;; collect as needed
(do (swap! logs concat (dev/get-performance-logs driver))
    true)

(count @logs)
;; 136

The logs->requests and logs->ajax functions convert already fetched logs into requests. Unlike get-requests and get-ajax, they are pure functions and won’t flush anything.

;; convert our fetched requests from our collector atom
(dev/logs->requests @logs)
(last (dev/logs->requests @logs))
;;    {:state 4,
;;     :id "14535.162",
;;     :type :ping,
;;     :xhr? false,
;;     :url
;;     "https://www.google.com/gen_204?atyp=i&r=1&ei=Zd2aYsrzLozStQbzgbqIBQ&ct=slh&v=t1&m=HV&pv=0.48715273690818806&me=1:1654316389931,V,0,0,1200,1053:0,B,1053:0,N,1,Zd2aYsrzLozStQbzgbqIBQ:0,R,1,1,0,0,1200,1053:93,x:42832,e,U&zx=1654316432856",
;;     :with-data? true,
;;     :request
;;     {:method :post,
;;      :headers
;;      {:Referer "https://www.google.com/",
;;       :sec-ch-ua-full-version-list
;;       "\" Not A;Brand\";v=\"99.0.0.0\", \"Chromium\";v=\"102.0.5005.61\", \"Google Chrome\";v=\"102.0.5005.61\"",
;;       :sec-ch-viewport-width "1200",
;;       :sec-ch-ua-platform-version "\"10.15.7\"",
;;       :sec-ch-ua
;;       "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"102\", \"Google Chrome\";v=\"102\"",
;;       :sec-ch-ua-platform "\"macOS\"",
;;       :sec-ch-ua-full-version "\"102.0.5005.61\"",
;;       :sec-ch-ua-wow64 "?0",
;;       :sec-ch-ua-model "",
;;       :sec-ch-ua-bitness "\"64\"",
;;       :sec-ch-ua-mobile "?0",
;;       :sec-ch-dpr "1",
;;       :sec-ch-ua-arch "\"x86\"",
;;       :User-Agent
;;       "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.5005.61 Safari/537.36"}},
;;     :response
;;     {:status nil,
;;      :headers
;;      {:alt-svc
;;       "h3=\":443\"; ma=2592000,h3-29=\":443\"; ma=2592000,h3-Q050=\":443\"; ma=2592000,h3-Q046=\":443\"; ma=2592000,h3-Q043=\":443\"; ma=2592000,quic=\":443\"; ma=2592000; v=\"46,43\"",
;;       :bfcache-opt-in "unload",
;;       :content-length "0",
;;       :content-type "text/html; charset=UTF-8",
;;       :date "Sat, 04 Jun 2022 04:20:32 GMT",
;;       :server "gws",
;;       :x-frame-options "SAMEORIGIN",
;;       :x-xss-protection "0"},
;;      :mime "text/html",
;;      :remote-ip "142.251.41.36"},
;;     :done? true}

When working with logs and requests, pay attention to their count and size. The maps have plenty of keys and the number of items in collections can become very large. Printing a slew of events might freeze your editor. Consider using clojure.pprint/pprint as it relies on max level and length limits.

Postmortem: Auto-save Artifacts in Case of Exception

Sometimes, it can be difficult to diagnose what went wrong during a failed UI test run. Use the with-postmortem to save useful data to disk before the exception was triggered:

Example:

(try
  (e/with-postmortem driver {:dir "target/etaoin-play/postmortem"}
    (e/click driver :non-existing-element))
  (catch Exception _e
    "yup, we threw!"))
;; => "yup, we threw!"

An exception will occur. Under target/etaoin-postmortem you will find three postmortem files named like so: <browser>-<host>-<port>-<datetime>.<ext>, for example:

$ tree target
target
└── etaoin-postmortem
    ├── chrome-127.0.0.1-49766-2022-06-04-12-26-31.html
    ├── chrome-127.0.0.1-49766-2022-06-04-12-26-31.json
    └── chrome-127.0.0.1-49766-2022-06-04-12-26-31.png

The available with-postmortem options are:

{;; directory to save artifacts
 ;; will be created if it does not already exist, defaults to current working directory
 :dir "/home/ivan/UI-tests"

 ;; directory to save screenshots; defaults to :dir
 :dir-img "/home/ivan/UI-tests/screenshots"

 ;; the same but for HTML sources
 :dir-src "/home/ivan/UI-tests/HTML"

 ;; the same but for console logs
 :dir-log "/home/ivan/UI-tests/console"

 ;; a string template to format a timestamp; See SimpleDateFormat Java class
 :date-format "yyyy-MM-dd-HH-mm-ss"}

Driver Options

When creating a driver instance, a map of additional parameters can optionally be passed to tweak the WebDriver and web browser behaviour.

Here, for example, we set an explicit path to the chrome WebDriver binary:

(def driver (e/chrome {:path-driver "/Users/ivan/downloads/chromedriver"}))
OptionDefaults

:host for WebDriver process. When:

Alternative: see :webdriver-url below.

Example: :host "192.68.1.12"

<not set>

:port for WebDriver process. If :port is found to already in use when creating a new local WebDriver process (see :host), a random port will be automatically selected.
See also Connecting to an Existing Running WebDriver.

Example: :port 9997

Random port when lanching local WebDriver process, else varies by vendor:

  • chrome 9515

  • firefox 4444

  • safari 4445

  • edge 17556

  • phantom 8910

:webdriver-url for WebDriver process. When:

  • omitted, creates a new local WebDriver process (unless :host was specified).

  • specified, attempts to connect to an existing running WebDriver process.

See Connecting to an Existing Running WebDriver.

Alternative: see :host above.

Example: "https://chrome.browserless.io/webdriver"

<not set>

:path-driver to WebDriver binary.
Typically used if your WebDriver is not on the PATH.

Example: :path-driver "/Users/ivan/Downloads/geckodriver"

As you would expect, varies by vendor:

  • chrome "chromedriver"

  • firefox "geckodriver"

  • safari "safaridriver"

  • edge "msedgedriver"

  • phantom "phantomjs"

:args-driver specifies extra command line arguments to WebDriver.

Example: :args-driver ["-b" "/path/to/firefox/binary"]

<not set>

:webdriver-failed-launch-retries

Introduced to compensate for mysterious but recoverable failed launches of safari driver.

  • safari 4 (for a total of 5 tries)

  • all other drivers 0

:path-browser to web browser binary.
Typically used if your browser is not on the PATH.

Example: :path-browser "/Users/ivan/Downloads/firefox/firefox"

By default, the WebDriver process automatically finds the web browser.

:args specifies extra command line arguments to web browser, see your web browser docs for what is available.

Example: :args ["--incognito" "--app" "http://example.com"]

<not set>

:log-level web browser minimal console log level. Only messages with this level and above will be collected. From least to most verbose:

  • nil, :off or :none for no messages

  • :err, :error, :severe, :crit or :critical

  • :warn or :warning

  • :debug

  • :all for all messages.

See Reading a Browser’s Console Logs

Example: :log-level :err

:all

driver-log-level WebDriver minimal log level. values vary by browser driver vendor:

  • chrome "OFF" "SEVERE" "WARNING" "INFO" or "DEBUG"

  • firefox "fatal" "error" "warn" "info" "config" "debug" or "trace"

  • phantomjs "ERROR" "WARN" "INFO" "DEBUG"

Example: :driver-log-level "INFO"

  • phantomjs "INFO"

:log-stdout and :log-stderr WebDriver stdout and stderr log files

Specify :inherit to have WebDriver process output destination inherit from calling process (for example, the console).

Example:

  :log-stdout "target/chromedriver-out.log"
  :log-stderr "target/chrmoedriver-err.log"

/dev/null, on Windows NUL

:profile path to custom web browser profile, see Setting the Browser Profile

Example:
:profile "/Users/ivan/Library/Application Support/Firefox/Profiles/iy4iitbg.Test"

<not set>

:env map of environment variables for WebDriver process.

Example: :env {:MOZ_CRASHREPORTER_URL "http://test.com"}

<not set>

:size initial web browser window width and height in pixels

Example: size: [640 480]

[1024 680]

:url default URL to open in web browser.+ Only works in Firefox at this time.

Example: :url "https://clojure.org"

<not set>

:user-agent overrides the web browser User-Agent. Useful for headless mode. See Managing User-Agent.

Example: :user-agent "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

Default is governed by WebDriver vendor.

:download-dir directory for web browser downloads files. See File Download Directory

Example: :download-dir "target/chrome-downloads"

Default is governed by browser vendor.

:headless run the web browser without a UI. See Using Headless Drivers.

Example :headless true

Normally false, but automatically set for driver creation functions like chrome-headless, with-chrome-headless etc.

:prefs map of web browser specific preferences.

Example: see one usage in File Download Directory.

<not set>

:proxy to set web browser proxy.

Example: see HTTP Proxy.

<not set>

:load-strategy controls how long the WebDriver should wait before interacting with a page. See Load Strategy.

Example: :load-strategy :none

:normal

:capabilities WebDriver-specific options. Read vendor docs for WebDriver before setting anything here. You’ll find an example usage under HTTP Proxy.

<none>

Using Headless Drivers

Google Chrome, Firefox, and Microsoft Edge can be run in headless mode. When headless, none of the UI windows appear on the screen. Running without a UI is helpful when:

  • running integration tests on servers that do not have a graphical output device

  • running local tests without having them take over your local UI

Ensure your browser supports headless mode by checking if it accepts --headless command-line argument when running it from the terminal. The Phantom.js driver is headless by its nature (it was never been developed for rendering UI).

When starting a driver, pass the :headless boolean flag to switch into headless mode. This flag is ignored for Safari which, as of June 2022, still does not support headless mode.

(require '[etaoin.api :as e])

(def driver (e/chrome {:headless true})) ;; runs headless Chrome
;; do some stuff
(e/quit driver)

or

(def driver (e/firefox {:headless true})) ;; runs headless Firefox
;; you can also check if a driver is in headless mode:
(e/headless? driver)
;; => true
(e/quit driver)
PhantomJS will always be in headless mode.

There are several shortcuts to run Chrome or Firefox in headless mode:

(def driver (e/chrome-headless))
;; do some stuff
(e/quit driver)

;; or

(def driver (e/firefox-headless {:log-level :all})) ;; with extra settings
;; do some stuff
(e/quit driver)

;; or

(require '[etaoin.api :as e])

(e/with-chrome-headless driver
  (e/go driver "https://clojure.org"))

(e/with-firefox-headless {:log-level :all} driver ;; notice extra settings
  (e/go driver "https://clojure.org"))

There are also the when-headless and when-not-headless macros that conditonally execute a block of commands:

(e/with-chrome driver
  (e/when-not-headless driver
    ;;... some actions that might be not available in headless mode
    )
  ;;... common actions for both versions
  )

File Download Directory

To specify a directory where the browser should download files, use the :download-dir option:

(def driver (e/chrome {:download-dir "target/etaoin-play/chrome-downloads"}))
;; do some downloading
(e/driver quit)

Now, when you click on a download link, the file will be saved to that folder. Currently, only Chrome and Firefox are supported.

Firefox requires specifying MIME-types of the files that should be downloaded without showing a system dialog. By default, when the :download-dir parameter is passed, the library adds the most common MIME-types: archives, media files, office documents, etc. If you need to add your own one, override that Firefox preference manually via the :prefs option:

(def driver (e/firefox {:download-dir "target/etaoin-play/firefox-downloads"
                        :prefs {:browser.helperApps.neverAsk.saveToDisk
                                "some-mime/type-1;other-mime/type-2"}}))
;; do some downloading
(e/driver quit)

To check whether a file was downloaded during UI tests, see Check Whether a File has been Downloaded.

Managing User-Agent

Set a custom User-Agent header with the :user-agent option when creating a driver, for example:

(e/with-firefox {:user-agent "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"}
                driver
  (e/get-user-agent driver))
;; => "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)"

Setting this header is important when using headless browsers as many websites implement some sort of blocking when the User-Agent includes the "headless" string. This can lead to 403 response or some weird behavior of the site.

HTTP Proxy

To set proxy settings use environment variables HTTP_PROXY/HTTPS_PROXY or pass a map of the following type:

{:proxy {:http "some.proxy.com:8080"
         :ftp "some.proxy.com:8080"
         :ssl "some.proxy.com:8080"
         :socks {:host "myproxy:1080" :version 5}
         :bypass ["http://this.url" "http://that.url"]
         :pac-url "localhost:8888"}}

;; example
(e/chrome {:proxy {:http "some.proxy.com:8080"
                   :ssl "some.proxy.com:8080"}})
A :pac-url is for a proxy autoconfiguration file. Used with Safari as other proxy options do not work in Safari.

To fine tune the proxy you use the original object and pass it to capabilities:

(e/chrome {:capabilities
           {:proxy
            {:proxyType "manual"
             :proxyAutoconfigUrl "some.proxy.com:8080"
             :ftpProxy "some.proxy.com:8080"
             :httpProxy "some.proxy.com:8080"
             :noProxy ["http://this.url" "http://that.url"]
             :sslProxy "some.proxy.com:8080"
             :socksProxy "some.proxy.com:1080"
             :socksVersion 5}}})

Connecting to an Existing Running WebDriver

To connect to an existing WebDriver, specify the :host parameter.

When neither the :host nor the :webdriver-url parameter is specified Etaoin will launch a new WebDriver process.

The :host can be a hostname (localhost, some.remote.host.net) or an IP address (127.0.0.1, 183.102.156.31). If the port is not specified, the default :port is assumed.

Both :host and :port are ignored if :webdriver-url is specified.

Example:

;; Connect to an existing chromedriver process on localhost on port 9515
(def driver (e/chrome {:host "127.0.0.1" :port 9515})) ;; for connection to driver on localhost on port 9515

;; Connect to an existing geckodriver process on remote most on default port
(def driver (e/firefox {:host "192.168.1.11"})) ;; the default port for firefox is 4444

;; Connect to a chrome instance on browserless.io via :webdriver-url
;; (replace YOUR-API-TOKEN with a valid browserless.io api token if you want to try this out)
(e/with-chrome {:webdriver-url "https://chrome.browserless.io/webdriver"
                :capabilities {"browserless:token" "YOUR-API-TOKEN"
                               "chromeOptions" {"args" ["--no-sandbox"]}}}
               driver
  (e/go driver "https://en.wikipedia.org/")
  (e/wait-visible driver [{:id :simpleSearch} {:tag :input :name :search}])
  (e/fill driver {:tag :input :name :search} "Clojure programming language")
  (e/fill driver {:tag :input :name :search} k/enter)
  (e/get-title driver))
;; => "Clojure programming language - Search results - Wikipedia"

Setting the Browser Profile

When running Chrome or Firefox, you may specify a special web browser profile made for test purposes. A profile is a folder that keeps browser settings, history, bookmarks, and other user-specific data.

Imagine, for example, that you’d like to run your integration tests against a user that turned off Javascript execution or image rendering.

This is a hypothetical example. Turning off JavaScript will affect/break certain WebDriver features. And it can affect certain WebDriver implementations, for example.

Create and Find a Profile in Chrome

  1. In the right top corner of the main window, click on a user button.

  2. In the dropdown, select "Manage People".

  3. Click "Add person", submit a name and press "Save".

  4. The new browser window should appear. Now, setup the new profile as you want.

  5. Open chrome://version/ page. Copy the file path that is beneath the Profile Path caption.

Create and Find a Profile in Firefox

  1. Run Firefox with -P, -p or -ProfileManager key as the official page describes.

  2. Create a new profile and run the browser.

  3. Setup the profile as you need.

  4. Open about:support page. Near the Profile Folder caption, press the Show in Finder button. A new folder window should appear. Copy its path from there.

Running a Driver with a Profile

Once you’ve got a profile path, launch a driver with the :profile key as follows:

;; Chrome
(def chrome-profile
  "/Users/ivan/Library/Application Support/Google/Chrome/Profile 2/Default")

(def chrome-driver (e/chrome {:profile chrome-profile}))

;; Firefox
(def ff-profile
  "/Users/ivan/Library/Application Support/Firefox/Profiles/iy4iitbg.Test")

(def firefox-driver (e/firefox {:profile ff-profile}))

Writing and Running Integration Tests For Your Application

Headless Testing

Is is not unusual for Continuous Integration services to have no display. This seems to be especially true for Linux runners.

When running your tests on Linux with no display, you have 2 choices:

  • run the WebDriver in headless mode

  • use a virtual display

Things being what they are, WebDrivers can behave differently when run headless.

The technologies we use for Etaoin’s CI testing on GitHub Actions for Linux are:

  • Xvfb - acts as an X virtual display

  • fluxbox - a lightweight windows manager (needed by geckodriver/Firefox to support window positioning operations)

You can see how we make use of these tools in the Etaoin test script, but in a nutshell:

To install:

sudo apt get install -y xvfb fluxbox
As of this writing Xvfb is pre-installed on the linux runner on GitHub Actions, but fluxbox is not.

Ensure DISPLAY env var is set:

export DISPLAY=:99.0

Launch the virtual display and fluxbox:

Xvfb :99 -screen 0 1024x768x24 &
fluxbox -display :99 &

Basic Fixture

It is desirable to have your tests be independent of one another. One way to achieve this is through the use of a test fixture. The fixture’s job is to, for each test:

  1. create a new driver

  2. run the test with the driver

  3. shutdown the driver

A dynamic *driver* var might be used to hold the driver.

(ns project.test.integration
  "A module for integration tests"
  (:require [clojure.test :refer [deftest is use-fixtures]]
            [etaoin.api :as e]))

(def ^:dynamic *driver*)

(defn fixture-driver
  "Executes a test running a driver. Bounds a driver
   with the global *driver* variable."
  [f]
  (e/with-chrome [driver]
    (binding [*driver* driver]
      (f))))

(use-fixtures
  :each ;; start and stop driver for each test
  fixture-driver)

;; now declare your tests

(deftest ^:integration
  test-some-case
  (doto *driver*
    (e/go url-project)
    (e/click :some-button)
    (e/refresh)
    ...
    ))

If for some reason you want to reuse a single driver instance for all tests:

(ns project.test.integration
  "A module for integration tests"
  (:require [clojure.test :refer [deftest is use-fixtures]]
            [etaoin.api :as e]))

(def ^:dynamic *driver*)

(defn fixture-browser [f]
  (e/with-chrome-headless {:args ["--no-sandbox"]} driver
    (e/disconnect-driver driver)
    (binding [*driver* driver]
      (f))
    (e/connect-driver driver)))

;; creating a session every time that automatically erases resources
(defn fixture-clear-browser [f]
  (e/connect-driver *driver*)
  (e/go *driver* "http://google.com")
  (f)
  (e/disconnect-driver *driver*))

;; this is run `once` before running the tests
(use-fixtures
  :once
  fixture-browser)

;; this is run `every` time before each test
(use-fixtures
  :each
  fixture-clear-browser)

...some tests

For faster testing you can use this example:

.....

(defn fixture-browser [f]
  (e/with-chrome-headless {:args ["--no-sandbox"]} driver
    (binding [*driver* driver]
      (f))))

;; note that resources, such as cookies, are deleted manually,
;; so this does not guarantee that the tests are clean
(defn fixture-clear-browser [f]
  (e/delete-cookies *driver*)
  (e/go *driver* "http://google.com")
  (f))

......

Multi-Driver Fixtures

In the example above, we examined a case when you run tests against a single type of driver. However, you may want to test your site on multiple drivers, say, Chrome and Firefox. In that case, your fixture may become a bit more complex:

(def driver-type [:firefox :chrome])

(defn fixture-drivers [f]
  (doseq [type driver-types]
    (e/with-driver type {} driver
      (binding [*driver* driver]
        (testing (format "Testing in %s browser" (name type))
          (f))))))

Now, each test will be run twice. Once for Firefox and then once Chrome. Please note the test call is prepended with the testing macro that puts the driver name into the report. Once you’ve got an error, you’ll easily find what driver failed the tests exactly.

See also Etaoin’s API tests for an example of this strategy.

Postmortem Handler To Collect Artifacts

To save some artifacts in case of an exception, wrap the body of your test into the with-postmortem handler as follows:

(deftest test-user-login
  (e/with-postmortem *driver* {:dir "/path/to/folder"}
    (doto *driver*
      (e/go "http://127.0.0.1:8080")
      (e/click-visible :login)
      ;; any other actions...
      )))

If any exception occurs in that test, artifacts will be saved.

To not copy and paste the options map, declare it at the top of the module. If you use Circle CI, it would be great to save the data into a special artifacts directory that might be downloaded as a zip file once the build has been finished:

(def pm-dir
  (or (System/getenv "CIRCLE_ARTIFACTS") ;; you are on CI
      "/some/local/path"))               ;; local machine

(def pm-opt
  {:dir pm-dir})

Now pass that map everywhere into PM handler:

  ;; test declaration
  (e/with-postmortem *driver* pm-opt
    ;; test body goes here
    )

Once an error occurs, you will find a PNG image that represents your browser page at the moment of exception and HTML dump.

See Postmortem: Auto-save Artifacts in Case of Exception.

Running Tests By Tag

Since UI tests may take lots of time to pass, it’s definitely a good practice to pass both server and UI tests independently from each other.

If you are using leiningen, here are a few tips.

First, add ^:integration tag to all the tests that are run under the browser like follows:

(deftest ^:integration
  test-password-reset-pipeline
  (doto *driver*
    (go url-password-reset)
    (click :reset-btn)
    ;; and so on...
  ))

Then, open your project.clj file and add test selectors:

:test-selectors {:default (complement :integration)
                 :integration :integration}

Now, when you launch lein test you will run all the tests except browser integration tests. To run integration tests, launch lein test :integration.

Check Whether a File has been Downloaded

Sometimes, a file starts to download automatically when you click on a link or just visit some page. In tests, you might need to ensure a file really has been downloaded successfully. A common scenario would be:

  • provide a custom empty download folder when running a browser (see File Download Directory).

  • Click on a link or perform any action needed to start file downloading.

  • Wait for some time; for small files, 5-10 seconds would be enough.

  • Using files API, scan that directory and try to find a new file. Check if it matches a proper extension, name, creation date, etc.

Example:

(require '[clojure.java.io :as io]
         '[clojure.string :as str])

;; Local helper that checks whether it is really an Excel file.
(defn xlsx? [file]
  (-> file
      .getAbsolutePath
      (str/ends-with? ".xlsx")))

;; Top-level declarations
(def DL-DIR "/Users/ivan/Desktop")
(def driver (e/chrome {:download-dir DL-DIR}))

;; Later, in tests...
(e/click-visible driver :download-that-application)
(e/wait driver 7) ;; wait for a file has been downloaded

;; Now, scan the directory and try to find a file:
(let [files (file-seq (io/file DL-DIR))
      found (some xlsx? files)]
  (is found (format "No *.xlsx file found in %s directory." DL-DIR)))

Running Selenium IDE files

Etaoin can play the files produced by Selenium IDE. Selenium IDE allows you to record web interactions for later playback. It is installed as an optional extension in your web browser.

Once installed, and activated, it records your actions into a JSON file with the .side extension. You can save that file and run it with Etaoin.

Let’s imagine you’ve installed the IDE and recorded some actions as per Selenium IDE documentation. Now that you have a test.side file, you could do something like this:

(require '[clojure.java.io :as io]
         '[etaoin.api :as e]
         '[etaoin.ide.flow :as flow])

(def driver (e/chrome))

(def ide-file (io/resource "ide/test.side"))

(def opt
    {;; The base URL redefines the one from the file.
     ;; For example, the file was written on the local machine
     ;; (http://localhost:8080), and we want to perform the scenario
     ;; on staging (https://preprod-001.company.com)
     :base-url "https://preprod-001.company.com"

     ;; keywords :test-.. and :suite-.. (id, ids, name, names)
     ;; are used to select specific tests. When not passed,
     ;; all tests get run. For example:

     :test-id "xxxx-xxxx..."         ;; a single test by its UUID
     :test-name "some-test"          ;; a single test by its name
     :test-ids ["xxxx-xxxx...", ...] ;; multiple tests by their ids
     :test-names ["some-test1", ...] ;; multiple tests by their names

     ;; the same for suites:

     :suite-id    ...
     :suite-name  ...
     :suite-ids   [...]
     :suite-names [...]})

(flow/run-ide-script driver ide-file opt)

Everything related to the IDE feature can be found under the etaoin.ide namespace.

CLI Arguments

You may also run a .side script from the command line. Here is a clojure example:

clojure -M -m etaoin.ide.main -d firefox -p '{:port 8888 :args ["--no-sandbox"]}' -r ide/test.side

As well as from an uberjar. In this case, Etaoin must be in the primary dependencies, not the :dev or :test related.

java -cp .../poject.jar -m etaoin.ide.main -d firefox -p '{:port 8888}' -f ide/test.side

We support the following arguments (check them out using the clojure -M -m etaoin.ide.main -h command):

  -d, --driver-name name   :chrome  The name of driver. The default is `:chrome`
  -p, --params params      {}       Parameters for the driver represented as an
                                    EDN string, e.g '{:port 8080}'
  -f, --file path                   Path to an IDE file on disk
  -r, --resource path               Path to an IDE resource
      --test-ids ids                Comma-separeted test ID(s)
      --suite-ids ids               Comma-separeted suite ID(s)
      --test-names names            Comma-separeted test name(s)
      --suite-names names           Comma-separeted suite name(s)
      --base-url url                Base URL for tests
  -h, --help

Pay attention to --params. This must be an EDN string representing a Clojure map. That’s the same map that you pass into a driver at creation time.

Please note the IDE support is still experimental. If you encounter unexpected behavior feel free to open an issue. At the moment, we only support Chrome and Firefox for IDE files.

Webdriver in Docker

To work with the driver in Docker, you can take ready-made images:

Example for Chrome:

docker run --name chromedriver -p 9515:4444 -d -e CHROMEDRIVER_WHITELISTED_IPS='' robcherry/docker-chromedriver:latest

for Firefox:

docker run --name geckodriver -p 4444:4444 -d instrumentisto/geckodriver

To connect to an existing running WebDriver process you need to specify the :host. In this example :host would be localhost or 127.0.0.1. The :port would be the appropirate port for the running WebDriver process as exposed by docker. If the port is not specified, the default port is set.

(def driver (e/chrome-headless {:host "localhost" :port 9515 :args ["--no-sandbox"]}))
(def driver (e/firefox-headless {:host "localhost"})) ;; will try to connect to port 4444

Troubleshooting

Old Versions of WebDrivers can have Limitations

Reproduction

For example, chromedriver used to throw an error when calling maximize:

(e/with-chrome driver
  (e/maximize driver))
;; an exception with "cannot get automation extension" was thrown
Cause

This was a bug in chromedriver that was fixed in chromdriver v2.28.

Solution

Updating to the current WebDriver resolved the issue.

New Versions of WebDrivers can Introduce Bugs

Reproduction

For example, chromedriver v103 started sporadically failing with unknown error: cannot determine loading status\nfrom unknown error: unexpected command response.

Cause

Likely a bug in chromedriver

Solution

Upgrade to newer version that fixes bug. For this particular bug I simply suffered retrying failed tests while waiting for newer version.

Clicking on a Link Does not Work with Safari

Reproduction

An attempt to click on a link does nothing.

Given the following HTML

<div class="faculty_course_title" title="Course Title">
 <a href="https://clojure.org">Course Title</a>
</div>

An attempt to click on the link does nothing:

(e/click-single driver [{:tag :div :class "faculty_course_title"} {:tag :a}])
Cause

Odd as it may seem, this is a long-standing (maybe on-again-off-again?) bug in safaridriver.

Work-around

You can, if you wish, employ JavaScript to click the link:

(let [elem (e/query driver [{:tag :div :class "faculty_course_title"} {:tag :a}])]
  (e/js-execute driver "arguments[0].click()" (e/el->ref elem)))

XPath and Searching from Root vs Current node

Reproduction
;; we intend to find an element with the text 'some' under an element with id 'mishmash'
(e/get-element-text driver [{:id :mishmash} "//*[contains(text(),'some')]"])
;; => "A little sample page to illustrate some concepts described in the Etaoin user guide."
;; but we've found the first element with text 'some'
Cause

In a vector, every expression searches from the previous one in a loop. Without a leading dot, the XPath "//..." clause means to find an element from the root of the whole page. With a dot, it means to find from the current node, which is one from the previous query, and so forth.

Solution

Add the XPath dot.

(e/get-element-text driver [{:id :mishmash} ".//*[contains(text(),'some')]"])
;; => "some other paragraph"
;; that's what we were looking for!

Clicking On Non-Visible Element

Reproduction
(e/click driver :cantseeme)
;; as of this writing, on chrome throws an exception with message containing 'not interactable'
Cause

You cannot interact with an element that is not visible or is so small that a human could not click on it.

Selectors not Working

Symptom

Selectors for locating elements are not working, even though the elements are clearly available.

Possible cause

Your script may have clicked a link that opened a new tab or window. Even though the new window is in the foreground, the driver instance is still connected to the original window.

Solution

Call switch-window-next when a new tab or window is opened to point the driver to the new tab/window.

Unpredictable errors in Chrome when the window is not active

Reproduction

when you focus on another window, a WebDriver session that is run under Google Chrome fails.

Solution

Google Chrome may suspend a tab when it has been inactive for some time. When the page is suspended, no operation can be done on it. No clicks, Js execution, etc. So try to keep Chrome window active during test session.

Invalid argument: can’t kill an exited process for Firefox

Reproduction

When you try to start the driver you get an error:

(def driver (e/firefox {:headless true}))
;; throws an exception containing message with 'invalid argument: can't kill an exited process'
Possible Cause

Running Firefox as root in a regular user’s session is not supported

To Diagnose

Run the driver with the path to the log files and the "trace" log level and explore the output.

(def driver (firefox {:log-stdout "ffout.log" :log-stderr "fferr.log" :driver-log-level "trace"}))
Similar Problem

https://github.com/mozilla/geckodriver/issues/1655

DevToolsActivePort file doesn’t exist error on Chrome

Reproduction

When you try to start the chromedriver you get an error:

(def driver (e/chrome))
;; throws an exception with message containing 'DevToolsActivePort file doesn't exist'
Possible Cause

A common cause for Chrome to crash during startup is running Chrome as root user (administrator) on Linux. While it is possible to work around this issue by passing --no-sandbox flag when creating your WebDriver session, such a configuration is unsupported and highly discouraged. You need to configure your environment to run Chrome as a regular user instead.

Potential Solution

Run driver with an argument --no-sandbox. Caution! This bypasses OS security model.

(e/with-chrome {:args ["--no-sandbox"]} driver
  (e/go driver "https://clojure.org"))
Similiar Problem

A similar problem is described here

Developer Guide❯

Can you improve this documentation? These fine people already did:
Lee Read, github-actions[bot], lread, Uday Verma, Raimon Grau & John KrasnayEdit on GitHub

cljdoc is a website building & hosting documentation for Clojure/Script libraries

Keyboard shortcutsReport a problemcljdoc on GitHub
× close