Understanding the DOM
The Document Object Model (DOM) is a programming interface for web documents. It represents the page so that programs can change the document structure, style, and content. The DOM represents the document as nodes and objects – essentially, it’s a tree-like structure of elements.
When we use CSS or XPath selectors, we’re navigating this tree structure to find specific elements.
CSS Selectors: The Basics
CSS (Cascading Style Sheets) selectors were originally designed for styling HTML elements, but they’re also perfect for locating elements. Let’s start with the fundamentals:
Basic Selectors
1. Tag Name: Selects all elements with the specified tag.
div /* Selects all div elements */
2. ID Selector: Selects an element with a specific ID. IDs must be unique within a page.
#login-button /* Selects the element with id="login-button" */
3. Class Selector: Selects all elements with a specific class.
.error-message /* Selects all elements with class="error-message" */
4. Attribute Selector: Selects elements based on attribute presence or value.
[type="submit"] /* Selects all elements with type="submit" */
[data-test-id] /* Selects all elements with the data-test-id attribute */
Combining Selectors
You can create more specific selectors by combining them:
1. Descendant Combinator: Selects all elements that are descendants of a specified element.
div p /* Selects all <p> elements inside <div> elements */
2. Child Combinator: Selects all elements that are direct children of a specified element.
ul > li /* Selects all <li> elements that are direct children of <ul> */
3. Adjacent Sibling Combinator: Selects an element that is directly after another specific element.
h2 + p /* Selects the first <p> element that immediately follows an <h2> */
4. General Sibling Combinator: Selects elements that are siblings of a specified element.
h2 ~ p /* Selects all <p> elements that follow an <h2> */
CSS Selectors: Intermediate Techniques
As we move to more complex pages, we need more powerful selection techniques:
Pseudo-classes and Pseudo-elements
1. Pseudo-classes: Select elements based on a special state
button:hover /* Selects buttons when the mouse hovers over them */
input:focus /* Selects input elements when they have focus */
li:first-child /* Selects the first <li> element in its parent */
li:nth-child(2) /* Selects the second <li> element in its parent */
2. Pseudo-elements: Select and style a part of an element.
p::first-line /* Selects the first line of each <p> element */
Attribute Selectors with Wildcards
[class^="btn-"] /* Selects elements with a class attribute value starting with "btn-" */
[class$="-primary"] /* Selects elements with a class attribute value ending with "-primary" */
[class*="menu"] /* Selects elements with a class attribute value containing "menu" */
CSS Selectors: Advanced Techniques
At the advanced level, we combine multiple techniques to create highly specific selectors:
/* Selects the third <input> of type "text" within forms that have class "registration" */
form.registration input[type="text"]:nth-of-type(3)
/* Selects disabled submit buttons */
input[type="submit"]:disabled
/* Selects checked checkboxes within elements with class "options" */
.options input[type="checkbox"]:checked
CSS Selector Best Practices for Automation
- Use IDs when available: They’re unique and the fastest selectors.
- Prefer class names over tag names: They’re more specific and less likely to change.
- Avoid overly complex selectors: They’re harder to maintain and more brittle.
- Use data attributes for testing: Create dedicated attributes like
data-test-id="login-button"in your HTML.
XPath: The Basics
XPath (XML Path Language) is a query language for selecting nodes from an XML document. Since HTML can be treated as XML, XPath is a powerful tool for locating elements on a webpage.
Basic XPath Expressions
- Absolute Path: Starts from the root element and follows a complete path to the target
/html/body/div/p
2. Relative Path: Starts from the current context node
//p /* Selects all <p> elements in the document */
3. Selecting by Attribute: Uses attribute values to select nodes.
//input[@type='text'] /* Selects all input elements with type="text" */
//button[@id='submit'] /* Selects the button with id="submit" */
4. Selecting by Text Content: Finds elements with specific text.
//button[text()='Log In'] /* Selects buttons with text exactly "Log In" */
XPath: Intermediate Techniques
XPath provides powerful navigation capabilities:
Navigation Axes
1. Parent: Select the parent of the current node
//input[@id='username']/parent::div
2. Child: Select children of the current node.
//form[@id='login']/child::input
3. Ancestor: Select all ancestors of the current node
//input[@id='username']/ancestor::form
4. Descendant: Select all descendants of the current node.
//div[@class='container']/descendant::input
5. Following-sibling: Select all siblings after the current node
//label[@for='username']/following-sibling::input
6. Preceding-sibling: Select all siblings before the current node
//input[@type='submit']/preceding-sibling::input
Using Functions in XPath
1. contains(): Check if an attribute contains specific text
//div[contains(@class, 'error')]
2. starts-with(): Check if an attribute starts with specific text.
//div[starts-with(@id, 'user-')]
3. normalize-space(): Normalize whitespace in text.
//button[normalize-space(text())='Log In']
XPath: Advanced Techniques
At the advanced level, we combine multiple conditions and use more complex functions:
Multiple Conditions
/* Selects <input> elements that have both class="form-control" and type="text" */
//input[@class='form-control' and @type='text']
/* Selects elements that are either <button> or <input> with type="button" */
//button | //input[@type='button']
Position-based Selection
/* Selects the first <tr> in a table */
//table//tr[1]
/* Selects the last <li> in a list */
//ul/li[last()]
/* Selects the second-to-last <div> in its parent */
//div[last()-1]
Using Variables and Advanced Functions
/* Selects elements where the text is not empty */
//span[string-length(normalize-space(text())) > 0]
/* Selects elements containing numbers */
//div[contains(text(), '0') or contains(text(), '1') or ... or contains(text(), '9')]
Using XPath for Dynamic Content
/* Selects elements whose class contains 'dynamic-' followed by any text */
//div[matches(@class, 'dynamic-.*')]
/* Finds elements containing specific text, ignoring case */
//span[contains(translate(text(), 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz'), 'error')]
How to Capture Locators
Now that we understand both CSS and XPath selectors, let’s explore practical methods for capturing them:
Using Browser DevTools
- Inspect Element: Right-click on an element and select “Inspect” or press F12.
- Copy Selector: Right-click on the element in the Elements panel:
- For CSS: “Copy > Copy selector”
- For XPath: “Copy > Copy XPath”
Using Console to Test Selectors
In the browser console:
// For CSS selectors
document.querySelector('.my-class')
// For XPath
$x("//button[contains(text(), 'Submit')]")
Using Browser Extensions
- ChroPath: Shows XPath and CSS selector for inspected elements
- Selector Gadget: Allows visual selection of elements to generate CSS selectors
- XPath Helper: Evaluates XPath expressions within the browser
Authoring Custom Selectors
When the automatically generated selectors are too brittle, you can create custom ones:
- Analyze the page structure: Look for unique identifiers or patterns
- Start broad, then narrow: Begin with general selectors and refine them
- Test across page states: Ensure selectors work after refreshes or UI changes
Practical Examples and Common Challenges
Handling Dynamic IDs
Elements with dynamic IDs (like id="form-item-5f3e8") require special approaches:
/* CSS */
[id^="form-item-"]
/* XPath */
//div[starts-with(@id, 'form-item-')]
Dealing with Iframes
Elements inside iframes need a different approach:
// First locate the iframe, then its contents
const iframe = document.querySelector('#my-iframe');
const button = iframe.contentDocument.querySelector('.submit-button');
Handling Shadow DOM
Modern web components often use Shadow DOM, which requires special handling:
// Navigate through Shadow DOM
const host = document.querySelector('.shadow-host');
const shadowRoot = host.shadowRoot;
const button = shadowRoot.querySelector('.shadow-button');
Locating Elements in Dynamic Lists
/* The third item in a list */
.list-item:nth-child(3)
/* The second item with a specific class */
.special-item:nth-of-type(2)
Comparative Analysis: CSS vs. XPath
Let’s compare these two locator strategies:
| Feature | CSS | XPath |
|---|---|---|
| Readability | Generally more readable | Can be more verbose |
| Browser Support | Excellent | Good |
| Performance | Usually faster | Can be slower in some browsers |
| Traversal | Limited (can’t select parent) | Comprehensive (can select parent/ancestor) |
| Text Selection | Limited | Can select by text content |
| Complexity | Simpler syntax | More powerful but complex syntax |
Best Practices for Automation
- Use Robust Selectors: Prefer selectors that are less likely to change with UI updates
- Add Test Attributes: Work with developers to add
data-test-idattributes - Layer Your Selectors: Create page object models with multiple selector strategies
- Avoid Hardcoding Text: Use partial text matches or store text in variables
- Keep Selectors Short: Shorter selectors are less brittle
- Test Selector Uniqueness: Ensure your selector returns exactly one element when needed
Practice Exercises
Let’s practice finding elements with both CSS and XPath. For these exercises, consider this HTML snippet:
<div class="container">
<header>
<h1 id="page-title">Shopping Cart</h1>
<div class="user-info">Welcome, <span class="username">JohnDoe</span></div>
</header>
<main>
<div class="cart-summary">
<h2>Your Cart (3 items)</h2>
<div class="total">Total: $145.99</div>
</div>
<ul class="product-list">
<li class="product-item" data-product-id="p123">
<div class="product-name">Wireless Headphones</div>
<div class="product-price">$79.99</div>
<div class="product-quantity">
<label for="qty-p123">Qty:</label>
<input type="number" id="qty-p123" value="1" min="1">
</div>
<button class="remove-button">Remove</button>
</li>
<li class="product-item" data-product-id="p456">
<div class="product-name">Phone Case</div>
<div class="product-price">$24.99</div>
<div class="product-quantity">
<label for="qty-p456">Qty:</label>
<input type="number" id="qty-p456" value="2" min="1">
</div>
<button class="remove-button">Remove</button>
</li>
<li class="product-item out-of-stock" data-product-id="p789">
<div class="product-name">USB Cable</div>
<div class="product-price">$14.99</div>
<div class="product-quantity">
<label for="qty-p789">Qty:</label>
<input type="number" id="qty-p789" value="1" min="1" disabled>
</div>
<div class="stock-status">Out of stock</div>
<button class="remove-button">Remove</button>
</li>
</ul>
</main>
<footer>
<button id="continue-shopping">Continue Shopping</button>
<button id="checkout" class="primary-button">Checkout</button>
</footer>
</div>
Questions:
- Write a CSS selector for the page title “Shopping Cart”.
- Write an XPath expression for the same page title.
- Find the username “JohnDoe” using both CSS and XPath.
- Select all product items using CSS and XPath.
- Find the out-of-stock product using both locator types.
- Select the “Checkout” button using both CSS and XPath.
- Find the quantity input for “Phone Case” using both CSS and XPath.
- Select all “Remove” buttons using both locator types.
- Find the total price ($145.99) using both CSS and XPath.
- Select the product with data-product-id=”p456″ using both CSS and XPath.
Tools for Learning and Practice
To solidify your learning, here are some tools where you can practice CSS and XPath selectors:
- CSS Diner: A fun game to learn CSS selectors
- XPath Playground: Test XPath expressions against HTML
- SelectorGadget: A visual tool to generate CSS selectors
- ChroPath: Browser extension for interactive XPath/CSS
- Selenium IDE: Record and playback testing tool with selectors
Resources for Further Study
Popular Resources
- MDN Web Docs: CSS Selectors: Comprehensive guide to CSS selectors
- W3Schools XPath Tutorial: Easy-to-follow XPath tutorial
- CSS-Tricks: Complex Selectors: Advanced CSS selector techniques
- Selenium Documentation: Official guide on locators
Underrated Resources
- The Art of XPath: In-depth e-book on XPath mastery
- Airbnb Style Guide for CSS: Best practices for maintainable selectors
- XPath Axes Explained: Visual guide to XPath axes
- Robust Selenium Locators: Strategies for building reliable test automation
GitHub Repositories
- Awesome Selectors: Curated list of selector resources
- WebDriverIO Selector Examples: Practical examples of selectors in automation
- XPath Helper: Open source browser extension for XPath
- CSS Selector Generator: Library to generate optimized CSS selectors
AI Tools for Improving Automation
- Playwright Codegen: Automatically generates selectors and test code
- Selenium IDE with AI: Generates more robust selectors using AI
- Applitools Eyes: Visual AI for testing that reduces dependency on brittle selectors
- Mabl: Intelligent test automation with self-healing locators
- TestCraft: AI-powered test automation with dynamic locators
