Tips for accessing elements inside #shadow-root (open) when clearing browsing data in Chrome Browser with the help of a cssSelector

Recently, I came across a discussion on how to automate shadow DOM elements using Selenium, specifically dealing with #shadow-root (open) elements.

During my attempts to locate the Clear data button within the popup for Clear browsing data, which appears when accessing the URL

chrome://settings/clearBrowserData
through Selenium, I encountered difficulties in locating the following element:

#shadow-root (open)
<settings-privacy-page>

Here is a snapshot for reference:

https://i.sstatic.net/RUUI6.png

Below are the coding trials and errors faced while using Selenium:

  • Attempt 1:

    WebElement root5 = shadow_root4.findElement(By.tagName("settings-privacy-page"));
    
    • Error:

      Exception in thread "main" org.openqa.selenium.JavascriptException: javascript error: b.getElementsByTagName is not a function
      
  • Attempt 2:

    WebElement root5 = shadow_root4.findElement(By.cssSelector("settings-privacy-page"));
    
    • Error:

      Exception in thread "main" org.openqa.selenium.NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"settings-privacy-page"}
      
  • Attempt 3:

    WebElement root5 = (WebElement)((JavascriptExecutor)shadow_root4).executeScript("return document.getElementsByTagName('settings-privacy-page')[0]");
    
    • Error:

      Exception in thread "main" java.lang.ClassCastException: org.openqa.selenium.remote.RemoteWebElement cannot be cast to org.openqa.selenium.JavascriptExecutor
      

If needed, the initial code block provided above works perfectly:

driver.get("chrome://settings/clearBrowserData");
WebElement root1 = driver.findElement(By.tagName("settings-ui"));
WebElement shadow_root1 = expand_shadow_element(root1);

WebElement root2 = shadow_root1.findElement(By.cssSelector("settings-main#main"));
WebElement shadow_root2 = expand_shadow_element(root2);

WebElement root3 = shadow_root2.findElement(By.cssSelector("settings-basic-page[role='main']"));
WebElement shadow_root3 = expand_shadow_element(root3);

WebElement root4 = shadow_root3.findElement(By.cssSelector("settings-section[page-title='Privacy and security']"));
WebElement shadow_root4 = expand_shadow_element(root4);

Note: The function expand_shadow_element() works without any issues.

Answer №1

If you need to access the 'Clear Data' element, you can utilize the following JavaScript code to locate the element and then take action on it.

return document.querySelector('settings-ui').shadowRoot.querySelector('settings-main').shadowRoot.querySelector('settings-basic-page').shadowRoot.querySelector('settings-section > settings-privacy-page').shadowRoot.querySelector('settings-clear-browsing-data-dialog').shadowRoot.querySelector('#clearBrowsingDataDialog').querySelector('#clearBrowsingDataConfirm')

Here is an example script to demonstrate this:

driver.get("chrome://settings/clearBrowserData");
driver.manage().window().maximize();
JavascriptExecutor js = (JavascriptExecutor) driver; 
WebElement clearData = (WebElement) js.executeScript("return document.querySelector('settings-ui').shadowRoot.querySelector('settings-main').shadowRoot.querySelector('settings-basic-page').shadowRoot.querySelector('settings-section > settings-privacy-page').shadowRoot.querySelector('settings-clear-browsing-data-dialog').shadowRoot.querySelector('#clearBrowsingDataDialog').querySelector('#clearBrowsingDataConfirm')");
// now you can click on the clear data button
clearData.click();

Edit 2: Explanation

Issue: Selenium does not directly support interacting with Shadow DOM elements due to them not being in the current DOM structure. This results in a NoSuchElementException when trying to access elements within the shadow dom.

Shadow DOM: https://i.sstatic.net/nHa2j.png

Note: For better comprehension, refer to the terms depicted in the image.

Solution:

To work with a shadow element, initially identify the shadow host to which the shadow DOM is attached. The following method can be used to obtain the shadow root based on the shadowHost.

private static WebElement getShadowRoot(WebDriver driver, WebElement shadowHost) {
    JavascriptExecutor js = (JavascriptExecutor) driver;
    return (WebElement) js.executeScript("return arguments[0].shadowRoot", shadowHost);
}

Then, navigate through the shadow tree element using the shadowRoot Element.

// locate the shadowHost in the original DOM via findElement
WebElement shadowHost = driver.findElement(By.cssSelector("shadowHost_CSS"));
// acquire the shadow root
WebElement shadowRoot = getShadowRoot(driver, shadowHost);
// access shadow tree element
WebElement shadowTreeElement = shadowRoot.findElement(By.cssSelector("shadow_tree_element_css"));

To streamline the above processes, the subsequent method has been devised.

public static WebElement getShadowElement(WebDriver driver, WebElement shadowHost, String cssOfShadowElement) {
    WebElement shadowRoot = getShadowRoot(driver, shadowHost);
    return shadowRoot.findElement(By.cssSelector(cssOfShadowElement));
}

Henceforth, the shadowTree Element can be fetched with just a single method invocation

WebElement shadowHost = driver.findElement(By.cssSelector("shadowHost_CSS_Goes_here));
WebElement shadowTreeElement = getShadowElement(driver, shadowHost, "shadow_tree_element_css");

Subsequently, standard operations like .click(), .getText() can be performed as usual.

shadowTreeElement.click()

This procedure appears uncomplicated for scenarios having only one level of shadow DOM. However, in this instance, there are multiple levels of shadow DOMs necessitating access to each element by traversing through every shadow host and root. https://i.sstatic.net/UvWec.png

The ensuing snippet showcases usage of the aforementioned methods (getShadowElement and getShadowRoot)

// Identify shadowHost on the prevailing DOM
WebElement shadowHostL1 = driver.findElement(By.cssSelector("settings-ui"));

// subsequently pinpoint the shadowElement while navigating all levels of shadow
WebElement shadowElementL1 = getShadowElement(driver, shadowHostL1, "settings-main");
WebElement shadowElementL2 = getShadowElement(driver, shadowElementL1,"settings-basic-page");
WebElement shadowElementL3 = getShadowElement(driver, shadowElementL2,"settings-section > settings-privacy-page");
WebElement shadowElementL4 = getShadowElement(driver, shadowElementL3,"settings-clear-browsing-data-dialog");
WebElement shadowElementL5 = getShadowElement(driver, shadowElementL4,"#clearBrowsingDataDialog");
WebElement clearData = shadowElementL5.findElement(By.cssSelector("#clearBrowsingDataConfirm"));
System.out.println(clearData.getText());
clearData.click();

All the preceding steps can be consolidated into a single JavaScript call as elucidated at the outset of the answer (included below chiefly to minimize confusion).

WebElement clearData = (WebElement) js.executeScript("return document.querySelector('settings-ui').shadowRoot.querySelector('settings-main').shadowRoot.querySelector('settings-basic-page').shadowRoot.querySelector('settings-section > settings-privacy-page').shadowRoot.querySelector('settings-clear-browsing-data-dialog').shadowRoot.querySelector('#clearBrowsingDataDialog').querySelector('#clearBrowsingDataConfirm')");

Screenshot: https://i.sstatic.net/k8PAO.png

Answer №2

Recently, I encountered a similar test that involved clearing the browsing history in Chrome. The only difference was that I cleared the data after navigating to the advanced section of the pop-up window. If you're struggling with clicking just the "Clear data" button, it's possible that you may have overlooked a hierarchy element or confused sibling and parent elements. Looking at your code, it seems like you understand the importance of proper sequencing to access shadow DOM elements, as explained earlier.
Now, addressing your current issue, here is a snippet of my code that effectively clears the browsing history and waits for the process to complete before proceeding:

public WebElement expandRootElement(WebElement element) {
WebElement ele = (WebElement) ((JavascriptExecutor) driver).executeScript("return arguments[0].shadowRoot",
        element);
return ele;
}

public void clearBrowsingHistory() throws Exception {

    WebDriverWait wait = new WebDriverWait(driver, 15);
    driver.get("chrome://settings/clearBrowserData");
// Get shadow root elements
    WebElement shadowRoot1 = expandRootElement(driver.findElement(By.xpath("/html/body/settings-ui")));

    WebElement root2 = shadowRoot1.findElement(By.cssSelector("settings-main"));
    WebElement shadowRoot2 = expandRootElement(root2);

    WebElement root3 = shadowRoot2.findElement(By.cssSelector("settings-basic-page"));
    WebElement shadowRoot3 = expandRootElement(root3);

    WebElement root4 = shadowRoot3
        .findElement(By.cssSelector("#advancedPage > settings-section > settings-privacy-page"));
    WebElement shadowRoot4 = expandRootElement(root4);

    WebElement root5 = shadowRoot4.findElement(By.cssSelector("settings-clear-browsing-data-dialog"));
    WebElement shadowRoot5 = expandRootElement(root5);

    WebElement root6 = shadowRoot5
        .findElement(By.cssSelector("cr-dialog div[slot ='button-container'] #clearBrowsingDataConfirm"));

    root6.click();
    wait.until(ExpectedConditions.invisibilityOf(root6));
}

If you stick to default options in the pop-up, this code should work smoothly for you (otherwise, additional code will be needed to select checkboxes). Let me know if this resolves your problem. Hopefully this information proves useful I've also included an image snapshot of the screen here- image

Answer №3

The Locator Strategy mentioned by @supputuri and utilizing document.querySelector() works flawlessly when using

However, if the targeted element is within the , it is necessary to apply a WebDriverWait for elementToBeClickable(). The following solution can be used:

  • Code Block:

    driver.get("chrome://settings/clearBrowserData");
    new WebDriverWait(driver, 5).until(ExpectedConditions.elementToBeClickable((WebElement) ((JavascriptExecutor)driver).executeScript("return document.querySelector('settings-ui').shadowRoot.querySelector('settings-main').shadowRoot.querySelector('settings-basic-page').shadowRoot.querySelector('settings-section > settings-privacy-page').shadowRoot.querySelector('settings-clear-browsing-data-dialog').shadowRoot.querySelector('#clearBrowsingDataDialog').querySelector('#clearBrowsingDataConfirm')"))).click();
    System.out.println("Clear data Button Clicked");
    
  • Console Output:

    Clear data Button Clicked
    

Answer №4

I encountered an InvalidArgumentEXception while attempting to locate a shadowRoot element in the DOM with Selenium 4.3.0 and Chrome Version 103.0.5060.134

The resolution is as follows:

SearchContext se = driver.findElement(By.xpath("...")).getShadowRoot();
where the return type is SearchContext. Try using xpath as the locator in the above line, and also attempt to find the element using the SearchContext reference like so:

WebElement we = se.findElement(By.cssSelector("....."));
Use cssSelector as the locator

And just like that, it works flawlessly!

I couldn't find this solution readily available and it took me quite some time to figure it out. Hopefully this helps others facing a similar issue!

Answer №5

Selenium 4 now provides built-in functionality for interacting with shadow DOM using CSS selectors. To tackle the issue at hand in Java, one could implement the following solution:

WebDriver driver = new ChromeDriver();
driver.get("chrome://settings/clearBrowserData");
TimeUnit.SECONDS.sleep(1); // Waiting for elements to load, not recommended in production.
// Begin by selecting the parent of the shadow root and calling .getShadowRoot().
driver.findElement(By.cssSelector("settings-ui")).getShadowRoot()
    .findElement(By.cssSelector("settings-main")).getShadowRoot()
    .findElement(By.cssSelector("settings-basic-page")).getShadowRoot()
    .findElement(By.cssSelector("settings-privacy-page")).getShadowRoot()
    .findElement(By.cssSelector("settings-clear-browsing-data-dialog")).getShadowRoot()
    .findElement(By.cssSelector("#clearBrowsingDataConfirm"));

A Python alternative might look something like this:

driver = webdriver.Chrome()
driver.get("chrome://settings/clearBrowserData")
time.sleep(1)  # Waiting for elements to load, not recommended in production.
# Begin by selecting the parent of the shadow root and accessing its .shadow_root property.
driver.find_element(By.CSS_SELECTOR, "settings-ui").shadow_root\
    .find_element(By.CSS_SELECTOR, "settings-main").shadow_root\
    .find_element(By.CSS_SELECTOR, "settings-basic-page").shadow_root\
    .find_element(By.CSS_SELECTOR, "settings-privacy-page").shadow_root\
    .find_element(By.CSS_SELECTOR, "settings-clear-browsing-data-dialog").shadow_root\
    .find_element(By.CSS_SELECTOR, "#clearBrowsingDataConfirm")

Note that only CSS selectors can be used within shadow root elements; xpath or id's are unsupported in this context.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Text within the div element causes the displacement of other adjacent div elements

When the text in mondaySpecial extends to a second line, it pushes tuesdaySpecial-saturdaySpecial further down on the page, disrupting the layout of the divs. The same issue occurs with other specials as well. Is there a way to prevent this from happening? ...

Get PDFs using Python's Selenium, then store them with custom filenames

Our goal is to scrape a multitude of PDFs for various companies and save them with their respective company names, all sourced from . Although my current code successfully downloads the PDFs, I am facing an issue when it comes to saving them with the comp ...

Exploring Navigation States with JQuery: Active, Hover, and Inactive

I've been struggling with the code below as it doesn't seem to be working. I had a simpler code before for swapping images, so if there's an easier way to achieve this, I'm open to suggestions. Just to recap, I'm trying to: 1. La ...

Centering text vertically in a horizontal form row with Bootstrap 4

How can a div with text be centered vertically in a Bootstrap 4 Horizontal form? For example, instead of a form input have a plain text div like "Password goes here," you can use the following code snippet: <form> <div class="form-group row"&g ...

Struggles with organizing tables in asp.net

My apologies for my lack of knowledge in this area, but I am struggling to make my table look correct on my webform. I begin by creating a table: <table> </table> And I want to add a border. The traditional way is to use the 'border&apo ...

Attempting to navigate and extract information from a webpage that is constantly being updated with new

I am attempting to gather all of the odds available for each game displayed on this particular webpage: Knowing that the website loads dynamically, my strategy was to use a scrolling function in order to load all existing odds while the page scrolls. Unfo ...

Unable to send data to Spring Controller from form submission

As a newcomer to Spring, I have been exploring tutorials online to enhance my knowledge. Recently, I developed a straightforward Spring MVC Login App utilizing AngularJS for the frontend and JSON for data posting to the Spring Controller. The login form I ...

Using Bootstrap 4 to create columns with fixed width and displaying the rest of the line in the card-footer

I have a bootstrap-4 card containing a footer. The footer consists of a fixed-width input field and a button. The button needs to expand to the width of 100% minus the width of the input field. https://jsfiddle.net/aq9Laaew/224543/ input { width:3rem; ...

Issue with non-responsive images in Bootstrap 3

Here is the bootstrap configuration that I am currently using: http://www.bootply.com/118172 I'm having an issue with the responsiveness of the image. It's not shrinking to match the height of the div on the left side. Can anyone help me trouble ...

Struggling to generate a new database

I am in the process of setting up a database, but I am facing an issue where the file explorer on the application does not display any existing databases or folders where the database should be located. I suspect this is because the database has not been c ...

Java - Adjusting the volume of a continuous Clip without interruption?

Currently, I am working with a looping Clip and adjusting the volume as shown below: clip.stop(); clip.flush(); FloatControl fc = (FloatControl) clip.getControl(FloatControl.Type.MASTER_GAIN); fc.setValue(new_volume); clip.loop(javax.sound.midi.Sequencer. ...

What causes the CSS height attribute to mess up UL lists?

I'm struggling to understand a problem in my nested list. When I set the height of LI elements, they end up overlapping each other. Can someone explain why this is happening and suggest a proper way to apply height without causing this overlap effect? ...

When an element is appended, its image height may sometimes be mistakenly reported as

I am dynamically adding divs and I need to retrieve the height and width of an image. Based on this information, I have to apply CSS to the MB-Container class. For example: if the image is portrait orientation, set container width to 100%. If it's ...

Selenium: A guide to specifying CSS for webpages with multiple elements sharing the same name

Looking for assistance with CSS selection: <table id="userPlaylistTable" cellspacing="0" cellpadding="0"> <div class="numCellWrapper"> ... When defining a css selector, how can I target a specific element among multiple elements of the same ...

Top Bootstrap dropdown menu overlaid with Leaflet zoom buttons

I'm encountering an issue depicted in the screenshot below. Is this related to bootstrap or leaflet? How can I adjust the position/style of the dropdown menu so that it appears above all other elements? https://i.sstatic.net/cQhN5.png ...

Retrieving a page using jQuery

Below is the JavaScript code that I am using: $.ajax({ url: "test.html", error: function(){ //handle error }, success: function(){ //perform actions here with the received data } }); After retrieving test.html, I am wo ...

Ways to align the icon and text to the left side of the screen?

I am looking to utilize Bootstrap's list group feature to create a navigation bar similar to the image below: https://i.sstatic.net/bndVl.png I am curious about how I can shift them to the left. Here is what I have attempted so far. .list-group-i ...

Strategies for efficiently loading 100,000 records into a table using Spring Boot on the server side and Angular on the front end

I am facing an issue with the speed of loading data from the back end to the front end without causing delays on the client side. I am using Spring Boot for the back end and Angular 7 for the front end. The problem arises when I submit a request from the f ...

What is the best way to create a 3x3 grid layout with 9 input text boxes that do not have a double border thickness?

I am having trouble creating an input square with 3 rows and 3 columns. The borders between each two input boxes are overlapping, resulting in a double thickness border. To see this issue in action, you can run the provided html. How can I fix this so that ...

Struggling to align list-items in a horizontal manner

I'm having trouble aligning the list-items in my horizontal navbar. Can anyone assist me in identifying the error? Below is the HTML code, which has a complex structure due to the use of the Wordpress thesis theme and Cufon for font replacement: < ...