"Using html_attr with the attribute "href" does not return any value in the rvest package

My objective is to extract the URLs linked with specific CSS elements on a website using rvest. Despite trying various methods, such as using the html_attr function with the 'href' argument, my current script only returns NA values instead of the expected URLs.

Code snippet for setting up variables

library(rvest)

my_url <- "http://www.sherdog.com/events/UFC-Fight-Night-111-Holm-vs-Correia-58241"

my_read_url <- read_html(my_url)

my_nodes <- html_nodes(my_read_url, ".fighter_result_data a span , .right_side a span , .left_side a span")

Verify if my_nodes correspond to athletes' names

html_text(my_nodes)

Display that my_nodes are selecting the desired CSS elements

[1] "Holly Holm"          "Bethe Correia"       "Marcin Tybura"      
 [4] "Andrei Arlovski"     "Colby Covington"     "Dong Hyun Kim"      
 [7] "Rafael dos Anjos"    "Tarec Saffiedine"    "Jon Tuck"           
[10] "Takanori Gomi"       "Walt Harris"         "Cyril Asker"        
[13] "Alex Caceres"        "Rolando Dy"          "Yuta Sasaki"        
[16] "Justin Scoggins"     "Jingliang Li"        "Frank Camacho"      
[19] "Russell Doane"       "Kwan Ho Kwak"        "Naoki Inoue"        
[22] "Carls John de Tomas" "Lucie Pudilova"      "Ji Yeon Kim"

Attempt to retrieve URLs for each athlete's unique pages

html_attr(my_nodes, "href")

The output indicates that my efforts only yield a list of NA values

[1] NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA

If anyone can provide assistance in successfully obtaining the URLs instead of these NA values, I would greatly appreciate it. Thank you!

htmlcssrweb-scrapingrvest

Answer №1

Make sure you are selecting the span elements, not the a elements when using the html_nodes function. Remember that only the a elements have an href= attribute, not the span elements. You should adjust your code to:

my_nodes <- html_nodes(my_read_url, ".fighter_result_data a, .right_side a, .left_side a")
html_text(my_nodes)
html_attr(my_nodes, "href")

Answer №2

Like what @MrFlick mentioned, the hyperlinks can be found within <a> tags and you need to access them.

my_url %>%
  read_html() %>%
  html_nodes('.fighter_result_data') %>% html_nodes('a') %>% 
  html_attr('href')
[1] "/fighter/Marcin-Tybura-86928"        "/fighter/Andrei-Arlovski-270"

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Material-inspired Design Device Compatible DIV slide with JS, JQuery, and CSS

My goal is to achieve something similar to this: Desired Live Website I am looking for a feature where clicking on the div will slide in content from the right, load an external page inside it, and close when prompted. The slider div should be device c ...

javascriptjquerycssfluid-layoutmaterial-design

Hover over to disable inline styling and restore default appearance

There are some unique elements (.worker) with inline styles that are dynamically generated through Perl. I want to change the background when hovering over them and then revert back to the original Perl-generated style. The only way to override the inline ...

javascriptjqueryhtmlcssperl

How can I add text to an HTML5 SVG similar to using the HTML5 <p> tag?

I am currently working on creating dynamic rectangular boxes and I am facing some difficulties with inserting text into the shapes. The SVG text requires setting x and y coordinates in separate text tags, and doesn't have built-in width and height pro ...

javascriptcsshtmljquery-pluginssvg

How to reset an HTML5 form using the submit button

Can you prevent a form from resetting when you click the submit button? As it stands, clicking the submit button in HTML5 clears all the form fields. Is there any way to retain the previous values instead of resetting them? ...

htmlforms

Tips for creating responsive designs with specific media queries for various device sizes

I am looking to create a responsive website using media queries and have already created different media queries for mobile, tablets, and desktops. However, I am unsure if I should be writing the same CSS code multiple times for various device sizes, such ...

htmlcssmedia-queries

Update the text on the form submit button after it has been submitted

Is there a way to change the text on a submit button after it has been clicked? I have a form with a button and I want to switch the text from "click" to "Next" once the form has been submitted. <form> <div class="form-grou ...

javascriptjqueryhtmlbuttontext

Begin an unnumbered hierarchical list commencing at 1.2.1

I am trying to create a nested unordered list with specific numbering. My goal is for the list to start at "1.2.1, 1.2.2, etc." Please refer to the attached fiddle for reference. The desired outcome is shown in the following image: https://i.stack.imgur ...

htmlcss

angular trustAsHtml does not automatically insert content

Two divs are present on the page. Upon clicking button1, an iframe is loaded into div1. The same applies to button2 and div2. These iframes are loaded via ajax and trusted using $sce.trustAsHtml. This is how the HTML looks: <div ng-bind-html="video.tru ...

javascripthtmlangularjsiframe

Creating a dual-direction infinite scroll effect with CSS through mouse dragging

I'm currently working on implementing an infinite scroll component for a project. After consulting this tutorial, I've encountered an issue. It seems that I can only achieve infinite scroll in one direction. Whenever I add elements to the leftmo ...

javascriptjqueryhtmlcssscroll

What could be preventing my XPath from selecting a link/button using its label text?

<a href="javascript:void(0)" title="home"> <span class="menu_icon">Possibly more content could go here</span> Home </a> When I use //a as the XPath in the above code, it is highlighted. However, when I try //a[contains(text ...

htmlxmlseleniumxpath

CSS: the max-width property does not constrain an absolute positioned container from overflowing

I've been trying to figure this out for hours, but I can't seem to get it right. My top menu has submenus that include a popup menu. In one of my popups, I need the container to have a maximum width of 170 pixels and all the items inside to wra ...

htmlcss

Manage several scrolls using overflow:auto

There are numerous popups on a page with the overflow:auto property. The structure is as follows - <div id="popup1"> <div>SomeHTMLSTRUC</div> <div>SomeHTMLSTRUC</div> <ul class="scroll"></ul> </div> ...

htmlcss

Executing a webservice method in an html page using javascript without the need to refresh the page

Is it possible to call a webservice from an index.html page using JavaScript? My webservice is located at "localhost/ws/service.asmx" and the specific web method I want to call is called HelloWorld. The index.html page contains an HTML submit button whic ...

javascriptjqueryhtmlweb-services

Techniques for Adding Background Color to Fieldset Border

Recently starting to learn HTML and stumbled upon a question: How can I create a background color or image for a field set border? Can I simply use regular color values, or do I need special codes for creating a background color in a field set? Any insig ...

htmlcss

Auto-fit HTML Webpage Resizer Simplified

Just finished my very first jQuery project, a simple full-width slider. I've been focusing on HTML & CSS and currently working with C#. The problem is that I don't want the page to be scrollable; I want it to autofit to the webpage. Imagine ope ...

javascriptjqueryhtmlcss

Transfer information from the client to the server using AJAX and PHP by

When attempting to post a JavaScript variable called posY to a PHP file, an error occurred with the message: Notice: Undefined index: data in C:\xampp\htdocs\Heads_in_the_clouds\submitposY.php The posY variable is defined in the JavaSc ...

javascripthtmlcanvas

Retrieve the information from a website and display it on the current webpage using an ajax request

Is there a way to insert parsed HTML content into my webpage using just a link from another page? I'm attempting to use an AJAX call, but I keep receiving an error. Below is the code I've written, and the browser doesn't seem to be the issue ...

javascriptjqueryhtmlajax

What could be causing my AJAX script to not execute upon form submission?

My form's ajax functionality is not working properly. When I try to submit the form, nothing happens. Can someone please assist me with this issue? $( '#my-form' ) .submit( function( e ) { $.ajax( { url: 'u_mainslid ...

phpjqueryhtmlajaxforms

Selenium mistakenly chooses the incorrect element by selecting the first sibling element instead of searching within the element itself

I've been trying to loop through a list of elements and display the text, but I've encountered a strange issue with Selenium. When I select an element inside another element, Selenium returns the element inside the first sibling element instead o ...

pythonhtmlseleniumselenium-webdriverweb-scraping

How can a JavaScript function be used to check a tag's status?

I currently have two select tags on my webpage. I want to ensure that only one option can be selected at a time from these two tags. If the user tries to select options from both tags, an error message should be displayed instructing them to choose only on ...

javascriptjqueryhtmldom