scrapy css last-child selector unable to target text

I am currently facing a challenge in selecting and matching elements within an HTML document using CSS selectors in the Scrapy framework. My issue lies with extracting information from a specific field using the last-child selector.

Below is the snippet of HTML code:

<td class="Table-Standard-AwardName Table-Scholarship-AwardName">

<a id="ctl00_ContentPlaceHolder1_ScholarshipDataControl_grvScholarshipSearch_ctl02_hylScholarshipName" class="bold" href="/Scholarships/14123/Family-Bursary,-The">Family Bursary, The</a>   

<br>

<span>Field of Study:</span> 

EcologyEnvironmental Science

</td>

The text "EcologyEnvironmental Science" is what I am trying to match. However, when I attempt to use the last-child selector, the output only displays 'Field of Study':

In [3]: response.css('td.Table-Standard-AwardName.Table-Scholarship-AwardName > *:last-child::text').extract_first()
Out[3]: 'Field of Study:'

Despite experimenting with different methods like nth-last-child() and combining sibling selectors, I have been unable to achieve the desired result. Any assistance would be greatly appreciated!

htmlcssscrapy

Answer №1

As previously mentioned, the content within the EcologyEnvironmental Science text is located within the td element. Therefore, you can simply extract its text using the following method:

values = response.css('.Table-Standard-AwardName.Table-Scholarship-AwardName::text').extract()
out = next(filter(None, map(methodcaller('strip'), values)))
# In your item assignment, consider setting the value to 'EcologyEnvironmental Science'

Answer №2

"EcologyEnvironmental Science" is not a stand-alone element such as a span or div; rather, it is simply part of the content within that td. Therefore, it does not meet the criteria outlined in the condition '... > * ...' which specifies any direct child of the td with that class.

In order to target only that specific portion of the content using CSS, it would need to be enclosed within a span element like this:

...
  <span>Field of Study:</span> 
  <span>EcologyEnvironmental Science</span>
</td>

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

What is causing me to have difficulty importing any of the images located in the public folder?

I'm currently tackling the NextJS framework, and I'm having trouble importing images that are stored in the public folder. Despite being able to navigate through folders, the images aren't displaying as expected. Here is the import line and ...

htmlreactjsnext.jstailwind-cssreact-fullstack

Safari not rendering SVG wave at full width reached

My SVG isn't scaling to 100% width in Safari like it is in IE, Chrome, and Firefox. Despite numerous attempts, I haven't been able to make my SVG stretch across the full width like it does in other browsers. The project is built using React. . ...

htmlcsssvg

Is it possible to create an HTML select that displays transparent text after a selection

I'm currently working on customizing the appearance of an HTML select element with various background color options. Everything is functioning properly, but I have a specific requirement. I want the text in the dropdown options to be visible only when ...

jqueryhtmlcolorshtml-select

Need help with a countdown function that seems to be stuck in a loop after 12 seconds. Any

I am facing an issue with a PHP page that contains a lot of data and functions, causing it to take around 12 seconds to load whenever I navigate to that specific page. To alert the user about the loading time, I added the following code snippet. However, ...

javascripthtml

Text will not be visible within the div container

I'm working on a mobile layout design for my css project. My goal is to have the header and footer remain fixed on the screen while allowing users to scroll through the content in between. I was able to achieve the fixed positioning, but now none of m ...

htmlcss

Showing a DIV multiple times with an additional text field in HTML and JS

As someone new to development, I am facing a requirement where I need to create a form with a dynamic field that can be added or removed using buttons. When the user clicks on the Add button, another field should appear, and when they click on Remove, the ...

javascriptphphtml

Seeing the Bootstrap css load immediately upon the page loading and switching between pages

Upon loading my webpage or navigating between subpages, I've noticed that the Bootstrap.css styles are interfering with my CSS transitions. For some reason, my regular links appear orange initially, but they turn blue during loading or when transition ...

htmljquerycssbootstrap-5

How can I alter the text color on hover within a Material UI card? My goal is to have the text color change when hovering over the card itself, rather than just the text

When I apply CSS to the card hover effect, it works fine. However, I also want to change the text color along with the card color on hover. card: { maxWidth: 350, height: 300, '&:hover': { backgroundColor: '#373737 !impor ...

cssmaterial-uicard

Issue with sticky navigation bar causing page content to shift

Any ideas on how to solve my issue with a sticky nav bar in Twitter Bootstrap? Whenever I scroll past a certain point, the navbar sticks but causes my content to jump. I want the content to stay in place without jumping. Here is my HTML code: <body> ...

jquerycsstwitter-bootstrap

The feature for automatically hiding other dropdowns when one is open is not functioning properly

Trying to finalize a mega menu that almost works. I need help with closing all other open mega menus when a user clicks on one. Check out the JavaScript code snippet I've written below. The mega menu currently toggles correctly, but it doesn't h ...

javascripthtml

Webkit browsers do not properly clip content with rounded corners when using position:relative

In webkit browsers like Chrome, rounded corners do not properly cut off content when using position:relative; Check out this demonstration. Here is the HTML: <div class="outer"> <div class="inner"> </div> <div> And here ...

htmlcsswebkitrounded-corners

An issue arose while attempting to retrieve information based on the specified ID within the database

I am facing a challenge that I need help with, I have a webpage that displays questions fetched from a database and I have created a comments section for each question. How can I retrieve comments based on the post_id? I attempted to fetch them as usual bu ...

phphtmlmysqllaravel

Guide on excluding certain words within a paragraph using PHP

In my database, there is a paragraph that looks like this: $str ="this is a paragraph i show shortly and when i click on the view more it will show completely for that i am using the ajax and retrieve it " I display it as follows: this is a paragrap ...

phpjavascriptmysqlhtmlcss

Whenever I try to include something within the `componentWillUnmount` function,

I'm currently learning React on my own. I've been trying to save session data in my componentWillUnmount method. However, when I add code inside componentWillUnmount, nothing seems to happen. I tried debugging by adding console logs and debugger ...

javascripthtmlreactjsreduxmaterial-ui

Enhance your reality with Intel XDK's augmented reality feature

I am looking to develop an app using HTML and Intel XDK. It's a simple process - just place the final html pages in the www folder of the project and your app is good to go. Now, I want to integrate this: https://github.com/krisrak/html5-augmented-rea ...

htmlintel-xdk

The novice image slideshow script in JavaScript is causing all images to disappear and generating errors

Trying to create a simple image slider that pulls information from various sources. CSS and HTML are set up properly, but adding the JavaScript logic causes all images to disappear. The console displays an error saying "Uncaught TypeError: Cannot read prop ...

javascripthtmlcssslideshowweb-maker

Rotating 3D cube with CSS, adjusting height during transition

I'm attempting to create a basic 2-sided cube rotation. Following the rotation, my goal is to click on one of the sides and have its height increase. However, I've run into an issue where the transition occurs from the middle instead of from the ...

javascriptjquerycss

Is there a way to control a single Angular application using multiple JavaScript scripts?

Summary: I am looking for a way to allow two script files to handle different tasks within the same angular app. One needs to initialize the app, while the other needs to assign a templateCache object to a section of JSON stored in localStorage. Backgrou ...

javascripthtmlangularjs

"Aligning the title of a table at the center using React material

I have integrated material-table into my react project and am facing an issue with centering the table title. Here is a live example on codesandbox: https://codesandbox.io/s/silly-hermann-6mfg4?file=/src/App.js I specifically want to center the title "A ...

javascriptcssreactjsmaterial-uimaterial-table

Moving a blank block to the top using CSS transformation

After trying to implement the translate3d(150px,0px,0px); on my "content" div, I encountered an issue where an empty block appears at the top of the page. Unfortunately, I am unsure about how to resolve this issue. <!DOCTYPE html> <html> <h ...

htmlcsstransformtranslate3d