scrapy css last-child selector unable to target text

I am currently facing a challenge in selecting and matching elements within an HTML document using CSS selectors in the Scrapy framework. My issue lies with extracting information from a specific field using the last-child selector.

Below is the snippet of HTML code:

<td class="Table-Standard-AwardName Table-Scholarship-AwardName">

<a id="ctl00_ContentPlaceHolder1_ScholarshipDataControl_grvScholarshipSearch_ctl02_hylScholarshipName" class="bold" href="/Scholarships/14123/Family-Bursary,-The">Family Bursary, The</a>   

<br>

<span>Field of Study:</span> 

EcologyEnvironmental Science

</td>

The text "EcologyEnvironmental Science" is what I am trying to match. However, when I attempt to use the last-child selector, the output only displays 'Field of Study':

In [3]: response.css('td.Table-Standard-AwardName.Table-Scholarship-AwardName > *:last-child::text').extract_first()
Out[3]: 'Field of Study:'

Despite experimenting with different methods like nth-last-child() and combining sibling selectors, I have been unable to achieve the desired result. Any assistance would be greatly appreciated!

htmlcssscrapy

Answer №1

As previously mentioned, the content within the EcologyEnvironmental Science text is located within the td element. Therefore, you can simply extract its text using the following method:

values = response.css('.Table-Standard-AwardName.Table-Scholarship-AwardName::text').extract()
out = next(filter(None, map(methodcaller('strip'), values)))
# In your item assignment, consider setting the value to 'EcologyEnvironmental Science'

Answer №2

"EcologyEnvironmental Science" is not a stand-alone element such as a span or div; rather, it is simply part of the content within that td. Therefore, it does not meet the criteria outlined in the condition '... > * ...' which specifies any direct child of the td with that class.

In order to target only that specific portion of the content using CSS, it would need to be enclosed within a span element like this:

...
  <span>Field of Study:</span> 
  <span>EcologyEnvironmental Science</span>
</td>

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Troubleshooting JavaScript for Sidebar Disappearance due to marginRight and display CSS Issue

I need to adjust the layout of my website so that the sidebar disappears when the window is resized below a certain width, and then reappears when the window is expanded. Here is the HTML code: <style type="text/css"> #sidebar{ width:290 ...

javascriptjquerycsshidesidebar

Automatically retrieve a URL from a website using Python

My approach to streamline my program and enhance user experience was by creating an executable file that consolidates all functions, including the download of various applications and their installation. However, I encountered a challenge where the link is ...

htmlpython-3.xbeautifulsouppython-requests

The inconsistency of Selenium's StaleElementReferenceException error and the variability of pageload completion codes is causing issues with clicking on elements

Big shoutout to the amazing stackoverflow community for always providing assistance. Lately, I've been grappling with the frustrating "StaleElementReferenceException" issue and haven't found a universal solution yet. Some helpful members have rec ...

javascriptjavajqueryhtmlselenium

Combining various types of media stylesheets with a print stylesheet

I'm struggling with a school assignment that is asking me to compare two different ways of linking a print stylesheet with another stylesheet in HTML. The assignment requires me to explain when each technique would be appropriate: <link rel="style ...

htmlcsswebprintingstylesheet

Is Safari causing issues with the CSS slider animation?

Currently, I am working on implementing a CSS-only slider (no jQuery) in WordPress using the code from this source: http://codepen.io/dudleystorey/pen/kFoGw An issue I have encountered is that the slider transitions are not functioning correctly in Safari ...

htmlcsswordpress

Ways to Retrieve HTML Snippet from Handler Request and Load it into a String

I'm encountering an issue with my ASP.NET MVC application. I'm attempting to retrieve the following HTML snippet from a Handler in a separate ASP.NET project, within one of my controllers, in order to display it in a View: <br /> <div i ...

htmlasp.net-mvccode-snippets

Creating an Interactive Menu System with Nested Options in Angular 2

I have a JSON structure that I need to transform into a menu with sub-menus based on the value of the sub-menu_location field. Here's an example: { "data": [ { "menu_name": "Primary Operations", "enabled": true, "sub-menu_lo ...

javascripthtmlcssangular

Attempting to extract a parameter from a URL and pass it as an argument to a function for the purpose of locating objects based on

I am trying to retrieve data from a URL http://localhost:3000/share/user=sampleuser to display objects with an author value that matches the one in the URL. However, I encountered an error when attempting to call a function that extracts the value from t ...

javascripthtmlnode.jsmustache

The `tailwind.min.css` file takes precedence over `tailwind.css` in my Nuxt

Having trouble configuring Tailwind to use a custom font without it overriding the tailwind.css file and not displaying the changes? https://i.stack.imgur.com/ExDCL.png Check out my files below. // tailwind.config.js const defaultTheme = require('ta ...

javascriptcssvue.jsnuxt.jstailwind-css

Can you include an optional selector in SASS?

Here is an interesting concept: adding a CSS-selector optionally to the existing selector stack within a mixin. This is what I envision: @mixin myMixin(mobileSelector:true) { @if(mobileSelector) { .foo .mobile:before, } .classXY .subclass:before ...

csssassdart-sass

Navigate within a single component on the Stack by scrolling

I'm a huge fan of CSS! Although it may seem like a simple task, I am completely stumped on how to tackle it. Mission: My goal is to enable scrolling only within the List section without affecting the entire page. Here's the structure of my ...

cssreactjsmaterial-ui

Connecting a string array to the navigation bar

Need Assistance with AngularJS: In my controller, I have a string array that looks like this: var app = angular.module('myApp', []); app.controller('mycontroller', function($scope) { $scope.menuitems =['Home','About&apos ...

javascriptcssangularjshtmlasp.net-web-api

The measurement of a HTML window's entire content height (not just the visible viewport height)

Currently, I am attempting to determine the total height of a webpage's content, not just what is visible. In my efforts, I have managed to achieve some success in FireFox using: document.getElementsByTagName('html')[0].offsetHeight. Howeve ...

javascripthtmlcss

Unable to set the width for a div element within the bootstrap grid layout

When trying to set the width of a div tag to 100% within a bootstrap grid system (col-lg-3), I noticed that the div ends up taking the full width of the browser instead of just 100% of its parent(col-lg-3). .sidebar { color: #fff; f ...

htmlcss

Ways to organize interconnected form elements on a PHP webpage

On my PHP page, I have a form designed to gather user information for job recruitment. This form consists of multiple input fields and can be considered as a candidate's resume. One section of the form requires users to enter their skill set in the f ...

phphtmlforms

Filtering data in an antd table by searching

Just starting out with React hooks, specifically using TypeScript, and I'm struggling to implement a search filter with two parameters. Currently, the search filter is only working with one parameter which is 'receiver?.name?'. However, I wo ...

javascripthtmlreactjstypescripthtml-table

Adjust the height of each child element to be half of the fixed height of the parent

There is a wrapper containing multiple boxes generated using ng-repeat from a list. The wrapper has a fixed height of 300px. If the list contains only one box, a class is added to fill the entire space of the wrapper. However, when there are more than on ...

htmlcssangularjsflexbox

Box-free calendar dash component

Encountering an issue within my application dashboard, here's a direct screenshot: https://i.stack.imgur.com/sEMrM.png The problem is that the Calendar days are extending beyond the borders of the calendar box. To implement this, I simply utilized ...

pythonhtmlcssplotly-dash

Using JQuery to enable the movement of divs on a screen through clicking and dragging, without the use of

Recently, I've been experimenting with a small project involving draggable divs. However, the code I've written doesn't seem to be functioning properly, causing JQuery to become unresponsive. Is there an alternative method that is simple an ...

javascriptjqueryhtmlcss

Having trouble tackling whitespace removal in the DIV? Usual methods seem ineffective?

I am currently working on a vertical-align "3 block" design and trying to eliminate the white space between the divs. I have experimented with setting the font-size of the body to 0px and then assigning individual font sizes to each child, but this approac ...

htmlcss