Red-colored text (as displayed in Firefox) is not recognized by JSoup when parsing HTML <img> tags

UPDATE: I was able to answer my own question. JSoup can indeed find all image tags.

I've been attempting to scrape content from but encountered a problem.

In the website's source code, the main images are displayed in red font, which seems to be causing issues with my JSoup select method and getElementsByTag method. It would be helpful if you could check the source code yourself due to formatting differences, but I'll provide the essential information here.

UPDATE: After inspecting the code on Chrome and IE, I noticed that the image tags are not displayed in red, indicating it might be an issue specific to Firefox. However, JSoup still fails to detect these images. (Further updates at the end of the post)

UPDATE 3: Instead of pasting my code, I have shared a screenshot here: . If you observe, the red blocks represent the user-uploaded images (the ones I need), while other img tags appear in different colors (mostly logos). When I execute the following code:

Elements imageElements = doc.select("img");

and print the results, I only get non-red image tags.

Since I don't have much experience with HTML or CSS, is there something crucial I'm missing? Could this be an issue with my code or some specific quirk of the webpage design? Is there a way to retrieve those "red font" images as well?

UPDATE 2: Upon further investigation, I found that the red HTML font in Firefox displays an error message stating: No space between attributes.

This has left me a bit perplexed considering Flickr is a renowned platform where everything seems to be functioning correctly. Could this behavior be intentional to prevent scraping? Is there still a way for me to access and download those images?

htmlcssfirefoxjsoup

Answer №1

Responding to my own inquiry.

I admit that I was wrong; JSoup actually does locate all the img tags. Although, I can't pinpoint exactly where I went wrong since I observed it working yesterday and have made changes to my code since then. I suspect my mistake had something to do with how I used .select, which might have caused those images to be excluded (the code in this post was simplified for clarity).

I'll keep this question posted as it may assist others grappling with malformed HTML in their source code, as there are several useful pointers in the comments section.

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Tips for deactivating the double class with jQuery

I need to implement a feature where all classes on a button are disabled if a specific class is present. I attempted to disable all classes, but it seems like I made an error somewhere. Jquery: if ($('.lgi_btn_cta_toggle').hasClass('lgi_ct ...

jquerycss

Passing a Javascript variable to the NAME attribute of an HTML <a href> tag: Steps to do it efficiently

I need assistance with passing a JavaScript variable to the NAME attribute of an HTML tag. Let's consider this script example: <script> var name = "Click here!"; </script> My goal is to pass the variable to some code in order for <a ...

javascriptjqueryhtml

Display the initial three image components on the HTML webpage, then simply click on the "load more" button to reveal the subsequent two elements

I've created a div with the id #myList, which contains 8 sub-divs each with an image. My goal is to initially load the first 3 images and then have the ability to load more when clicking on load more. I attempted to follow this jsfiddle example Bel ...

javascriptjqueryhtmlcss

What could be causing Vuejs to not update elements promptly?

Currently, I am encountering a scenario where I am adding options to a select element using Vue.js when the @change event of that specific element is triggered. An issue arises where the new option is not 'registered' until I exit the function. ...

javascripthtmlvue.js

Rearrange the middle column to be the top column on smaller screens

I am trying to create a layout with 3 columns in a row. The challenge is to prioritize the middle column on top when viewed on small screens, with the left and right columns below it. To clarify, on large screens the order should be 1 2 3, while on small s ...

csstwitter-bootstrap

Guide on inserting tooltip to designated header column in primeNG data table

Html <p-table #dt1 [columns]="cols" [value]="cars1"> <ng-template pTemplate="header" let-columns> <tr> <th *ngFor="let col of columns"> {{col.header}} </th> ...

htmlangulartypescriptprimengprimeng-datatable

Error loading custom Javascript in MVC 4 view during the first page load

I'm working on an MVC 4 application that utilizes jQuery Mobile. I have my own .JS file where all the functionality is stored. However, when I navigate to a specific view and check the page source, I notice that all scripts files are loaded except fo ...

javascriptjqueryhtmlasp.net-mvcjquery-mobile

I am in the process of creating several checkboxes and am looking to incorporate some added functionality

Currently, I am working on a project that involves creating multiple checkboxes. My goal is to implement a specific functionality where only one checkbox can be checked in each group with the correct or incorrect value. Once all groups have been selected, ...

javascriptjquerycssarrayshtml

The component triggering the redirect prematurely, interrupting the completion of useEffect

I set up a useEffect to fetch data from an endpoint, and based on the response, I want to decide whether to display my component or redirect to another page. The problem I'm facing is that the code continues to run before my useEffect completes, lead ...

javascripthtmlreactjs

Is it possible to generate an HTML element by utilizing an array of coordinates?

I have a set of 4 x/y coordinates that looks like this: [{x: 10, y: 5}, {x:10, y:15}, {x:20, y:10}, {x:20, y:20}] Is there a way to create an HTML element where each corner matches one of the coordinates in the array? I am aware that this can be done usi ...

javascripthtmlcss

What is the best way to vertically align a container beneath a navbar without triggering the appearance of a scrollbar?

I'm in the process of developing an application using Bootstrap 4. The app features a navigation bar and a container located beneath it. Below you'll find the structure of the application: <body> <div id="root"> ...

htmlcssbootstrap-4

.scss compiling with errors

Recently, I embarked on a new Vue(3) project. Within this project, I have set up some basic styling in the App.scss file and created a HomeView.vue with a corresponding HomeView.scss file (located in the /src/views/Home directory). The styling from both fi ...

cssvue.jssass

Using setInterval together with jQuery to animate CSS based on changes in window size

I've been troubleshooting this issue and trying various methods, but despite it seeming logical, nothing is working. Any assistance would be greatly appreciated since I'm not very skilled in coding. My current challenge involves animating an obj ...

jquerycssanimationsetintervalwindow-resize

Guidance on incorporating static files with Spring MVC and Thymeleaf

I'm seeking guidance on how to properly incorporate static files such as CSS and images in my Spring MVC application using Thymeleaf. Despite researching extensively on this topic, I have not found a solution that works for me. Based on the recommenda ...

javacssspringspring-mvcthymeleaf

AngularJS tips for resolving an issue when trying to add duplicates of a string to an array

Currently dealing with a bug that occurs when attempting to push the same string into an array that has already been added. The app becomes stuck and prevents the addition of another string. How can I prevent the repeat from causing the app to get stuck w ...

javascripthtmlangularjsarrays

Offspring of the superior element resting above another element

I'm dealing with a unique situation involving the Z-INDEX property. Check out my HTML setup below. <div class="player"> <div class="player-line"> <div class="player-handle"></div> <!-- /.player-handle --> </d ...

htmlcssz-index

Customizing the DatePicker with a unique button in material-ui

For my current project, I am utilizing a Datepicker component. I am looking to incorporate a custom information button in the upper right corner of the calendar layout, similar to the example image provided below: https://i.stack.imgur.com/fHMbn.png Unfo ...

javascripthtmlcssreactjsmaterial-ui

Assist me in temporarily altering the color of menu items in a sequential manner upon the page's loading

Currently, I'm working on a website that features a subtle dark grey menu at the top of every page. The menu is built using HTML and CSS with a list structure. To highlight the corresponding menu item based on the current page, I am utilizing the ID a ...

htmlcssmenuonload

The element is unclickable due to being obstructed by another element

My task is to click on a button using this code: browser.find_element_by_id('btnSearch') However, the button is being blocked by a div tag with the following attributes: <div id="actionSearch" class="row pull-right"> How can I work around ...

pythonhtmlcssseleniumselenium-chromedriver

The method of utilizing React with Redux to display component properties

I am currently trying to include my common component in my main.js file Successfully implemented this However, when attempting to print my Redux data values in the common component, I created a method called handleClickForRedux to handle this task. Even af ...

javascripthtmlcssreactjsredux