What is the best way to tally the frequency of words in a collection of URLs using JavaScript?

I have a JSON object in WordPress that contains a list of URLs. I would like to determine the frequency of occurrence of the second part of each URL.

Currently, the code snippet below is able to extract the remaining part of the URL after the prefix https://www.example.co. My next objective is to calculate the occurrence of the second part of the URL, which includes cat1, cat3, cat2, xmlrpc.php.

var urlList = [
  {
    "URL": "https://www.example.co/cat1/aa/bb/cc",
    "Last crawled": "Jun 23, 2019"
  },
  {
    "URL": "https://www.example.co/cat2/aa",
    "Last crawled": "Jun 23, 2019"
  },
  // ...Additional URL objects
  {
    "URL": "https://www.example.co/xmlrpc.php",
    "Last crawled": "Jun 19, 2019"
  }
]

const paths = urlList.map(value => value.URL.replace('https://www.example.co', ''));

//console.log(paths);

paths.forEach(function(item) {
    var urlSecondPart = item.split("/")[1];
    console.log(urlSecondPart);
});

Is there a way to achieve this calculation within my current forEach loop?

Any assistance on this matter would be highly appreciated. Thank you.

Answer №1

Utilize a regular expression to identify non-/ characters that appear after .co/:

var urlList = [
  {
    "URL": "https://www.example.co/cat1/aa/bb/cc",
    "Last crawled": "Jun 23, 2019"
  },
  {
    "URL": "https://www.example.co/cat2/aa",
    "Last crawled": "Jun 23, 2019"
  },
  {
    "URL": "https://www.example.co/cat1/aa/bb/cc/dd/ee",
    "Last crawled": "Jun 23, 2019"
  },
  {
    "URL": "https://www.example.co/cat3/aa/bb/cc/",
    "Last crawled": "Jun 23, 2019"
  },
  {
    "URL": "https://www.example.co/cat2/aa/bb",
    "Last crawled": "Jun 23, 2019"
  },
  {
    "URL": "https://www.example.co/cat1/aa/bb",
    "Last crawled": "Jun 23, 2019"
  },
  {
    "URL": "https://www.example.co/xmlrpc.php",
    "Last crawled": "Jun 19, 2019"
  }
]

const paths = urlList.map(
  ({ URL }) => URL.match(/\.co\/([^\/]+)/)[1]
);
console.log(paths);

const counts = paths.reduce((a, str) => {
  a[str] = (a[str] || 0) + 1;
  return a;
}, {});
console.log(counts);

On newer engines, you can employ lookbehind instead of extracting the capture group:

const paths = urlList.map(
  ({ URL }) => URL.match(/(?<=\.co\/)[^\/]+/)[0]
);

If you wish to monitor all full URLs utilized, reduce not only into a count, but also into an array of those full URLs:

var urlList = [
  {
    "URL": "https://www.example.co/cat1/aa/bb/cc",
    "Last crawled": "Jun 23, 2019"
  },
  {
    "URL": "https://www.example.co/cat2/aa",
    "Last crawled": "Jun 23, 2019"
  },
  {
    "URL": "https://www.example.co/cat1/aa/bb/cc/dd/ee",
    "Last crawled": "Jun 23, 2019"
  },
  {
    "URL": "https://www.example.co/cat3/aa/bb/cc/",
    "Last crawled": "Jun 23, 2019"
  },
  {
    "URL": "https://www.example.co/cat2/aa/bb",
    "Last crawled": "Jun 23, 2019"
  },
  {
    "URL": "https://www.example.co/cat1/aa/bb",
    "Last crawled": "Jun 23, 2019"
  },
  {
    "URL": "https://www.example.co/xmlrpc.php",
    "Last crawled": "Jun 19, 2019"
  }
]

const getSecond = url => url.match(/\.co\/([^\/]+)/)[1];

const counts = urlList.reduce((a, { URL }) => {
  const second = getSecond(URL);
  if (!a[second]) {
    a[second] = { count: 0, fullUrls: [] };
  }
  a[second].count++;
  a[second].fullUrls.push(URL);
  return a;
}, {});
console.log(counts);

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Logo-enhanced Navigation Bar in Bootstrap

Hey everyone, I've been experimenting with Bootstrap headers that include a logo before the "Brand" text. I'm facing an issue where the navbar doesn't resize properly, as shown in the screenshot below: - The navbar doesn't adjust to t ...

Simple steps to successfully pass two parameters to jsonpCallback

Hey there! Below is the code snippet where I am using an ajax call in jQuery to invoke function 1 named "setEmailAFriendCount". In this function, we are passing a variable with JSON data type. However, I now need to call the same function from an ajax call ...

Interested in incorporating href within a PHP variable?

My goal is to send a verification email without displaying the link in the email itself. Instead, I want to include text that says "click here to verify." { $verify_email=" Hello, Thank you for signing up. To verify your e-mail, please visit http://".$sit ...

Extracting information from dynamically generated tables using Python 2.7, Beautiful Soup, and Selenium

I am in need of assistance with scraping a JavaScript generated table and saving specific data to a csv file. The tools available to me are limited to python 2.7, Beautiful Soup, and/or Selenium. Although I have referred to the code provided in question 14 ...

The click functionality is not functioning properly within the .each() loop

For my event handler, I am trying to use click() but it doesn't seem to be working. Here is the code snippet: $('.ajax-close').click(function( event ){ event.preventDefault(); alert('hi'); $( ' ...

What methods can be used to modify the behavior of tiptap when pasting plain text?

Currently, my goal is to create a visual editor by utilizing the tiptap library. Although v2 of tiptap is more commonly used, there are instances where v1 is necessary. However, I encountered an issue with tiptap's behavior when pasting plain text, ...

Executing a task within a Grunt operation

I have integrated Grunt (a task-based command line build tool for JavaScript projects) into my project. One of the tasks I've created is a custom tag, and I am curious if it is feasible to execute a command within this tag. Specifically, I am working ...

Using HTML5 to display the {{data}} extracted from a JSON file right in the center of the text

Can someone help me with a question about working with html and angular5? <span [innerHTML]="'client.acceptance.explanation'| translate"></span> <span><b>{{data}}</b></span> I have text stored in a json file ...

I am interested in displaying a tooltip when a button is hovered over

In my application, there is a button with the ID #download_setup. Currently, an AJAX call is triggered upon clicking this button. However, there are certain users for whom I need to prevent this AJAX download and instead display a tooltip saying "you don&a ...

Is there a way for me to store the output of an AJAX call?

One of the challenges I'm facing involves an AJAX request: $.ajax({ url: 'script.php?val1=' + value1 + '&val2=' + value2, dataType: "json", success: function (data) { var newValue1 = data[0]; ...

The filter() and some() functions are not producing the anticipated output

Currently, I am in the process of developing a filtering mechanism to sift through a dataset obtained from an API. The array that requires filtering contains objects with various parameters, but my aim is to filter based only on specific parameters. For ...

Adjusting Text Size Depending on Width

I recently used an online converter to transform a PDF into HTML. Check out the result here: http://www.example.com/pdf-to-html-converted-file The conversion did a decent job, but I'm wondering if it's feasible to have the content scale to 100% ...

Come up with various transition animations for multiple child elements that activate when the cursor hovers over the parent element

I have a CSS & HTML code snippet that looks like this: .item-video-kami { width: 480px; overflow: hidden; position: relative; } .item-video-kami>.play-icon.full-height>svg { width: 106px; color: #ffffff50; } .item-video-kami img { ...

Fix background transition and add background dim effect on hover - check out the fiddle!

I'm facing a challenging situation here. I have a container with a background image, and inside it, there are 3 small circles. My goal is to make the background image zoom in when I hover over it, and dim the background image when I hover over any of ...

Implementing a secure route in React that asynchronously checks user authentication status

Currently, I have a React app utilizing Auth from the aws-amplify package (version 5.3.8) and react-router-dom (version 5.3.0). Although there is a version specifically for React available, it necessitates using at least version 5.0.0 of react-scripts, whe ...

What is the most effective way to extract data that includes an array within it?

const flightList = [{ number: 343, from: "Singapore", to: "India", upgradeTypes: ["Economy to Premium Economy", "Economy to Business Class"] }, . { number: 363, from: "Chennai", to: "Sing ...

Error: Trying to add an element to an undefined variable (d3 force layout)

I've been racking my brain trying to solve this error I keep encountering while working with a force layout. The problem arose when I switched to reading nodes from a JSON file. When I comment out certain lines, the error disappears. But if I leave th ...

When browserify is utilized, the error "function is not defined" may show up as an Un

Exploring an example at and attempting a function call as shown below: My HTML code is: <!DOCTYPE html> <html> <head> <title>Testing Browserify</title> <script src="bundle.js"></script> </head> <body& ...

Tips for overlaying text on the background in Next.js:

I'm currently working on creating an image element with overlay text. Check out my jsx code below: <div className={styles.img}> <img src={src} alt="" /> <p>{`(${size})`}</p> </div> And here is t ...

"Utilizing GroupBy and Sum functions for data aggregation in Prisma

I am currently working with a Prisma schema designed for a MongoDB database model orders { id String @id @default(auto()) @map("_id") @db.ObjectId totalAmount Int createdAt DateTime @db.Date } My ...