Simple web scraper that reads all the links to JSON files in JS
Midhun
Posted on January 17, 2022
I had to get a list of all links on a webpage for a task I was working on. here I am sharing the snippet of code that I used. Let's discuss how to improve it
var tag = document.querySelectorAll("a");
var myarray = []
for (var i = 0; i < tag.length; i++) {
var nametext = tag[i].textContent;
var cleantext = nametext.replace(/\s+/g, ' ').trim();
var cleanlink = tag[i].href;
myarray.push([cleantext, cleanlink]);
};
function generateJson() {
var hrefArray = [];
for (var i = 0; i < myarray.length; i++) {
let t = {}
t.n = myarray[i][0]; t.m = myarray[i][1];
hrefArray.push(t);
};
var win = window.open("Json");
win.document.write(JSON.stringify(hrefArray));
}
generateJson()
Steps
- You will need to open the website in your browser to get all links
- Go to the console tab in Inspect element
- Please paste the above code and press enter. A json file will open in a new window
Screenshots
- How to Run
- Result
Please let me know your thoughts after reading
π πͺ π
π©
Midhun
Posted on January 17, 2022
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
javascript Leveraging JavaScript's Set and Map for an Efficient Content Management System
September 2, 2024
javascript How to add a Who-Is-Online feature on your application with JavaScript
September 19, 2024
javascript Understanding the Fetch API: The Future of Network Requests in Web Development
August 20, 2024