Page 1 of 1

Grab URLs

PostPosted: Mon Jan 05, 2015 6:04 pm
by stu
Any plugin or magic that would allow to get a list of urls from a text. Like InternetGet "URL" "[HTML]"

And then somehow I get a list of all urls within the var?

Cheers.

Re: Grab URLs

PostPosted: Tue Jan 06, 2015 3:46 am
by dec
Hello,

Maybe the use of Regular Expression can be useful here. You can try my npRexp plugin.

Anyway it's not an easy thing, as you can see, then may you can be a little more specific... ;)

Re: Grab URLs

PostPosted: Tue Jan 06, 2015 7:11 am
by stu
I think the Techmedia dude had an URL scanner that would get all JPGs, JSs, etc from a designated URL... Of course, his out of the picture so I was wondering if there's any alternative way of getting links and such from a text... but yeah, maybe regural expressions would aid... I'll check it out. :D

Re: Grab URLs

PostPosted: Tue Jan 06, 2015 9:37 am
by David de Argentina
Hi stu,

You could addapt my NeoThinBasic sample for extract any image from a .EML file.
In fact, an .EML is almost equal an .HTML file.
Take a look of this:
viewtopic.php?f=34&t=20213

My devaluated cent,
David de Argentina

Re: Grab URLs

PostPosted: Wed Jan 07, 2015 8:36 am
by dec
Hello,

Take a look at the below sample:

http://www.neoplugins.com/Humm/Sites/Ma ... abURLs.zip

Instead of searching for URLs, I use a Regular Expression to search any "href" or "src" URL.

Re: Grab URLs

PostPosted: Wed Jan 07, 2015 9:19 am
by stu
That's very nice, only one button... Thanks for the example, I'll take a closer look at npRexp. =)

Re: Grab URLs

PostPosted: Wed Jan 07, 2015 9:28 am
by dec
Hi!

stu wrote:That's very nice, only one button... Thanks for the example, I'll take a closer look at npRexp. =)


You're welcome stu!

Re: Grab URLs

PostPosted: Thu Jan 08, 2015 5:58 pm
by stu
Hello! =)

I was trying to use the example to grab some URLs from Google search through the custom search API but im not getting postive results, any idea? I can clearly see URLs there and I try with other sites and works fine (?)

check:

Code: Select all
https://www.googleapis.com/customsearch/v1?key=AIzaSyBFD2lX-rzYdCYNabpCMnL5xuHgWJpkRHg&cx=017576662512468239146:omuauf_lfve&q=lectures


:wink:

Re: Grab URLs

PostPosted: Fri Jan 09, 2015 3:01 am
by dec
Hello,

The above sample it's based in the "href" and "src" HTML tags attributes. However, the link you shown:

Code: Select all
https://www.googleapis.com/customsearch/v1?key=AIzaSyBFD2lX-rzYdCYNabpCMnL5xuHgWJpkRHg&cx=017576662512468239146:omuauf_lfve&q=lectures


... Do not offer HTML source, but JSON, if I am not wrong. Then yes, we can view here some URLs, but not in the "href" or "src" HTML tags, because there is no HTML tags.

If you take a look at this other sample:

http://www.neoplugins.com/Humm/Sites/Ma ... bURLs2.zip

What I do is to change the Regular Expression in order to grab any URL in the source JSON "link:" property, something like:

Code: Select all
npRexpStr "[ID]" "[#34]link[#34]: [#34](.*)[#34]" "[Result]"


And now yes... you can view how we can grab such URLs from the above JSON source.

Re: Grab URLs

PostPosted: Sun Aug 30, 2015 8:05 am
by fkapnist
Put something like this in your JS Functions folder and Call to it.... It writes a list to a IE window but you can modify it to save as a file, etc...

var allLinks = document.links;
for (var i=0; i<allLinks.length; i++) {
win.document.write(allLinks[i].href+"<BR/>"); // this works


8)

.