Creating a Web Spider with PHP
I used this for spidering my site.
function get_links($url) {
$req = new httprequest();
$html = $req->get($url);
$regex = '/mhref/m';
$preg = '/<a href="([^0-9].+?)"/'; // 1 or more
$preg = '/<a href="([^0-9].*?)"/'; // 0 or more
$urls = array ();
preg_match_all($preg, $html, $urls);
$count = 0;
foreach ($urls as $url => $links) {
if ($count == 1) {
foreach ($links as $link) {
echo "link-> " . $link . "\n";
}
}
$count++;
}
echo "\n\n";
//print_r($urls);
}
Did you enjoy this post? Why not leave a comment below and continue the conversation, or subscribe to my feed and get articles like this delivered automatically to your feed reader.
Comments
No comments yet.
Leave a comment