Saturday 15 March 2008

Site Disappeared But Back Again

In my last post, I mentioned that rssorange.com, had gotten itself listed on Google, the day after it had been registered. It was, however, a fleeting listing, as a couple of days later it dissapeared back into the ether.


Four weeks have passed since then & today I noticed that one page, the C/C++ rss feeds category page, has surfaced on the first page of Google for the search term "rss orange".


This reappearance (and appearance on the other major search engines) is undoubtably due to the crawlers finding inbound links to the site from a handful or so other websites that I have placed links on. In the majority, these have been directory sites that do not require a reciprocal link. These linked from sites appear to have been crawled recently and it has sent the search engine spiders in, en masse.


To gain intel on which spiders have been crawling, I haven't been trawling through the website logfiles, instead I have implemented a php script that sends me an email everytime a crawler visits. It is preconfigured to register the visits of the major bots but can be altered / added to if you want to monitor other spiders. This script (PHP Robot Visit) is a free download from stevedawson.com. Please do read the notes at the bottom of the information page as you will receive a lot of email - I received over 200 in 2 days from googlebot, alone.


Next steps are to find more choice sites to gain inbound links from & to work on refining the SEO techniques to get search results on broader search terms like rss, rss feeds, etc.

No comments: