Info for webmasters from CrawlTrack

As a webmaster you will find on this page informations that will help you. You will be able to better manage your site, protect it and improve his ranking and then of course the number of visitors on it.

Identified search engines crawlers

Google, Bing, Yahoo or Baidu are constantly enthusiasm to search the web for new pages to add to their index. You can used CrawlTrack to find out when the crawlers went on your site, but if you want more information on IP addresses and user-agents used by the crawlers this article will interest you.

Crawlers known by CrawlTrack

CrawlTrack can detect and identified hundreds of different crawlers. The complete list is available by clicking on the link below.

Tool to identified the origin of an IP

The tool propose here can, with the help of Maxmind database, identified the country of origin of an IP. This combined with the database of the CrawlTrack used on that site, give the posibility to know if that IP is known as used by a crawler or a hacker. Don't hesitate to use it.

Avoid duplicate content

As experimented webmaster you know that duplicate content is very bad for your ranking. You are very cautious to avoid to have the same content duplicate on differents pages. But are you sure that your pages are really unique? If you didn't pay attention, you can have all your pages with two different url: www.example.com/page.htm and example.com/page.htm. For your home page it can be up to four different url !!! You will find here solution to avoid that.

How to protect your site from hacking attempts?

The web is not a peacefull world, you have to know that soon your site will be a hacker target. On www.crawltrack.net, CrawlTrack detect between 500 and 10 000 attacks per day. You cannot ignore that, or one day, you will loose the control on your site!!!