CrawlTrack, webmaster dashboard.
Web analytic, SEO and protection
CrawlProtect, your website safety.
Two php/MySQL scripts, free and easy to install
The tools you need to manage and keep control of your site.
As a webmaster you will find on this page informations that will help you. You will be able to better manage your site, protect it and improve his ranking and then of course the number of visitors on it. I will use also that blog to give informatuion about CrawlTrack and it's development. And as there is not only internet and script in life, sometimes I will speak of different subject sutch as travel for example.
After several months of work, I'm please to release the new version of CrawlProtect. That version is a complete review in order to add new features but also to have an easy to maintain code (base on the same framework than CrawlTrack). That new version is using an MySQL database to avoid any compatibilities issues with hosting equipped with Suhosin (or equivalent script), blocking any file creation by a script. The main new features are: fully manageable htaccess, spam filtering on forum and/or blog, picture hotlinking blocage and the last but not the least, the possibility to follow all your sites from the same interface. New languages are also available (Italian and russian). The best to see all these improvment is to download it.
In April 2008, I have spent two weeks in Japan with my 15 years old daughter. We spent one week in Tokyo, then we comute to Kyoto in Shinkansen to spend the second week there.
It's a new start for CrawlTrack development, to allow a team work, Crawltrack is now on SourceForge in order to use Subversion and tracker. Philippe, known as Kissifrot, who has already done a lot of work on the lasts releases is the first developper to join me. If you would like to join us, don't hesitate to contact me.
After only one month of running, Microsoft has change BingBot user-agent.
It's a tiny change, just a ; more. The new user-agent is now:
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
CrawlTrack users have to update their CrawlerList to continue to detect BingBot, the updated list is the CrawlerList 96.
It should be the start of glory, I have been recently contacted two times to answer to interviews.
The first one has been publish on componente-calculatoare.blogspot.com in romanian, there is a translation tool at the bottom of the page if you don't speak romanian.
The second one is publish (in french) on blog.referencementschool.com.
In order to concentrate more my activities, from today CrawlProtect is now host on www.crawltrack.com
You will find here all the content of the old www.crawlprotect.com, and if you have a look on the documentation page you will be able to find new function which will appear in the next CrawlProtect release.
Bing, Microsoft searchengine, had annonced it a few months ago, the old MSNBot will be relayed by a new bot, Bingbot.
The annoncement were speaking of a start-up in October 2010, in fact Bingbot was a little bit in advance as I have detected it's first visit September the 29th around 1 am.
So, if it's not already done, don't forget to upgrade your CrawlTrack to be able to detect it (Crawltrack detect Bingbot since release 3.2.5 and crawlerlist 95).
Bingbot use the user agent:
Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm).
Today, MSNBot is still working, but we can think that it will progresssively disapear.
Google, Bing, Yahoo or Baidu are constantly enthusiasm to search the web for new pages to add to their index. You can used CrawlTrack to find out when the crawlers went on your site, but if you want more information on IP addresses and user-agents used by the crawlers this article will interest you.
CrawlTrack can detect and identified hundreds of different crawlers. The complete list is available by clicking on the link below.
The tool propose here can, with the help of Maxmind database, identified the country of origin of an IP. This combined with the database of the CrawlTrack used on that site, give the posibility to know if that IP is known as used by a crawler or a hacker. Don't hesitate to use it.
As experimented webmaster you know that duplicate content is very bad for your ranking. You are very cautious to avoid to have the same content duplicate on differents pages. But are you sure that your pages are really unique? If you didn't pay attention, you can have all your pages with two different url: www.example.com/page.htm and example.com/page.htm. For your home page it can be up to four different url !!! You will find here solution to avoid that.
The web is not a peacefull world, you have to know that soon your site will be a hacker target. On www.crawltrack.net, CrawlTrack detect between 500 and 10 000 attacks per day. You cannot ignore that, or one day, you will loose the control on your site!!!