UrFix's Blog

What is Referrer Spam?

According to Wikipedia: Referrer spam (also known as log spam or referrer bombing) is a kind of spamdexing (spamming aimed at search engines). The technique involves making repeated web site requests using a fake referrer url that points to the site the spammer wishes to advertise. Sites that publicize their access logs, including referrer statistics, will then end up linking to the spammer’s site, which will in turn be indexed by the search engines as they crawl the access logs.

This benefits the spammer because of the free link, and also gives the spammer’s site improved search engine ranking due to link-counting algorithms that search engines use.

What Does a Referrer Spam script look like?

Here is a example of a referrer spam script.

count=0
for i in `cat list`
do
count=`expr $count + 1`
curl $i -e “https://blog.urfix.com” -A “Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.6) Gecko/20100107 Fedora/3.5.6-1.fc12 Firefox/3.5.6”
echo $i
echo counting
done
echo “End of file”
echo done

What this script does is it reads urls from a file called list (for i in `cat list`)

It then sends a request to the first url spooifng the referral making it seem that it is coming from https://blog.urfix.com (-e “https://blog.urfix.com”)

It also spoofs the Agent and OS to make it seem like a real user and not some curl bot (-A “Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.6) Gecko/20100107 Fedora/3.5.6-1.fc12 Firefox/3.5.6”)

the script continues by looping and doing the same with second url the third and so forth until it reaches the bottom of the list.

Solutions

Referrer Karma WordPress Plugin
Referrer Karma is a rather simple script that prevents malicious bots from accessing your pages, flooding your logs and possibly draining your server’s bandwidth. All it does is check that an incoming bot has a valid referrer field URL (i.e. that the page it claims to come from, does exist and does have a link to your site). If RK thinks the incoming visitor is a malicious bot, it displays a 403 error page (which will not be counted as a visit by log analyzer tools) and uses HTML redirecting to the original URL to avoid blocking legitimate visitors (See site for details).

stop-spam-referer-php-script

• Really easy install – two minuts
• Stop bad referers ( domains list )
• Stop domains by keywords
• Bandwidth safe
• and much more

With this simple php script you can stop all your referer spam easily :)

.htaccess

Use an .htaccess file to block requests from whatever you have managed to identify as either a crawler designed to find URLs to spam or a spamming URL. This is a relatively simple blacklist, and though it cannot work as a long term solution to this problem, it should keep you happy for now.

Conclusion

There are plenty of techniques available for blocking referrer spam, and everyone has their favorite. Personally, I use a combination the three techniques mentioned in this article.

Good luck with getting rid of your spam referrals.