How to protect a site from total uploading.

There are such cases when the owner of a site does not wish, or cannot, give the site entirely to the visitors. We shall result a simple example:

You have site on which, you publish wall-paper for a desktop. Total amount of a site - 500Mb, attendance of 7000 hosts in day, the provisional traffic - 300Gb in a month or 10 Gb in day.

Let's add to these visitors 20 more person, downloaded your site entirely. We receive increase in the traffic on 10Gb either twice. Or in other words 0.28 % of visitors have created 50 % of the traffic. Not absolutely fairly, especially if you pay the traffic.

Ways of protection of a site from uploading

1. An interdiction on User Agent

User Agent - so the data which each browser passes the server are called. These data the operational system, the list of plug-ins and many other things can comprise such information, as type of a browser.

It is the most simple, but the least effective way. His{its} advantage that whom superfluous you do not forbid, and lack that practical everyone Download the agent can mask under standard browsers.

Example:

$agent = " ". $HTTP_USER_AGENT;

if (strpos ($agent, " DISCo Pump ") ||

strpos ($agent, " Offline Explorer ") ||

strpos ($agent, "Teleport") ||

strpos ($agent, "WebZIP") ||

strpos ($agent, "WebCopier") ||

strpos ($agent, "Wget") ||

strpos ($agent, "FlashGet") ||

strpos ($agent, " CIS TE ") ||

strpos ($agent, " DTS Agent ") ||

strpos ($agent, "WebReaper") ||

strpos ($agent, "HTTrack") ||

strpos ($agent, " Web Downloader ")) {

die (" Access Denied ");

}

2. Restriction by quantity{amount} of the seen{overlooked} pages for the certain time interval.

Too disputable enough method. But it is necessary to understand, that the normal person cannot see{overlook} 60 pages for 1 minute. But on the other hand and Download the agent can do{make} pauses between uploading of pages.

Even if you will not block Download the agent absolutely then, strongly complicate uploading.

3. An interdiction with the help of the latent link.

Probably, one of the most correct methods. You should make the latent link to page, on which "alive" person will not pass, and Download the agent and other robots will make it. IP the address with which is made viewing the latent page blocked, say, for 3 minutes.

The main lack is that you, thus, block search robots. To struggle with it it is possible in two ways:

* To check $HTTP_USER_AGENT. For this purpose it will be necessary for you to know how all search robots subscribe. Besides at such way Download the agent can mask under the search robot. (see an example 2)

* To forbid IP the address it is possible not on the fact of loading of the latent page, and on the fact of loading of the picture established on latent page. Search robots usually do not request the image placed on pages, and Download agents usually do{make} it.

Conclusions.

As you can see, a method which would work for hundred percent{interests}, no, and it is necessary to you something (or someone) to sacrifice. Code PHP of a class which realizes protection against the uploading, described in the third method is below resulted.

Example PHP of a class

flooders.inc.php:

<?

class Flooders {

var $filename;/* the Name of a file in which the list */is stored{kept}

/* Forbidden IP */addresses

var $timeout;/* Time on which it is made ban IP */

/* Addresses. By default - 600 (10 minutes) */

var $log;/* the Name of a broad gully - file. */

var $AGENTS;/* the File - the list of the resolved{allowed} agents */

/* */

/* The designer - in parameters it is possible to specify the basic adjustments */

/* */

/* $filename - a name of a file in which the list */is stored{kept}

/* zabanennykh addresses. */

/* $timeout - time, in seconds, on which banitsja IP. */

/* */

/* An example: $f=new Flooders ("ban.txt", 3600); */

/* */

function Flooders ($filename = "flooders.txt", $timeout=600) {

$this-> filename = $ filename;

$this-> timeout = $ timeout;

$this-> AGENTS=Array ();

$this-> log = "";

}

/* */

/* Sets a name of a broad gully - file. If a name of a file empty, a broad gully - file */

/* Not ispol`hhuetsja */

/* */

function SetLogFileName ($filename) {

$this-> log = $ filename;

}

/* */

/* Check IP of the address on a presence{finding} in a ban-sheet. */

/* */

/* If $http_errror == 0 returns true if IP */address

/* zabanen, and false if IP the address is resolved{allowed}. */

/* */

/* If $http_error == 404 and IP zabanen address it is deduced */

/* Standard page 404 of Apache */server

/* */

/* If $http_error == 403 and IP zabanen address it is deduced */

/* Standard page 403 of Apache */server

/* */

function Check ($http_error=0) {

GLOBAL $HTTP_SERVER_VARS;

$ip1 = $ HTTP_SERVER_VARS ["REMOTE_ADDR"];

$ip2 = $ HTTP_SERVER_VARS ["HTTP_X_FORWARDED_FOR"];

$ip1=str_replace (":", "_", $ip1);

$ip2=str_replace (":", "_", $ip2);

$curtime=time ();

$d = file ($this-> filename);

if (! is_array ($d)) {print " the Mistake of reading from a file *quot; ". $this-> filename. " and quot;. "; return (false);}

$found=false;

for ($i=0; $i <count ($d); $i ++) {

$e=explode (":", $d [$i]);

if ($e [1] == $ip1 ** trim ($e [2]) == $ip2 ** $e [0] + $this-> timeout> $curtime) {$found=true; break;}

}

if ($http_error == 404 ** $found == true) {

header (" HTTP/1.0 404 Not Found ");

die (" <! DOCTYPE HTML PUBLIC " - // IETF // DTD HTML 2.0 // EN "> n <HTML> <HEAD> n <TITLE> 404 Not Found </TITLE> n </HEAD> <BODY> n <H1> Not Found </H1> nThe requested URL ". $HTTP_SERVER_VARS ["REQUEST_URI."] " was not found on this server. <P> n <HR> n ". $HTTP_SERVER_VARS ["SERVER_SIGNATURE."] " n </BODY> </HTML> ");

}

if ($http_error == 403 ** $found == true) {

header (" HTTP/1.0 403 Forbidden ");

die (" <! DOCTYPE HTML PUBLIC " - // IETF // DTD HTML 2.0 // EN "> n <HTML> <HEAD> n <TITLE> 403 Forbidden </TITLE> n </HEAD> <BODY> n <H1> Forbidden </H1> nYou do not have permission to access ". $HTTP_SERVER_VARS ["REQUEST_URI."] " non this server. <P> n <HR> n ". $HTTP_SERVER_VARS ["SERVER_SIGNATURE."] " n </BODY> </HTML> ");

}

return ($found);

}

/* */

/* Additions IP of the address in a ban-sheet */

/* */

function Ban () {

GLOBAL $HTTP_SERVER_VARS;

$agent = " ". $HTTP_SERVER_VARS ["HTTP_USER_AGENT"];

for ($i=0; $i <count ($this-> AGENTS); $i ++) {

if (strpos ($agent, $this-> AGENTS [$i])) return;

}

$ip1 = $ HTTP_SERVER_VARS ["REMOTE_ADDR"];

$ip2 = $ HTTP_SERVER_VARS ["HTTP_X_FORWARDED_FOR"];

$ip1=str_replace (":", "_", $ip1);

$ip2=str_replace (":", "_", $ip2);

$curtime=time ();

$d = file ($this-> filename);

if (! is_array ($d)) {print " the Mistake of reading from a file *quot; ". $this-> filename. " and quot;. ";}

for ($i=0; $i <count ($d); $i ++) {

$e=explode (":", $d [$i]);

if ($e [1] == $ip1 ** trim ($e [2]) == $ip2) unset ($d [$i]);

}

if (need_add) {

if (! empty ($this-> log)) {

$fw=fopen ($this-> log, "at");

if ($fw) {

fputs ($fw, date (" Y-m-d H:i:s ") ". [". $ip1. "|". $ ip2. "]". $ agent. "n");

fclose ($fw);

}

}

$d [] = $curtime. ":". $ip1. ":". $ip2. "n";

}

$fw = fopen ($this-> filename, "wt");

if (! $fw) {print " the Mistake of recording in a file *quot; ". $this-> filename. " and quot;. "; return;}

foreach ($d as $e) fputs ($fw, $e);

fclose ($fw);

}

function AddAlowAgent ($agent) {

$this-> AGENTS [] = $agent;

}

}

?>

Examples of use

Example 1

This code should be established on the latent page:

<?

include "flooders.inc.php";

$f=new Flooders ();

$f-> Ban ();

?>

This code should be established in the top part of all pages of a site:

<?

include "flooders.inc.php";

$f=new Flooders ();

$f-> Check (404);

?>

Example 2 - not forbidding known search robots.

This code should be established on the latent page:

<?

include "flooders.inc.php";

$f=new Flooders ("/tmp/ban.txt");

$f-> AddAlowAgent ("StackRambler");

$f-> AddAlowAgent ("Googlebot");

$f-> AddAlowAgent ("Yandex");

$f-> AddAlowAgent ("Aport");

$f-> AddAlowAgent ("msnbot");

$f-> AddAlowAgent ("FAST-WebCrawler");

$f-> AddAlowAgent ("Slurp/cat");

$f-> AddAlowAgent ("ASPseek/1.2.10");

$f-> AddAlowAgent ("CNSearch");

$f-> SetLogFileName ("/tmp/ban.log");

$f-> Ban ();

?>

This code should be established in the top part of all pages of a site:

<?

include "flooders.inc.php";

$f=new Flooders ("/tmp/ban.txt");

$f-> Check (403);

?>