Script to see Concurrent Users

Connect with other users about what to run on your webhosting (and how to run it) here.
Post Reply
Jaminb2030
New to forums
New to forums
Posts: 2
https://www.youtube.com/channel/UC40BgXanDqOYoVCYFDSTfHA
Joined: Fri May 31, 2013 10:10 am

Script to see Concurrent Users

Post by Jaminb2030 »

So I wanted a way to see how many concurrent users i had, i didn't want people to be getting timed out due to hitting this limit without me knowing. So i made the script below, it parses the access_log and takes an educated guess at how many concurrent users there are.

Note that this is not an exact science, maybe with changes to the log format it can be refined but you will likely need to get a VM as NFO probably will not change that.

Any requests from a unique IP in the last 3 seconds is considered a Concurrent User. Multiple users behind a Natted private address will only show as 1 Concurrent User.
Even so, unless your website markets to large corporations this should be fairly accurate!

This script also has a built in search function allowing you see dig through the connection logs and see whats hitting your site.

(IPs in the image below are all web-crawlers)
Image

You can do alot more with this like reverse-DNSing the IP to see who is hitting your webserver and other stuff so feel free to add to it! Cheers!


Add this concurrent_users.php script to your main directory (inside Public), if you put it somewhere else make sure to point file('../access_log') at the top of the script to the right place. If you don't know PHP ../ mean up 1 directory.

Code: Select all

<?php
$log_file = file('../access_log');


$ip_list = array();
$now = strtotime("NOW");
$concurrent_users = 0;


foreach($log_file as $key => $data){ // for each line in the log
	$data = str_replace("\n", "", $data);
	$temp = explode(" ", $data); // Array by spaces
	$website = $temp[0]; // first non-space chunk is the website accessed
	$Connecting_ip = $temp[1]; // IP is next non-space chunk
	if(!isset($ip_list[$Connecting_ip])){
		$ip_list[$Connecting_ip] = array(); // Builds an array by IP
	}
	array_push($ip_list[$Connecting_ip], Array()); // Creates a new array for each connection request by this IP.
	end($ip_list[$Connecting_ip]);         // move the internal pointer to the end of the array
	$key = key($ip_list[$Connecting_ip]);  // fetches the key of the element pointed to by the internal pointer
	$ip_list[$Connecting_ip][$key]['website'] = $website;

	preg_match('/\d{1,2}\/\w*\/\d{4}:\d{2}:\d{2}:\d{2}\s-\d+/', $data, $access_time); // DateTime Regex
	$ip_list[$Connecting_ip][$key]['access_time'] = $access_time[0];
	
	$string_details = substr($data, (strpos($data, '"')+1)); // The rest of the line minus the starting "
	$request_type = explode(" ", $string_details);
	$ip_list[$Connecting_ip][$key]['request_type'] = $request_type[0]; // Ok first part of this is the request method
	$ip_list[$Connecting_ip][$key]['file_accesses'] = $request_type[1]; // Next is the directory requested
	$ip_list[$Connecting_ip][$key]['protocol_used'] = substr($request_type[2], 0, (strlen($request_type[2])-1)); //then the protocol used minus the "
	$ip_list[$Connecting_ip][$key]['return_code'] = $request_type[3]; // HTTP return code
	$ip_list[$Connecting_ip][$key]['response_size'] = $request_type[4]; // response size
	$client_information = ''; // Just lump the rest of the exploded array into a string
	for($id=6; $id<=count($request_type); $id++){
		$client_information .= $request_type[$id];
	}
	$ip_list[$Connecting_ip][$key]['client_information'] = str_replace('"', '', $client_information); // The end of the string is client information, we do not want Quotes
	if(strlen($access_time[0]) > 0){
		$time = ($now - strtotime($access_time[0])); // time between request and now
		if($time <= 3 && !isset($ip_list[$Connecting_ip]['active_user'])){// PHP does not track miliseconds so, anything 3 seconds or less ago is considered a "Concurrent user", no not perfect but good enough for general ideas
			$concurrent_users++;
			$ip_list[$Connecting_ip]['active_user'] = 1;
		}
		$ip_list[$Connecting_ip][$key]['time_since_connect'] = $time;
	}
}
echo '<h1> Concurrent Users: '.$concurrent_users.'</h1>';


// ----------------------------------------------- IF YOU have a very large access_log.txt file, the load times for this script can become quite long, if so i would delete everything from here down and just use the $concurrent_users varable.



$http_codes = array( // https://gist.github.com/henriquemoody/6580488
    100 => 'Continue',
    101 => 'Switching Protocols',
    102 => 'Processing', // WebDAV; RFC 2518
    200 => 'OK',
    201 => 'Created',
    202 => 'Accepted',
    203 => 'Non-Authoritative Information', // since HTTP/1.1
    204 => 'No Content',
    205 => 'Reset Content',
    206 => 'Partial Content',
    207 => 'Multi-Status', // WebDAV; RFC 4918
    208 => 'Already Reported', // WebDAV; RFC 5842
    226 => 'IM Used', // RFC 3229
    300 => 'Multiple Choices',
    301 => 'Moved Permanently',
    302 => 'Found',
    303 => 'See Other', // since HTTP/1.1
    304 => 'Not Modified',
    305 => 'Use Proxy', // since HTTP/1.1
    306 => 'Switch Proxy',
    307 => 'Temporary Redirect', // since HTTP/1.1
    308 => 'Permanent Redirect', // approved as experimental RFC
    400 => 'Bad Request',
    401 => 'Unauthorized',
    402 => 'Payment Required',
    403 => 'Forbidden',
    404 => 'Not Found',
    405 => 'Method Not Allowed',
    406 => 'Not Acceptable',
    407 => 'Proxy Authentication Required',
    408 => 'Request Timeout',
    409 => 'Conflict',
    410 => 'Gone',
    411 => 'Length Required',
    412 => 'Precondition Failed',
    413 => 'Request Entity Too Large',
    414 => 'Request-URI Too Long',
    415 => 'Unsupported Media Type',
    416 => 'Requested Range Not Satisfiable',
    417 => 'Expectation Failed',
    418 => 'I\'m a teapot', // RFC 2324
    419 => 'Authentication Timeout', // not in RFC 2616
    420 => 'Enhance Your Calm', // Twitter
    420 => 'Method Failure', // Spring Framework
    422 => 'Unprocessable Entity', // WebDAV; RFC 4918
    423 => 'Locked', // WebDAV; RFC 4918
    424 => 'Failed Dependency', // WebDAV; RFC 4918
    424 => 'Method Failure', // WebDAV)
    425 => 'Unordered Collection', // Internet draft
    426 => 'Upgrade Required', // RFC 2817
    428 => 'Precondition Required', // RFC 6585
    429 => 'Too Many Requests', // RFC 6585
    431 => 'Request Header Fields Too Large', // RFC 6585
    444 => 'No Response', // Nginx
    449 => 'Retry With', // Microsoft
    450 => 'Blocked by Windows Parental Controls', // Microsoft
    451 => 'Redirect', // Microsoft
    451 => 'Unavailable For Legal Reasons', // Internet draft
    494 => 'Request Header Too Large', // Nginx
    495 => 'Cert Error', // Nginx
    496 => 'No Cert', // Nginx
    497 => 'HTTP to HTTPS', // Nginx
    499 => 'Client Closed Request', // Nginx
    500 => 'Internal Server Error',
    501 => 'Not Implemented',
    502 => 'Bad Gateway',
    503 => 'Service Unavailable',
    504 => 'Gateway Timeout',
    505 => 'HTTP Version Not Supported',
    506 => 'Variant Also Negotiates', // RFC 2295
    507 => 'Insufficient Storage', // WebDAV; RFC 4918
    508 => 'Loop Detected', // WebDAV; RFC 5842
    509 => 'Bandwidth Limit Exceeded', // Apache bw/limited extension
    510 => 'Not Extended', // RFC 2774
    511 => 'Network Authentication Required', // RFC 6585
    598 => 'Network read timeout error', // Unknown
    599 => 'Network connect timeout error' // Unknown
);



$table_header = '<table border="2" ID="Log_table"><thead bgcolor="gray"><tr><td style="width:10%"><strong><center>Connecting IP</center></strong></td><td style="width:10%"><strong><center>Time Since</center></strong></td><td style="width:30%"><strong><center>File Accessed</center></strong></td><td style="width:10%"><strong><center>Return Code</center></strong></td><td style="width:10%"><strong><center>Protocol Used</center></strong></td><td style="width:10%"><strong><center>Website</center></strong></td><td style="width:10%"><strong><center>Response Size</center></strong></td><td style="width:10%"><strong><center>Client Info</center></strong></td></thead></tr>';
$table_body = '';

function humanTiming ($time) // ulx time to human readable time
{

    $time = ($time<1)? 1 : $time;
    $tokens = array (
        31536000 => 'year',
        2592000 => 'month',
        604800 => 'week',
        86400 => 'day',
        3600 => 'hour',
        60 => 'minute',
        1 => 'second'
    );

    foreach ($tokens as $unit => $text) {
        if ($time < $unit) continue;
        $numberOfUnits = floor($time / $unit);
        return $numberOfUnits.' '.$text.(($numberOfUnits>1)?'s':'');
    }

}

?>
<script type="text/javascript">
	function makeTableScroll() {
		var maxRows = 4;

		var table = document.getElementById('Log_table');
		var wrapper = table.parentNode;
		var rowsInTable = table.rows.length;
		var height = 0;
		if (rowsInTable > maxRows) {
			for (var i = 0; i < maxRows; i++) {
				height += table.rows[i].clientHeight;
			}
			wrapper.style.height = height + "px";
		}
	}
</script>
<style>
* {
  box-sizing: border-box;
}

#myInput {
  background-image: url('');
  background-position: 10px 10px;
  background-repeat: no-repeat;
  width: 100%;
  font-size: 16px;
  padding: 12px 20px 12px 40px;
  border: 1px solid #ddd;
  margin-bottom: 12px;
}
</style>
<input type="text" id="myInput" onkeyup="myFunction()" placeholder="Search Access List" autofocus>
<?php


foreach($ip_list as $key => $ip){
	foreach($ip as $key2 => $value){
		if(is_numeric($key2)){
			if($value['file_accesses'] == '/'){
				$value['file_accesses'] = 'root'; // saying root is nicer then /
			}
			$table_body .= '<tr><td>'.$key.'</td><td>'.humanTiming($value['time_since_connect']).' ago</td><td>'.$value['file_accesses'].'</td><td>'.$value['return_code'].' - '.$http_codes[$value['return_code']].'</td><td>'.$value['protocol_used'].'</td><td>'.$value['website'].'</td><td>'.$value['response_size'].'</td><td>'.$value['client_information'].'</td></tr>';
		}
	}
}
$table_body .= '</table>';


echo $table_header.$table_body;
//echo '<pre>' , var_dump($ip_list), '</pre>'; // uncomment if you want to see all the data.
?>
<script>
function myFunction() {
  var input, filter, table, tr, td, i, txtValue;
  input = document.getElementById("myInput");
  filter = input.value.toUpperCase();
  table = document.getElementById("Log_table");
  tr = table.getElementsByTagName("tr");
  for (i = 1; i < tr.length; i++) {
	td = tr[i].getElementsByTagName("td");
	if (td) {
		pass = 0;
		for (e = 1; e < td.length; e++) {
			txtValue = td[e].textContent || td[e].innerText;
			if (txtValue.toUpperCase().indexOf(filter) > -1) {
				pass = 1;
			}
		}
		if (pass == 1) {
			tr[i].style.display = "";
		} else {
			tr[i].style.display = "none";
		}
	}       
  }
}
</script>
User avatar
Edge100x
Founder
Founder
Posts: 12945
Joined: Thu Apr 18, 2002 11:04 pm
Location: Seattle
Contact:

Re: Script to see Concurrent Users

Post by Edge100x »

If you are interested in knowing how close you are to the limit that we have for our webhosting plans, this command will give you the number of running Apache processes for your user, which should be close to what Apache considers concurrent sessions:

Code: Select all

ps -u yourusername | grep apache2 | wc -l
If you reach that limit, you shouldn't see timeouts; instead, users should start seeing 503 errors. If a user is seeing timeouts, it could be something else, such as a per-IP anti-DDoS limit on our end. If you contact us, we can investigate that further.
Post Reply