Varnish is a high-performance open source HTTP accelerator. The largest online newspaper in Norway, Verdens Gang, uses 3 Varnishes instead of the original 12 Squids, which performs better than before.
But compared with the old Squid, each has its own advantages and disadvantages. The large number of relative comparisons on the Internet are only used to maximize the use of the applications they are familiar with. Perhaps Squid can only be sufficient to exert its most powerful power when it is capable.
Varnish adopts "Visual Page Cache" technology. In terms of memory utilization, Varnish has an advantage over Squid. It avoids Squid frequently swapping files in memory and disk, and its performance is higher than Squid.
Through the Varnish management port, regular expressions can be used to quickly and batch clear some caches, which is something that Squid cannot have.
I will give a brief introduction and notes on some insights and configuration methods of varnish
Laboratory environment: Red Hat Enterprise Linux Server release 5.4 (Tikanga)
Kernel 2.6.18-164.el5
yum install pcre-devel ##Preinstall a software package, otherwise an error will be prompted
tar zxvf varnish-2.1.
cd varnish-2.1.3
./configure --prefix=/usr/local/varnish-2.1.3
make && make install
Edit the configuration file, there are templates, but there are too many comments, it is best to create a new one by yourself
vim /usr/local/varnish-2.1.3/etc/varnish/
###############################################################################################################################################################################################################################################################
#http request processing process
#1, receive the entry status of the request, judge whether pass or lookup local query based on vcl
#lookup, search for data in the hash table, if found, enter hit state, otherwise enter fetch state
#pass, select the background and enter the fetch state
#fetch, backend fetch requests, send requests, obtain data, and store locally
#deliver, send data to the client, enter done
#done, processing ends
###########Configure the backend server###########################
backend linuxidc01 {
.host = "192.168.1.142";
.port = "7070";
.probe = {
.timeout = 5s;
.interval = 2s;
.window = 10;
.threshold = 8;
}
}
backend linuxidc02 {
.host = "192.168.1.141";
.port = "7070";
.probe = {
.timeout = 5s;
.interval = 2s;
.window = 10;
.threshold = 8;
}
}
###################Configure the backend server group, perform health detection for 6 seconds, and set the weight using random method ############
########## Another way round-robin is the default polling mechanism #########################################################################################################################################################################################################################################
director linuxidc15474 random
{ .retries = 6;
{ .backend = linuxidc02;
.weight = 2;
}
{ .backend = linuxidc01;
.weight = 2;
}
}
############### Define the access list, allowing the following addresses to clear the varnish cache################################################################################################
acl local {
"localhost";
"127.0.0.1";
}
#########Judges the following server and cache configuration from the url##########################################################################################
sub vcl_recv
{
if ( ~ "^") #Match the domain name and jump to the background server
{ set = linuxidc15474; }
else { error 404 "Unknown HostName!"; }
if ( == "PURGE") #Not allowed to clear the varnish cache in non-access control list
{ if (! ~ local)
{
error 405 "Not Allowed.";
return (lookup);
}
}
#Clear cookies with jpg and other files in the url
if ( == "GET" && ~ "\.(jpg|png|gif|swf|jpeg|ico)$")
{
unset ;
}
#Judgement-Forwarded-For If the front-end has multiple reverse proxy, you can get the client IP address.
if (-forwarded-for)
{
set -Forwarded-For = -Forwarded-For ", " ;
}
else { set -Forwarded-For = ; }
##varnish implements anti-theft chain of pictures
# if ( ~ "http://.*)
# {
# if ( !( ~ "http://.*vicp\.net" ||
# ~ "http://.*linuxidc15474\.net" ) )
# {
# set = "";
# set = "/";
# }
# return(lookup);
# }
# else {return(pass);}
if ( != "GET" &&
!= "HEAD" &&
!= "PUT" &&
!= "POST" &&
!= "TRACE" &&
!= "OPTIONS" &&
!= "DELETE")
{ return (pipe); }
#Direct forwarding of non-GET|HEAD requests to the backend server
if ( != "GET" && != "HEAD")
{ return (pass); }
##For GET requests and end with .php and .php? in the url, they will be forwarded directly to the backend server.
if ( == "GET" && ~ "\.(php)($|\?)")
{ return (pass); }
##For verification and cookies in the request, it will be forwarded directly to the backend server
if ( || )
{ return (pass);}
{
##In addition to the above access requests, look up from the cache
return (lookup);
}
##The specified font directory is not cached
if ( ~ "^/fonts/")
{ return (pass); }
}
sub vcl_pipe
{ return (pipe); }
## Enter pass mode, the request is sent to the backend, and the backend returns data to the client, but does not enter cache processing
sub vcl_pass
{ return (pass); }
sub vcl_hash
{
set += ;
if ()
{ set += ; }
else { set += ; }
return (hash);
}
##If the requested cache is found in the cache after lookingup, the following keywords usually end
sub vcl_hit
{
if (!)
{ return (pass); }
return (deliver);
}
##Called when the cache is not found after lookingup, the following keywords end, and the fetch parameter is called to retest whether to add the cache
sub vcl_miss
{ return (fetch); }
#Let the type of varnish server cache, get data from the backend and call it
sub vcl_fetch
{ if (!)
{ return (pass); }
if (-Cookie)
{ return (pass); }
##WEB server indicates content that is not cached, and the varnish server does not cache
if ( ~ "no-cache" || -Control ~ "no-cache" || -Control ~ "private")
{ return (pass); }
##Cached files containing jpg, png and other formats in the access get. The cache time is 7 days and s is seconds.
if ( == "GET" && ~ "\.(js|css|mp3|jpg|png|gif|swf|jpeg|ico)$")
{ set = 7d; }
##For accessing static pages such as htm in the get, cache for 300 seconds
if ( == "GET" && ~ "\/[0-9]\.htm$")
{ set = 300s; }
return (deliver);
}
####Add to view cache hits in the page header information##########
sub vcl_deliver
{
set -hits = ;
if ( > 0)
{ set -Cache = "HIT cqtel-bbs"; }
else { set -Cache = "MISS cqtel-bbs"; }
}
###############################################################################################################################
Create a user:
groupadd www
useradd www -g www
Create a cache location for varnish_cache
mkdir /data/varnish_cache
Start varnish
ulimit -SHn 8192 #### Set file descriptor, because my machine performance is not good, you can set it according to your own configuration
/usr/local/varnish-2.1.3/sbin/varnishd -u www -g www -f /usr/local/varnish-2.1.3/etc/varnish/ -a 0.0.0.0:80 -s file,/data/varnish_cache/varnish_cache.data,100M -w 1024,8192,10 -t 3600 -T 127.0.0.1:3500
####-u What to run -g What group to run -f varnish configuration file -a Bind IP and port -s varnish cache file location and size -w Minimum, maximum thread and timeout -T varnish management port, mainly used to clear cache
#End the varnishd process
pkill varnishd
Start varnishncsa to write Varnish access logs to log files:
/usr/local/varnish-2.1.3/bin/varnishncsa -w /data/logs/ &
Run at 0 o'clock every day, cut the Varnish logs by day, generate a compressed file, and delete the script of the old log of the last month (/var/logs/):
vim /usr/local/varnish-2.1.3/etc/varnish/cut_varnish_log.sh
Write the following script:
#!/bin/sh
# This file run at 00:00
date=$(date -d "yesterday" +"%Y-%m-%d")
pkill -9 varnishncsa
mv /data/logs/ /data/logs/${date}.log
/usr/local/varnish-2.1.3/bin/varnishncsa -w /data/logs/ &
mkdir -p /data/logs/varnish/
gzip -c /data/logs/${date}.log > /data/logs/varnish/${date}.
rm -f /data/logs/${date}.log
rm -f /data/logs/varnish/$(date -d "-1 month" +"%Y-%m*").
Timing tasks:
crontab -e
00 00 * * * /usr/local/varnish-2.1.3/etc/varnish/cut_varnish_log.sh
Optimize Linux kernel parameters
vi /etc/
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.ip_local_port_range = 5000 65000
Make the configuration effective
/sbin/sysctl -p
Batch clear cache using regular expressions via Varnish management port
Clear all caches
/usr/local/varnish-2.1.3/bin/varnishadm -T 127.0.0.1:3500 *$
Clear all caches in the image directory
/usr/local/varnish-2.1.3/bin/varnishadm -T 127.0.0.1:3500 /image/
127.0.0.1:3500 is the cleared cache server address. The cleared domain name /static/image/ is the cleared URL address list.
/usr/local/varnish-2.1.3/bin/varnishadm -T 127.0.0.1:3500 purge " ~ $ && ~ /static/image/"
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
A PHP function that clears Squid cache
<?php
function purge($ip, $url)
{
$errstr = '';
$errno = '';
$fp = fsockopen ($ip, 80, $errno, $errstr, 2);
if (!$fp)
{
return false;
}
else
{
$out = "PURGE $url HTTP/1.1\r\n";
$out .= "Host:blog.\r\n";
$out .= "Connection: close\r\n\r\n";
fputs ($fp, $out);
$out = fgets($fp , 4096);
fclose ($fp);
return true;
}
}
purge("192.168.0.4", "/");
?>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Configure the automatic start of Varnish
vim /etc//
Write the following on the last line:
ulimit -SHn 8192
/usr/local/varnish-2.1.3/sbin/varnishd -u www -g www -f /usr/local/varnish-2.1.3/etc/varnish/ -a 0.0.0.0:80 -s file,/data/varnish_cache/varnish_cache.data,100M -w 1024,8192,10 -t 3600 -T 127.0.0.1:3500
/usr/local/varnish-2.1.3/bin/varnishncsa -w /data/logs/ &
Check the number of Varnish server connections and hits:
/usr/local/varnish-2.1.3/bin/varnishstat
The above is the status of varnish.
1675 0.00 �
179 0.00
11
Use help to see which Varnish commands you can use:
/usr/local/varnish-2.1.3/bin/varnishadm -T 127.0.0.1:3500 help
But compared with the old Squid, each has its own advantages and disadvantages. The large number of relative comparisons on the Internet are only used to maximize the use of the applications they are familiar with. Perhaps Squid can only be sufficient to exert its most powerful power when it is capable.
Varnish adopts "Visual Page Cache" technology. In terms of memory utilization, Varnish has an advantage over Squid. It avoids Squid frequently swapping files in memory and disk, and its performance is higher than Squid.
Through the Varnish management port, regular expressions can be used to quickly and batch clear some caches, which is something that Squid cannot have.
I will give a brief introduction and notes on some insights and configuration methods of varnish
Laboratory environment: Red Hat Enterprise Linux Server release 5.4 (Tikanga)
Kernel 2.6.18-164.el5
yum install pcre-devel ##Preinstall a software package, otherwise an error will be prompted
tar zxvf varnish-2.1.
cd varnish-2.1.3
./configure --prefix=/usr/local/varnish-2.1.3
make && make install
Edit the configuration file, there are templates, but there are too many comments, it is best to create a new one by yourself
vim /usr/local/varnish-2.1.3/etc/varnish/
###############################################################################################################################################################################################################################################################
#http request processing process
#1, receive the entry status of the request, judge whether pass or lookup local query based on vcl
#lookup, search for data in the hash table, if found, enter hit state, otherwise enter fetch state
#pass, select the background and enter the fetch state
#fetch, backend fetch requests, send requests, obtain data, and store locally
#deliver, send data to the client, enter done
#done, processing ends
###########Configure the backend server###########################
Copy the codeThe code is as follows:
backend linuxidc01 {
.host = "192.168.1.142";
.port = "7070";
.probe = {
.timeout = 5s;
.interval = 2s;
.window = 10;
.threshold = 8;
}
}
backend linuxidc02 {
.host = "192.168.1.141";
.port = "7070";
.probe = {
.timeout = 5s;
.interval = 2s;
.window = 10;
.threshold = 8;
}
}
###################Configure the backend server group, perform health detection for 6 seconds, and set the weight using random method ############
########## Another way round-robin is the default polling mechanism #########################################################################################################################################################################################################################################
Copy the codeThe code is as follows:
director linuxidc15474 random
{ .retries = 6;
{ .backend = linuxidc02;
.weight = 2;
}
{ .backend = linuxidc01;
.weight = 2;
}
}
############### Define the access list, allowing the following addresses to clear the varnish cache################################################################################################
Copy the codeThe code is as follows:
acl local {
"localhost";
"127.0.0.1";
}
#########Judges the following server and cache configuration from the url##########################################################################################
Copy the codeThe code is as follows:
sub vcl_recv
{
if ( ~ "^") #Match the domain name and jump to the background server
{ set = linuxidc15474; }
else { error 404 "Unknown HostName!"; }
if ( == "PURGE") #Not allowed to clear the varnish cache in non-access control list
{ if (! ~ local)
{
error 405 "Not Allowed.";
return (lookup);
}
}
#Clear cookies with jpg and other files in the url
if ( == "GET" && ~ "\.(jpg|png|gif|swf|jpeg|ico)$")
{
unset ;
}
#Judgement-Forwarded-For If the front-end has multiple reverse proxy, you can get the client IP address.
if (-forwarded-for)
{
set -Forwarded-For = -Forwarded-For ", " ;
}
else { set -Forwarded-For = ; }
##varnish implements anti-theft chain of pictures
# if ( ~ "http://.*)
# {
# if ( !( ~ "http://.*vicp\.net" ||
# ~ "http://.*linuxidc15474\.net" ) )
# {
# set = "";
# set = "/";
# }
# return(lookup);
# }
# else {return(pass);}
if ( != "GET" &&
!= "HEAD" &&
!= "PUT" &&
!= "POST" &&
!= "TRACE" &&
!= "OPTIONS" &&
!= "DELETE")
{ return (pipe); }
#Direct forwarding of non-GET|HEAD requests to the backend server
if ( != "GET" && != "HEAD")
{ return (pass); }
##For GET requests and end with .php and .php? in the url, they will be forwarded directly to the backend server.
if ( == "GET" && ~ "\.(php)($|\?)")
{ return (pass); }
##For verification and cookies in the request, it will be forwarded directly to the backend server
if ( || )
{ return (pass);}
{
##In addition to the above access requests, look up from the cache
return (lookup);
}
##The specified font directory is not cached
if ( ~ "^/fonts/")
{ return (pass); }
}
sub vcl_pipe
{ return (pipe); }
## Enter pass mode, the request is sent to the backend, and the backend returns data to the client, but does not enter cache processing
sub vcl_pass
{ return (pass); }
sub vcl_hash
{
set += ;
if ()
{ set += ; }
else { set += ; }
return (hash);
}
##If the requested cache is found in the cache after lookingup, the following keywords usually end
sub vcl_hit
{
if (!)
{ return (pass); }
return (deliver);
}
##Called when the cache is not found after lookingup, the following keywords end, and the fetch parameter is called to retest whether to add the cache
sub vcl_miss
{ return (fetch); }
#Let the type of varnish server cache, get data from the backend and call it
sub vcl_fetch
{ if (!)
{ return (pass); }
if (-Cookie)
{ return (pass); }
##WEB server indicates content that is not cached, and the varnish server does not cache
if ( ~ "no-cache" || -Control ~ "no-cache" || -Control ~ "private")
{ return (pass); }
##Cached files containing jpg, png and other formats in the access get. The cache time is 7 days and s is seconds.
if ( == "GET" && ~ "\.(js|css|mp3|jpg|png|gif|swf|jpeg|ico)$")
{ set = 7d; }
##For accessing static pages such as htm in the get, cache for 300 seconds
if ( == "GET" && ~ "\/[0-9]\.htm$")
{ set = 300s; }
return (deliver);
}
####Add to view cache hits in the page header information##########
sub vcl_deliver
{
set -hits = ;
if ( > 0)
{ set -Cache = "HIT cqtel-bbs"; }
else { set -Cache = "MISS cqtel-bbs"; }
}
###############################################################################################################################
Create a user:
groupadd www
useradd www -g www
Create a cache location for varnish_cache
mkdir /data/varnish_cache
Start varnish
ulimit -SHn 8192 #### Set file descriptor, because my machine performance is not good, you can set it according to your own configuration
/usr/local/varnish-2.1.3/sbin/varnishd -u www -g www -f /usr/local/varnish-2.1.3/etc/varnish/ -a 0.0.0.0:80 -s file,/data/varnish_cache/varnish_cache.data,100M -w 1024,8192,10 -t 3600 -T 127.0.0.1:3500
####-u What to run -g What group to run -f varnish configuration file -a Bind IP and port -s varnish cache file location and size -w Minimum, maximum thread and timeout -T varnish management port, mainly used to clear cache
#End the varnishd process
pkill varnishd
Start varnishncsa to write Varnish access logs to log files:
/usr/local/varnish-2.1.3/bin/varnishncsa -w /data/logs/ &
Run at 0 o'clock every day, cut the Varnish logs by day, generate a compressed file, and delete the script of the old log of the last month (/var/logs/):
vim /usr/local/varnish-2.1.3/etc/varnish/cut_varnish_log.sh
Write the following script:
#!/bin/sh
# This file run at 00:00
date=$(date -d "yesterday" +"%Y-%m-%d")
pkill -9 varnishncsa
mv /data/logs/ /data/logs/${date}.log
/usr/local/varnish-2.1.3/bin/varnishncsa -w /data/logs/ &
mkdir -p /data/logs/varnish/
gzip -c /data/logs/${date}.log > /data/logs/varnish/${date}.
rm -f /data/logs/${date}.log
rm -f /data/logs/varnish/$(date -d "-1 month" +"%Y-%m*").
Timing tasks:
crontab -e
00 00 * * * /usr/local/varnish-2.1.3/etc/varnish/cut_varnish_log.sh
Optimize Linux kernel parameters
vi /etc/
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.ip_local_port_range = 5000 65000
Make the configuration effective
/sbin/sysctl -p
Batch clear cache using regular expressions via Varnish management port
Clear all caches
/usr/local/varnish-2.1.3/bin/varnishadm -T 127.0.0.1:3500 *$
Clear all caches in the image directory
/usr/local/varnish-2.1.3/bin/varnishadm -T 127.0.0.1:3500 /image/
127.0.0.1:3500 is the cleared cache server address. The cleared domain name /static/image/ is the cleared URL address list.
/usr/local/varnish-2.1.3/bin/varnishadm -T 127.0.0.1:3500 purge " ~ $ && ~ /static/image/"
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
A PHP function that clears Squid cache
Copy the codeThe code is as follows:
<?php
function purge($ip, $url)
{
$errstr = '';
$errno = '';
$fp = fsockopen ($ip, 80, $errno, $errstr, 2);
if (!$fp)
{
return false;
}
else
{
$out = "PURGE $url HTTP/1.1\r\n";
$out .= "Host:blog.\r\n";
$out .= "Connection: close\r\n\r\n";
fputs ($fp, $out);
$out = fgets($fp , 4096);
fclose ($fp);
return true;
}
}
purge("192.168.0.4", "/");
?>
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Configure the automatic start of Varnish
vim /etc//
Write the following on the last line:
ulimit -SHn 8192
/usr/local/varnish-2.1.3/sbin/varnishd -u www -g www -f /usr/local/varnish-2.1.3/etc/varnish/ -a 0.0.0.0:80 -s file,/data/varnish_cache/varnish_cache.data,100M -w 1024,8192,10 -t 3600 -T 127.0.0.1:3500
/usr/local/varnish-2.1.3/bin/varnishncsa -w /data/logs/ &
Check the number of Varnish server connections and hits:
/usr/local/varnish-2.1.3/bin/varnishstat
The above is the status of varnish.
1675 0.00 �
179 0.00
11
Use help to see which Varnish commands you can use:
/usr/local/varnish-2.1.3/bin/varnishadm -T 127.0.0.1:3500 help