<s1 title="Reliability and Load Balancing">

<summarylist/>

<p>As traffic increases, web sites need to add additional web
servers and servlet engines.  Distributing the traffic across the
servers and coping when a server restarts is the challenge of
load balancing.</p>

<ul>
<li>What is balancing the load?
<ul>
<li>Hardware load-balancer
<li>Resin web server using LoadBalanceServlet
<li>Web Server (Apache/IIS) with plugin
</ul>
<li>How is session consistency maintained?
<ul>
<li>Sticky sessions
<li><a href="sessions.xtp">distributed sessions</a>
</ul>
</ul>

<p>In general, the hardware load-balancer will have the best results
while using the Resin or Apache/IIS is a low-cost alternative for
medium sites.</p>

<s2 title="Hardware Load Balancing">

<p>Sites with a hardware load balancer will generally put one Resin JVM
on each server and configure the load balancer to distribute the load
across those JVMs.  Although it's possible to configure Resin with Apache/IIS
in this configuration, it's not necessary and running the Resin as a web
server reduces the configuration complexity.</p>

<p>Sites using sessions will configure <a
href="sessions.xtp">distributed sessions</a> to make sure the users
see the same session values.  The IP-based sticky sessions provided by
hardware load balancers should be used to increase efficiency.  The
IP-sessions will usually send the request to the right server, but
there are clients behind firewalls and proxies which will have
different IPs for each request even though the session is the same.
IP-sessions are only mostly sticky.</p>

<p>A typical configuration will use the same resin.conf for all
servers and use the -server flag to start the correct one:</p>

<example title='resin.conf for all servers'>
&lt;http-server>
  &lt;srun id='a' host='192.168.0.1' port='6802' srun-index='1'/>
  &lt;srun id='b' host='192.168.0.2' port='6802' srun-index='2'/>
  &lt;srun id='c' host='192.168.0.3' port='6802' srun-index='3'/>
  &lt;srun id='d' host='192.168.0.4' port='6802' srun-index='4'/>

  &lt;session-config>
    &lt;tcp-store/>
  &lt;/session-config>

  ...
&lt;/http-server>
</example>

<p>On Unix, the servers will generally be started using some
<a href="httpd.xtp#script">startup script</a>.  Each server will have
a different value for -server and for -pid.</p>

<example title="Starting each server on Unix">
unix> bin/resin -server a -pid server-a.pid start
</example>

<example title="Installing each server on Windows">
resin> bin/httpd -server a -install-as resin-a
resin> net start resin-a
</example>
  
</s2>

<s2 title="Using Resin as the Load Balancer">

<p>Resin 2.1 includes a LoadBalanceServlet which will balance
requests to backend servers.  Because it's implemented as a servlet,
this configuration is the most flexible.  A site might send all
requests for /foo to the backend host 192.168.0.1  and all requests to
/bar to the backend host 192.168.0.2.  Since Resin has an
integrated HTTP proxy cache, the front-end machine can cache results
for the backend servers.</p>

<p>Using Resin as the load balancing web server, requires a minimum
of two configuration files: one for the load balancing server, and one
for the backend servers.  The front configuration will dispatch
several url-patterns to the backend servers, while the backend will
need to actually serve the requests.</p>

<p>The following example sends all URLs starting with /foo to one
of the servers in srun-group <var/a/>.</p>

<example title="front.conf">
&lt;http-server>
  &lt;http port='80'/>

  &lt;srun id='back1' srun-group='a' srun-index='1'
           host='192.168.0.1' port='6802'/>
  &lt;srun id='back2' srun-group='a' srun-index='2'
           host='192.168.0.2' port='6802'/>
  &lt;srun id='back3' srun-group='b' srun-index='3'
           host='192.168.0.3' port='6802'/>

  &lt;servlet>
    &lt;servlet-name>balance-a&lt;/servlet-name>
    &lt;servlet-class>com.caucho.http.servlet.LoadBalanceServlet&lt;/servlet-class>
    &lt;init-param srun-group='a'/>
  &lt;/servlet>

  &lt;servlet-mapping url-pattern='/foo/*' servlet-name='balance-a'/>
&lt;/http-server>
</example>

<ul>
<li>The &lt;http> and &lt;srun> must have different values for <var/id/>.
<li>The &lt;srun> must have an <var/id/> so they will not be
started along with the &lt;http>.
<li>Since the &lt;http> configuration has no <var/id/> it will be
used when you start Resin without a <var/-server/> configuration.
<li>The <var/srun-index/> is important and must match the <var/srun-index/>
in the backend configuration.
</ul>

<p>The backend server configuration is identical to the server
configuration for the hardware load balancer.  In general, most sites
will use some form of distributed session management.</p>

<example title='back.conf for all backend servers'>
&lt;http-server>
  &lt;srun id='a' host='192.168.0.1' port='6802' srun-index='1'/>
  &lt;srun id='b' host='192.168.0.2' port='6802' srun-index='2'/>

  &lt;session-config>
    &lt;tcp-store>
  &lt;/session-config>

  ...
&lt;/http-server>
</example>

</s2>

<s2 title="Balancing with the Apache/IIS plugin">

<p>To understand how Resin's load balancing works, it's important to review
how the plugin dispatches requests to the backend JVM, i.e. the srun.  The
following sequence describes a typical request:</p>

<ol>
<li>Request arrives at web server.
<li>Plugin (mod_caucho, mod_isapi, etc) checks if it's a Resin request
<li>Plugin selects a backend JVM, i.e. a &lt;srun>
<ul>
<li>If it's an old session, send it to the owner JVM. (sticky-sessions)
<li>If it's a new request, send it to the next &lt;srun>, using a
round-robin policy.
</ul>
<li>Plugin sends the request to the backend JVM with a TCP socket.
<li>Plugin receives the response from the backend JVM with the same TCP socket.
</ol>

<p>The plugin needs to know which requests should go to Resin, i.e. the
servlet-mappings.  And it needs to know the TCP host/port names of the
backend machines, i.e. the &lt;srun> and &lt;srun-backup> tags.
/caucho-status shows all that information in one table.  All the
plugins and the JVMs can read the same resin.conf, making maintenance
easier.</p>

<p>The plugin controls the load balancing since it needs to decide
which JVM to use.  Because the plugin is key in load-balancing, looking at
the /caucho-status will tell you exactly how your system is configured.
The JVMs are just passive, waiting for the next request.  From the
JVM-perspective, a request from a plugin is identical to
an HTTP request, except it uses a slightly different encoding.  In fact, with
Resin 1.2, the same JVM can server as an srun and as an httpd listening to
port 8080, for example.  The dual srun/http configuration can be
useful for debugging.
</p>

<p>Selecting a free JVM is done using a round-robin policy.
Although the round-robin policy is simple, in practice it is as
effective as complicated balancing policies.  In addition, because
it's simple, round-robin is more robust and faster than adaptive policies.
</p>

</s2>

<s2 title='Single Machine'>

<p>The cheapest backup strategy just uses a single machine for the web
server and two JVMs.  One JVM is designated as primary and the other
is a backup.  If the first fails for some reason, the second will take
over.  Because the backup is normally not used, it doesn't really take
up much system resources.</p>

<figure src="backup.gif"/>

<example title="resin.conf">
&lt;caucho.com>
&lt;http-server>

  &lt;srun id="a" host='localhost' port='6802' srun-index='1'/>
  &lt;srun-backup id="b" host='localhost' port='6803' srun-index='2'/>

  ...
  
&lt;/http-server>
&lt;/caucho.com>
</example>

<sidebar>Use <var//caucho-status/> to check your configuration</sidebar>

<p>You will start the two srun processes separately.  Each srun needs to
know which port to listen to.  The <var/-server/> argument to httpd.sh selects
a srun block to start.  <var/-server a/> selects the first one, i.e.
port 6802 and <var/-server b/> selects the second one.</p>

<p>Here's how to start them on unix:</p>

<example>
unix> httpd.sh -pid srun1.pid -server a start
unix> httpd.sh -pid srun2.pid -server b start
</example>

<p>On Unix, the <var/-pid/> is used to keep track of the live srun so
a later <var/httpd.sh stop/> will work.  It just names a file that will
contain the process id (pid) of the started process.</p>

<p>Use the -install-as to install them on NT.  <note>The -install-as must
occur before the -server</note>.</p>

<example>
c:\resin1.2> bin/httpd -install-as ResinA -server a
c:\resin1.2> bin/httpd -install-as ResinB -server b
</example>

<p>On Unix, the <var/-pid/> is used to keep track of the live srun so
a later <var/httpd.sh stop/> will work.  It just names a file that will
contain the process id (pid) of the started process.</p>

<p>To make sure that your web server understands the configuration,
look at <var/http://host/caucho-status/>.  caucho-status will
show the current state of all the JVMs.</p>

<s3 title="Summary">

<ul>
<li>The webserver plugin (mod_caucho, or isapi_srun) looks in the resin.conf
for &lt;srun> and &lt;srun-backup> elements.
<li><var//caucho-status/> shows all the servlet runners and backups.
<li><var/httpd.sh -server a/> start the srun with id="a".
<li>The plugin will always send a session to the same backend srun.
</ul>

</s3>

</s2>

<s2 title='Single Web Server Load Balancing'>

<p>Once you start using multiple computers, you can start distributing
the load between a single web server and multiple JVMs.
This is a cheap alternative to a router-based load balancer.  Also, by
using Resin's load balancing, you can make sure that sessions stay on
the same machine.</p>

<figure src="sticky-sessions.gif"/>

<example>
&lt;caucho.com>
&lt;http-server>

  &lt;srun id="1" host='host1' port='6802' srun-index='1'/>
  &lt;srun id="2" host='host2' port='6802' srun-index='2'/>
  &lt;srun id="3" host='host3' port='6802' srun-index='3'/>

  ...
  
&lt;/http-server>
&lt;/caucho.com>
</example>

<p>Each host and the plugin can share the same resin.conf.  Sharing the
resin.conf makes maintenance easier, since the files won't get out of sync.</p>

<p>However, you could use a different resin.conf for plugin and
for each of the JVMs.  The resin.conf for the plugin, e.g. mod_caucho,
just needs enough information to list the &lt;srun> hosts and enough
servlet-mappings to tell the plugin where to send the requests.</p>

<s3 title="Summary">

<ul>
<li>The webserver plugin (mod_caucho, or isapi_srun) looks in the resin.conf
for &lt;srun> and &lt;srun-backup> elements.
<li>The webserver plugin can share the same resin.conf as the JVMs.
<li><var/httpd.sh -server a/> starts the srun with id="a".
<li>The plugin sends a session to the same backend srun.
<li>New sessions or non-session requests balance the load with a round-robin
policy.
</ul>

</s3>

</s2>

<s2 title='What about sessions?'>

<p>A session needs to stay on the same JVM that started it.
Otherwise, each JVM would only see every second or third request and
get confused.</p>

<p>To make sure that sessions stay on the same JVM, Resin encodes the
cookie with the host number.  In the previous example, the hosts would
generate cookies like:</p>

<deftable>
<tr><th>srun-index<th>cookie prefix
<tr><td>1<td><var/a/>xxx
<tr><td>2<td><var/b/>xxx
<tr><td>3<td><var/c/>xxx
</deftable>

<note>srun-index only exists in Resin 1.2.3.  If unspecified, it defaults
to the index of the srun in the srun list.</note>

<p>On the web server, mod_caucho will decode the cookie and send it
to the appropriate host.  So <var/bX8ZwooOz/> would go to host2.</p>

<p>In the infrequent case that host2 fails, Resin will send the
request to host3.  The user will lose the session but that's a minor
problem compared to showing a connection failure error.  To save sessions,
you'll need to use <a href="sessions.xtp">distributed sessions</a>.
Also take a look at
<a href="../java_tut/tcp-sessions.xtp">tcp sessions</a>.</p>

<p>The following example is a typical configuration for a distributed
server using an external hardware load-balancer, i.e. where each Resin is
acting as the HTTP server.  Each server will be started
as <var/-server a/> or <var/-server b/> to grab its specific configuration.</p>

<p>In this example, sessions will only be stored when the server shuts down,
either for maintenance or with a new version of the server.  This is the most
lightweight configuration, and doesn't affect performance significantly.
If the hardware or the JVM crashes, however, the sessions will be lost.
(If you want to save sessions for hardware or JVM crashes,
remove the &lt;save-on-shutdown/> flag.)</p>

<example title="resin.conf">
&lt;caucho.com>
&lt;http-server>
  &lt;http id='a' port='80'/>
  &lt;srun id='a' port='6802' host='192.168.0.1' srun-index='1'/>

  &lt;http id='b' port='80'/>
  &lt;srun id='b' port='6802' host='192.168.0.2' srun-index='2'/>

  &lt;http id='c' port='80'/>
  &lt;srun id='c' port='6802' host='192.168.0.3' srun-index='3'/>

  &lt;!-- enable tcp-store for all hosts/web-apps -->
  &lt;session-config>
    &lt;tcp-store/>
    &lt;save-on-shutdown/>
  &lt;/session-config>

  ...
&lt;/http-server>
&lt;/caucho.com>
</example>

</s2>

<s2 title='Multiple Web Servers'>

<p>Many larger sites like to use multiple web servers with a JVM and a
web server on each machine.  A router will distribute the load between
the machines.</p>

<p>In this configuration, the site needs to take control of its own
sessions.  Because the router will distribute the load randomly, any
persistent session state needs to be handled by a centralized server
like a database or use Resin's TCP-ring storage.
</p>

<p>Even in this configuration, you can use Resin's load balancing to
increase reliability.  Each web server should choose its own JVM
first, but use another machine as a backup.
</p>

<p>In this case, you can use the trick that <var/localhost/> refers
to the preferred host.  The configuration would look like:</p>

<example>
&lt;caucho.com>
&lt;http-server>

  &lt;srun id="a" host='localhost' port='6802' srun-index='1'/>
  &lt;srun-backup id="b" host='host1' port='6802' srun-index='2'/>
  &lt;srun-backup id="c" host='host2' port='6802' srun-index='3'/>
  &lt;srun-backup id="d" host='host3' port='6802' srun-index='4'/>
  ...
  
&lt;/http-server>
&lt;/caucho.com>
</example>

<p>Alternately, if you're using Apache, you could configure the sruns
in the httpd.conf.</p>

<example title='host1 httpd.conf'>
CauchoConfigFile /home/www/resin/conf/resin.conf
CauchoHost host1 6802
CauchoBackup host2 6802
</example>

<example title='host2 httpd.conf'>
CauchoConfigFile /home/www/resin/conf/resin.conf
CauchoBackup host1 6802
CauchoHost host2 6802
</example>

<p><b>The order must be consistent for all servers so sessions will always
go to the correct machine.</b>  <var/bXXX/> must always go to host2.</p>

</s2>

<s2 title='Multiple Web Servers, Single JVM'>

<p>Multiple web servers can use the same JVM.  For example, a fast
plain webserver and an SSL web server may only need a single JVM.
(Although a backup would be good.)  Since the JVM doesn't care where
the request comes from, it can treat each request identically.</p>

<p>This simplifies SSL development.  A servlet just needs to check
the <var/request.isSecure()/> method to see if the request is SSL or
not.  Other than that, all requests are handled identically.</p>

</s2>

<s2 title='When everything fails'>

<p>By default, if <var/mod_caucho/> can't connect to any of the JVMs,
it will return a basic "can't connect" page to the user.  Sites which
want a more professional response can redirect the user to an error
page.</p>

<p>In the resin.conf, you'll use:</p>

<example>
&lt;caucho.com>
&lt;http-server>
  &lt;error-page exception-type='connection'
               location='/error.html'/>
&lt;/http-server>
&lt;/caucho.com>
</example>

<p>The error page must be an absolute path because it could be called
from any url.  Of course, it can't refer to a servlet or to a JSP file.</p>

</s2>
</s1>
