
#include "bacula_template.inc" title="Recycling Volumes" num=11

<h1>Automatic Volume Recycling</h1>
Normally, <b>Bacula</b> will write on a volume, and once the
tape is written, it may append to the volume, but it will 
never overwrite the data thus destroying it. When we
speak of <b>recycling</b> volumes, we mean that <b>Bacula</b>
can write over the previous contents of a volume. Thus all
previous data will be lost.   
<p>
If you are like me, you may not want <b>Bacula</b> to automatically
recycle (reuse) tapes. This requires a large number of tapes, and
when I need  a tape, I manually recycle it. For more on 
manual recycling, see the section entitled 
<a href="recycling.html#manualrecycling">
Manually Recycling Volumes</a> below in this chapter.
<p>
Most people prefer to have a Pool
of tapes that are used for daily backups and recycled once a week,
another Pool of tapes that are used for Full backups once a week
and recycled monthly, and finally a Pool of tapes that are used
once a month and recycled after a year or two. With a scheme
like this, your pool of tapes remains constant.      
<p>By properly defining your Volume Pools with appropriate Retention
periods, <b>Bacula</b> can manage the recycling (such as defined 
above) automatically.
<p>Automatic recycling of Volumes is controlled by three
records in the <b>Pool</b> resource definition in the Director's
configuration file. These three records are:
<b>
<ul type="disc">
<li>AutoPrune = yes</li>
<li>VolumeRetention = &lt;time&gt;</li>
<li>Recycle = yes</li>
</ul>
</b>
Automatic recycling of Volumes is performed by <b>Bacula</b> only when
it wants a new Volume and no appendable Volumes are available in the
Pool.  It will then search the Pool for any Volumes with the
<b>Recycle</b> flag set and whose Volume Status is <b>Full</b>.  At that
point, the recycling occurs in two steps.  The first is that a Volume
must be purged of all Jobs and Files, and the second step is the actual
recycling of the Volume. The Volume will be purged if the VolumeRetention
period has expired.  If no volumes can be recycled for any of the reasons
stated above, <b>Bacula</b> will request operator intervention (i.e.
it will ask you to label a new volume).
<p>

<h2>Automatic Pruning</h2>
By setting <b>AutoPrune</b> to <b>yes</b> you will permit <b>Bacula</b>
to automatically prune all Volumes in the Pool when a Job needs   
another Volume. When a Job requests another volume and
there are no Volumes with Volume Status
<b>Append</b> available, <b>Bacula</b> will begin volume
pruning. This means that all Jobs that
are older than the <b>VolumeRetention</b> period will be pruned from
every Volume that has Volume Status <b>Full</b> and has Recycle set
to <b>yes</b>. When all files are pruned from a Volume, it will be marked
as <b>Purged</b> implying that no Jobs remain on the volume.
The records that control the pruning are described below.
<dl>
   <dt>AutoPrune = &lt;yes/no&gt;</dt><dd>If AutoPrune is set to
       <b>yes</b> (default), Bacula (version 1.20 or greater) will
       automatically apply the Volume retention period when running a Job
       and it needs a new Volume but no appendable volumes are available.
       At that point, Bacula will prune all Volumes that can be pruned
       (i.e. AutoPrune set) in an attempt to find a usable volume. If
       during the autoprune, all files are pruned from the Volume, it
       will be marked with VolStatus <b>Purged</b>.
       The default is <b>yes</b>.</dd>

   <dt>Volume Retention = &lt;time-period-specification&gt;</dt><dd>The
       Volume Retention record defines the length of time that
       <b>Bacula</b> will guarantee that the Volume is not reused counting
       from the time the last job stored on the Volume terminated.
       <p>
       When this time period expires, and if <b>AutoPrune</b> is set to
       <b>yes</b>, and a new Volume is needed, but no appendable Volume
       is available, <b>Bacula</b> will prune (remove) Job records that
       are older than the specified Volume Retention period. 
       <p>The Volume Retention period takes precedence 
       over any Job Retention period you have specified in the
       Client resource. It should also be noted, that the Volume
       Retention period is obtained by reading the Catalog Database Media
       record rather than the Pool resource record. This means that
       if you change the VolumeRetention in the Pool resource record,
       you must ensure that the corresponding change is made in the
       catalog. When all files are removed from the volume, its VolStatus
       is set to <b>Purged</b>.

       <p>Retention periods are specified in seconds, but as
       a convenience, there are a number of modifiers that permit easy
       specification in terms of minutes, hours, days, weeks, months,
       quarters, or years on the record.  See the 
       <a href="configure.html#Time">Configuration chapter</a> of this
       manual for additional details of modifier specification.  
       <p>The default is 1 year.  </dd>

   <dt>Recycle = &lt;yes/no&gt;</dt><dd>This statement tells Bacula
       whether or not the particular Volume can be recycled (i.e.
       rewritten). If Recycle is set to <b>no</b>
       (the default), then even if <b>Bacula</b> prunes all the Jobs
       on the volume, it will not consider the tape for recycling. If
       Recycle is set to <b>yes</b> and all Jobs have been pruned, the
       volume status will be set to <b>Purged</b> and the volume may
       then be reused when another volume is needed.  If the volume is
       reused, it is relabeled with the same Volume Name and all previous
       data will be lost.</dd>

</dl>

<h2>Recycling</h2>
After all Volumes of a Pool have been pruned (as mentioned above, this
happens when a Job needs a new Volume and no appendable Volumes are
available), <b>Bacula</b> will look for the oldest Volume that is
Purged (all Jobs and Files expired), and if the <b>Recycle</b> flag
is on (Recycle=yes) for that Volume, <b>Bacula</b> will relabel it
and write new data on it.

<h2>Recycle Status</h2>
Each Volume inherits the Recycle status (yes or no) from the
Pool resource record when the Media
record is created (normally when the Volume is labeled). This
Recycle status is stored in the Media record of the Catalog.
Using the the Console program, you 
may subsequently change the Recycle status for each Volume.
For example in the following output from <b>list volumes</b>:
<font size=-2>
<pre>
+------------+-----------+-----------+----------+------------------+----------+-------+
| VolumeName | MediaType | VolStatus | VolBytes | LastWritten      | VolReten | Recyc |
+------------+-----------+-----------+----------+------------------+----------+-------+
| File0001   | File      | Full      | 4190055  | 2002-05-25 18:42 | 14400    | 1     |
| File0002   | File      | Full      | 1896460  | 2002-05-26 18:05 | 14400    | 1     |
| File0003   | File      | Full      | 1896460  | 2002-05-26 20:05 | 14400    | 1     |
| File0004   | File      | Full      | 1896460  | 2002-05-26 21:35 | 14400    | 1     |
| File0005   | File      | Full      | 1896460  | 2002-05-26 22:05 | 14400    | 1     |
| File0006   | File      | Full      | 1896460  | 2002-05-26 19:35 | 14400    | 1     |
| File0007   | File      | Purged    | 1896466  | 2002-05-26 18:05 | 14400    | 1     |
+------------+-----------+-----------+----------+------------------+----------+-------+
</pre></font>
all the volumes are marked as recyclable, and the last Volume, <b>File0007</b>
has been purged, so it may be immediately recycled. The other volumes are all 
marked recyclable and when their Volume Retention period (14400 seconds or 4 hours)
expires, they will be eligible for pruning, and possible recycling.
Even though Volume <b>File0007</b> has been purged, all the data on the
Volume is still recoverable. A purged Volume simply means that there are
no entries in the Catalog. Even if the Volume Status is changed to 
<b>Recycle</b>, the data on the Volume will be recoverable. The data
is lost only when the Volume is re-labeled and re-written.
<p>
To modify Volume <b>File0001</b> so that it cannot be recycled, you
use the <b>update volume pool=File</b> command in the console program,
or simply <b>update</b> and <b>Bacula</b> will prompt you for the information.
<font size=-2>
<pre>
+------------+-----------+-----------+----------+------------------+----------+-------+
| VolumeName | MediaType | VolStatus | VolBytes | LastWritten      | VolReten | Recyc |
+------------+-----------+-----------+----------+------------------+----------+-------+
| File0001   | File      | Full      | 4190055  | 2002-05-25 18:42 | 14400    | 0     |
| File0002   | File      | Full      | 1897236  | 2002-05-26 23:35 | 14400    | 1     |
| File0003   | File      | Full      | 1896460  | 2002-05-26 20:05 | 14400    | 1     |
| File0004   | File      | Full      | 1896460  | 2002-05-26 21:35 | 14400    | 1     |
| File0005   | File      | Full      | 1896460  | 2002-05-26 22:05 | 14400    | 1     |
| File0006   | File      | Full      | 1896460  | 2002-05-26 19:35 | 14400    | 1     |
| File0007   | File      | Purged    | 1896466  | 2002-05-26 18:05 | 14400    | 1     |
+------------+-----------+-----------+----------+------------------+----------+-------+
</pre> </font>
In this case, <b>File0001</b> will never be automatically recycled. The
same effect can be achieved by setting the Volume Status to Read-Only.

<h2><a name="singletape">Making Bacula Use a Single Tape</a></h2>
Most people will want <b>Bacula</b> to fill a tape and when
it is full, a new tape will be mounted, and so on.  However, as
an extreme example, it is possible for <b>Bacula</b> to write
on a single tape, and every night to rewrite it. To get this
to work, you must do two things: first, set the VolumeRetention
to less than your save period (one day), and the second item
is to make <b>Bacula</b> mark the tape as full after using
it once. This is done using <b>UseVolumeOnce = yes</b>. If
this latter record is not used and the tape is not full
after the first time it is written, <b>Bacula</b> will simply
append to the tape and eventually request another volume.
Using the tape only once, forces the tape to be marked
<b>Full</b> after each use, and the next time <b>Bacula</b>
runs, it will recycle the tape.
<p>An example Pool resource that does this is:
<pre>
Pool {
  Name = DDS-4
  Use Volume Once = yes
  Pool Type = Backup
  AutoPrune = yes
  VolumeRetention = 12h # expire after 12 hours
  Recycle = yes
}
</pre>

<h2><a name="usageexample">A Daily, Weekly, Monthly Tape Usage Example</a></h2>
This example is meant to show you how one could define
a fixed set of volumes that <b>Bacula</b> will rotate through
on a regular schedule. There are an infinite number of such
schemes, all of which have various advantages and disadvantages.
<p>We start with the following assumptions:
<ul>
  <li>A single tape has more than enough capacity to do
      a full save.</li>
  <li>There are 10 tapes that are used on a daily basis
      for incremental backups. They are prelabled Daily1 ...
      Daily10.
  <li>There are 4 tapes that are used on a weekly basis
      for full backups. They are labeled Week1 ... Week4.</li>
  <li>There are 12 tapes that are used on a monthly basis
      for full backups. They are numbered Month1 ... Month12</li>
  <li>A full backup is done every Saturday evening (tape inserted
      Friday evening before leaving work).</li>
  <li>No backups are done over the weekend (this is easy to
      change).</li>
  <li>The first Friday of each month, a Monthly tape is used for
      the Full backup.</li>
  <li>Incremental backups are done Monday - Friday (actually
      Tue-Fri mornings).
</ul>
We start the system by doing a Full save to one of the weekly volumes
or one of the monthly volumes.  The next morning, we remove the tape 
and insert a Daily tape. Friday evening, we remove the Daily tape
and insert the next tape in the Weekly series. Monday, we remove the
Weekly tape and re-insert the Daily tape.  On the first Friday of the
next month, we insert the next Monthly tape in the series rather than
a Weekly tape, then continue. When a Daily tape finally fills up,
<b>Bacula</b> will request the next one in the series, and the next
day when you notice the email message, you will mount it and <b>Bacula</b>
will finish the unfinished incremental backup.

<p>What does this give?  Well, at any point, you will have a 
the last complete Full save plus several Incremental saves. 
For any given file your want to recover (or your whole system), you
will have a copy of that file every day for at least the last 14 days.
For older versions, you will have at least 3 and probably 4 Friday
full saves of that file, and going back further, you will have a copy
of that file made on the beginning of the month for at least a year.
<p>So you have copies of any file (or your whole system) for at least
a year, but as you go back in time, the time between copies increases
from daily to weekly to monthly.
<p>What would the <b>Bacula</b> configuration look like to implement
such a scheme?
<pre>
Schedule {
  Name = "NightlySave"
  Run = Level=Full Pool=Monthly 1sat at 03:05
  Run = Level=Full Pool=Weekly 2sat-5sat at 03:05
  Run = Level=Incremental Pool=Daily tue-fri at 03:05
}

Job {
  Name = "NightlySave"
  Type = Backup
  Level = Full
  Client = LocalMachine
  FileSet = "File Set"
  Messages = Standard
  Storage = DDS-4
  Pool = FullSaves
  Schedule = "NightlySave"
}

# Definition of file storage device
Storage {
  Name = DDS-4
  Address = localhost
  SDPort = 9103
  Password = XXXXXXXXXXXXX
  Device = FileStorage
  Media Type = 8mm
}

FileSet {
  Name = "File Set"
  Include = signature=MD5 { 
    fffffffffffffffff
  }
  Exclude = { *.o }
}

Pool {
  Name = Daily
  Pool Type = Backup
  AutoPrune = yes
  VolumeRetention = 10d   # recycle in 10 days
  Maximum Volumes = 10
  Recycle = yes
}

Pool {
  Name = Weekly
  Use Volume Once = yes
  Pool Type = Backup
  AutoPrune = yes
  VolumeRetention = 30d  # recycle in 30 days (default)
  Recycle = yes
}

Pool {
  Name = Monthly
  Use Volume Once = yes
  Pool Type = Backup
  AutoPrune = yes
  VolumeRetention = 365d  # recycle in 1 year
  Recycle = yes
}
</pre>

<h2><a name="PruningExample">
Automatic Pruning and Recycling Example</a></h2>  
Perhaps the best way to understand the various resource records
that come into play during automatic pruning and recycling is to
run a Job that goes through the whole cycle.  If you add the
following resources to your Director's configuration file:
<pre>
Schedule {
  Name = "30 minute cycle"
  Run = Level=Full Pool=File Messages=Standard Storage=File hourly at 0:05
  Run = Level=Full Pool=File Messages=Standard Storage=File hourly at 0:35
}

Job {
  Name = "Filetest"
  Type = Backup
  Level = Full
  Client=XXXXXXXXXX
  FileSet="Test Files"
  Messages = Standard
  Storage = File
  Pool = File
  Schedule = "30 minute cycle"
}

# Definition of file storage device
Storage {
  Name = File
  Address = XXXXXXXXXXX
  SDPort = 9103
  Password = XXXXXXXXXXXXX
  Device = FileStorage
  Media Type = File
}

FileSet {
  Name = "Test Files"
  Include = signature=MD5 { 
    fffffffffffffffff
  }
  Exclude = { *.o }
}

Pool {
  Name = File
  Use Volume Once = yes
  Pool Type = Backup
  LabelFormat = File
  AutoPrune = yes
  VolumeRetention = 4h
  Maximum Volumes = 12
  Recycle = yes
}
</pre>
Where you will need to replace the <b>ffffffffff</b>'s by the 
appropriate files to be saved for your configuration. For the FileSet Include,
choose a directory that has one or two megabytes maximum since there
will probably be approximately 8 copies of the directory that
<b>Bacula</b> will cycle through.
<p>In addition, you will need to add the following to your
Storage daemon's configuration file:
<pre>
Device {
  Name = FileStorage
  Media Type = File
  Archive Device = /tmp
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
}
</pre>
<p>With the above resources, <b>Bacula</b> will start a Job every half 
hour that saves a copy of the directory you chose to /tmp/File0001 ...
/tmp/File0012. After 4 hours, Bacula will start recycling the backup
Volumes (/tmp/File0001 ...). You should see this happening in the 
output produced. Bacula will automatically create the Volumes (Files)
the first time it uses them.
<p>To turn it off, either delete all the resources you've added, or
simply comment out the <b>Schedule</b> record in the <b>Job</b>
resource.

<h2><a name="manualrecycling">Manually Recycling Tapes</h2>
Although automatic recycling of tapes is implemented
in version 1.20 and later (see the <a href="recycling.html">Automatic
Recycling of Volumes</a> chapter of this manual),
you may want to manually force reuse (recycling) of a tape.

<p>Assuming that you want to keep the tape Volume name, but
you simply want to write new data on the tape, the steps to
take are:
<ul>
  <li>Use the <b>update volume</b> command in the Console to   
      ensure that the <b>Recycle</b> field is set to <b>1</b></li>
  <li>Use the <b>purge jobs volume</b> command in the Console
      to mark the Volume as <b>Purged</b>.  Check by using
      <b>list volumes</b>.</li>
</ul>
Once the Volume is marked Purged, it will be recycled the
next time a Volume is needed.

<p>If you wish to reuse the tape by giving it a new name,
follow the following steps:  
<ul>
  <li>Use the <b>purge jobs volume</b> command in the Console
      to mark the Volume as <b>Purged</b>.  Check by using
      b>list volumes</b>.</li>
  <li>In Bacula version 1.30 or greater, use the Console
      <b>relabel</b> command to relabel the Volume.</li>
</ul>
For Bacula versions prior to 1.30 or to manually relabel the
Volume, use the instructions below:   
<ul>
  <li>Use the <b>delete volume</b> command in the Console
      to delete the Volume from the Catalog.</li>
  <li>If the a different tape is mounted, use the <b>unmount</b>
      command, remove the tape, and insert the tape to be
      renamed.</b>
  <li>Write an EOF mark in the tape using the following 
      commands:
<pre>
  mt -f /dev/nst0 rewind
  mt -f /dev/nst0 weof
</pre>
      where you replace <b>/dev/nst0</b> with the appropriate
      device name on your system.</li>
   <li>Use the <b>label</b> command to write a new label to
       the tape and to enter it in the catalog.</li>
</ul>
<p>
With <b>Bacula</b> versions prior to 1.19, the <b>delete</b>
command will only delete the selected Volume from the database 
and not its associated Job and File records. With <b>Bacula</b>
version 1.19 and later, all associated records are deleted from the database.
<p>
Please be aware that the <b>delete</b> command can be dangerous.
Once it is done, to recover the File records, you must either
restore your database as it was before the <b>delete</b> command,
or use the <b>bscan</b> utility program to scan the tape and
recreate the database entries.

<page_prev name="Catalog Maintenance" url="catmaintenance.html">
<page_next name="Backing Up to Disk Volumes" url="disk.html">
