                         The BLAST Databases
                    Last updated on July 21, 2003

This document describes the "BLAST" databases available on the NCBI 
FTP site under the "blast/db" subdirectory.  The direct URL to this 
subdirectory is:
      ftp://ftp.ncbi.nih.gov/blast/db

I. General Introduction

NCBI BLAST home pages (http://www.ncbi.nih.gov/BLAST/) use a standard 
set of BLAST databases for Nucleotide, Protein, and Translated BLAST 
searches.  These databases are made available in the db directory as 
compressed archives (ftp://ftp.ncbi.nih.gov/blast/db/) in preformatted 
format.  The FASTA databases now reside under the blast/FASTA 
subdirectory.

The preformatted databases offer the following advantages:

	* The preformatted databases are smaller in size and are
                faster to download;
	* Preformatting removes the need to run formatdb;
	* Taxonomy information is available for each database entry.

Preformatted databases must be downloaded in binary mode and inflated 
with gzip or other decompress tools. The BLAST database files can then 
be extracted out of the resulting tar file using “tar” program on Unix/Linux 
or WinZip and StuffIt Expander on Windows and Macintosh platforms, 
respectively.  

Large databases are formatted in multiple 1 Gigabytes volumes, which 
are named using the “database.##.tar.gz” convention.  All relevant volumes
are required. An alias file is provided so that the database can be called
using the alias name without the extension (.nal or .pal). For example, 
to call est database, simply use “–d est” option in the commandline 
(without the quotes). 

Certain databases are subsets of a larger parental database. For those 
databases, mask files, rather than actual databases, are provided. The 
mask file needs the parent database to function properly. The parent 
databases should be generated on the same day as the mask file. For 
example, to use swissprot preformatted database, swissprot.tar.gz, one 
will need to get the nr.tar.gz with the same date stamp.

Additional BLAST databases that are not provided in preformatted 
formats are available in the FASTA subdirectory.  For genomic BLAST 
databases, please check the genomes ftp directory at:
	ftp://ftp.ncbi.nih.gov/genomes/


2. Contents of the /blast/db/ directory

The formatted databases are archived in this directory.  The name of 
these databases and their contents are listed below.
+--------------------+-----------------------------------------------+
|File Name           | Content Description                           |
+--------------------+-----------------------------------------------+
/FASTA                 subdirectory for FASTA formatted sequences
	
README                 README for this subdirectory (this file)

est.00.tar.gz          | three volumes of the formatted est database
est.01.tar.gz          | from the EST division of GenBank, EMBL, 
est.02.tar.gz          | and DDBJ

est_human.tar.gz       | mask file for human subset of the est
est_mouse.tar.gz       | mask file for mouse subset of the est
est_others.tar.gz      | mask file for non-human and non-mouse subset
                       | of the est database
                       | These three mask files need all volumes of
                       | est to function properly.

gss.00.tar.gz          | two volumes of the formatted gss database
gss.01.tar.gz          | from the GSS division of GenBank, EMBL, and
                       | DDBJ

htgs.00.tar.gz         | three volumes of htgs database with entries
htgs.01.tar.gz         | from HTG division of GenBank, EMBL, and DDBJ
htgs.01.tar.gz         |

human_genomic.tar.gz   human RefSeq (NC_######) chromosome records
                         with gap adjusted concatenated NT_ contigs
 
nr.tar.gz              non-redundant protein sequence database with 
                         entries from GenPept, Swissprot, PIR, PDF, PDB,
                         and NCBI RefSeq

nt.00.tar.gz           | nucleotide sequence database, with entries 
nt.01.tar.gz           | from all traditional divisions of GenBank,  
nt.02.tar.gz           | EMBL, and DDBJ excluding bulk divisions (gss, 
                       | sts, pat, est, and htg divisions. wgs entries
                       | are also excluded. Not non-redundant.

other_genomic.tar.gz   RefSeq chromosome records (NC_######) for 
                         organisms other than human

pataa.tar.gz           | patent protein sequence database
patnt.tar.gz           | patent nucleotide sequence database
                       | The above two databases are directly from 
                       | USPTO or from EU/Japan Patent Agencies via 
                       | EMBL/DDBJ

pdbaa.tar.gz           protein sequences from pdb protein structures
pdbnt.tar.gz           nucleotide sequences from pdb nucleic acid 
                         structures. They are NOT the protein coding 
                         sequences for the corresponding pdbaa entries.

sts.tar.gz             Sequences from the STS division of GenBank, EMBL,
                         and DDBJ

swissprot.tar.gz       swiss-prot sequence databases (last major update)

taxdb.tar.gz           Taxonomy information for the formatted database 

wgs.00.tar.gz          | Whole genome shotgun sequence assemblies for 
wgs.01.tar.gz          | different organisms, broken up into 1 GB 
wgs.02.tar.gz          | volumes. 
wgs.03.tar.gz
wgs.04.tar.gz
wgs.05.tar.gz
+--------------------+-----------------------------------------------+


3. Content of the /db/FASTA Subdirectory

This subdirectory contains FASTA formatted sequence files, formerly 
available under /db directory. The file names and database contents 
are listed below. These files are now archived in .gz format and must 
be processed through formatdb before they can be used by the BLAST 
programs. 

+--------------------+-----------------------------------------------+
|File Name           | Content Description                           |
+--------------------+-----------------------------------------------+
alu.a.gz                translation of alu.n repeats
alu.n.gz                alu repeat elements

drosoph.aa.gz           CDS translations from drosophila.nt  
drosoph.nt.gz           genomic sequences for drosophila

ecoli.aa.gz		CDS translations from ecoli.nt
ecoli.nt.gz             Escherichia coli K-12 genomic sequences

est_human.gz*           | human subset of the est database (see Note 1)
est_mouse.gz*           | mouse subset of the est database
est_others.gz*          | non-human and non-mouse subset of the est 
                          database

gss.gz*                 sequences from the GSS division of GenBank,
                          EMBL, and DDBJ

htg.gz*                 htgs database with high throughput genomic 
                          entries from the htg division of GenBank, 
                          EMBL, and DDBJ 

human_genomic.gz*       human RefSeq (NC_######) chromosome records
                          with gap adjusted concatenated NT_ contigs 

igSeqNt.gz              human and mouse immunoglobulin nucleotide 
                          sequences
igSeqProt.gz            human and mouse immunoglobulin protein 
                          sequences

mito.aa.gz              CDS translations of complete mitochondrial 
                          genomes
mito.nt.gz              complete mitochondrial genomes

month.aa.gz             | newly released/updated protein sequences 
                            (See Note 2)
month.est_human.gz      | newly released/updated human est sequences	
month.est_mouse.gz      | newly released/updated mouse est sequences
month.est_others.gz     | newly released/updated est other than 
                        |   human/mouse
month.gss.gz 		| newly released/updated gss sequences 
month.htgs.gz		| newly released/updated htgs sequences
month.nt.gz             | newly released/updated sequences for the nt
                          database 

nr.gz*                  non-redundant protein sequence database with 
                          entries from GenPept, Swissprot, PIR, PDF, 
                          PDB, and RefSeq

nt.gz*                  nucleotide sequence database, with entries
                          from all traditional divisions of GenBank, 
                          EMBL, and DDBJ excluding bulk divisions 
                          (gss, sts, pat, est, htg divisions) and wgs 
                          entries. Not non-redundant.

other_genomic.gz*       RefSeq chromosome records (NC_######) for 
                          organisms other than human

pataa.gz*               | patent protein sequence database
patnt.gz*               | patent nucleotide sequence database
                        | The above two dbs are directly from USPTO 
                        | of from EU/Japan Patent Agency via EMBL/DDBJ

pdbaa.gz*               protein sequences from pdb protein structures
pdbnt.gz*               nucleotide sequences from pdb nucleic acid 
                         structures. They are NOT the protein coding 
                         sequences for the corresponding pdbaa entries.

sts.gz*			database for sequence tag site entries 

swissprot.gz*		swiss-prot database (last major release)

vector.gz		vector sequence database (See Note 3)

wgs.gz*			whole genome shotgun genome assemblies

yeast.aa.gz             protein translations from yeast genome
yeast.nt.gz             yeast genomes.
+--------------------+-----------------------------------------------+
NOTE: 
(1) we do not provide the complete est database in FASTA format. One
    need  to get all three subsets(est_human, est_mouse, and est_others
    and concatenate them into the complete est fasta database.
(2) month.### databases are the sequences newly released or updated
    within the last 30 days for that database.
(3) For vector contamination screening, use the UniVec database from:
    ftp://ftp.ncbi.nih.gov/pub/UniVec/			
 *  marked files have preformatted counterparts. 


4. Database updates

The BLAST databases are updated daily.  Update of existing databases 
by merging of new records from the month database using fmerge is no 
longer supported. We do not have an established incremental update 
scheme at this time. We recommend downloading the databases regularly 
to keep their content current.

5. Non-redundant defline syntax

The only non-redundant database is the protein nr. In it, identical 
sequences are merged into one entry. To be merged two sequences must 
have identical lengths and every residue at every position must be the 
same.  The FASTA deflines for the different entries that belong to one 
nr record are separated by control-A characters invisible to most 
programs. In the example below both entries gi|1469284 and gi|1477453 
have the same sequence, in every respect:

>gi|3023276|sp|Q57293|AFUC_ACTPL   Ferric transport ATP-binding protein afuC 
^Agi|1469284|gb|AAB05030.1|   afuC gene product ^Agi|1477453|gb|AAB17216.1|   
afuC [Actinobacillus pleuropneumoniae]
MNNDFLVLKNITKSFGKATVIDNLDLVIKRGTMVTLLGPSGCGKTTVLRLVAGLENPTSGQIFIDGEDVT
KSSIQNRDICIVFQSYALFPHMSIGDNVGYGLRMQGVSNEERKQRVKEALELVDLAGFADRFVDQISGGQ
QQRVALARALVLKPKVLILDEPLSNLDANLRRSMREKIRELQQRLGITSLYVTHDQTEAFAVSDEVIVMN
KGTIMQKARQKIFIYDRILYSLRNFMGESTICDGNLNQGTVSIGDYRFPLHNAADFSVADGACLVGVRPE
AIRLTATGETSQRCQIKSAVYMGNHWEIVANWNGKDVLINANPDQFDPDATKAFIHFTEQGIFLLNKE

The syntax of sequence header lines used by the NCBI BLAST server 
depends on the database from which each sequence was obtained.  The table 
below lists the identifiers for the databases from which the sequences 
were derived.

  Database Name                     Identifier Syntax
  ============================      ========================
  GenBank                           gb|accession|locus
  EMBL Data Library                 emb|accession|locus
  DDBJ, DNA Database of Japan       dbj|accession|locus
  NBRF PIR                          pir||entry
  Protein Research Foundation       prf||name
  SWISS-PROT                        sp|accession|entry name
  Brookhaven Protein Data Bank      pdb|entry|chain
  Patents                           pat|country|number 
  GenInfo Backbone Id               bbs|number 
  General database identifier	    gnl|database|identifier
  NCBI Reference Sequence           ref|accession|locus
  Local Sequence identifier         lcl|identifier

"gi" identifiers are being assigned by NCBI for all sequences contained 
within NCBI's sequence databases.  The "gi" identifier provides a uniform 
and stable naming convention whereby a specific sequence is assigned its 
unique gi identifier.  If a nucleotide or protein sequence changes, 
however, a new gi identifier is assigned, even if the accession number 
of the record remains unchanged. Thus gi identifiers provide a mechanism 
for identifying the exact sequence that was used or retrieved in a given 
search. 

We recommend that "gi display option" be activated in local blast search 
by setting the -I option to T, which was set to false by default: 

  -I  Show GI's in deflines [T/F]
    default = F

For databases whose entries are not from official NCBI sequence databases, 
such as Trace database, the gnl| convention is used. For custom database, 
this convention should be followed and the id for each sequence must be 
unique, if one would like to take the advantage of indexed database, 
which enables specific sequence retrieval using fastacmd program included 
in the blast executable package.  One should refer to documents 
distributed in the standalone BLAST package for more details. 


6. Formatting the FASTA database

FASTA database files need to be formatted with formatdb before they can be 
used in local blast search.  For those from NCBI, the following formatdb 
are recommended:
	formatdb –i input_db –p F –o T	for nucleotide
	formatdb –i input_db –p T –o T	for protein

The -A option introduced in 2.2.3 is now built into the formatdb program 
and thus removed from the list of configurable options since 2.2.8. This 
enables formatdb to properly handle large sequence files (longer than 16
million bases).  Please refer to formatdb.txt under the /blast/documents 
directory for more information.  Database preprared using 2.2.8 formatdb 
will not be backward compatible with blast programs old than version 2.2.3.


7. Technical Support

Questions and comments on this document and NCBI BLAST related questions 
should be sent to blast-help group at:
      blast-help@ncbi.nlm.nih.gov

For information about other NCBI resources/services, please send email to 
NCBI User Serivce at:
      info@ncbi.nlm.nih.gov


