This is semantic.info, produced by makeinfo version 4.3 from
semantic.texi.

This manual documents Semantic, a new intrastructure for parser based
text analysis for Emacs.

   Copyright (C) 1999, 2000, 2001, 2002, 2003, 2004 Eric M. Ludlam
Copyright (C) 2001, 2002, 2003 David Ponce Copyright (C) 2002, 2003
Richard Y. Kim

     Permission is granted to copy, distribute and/or modify this
     document under the terms of the GNU Free Documentation License,
     Version 1.1 or any later version published by the Free Software
     Foundation; with the Invariant Sections being list their titles,
     with the Front-Cover Texts being list, and with the Back-Cover
     Texts being list.  A copy of the license is included in the
     section entitled "GNU Free Documentation License".
   
INFO-DIR-SECTION Emacs
START-INFO-DIR-ENTRY
* Semantic: (semantic).	Infrastructure for parser based text analysis.
END-INFO-DIR-ENTRY

   This file documents Semantic.  _Infrastructure for parser based text
analysis in Emacs_

   Copyright (C) 2002 Eric M. Ludlam, David Ponce, and Richard Y. Kim


File: semantic.info,  Node: Top,  Next: Overview,  Up: (dir)

Semantic Manual
***************

semantic is a program for Emacs which includes, at its core, a lexer,
and two compiler compilers (bovinator and wisent).  Additional tools
include, support for imenu, speedbar, whichfunc, eldoc, hippie-expand,
and several other build in tools.

The core utility is the parser infrastructure which allows different
types of parsers to be linked into the system.  Two build in parsers
include the "bovine" parser, and the "wisent" parser.

To send bug reports, or participate in discussions about semantic, use
the mailing list cedet-semantic@sourceforge.net via the URL:
<http://lists.sourceforge.net/lists/listinfo/cedet-semantic>

* Menu:

* Overview::
* Installation::
* Semantic manuals::
* Semantic Internals::
* Glossary::
* GNU Free Documentation License::
* Index::


File: semantic.info,  Node: Overview,  Next: Installation,  Prev: Top,  Up: Top

Overview
********

This chapter gives the overview of semantic and its goals.

With Emacs, regular expressions (and syntax tables) are the basis of
identifying components in a programming language source for purposes
such as color highlighting.  This approach has proved is usefulness,
but have limitations.

semantic provides a new intrastructure that goes far beyond text
analysis based on regular expressions.

semantic uses "parsers" to analyze programming language sources.  For
languages that can be described using a context-free grammar, parsers
can be based on the grammar of the language.  Or they can be "external
parsers" implemented using any means.  This allows the use of a regular
expression parser for non-regular languages, or external programs for
speed.

semantic provides extensive tools to help support a new language.  An
original LL parser, and a Bison-like LALR parser are included.  So, for
a regular language, all that the developer needs to do is write a
grammar file along with appropriate semantic rules.

semantic allows an uniform representation of language components, and
provides a common API so that programmers can develop applications that
work for all languages.  The distribution includes good set of tools
and examples for the application writers, that demonstrate the
usefulness of semantic.

The following diagram illustrates the benefits of using semantic:

*Please Note:*
     The words in all-capital are those that semantic itself provides.
     Others are current or future languages or applications that are not
     distributed along with semantic.

                                                                  Applications
                                                                      and
                                                                   Utilities
                                                                     -------
                                                                    /       \
                    +---------------+    +--------+    +--------+
              C --->| C      PARSER |--->|        |    |        |
                    +---------------+    |        |    |        |
                    +---------------+    | COMMON |    | COMMON |<--- SPEEDBAR
           Java --->| JAVA   PARSER |--->|        |    |        |
                    +---------------+    | PARSE  |    | PARSE  |<--- SENATOR
                    +---------------+    |        |    |        |
         Python --->| PYTHON PARSER |--->| TREE   |    | TREE   |<--- DOCUMENT
                    +---------------+    |        |    |        |
                    +---------------+    | FORMAT |    | API    |<--- SEMANTICDB
         Scheme --->| SCHEME PARSER |--->|        |    |        |
                    +---------------+    |        |    |        |<--- jdee
                    +---------------+    |        |    |        |
        Texinfo --->| TEXI.  PARSER |--->|        |    |        |<--- ecb
                    +---------------+    |        |    |        |
     
                         ...                ...           ...         ...
     
                    +---------------+    |        |    |        |<--- app. 1
        Lang. A --->| A      Parser |--->|        |    |        |
                    +---------------+    |        |    |        |<--- app. 2
                    +---------------+    |        |    |        |
        Lang. B --->| B      Parser |--->|        |    |        |<--- app. 3
                    +---------------+    |        |    |        |
     
                          ...        ...     ...          ...       ...
     
                    +---------------+    |        |    |        |
        Lang. Y --->| Y      Parser |--->|        |    |        |<--- app. ?
                    +---------------+    |        |    |        |
                    +---------------+    |        |    |        |<--- app. ?
        Lang. Z --->| Z      Parser |--->|        |    |        |
                    +---------------+    +--------+    +--------+

* Menu:

* Semantic Components::


File: semantic.info,  Node: Semantic Components,  Up: Overview

Semantic Components
===================

This chapter gives an overview of major components of semantic and how
they interact with each other to perform its job.

The first step of parsing is to break up the input file into its
fundamental components.  This step is called lexical analysis.  The
output of the lexical analyzer is a list of tokens that make up the
file.

             syntax table, keywords list, and options
                              |
                              |
                              v
         input file  ---->  Lexer   ----> token stream

The next step is the parsing shown below.

                         parser tables
                              |
                              v
         token stream --->  Parser  ----> parse tree

The end result, the parse tree, is created based on the parser tables,
which are in the internal representation of the language grammar used by
semantic.

The semantic database provides caching of the parse trees by saving them
into files named `semantic.cache' automatically when loading them when
appropriate instead of re-parsing.  The reason for this is to save the
time it takes to parse a file which could take several seconds or more
for large files.

Finally, semantic provides an API for the Emacs Lisp programmer to
access the information in the parse tree.


File: semantic.info,  Node: Installation,  Next: Semantic manuals,  Prev: Overview,  Up: Top

Installation
************

To install semantic, untar the distribution into a subdirectory, such as
`/usr/share/emacs/site-lisp/semantic-#.#'.  Next, add the following
lines into your individual `.emacs' file, or into
`site-lisp/site-start.el'.

     (setq semantic-load-turn-everything-on t)
     (load-file "/path/to/semantic/semantic-load.el")

If you would like to turn individual tools on or off in your init file,
skip the first line.

Semantic is a part of CEDET.  If you are installing semantic as a part
of CEDET, see the CEDET `INSTALL' file for directions.


File: semantic.info,  Node: Semantic manuals,  Next: Semantic Internals,  Prev: Installation,  Up: Top

Semantic manuals
****************

To know how to use applications enabled by semantic, *note the User's
Guide: (semantic-user)top.

If you are an application developer and need to know the functions and
concepts, *note the Application Development Manual:
(semantic-appdev)top.

To support a new language, *note the Language Support Developer's
Guide: (semantic-langdev)top.

To use the grammar writing environment, and write in the rule based
language, *note the Grammar Framework Manual: (grammar-fw)top.

For details on the original semantic LL parser, *note the Bovine Parser
Manual: (bovine)top.

For details on the Bison-like LALR parser, *note the Wisent Parser
Manual: (wisent)top.


File: semantic.info,  Node: Semantic Internals,  Next: Glossary,  Prev: Semantic manuals,  Up: Top

Semantic Internals
******************

This chapter explains the internals of semantic.  This information
would not be needed by neither applicatioan developers nor grammar
developers.

It would be useful mostly for the hackers who would like to learn more
about how semantic works.


File: semantic.info,  Node: Glossary,  Next: GNU Free Documentation License,  Prev: Semantic Internals,  Up: Top

Glossary
********


     BNF
     In semantic 1.4, a BNF file represented "Bovine Normal Form", the
     grammar file used for the 1.4 parser generator.  This was a play on
     Backus-Naur Form which proved too confusing.


     bovinate
     A verb representing what happens when a bovine parser parsers a
     file.


     bovine lambda
     In a bovine, or LL parser, the bovine lambda is a function to
     execute when a specific set of match rules has succeeded in
     matches text from the buffer.


     bovine parser
     A parser using the bovine parser generator.  It is an LL parser
     suitible for small simple languages.


     context


     LALR


     lexer
     A program which converts text into a stream of tokens by analyzing
     them lexially.  Lexers will commonly create strings, symbols,
     keywords and punctuation, and strip whitespace and comments.


     LL


     nonterminal


     overloadable
     Some functions are defined via `define-overload'.  These can be
     overloaded via ....


     parser
     A program that converts tokens to tags.


     tag
     A tag is a representation of some entity in a language file, such
     as a functin, variable, or include statement.  In semantic, the
     word tag is used the same way it is used for the etags or ctags
     tools.


     token
     A single atomic item returned from a lexer.  It represents some set
     of characters found in a buffer.


     token stream
     The output of the lexer as well as the input to the parser.


     wisent parser
     A parser using the wisent parser generator.  It is a port of bison
     to Emacs Lisp.  It is an LALR parser suitable for complex
     languages.


File: semantic.info,  Node: GNU Free Documentation License,  Next: Index,  Prev: Glossary,  Up: Top

GNU Free Documentation License
******************************

                        Version 1.1, March 2000
     Copyright (C) 2000  Free Software Foundation, Inc.
     59 Temple Place, Suite 330, Boston, MA  02111-1307  USA
     
     Everyone is permitted to copy and distribute verbatim copies
     of this license document, but changing it is not allowed.


  0. PREAMBLE

     The purpose of this License is to make a manual, textbook, or other
     written document "free" in the sense of freedom: to assure everyone
     the effective freedom to copy and redistribute it, with or without
     modifying it, either commercially or noncommercially.  Secondarily,
     this License preserves for the author and publisher a way to get
     credit for their work, while not being considered responsible for
     modifications made by others.

     This License is a kind of "copyleft", which means that derivative
     works of the document must themselves be free in the same sense.
     It complements the GNU General Public License, which is a copyleft
     license designed for free software.

     We have designed this License in order to use it for manuals for
     free software, because free software needs free documentation: a
     free program should come with manuals providing the same freedoms
     that the software does.  But this License is not limited to
     software manuals; it can be used for any textual work, regardless
     of subject matter or whether it is published as a printed book.
     We recommend this License principally for works whose purpose is
     instruction or reference.


  1. APPLICABILITY AND DEFINITIONS

     This License applies to any manual or other work that contains a
     notice placed by the copyright holder saying it can be distributed
     under the terms of this License.  The "Document", below, refers to
     any such manual or work.  Any member of the public is a licensee,
     and is addressed as "you".

     A "Modified Version" of the Document means any work containing the
     Document or a portion of it, either copied verbatim, or with
     modifications and/or translated into another language.

     A "Secondary Section" is a named appendix or a front-matter
     section of the Document that deals exclusively with the
     relationship of the publishers or authors of the Document to the
     Document's overall subject (or to related matters) and contains
     nothing that could fall directly within that overall subject.
     (For example, if the Document is in part a textbook of
     mathematics, a Secondary Section may not explain any mathematics.)
     The relationship could be a matter of historical connection with
     the subject or with related matters, or of legal, commercial,
     philosophical, ethical or political position regarding them.

     The "Invariant Sections" are certain Secondary Sections whose
     titles are designated, as being those of Invariant Sections, in
     the notice that says that the Document is released under this
     License.

     The "Cover Texts" are certain short passages of text that are
     listed, as Front-Cover Texts or Back-Cover Texts, in the notice
     that says that the Document is released under this License.

     A "Transparent" copy of the Document means a machine-readable copy,
     represented in a format whose specification is available to the
     general public, whose contents can be viewed and edited directly
     and straightforwardly with generic text editors or (for images
     composed of pixels) generic paint programs or (for drawings) some
     widely available drawing editor, and that is suitable for input to
     text formatters or for automatic translation to a variety of
     formats suitable for input to text formatters.  A copy made in an
     otherwise Transparent file format whose markup has been designed
     to thwart or discourage subsequent modification by readers is not
     Transparent.  A copy that is not "Transparent" is called "Opaque".

     Examples of suitable formats for Transparent copies include plain
     ASCII without markup, Texinfo input format, LaTeX input format,
     SGML or XML using a publicly available DTD, and
     standard-conforming simple HTML designed for human modification.
     Opaque formats include PostScript, PDF, proprietary formats that
     can be read and edited only by proprietary word processors, SGML
     or XML for which the DTD and/or processing tools are not generally
     available, and the machine-generated HTML produced by some word
     processors for output purposes only.

     The "Title Page" means, for a printed book, the title page itself,
     plus such following pages as are needed to hold, legibly, the
     material this License requires to appear in the title page.  For
     works in formats which do not have any title page as such, "Title
     Page" means the text near the most prominent appearance of the
     work's title, preceding the beginning of the body of the text.

  2. VERBATIM COPYING

     You may copy and distribute the Document in any medium, either
     commercially or noncommercially, provided that this License, the
     copyright notices, and the license notice saying this License
     applies to the Document are reproduced in all copies, and that you
     add no other conditions whatsoever to those of this License.  You
     may not use technical measures to obstruct or control the reading
     or further copying of the copies you make or distribute.  However,
     you may accept compensation in exchange for copies.  If you
     distribute a large enough number of copies you must also follow
     the conditions in section 3.

     You may also lend copies, under the same conditions stated above,
     and you may publicly display copies.

  3. COPYING IN QUANTITY

     If you publish printed copies of the Document numbering more than
     100, and the Document's license notice requires Cover Texts, you
     must enclose the copies in covers that carry, clearly and legibly,
     all these Cover Texts: Front-Cover Texts on the front cover, and
     Back-Cover Texts on the back cover.  Both covers must also clearly
     and legibly identify you as the publisher of these copies.  The
     front cover must present the full title with all words of the
     title equally prominent and visible.  You may add other material
     on the covers in addition.  Copying with changes limited to the
     covers, as long as they preserve the title of the Document and
     satisfy these conditions, can be treated as verbatim copying in
     other respects.

     If the required texts for either cover are too voluminous to fit
     legibly, you should put the first ones listed (as many as fit
     reasonably) on the actual cover, and continue the rest onto
     adjacent pages.

     If you publish or distribute Opaque copies of the Document
     numbering more than 100, you must either include a
     machine-readable Transparent copy along with each Opaque copy, or
     state in or with each Opaque copy a publicly-accessible
     computer-network location containing a complete Transparent copy
     of the Document, free of added material, which the general
     network-using public has access to download anonymously at no
     charge using public-standard network protocols.  If you use the
     latter option, you must take reasonably prudent steps, when you
     begin distribution of Opaque copies in quantity, to ensure that
     this Transparent copy will remain thus accessible at the stated
     location until at least one year after the last time you
     distribute an Opaque copy (directly or through your agents or
     retailers) of that edition to the public.

     It is requested, but not required, that you contact the authors of
     the Document well before redistributing any large number of
     copies, to give them a chance to provide you with an updated
     version of the Document.

  4. MODIFICATIONS

     You may copy and distribute a Modified Version of the Document
     under the conditions of sections 2 and 3 above, provided that you
     release the Modified Version under precisely this License, with
     the Modified Version filling the role of the Document, thus
     licensing distribution and modification of the Modified Version to
     whoever possesses a copy of it.  In addition, you must do these
     things in the Modified Version:

     A. Use in the Title Page (and on the covers, if any) a title
     distinct    from that of the Document, and from those of previous
     versions    (which should, if there were any, be listed in the
     History section    of the Document).  You may use the same title
     as a previous version    if the original publisher of that version
     gives permission.
     B. List on the Title Page, as authors, one or more persons or
     entities    responsible for authorship of the modifications in the
     Modified    Version, together with at least five of the principal
     authors of the    Document (all of its principal authors, if it
     has less than five).
     C. State on the Title page the name of the publisher of the
     Modified Version, as the publisher.
     D. Preserve all the copyright notices of the Document.
     E. Add an appropriate copyright notice for your modifications
     adjacent to the other copyright notices.
     F. Include, immediately after the copyright notices, a license
     notice    giving the public permission to use the Modified Version
     under the    terms of this License, in the form shown in the
     Addendum below.
     G. Preserve in that license notice the full lists of Invariant
     Sections    and required Cover Texts given in the Document's
     license notice.
     H. Include an unaltered copy of this License.
     I. Preserve the section entitled "History", and its title, and add
     to    it an item stating at least the title, year, new authors, and
       publisher of the Modified Version as given on the Title Page.
     If    there is no section entitled "History" in the Document,
     create one    stating the title, year, authors, and publisher of
     the Document as    given on its Title Page, then add an item
     describing the Modified    Version as stated in the previous
     sentence.
     J. Preserve the network location, if any, given in the Document for
       public access to a Transparent copy of the Document, and
     likewise    the network locations given in the Document for
     previous versions    it was based on.  These may be placed in the
     "History" section.     You may omit a network location for a work
     that was published at    least four years before the Document
     itself, or if the original    publisher of the version it refers
     to gives permission.
     K. In any section entitled "Acknowledgements" or "Dedications",
     preserve the section's title, and preserve in the section all the
      substance and tone of each of the contributor acknowledgements
     and/or dedications given therein.
     L. Preserve all the Invariant Sections of the Document,
     unaltered in their text and in their titles.  Section numbers
     or the equivalent are not considered part of the section titles.
     M. Delete any section entitled "Endorsements".  Such a section
     may not be included in the Modified Version.
     N. Do not retitle any existing section as "Endorsements"    or to
     conflict in title with any Invariant Section.

     If the Modified Version includes new front-matter sections or
     appendices that qualify as Secondary Sections and contain no
     material copied from the Document, you may at your option
     designate some or all of these sections as invariant.  To do this,
     add their titles to the list of Invariant Sections in the Modified
     Version's license notice.  These titles must be distinct from any
     other section titles.

     You may add a section entitled "Endorsements", provided it contains
     nothing but endorsements of your Modified Version by various
     parties-for example, statements of peer review or that the text has
     been approved by an organization as the authoritative definition
     of a standard.

     You may add a passage of up to five words as a Front-Cover Text,
     and a passage of up to 25 words as a Back-Cover Text, to the end
     of the list of Cover Texts in the Modified Version.  Only one
     passage of Front-Cover Text and one of Back-Cover Text may be
     added by (or through arrangements made by) any one entity.  If the
     Document already includes a cover text for the same cover,
     previously added by you or by arrangement made by the same entity
     you are acting on behalf of, you may not add another; but you may
     replace the old one, on explicit permission from the previous
     publisher that added the old one.

     The author(s) and publisher(s) of the Document do not by this
     License give permission to use their names for publicity for or to
     assert or imply endorsement of any Modified Version.

  5. COMBINING DOCUMENTS

     You may combine the Document with other documents released under
     this License, under the terms defined in section 4 above for
     modified versions, provided that you include in the combination
     all of the Invariant Sections of all of the original documents,
     unmodified, and list them all as Invariant Sections of your
     combined work in its license notice.

     The combined work need only contain one copy of this License, and
     multiple identical Invariant Sections may be replaced with a single
     copy.  If there are multiple Invariant Sections with the same name
     but different contents, make the title of each such section unique
     by adding at the end of it, in parentheses, the name of the
     original author or publisher of that section if known, or else a
     unique number.  Make the same adjustment to the section titles in
     the list of Invariant Sections in the license notice of the
     combined work.

     In the combination, you must combine any sections entitled
     "History" in the various original documents, forming one section
     entitled "History"; likewise combine any sections entitled
     "Acknowledgements", and any sections entitled "Dedications".  You
     must delete all sections entitled "Endorsements."

  6. COLLECTIONS OF DOCUMENTS

     You may make a collection consisting of the Document and other
     documents released under this License, and replace the individual
     copies of this License in the various documents with a single copy
     that is included in the collection, provided that you follow the
     rules of this License for verbatim copying of each of the
     documents in all other respects.

     You may extract a single document from such a collection, and
     distribute it individually under this License, provided you insert
     a copy of this License into the extracted document, and follow
     this License in all other respects regarding verbatim copying of
     that document.

  7. AGGREGATION WITH INDEPENDENT WORKS

     A compilation of the Document or its derivatives with other
     separate and independent documents or works, in or on a volume of
     a storage or distribution medium, does not as a whole count as a
     Modified Version of the Document, provided no compilation
     copyright is claimed for the compilation.  Such a compilation is
     called an "aggregate", and this License does not apply to the
     other self-contained works thus compiled with the Document, on
     account of their being thus compiled, if they are not themselves
     derivative works of the Document.

     If the Cover Text requirement of section 3 is applicable to these
     copies of the Document, then if the Document is less than one
     quarter of the entire aggregate, the Document's Cover Texts may be
     placed on covers that surround only the Document within the
     aggregate.  Otherwise they must appear on covers around the whole
     aggregate.

  8. TRANSLATION

     Translation is considered a kind of modification, so you may
     distribute translations of the Document under the terms of section
     4.  Replacing Invariant Sections with translations requires special
     permission from their copyright holders, but you may include
     translations of some or all Invariant Sections in addition to the
     original versions of these Invariant Sections.  You may include a
     translation of this License provided that you also include the
     original English version of this License.  In case of a
     disagreement between the translation and the original English
     version of this License, the original English version will prevail.

  9. TERMINATION

     You may not copy, modify, sublicense, or distribute the Document
     except as expressly provided for under this License.  Any other
     attempt to copy, modify, sublicense or distribute the Document is
     void, and will automatically terminate your rights under this
     License.  However, parties who have received copies, or rights,
     from you under this License will not have their licenses
     terminated so long as such parties remain in full compliance.

 10. FUTURE REVISIONS OF THIS LICENSE

     The Free Software Foundation may publish new, revised versions of
     the GNU Free Documentation License from time to time.  Such new
     versions will be similar in spirit to the present version, but may
     differ in detail to address new problems or concerns.  See
     http://www.gnu.org/copyleft/.

     Each version of the License is given a distinguishing version
     number.  If the Document specifies that a particular numbered
     version of this License "or any later version" applies to it, you
     have the option of following the terms and conditions either of
     that specified version or of any later version that has been
     published (not as a draft) by the Free Software Foundation.  If
     the Document does not specify a version number of this License,
     you may choose any version ever published (not as a draft) by the
     Free Software Foundation.


ADDENDUM: How to use this License for your documents
====================================================

To use this License in a document you have written, include a copy of
the License in the document and put the following copyright and license
notices just after the title page:


       Copyright (C)  YEAR  YOUR NAME.
       Permission is granted to copy, distribute and/or modify this document
       under the terms of the GNU Free Documentation License, Version 1.1
       or any later version published by the Free Software Foundation;
       with the Invariant Sections being LIST THEIR TITLES, with the
       Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST.
       A copy of the license is included in the section entitled ``GNU
       Free Documentation License''.
If you have no Invariant Sections, write "with no Invariant
Sections" instead of saying which ones are invariant.  If you have no
Front-Cover Texts, write "no Front-Cover Texts" instead of "Front-Cover
Texts being LIST"; likewise for Back-Cover Texts.

If your document contains nontrivial examples of program code, we
recommend releasing these examples in parallel under your choice of
free software license, such as the GNU General Public License, to
permit their use in free software.


File: semantic.info,  Node: Index,  Prev: GNU Free Documentation License,  Up: Top

Index
*****

* Menu:


Tag Table:
Node: Top1089
Node: Overview1960
Node: Semantic Components6140
Node: Installation7552
Node: Semantic manuals8217
Node: Semantic Internals9014
Node: Glossary9400
Ref: BNF9535
Ref: bovinate9740
Ref: bovine lambda9836
Ref: bovine parser10024
Ref: context10156
Ref: LALR10171
Ref: lexer10183
Ref: LL10401
Ref: nonterminal10411
Ref: overloadable10430
Ref: parser10544
Ref: tag10603
Ref: token10835
Ref: token stream10959
Ref: wisent parser11044
Node: GNU Free Documentation License11216
Node: Index30925

End Tag Table
