Class | YARD::Parser::SourceParser
In: | lib/yard/parser/source_parser.rb
Parent: | Object

Responsible for parsing a source file into the namespace. Parsing also invokes handlers to process the parsed statements and generate any code objects that may be recognized.

## Custom Parsers

SourceParser allows custom parsers to be registered and called when a certain filetype is recognized. To register a parser and hook it up to a set of file extensions, call {register_parser_type}

@see register_parser_type @see Handlers::Base @see CodeObjects::Base

### Methods

after_parse_file after_parse_file_callbacks after_parse_list after_parse_list_callbacks before_parse_file before_parse_file_callbacks before_parse_list before_parse_list_callbacks new parse parse parse_string parser_type= parser_type_extensions parser_type_extensions= parser_type_for_extension parser_types parser_types= register_parser_type tokenize tokenize validated_parser_type

### Constants

SHEBANG_LINE | = | /\A\s*#!\S+/
ENCODING_LINE | = | /\A(?:\s*#*!.*\r?\n)?\s*(?:#+|\/\*+|\/\/+).*coding\s*[:=]{1,2}\s*([a-z\d_\-]+)/i
ENCODING_BYTE_ORDER_MARKS | = | { 'utf-8' => "\xEF\xBB\xBF", # Not yet supported #'utf-16be' => "\xFE\xFF", #'utf-16le' => "\xFF\xFE", #'utf-32be' => "\x00\x00\xFF\xFE", #'utf-32le' => "\xFF\xFE", } |  | Byte order marks for various encodings @since 0.7.0

### Attributes

contents | [R] | @return [String] the contents of the file to be parsed @since 0.7.0
file | [R] | @return [String] the filename being parsed by the parser.

globals

[R]

@return [OpenStruct] an open struct containing arbitrary global state

      shared between files and handlers.

@since 0.7.0

parser_type | [R] | @return [Symbol] the default parser type (defaults to :ruby)

parser_type

[R]

@return [Symbol] the parser type associated with the parser instance.

      This should be set by the {#initialize constructor}.

parser_type_extensions | [R] | @return [Hash] a list of registered parser type extensions @private @since 0.5.6
parser_types | [R] | @return [Hash{Symbol=>Object}] a list of registered parser types @private @since 0.5.6

### Public Class methods

after_parse_file(&block)

Registers a callback to be called after an individual file is parsed. The block passed to this method will be called on subsequent parse calls.

To register a callback that is called after the entire list of files is processed, see {after_parse_list}.

@example Printing the length of each file after it is parsed

      SourceParser.after_parse_file do |parser|
        puts "#{parser.file} is #{parser.contents.size} characters"
      end
      YARD.parse('lib/**/*.rb')
      # prints:
      "lib/foo.rb is 1240 characters"
      "lib/foo_bar.rb is 248 characters"

@yield [parser] the yielded block is called once after each file

      that is parsed. This might happen many times for a single codebase.

@yieldparam [SourceParser] parser the parser object that parsed

      the file.

@yieldreturn [void] the return value for the block is ignored. @return [Proc] the yielded block @see before_parse_file @see after_parse_list @since 0.7.0

after_parse_file_callbacks()

@return [Array<Proc>] the list of callbacks to be called after

      parsing a file. Should only be used for testing.

@since 0.7.0

after_parse_list(&block)

Registers a callback to be called after a list of files is parsed via {parse}. The block passed to this method will be called on subsequent parse calls.

@example Printing results after parsing occurs

      SourceParser.after_parse_list do
        puts "Finished parsing!"
      end
      YARD.parse
      # Prints "Finished parsing!" after parsing files

@yield [files, globals] the yielded block is called once before

      parsing all files

@yieldparam [Array<String>] files the list of files that will be parsed. @yieldparam [OpenStruct] globals a global structure to store arbitrary

      state for post processing (see {Handlers::Processor#globals})

@yieldreturn [void] the return value for the block is ignored. @return [Proc] the yielded block @see before_parse_list @see before_parse_file @since 0.7.0

after_parse_list_callbacks()

@return [Array<Proc>] the list of callbacks to be called after

      parsing a list of files. Should only be used for testing.

@since 0.7.0

before_parse_file(&block)

Registers a callback to be called before an individual file is parsed. The block passed to this method will be called on subsequent parse calls.

To register a callback that is called before the entire list of files is processed, see {before_parse_list}.

@example Installing a simple callback

      SourceParser.before_parse_file do |parser|
        puts "I'm parsing #{parser.file}"
      end
      YARD.parse('lib/**/*.rb')
      # prints:
      "I'm parsing lib/foo.rb"
      "I'm parsing lib/foo_bar.rb"
      "I'm parsing lib/last_file.rb"

@example Cancel parsing of any test_*.rb files

      SourceParser.before_parse_file do |parser|
        return false if parser.file =~ /^test_.+\.rb$/
      end

@yield [parser] the yielded block is called once before each

      file that is parsed. This might happen many times for a single
      codebase.

@yieldparam [SourceParser] parser the parser object that will {parse}

      the file.

@yieldreturn [Boolean] if the block returns `false`, parsing for

      the file is cancelled.

@return [Proc] the yielded block @see after_parse_file @see before_parse_list @since 0.7.0

before_parse_file_callbacks()

@return [Array<Proc>] the list of callbacks to be called before

      parsing a file. Should only be used for testing.

@since 0.7.0

before_parse_list(&block)

Registers a callback to be called before a list of files is parsed via {parse}. The block passed to this method will be called on subsequent parse calls.

@example Installing a simple callback

      SourceParser.before_parse_list do |files, globals|
        puts "Starting to parse..."
      end
      YARD.parse('lib/**/*.rb')
      # prints "Starting to parse..."

@example Setting global state

      SourceParser.before_parse_list do |files, globals|
        globals.method_count = 0
      end
      SourceParser.after_parse_list do |files, globals|
        puts "Found #{globals.method_count} methods"
      end
      class MyCountHandler < Handlers::Ruby::Base
        handles :def, :defs
        process { globals.method_count += 1 }
      end
      YARD.parse
      # Prints: "Found 37 methods"

@example Using a global callback to cancel parsing

      SourceParser.before_parse_list do |files, globals|
        return false if files.include?('foo.rb')
      end

      YARD.parse(['foo.rb', 'bar.rb']) # callback cancels this method
      YARD.parse('bar.rb') # parses normally

@yield [files, globals] the yielded block is called once before

      parsing all files

@yieldparam [Array<String>] files the list of files that will be parsed. @yieldparam [OpenStruct] globals a global structure to store arbitrary

      state for post processing (see {Handlers::Processor#globals})

@yieldreturn [Boolean] if the block returns `false`, parsing is

      cancelled.

@return [Proc] the yielded block @see after_parse_list @see before_parse_file @since 0.7.0

before_parse_list_callbacks()

@return [Array<Proc>] the list of callbacks to be called before

      parsing a list of files. Should only be used for testing.

@since 0.7.0

new(parser_type = SourceParser.parser_type, load_order_errors = false, globals = nil)

Creates a new parser object for code parsing with a specific parser type.

@param [Symbol] parser_type the parser type to use @param [Boolean] load_order_errors whether or not to raise the {LoadOrderError} @param [OpenStruct] globals global state to be re-used across separate source files

parse(paths = ["{lib,app}/**/*.rb", "ext/**/*.c"], excluded = [], level = log.level)

Parses a path or set of paths

@param [String, Array<String>] paths a path, glob, or list of paths to

      parse

@param [Array<String, Regexp>] excluded a list of excluded path matchers @param [Fixnum] level the logger level to use during parsing. See

      {YARD::Logger}

@return [void]

parse_string(content, ptype = parser_type)

Parses a string `content`

@param [String] content the block of code to parse @param [Symbol] ptype the parser type to use. See {parser_type}. @return the parser object that was used to parse `content`

parser_type=(value)
parser_type_extensions()
parser_type_extensions=(value)
parser_type_for_extension(extension)

Finds a parser type that is registered for the extension. If no type is found, the default Ruby type is returned.

@return [Symbol] the parser type to be used for the extension @since 0.5.6

parser_types()
parser_types=(value)
register_parser_type(type, parser_klass, extensions = nil)

Registers a new parser type.

@example Registering a parser for "java" files

      SourceParser.register_parser_type :java, JavaParser, 'java'

@param [Symbol] type a symbolic name for the parser type @param [Base] parser_klass a class that implements parsing and tokenization @param [Array<String>, String, Regexp] extensions a list of extensions or a

      regex to match against the file extension

@return [void] @see Parser::Base

tokenize(content, ptype = parser_type)

Tokenizes but does not parse the block of code

@param [String] content the block of code to tokenize @param [Symbol] ptype the parser type to use. See {parser_type}. @return [Array] a list of tokens

validated_parser_type(type)

Returns the validated parser type. Basically, enforces that :ruby type is never set if the Ripper library is not available

@param [Symbol] type the parser type to set @return [Symbol] the validated parser type @private

### Public Instance methods

parse(content = __FILE__)

The main parser method. This should not be called directly. Instead, use the class methods {parse} and {parse_string}.

@param [String, read, Object] content the source file to parse @return [Object, nil] the parser object used to parse the source

tokenize(content)

Tokenizes but does not parse the block of code using the current {parser_type}

@param [String] content the block of code to tokenize @return [Array] a list of tokens

[Validate]