Class: Syntax::Token

Class Syntax::Token |
In: | lib/syntax/common.rb
Parent: | String

A single token extracted by a tokenizer. It is simply the lexeme itself, decorated with a ‘group’ attribute to identify the type of the lexeme.

Methods

- new

Attributes

[R] | group | the type of the lexeme that was extracted.
[R] | instruction | the instruction associated with this token (:none, :region_open, or :region_close)

Public Class methods
new( text, group, instruction = :none )

Create a new Token representing the given text, and belonging to the given group.

[ show source ]

        # File lib/syntax/common.rb, line 19
    19:     def initialize( text, group, instruction = :none )
    20:       super text
    21:       @group = group
    22:       @instruction = instruction
    23:     end