Extended maintenance of Ruby 1.9.3 ended on February 23, 2015. Read more
Collection of methods for writing parsers against RDoc::RubyLex and RDoc::RubyToken
Adds a token listener obj
, but you should probably use #token_listener
# File rdoc/parser/ruby_tools.rb, line 12 def add_token_listener(obj) @token_listeners ||= [] @token_listeners << obj end
Fetches the next token from the scanner
# File rdoc/parser/ruby_tools.rb, line 20 def get_tk tk = nil if @tokens.empty? then tk = @scanner.token @read.push @scanner.get_readed puts "get_tk1 => #{tk.inspect}" if $TOKEN_DEBUG else @read.push @unget_read.shift tk = @tokens.shift puts "get_tk2 => #{tk.inspect}" if $TOKEN_DEBUG end tk = nil if TkEND_OF_SCRIPT === tk if TkSYMBEG === tk then set_token_position tk.line_no, tk.char_no case tk1 = get_tk when TkId, TkOp, TkSTRING, TkDSTRING, TkSTAR, TkAMPER then if tk1.respond_to?(:name) then tk = Token(TkSYMBOL).set_text(":" + tk1.name) else tk = Token(TkSYMBOL).set_text(":" + tk1.text) end # remove the identifier we just read (we're about to replace it with a # symbol) @token_listeners.each do |obj| obj.pop_token end if @token_listeners else tk = tk1 end end # inform any listeners of our shiny new token @token_listeners.each do |obj| obj.add_token(tk) end if @token_listeners tk end
Reads and returns all tokens up to one of tokens
. Leaves the
matched token in the token list.
# File rdoc/parser/ruby_tools.rb, line 68 def get_tk_until(*tokens) read = [] loop do tk = get_tk case tk when *tokens then unget_tk tk; break end read << tk end read end
Retrieves a String representation of the read tokens
# File rdoc/parser/ruby_tools.rb, line 83 def get_tkread read = @read.join("") @read = [] read end
Peek equivalent for #get_tkread
# File rdoc/parser/ruby_tools.rb, line 92 def peek_read @read.join('') end
Peek at the next token, but don’t remove it from the stream
# File rdoc/parser/ruby_tools.rb, line 99 def peek_tk unget_tk(tk = get_tk) tk end
Removes the token listener obj
# File rdoc/parser/ruby_tools.rb, line 107 def remove_token_listener(obj) @token_listeners.delete(obj) end
Resets the tools
# File rdoc/parser/ruby_tools.rb, line 114 def reset @read = [] @tokens = [] @unget_read = [] @nest = 0 end
Skips whitespace tokens including newlines if skip_nl
is true
# File rdoc/parser/ruby_tools.rb, line 124 def skip_tkspace(skip_nl = true) # HACK dup tokens = [] while TkSPACE === (tk = get_tk) or (skip_nl and TkNL === tk) do tokens.push tk end unget_tk tk tokens end
Commenting is here to help enhance the documentation. For example, code samples, or clarification of the documentation.
If you have questions about Ruby or the documentation, please post to one of the Ruby mailing lists. You will get better, faster, help that way.
If you wish to post a correction of the docs, please do so, but also file bug report so that it can be corrected for the next release. Thank you.
If you want to help improve the Ruby documentation, please visit Documenting-ruby.org.