🕷 software releases

by ryan davis


ruby_parser version 3.13.1 has been released!

Published 2019-03-25 @ 17:19

ruby_parser (RP) is a ruby parser written in pure ruby (utilizing racc–which does by default use a C extension). RP’s output is the same as ParseTree’s output: s-expressions using ruby’s arrays and base types.

As an example:

def conditional1 arg1
  return 1 if arg1 == 0
  return 0


s(:defn, :conditional1, s(:args, :arg1),
    s(:call, s(:lvar, :arg1), :==, s(:lit, 0)),
    s(:return, s(:lit, 1)),
  s(:return, s(:lit, 0)))

Tested against 801,039 files from the latest of all rubygems (as of 2013-05):

  • 1.8 parser is at 99.9739% accuracy, 3.651 sigma
  • 1.9 parser is at 99.9940% accuracy, 4.013 sigma
  • 2.0 parser is at 99.9939% accuracy, 4.008 sigma


3.13.1 / 2019-03-25

  • 1 major enhancement:

    • Overhauled RubyLexer#lex_state to allow for multi-state value.
  • 5 minor enhancements:

    • Added RubyParser#current so scripts can poke around inside.
    • Cleaned up and modernized bin/ruby_parse_extract_error.
    • Removed RubyLexer#in_lex_state? in favor of State#=~ and #==.
    • Renamed RubyLexer#in_arg_state? to is_after_operator?.
    • Replaced all use of :expr_<*> symbols w/ bitfields.
  • 15 bug fixes:

    • Added tests to show #233 already fixed.
    • Fixed a lexer state bug dealing with complex edge case in hash key w/ interpolated symbol and an array value. Not a great fix. (presidentbeef)
    • Fixed a number of lex_state check problems.
    • Fixed alias and undef keyword data.
    • Fixed bug normalizing to utf-8 on tail of dstr (post-interpolation). (mvz)
    • Fixed handling of hex chars for bare unicode escape. (mvz)
    • Fixed handling of invalid string octals more gracefully. Needs generalization.
    • Fixed lexing of regexps w/ backslashed unicode. I hope.
    • Fixed parse_percent for EXPR_FITEM edge cases. Fixed bug #125.
    • Fixed parsing of op_asgn on constants (eg ::X   = 1).
    • Fixed precedence of rescue modifier on assignments. (mvz/whitequark)
    • Fixed some EXPR_FNAME edge cases.
    • Fixed using unicode in gvars. Fixed bug 202. (331299)
    • Handle invalid string literals more gracefully.
    • Minor fix lexing %s[]
  • home: https://github.com/seattlerb/ruby_parser
  • bugs: https://github.com/seattlerb/ruby_parser/issues
  • rdoc: http://docs.seattlerb.org/ruby_parser