aboutsummaryrefslogtreecommitdiffstats
path: root/ast.rb
Commit message (Collapse)AuthorAgeFilesLines
* ast.rb: Fix bug for source of multibyte charactersalpaca-tc2023-11-051-2/+2
| | | | | first_column and last_column return byte positions, but existing implementations did not consider multibyte.
* Move SCRIPT_LINES__ away from parse.yNobuyoshi Nakada2023-08-251-7/+7
|
* Fix spelling (#7389)John Bampton2023-02-271-1/+1
|
* [DOC] Document AST.parse's keyword argumentszverok2022-12-241-6/+36
|
* [DOC] Fix AST documentsNobuyoshi Nakada2022-11-251-10/+11
| | | | | | - Fix indents of `tokens`, to make the contents of Token a list - Move the example of `tokens` to separate from the above list - Markup keyword argument and method name
* Enhance keep_tokens option for RubyVM::AbstractSyntaxTree parsing methodsyui-knk2022-11-211-6/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implementation for Language Server Protocol (LSP) sometimes needs token information. For example both `m(1)` and `m(1, )` has same AST structure other than node locations then it's impossible to check the existence of `,` from AST. However in later case, it might be better to suggest variables list for the second argument. Token information is important for such case. This commit adds these methods. * Add `keep_tokens` option for `RubyVM::AbstractSyntaxTree.parse`, `.parse_file` and `.of` * Add `RubyVM::AbstractSyntaxTree::Node#tokens` which returns tokens for the node including tokens for descendants nodes. * Add `RubyVM::AbstractSyntaxTree::Node#all_tokens` which returns all tokens for the input script regardless the receiver node. [Feature #19070] Impacts on memory usage and performance are below: Memory usage: ``` $ cat test.rb root = RubyVM::AbstractSyntaxTree.parse_file(File.expand_path('../test/ruby/test_keyword.rb', __FILE__), keep_tokens: true) $ /usr/bin/time -f %Mkb /usr/local/bin/ruby -v ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] 11408kb # keep_tokens :false $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 17508kb # keep_tokens :true $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 30960kb ``` Performance: ``` $ cat ../ast_keep_tokens.yml prelude: | src = <<~SRC module M class C def m1(a, b) 1 + a + b end end end SRC benchmark: without_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: false) with_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: true) $ make benchmark COMPARE_RUBY="./ruby" ARGS=../ast_keep_tokens.yml /home/kaneko.y/.rbenv/shims/ruby --disable=gems -rrubygems -I../benchmark/lib ../benchmark/benchmark-driver/exe/benchmark-driver \ --executables="compare-ruby::./ruby -I.ext/common --disable-gem" \ --executables="built-ruby::./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems --disable-gem" \ --output=markdown --output-compare -v ../ast_keep_tokens.yml compare-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] built-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] warming up.. | |compare-ruby|built-ruby| |:--------------------|-----------:|---------:| |without_keep_tokens | 21.659k| 21.303k| | | 1.02x| -| |with_keep_tokens | 6.220k| 5.691k| | | 1.09x| -| ```
* Add `node_id_for_backtrace_location` functioneileencodes2022-10-311-0/+15
| | | | | | | | | | | | | | | We want to use error highlight with eval'd code, specifically ERB templates. We're able to recover the generated code for eval'd templates and can get a parse tree for the ERB generated code, but we don't have a way to get the node id from the backtrace location. So we can't pass the right node into error highlight. This patch gives us an API to get the node id from the backtrace location so we can find the node in the AST. Error Highlight PR: https://github.com/ruby/error_highlight/pull/26 Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
* Add error_tolerant option to RubyVM::ASTyui-knk2022-10-081-6/+6
| | | | | | | If this option is enabled, SyntaxError is not raised and Node is returned even if passed script is broken. [Feature #19013]
* ast.c: Rename "save_script_lines" to "keep_script_lines"Yusuke Endoh2021-08-201-6/+6
| | | | | | | ... as per ko1's preference. He is preparing to extend this feature to ISeq for his new debugger. He prefers "keep" to "save" for this wording. This API is internal and not included in any released version, so I change it in advance.
* Experimentally expose RubyVM::AST::Node#node_idYusuke Endoh2021-06-211-0/+11
| | | | | | Now ISeq#to_a includes the node_id list for each bytecode instruction. I want a way to retrieve the AST::Node instance corresponding to an instruction for a research purpose including TypeProf-based LSP server.
* ast.rb: RubyVM::AST.parse and .of accepts `save_script_lines: true`Yusuke Endoh2021-06-181-6/+42
| | | | | | | This option makes the parser keep the original source as an array of the original code lines. This feature exploits the mechanism of `SCRIPT_LINES__` but records only the specified code that is passed to RubyVM::AST.of or .parse, instead of recording all parsed program texts.
* ast.rb: reduce nestingNobuyoshi Nakada2021-06-061-117/+114
|
* Fix RubyVM::AbstractSyntaxTree documentMasataka Pocke Kuwabara2020-06-241-2/+2
| | | | RubyVM::AbstractSyntaxTree is a module actually, but the document says "class".
* [Feature #16254] Use `Primitive.func` styleNobuyoshi Nakada2020-06-191-10/+10
|
* [Feature #16254] Use `__builtin.func` styleNobuyoshi Nakada2020-06-191-10/+10
|
* Fix variable name and add more example [ci skip]Kazuhiro NISHIYAMA2020-01-031-2/+4
|
* Fix example of node.type [ci skip]Kazuhiro NISHIYAMA2020-01-031-1/+1
| | | | | | | | | | | | | | | ``` % docker run -it --rm rubylang/all-ruby env ALL_RUBY_SINCE=ruby-2.6 ./all-ruby -e 'root = RubyVM::AbstractSyntaxTree.parse("x = 1 + 2") p [root, root.type] call = root.children[2] p [call, call.type] ' ruby-2.6.0 [#<RubyVM::AbstractSyntaxTree::Node:SCOPE@1:0-1:9>, :SCOPE] [#<RubyVM::AbstractSyntaxTree::Node:LASGN@1:0-1:9>, :LASGN] ... ruby-2.7.0 [#<RubyVM::AbstractSyntaxTree::Node:SCOPE@1:0-1:9>, :SCOPE] [#<RubyVM::AbstractSyntaxTree::Node:LASGN@1:0-1:9>, :LASGN] ```
* Improve highlighting in RubyVM::AbstractSyntaxTree docs when using `ri`Benoit Daloze2019-12-141-1/+1
|
* Clarify in the documentation that RubyVM::AbstractSyntaxTree is not stable APIBenoit Daloze2019-12-141-0/+10
| | | | * See [Feature #14844].
* fix line break code (fix to LF)Koichi Sasada2019-11-081-134/+134
|
* use builtin for RubyVM::AbstractSyntaxTree.Koichi Sasada2019-11-081-0/+134
Define RubyVM::AbstractSyntaxTree in ast.rb with __builtin functions.
generated by cgit v1.2.3 (git 2.39.1) at 2024-04-27 13:34:18 +0000