aboutsummaryrefslogtreecommitdiffstats
path: root/ast.c
Commit message (Collapse)AuthorAgeFilesLines
* Remove duplicate to_path conversionNobuyoshi Nakada2023-11-021-1/+0
| | | | | `rb_file_open_str` calls `FilePathValue`, and the converted result is not used in this function.
* Embed `rb_args_info` in `rb_node_args_t`Nobuyoshi Nakada2023-10-301-1/+1
|
* Expand OP_ASGN1 nd_args to nd_index and nd_rvalueyui-knk2023-10-201-2/+2
| | | | | | ARGSCAT has been used for nd_args to hold index and rvalue, because there was limitation on the number of members for Node. We can easily change structure of node now, let's expand it.
* Differentiate VAR nodesyui-knk2023-10-091-2/+6
|
* Differentiate CALL nodesyui-knk2023-10-091-2/+8
|
* Differentiate ASGN nodesyui-knk2023-10-071-4/+11
|
* Pass nd_value to NODE_REQUIRED_KEYWORD_Pyui-knk2023-10-071-1/+1
|
* Remove `NODE_VALUES`Nobuyoshi Nakada2023-10-061-1/+0
| | | | | | This node type was added for the multi-value experiment back in 2004. The feature itself was removed after a few years, but this is its remnant.
* Differentiate `NODE_BREAK`/`NODE_NEXT`/`NODE_RETURN`Nobuyoshi Nakada2023-10-051-1/+3
|
* Move internal NODE_DEF_TEMP to parse.yNobuyoshi Nakada2023-10-051-1/+0
|
* Use rb_node_args_t and rb_node_args_aux_t instead of NODEyui-knk2023-10-011-1/+1
|
* Use rb_node_opt_arg_t and rb_node_kw_arg_t instead of NODEyui-knk2023-10-011-2/+2
|
* Expand pattern_info struct into ARYPTN Node and FNDPTN Nodeyui-knk2023-09-301-8/+6
|
* Merge NODE_DEF_TEMP and NODE_DEF_TEMP2yui-knk2023-09-291-1/+0
|
* Merge RNode_OP_ASGN2 and RNode_OP_ASGN22yui-knk2023-09-291-6/+3
|
* Change RNode structure from union to structyui-knk2023-09-281-109/+116
| | | | | | | | | | | | | | | | | | | | | | | All kind of AST nodes use same struct RNode, which has u1, u2, u3 union members for holding different kind of data. This has two problems. 1. Low flexibility of data structure Some nodes, for example NODE_TRUE, don’t use u1, u2, u3. On the other hand, NODE_OP_ASGN2 needs more than three union members. However they use same structure definition, need to allocate three union members for NODE_TRUE and need to separate NODE_OP_ASGN2 into another node. This change removes the restriction so make it possible to change data structure by each node type. 2. No compile time check for union member access It’s developer’s responsibility for using correct member for each node type when it’s union. This change clarifies which node has which type of fields and enables compile time check. This commit also changes node_buffer_elem_struct buf management to handle different size data with alignment.
* Move SCRIPT_LINES__ away from parse.yNobuyoshi Nakada2023-08-251-7/+16
|
* [Feature #19719] Universal Parseryui-knk2023-06-121-1/+1
| | | | | | | | | Introduce Universal Parser mode for the parser. This commit includes these changes: * Introduce `UNIVERSAL_PARSER` macro. All of CRuby related functions are passed via `struct rb_parser_config_struct` when this macro is enabled. * Add CI task with 'cppflags=-DUNIVERSAL_PARSER' for ubuntu.
* Rename `rb_node_name` to the original nameyui-knk2023-05-241-2/+2
| | | | | | 98637d421dbe8bcf86cc2effae5e26bb96a6a4da changes the name of the function. However this function is exported as global, then change the name to origin one for keeping compatibility.
* Move `ruby_node_name` to node.c and rename prefix of the functionyui-knk2023-05-231-2/+2
|
* Add utility macros `DECIMAL_SIZE_OF` and `DECIMAL_SIZE_OF_BYTES`Nobuyoshi Nakada2023-02-141-1/+1
|
* Check if the argument is Thread::Backtrace::Location objectyui-knk2023-01-061-0/+5
| | | | [Bug #19262]
* Enhance keep_tokens option for RubyVM::AbstractSyntaxTree parsing methodsyui-knk2022-11-211-13/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Implementation for Language Server Protocol (LSP) sometimes needs token information. For example both `m(1)` and `m(1, )` has same AST structure other than node locations then it's impossible to check the existence of `,` from AST. However in later case, it might be better to suggest variables list for the second argument. Token information is important for such case. This commit adds these methods. * Add `keep_tokens` option for `RubyVM::AbstractSyntaxTree.parse`, `.parse_file` and `.of` * Add `RubyVM::AbstractSyntaxTree::Node#tokens` which returns tokens for the node including tokens for descendants nodes. * Add `RubyVM::AbstractSyntaxTree::Node#all_tokens` which returns all tokens for the input script regardless the receiver node. [Feature #19070] Impacts on memory usage and performance are below: Memory usage: ``` $ cat test.rb root = RubyVM::AbstractSyntaxTree.parse_file(File.expand_path('../test/ruby/test_keyword.rb', __FILE__), keep_tokens: true) $ /usr/bin/time -f %Mkb /usr/local/bin/ruby -v ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] 11408kb # keep_tokens :false $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 17508kb # keep_tokens :true $ /usr/bin/time -f %Mkb /usr/local/bin/ruby test.rb 30960kb ``` Performance: ``` $ cat ../ast_keep_tokens.yml prelude: | src = <<~SRC module M class C def m1(a, b) 1 + a + b end end end SRC benchmark: without_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: false) with_keep_tokens: | RubyVM::AbstractSyntaxTree.parse(src, keep_tokens: true) $ make benchmark COMPARE_RUBY="./ruby" ARGS=../ast_keep_tokens.yml /home/kaneko.y/.rbenv/shims/ruby --disable=gems -rrubygems -I../benchmark/lib ../benchmark/benchmark-driver/exe/benchmark-driver \ --executables="compare-ruby::./ruby -I.ext/common --disable-gem" \ --executables="built-ruby::./miniruby -I../lib -I. -I.ext/common ../tool/runruby.rb --extout=.ext -- --disable-gems --disable-gem" \ --output=markdown --output-compare -v ../ast_keep_tokens.yml compare-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] built-ruby: ruby 3.2.0dev (2022-11-19T09:41:54Z 19070-keep_tokens d3af1b8057) [x86_64-linux] warming up.. | |compare-ruby|built-ruby| |:--------------------|-----------:|---------:| |without_keep_tokens | 21.659k| 21.303k| | | 1.02x| -| |with_keep_tokens | 6.220k| 5.691k| | | 1.09x| -| ```
* Add `node_id_for_backtrace_location` functioneileencodes2022-10-311-0/+12
| | | | | | | | | | | | | | | We want to use error highlight with eval'd code, specifically ERB templates. We're able to recover the generated code for eval'd templates and can get a parse tree for the ERB generated code, but we don't have a way to get the node id from the backtrace location. So we can't pass the right node into error highlight. This patch gives us an API to get the node id from the backtrace location so we can find the node in the AST. Error Highlight PR: https://github.com/ruby/error_highlight/pull/26 Co-authored-by: Aaron Patterson <tenderlove@ruby-lang.org>
* Move `error` from top_stmts and top_stmt to stmtyui-knk2022-10-081-0/+2
| | | | | | | | | | | | | | | | | | | By this change, syntax error is recovered smaller units. In the case below, "DEFN :bar" is same level with "CLASS :Foo" now. ``` module Z class Foo foo. end def bar end end ``` [Feature #19013]
* Add error_tolerant option to RubyVM::ASTyui-knk2022-10-081-13/+16
| | | | | | | If this option is enabled, SyntaxError is not raised and Node is returned even if passed script is broken. [Feature #19013]
* Add ISEQ_BODY macroPeter Zhu2022-03-241-2/+2
| | | | | | Use ISEQ_BODY macro to get the rb_iseq_constant_body of the ISeq. Using this macro will make it easier for us to change the allocation strategy of rb_iseq_constant_body when using Variable Width Allocation.
* Make RubyVM::AST.of work with code written in `-e` command-line optionYusuke Endoh2021-12-261-4/+7
| | | | [Bug #18434]
* Make AST.of possible even under eval when keep_script_lines is enabledYusuke Endoh2021-12-191-2/+2
| | | | | | | | | | | | | | | Now the following code works without an exception. ``` RubyVM.keep_script_lines = true eval(<<END) def foo end END p RubyVM::AbstractSyntaxTree.of(method(:foo)) ```
* Make RubyVM::AbstractSyntaxTree.of raise for backtrace location in evalYusuke Endoh2021-12-191-11/+15
| | | | | | | | | This check is needed to fix a bug of error_highlight when NameError occurred in eval'ed code. https://github.com/ruby/error_highlight/pull/16 The same check for proc/method has been already introduced since 64ac984129a7a4645efe5ac57c168ef880b479b2.
* Remove `NODE_DASGN_CURR` [Feature #18406]Nobuyoshi Nakada2021-12-131-1/+0
| | | | | | | This `NODE` type was used in pre-YARV implementation, to improve the performance of assignment to dynamic local variable defined at the innermost scope. It has no longer any actual difference with `NODE_DASGN`, except for the node dump.
* Add `nd_type_p` macroS.H2021-12-041-3/+3
|
* Refactor hacky ID tables to struct rb_ast_id_table_tYusuke Endoh2021-11-211-3/+3
| | | | | | | | | The implementation of a local variable tables was represented as `ID*`, but it was very hacky: the first element is not an ID but the size of the table, and, the last element is (sometimes) a link to the next local table only when the id tables are a linked list. This change converts the hacky implementation to a normal struct.
* ast.c: Use kept script_lines data instead of re-opening the source file (#5019)Yusuke Endoh2021-10-261-4/+5
| | | ast.c: Use kept script_lines data instead of re-open the source file
* ast.c: AST.of against C method should return nil (as Ruby 2.6--3.0)Yusuke Endoh2021-09-181-1/+1
|
* ast.c: AST.of checks if a given method object is defined in CYusuke Endoh2021-09-181-0/+3
| | | | [Bug #18178]
* Replace RBOOL macroS-H-GAMELINKS2021-09-051-2/+2
|
* ast.c: Rename "save_script_lines" to "keep_script_lines"Yusuke Endoh2021-08-201-16/+16
| | | | | | | ... as per ko1's preference. He is preparing to extend this feature to ISeq for his new debugger. He prefers "keep" to "save" for this wording. This API is internal and not included in any released version, so I change it in advance.
* Make RubyVM::AbstractSyntaxTree.of raise for method/proc created in evalJeremy Evans2021-07-291-0/+3
| | | | | | | | | | | This changes Thread::Location::Backtrace#absolute_path to return nil for methods/procs defined in eval. If the realpath of an iseq is nil, that indicates it was defined in eval, in which case you cannot use RubyVM::AbstractSyntaxTree.of. Fixes [Bug #16983] Co-authored-by: Koichi Sasada <ko1@atdot.net>
* Experimentally expose RubyVM::AST::Node#node_idYusuke Endoh2021-06-211-6/+1
| | | | | | Now ISeq#to_a includes the node_id list for each bytecode instruction. I want a way to retrieve the AST::Node instance corresponding to an instruction for a research purpose including TypeProf-based LSP server.
* Enable USE_ISEQ_NODE_ID by defaultYusuke Endoh2021-06-181-2/+2
| | | | | | | | ... which is formally called EXPERIMENTAL_ISEQ_NODE_ID. See also ff69ef27b06eed1ba750e7d9cab8322f351ed245. https://bugs.ruby-lang.org/issues/17930
* Make it possible to get AST::Node from Thread::Backtrace::LocationYusuke Endoh2021-06-181-12/+15
| | | | | | | | | RubyVM::AST.of(Thread::Backtrace::Location) returns a node that corresponds to the location. Typically, the node is a method call, but not always. This change also includes iseq's dump/load support of node_ids for each instructions.
* node.h: Reduce struct size to fit with Ruby object size (five VALUEs)Yusuke Endoh2021-06-181-1/+1
| | | | | | | | by merging `rb_ast_body_t#line_count` and `#script_lines`. Fortunately `line_count == RARRAY_LEN(script_lines)` was always satisfied. When script_lines is saved, it has an array of lines, and when not saved, it has a Fixnum that represents the old line_count.
* ast.rb: RubyVM::AST.parse and .of accepts `save_script_lines: true`Yusuke Endoh2021-06-181-16/+32
| | | | | | | This option makes the parser keep the original source as an array of the original code lines. This feature exploits the mechanism of `SCRIPT_LINES__` but records only the specified code that is passed to RubyVM::AST.of or .parse, instead of recording all parsed program texts.
* compile.c: Pass node instead of nd_line(node) to ADD_INSN* functionsYusuke Endoh2021-05-071-0/+14
| | | | | | | | | | | | | | | ... then, new_insn_core extracts nd_line(node). Also, if a macro "EXPERIMENTAL_ISEQ_NODE_ID" is defined, this changeset keeps nd_node_id(node) for each instruction. This is intended for TypeProf to identify what AST::Node corresponds to each instruction. This patch is originally authored by @yui-knk for showing which column a NoMethodError occurred. https://github.com/ruby/ruby/compare/master...yui-knk:feature/node_id Co-Authored-By: Yuichiro Kaneko <yui-knk@ruby-lang.org>
* Remove unused rb_ast_parse_array declarationS.H2021-03-201-1/+0
|
* Unfreeze string-literal-only interpolated string-literalNobuyoshi Nakada2020-09-301-3/+9
| | | | [Feature #17104]
* Hoisted out functions for no name rest argument symbolNobuyoshi Nakada2020-07-081-8/+19
|
* Constified NODE pointer in ASTNodeDataNobuyoshi Nakada2020-07-081-7/+7
|
* Added `NODE_SPECIAL_EXCESSIVE_COMMA` info to `ARGS` of ↵manga_osyo2020-07-081-1/+3
| | | | `RubyVM::AbstractSyntaxTree`.
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-29 22:23:42 +0000