aboutsummaryrefslogtreecommitdiffstats
path: root/parse.y
Commit message (Collapse)AuthorAgeFilesLines
* parse.y: Fix location of asgn nodeyui-knk2017-11-161-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | * parse.y (new_op_assign_gen): Fix location of asgn node. Assignable node (e.g. NODE_LASGN) is generated before rhs is generated, so we reset the location when nd_value of asgn is fixed. e.g. : ``` a -= 10 ``` * Before ``` NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 1) ``` * After ``` NODE_DASGN_CURR (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 7) ``` git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60781 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Preserve previous line and restore it when read '\n'yui-knk2017-11-161-3/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * parse.y (parser_params): Add prevline to store previous line. * parse.y (yycompile0): Initialize prevline with 0. * parse.y (parser_nextline): Store previous line on prevline. * parse.y (parser_nextc): Check parser is on EOF or has nextline. Now parser_yylex does not always set lex_p as lex_pend, we should check EOF flag and nextline is set. * parse.y (parser_yylex): Restore previous line, set lex_p and tokp on '\n'. Before this commit, tokp is on the head of next line of '\n' and lex_p is on the tail of next line when next token is '\n'. By this behavior, in some case the last column of NODE_CALL (or NODE_QCALL) is set to the last column of next line. NODE_CALL can be generated via `primary_value call_op operation2 {} opt_paren_args` and opt_paren_args can be none. If none is generated with next token '\n', the last column of none is set to the last column of next line. e.g. : ``` a.b cd.ef ``` The location of NODE_CALL of first line is set to, * Before ``` NODE_CALL (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 6) ``` * After ``` NODE_CALL (line: 1, first_lineno: 1, first_column: 0, last_lineno: 1, last_column: 3) ``` * parse.y (parser_mark): GC mark prevline. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60780 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: zero codepointsnobu2017-11-141-2/+2
| | | | | | | | | * parse.y (parser_tokadd_utf8): relax restriction to allow zero or more codepoints. fixup r59417. https://github.com/ruby/ruby/commit/7e8b910#commitcomment-25602670 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60766 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Initialize last column with -1 and lineno with 0yui-knk2017-11-131-0/+2
| | | | | | | | * parse.y (node_newnode): Initialize last column of nodes with -1 and lineno with 0 to make it easy to detect nodes which we forget to set a column number or lineno. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60751 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Store last location of a node on RNodeyui-knk2017-11-131-531/+396
| | | | | | | | | | | | | | | | | | | * node.c (rb_node_init): Initialize last location with 0. * node.h (struct rb_code_range_struct): Define a structure which contains first location and last location of a node. * node.h (struct RNode): Use rb_code_range_t to store last location of a node. * node.h (nd_column, nd_set_column, nd_lineno, nd_set_lineno): Follow-up the change of struct RNode. * node.h (nd_last_column, nd_set_last_column, nd_last_lineno, nd_set_last_lineno): Define getter/setter macros for last location of RNode. * parse.y : Set last location of tokens. Thanks to Yusuke Endoh (mame) for design of data structures. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60750 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: constified YYLTYPEnobu2017-11-111-119/+119
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60742 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* use `rb_ast_dispose` instead of `rb_ast_free`mame2017-11-091-1/+1
| | | | | | | `rb_ast_dispose` calls `rb_gc_writebarrier_remember`. Unless we call it, the marked objects may not be GC'ed until `rb_ast_t` is GC'ed. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60722 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: fix r60644nobu2017-11-081-5/+2
| | | | | | | | | * parse.y (rb_strterm_heredoc_t): should not define sourceline as union, to fix wrong read on big-endian platforms. as rb_imemo_new stores all members as VALUEs, should read it as VALUE too. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60720 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: remove in_singlenobu2017-11-081-15/+11
| | | | | | | * parse.y (parser_params): removed in_single, which is referenced with in_def always. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60716 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Omit first argument of NEW_CASE2yui-knk2017-11-081-1/+1
| | | | | | | | * node.h (NEW_CASE2): Omit first argument of NEW_CASE2, bacause the first argument of NEW_CASE2 is always 0. * parse.y: Ditto git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60714 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* lexer.rb: Ripper::Lexer::Statenobu2017-11-051-1/+1
| | | | | | | | | | * ext/ripper/lib/ripper/lexer.rb (Ripper::Lexer::State): wrapper of lex_state values. * parse.y (rb_parser_lex_state_name): return shared strings. lex state combinations are very restricted. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60665 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Revert "Introduce rb_code_range_t and replace YYLTYPE with it"mame2017-11-051-660/+794
| | | | | | | | | | r60655 changed the usage of bison by defining a customized YYLTYPE, which seemed to cause a random stall at rb_thread_terminate_all. Kazuki Tsujimoto investigated the issue and pointed out the commit that caused it. Thanks! git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60660 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Introduce rb_code_range_t and replace YYLTYPE with itmame2017-11-041-794/+660
| | | | | | | | | | | rb_code_range_t has two t_code_location_t, i.e., the first and last locations. This is used for YYLTYPE, tracked locations of bison, and will be also used for representing the "range", the first and the last locations of each NODE. Currently, each NODE keeps only the first location, though. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60655 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: fix duplicate symbolnobu2017-11-041-0/+2
| | | | | | | * parse.y (rb_strterm_mark): do not define in ripper to get rid of duplicate symbol with static-linked-ext. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60654 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Avoid usage of the magic number `(NODE*)-1`mame2017-11-041-11/+11
| | | | | | | | | | | | | This magic number has two meanings depending upon the context: * "required keyword argument (no name)" on NODE_LASGN (`def foo(x:)`) * "rest argument (no name)" on NODE_MASGN and NODE_POSTARG ('a, b, * = ary` or `a, b, *, z = ary`) To show this intention explicitly, two macros are introduced: NODE_SPECIAL_REQUIRED_KEYWORD and NODE_SPECIAL_NO_NAME_REST. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60650 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_args_tail_gen): returns a NODE instead of imemomame2017-11-041-3/+5
| | | | | | | `new_args_tail_gen` returned imemo, but the value was later accessed as `NODE*`. This prevented change of NODE structure. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60647 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: suppress warning in rippernobu2017-11-041-0/+2
| | | | | | | * parse.y (rb_discard_node_gen): rb_discard_node() is not used in ripper right now. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60645 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: suppress warningsnobu2017-11-041-11/+8
| | | | | | | | | | | * parse.y (parser_set_line): removed no longer used function. * parse.y (rb_strterm_heredoc_t): adjust type of sourceline to ruby_sourceline. * parse.y (rb_strterm_t): get rid of redefinition. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60644 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Rename imemo_strterm to imemo_parser_strtermmame2017-11-041-3/+3
| | | | | | Per ko1's request. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60642 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Make Ripper use NODE buffermame2017-11-041-20/+27
| | | | | | | | | | | | | | This is a follow-up of r60488. Not only the Ruby parser but also Ripper now use NODE buffer instead of NODE objects managed by GC. Now we can change the struct of NODEs so that it can keep detailed location information, perhaps (not tried yet). Note that, however, the first word of each NODE must be still compatible with RBasic structure. This is because Ripper handles NODEs and other objects uniformly; especially, it still uses `RB_TYPE_P(obj, T_NODE)` for distinguishing between NODEs and other objects. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60639 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Fix the alignment bug of r60634 for LLP64mame2017-11-041-16/+31
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60638 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Revert "Revert "Replace NODE_STRTERM and NODE_HEREDOC with imemo_strterm""mame2017-11-041-52/+79
| | | | | | Retry r60634 git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60637 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Revert "Replace NODE_STRTERM and NODE_HEREDOC with imemo_strterm"mame2017-11-041-79/+52
| | | | | | Due to build failure on mswin and mingw. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60635 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Replace NODE_STRTERM and NODE_HEREDOC with imemo_strtermmame2017-11-041-52/+79
| | | | | | | | | | | Just refactoring. NODE_STRTERM and NODE_HEREDOC are not an internal node of AST, but a temporary storage for managing termination of string literals and heredocs. Instead of NODE abuse, I want to use imemo for the storage in order to avoid (my) confusion. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60634 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Remove unneeded complexitymame2017-10-311-4/+0
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60587 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Use NODE_CASE2 if case expressions don't existyui-knk2017-10-311-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When NODE_WHEN is compiled by iseq_compile_each0, the node passed to compile_when is NODE_WHEN (not NODE_CASE). So we can not handle the location of NODE_CASE of case statements which don't have case expressions. e.g. : ``` case; when 1; foo; when 2; bar; else baz; end ``` This commit adds NODE_CASE2, and compiles it by iseq_compile_each0. * compile.c (compile_case): Does not call COMPILE_ when NODE_CASE does not have case expressions. * compile.c (compile_case2): Compile NODE_CASE2 by compile_case2. * compile.c (compile_when): Delete an obsoleted function. * compile.c (iseq_compile_each0): Compile NODE_CASE2. * ext/objspace/objspace.c (count_nodes): Add NODE_CASE2 case. * node.c (dump_node, rb_gc_mark_node): Add NODE_CASE2 case. * node.h (node_type): Add NODE_CASE2. * node.h (NEW_CASE2): Add a macro which generates NODE_CASE2. * parse.y: Generate NODE_CASE2 if case expressions don't exist. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60585 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Store lineno on RNodeyui-knk2017-10-301-582/+741
| | | | | | | | | | | | | | | * node.c (A_NODE_HEADER): Print lineno. * node.h (nd_column, nd_set_column): Store column number with 16-bit. * node.h (nd_lineno, nd_set_lineno): Define getter/setter macros for lineno of RNode. * parse.y : Set first lineno of tokens. Thanks to takeshinoda for review. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60580 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Remove duplicated column setyui-knk2017-10-291-1/+0
| | | | | | | * parse.y : Column of resq is set by new_resbody, so this line is needless. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60576 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_regexp_gen): Set column for NODE_SCOPEyui-knk2017-10-291-0/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60575 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * node.h (ast_t): renamed to `rb_ast_t`.ko12017-10-291-12/+12
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60565 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: Need to mark the third element of `NODE_HEREDOC`mame2017-10-271-0/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60489 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Revert "Revert "Manage AST NODEs out of GC""mame2017-10-271-25/+55
| | | | | | | This re-introduces r60485. This reverts commit 5a176b75b1187cbd3861c387bde65ff66396a07c. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60488 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Revert "Manage AST NODEs out of GC"mame2017-10-271-55/+25
| | | | | | This reverts commit 620ba74778bfdbdc34ffbb142d49ce84a0ef58e9. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60486 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Manage AST NODEs out of GCmame2017-10-271-25/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | NODEs in AST are no longer objects managed by GC. This change will remove the restriction imposed by the GC. For example, a NODE can use more than five words (this is my primary purpose; we want to store the position data for each NODE, for coverage library), or even a NODE can have variable length (some kinds of NODEs have unused fields). To do this, however, we need more work, since Ripper still uses T_NODE objects managed by the GC. The life time of NODEs is more obvious than other kinds of objects; they are created at parsing, and they become disused immediately after compilation. This change releases all NODEs by a few `xfree`s after compilation, so performance will be improved a bit. In extreme example, `eval("x=1;" * 10000000)` runs much faster (40 sec. -> 7.8 sec. on my machine). The most important part of this change is `ast_t` struct, which has three contents: (1) NODE buffer (malloc'ed memory), (2) a reference to the root NODE, and (3) an array that contains objects that must be marked during parsing (such as literal objects). Some functions that had received `NODE*` arguments, must now receive `ast_t*`. * node.c, node.h: defines `ast_t` struct and related operations. * gc.c, internal.h: defines `imemo_ast`. * parse.y: makes `parser_params` struct have a reference to `ast_t`. Instead of `rb_node_newnode`, use `rb_ast_newnode` to create a NODE. * iseq.c, load.c, ruby.c, template/prelude.c.tmpl: modifies some functions to handle `ast_t*` instead of `NODE*`. * test/ruby/test_gc.rb: ad-hoc fix for a failed test. The test assumes GC eden is increased at startup by NODE object creation. However, this change now create no NODE object, so GC eden is not necessarily increased. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60485 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Add explicit markers for literal objects generated in parse.ymame2017-10-271-15/+36
| | | | | | | This is just a preparation to manage AST NODEs out of GC. Currently `add_mark_object` does nothing. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60481 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Refactoring by adding `rb_imemo_alloc_new` to create imemo_alloc buffermame2017-10-251-4/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60429 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Make imemo_alloc writebarrier-unprotectedmame2017-10-251-0/+3
| | | | | | | | | imemo_alloc provides a memory buffer whose contents are marked by GC. C code can access imemo_alloc buffer freely, so imemo_alloc must be considered writebarrier-unprotected. But T_IMEMO is writebarrier- protected by default, which caused a GC bug. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60427 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Removed NODE_OPT_Nnobu2017-10-241-1/+1
| | | | | | | | | * node.h (NODE_OPT_N): removed. * parse.y (parser_append_options): expand -n option loop to while gets loop. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60394 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: rb_parser_set_optionsnobu2017-10-241-63/+36
| | | | | | | | | | * parse.y (yycompile0): append top-level addenda before appending prelude nodes. * parse.y (rb_parser_set_options): set top-level addendum options before parsing. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60393 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Avoid many type castsmame2017-10-241-10/+16
| | | | | | | | The idiom `rb_gc_force_recycle((VALUE)...)` was used heavily. This change defines `rb_discard_nod(NODE*)` and allows C compiler to typecheck. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60391 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Remove special handling for (NODE*)1mame2017-10-241-1/+0
| | | | | | | | The magic number was used to mark NODE_ATTRASGN when its receiver is self. But the hack was refactored at r46366. So the remaining code fragments are no longer needed. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60389 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y (new_scope_gen): adjust indentyui-knk2017-10-221-3/+3
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60350 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* parse.y: workaround for warningsnobu2017-10-221-0/+34
| | | | | | | * parse.y (mark_lvar_used): enable workaround to suppress unused local variables. [ruby-core:82656] [Bug #13872] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60339 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* fix up r60224nobu2017-10-221-13/+13
| | | | | | * parse.y (match_op_gen): fix optimization at DREGX_ONCE. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60338 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Remove not used node_type NODE_IASGN2yui-knk2017-10-211-1/+0
| | | | | | | | | | | * compile.c (compile_massign_opt, iseq_compile_each0): This node_type has not been used since r11813. * ext/objspace/objspace.c (count_nodes): ditto * node.c (dump_node, rb_gc_mark_node): ditto * node.h (node_type, NEW_IASGN2): ditto * parse.y (node_assign_gen): ditto git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60302 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Do not pass a not used argumentyui-knk2017-10-211-3/+3
| | | | | | | * parse.y (new_op_assign_gen): new_op_assign_gen for ripper does not need column information. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60289 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Define the macro separately for ruby parser and for ripperyui-knk2017-10-211-2/+2
| | | | | | | * parse.y (new_op_assign): Define the macro separately for ruby parser and for ripper. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60286 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* fix up imemo_allocnobu2017-10-211-1/+1
| | | | | | | | | * internal.h (rb_imemo_alloc_struct), gc.c (gc_mark_imemo): turned next into the pointer to chain. * parse.y (NEWHEAP): needs a cast. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60244 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Replace NODE_ALLOCA with T_IMEMO (imemo_alloc)mame2017-10-211-15/+15
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60239 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Do not literal_flush if float is read correctlyyui-knk2017-10-211-1/+0
| | | | | | | | | | | | | | | | | | | | * parse.y (parse_numeric): Do not literal_flush if float is read correctly. This will fix the first column of float. before: ``` 1.2i1.1 ^ ``` after: ``` 1.2i1.1 ^~~ ``` git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@60225 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-08 13:45:21 +0000