aboutsummaryrefslogtreecommitdiffstats
path: root/ext
Commit message (Collapse)AuthorAgeFilesLines
* Remove `NODE_DASGN_CURR` [Feature #18406]Nobuyoshi Nakada2021-12-131-1/+0
| | | | | | | This `NODE` type was used in pre-YARV implementation, to improve the performance of assignment to dynamic local variable defined at the innermost scope. It has no longer any actual difference with `NODE_DASGN`, except for the node dump.
* [ruby/cgi] Check integer overflow in long rangeNobuyoshi Nakada2021-12-121-2/+11
| | | | | | https://hackerone.com/reports/1328463 https://github.com/ruby/cgi/commit/ccaf6027e0
* [ruby/io-wait] Bump up 0.2.1Hiroshi SHIBATA2021-12-091-1/+1
| | | | https://github.com/ruby/io-wait/commit/c97ab9a212
* ext/ripper/lib/ripper/lexer.rb: Do not deprecate Ripper::Lexer::State#[]Yusuke Endoh2021-12-091-14/+11
| | | | | | | | | | | The old code of IRB still uses this method. The warning is noisy on rails console. In principle, Ruby 3.1 deprecates nothing, so let's avoid the deprecation for the while. I think It is not so hard to continue to maintain it as it is a trivial shim. https://github.com/ruby/ruby/pull/5093
* [Bug #18382] Fix crash in compaction for ObjectSpace.trace_object_allocationsPeter Zhu2021-12-021-1/+2
| | | | | ObjectSpace.trace_object_allocations can crash when auto-compaction is enabled.
* Define Ripper::Lexer::Elem#to_sNobuyoshi Nakada2021-12-021-0/+2
| | | | | | Alias `#inspect` as `#to_s` also in the new `Ripper::Lexer::Elem` class, so that `puts Ripper::Lexer.new(code).scan` shows the attributes.
* Deprecate `Lexer::Elem#[]` and `Lexer::State#[]`schneems2021-12-021-0/+31
| | | | | | | Discussed in https://github.com/ruby/ruby/pull/5093#issuecomment-964426481. > it would be enough to mimic only [] for almost all cases This adds back the `Lexer::Elem#[]` and `Lexer::State#[]` and adds deprecation warnings for them.
* Only iterate Lexer heredoc arraysschneems2021-12-021-10/+12
| | | | The last element in the `@buf` may be either an array or an `Elem`. In the case it is an `Elem` we iterate over every element, when we do not need to. This check guards that case by ensuring that we only iterate over an array of elements.
* ~1.10x faster Change Ripper.lex structs to classesschneems2021-12-021-8/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ## Concept I am proposing we replace the Struct implementation of data structures inside of ripper with real classes. This will improve performance and the implementation is not meaningfully more complicated. ## Example Struct versus class comparison: ```ruby Elem = Struct.new(:pos, :event, :tok, :state, :message) do def initialize(pos, event, tok, state, message = nil) super(pos, event, tok, State.new(state), message) end # ... def to_a a = super a.pop unless a.empty? a end end class ElemClass attr_accessor :pos, :event, :tok, :state, :message def initialize(pos, event, tok, state, message = nil) @pos = pos @event = event @tok = tok @state = State.new(state) @message = message end def to_a if @message [@pos, @event, @tok, @state, @message] else [@pos, @event, @tok, @state] end end end # stub state class creation for now class State; def initialize(val); end; end ``` ## MicroBenchmark creation ```ruby require 'benchmark/ips' require 'ripper' pos = [1, 2] event = :on_nl tok = "\n".freeze state = Ripper::EXPR_BEG Benchmark.ips do |x| x.report("struct") { Elem.new(pos, event, tok, state) } x.report("class ") { ElemClass.new(pos, event, tok, state) } x.compare! end; nil ``` Gives ~1.2x faster creation: ``` Warming up -------------------------------------- struct 263.983k i/100ms class 303.367k i/100ms Calculating ------------------------------------- struct 2.638M (± 5.9%) i/s - 13.199M in 5.023460s class 3.171M (± 4.6%) i/s - 16.078M in 5.082369s Comparison: class : 3170690.2 i/s struct: 2638493.5 i/s - 1.20x (± 0.00) slower ``` ## MicroBenchmark `to_a` (Called by Ripper.lex for every element) ```ruby require 'benchmark/ips' require 'ripper' pos = [1, 2] event = :on_nl tok = "\n".freeze state = Ripper::EXPR_BEG struct = Elem.new(pos, event, tok, state) from_class = ElemClass.new(pos, event, tok, state) Benchmark.ips do |x| x.report("struct") { struct.to_a } x.report("class ") { from_class.to_a } x.compare! end; nil ``` Gives 1.46x faster `to_a`: ``` Warming up -------------------------------------- struct 612.094k i/100ms class 893.233k i/100ms Calculating ------------------------------------- struct 6.121M (± 5.4%) i/s - 30.605M in 5.015851s class 8.931M (± 7.9%) i/s - 44.662M in 5.039733s Comparison: class : 8930619.0 i/s struct: 6121358.9 i/s - 1.46x (± 0.00) slower ``` ## MicroBenchmark data access ```ruby require 'benchmark/ips' require 'ripper' pos = [1, 2] event = :on_nl tok = "\n".freeze state = Ripper::EXPR_BEG struct = Elem.new(pos, event, tok, state) from_class = ElemClass.new(pos, event, tok, state) Benchmark.ips do |x| x.report("struct") { struct.pos[1] } x.report("class ") { from_class.pos[1] } x.compare! end; nil ``` Gives ~1.17x faster data access: ``` Warming up -------------------------------------- struct 1.694M i/100ms class 1.868M i/100ms Calculating ------------------------------------- struct 16.149M (± 6.8%) i/s - 81.318M in 5.060633s class 18.886M (± 2.9%) i/s - 95.262M in 5.048359s Comparison: class : 18885669.6 i/s struct: 16149255.8 i/s - 1.17x (± 0.00) slower ``` ## Full benchmark integration of this inside of Ripper.lex Inside of this repo with this commit ``` $ cd ext/ripper $ make $ cat test.rb file = File.join(__dir__, "../../array.rb") source = File.read(file) bench = Benchmark.measure do 10_000.times.each do Ripper.lex(source) end end puts bench ``` Then execute with and without this change 50 times: ``` rm new.txt rm old.txt for i in {0..50} do `ruby -Ilib -rripper -rbenchmark ./test.rb >> new.txt` `ruby -rripper -rbenchmark ./test.rb >> old.txt` done ``` I used derailed benchmarks internals to compare the results: ``` dir = Pathname(".") branch_info = {} branch_info["old"] = { desc: "Struct lex", time: Time.now, file: dir.join("old.txt"), name: "old" } branch_info["new"] = { desc: "Class lex", time: Time.now, file: dir.join("new.txt"), name: "new" } stats = DerailedBenchmarks::StatsFromDir.new(branch_info) stats.call.banner ``` Which gave us: ``` ❤️ ❤️ ❤️ (Statistically Significant) ❤️ ❤️ ❤️ [new] (3.3139 seconds) "Class lex" ref: "new" FASTER 🚀🚀🚀 by: 1.1046x [older/newer] 9.4700% [(older - newer) / older * 100] [old] (3.6606 seconds) "Struct lex" ref: "old" Iterations per sample: Samples: 51 Test type: Kolmogorov Smirnov Confidence level: 99.0 % Is significant? (max > critical): true D critical: 0.30049534876137013 D max: 0.9607843137254902 Histograms (time ranges are in seconds): [new] description: [old] description: "Class lex" "Struct lex" ┌ ┐ ┌ ┐ [3.0, 3.3) ┤▇ 1 [3.0, 3.3) ┤ 0 [3.3, 3.6) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 47 [3.3, 3.6) ┤ 0 [3.5, 3.8) ┤▇▇ 2 [3.5, 3.8) ┤▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 46 [3.8, 4.1) ┤▇ 1 [3.8, 4.1) ┤▇▇▇ 4 [4.0, 4.3) ┤ 0 [4.0, 4.3) ┤ 0 [4.3, 4.6) ┤ 0 [4.3, 4.6) ┤▇ 1 └ ┘ └ ┘ # of runs in range # of runs in range ``` To sum this up, the "new" version of this code (using real classes instead of structs) is 10% faster across 50 runs with a statistical significance confidence level of 99%. Histograms are for visual checksum.
* [ruby/win32ole] Fix typos [ci skip]Nobuyoshi Nakada2021-11-251-1/+1
| | | | https://github.com/ruby/win32ole/commit/8d46bd0c93
* [ruby/win32ole] LICENSEHiroshi SHIBATA2021-11-251-0/+1
| | | | https://github.com/ruby/win32ole/commit/62fd78078b
* Keep the generated source files when clean [Bug #18363]Nobuyoshi Nakada2021-11-251-1/+2
|
* [ruby/zlib] [Bug #18358] Fix crash in zlib when in progressPeter Zhu2021-11-241-40/+77
| | | | | | | | When Zlib::Inflate#inflate or Zlib::Deflate#deflate is called recursively inside the block, a crash can occur because of an use-after-free bug. https://github.com/ruby/zlib/commit/50fb8a0338
* [ruby/cgi] Fix integer overflowNobuyoshi Nakada2021-11-241-1/+2
| | | | | | | | Make use of the check in rb_alloc_tmp_buffer2. https://hackerone.com/reports/1328463 https://github.com/ruby/cgi/commit/c728632c1c
* Avoid assert failure when NULL EC is expectedAlan Wu2021-11-221-0/+31
| | | | | | | | | | | | | | | | | | | After 5680c38c75aeb5cbd219aafa8eb48c315f287d97, postponed job APIs now expect to be called on native threads not managed by Ruby and handles getting a NULL execution context. However, in debug builds the change runs into an assertion failure with GET_EC() which asserts that EC is non-NULL. Avoid the assertion failure by passing `false` for `expect_ec` instead as the intention is to handle when there is no EC. Add a test from John Crepezzi and John Hawthorn to exercise this situation. See GH-4108 See GH-5094 [Bug #17573] Co-authored-by: John Hawthorn <john@hawthorn.email> Co-authored-by: John Crepezzi <john.crepezzi@gmail.com>
* Update dependenciesNobuyoshi Nakada2021-11-2187-1521/+945
|
* [ruby/date] Bump version to 3.2.2Hiroshi SHIBATA2021-11-181-1/+1
| | | | https://github.com/ruby/date/commit/21d91c5f4f
* [ruby/date] Expose Date::VERSIONJean Boussier2021-11-182-1/+7
| | | | | | | | | | An almost universal convention for gems is to expose `Namespace::VERSION` which makes it mcuh easier when debugging etc. Many gems extracted from ruby don't do this, even though it would be even more useful because they ship with ruby, so it's less clear which version it is. https://github.com/ruby/date/commit/fef7ec18d8
* Prevent "already initialized constant Digest::VERSION"Yusuke Endoh2021-11-171-2/+5
| | | | | | | | | | | http://rubyci.s3.amazonaws.com/ubuntu1804/ruby-master/log/20211117T033003Z.log.html.gz ``` installing default gems from ext: /home/chkbuild/chkbuild/tmp/build/20211117T033003Z/lib/ruby/gems/3.1.0 /home/chkbuild/chkbuild/tmp/build/20211117T033003Z/ruby/ext/digest/lib/digest/version.rb:4: warning: already initialized constant Digest::VERSION /home/chkbuild/chkbuild/tmp/build/20211117T033003Z/ruby/.ext/common/digest/version.rb:4: warning: previous definition of VERSION was here ``` This hack is copied from ext/psych/psych.gemspec
* ext/io/wait/depend: make it work for Ruby 3.1.0-devYusuke Endoh2021-11-171-1/+2
| | | | | | | | | | | | | | | | | Reverts https://github.com/ruby/io-wait/commit/2eb3841e9c8f3279955253a93d3db98b80787548 because it fails on "update-deps" check in the ruby/ruby CI. https://github.com/ruby/ruby/runs/4230891140?check_suite_focus=true ``` diff --git a/ext/io/wait/depend b/ext/io/wait/depend index 7b314b9..449e9fe 100644 --- a/ext/io/wait/depend +++ b/ext/io/wait/depend ... ``` Maybe now it does not work on Ruby 2.6. This file must be changed for each Ruby version. I have no good idea to fix this issue.
* [ruby/date] check_limit: also handle symbolsJean Boussier2021-11-161-0/+1
| | | | https://github.com/ruby/date/commit/376c65942b
* [ruby/date] `Date._<format>(nil)` should return an empty HashJean Boussier2021-11-161-0/+2
| | | | | | | | | | | | | Fix: https://github.com/ruby/date/issues/39 This is how versions previous to 3.2.1 behaved and Active Support currently rely on this behavior. https://github.com/rails/rails/blob/90357af08048ef5076730505f6e7b14a81f33d0c/activesupport/lib/active_support/values/time_zone.rb#L383-L384 Any Rails application upgrading to date `3.2.1` might run into unexpected errors. https://github.com/ruby/date/commit/8f2d7a0c7e
* [ruby/nkf] Fix a typo [ci skip]Nobuyoshi Nakada2021-11-161-1/+1
| | | | https://github.com/ruby/nkf/commit/18c118c83c
* [ruby/date] Add length limit option for methods that parses date stringsYusuke Endoh2021-11-162-89/+297
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | `Date.parse` now raises an ArgumentError when a given date string is longer than 128. You can configure the limit by giving `limit` keyword arguments like `Date.parse(str, limit: 1000)`. If you pass `limit: nil`, the limit is disabled. Not only `Date.parse` but also the following methods are changed. * Date._parse * Date.parse * DateTime.parse * Date._iso8601 * Date.iso8601 * DateTime.iso8601 * Date._rfc3339 * Date.rfc3339 * DateTime.rfc3339 * Date._xmlschema * Date.xmlschema * DateTime.xmlschema * Date._rfc2822 * Date.rfc2822 * DateTime.rfc2822 * Date._rfc822 * Date.rfc822 * DateTime.rfc822 * Date._jisx0301 * Date.jisx0301 * DateTime.jisx0301 https://github.com/ruby/date/commit/3959accef8
* [ruby/io-wait] Fix dependency for ruby 2.6Nobuyoshi Nakada2021-11-161-1/+1
| | | | https://github.com/ruby/io-wait/commit/2eb3841e9c
* [ruby/io-wait] Allow earlier versionsNobuyoshi Nakada2021-11-161-1/+1
| | | | https://github.com/ruby/io-wait/commit/1060f9348c
* [ruby/io-wait] Fix backward compatibility with earlier versionsNobuyoshi Nakada2021-11-162-7/+108
| | | | https://github.com/ruby/io-wait/commit/898248931f
* Merge digest-3.0.3.pre3 and ↵Hiroshi SHIBATA2021-11-165-6/+23
| | | | https://github.com/ruby/digest/commit/efd76821b8a467c193c753104c29b476debbb2c9
* [ruby/digest] Abort loading if being loaded by gem/bundle pre Ruby 3.0.3Akinori MUSHA2021-11-161-14/+12
| | | | https://github.com/ruby/digest/commit/efd76821b8
* [ruby/digest] Avoid null pointer subtraction in digest/md5xtkoba2021-11-161-1/+1
| | | | | | | | Fixes warning on Clang 13. Fixes [Bug #18076] https://github.com/ruby/digest/commit/32135c7487
* [Feature #18290] Remove all usages of rb_gc_force_recyclePeter Zhu2021-11-082-5/+0
| | | | | This commit removes usages of rb_gc_force_recycle since it is a burden to maintain and makes changes to the GC difficult.
* Prefer ANSI-style prototypes over old K&R-style definitionsNobuyoshi Nakada2021-11-081-2/+6
|
* Fix filename typo [Bug #18140]Nobuyoshi Nakada2021-11-082-160/+160
|
* [ruby/openssl] Fix typos [ci skip]Nobuyoshi Nakada2021-11-032-2/+2
| | | | https://github.com/ruby/openssl/commit/708ebf2f7a
* ext/socket/extconf.rb: Fix the chech if if_indextoname is availableYusuke Endoh2021-11-021-2/+5
| | | | The check had not work because "headers" were not passed.
* Fix typosNobuyoshi Nakada2021-11-021-1/+1
|
* [ruby/openssl] x509name: improve docs for X509::NameKazuki Yamaguchi2021-11-012-2/+27
| | | | | | | | | | | | Add docs for X509::Name.parse_openssl and X509::Name.parse_rfc2253, which are currently undocumented despite being widely used. Small changes are also made to #to_s and the class description to recommend using RFC 2253-based methods. Fixes: https://github.com/ruby/openssl/issues/470 https://github.com/ruby/openssl/commit/74041a35d4
* [ruby/openssl] ssl: disallow reading/writing to unstarted SSL socketKazuki Yamaguchi2021-11-011-139/+92
| | | | | | | | | | | | | | | OpenSSL::SSL::SSLSocket allowed #read and #write to be called before an SSL/TLS handshake is completed. They passed unencrypted data to the underlying socket. This behavior is very odd to have in this library. A verbose mode warning "SSL session is not started yet" was emitted whenever this happened. It also didn't behave well with OpenSSL::Buffering. Let's just get rid of it. Fixes: https://github.com/ruby/openssl/issues/9 https://github.com/ruby/openssl/commit/bf780748b3
* [Feature #18239] Implement VWA for stringsPeter Zhu2021-10-252-5/+14
| | | | | This commit adds support for embedded strings with variable capacity and uses Variable Width Allocation to allocate strings.
* Make Coverage suspendable (#4856)Yusuke Endoh2021-10-251-5/+118
| | | | | | | * Make Coverage suspendable Add `Coverage.suspend`, `Coverage.resume` and some methods. [Feature #18176] [ruby-core:105321]
* [ruby/openssl] bn: expand BIGNUM_RAND and BIGNUM_RAND_RANGE macrosKazuki Yamaguchi2021-10-251-50/+50
| | | | | | | Now that BN.pseudo_rand{,_range} are alias, those macros are only used once. Let's expand the macros for better readability. https://github.com/ruby/openssl/commit/7c2fc00dee
* [ruby/openssl] bn: make BN.pseudo_rand{,_range} an alias of BN.rand{,_range}Kazuki Yamaguchi2021-10-251-16/+2
| | | | | | | | BN_pseudo_rand() and BN_pseudo_rand_range() are deprecated in OpenSSL 3.0. Since they are identical to their non-'pseudo' version anyway, let's make them alias. https://github.com/ruby/openssl/commit/2d34e85ddf
* [ruby/openssl] pkey, ssl: use EVP_PKEY_eq() instead of EVP_PKEY_cmp()Kazuki Yamaguchi2021-10-254-3/+8
| | | | | | | OpenSSL 3.0 renamed EVP_PKEY_cmp() to EVP_PKEY_eq() because that was a confusing name. https://github.com/ruby/openssl/commit/d42bd7fcdb
* [ruby/openssl] pkey/ec: use EC_GROUP_free() instead of EC_GROUP_clear_free()Kazuki Yamaguchi2021-10-251-1/+1
| | | | | | | | | EC_GROUP_clear_free() is deprecated in OpenSSL 3.0. EC_GROUP does not include any sensitive data, so we can safely use EC_GROUP_free() instead. https://github.com/ruby/openssl/commit/e93a5fdffc
* [ruby/openssl] pkey/ec: deprecate PKey::EC::Point#make_affine! and make it a ↵Kazuki Yamaguchi2021-10-251-0/+5
| | | | | | | | | | | | | no-op It converts the internal representation of the point object to the affine coordinate system. However, it had no real use case because the difference in the internal representation has not been visible from Ruby/OpenSSL at all. EC_POINT_make_affine() is marked as deprecated in OpenSSL 3.0. https://github.com/ruby/openssl/commit/e2cc81fef7
* [ruby/openssl] hmac: use EVP_MD_CTX_get_pkey_ctx() instead of ↵Kazuki Yamaguchi2021-10-253-5/+14
| | | | | | | | | | EVP_MD_CTX_pkey_ctx() OpenSSL 3.0 renamed EVP_MD_CTX_pkey_ctx() to include "get" in the function name. Adjust compatibility macro so that we can use the new function name for all OpenSSL 1.0.2-3.0. https://github.com/ruby/openssl/commit/c106d888c6
* [ruby/openssl] digest: use EVP_MD_CTX_get0_md() instead of EVP_MD_CTX_md() ↵Kazuki Yamaguchi2021-10-254-4/+9
| | | | | | | | | | | | | | if exists The function was renamed in OpenSSL 3.0 due to the change of the lifetime of EVP_MD objects. They are no longer necessarily statically allocated and can be reference-counted -- when an EVP_MD_CTX is free'd, the associated EVP_MD can also become inaccessible. Currently Ruby/OpenSSL only handles builtin algorithms, so no special handling is needed except for adapting to the rename. https://github.com/ruby/openssl/commit/0a253027e6
* [ruby/openssl] bn: use BN_check_prime() in OpenSSL::BN#prime{,_fasttest}?Kazuki Yamaguchi2021-10-252-49/+22
| | | | | | | In OpenSSL 3.0, BN_is_prime_ex() and BN_is_prime_fasttest_ex() are deprecated in favor of BN_check_prime(). https://github.com/ruby/openssl/commit/90d51ef510
* [ruby/openssl] ssl: use SSL_get_rbio() to check if SSL is started or notKazuki Yamaguchi2021-10-251-2/+2
| | | | | | | | | | Use SSL_get_rbio() instead of SSL_get_fd(). SSL_get_fd() internally calls SSL_get_rbio() and it's enough for our purpose. In OpenSSL 3.0, SSL_get_fd() leaves an entry in the OpenSSL error queue if BIO has not been set up yet, and we would have to clean it up. https://github.com/ruby/openssl/commit/e95ee24867
* [ruby/openssl] ssl: use SSL_CTX_load_verify_{file,dir}() if availableKazuki Yamaguchi2021-10-252-0/+8
| | | | | | | SSL_CTX_load_verify_locations() is deprecated in OpenSSL 3.0 and replaced with those two separate functions. Use them if they exist. https://github.com/ruby/openssl/commit/5375a55ffc
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-01 16:10:28 +0000