aboutsummaryrefslogtreecommitdiffstats
path: root/encoding.c
Commit message (Collapse)AuthorAgeFilesLines
* No sync on ASCII/US_ASCCII/UTF-8Koichi Sasada2020-12-171-3/+10
| | | | | | | | | | | | rb_enc_from_index(index) doesn't need locking if index specify ASCII/US_ASCCII/UTF-8. rb_enc_from_index() is called frequently so it has impact. user system total real r_parallel/miniruby 174 0.000209 0.000000 5.559872 ( 1.811501) r_parallel/master_mini 175 0.000238 0.000000 12.664707 ( 3.523641) (repeat x1000 `s.split(/,/)` where s = '0,,' * 1000)
* Set default for Encoding.default_external to UTF-8 on Windows (#2877)Lars Kanis2020-12-081-1/+3
| | | | | | * Use UTF-8 as default for Encoding.default_external on Windows * Document UTF-8 change on Windows to Encoding.default_external fix https://bugs.ruby-lang.org/issues/16604
* fix public interfaceKoichi Sasada2020-11-181-1/+0
| | | | | | | | | | | | | | | | | | To make some kind of Ractor related extensions, some functions should be exposed. * include/ruby/thread_native.h * rb_native_mutex_* * rb_native_cond_* * include/ruby/ractor.h * RB_OBJ_SHAREABLE_P(obj) * rb_ractor_shareable_p(obj) * rb_ractor_std*() * rb_cRactor and rm ractor_pub.h and rename srcdir/ractor.h to srcdir/ractor_core.h (to avoid conflict with include/ruby/ractor.h)
* Don't redefine #rb_intern over and over againStefan Stüben2020-10-211-2/+0
|
* Revert "reduce lock for encoding"Koichi Sasada2020-10-201-72/+53
| | | | | | | | This reverts commit de17e2dea137bc5ba9f00e3acec32792d0dbb2eb. This patch can introduce race condition because of conflicting read/write access for enc_table::default_list. Maybe we need to freeze default_list at the end of Init_encdb() in enc/encdb.c.
* reduce lock for encodingKoichi Sasada2020-10-191-53/+72
| | | | | | | | | To reduce the number of locking for encoding manipulation, enc_table::list is splited to ::default_list and ::additional_list. ::default_list is pre-allocated and no need locking to access to the ::default_list. If additional encoding space is needed, use ::additional_list and this list need to use locking. However, most of case, ::default_list is enough.
* Check encoding name to replicateNobuyoshi Nakada2020-10-151-14/+28
| | | | https://hackerone.com/reports/954433
* freeze Encoding objectsKoichi Sasada2020-10-141-1/+4
| | | | | | Encoding objects can be accessed in multi-ractors and there is no state to mutate. So we can mark it as frozen and shareable. [Bug #17188]
* sync enc_table and rb_encoding_listKoichi Sasada2020-10-141-174/+371
| | | | | | | | | | | | | | | | | enc_table which manages Encoding information. rb_encoding_list also manages Encoding objects. Both are accessed/modified by ractors simultaneously so that they should be synchronized. For enc_table, this patch introduced GLOBAL_ENC_TABLE_ENTER/LEAVE/EVAL to access this table with VM lock. To make shortcut, three new global variables global_enc_ascii, global_enc_utf_8, global_enc_us_ascii are also introduced. For rb_encoding_list, we split it to rb_default_encoding_list (256 entries) and rb_additional_encoding_list. rb_default_encoding_list is fixed sized Array so we don't need to synchronized (and most of apps only needs it). To manage 257 or more encoding objects, they are stored into rb_additional_encoding_list. To access rb_additional_encoding_list., VM lock is needed.
* Fallback to built-in UTF-8 for minirubyNobuyoshi Nakada2020-05-161-1/+1
| | | | Source code encoding is defaulted to UTF-8 now too.
* sed -i 's|ruby/impl|ruby/internal|'卜部昌平2020-05-111-1/+1
| | | | To fix build failures.
* sed -i s|ruby/3|ruby/impl|g卜部昌平2020-05-111-1/+1
| | | | This shall fix compile errors.
* Added more NORETURN declarationsNobuyoshi Nakada2020-05-111-1/+2
|
* Merge pull request #2991 from shyouhei/ruby.h卜部昌平2020-04-081-1/+1
| | | Split ruby.h
* Get rid of warnings/exceptions at cleanupNobuyoshi Nakada2020-02-131-0/+10
| | | | | | | | After the encoding index instance variable is removed when all instance variables are removed in `obj_free`, then `rb_str_free` causes uninitialized instance variable warning and nil-to-integer conversion exception. Both cases result in object allocation during GC, and crashes.
* rb_enc_str_asciionly_p expects T_STRING卜部昌平2020-02-101-1/+1
| | | | | | This `str2` variable can be non-string (regexp etc.) but the previous code passed it directly to rb_enc_str_asciionly_p(), which expects its argument be a string. Let's enforce that constraint.
* more on NULL versus functions.卜部昌平2020-02-071-1/+1
| | | | | | Function pointers are not void*. See also ce4ea956d24eab5089a143bba38126f2b11b55b6 8427fca49bd85205f5a8766292dd893f003c0e48
* Fix description of Encoding.default_(in|ex)ternalLars Kanis2020-02-031-3/+2
| | | | | | | | Data written to files is not transcoded per default, but only when default_internal is set. The default for default_internal is nil and doesn't depend on the source file encoding.
* decouple internal.h headers卜部昌平2019-12-261-4/+14
| | | | | | | | | | | | | | | | | | Saves comitters' daily life by avoid #include-ing everything from internal.h to make each file do so instead. This would significantly speed up incremental builds. We take the following inclusion order in this changeset: 1. "ruby/config.h", where _GNU_SOURCE is defined (must be the very first thing among everything). 2. RUBY_EXTCONF_H if any. 3. Standard C headers, sorted alphabetically. 4. Other system headers, maybe guarded by #ifdef 5. Everything else, sorted alphabetically. Exceptions are those win32-related headers, which tend not be self- containing (headers have inclusion order dependencies).
* [Doc] Fix sample in Encoding#namesQuestionDriven2019-12-221-1/+1
|
* [Doc] Fix wrong example in Encoding.aliasesQuestionDriven2019-12-221-2/+2
|
* Deprecate taint/trust and related methods, and make the methods no-opsJeremy Evans2019-11-181-1/+0
| | | | | | This removes the related tests, and puts the related specs behind version guards. This affects all code in lib, including some libraries that may want to support older versions of Ruby.
* Warn on access/modify of $SAFE, and remove effects of modifying $SAFEJeremy Evans2019-11-181-1/+1
| | | | | | | | | | | | | | | | | This removes the security features added by $SAFE = 1, and warns for access or modification of $SAFE from Ruby-level, as well as warning when calling all public C functions related to $SAFE. This modifies some internal functions that took a safe level argument to no longer take the argument. rb_require_safe now warns, rb_require_string has been added as a version that takes a VALUE and does not warn. One public C function that still takes a safe level argument and that this doesn't warn for is rb_eval_cmd. We may want to consider adding an alternative method that does not take a safe level argument, and warn for rb_eval_cmd.
* Moved Init_encoding from wrong place [Bug #16292]Nobuyoshi Nakada2019-11-051-6/+6
|
* avoid overflow in integer multiplication卜部昌平2019-10-091-1/+1
| | | | | | | This changeset basically replaces `ruby_xmalloc(x * y)` into `ruby_xmalloc2(x, y)`. Some convenient functions are also provided for instance `rb_xmalloc_mul_add(x, y, z)` which allocates x * y + z byes.
* Better wording for __ENCODING__Lars Kanis2019-08-041-1/+1
| | | | | | | "locale encoding" is misleading since it doesn't mean Encoding.find("locale") but the encoding used to interpret the script file. It's therefore better to call it "script encoding" as in the paragraphs above. Closes: https://github.com/ruby/ruby/pull/1611
* Let the index boundary check in rb_enc_from_index be flagged as unlikelyLourens Naudé2019-07-231-1/+1
| | | | | | [Misc #15806] Closes: https://github.com/ruby/ruby/pull/2128
* Explicitly initialise encodings on init to remove branches on encoding lookupLourens Naudé2019-07-231-17/+6
| | | | | | [Misc #15806] Closes: https://github.com/ruby/ruby/pull/2128
* respect RUBY_DEBUG tooKoichi Sasada2019-07-151-1/+1
|
* encoding.c (enc_table_expand): prefer xrealloc to reallocYusuke Endoh2019-07-151-2/+4
| | | | And raise an exception when failed to register an encoding
* [DOC] fix markups [ci skip]nobu2019-03-221-10/+11
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@67337 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c: check external encodingnobu2018-09-151-1/+1
| | | | | | | * encoding.c (rb_enc_get_index): external encoding may not be Data object. [ruby-core:89016] [Bug #15122] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64753 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c (rb_enc_get_index): return -1 for non-encoding capable objectseregon2018-08-031-8/+10
| | | | | | | * Clarify logic and add spec. * Now passes test-all with the JSON fix. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64178 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* Revert "encoding.c (rb_enc_get_index): return -1 for non-encoding capable ↵eregon2018-08-031-10/+8
| | | | | | | | | objects" * This reverts commit fb253d2032e51f333e6577aa4fcf0c03f9c6fc02. * The CI is failing, this seems a bug in the JSON C extension. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64174 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c (rb_enc_get_index): return -1 for non-encoding capable objectseregon2018-08-031-8/+10
| | | | | | * Clarify logic and add spec. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64173 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c (enc_set_index): raise instead of rb_bug() for non-encoding ↵eregon2018-08-031-1/+3
| | | | | | | | capable objects * Add spec. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@64172 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* check enc_capable.ko12018-06-281-0/+8
| | | | | | | | | | | | | | | | | | | | | | | * encoding.c (rb_enc_capable): make it extern to check enc_capable. enc_index can be set to limited types such as T_STRING, T_REGEX and so on. This function check an object is this kind of types. * include/ruby/encoding.h: ditto. * encoding.c (enc_set_index): check a given object is enc_capable. * include/ruby/encoding.h (PUREFUNC): * marshal.c (encoding_name): check `rb_enc_capable` first. * marshal.c (r_ivar): ditto. If it is not enc_capable, it should be malformed data. * spec/ruby/optional/capi/encoding_spec.rb: remove tests depending on the wrong feature: all objects can set enc_index. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63777 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c: remove rb_gc_mark_encodingsnormal2018-06-051-5/+0
| | | | | | | | rb_gc_mark_encodings has been empty for a decade (since r17875 / 28b216ac45262819f8bc7be0b1417a707974f94b). Just remove it and its only caller in gc.c git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@63582 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* load.c: use fstring instead of OBJ_FREEZEnormal2018-01-241-5/+3
| | | | | | | These strings already exist (or will exist soon) in the fstring table, so avoid the duplicate, sooner. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@62030 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* internal.h: remove dependecy on ruby/encoding.hnobu2018-01-091-0/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61713 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c: rb_enc_find_index2nobu2017-12-221-0/+11
| | | | | | | * string.c (str_undump): use rb_enc_find_index2 to find encoding by unterminated string. check the format before encoding name. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@61396 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c: handle needmore error from rb_enc_precise_mbclen()rhe2016-12-141-1/+2
| | | | | | | | | | | | rb_enc_ascget() erroneously reports success even if the given byte sequence is incomplete, for non-ASCII compatible encoding strings. rb_enc_precise_mbclen() may return a negative value on error, and thus rb_enc_ascget() must not store the return value in 'unsigned int'; otherwise the subsequent MBCLEN_CHARFOUND_P() check won't catch the error. [ruby-core:78646] [Bug #13034] git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@57078 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * string.c (rb_str_change_terminator_length): New function to changengoto2016-07-051-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | termlen and resize heap for the terminator. This is split from rb_str_fill_terminator (str_fill_term) because filling terminator and changing terminator length are different things. [Bug #12536] * internal.h: declaration for rb_str_change_terminator_length. * string.c (str_fill_term): Simplify only to zero-fill the terminator. For non-shared strings, it assumes that (capa + termlen) bytes of heap is allocated. This partially reverts r55557. * encoding.c (rb_enc_associate_index): rb_str_change_terminator_length is used, and it should be called whenever the termlen is changed. * string.c (str_capacity): New static function to return capacity of a string with the given termlen, because the termlen may sometimes be different from TERM_LEN(str) especially during changing termlen or filling terminator with specific termlen. * string.c (rb_str_capacity): Use str_capacity. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@55575 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * encoding.c: Fix return value of `Encoding::ISO8859_1.name`hsbt2016-04-251-1/+1
| | | | | | | [Bug #12313][ruby-core:75147][ci skip] * ext/bigdecimal/bigdecimal.c: Fix code sample of `BigDecimal.new` git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54755 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/ruby.h (rb_isupper, rb_islower, rb_isalpha, rb_isdigit,naruse2016-03-291-28/+0
| | | | | | | | | | | rb_isalnum, rb_isxdigit, rb_isblank, rb_isspace, rb_isblank, rb_iscntrl, rb_isprint, rb_ispunct, rb_isgraph, rb_tolower, rb_toupper): use inline function to avoid function call. * include/ruby/ruby.h (rb_isascii): use inline function to clarify the logic. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@54391 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* RUBY_ASSERTnobu2016-01-221-1/+1
| | | | | | | * error.c (rb_assert_failure): assertion with stack dump. * ruby_assert.h (RUBY_ASSERT): new header for the assertion. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@53615 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * introduce new ISeq binary format serializer/de-serializerko12015-12-081-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | and a pre-compilation/runtime loader sample. [Feature #11788] * iseq.c: add new methods: * RubyVM::InstructionSequence#to_binary_format(extra_data = nil) * RubyVM::InstructionSequence.from_binary_format(binary) * RubyVM::InstructionSequence.from_binary_format_extra_data(binary) * compile.c: implement body of this new feature. * load.c (rb_load_internal0), iseq.c (rb_iseq_load_iseq): call RubyVM::InstructionSequence.load_iseq(fname) with loading script name if this method is defined. We can return any ISeq object as a result value. Otherwise loading will be continue as usual. This interface is not matured and is not extensible. So that we don't guarantee the future compatibility of this method. Basically, you should'nt use this method. * iseq.h: move ISEQ_MAJOR/MINOR_VERSION (and some definitions) from iseq.c. * encoding.c (rb_data_is_encoding), internal.h: added. * vm_core.h: add several supports for lazy load. * add USE_LAZY_LOAD macro to specify enable or disable of this feature. * add several fields to rb_iseq_t. * introduce new macro rb_iseq_check(). * insns.def: some check for lazy loading feature. * vm_insnhelper.c: ditto. * proc.c: ditto. * vm.c: ditto. * test/lib/iseq_loader_checker.rb: enabled iff suitable environment variables are provided. * test/runner.rb: enable lib/iseq_loader_checker.rb. * sample/iseq_loader.rb: add sample compiler and loader. $ ruby sample/iseq_loader.rb [dir] will compile all ruby scripts in [dir]. With default setting, this compile creates *.rb.yarb files in same directory of target .rb scripts. $ ruby -r sample/iseq_loader.rb [app] will run with enable to load compiled binary data. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52949 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c: make fstringnobu2015-12-021-5/+3
| | | | | | | * encoding.c (enc_name, rb_enc_name_list_i, rb_enc_aliases_enc_i): make fstring instead of making each copies. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52859 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c: defer finding encodingnobu2015-12-021-1/+18
| | | | | | | | | | | | | * encoding.c (enc_m_loader): defer finding encoding object not to be infected by marshal source. [ruby-core:71793] [Bug #11760] * marshal.c (r_object0): enable compatible loader on USERDEF class. the loader function is called with the class itself, instead of an allocated object, and the loaded data. * marshal.c (compat_allocator_table): intialize compat_allocator_tbl on demand. * object.c (rb_undefined_alloc): extract from rb_obj_alloc. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52856 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* encoding.c: use rb_check_aritynobu2015-12-021-1/+1
| | | | | | | * encoding.c (enc_dump): use rb_check_arity to just check number of arguments. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@52854 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.39.1) at 2024-04-28 20:47:43 +0000