| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
| |
* test/ruby: use better assertions instead of mere assert.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@44173 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
* test/ruby/test_econv.rb (test_default_external): use
assert_separately for clearer messages.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@42383 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36534 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
| |
drop stderr.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33996 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33994 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33992 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
ascii only. [ruby-core:41556] [Bug #5733]
tests are contributed by nobu.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33991 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@33200 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
decorator according to open mode.
* transcode.c (rb_econv_prepare_options): new function, to prepare
econv options with newline flags.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@31355 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
| |
whose result is 2 bytes. [ruby-core:30751]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@28307 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26663 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
after \r\n detection instead of just after \r.
[ruby-list:45988] [ruby-core:25881] [ruby-core:26788]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25883 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
* enc/trans/utf8_mac-tbl.rb: ditto.
* test/ruby/test_econv.rb: tests for above.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23296 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@22784 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
| |
rb_eConversionUndefinedError.
(rb_eConverterNotFoundError): renamed from rb_eNoConverterError.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19554 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
| |
a patch from Tadashi Saito <shiba at mail2.accsnet.ne.jp> in
[ruby-dev:36346].
* encoding.c (Init_Encoding): rename EncodingCompatibilityError to
Encoding::CompatibilityError. [ruby-dev:36366]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19407 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
a broken string is specified as a replacement.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19316 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
* transcode.c: follow the renaming.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19311 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
(rb_econv_init_by_convpath): new function.
(econv_init): use rb_econv_init_by_convpath.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19310 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(rb_econv_open): use decorator_names.
(econv_args): extracted from econv_init.
(econv_init): use econv_args.
(decorate_convpath): new function.
(search_convpath_i): new function.
(econv_s_search_convpath): new method.
(Init_transcode): new method defined.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19305 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
| |
(make_encobj): new function.
(econv_s_asciicompat_encoding): use make_encoding.
(rb_econv_open_exc): use SUPPLEMENTAL_CONVERSION.
(econv_convpath): use encoding object in the result.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19288 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19286 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
econv_output_followed_by_input.
(ECONV_AFTER_OUTPUT): renamed from ECONV_OUTPUT_FOLLOWED_BY_INPUT.
* transcode.c: follow the renaming.
* io.c: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19278 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ECONV_ENCODER_MASK and ECONV_DECORATOR_MASK.
(ECONV_UNIVERSAL_NEWLINE_DECORATOR): renamed from
ECONV_UNIVERSAL_NEWLINE_DECODER.
(ECONV_CRLF_NEWLINE_DECORATOR): renamed from
ECONV_CRLF_NEWLINE_ENCODER.
(ECONV_CR_NEWLINE_DECORATOR): renamed from ECONV_CR_NEWLINE_ENCODER.
(ECONV_XML_TEXT_DECORATOR): renamed from ECONV_XML_TEXT_ENCODER.
(ECONV_XML_ATTR_CONTENT_DECORATOR): renamed from
ECONV_XML_ATTR_CONTENT_ENCODER.
(ECONV_STATEFUL_DECORATOR_MASK): renamed from
ECONV_STATEFUL_ENCODER_MASK.
(ECONV_XML_ATTR_QUOTE_DECORATOR): renamed from
ECONV_XML_ATTR_CONTENT_DECORATOR.
* io.c: follow the renaming.
* transcode.c: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19271 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(rb_econv_decorate_at_last): declared.
* transcode.c (rb_econv_open_by_transcoder_entries): initialize
replacement_enc. allocate outbuf for the last transcoder.
(rb_econv_open0): extracted from rb_econv_open.
(rb_econv_open): use rb_econv_open0 and decorate the result using
rb_econv_decorate_at_first and rb_econv_decorate_at_last.
(rb_econv_decorate_at): new function.
(rb_econv_decorate_at_first): ditto.
(rb_econv_decorate_at_last): ditto.
(rb_econv_binmode): fix iteration end condition.
(econv_init): don't set source_encoding_name and
destination_encoding_name because they are set in rb_econv_open0.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19262 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
from rb_econv_stateless_encoding to apply stateless ASCII
incompatible encodings such as UTF-16BE.
* io.c (make_writeconv): use rb_econv_asciicompat_encoding.
* transcode_data.h (rb_transcoder_asciicompat_type_t): renamed from
rb_transcoder_stateful_type_t.
(rb_transcoder): use rb_transcoder_asciicompat_type_t.
* transcode.c: follow the type change.
(asciicompat_encoding_i): renamed from stateless_encoding_i.
(rb_econv_asciicompat_encoding): renamed from
rb_econv_stateless_encoding.
(econv_s_asciicompat_encoding): method renamed.
* tool/transcode-tblgen.rb: follow the type change.
* enc/trans/utf_16_32.trans: follow the type change.
rb_from_UTF_16BE to UTF-8 is asciicompat_decoder.
rb_from_UTF_16LE to UTF-8 is asciicompat_decoder.
rb_from_UTF_32BE to UTF-8 is asciicompat_decoder.
rb_from_UTF_32LE to UTF-8 is asciicompat_decoder.
UTF-8 to rb_to_UTF_16BE is asciicompat_encoder.
UTF-8 to rb_to_UTF_16LE is asciicompat_encoder.
UTF-8 to rb_to_UTF_32BE is asciicompat_encoder.
UTF-8 to rb_to_UTF_32LE is asciicompat_encoder.
* enc/trans/newline.trans: follow the type change. universal newline
decoder is asciicompat_converter.
* enc/trans/escape.trans: follow the type change.
* enc/trans/iso2022.trans: ditto.
* enc/trans/japanese.trans: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19249 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
after replacement.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19247 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
| |
undefined conversion error between iso-2022-jp and the corresponding
stateless encoding.
* enc/emacs_mule.c: replicate emacs-mule as stateless-iso-2022-jp.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19220 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(ECONV_STATEFUL_ENCODER_MASK): defined.
(ECONV_XML_ATTR_QUOTE_ENCODER): defined.
(ECONV_XML_ATTR_ENCODER): removed.
* enc/trans/escape.trans (rb_escape_xml_attr_content): defined.
(rb_escape_xml_attr_quote): defined.
(rb_escape_xml_attr): removed.
* io.c (NEED_WRITECONV): writeconv is required if supplemental
converter is used.
(make_writeconv): apply stateful encoder in writeconv.
* transcode.c: follow the constant change.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19209 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ECONV_HTML_TEXT_ENCODER.
(ECONV_XML_ATTR_ENCODER): renamed from ECONV_HTML_ATTR_ENCODER.
* enc/trans/escape.trans: follow the renaming.
* transcode.c: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19191 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
ECONV_HTML_ATTR_ENCODER.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19184 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
| |
(sym_text): ditto.
(sym_attr): ditto.
(econv_opts): check :html=>:text and :html=>:attr.
(Init_transcode): initialize the above variables.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19180 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(ECONV_HTML_ATTR_ENCODER): ditto.
* transcode.c (rb_econv_open): check ECONV_HTML_TEXT_ENCODER and
ECONV_HTML_ATTR_ENCODER.
(Init_transcode): Encoding::Converter::HTML_TEXT_ENCODER and
Encoding::Converter::HTML_ATTR_ENCODER defined.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19179 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
Encoding::Converter.stateless_encoding("html-attr-escaped") should be
nil.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19174 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
| |
(fun_so_escape_html_attr): new function.
(escape_html_attr_finish): new function.
(rb_escape_html_attr): use them to quote the converted result.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19173 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19172 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
newline conversion.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19167 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19165 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19163 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
| |
* transcode.c (output_hex_charref): new function.
(rb_econv_convert): call output_hex_charref if
ECONV_UNDEF_HEX_CHARREF.
(Init_transcode): Encoding::Converter::UNDEF_HEX_CHARREF added.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19162 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19156 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
(econv_primitive_convert): accept a hash as 5th argument as well.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19131 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
| |
(sym_crlf_newline_encoder): ditto.
(sym_cr_newline_encoder): ditto.
(econv_opts): check newline converter options.
(econv_init): make 3rd argument hash/nil only.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19130 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
it tend to cause security problem. If the behaviour is really
required, ECONV_INVALID_REPLACE with empty string can be used.
For example, CVE-2006-2313, CVE-2008-1036, [ruby-core:15645]
(ECONV_UNDEF_IGNORE): ditto.
* transcode.c (rb_econv_convert): follow the above change.
(econv_opts): ditto.
(Init_transcode): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19123 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
writeconv_pre_ecopts.
(MakeOpenFile): initialize them.
* include/ruby/encoding.h (rb_str_transcode): take ecopts argument.
(rb_econv_flags): removed.
(rb_econv_prepare_opts): declared.
(rb_econv_open_opts): declared.
* io.c (make_writeconv): use rb_econv_open_opts.
(make_readconv): ditto.
(io_fwrite): follow rb_str_transcode change.
(rb_io_extract_modeenc): use rb_econv_prepare_opts.
(rb_file_open_generic): initialize encs.ecopts.
(rb_file_open_internal): ditto.
(rb_io_reopen): ditto.
(argf_ecopts): defined.
(argf_next_argv): set encs.ecopts.
(io_encoding_set): use rb_econv_prepare_opts.
(argf_set_encoding): set argf_ecopts.
* gc.c (gc_mark_children): mark encs.ecopts and writeconv_pre_ecopts
in T_FILE.
* transcode.c (transcode_loop): take ecopts argument. use
rb_econv_open_opts.
(rb_econv_flags): removed.
(rb_econv_prepare_opts): defined.
(rb_econv_open_opts): defined.
(str_transcode0): take ecopts.
(str_transcode): use rb_econv_prepare_opts.
(rb_str_transcode): take ecopts.
(econv_init): accept hash argument.
(econv_insert_output): follow rb_str_transcode change.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19109 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* transcode.c (rb_econv_t): new fields: replacement_str,
replacement_len, replacement_enc and replacement_allocated.
(get_replacement_character): make len as size_t.
(rb_econv_open_by_transcoder_entries): initialize the new fields.
(rb_econv_close): deallocate replacement_str if it allocated.
(make_replacement): new function.
(output_replacement_character): use make_replacement.
(rb_econv_set_replacemenet): defined.
(econv_get_replacement): new method.
(econv_set_replacement): new method.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19108 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19086 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
| |
destination_encoding as encoding object.
(ecerr_source_encoding): new method.
(ecerr_destination_encoding): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18993 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
| |
for storing encoding names.
(ecerr_source_encoding_name): method renamed.
(ecerr_destination_encoding_name): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18987 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18986 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|