| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
"suggest parentheses around assignment used as truth value"
* transcode_data.h (getGB4bt1): clean a warning.
"suggest parentheses around arithmetic in operand of |"
* transcode_data.h (getGB4bt3): ditto.
* vm.c (thread_free): clean a warning. "format 'p' expects type
'void *', but argument 3 has type 'struct rb_mutex_struct *'"
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@28359 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
| |
support for new transcoding instruction FUNsio (with Tatsuya Mizuno)
* enc/trans/gb18030.trans: Significantly reduced GB18030 conversion
table footprint using FUNsio and differences (with Tatsuya Mizuno)
* test/ruby/test_transcode.rb: Minor name fix (from Tatsuya Mizuno)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@26065 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@25635 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* transcode.c: NOMAP is now multibyte direct map.
* transcode.c: remove ASIS.
* transcode_data.h: ditto.
* tool/transcode-tb (ActionMap#generate_info): remove :asis.
* tool/transcode-tb (ActionMap#generate_info): add :nomap0.
* enc/trans/utf8_mac.trans: replace :asis by :nomap0.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23344 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
| |
* transcode.c (transcode_restartable0): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@23294 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
| |
numeric.c, pack.c, strftime.c, string.c, thread.c, transcode.c,
transcode_data.h, util.c, variable.c, vm_dump.c,
include/ruby/encoding.h, missing/crypt.c, missing/vsnprintf.c:
suppress VC type warnings. [ruby-core:22726]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@22914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@22906 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
new Chinese GB18030 transcoding (from Yoshihiro Kambayashi)
* test/ruby/test_transcode.rb: added tests for the above
(from Yoshihiro Kambayashi)
* transcode_data.h, transcode.c, tool/transcode_tblgen.rb:
added support for GB18030-specific 4-byte sequences
(with Yoshihiro Kambayashi)
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@21509 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(makeSTR1LEN): defined.
* tool/transcode-tblgen.rb: use makeSTR1LEN. generate STR1 for 4 to
259 bytes.
* transcode.c (rb_transcoding): new field: output_index.
(transcode_restartable0): use STR1_LENGTH.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19366 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
| |
also returns ssize_t.
* enc/trans/iso2022.trans: follow the type change.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19354 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* transcode.c (transcode_restartable0): don't need to cast the result
of output functions.
* enc/trans/newline.trans: follow the type change.
* enc/trans/escape.trans: ditto.
* enc/trans/utf_16_32.trans: ditto.
* enc/trans/iso2022.trans: ditto.
* enc/trans/japanese.trans: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19351 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* transcode.c: give output buffer size for output functions.
* enc/trans/newline.trans: follow the type change.
* enc/trans/escape.trans: ditto.
* enc/trans/utf_16_32.trans: ditto.
* enc/trans/iso2022.trans: ditto.
* enc/trans/japanese.trans: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19350 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
from rb_econv_stateless_encoding to apply stateless ASCII
incompatible encodings such as UTF-16BE.
* io.c (make_writeconv): use rb_econv_asciicompat_encoding.
* transcode_data.h (rb_transcoder_asciicompat_type_t): renamed from
rb_transcoder_stateful_type_t.
(rb_transcoder): use rb_transcoder_asciicompat_type_t.
* transcode.c: follow the type change.
(asciicompat_encoding_i): renamed from stateless_encoding_i.
(rb_econv_asciicompat_encoding): renamed from
rb_econv_stateless_encoding.
(econv_s_asciicompat_encoding): method renamed.
* tool/transcode-tblgen.rb: follow the type change.
* enc/trans/utf_16_32.trans: follow the type change.
rb_from_UTF_16BE to UTF-8 is asciicompat_decoder.
rb_from_UTF_16LE to UTF-8 is asciicompat_decoder.
rb_from_UTF_32BE to UTF-8 is asciicompat_decoder.
rb_from_UTF_32LE to UTF-8 is asciicompat_decoder.
UTF-8 to rb_to_UTF_16BE is asciicompat_encoder.
UTF-8 to rb_to_UTF_16LE is asciicompat_encoder.
UTF-8 to rb_to_UTF_32BE is asciicompat_encoder.
UTF-8 to rb_to_UTF_32LE is asciicompat_encoder.
* enc/trans/newline.trans: follow the type change. universal newline
decoder is asciicompat_converter.
* enc/trans/escape.trans: follow the type change.
* enc/trans/iso2022.trans: ditto.
* enc/trans/japanese.trans: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19249 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(STR1_BYTEINDEX): defined.
(makeSTR1): defined.
* tool/transcode-tblgen.rb: generate STR1.
* transcode.c (transcode_restartable0): interpret STR1.
* enc/trans/escape.trans (fun_so_escape_xml_chref): removed. STR1 is
used instead.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19214 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
| |
from_encoding -> src_encoding, to_encoding -> dst_encoding.
* transcode.c: follow the renaming.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19142 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
* transcode.c (PType): don't define here.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19127 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19107 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
| |
(TRANSCODING_READBUF): ditto.
(TRANSCODING_WRITEBUF): ditto.
(TRANSCODING_STATE_EMBED_MAX): ditto.
(TRANSCODING_STATE): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19097 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
add state field.
(TRANSCODING_STATE): defined.
(rb_transcoder): add fields: state_size, state_init_func,
state_fini_func.
change rb_transcoding* argument to void*.
* transcode.c (transcode_restartable0): use TRANSCODING_STATE for
first arguments of transcoder functions.
(rb_transcoding_open_by_transcoder): initialize state field.
(rb_transcoding_close): finalize state field.
* tool/transcode-tblgen.rb: provide state size/init/fini.
* enc/trans/newline.trans (universal_newline_init): defined.
(fun_so_universal_newline): take void* as a state pointer.
(rb_universal_newline): provide state size/init/fini.
(rb_crlf_newline): ditto.
(rb_cr_newline): ditto.
* enc/trans/iso2022.trans (iso2022jp_init): defined.
(fun_si_iso2022jp_to_eucjp): take void* as a state pointer.
(fun_so_iso2022jp_to_eucjp): ditto.
(fun_so_eucjp_to_iso2022jp): ditto.
(iso2022jp_reset_sequence_size): ditto.
(finish_eucjp_to_iso2022jp): ditto.
(rb_ISO_2022_JP_to_EUC_JP): provide state size/init/fini.
(rb_EUC_JP_to_ISO_2022_JP): ditto.
* enc/trans/utf_16_32.trans (fun_so_from_utf_16be): take void* as a
state pointer.
(fun_so_to_utf_16be): ditto.
(fun_so_from_utf_16le): ditto.
(fun_so_to_utf_16le): ditto.
(fun_so_from_utf_32be): ditto.
(fun_so_to_utf_32be): ditto.
(fun_so_from_utf_32le): ditto.
(fun_so_to_utf_32le): ditto.
(rb_from_UTF_16BE): provide state size/init/fini.
(rb_to_UTF_16BE): ditto.
(rb_from_UTF_16LE): ditto.
(rb_to_UTF_16LE): ditto.
(rb_from_UTF_32BE): ditto.
(rb_to_UTF_32BE): ditto.
(rb_from_UTF_32LE): ditto.
(rb_to_UTF_32LE): ditto.
* enc/trans/japanese.trans (fun_so_eucjp2sjis): take void* as a state
pointer.
(fun_so_sjis2eucjp): ditto.
(rb_eucjp2sjis): provide state size/init/fini.
(rb_sjis2eucjp): provide state size/init/fini.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19096 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(WORDINDEX2INFO): defined.
(INFO2WORDINDEX): defined.
* tool/transcode-tblgen.rb: use WORDINDEX2INFO.
* transcode.c: use INFO2WORDINDEX.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19093 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
| |
word_array_length.
* tool/transcode-tblgen.rb (transcode_generated_code): generate
byte_array_length and word_array_length.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19088 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(BYTE_LOOKUP): removed.
(BYTE_LOOKUP_BASE): don't cast.
(BYTE_LOOKUP_INFO): ditto.
(PType): unsigned int, instead of uintptr_t.
(rb_transcoding): change type of next_field, conv_tree_start and
word_array.
* tool/transcode-tblgen.rb: generate word_array as array of unsigned
int.
* transcode.c (transcode_restartable0): follow the above type change.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19070 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
| |
(o4): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19048 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
word_array to avoid relocation.
* transcode.c (transcode_restartable0): add word_array to get infos
and BYTE_LOOKUPs.
* transcode_data.h (BYTE_LOOKUP_INFO): change return type to
uintptr_t.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19045 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
byte_array to avoid relocation.
* transcode.c (transcode_restartable0): add byte_array to get offsets
array.
* transcode_data.h (BYTE_LOOKUP_BASE): change return type to
uintptr_t.
(rb_transcoder): add fields: byte_array, word_array and word_size.
* enc/trans/newline.trans: follow rb_transcoder change.
* enc/trans/iso2022.trans: ditto.
* enc/trans/utf_16_32.trans: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19043 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(BYTE_LOOKUP_BASE): follow the type change.
(BYTE_LOOKUP_INFO): ditto.
(PType): ditto.
(rb_transcoding): ditto.
* tool/transcode-tblgen.rb: follow the type change.
* transcode.c: ditto.
* enc/trans/newline.trans: ditto.
* enc/trans/iso2022.trans: ditto.
* enc/trans/utf_16_32.trans: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19038 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
| |
BYTE_LOOKUP.
(BYTE_LOOKUP_INFO): ditto.
* transcode.c (transcode_restartable0): use BYTE_LOOKUP_BASE and
BYTE_LOOKUP_INFO.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19037 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(rb_transcoder): add field: stateful_type.
* tool/transcode-tblgen.rb: generate stateful_type field as
stateless_converter.
* enc/trans/iso2022.trans: follow rb_transcoder change.
* enc/trans/newline.trans: ditto.
* enc/trans/utf_16_32.trans: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18650 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* transcode_data.h (rb_transcoder): add resetsize_func field.
* enc/trans/iso2022.trans (iso2022jp_reset_sequence_size): defined.
(rb_EUC_JP_to_ISO_2022_JP): provede resetsize_func.
* tool/transcode-tblgen.rb: set NULL for resetsize_func.
* transcode.c (rb_econv_output): new function for inserting output.
(output_replacement_character): use rb_econv_output.
(transcode_loop): check return value of
output_replacement_character.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18628 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
transcode_data.h.
(rb_econv_elem_t): ditto.
(rb_econv_t): ditto. source_encoding and destination_encoding field
is added.
(rb_econv_open): declared.
(rb_econv_convert): ditto.
(rb_econv_close): ditto.
* transcode.c (rb_econv_open_by_transcoder_entries): initialize
source_encoding and destination_encoding field as NULL.
(rb_econv_open): make it external linkage.
(rb_econv_close): ditto.
(rb_econv_convert): ditto. renamed from rb_econv_conv.
(make_encoding): new function.
(econv_init): use make_encoding and store rb_encoding* in
rb_econv_t.
(econv_source_encoding): new method
Encoding::Converter#source_encoding.
(econv_destination_encoding): new method
Encoding::Converter#destination_encoding.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18625 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
| |
constant's prefix: transcode_ to econv_.
* transcode.c: follow the constant prefix change.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18624 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
transcode_invalid_input.
(transcode_destination_buffer_full): renamed from transcode_obuf_full.
(transcode_source_buffer_empty): renamed from transcode_ibuf_empty.
(rb_econv_result_t): renamed from rb_trans_result_t.
(rb_econv_elem_t): renamed from rb_trans_elem_t.
(rb_econv_t): renamed from rb_trans_t.
* transcode.c (UNIVERSAL_NEWLINE_DECODER): renamed from
UNIVERSAL_NEWLINE.
(CRLF_NEWLINE_ENCODER): renamed from CRLF_NEWLINE.
(CR_NEWLINE_ENCODER): renamed from CR_NEWLINE.
(rb_econv_open): renamed from rb_trans_open.
(rb_econv_close): renamed from rb_trans_close.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18618 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
transcode_output_followed_by_input.
* transcode.c (OUTPUT_FOLLOWED_BY_INPUT): new flag.
(transcode_restartable0): suspend when output followed by input if
OUTPUT_FOLLOWED_BY_INPUT is specified.
(trans_sweep): check OUTPUT_FOLLOWED_BY_INPUT.
(rb_trans_conv): support OUTPUT_FOLLOWED_BY_INPUT.
(econv_primitive_convert): return :output_followed_by_input for
transcode_output_followed_by_input.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18608 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
writebuf_len and writebuf.
(TRANSCODING_WRITEBUF): new macro.
* transcode.c (transcode_restartable0): output until output buffer is
really full.
(rb_transcoding_open_by_transcoder): initialize writebuf_len,
writebuf_off and writebuf.
(rb_transcoding_close): finalize writebuf.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18573 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* transcode_data.h (rb_trans_t): add last_tc field.
* transcode.c (UNIVERSAL_NEWLINE): defined.
(CRLF_NEWLINE): defined.
(CR_NEWLINE): defined.
(rb_trans_open_by_transcoder_entries): initialize last_tc.
(trans_open_i): allocate one more room for newline converter.
(rb_trans_open): universal newline implemented.
(more_output_buffer): take max_output argument instead ts.
(output_replacement_character): take tc argument instead of ts.
(transcode_loop): use last_tc field.
(econv_init): add flags argument for rb_trans_open.
(Init_transcode): Encoding::Converter::UNIVERSAL_NEWLINE defined.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18556 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
| |
* transcode.c (trans_open_i): just record from and to.
(rb_trans_open): load transcodings.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18531 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
| |
* tool/transcode-tblgen.rb: 8bit byte of ASCII-8BIT is a valid
(but unique to ASCII-8BIT) character.
* transcode.c (rb_eConversionUndefined): new error.
(rb_eInvalidByteSequence): ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18524 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
resetting a state of stateful encoding.
* enc/trans/iso2022.trans (rb_EUC_JP_to_ISO_2022_JP): specify
finish_eucjp_to_iso2022jp for resetstate_func.
* tool/transcode-tblgen.rb: specify NULL for resetstate_func.
* transcode.c (output_replacement_character): call resetstate_func
before appending the replacement character.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18503 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
| |
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18501 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(rb_trans_elem_t): new type.
(rb_trans_t): new type.
* transcode.c (transcode_dispatch_cb): removed.
(transcode_dispatch): removed.
(rb_transcoding_result_t): moved to rb_trans_result_t in
transcode_data.h.
(transcode_restartable0): goto follow_info when FUNsi.
(rb_transcoding_open): use get_transcoder_entry.
(rb_trans_open): new function.
(rb_trans_conv): ditto.
(rb_trans_close): ditto.
(trans_open_i): ditto.
(trans_sweep): ditto.
(more_output_buffer): take rb_trans_t instead of rb_transcoding as
an argument.
(transcode_loop): take from_encoding and to_encoding instead of tr
as arguments. use rb_trans_open/rb_trans_conv/rb_trans_close.
(str_transcode): don't use transcode_dispatch.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18498 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
| |
readlen -> recognized_len.
feedlen -> readagain_len.
* transcode.c: follow the field renaming.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18481 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* transcode.c (load_transcoder): extracted from transcode_dispatch_cb.
(rb_transcoding_result_t): renamed from transcode_result_t.
(rb_transcoding_open): new function.
(rb_transcoding_convert): ditto.
(rb_transcoding_close): ditto.
(transcode_loop): use rb_transcoding_open, rb_transcoding_convert
and rb_transcoding_close.
(str_transcode): don't need rb_transcoding.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18474 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* transcode.c (transcode_restartable0): renamed from
transcode_restartable.
save input buffer into feed buffer if next character is started the
point before input buffer. for example, "\x00\xd8\x01" then "\x02"
in UTF-16LE. \x02 causes invalid and next character is started from
\x01.
(transcode_restartable): new function to call
transcode_restartable0. if feed buffer is not empty, convert it at
first.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18467 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
as parameters.
(more_output_buffer): ditto.
(str_transcoding_resize): argument changed from rb_transcoding* to
VALUE.
(str_transcode): call transcode_loop with destination string and its
resize function.
* transcode_data.h (rb_transcoding): move ruby_string_dest and
flush_func to transcode_loop parameters.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18458 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
| |
* transcode.c (transcode_restartable): save/restore next_info and
next_byte.
sync readlen and in_p when invalid.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18454 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
transcoding.
(rb_transcoder): add max_input field.
from_unit_length field is renamed to input_unit_length.
* tool/transcode-tblgen.rb: generate max_input field.
* enc/trans/iso2022.erb.c: follow rb_transcoder change.
* enc/trans/utf_16_32.erb.c: ditto.
* transcode.c (PARTIAL_INPUT): new constant.
(transcode_char_start): new function.
(transcode_result_t): new type.
(transcode_restartable): new function.
(more_output_buffer): new function.
(transcode_loop): use transcode_restartable.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18452 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
from_utf8 field removed.
* tool/transcode-tblgen.rb: generate offsets range.
follow rb_transcoder change.
* transcode.c (transcode_loop): don't use from_utf8.
make invalid region from_unit_length wise.
* enc/trans/iso2022.erb.c: follow rb_transcoder and
transcode_generate_node change.
* enc/trans/utf_16_32.erb.c: follow rb_transcoder and
transcode_generate_node change.
explicit :invalid map removed.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18445 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(rb_transcoder): preprocessor and postprocessor field removed.
change arguments of func_ii, func_si, func_io and func_so.
new field "finish_func".
* tool/transcode-tblgen.rb: make FUNii, FUNsi and FUNio
generatable.
* transcode.c (transcoder_lib_table): removed.
(transcoder_table): change structure.
(transcoder_key): removed because the above structure change.
(make_transcoder_entry): new function.
(get_transcoder_entry): ditto.
(rb_register_transcoder): follow the structure change.
(declare_transcoder): ditto.
(transcode_search_path): new function for breadth first search to
find a list of converters.
(transcode_search_path_i): new function.
(transcode_dispatch_cb): ditto.
(transcode_dispatch): use transcode_search_path.
(transcode_loop): follow the argument change.
(str_transcode): preprocessor and postprocessor stuff removed.
* enc/trans/iso2022.erb.c: new file. ISO-2022-JP conversion
re-implemented.
* enc/trans/japanese.erb.c: ISO-2022-JP stuff removed.
nute(23:52:53)% head -40 ChangeLog
Thu Aug 7 23:43:11 2008 Tanaka Akira <akr@fsij.org>
* transcode_data.h (rb_transcoding): new field "stateful".
(rb_transcoder): preprocessor and postprocessor field removed.
change arguments of func_ii, func_si, func_io and func_so.
new field "finish_func".
* tool/transcode-tblgen.rb: make FUNii, FUNsi and FUNio
generatable.
* transcode.c (transcoder_lib_table): removed.
(transcoder_table): change structure.
(transcoder_key): removed because the above structure change.
(make_transcoder_entry): new function.
(get_transcoder_entry): ditto.
(rb_register_transcoder): follow the structure change.
(declare_transcoder): ditto.
(transcode_search_path): new function for breadth first search to
find a list of converters.
(transcode_search_path_i): new function.
(transcode_dispatch_cb): ditto.
(transcode_dispatch): use transcode_search_path.
(transcode_loop): follow the argument change.
(str_transcode): preprocessor and postprocessor stuff removed.
* enc/trans/iso2022.erb.c: new file. ISO-2022-JP conversion
re-implemented.
* enc/trans/japanese.erb.c: ISO-2022-JP stuff removed.
* enc/trans/utf_16_32.erb.c: follow argument change of FUNso.
[ruby-dev:35798]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18419 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
| |
exception, would be changed later.
* enc/trans/japanese.c (UNSUPPORTED_MODE): unsupported mode transition
exception.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18363 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|
|
|
|
|
|
|
|
|
|
|
|
| |
* transcode.c (str_transcode): rb_str_set_len() sets a delimiter.
* transcode_data.h (rb_transcoder): constified preprocessor and
postprocessor input.
* enc/trans/japanese.c: ditto.
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18066 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
|