aboutsummaryrefslogtreecommitdiffstats
path: root/transcode.c
Commit message (Collapse)AuthorAgeFilesLines
...
* * transcode.c (enc_arg): extracted from str_transcode_enc_args.akr2008-09-051-16/+27
| | | | | | | (str_transcode_enc_args): use enc_arg. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19154 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (rb_transcoder): rename fields:akr2008-09-041-22/+22
| | | | | | | | | | from_encoding -> src_encoding, to_encoding -> dst_encoding. * transcode.c: follow the renaming. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19142 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c: variables renamed for consistency.akr2008-09-041-55/+55
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19141 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* rdoc update.akr2008-09-041-14/+19
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19134 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (econv_init): accept an integer as 3rd argument as well.akr2008-09-041-4/+15
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19132 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (sym_partial_input): new variable.akr2008-09-041-11/+30
| | | | | | | (econv_primitive_convert): accept a hash as 5th argument as well. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19131 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (sym_universal_newline_decoder): new variable.akr2008-09-041-31/+44
| | | | | | | | | | (sym_crlf_newline_encoder): ditto. (sym_cr_newline_encoder): ditto. (econv_opts): check newline converter options. (econv_init): make 3rd argument hash/nil only. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19130 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (PType): defined unconditionaly.akr2008-09-041-1/+0
| | | | | | | | * transcode.c (PType): don't define here. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19127 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (ECONV_INVALID_IGNORE): removed becauseakr2008-09-041-16/+2
| | | | | | | | | | | | | | | it tend to cause security problem. If the behaviour is really required, ECONV_INVALID_REPLACE with empty string can be used. For example, CVE-2006-2313, CVE-2008-1036, [ruby-core:15645] (ECONV_UNDEF_IGNORE): ditto. * transcode.c (rb_econv_convert): follow the above change. (econv_opts): ditto. (Init_transcode): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19123 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/io.h (rb_io_t): new fields: encs.ecopts andakr2008-09-031-21/+86
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | writeconv_pre_ecopts. (MakeOpenFile): initialize them. * include/ruby/encoding.h (rb_str_transcode): take ecopts argument. (rb_econv_flags): removed. (rb_econv_prepare_opts): declared. (rb_econv_open_opts): declared. * io.c (make_writeconv): use rb_econv_open_opts. (make_readconv): ditto. (io_fwrite): follow rb_str_transcode change. (rb_io_extract_modeenc): use rb_econv_prepare_opts. (rb_file_open_generic): initialize encs.ecopts. (rb_file_open_internal): ditto. (rb_io_reopen): ditto. (argf_ecopts): defined. (argf_next_argv): set encs.ecopts. (io_encoding_set): use rb_econv_prepare_opts. (argf_set_encoding): set argf_ecopts. * gc.c (gc_mark_children): mark encs.ecopts and writeconv_pre_ecopts in T_FILE. * transcode.c (transcode_loop): take ecopts argument. use rb_econv_open_opts. (rb_econv_flags): removed. (rb_econv_prepare_opts): defined. (rb_econv_open_opts): defined. (str_transcode0): take ecopts. (str_transcode): use rb_econv_prepare_opts. (rb_str_transcode): take ecopts. (econv_init): accept hash argument. (econv_insert_output): follow rb_str_transcode change. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19109 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_econv_set_replacemenet): declared.akr2008-09-031-10/+144
| | | | | | | | | | | | | | | | | * transcode.c (rb_econv_t): new fields: replacement_str, replacement_len, replacement_enc and replacement_allocated. (get_replacement_character): make len as size_t. (rb_econv_open_by_transcoder_entries): initialize the new fields. (rb_econv_close): deallocate replacement_str if it allocated. (make_replacement): new function. (output_replacement_character): use make_replacement. (rb_econv_set_replacemenet): defined. (econv_get_replacement): new method. (econv_set_replacement): new method. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19108 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (transcode_restartable0): avoid VC++6's bug.usa2008-09-031-8/+8
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19105 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_econv_option_t): removed. Sinceakr2008-09-031-69/+56
| | | | | | | | | | | | | | | rb_econv_option_t has only one field, int flags, rb_econv_option_t is replaced by int. * include/ruby/io.h: follow the above change. * io.c: ditto. * transcode.c: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19103 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (rb_transcoding): moved from transcode_data.h.akr2008-09-031-0/+42
| | | | | | | | | | (TRANSCODING_READBUF): ditto. (TRANSCODING_WRITEBUF): ditto. (TRANSCODING_STATE_EMBED_MAX): ditto. (TRANSCODING_STATE): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19097 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (rb_transcoding): remove stateful field.akr2008-09-031-9/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | add state field. (TRANSCODING_STATE): defined. (rb_transcoder): add fields: state_size, state_init_func, state_fini_func. change rb_transcoding* argument to void*. * transcode.c (transcode_restartable0): use TRANSCODING_STATE for first arguments of transcoder functions. (rb_transcoding_open_by_transcoder): initialize state field. (rb_transcoding_close): finalize state field. * tool/transcode-tblgen.rb: provide state size/init/fini. * enc/trans/newline.trans (universal_newline_init): defined. (fun_so_universal_newline): take void* as a state pointer. (rb_universal_newline): provide state size/init/fini. (rb_crlf_newline): ditto. (rb_cr_newline): ditto. * enc/trans/iso2022.trans (iso2022jp_init): defined. (fun_si_iso2022jp_to_eucjp): take void* as a state pointer. (fun_so_iso2022jp_to_eucjp): ditto. (fun_so_eucjp_to_iso2022jp): ditto. (iso2022jp_reset_sequence_size): ditto. (finish_eucjp_to_iso2022jp): ditto. (rb_ISO_2022_JP_to_EUC_JP): provide state size/init/fini. (rb_EUC_JP_to_ISO_2022_JP): ditto. * enc/trans/utf_16_32.trans (fun_so_from_utf_16be): take void* as a state pointer. (fun_so_to_utf_16be): ditto. (fun_so_from_utf_16le): ditto. (fun_so_to_utf_16le): ditto. (fun_so_from_utf_32be): ditto. (fun_so_to_utf_32be): ditto. (fun_so_from_utf_32le): ditto. (fun_so_to_utf_32le): ditto. (rb_from_UTF_16BE): provide state size/init/fini. (rb_to_UTF_16BE): ditto. (rb_from_UTF_16LE): ditto. (rb_to_UTF_16LE): ditto. (rb_from_UTF_32BE): ditto. (rb_to_UTF_32BE): ditto. (rb_from_UTF_32LE): ditto. (rb_to_UTF_32LE): ditto. * enc/trans/japanese.trans (fun_so_eucjp2sjis): take void* as a state pointer. (fun_so_sjis2eucjp): ditto. (rb_eucjp2sjis): provide state size/init/fini. (rb_sjis2eucjp): provide state size/init/fini. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19096 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (WORDINDEX_SHIFT_BITS): defined.akr2008-09-031-1/+1
| | | | | | | | | | | | | (WORDINDEX2INFO): defined. (INFO2WORDINDEX): defined. * tool/transcode-tblgen.rb: use WORDINDEX2INFO. * transcode.c: use INFO2WORDINDEX. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19093 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* refactor last change.akr2008-09-031-2/+2
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19091 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (BL_MIN_BYTE): defined.akr2008-09-031-5/+8
| | | | | | | | | | | | (BL_MAX_BYTE): defined. (BL_OFFSET): defined. (BL_ACTION): defined. (transcode_restartable0): use BL_MIN_BYTE, BL_MAX_BYTE and BL_ACTION. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19090 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (BYTE_ADDR): defined.akr2008-09-031-4/+5
| | | | | | | | | (WORD_ADDR): ditto. (BL_BASE): use BYTE_ADDR and WORD_ADDR. (BL_INFO): use WORD_ADDR. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19089 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (base_element): removed.akr2008-09-021-4/+6
| | | | | | | | | | | | | | | | | | (BYTE_LOOKUP): removed. (BYTE_LOOKUP_BASE): don't cast. (BYTE_LOOKUP_INFO): ditto. (PType): unsigned int, instead of uintptr_t. (rb_transcoding): change type of next_field, conv_tree_start and word_array. * tool/transcode-tblgen.rb: generate word_array as array of unsigned int. * transcode.c (transcode_restartable0): follow the above type change. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19070 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/transcode-tblgen.rb: record infos and BYTE_LOOKUPs as index ofakr2008-09-011-3/+4
| | | | | | | | | | | | | | word_array to avoid relocation. * transcode.c (transcode_restartable0): add word_array to get infos and BYTE_LOOKUPs. * transcode_data.h (BYTE_LOOKUP_INFO): change return type to uintptr_t. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19045 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * tool/transcode-tblgen.rb: record offsets array as index ofakr2008-09-011-2/+3
| | | | | | | | | | | | | | | | | | | | | byte_array to avoid relocation. * transcode.c (transcode_restartable0): add byte_array to get offsets array. * transcode_data.h (BYTE_LOOKUP_BASE): change return type to uintptr_t. (rb_transcoder): add fields: byte_array, word_array and word_size. * enc/trans/newline.trans: follow rb_transcoder change. * enc/trans/iso2022.trans: ditto. * enc/trans/utf_16_32.trans: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19043 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (BYTE_LOOKUP): change to uintptr_t array.akr2008-09-011-1/+1
| | | | | | | | | | | | | | | | | | | | | (BYTE_LOOKUP_BASE): follow the type change. (BYTE_LOOKUP_INFO): ditto. (PType): ditto. (rb_transcoding): ditto. * tool/transcode-tblgen.rb: follow the type change. * transcode.c: ditto. * enc/trans/newline.trans: ditto. * enc/trans/iso2022.trans: ditto. * enc/trans/utf_16_32.trans: ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19038 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode_data.h (BYTE_LOOKUP_BASE): abstract accessor forakr2008-09-011-3/+3
| | | | | | | | | | | | BYTE_LOOKUP. (BYTE_LOOKUP_INFO): ditto. * transcode.c (transcode_restartable0): use BYTE_LOOKUP_BASE and BYTE_LOOKUP_INFO. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19037 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* rdoc update.akr2008-08-311-14/+33
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@19002 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* fix typos.akr2008-08-311-2/+2
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18999 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* rdoc update.akr2008-08-311-14/+121
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18996 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (transcode.c): set source_encoding andakr2008-08-311-6/+28
| | | | | | | | | destination_encoding as encoding object. (ecerr_source_encoding): new method. (ecerr_destination_encoding): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18993 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (make_econv_exception): rename instance variable namesakr2008-08-311-12/+12
| | | | | | | | | for storing encoding names. (ecerr_source_encoding_name): method renamed. (ecerr_destination_encoding_name): ditto. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18987 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (econv_putback): associate encoding to the result.akr2008-08-311-0/+28
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18986 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (econv_insert_output): raise ArgumentError on failure.akr2008-08-311-3/+23
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18984 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (econv_last_error): new method.akr2008-08-311-0/+34
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18980 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (econv_primitive_convert): make two arguments,akr2008-08-311-1/+6
| | | | | | | destination_byteoffset and destination_bytesize, optional. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18979 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (make_econv_exception): error message simplified.akr2008-08-311-4/+4
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18978 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (econv_putback): make max argument optional.akr2008-08-301-3/+5
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18967 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (rb_econv_putback): putback from last byte.akr2008-08-301-1/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18966 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (econv_putback): renamed from econv_primitive_putback.akr2008-08-301-2/+2
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18965 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (econv_insert_output): renamed fromakr2008-08-301-2/+2
| | | | | | | econv_primitive_insert_output. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18964 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (econv_primitive_errinfo): 5th element (partial_input)akr2008-08-301-15/+10
| | | | | | | | removed from result. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18962 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* rdoc update.akr2008-08-291-0/+11
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18929 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* rdoc update.akr2008-08-281-0/+1
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18914 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (econv_finish): new method.akr2008-08-281-0/+50
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18913 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (econv_convert): new method.akr2008-08-281-0/+54
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18912 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (econv_primitive_convert): accept nil asakr2008-08-281-7/+29
| | | | | | | destination_bytesize for unlimited destination size. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18911 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (econv_primitive_convert): accept nil as input for emptyakr2008-08-281-5/+17
| | | | | | | | input. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18910 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (sym_invalid_byte_sequence): new variable.akr2008-08-281-7/+23
| | | | | | | | | | | | | | | (sym_undefined_conversion): ditto. (sym_destination_buffer_full): ditto. (sym_source_buffer_empty): ditto. (sym_finished): ditto. (sym_output_followed_by_input): ditto. (sym_incomplete_input): ditto. (econv_result_to_symbol): use above variables. (Init_transcode): initialize above variables. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18908 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (transcode_loop): simplified.akr2008-08-261-11/+2
| | | | git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18876 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_econv_result_t): enumeration constant:akr2008-08-261-12/+42
| | | | | | | | | | | | | | | | | | | | | | econv_incomplete_input. * io.c (finish_writeconv): check econv_incomplete_input. * transcode.c (transcode_restartable0): return econv_incomplete_input for unexpected end of source buffer. (trans_sweep): check econv_incomplete_input. (rb_trans_conv): ditto. (rb_econv_convert0): ditto. (rb_econv_convert): ditto. (transcode_loop): ditto. (make_econv_exception): change message for econv_incomplete_input. (econv_result_to_symbol): return :incomplete_input for econv_incomplete_input. (ecerr_incomplete_input): new method. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18875 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * include/ruby/encoding.h (rb_econv_elem_t): move to transcode.cakr2008-08-261-0/+42
| | | | | | | | | | | (rb_econv_t): defined as an incomplete type. * transcode.c (rb_econv_elem_t): moved from encoding.h. (rb_econv_t): complete type defined. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18872 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
* * transcode.c (rb_econv_open): disable newline conversion for ASCIIakr2008-08-261-14/+33
| | | | | | | | incompatible encodings. (str_transcode0): don't need disable newline conversion here. git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@18870 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-30 13:31:22 +0000