Raise an exception if the IO object passed to SSLSocket isn't a file

SSLSocket#connect eventually calls `GetOpenFile` in order to get the
underlying file descriptor for the IO object passed in on
initialization.  `GetOpenFile` assumes that the Ruby object passed in is
a T_FILE object and just casts it to a T_FILE without any checks.  If
you pass an object that *isn't* a T_FILE to that function, the program
will segv.

Since we assume the IO object is a file in the `connect` method, this
commit adds a `CheckType` in the initialize method to ensure that the IO
object is actually a T_FILE.  If the object *isn't* a T_FILE, this class
will segv on `connect`, so I think this is a backwards compatible
change.

2 files changed, 12 insertions, 0 deletions

diff --git a/ext/openssl/ossl_ssl.c b/ext/openssl/ossl_ssl.c
index d6d321e4..1de0f989 100644
--- a/ext/openssl/ossl_ssl.c
+++ b/ext/openssl/ossl_ssl.c
@@ -1527,6 +1527,7 @@ ossl_ssl_initialize(int argc, VALUE *argv, VALUE self)
 
     if (rb_respond_to(io, rb_intern("nonblock=")))
 	rb_funcall(io, rb_intern("nonblock="), 1, Qtrue);
+    Check_Type(io, T_FILE);
     rb_ivar_set(self, id_i_io, io);
 
     ssl = SSL_new(ctx);
diff --git a/test/openssl/test_ssl.rb b/test/openssl/test_ssl.rb
index 2a52f272..c575134d 100644
--- a/test/openssl/test_ssl.rb
+++ b/test/openssl/test_ssl.rb
@@ -4,6 +4,17 @@ require_relative "utils"
 if defined?(OpenSSL)
 
 class OpenSSL::TestSSL < OpenSSL::SSLTestCase
+  def test_bad_socket
+    bad_socket = Struct.new(:sync).new
+    assert_raises TypeError do
+      socket = OpenSSL::SSL::SSLSocket.new bad_socket
+      # if the socket is not a T_FILE, `connect` will segv because it tries
+      # to get the underlying file descriptor but the API it calls assumes
+      # the object type is T_FILE
+      socket.connect
+    end
+  end
+
   def test_ctx_options
     ctx = OpenSSL::SSL::SSLContext.new