Add multiple fixes for ffc key generation using invalid p,q,g parameters.

Fixes #11864

- The dsa keygen assumed valid p, q, g values were being passed. If this is not correct then it is
  possible that dsa keygen can either hang or segfault.
  The fix was to do a partial validation of p, q, and g inside the keygen.
- Fixed a potential double free in the dsa keypair test in the case when in failed (It should never fail!).
  It freed internal object members without setting them to NULL.
- Changed the FFC key validation to accept 1024 bit keys in non fips mode.
- Added tests that use both the default provider & fips provider to test these cases.

Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/12176)

1 files changed, 46 insertions, 0 deletions

diff --git a/test/recipes/30-test_evp_libctx.t b/test/recipes/30-test_evp_libctx.t
new file mode 100644
index 0000000000..8fcc71a1cd
--- /dev/null
+++ b/test/recipes/30-test_evp_libctx.t
@@ -0,0 +1,46 @@
+#! /usr/bin/env perl
+# Copyright 2020 The OpenSSL Project Authors. All Rights Reserved.
+#
+# Licensed under the Apache License 2.0 (the "License").  You may not use
+# this file except in compliance with the License.  You can obtain a copy
+# in the file LICENSE in the source distribution or at
+# https://www.openssl.org/source/license.html
+
+use strict;
+use warnings;
+
+use OpenSSL::Test qw(:DEFAULT bldtop_dir srctop_dir srctop_file bldtop_file);
+use OpenSSL::Test::Utils;
+
+BEGIN {
+    setup("test_evp_libctx");
+}
+
+my $no_fips = disabled('fips') || ($ENV{NO_FIPS} // 0);
+
+use lib srctop_dir('Configurations');
+use lib bldtop_dir('.');
+use platform;
+
+my $infile = bldtop_file('providers', platform->dso('fips'));
+# If no fips then run the test with no extra arguments.
+my @test_args = ( );
+
+plan tests =>
+    ($no_fips ? 0 : 1)          # FIPS install test
+    + 1;
+
+unless ($no_fips) {
+    @test_args = ("-config", srctop_file("test","fips.cnf"),
+                  "-provider", "fips");
+
+    ok(run(app(['openssl', 'fipsinstall',
+               '-out', bldtop_file('providers', 'fipsmodule.cnf'),
+               '-module', $infile,
+               '-provider_name', 'fips', '-mac_name', 'HMAC',
+               '-macopt', 'digest:SHA256', '-macopt', 'hexkey:00',
+               '-section_name', 'fips_sect'])),
+       "fipsinstall");
+}
+
+ok(run(test(["evp_libctx_test", @test_args])), "running evp_libctx_test");