aboutsummaryrefslogtreecommitdiffstats
path: root/test/testlib
Commit message (Collapse)AuthorAgeFilesLines
* Create Certificate messages in TLS1.3 formatMatt Caswell2017-01-061-3/+13
| | | | | | | Also updates TLSProxy to be able to understand the format and parse the contained extensions. Reviewed-by: Rich Salz <rsalz@openssl.org> (Merged from https://github.com/openssl/openssl/pull/2020)
* Add a test to check the EC point formats extension appears when we expectMatt Caswell2016-12-291-1/+4
| | | | | | | | | The previous commit fixed a bug where the EC point formats extensions did not appear in the ServerHello. This should have been caught by 70-test_sslmessages but that test never tries an EC ciphersuite. This updates the test to do that. Reviewed-by: Richard Levitte <levitte@openssl.org> (Merged from https://github.com/openssl/openssl/pull/2153)
* Test framework: Add the possibility to have a test specific data dirRichard Levitte2016-12-091-0/+36
| | | | | | | | This data directory is formed automatically by taking the recipe name and changing '.t' to '_data'. Files in there can be reached with the new function data_file() Reviewed-by: Rich Salz <rsalz@openssl.org> (Merged from https://github.com/openssl/openssl/pull/2027)
* Move the checkhandshake.pm module into test/testlibMatt Caswell2016-12-081-0/+128
| | | | | | | | | | | Move this module into the same place as other test helper modules. It simplifies the code and keeps like things together. Perl changes reviewed by Richard Levitte. Non-perl changes reviewed by Rich Salz Reviewed-by: Rich Salz <rsalz@openssl.org> Reviewed-by: Richard Levitte <levitte@openssl.org>
* OpenSSL::Test - small fixupRichard Levitte2016-10-191-7/+2
| | | | Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/1686)
* OpenSSL::Test cleanup - no forward declarations neededRichard Levitte2016-10-191-15/+0
| | | | Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/1686)
* Add documentation of internal OpenSSL::Test functionsRichard Levitte2016-10-191-1/+43
| | | | | | Also, fix __wrap_cmd so it doesn't return unnecessary empty strings Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/1686)
* Make OpenSSL::Test a bit more flexibleRichard Levitte2016-10-191-109/+164
| | | | | | | | | | | | | | | | So far, apps and test programs, were a bit rigidely accessible as executables or perl scripts. But what about scripts in some other language? Or what about running entirely external programs? The answer is certainly not to add new functions to access scripts for each language or wrapping all the external program calls in our magic! Instead, this adds a new functions, cmd(), which is useful to access executables and scripts in a more generalised manner. app(), test(), fuzz(), perlapp() and perltest() are rewritten in terms of cmd(), and serve as examples how to do something similar for other scripting languages, or constrain the programs to certain directories. Reviewed-by: Matt Caswell <matt@openssl.org> (Merged from https://github.com/openssl/openssl/pull/1686)
* Run the fuzzing corpora as tests.Ben Laurie2016-07-011-1/+20
| | | | | Reviewed-by: Richard Levitte <levitte@openssl.org> Reviewed-by: Rich Salz <rsalz@openssl.org>
* OpenSSL::Test: Fix directory calculations in __cwd()Richard Levitte2016-06-221-1/+6
| | | | | | | The previous fix contained a mistake, where any absolute path in %directories would be cleared away instead of just being left alone. Reviewed-by: Rich Salz <rsalz@openssl.org>
* OpenSSL::Test: Fix directory calculations in __cwd()Richard Levitte2016-06-211-8/+18
| | | | | | | | | | | | | We recalculate the location of the directories we keep track of. However, we did so after having moved to the new directory already, so the data we did the calculations from were possibly not quite correct. This change moves the calculations to happen before moving to the new directory. This issue is sporadic, and possibly dependent on the platform. Reviewed-by: Matt Caswell <matt@openssl.org>
* tests: fix the shutting up of the shellRichard Levitte2016-06-061-4/+6
| | | | | | | | If we're going to redirect STDERR and STDOUT, it's better to do it by the book. This fix is a straight copy of examples in the perlfunc manual. Reviewed-by: Matt Caswell <matt@openssl.org>
* tests: Shut the shell up unless verboseRichard Levitte2016-06-061-0/+17
| | | | | | | In rare cases, the shell we run test programs in may have complaints. Shut those up unless testing verbosely. Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
* testlib/OpenSSL/Test.pm: address 5.10 warnings.Andy Polyakov2016-05-061-4/+4
| | | | Reviewed-by: Richard Levitte <levitte@openssl.org>
* Copyright consolidation; .pm and ConfigureRich Salz2016-04-203-1/+22
| | | | Reviewed-by: Richard Levitte <levitte@openssl.org>
* OpenSSL::Test: when moving directory, affect env as wellRichard Levitte2016-04-071-0/+22
| | | | | | | | | The environment variables TOP, SRCTOP, BLDTOP, ... are used to affect the testing framework. However, subprocesses may want to use them as well, and therefore need their values corrected when we move to a different directory. Reviewed-by: Emilia Käsper <emilia@openssl.org>
* Enhance OpenSSL::Test::cmdstr to give cmd string variantsRichard Levitte2016-04-021-2/+25
| | | | | | | | | | | Within OpenSSL::Test, all commands end up existing in two variants, one that has redirections that are needed internally to work well together with the test harness, and one without those redirections. Depending on what the result is going to be used for, the caller may want one for or the other, so we give them the possibility. Reviewed-by: Rich Salz <rsalz@openssl.org>
* OpenSSL::Test: have cmdstr() return the true command.Richard Levitte2016-03-311-2/+2
| | | | | | On the other hand, have run() display the display variant. Reviewed-by: Rich Salz <rsalz@openssl.org>
* When looking for executables, don't forget the extensionRichard Levitte2016-03-311-6/+13
| | | | | | | | Because some operating systems have executable extensions, typically ".exe", we need to append it when looking for files in test() and app() (or rather, their subroutines). Reviewed-by: Rich Salz <rsalz@openssl.org>
* When verbosity is requested, OpenSSL::Test::run() displays the commandRichard Levitte2016-03-301-0/+3
| | | | | | It displays the command's exit code as well. Reviewed-by: Emilia Käsper <emilia@openssl.org>
* Have OpenSsl..Test::app() and friends look for file in source as wellRichard Levitte2016-03-301-9/+21
| | | | | | | | If the command file that app(), test(), perlapp(9 and perltest() are looking for doesn't exist in the build tree, look for it in the source tree as well. Reviewed-by: Emilia Käsper <emilia@openssl.org>
* Add the option 'interpreter_args' to perlapps() and perltest()Richard Levitte2016-03-301-3/+21
| | | | | | | The intention with that option is to allow extra flags to the perl interpreter itself. Reviewed-by: Emilia Käsper <emilia@openssl.org>
* Make OpenSSL::Test::run() sensitive to signalsRichard Levitte2016-03-181-2/+7
| | | | | | | | | | | | | | | $? in perl gets the status value from wait(2), which is a word with the exit code in the upper half and the number of a raised signal in the lower half. OpenSSL::Test::run() ignored the signal half up until now. With this change, we recalculate an exit code the same way the Unix shells do, using this formula: ($? & 0x7f) ? ($? & 0x7f)|0x80 : ($? >> 8); Reviewed-by: Rich Salz <rsalz@openssl.org> Reviewed-by: Emilia Käsper <emilia@openssl.org>
* Make OpenSSL::Test::setup() a bit more forgivingRichard Levitte2016-03-071-2/+8
| | | | | | | | | It was unexpected that OpenSSL::Test::setup() should be called twice by the same recipe. However, that may happen if a recipe combines OpenSSL::Test and OpenSSL::Test::Simple, which can be a sensible thing to do. Therefore, we now allow it. Reviewed-by: Rich Salz <rsalz@openssl.org>
* Rethink logging of test recipesRichard Levitte2016-02-121-31/+8
| | | | | | | | | | | | | | | | | | | | | | | | | The logging that was performed in OpenSSL::Test was initially set up as a means not to let messages that test programs write to STDERR get displayed when a test isn't running in verbose mode. However, the way it was implemented, it meant that those messages were never displayed, and you had to look in a test log. This also meant that output to STDERR and output to STDOUT got broken apart, which isn't optimal. So, we remove the whole test log file implementation, and instead, we're sending STDERR to the null device unless one of these conditions apply: - the test recipe already redirects stderr. Just let it. - the environment variable HARNESS_ACTIVE is undefined, meaning the recipe is run directly as a perl script instead of being harnessed by Test::Harness - the environment variable HARNESS_VERBOSE is set. Getting a full log of the tests now becomes as simple as this: HARNESS_VERBOSE=yes make test 2>&1 | tee tests.log Reviewed-by: Rich Salz <rsalz@openssl.org>
* unified build scheme: adjust test framework for out of source build treeRichard Levitte2016-02-092-43/+106
| | | | | | | | | | | | | | | | To be able to run tests when we've built in a directory other than the source tree, the testing framework needs a few adjustments. test/testlib/OpenSSL/Test.pm needs to know where it can find shlib_wrap.sh, and a number of other tests need to be told a different place to find engines than what they may be able to figure out on their own. Relying to $TOP is not enough, $SRCTOP and $BLDTOP can be used as an alternative. As part of this change, top_file and top_dir are removed and srctop_file, bldtop_file, srctop_dir and bldtop_dir take their place. Reviewed-by: Ben Laurie <ben@openssl.org>
* Use File::Path::rmtree rather than File::Path::remove_treeRichard Levitte2016-02-081-3/+3
| | | | | | | | Just like File::Path::make_path, File::Path::remove_tree didn't show up before File::Path 2.06 / perl v5.10.1, so we prefer the legacy function here as well. Reviewed-by: Andy Polyakov <appro@openssl.org>
* Initialise with -1 rather than 1Richard Levitte2016-02-051-1/+1
| | | | | | A small typo crept in. Reviewed-by: Kurt Roeckx <kurt@openssl.org>
* Add checks for IPv4 and IPv6 in OpenSSL::Test::Utils and use themRichard Levitte2016-02-041-1/+77
| | | | | | | | | | | | | This uilds on the same way of checking for availability as we do in TLSProxy. We use all IP factories we know of, starting with those who know both IPv6 and IPv4 and ending with the one that only knows IPv4 and cache their possible success as foundation for checking the available of each IP domain. 80-test_ssl.t has bigger chances of working on platforms that do not run both IP domains. Reviewed-by: Rich Salz <rsalz@openssl.org>
* Have OpenSSL::Test::Utils::available_protocols load configdata as wellRichard Levitte2016-02-041-0/+1
| | | | | | | Otherwise, it could typically always return an empty list, since it's often called first if at all. Reviewed-by: Ben Laurie <ben@openssl.org>
* When checking if there's a VMS directory spec, don't forget the possible deviceRichard Levitte2016-01-301-1/+1
| | | | Reviewed-by: Rich Salz <rsalz@openssl.org>
* Be careful when applying EXE_SHELLRichard Levitte2016-01-271-4/+9
| | | | | | | $EXE_SHELL should only be used with out own programs, not with surrounding programs such as the perl interpreter. Reviewed-by: Matt Caswell <matt@openssl.org>
* Have OpenSSL::Test handle perl scripts like any programRichard Levitte2016-01-261-11/+77
| | | | | | | | | | | | | | | Since we're building some of our perl scripts and the result might not end up in apps/ (*), we may need to treat them like the compile programs we use for testing. This introduces perlapp() and perltest(), which behave like app() and test(), but will add the perl executable in the command line. ----- (*) For example, with a mk1mf build, the result will end up in $(BIN_D) Reviewed-by: Matt Caswell <matt@openssl.org>
* Use Configure's @disablables and %disabled through configdata.pmRichard Levitte2016-01-261-34/+87
| | | | | | | | Enhances the routines in OpenSSL::Test::Utils for checking disabled stuff to get their information directly from Configure instead of 'openssl list -disabled'. Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
* Make tests use configdata.pm rather than parsing MakefileRichard Levitte2016-01-251-2/+20
| | | | Reviewed-by: Rich Salz <rsalz@openssl.org>
* Fix OpenSSL::Test::Simple to take more than one algorithmRichard Levitte2016-01-251-4/+10
| | | | | | | Some test programs may depend on more than just one TLS version, for example. Reviewed-by: Viktor Dukhovni <viktor@openssl.org>
* In __cwd, make sure the given directory is seen as such and not a fileRichard Levitte2016-01-141-2/+1
| | | | | | On Unixly platforms, this doesn't matter. On VMS, it does. Reviewed-by: Rich Salz <rsalz@openssl.org>
* Add a directry spec for mcr if there is noneRichard Levitte2016-01-141-1/+1
| | | | | | | | | | On VMS, the command MCR will assume SYS$SYSTEM: when the first argument lacks a directory spec. So for programs in the current directory, we add [] to tell MCR it is in the current directory. It's the same as having ./ at the start of a program on Unix so the shell doesn't start looking along $PATH. Reviewed-by: Rich Salz <rsalz@openssl.org>
* testlib/OpenSSL/Test.pm: remove redundant 'cmd /c', MSWin32 Perl can take ↵Andy Polyakov2015-10-131-2/+0
| | | | | | care of itself. Reviewed-by: Richard Levitte <levitte@openssl.org>
* Test suite: minimal required to get mingw 'make test' work under Linux.Andy Polyakov2015-10-132-3/+11
| | | | | | (part by Alessandro Ghedini) Reviewed-by: Richard Levitte <levitte@openssl.org>
* Simplify Simple.pm further, and make it more verboseRichard Levitte2015-09-201-5/+4
| | | | Reviewed-by: Stephen Henson <steve@openssl.org>
* typoDr. Stephen Henson2015-09-201-1/+1
| | | | Reviewed-by: Richard Levitte <levitte@openssl.org>
* Update Simple.pm to use disabled()Dr. Stephen Henson2015-09-201-1/+2
| | | | Reviewed-by: Richard Levitte <levitte@openssl.org>
* Add Utils.pmDr. Stephen Henson2015-09-201-0/+84
| | | | | | | | Add Utils.pm for test utilities. This currently just contains one function: disabled which checks if a feature is disabled based on the output of openssl list -disabled Reviewed-by: Richard Levitte <levitte@openssl.org>
* Make sure the temporary error log resides in a well defined locationRichard Levitte2015-09-091-1/+2
| | | | | | | | | | | | | | | | | | | If a test recipe does something like this: indir "foo.$$" => sub { chmod 0500, File::Spec->curdir(); ok(run(app(["something"]))); } we get a problem, because we were storing the temporary stderr file in the current directory at all times (so while inside the 'indir', we would attemp to store it in "foo.$$"). So, change our ways to always store that temporary file in the exact same location, defined by the environment variable RESULT_D, or failing that TEST_D, or failing that $TOP/test. Reviewed-by: Rich Salz <rsalz@openssl.org>
* Small fix in OpenSSL::TestRichard Levitte2015-09-071-2/+4
| | | | | | | | Be careful when shifting in a function argument, you end up changing the caller's value. Instead, when it is an array, make a shallow copy and shift in that instead. Reviewed-by: Rich Salz <rsalz@openssl.org>
* Document OpenSSL::Test and OpenSSL::Test::SimpleRichard Levitte2015-09-072-183/+577
| | | | | | | For OpenSSL::Test, it meant rearranging the code to better suite the structure of the documentation. Reviewed-by: Rich Salz <rsalz@openssl.org>
* Change OpenSSL::Test to be an extension of Test::MoreRichard Levitte2015-09-072-4/+5
| | | | | | | | | | | It became tedious as well as error prone to have all recipes use Test::More as well as OpenSSL::Test. The easier way is to make OpenSSL::Test an extension of Test::More, thereby having all version checks as well as future checks firmly there. Additionally, that allows us to extend existing Test::More functions if the need would arise. Reviewed-by: Rich Salz <rsalz@openssl.org>
* New feature: STOPTESTRichard Levitte2015-09-071-0/+11
| | | | | | | | When the environment variable STOPTEST is defined (with any value other than the empty string), the test machinery in OpenSSL::Test goes into a different mode that will stop all testing at the end of a failing recipe. Reviewed-by: Rich Salz <rsalz@openssl.org>
* Add version numbers on some modules we use.Richard Levitte2015-09-072-3/+3
| | | | Reviewed-by: Rich Salz <rsalz@openssl.org>