hardenedbsd/HardenedBSD
1
0
Fork 0
mirror of https://git.hardenedbsd.org/hardenedbsd/HardenedBSD.git synced 2024-11-14 22:32:30 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity
3fe0a5d2f6
HardenedBSD/share/mk/atf.test.mk

112 lines
3.2 KiB
Makefile
Raw Normal View History

Enable ATF testing. Submitted by: Garrett Cooper Approved by: marcel (mentor)
2012-11-07 23:02:02 +01:00
# $FreeBSD$
#
Make bsd.test.mk the only public mk fragment for the building of tests. Change {atf,plain,tap}.test.mk to be internal implementation details of bsd.test.mk. Makefiles that build tests should now only include bsd.test.mk and declaratively specify what they want to build, without worrying about the internal implementation of the mk files. The reason for this change is to permit building test programs of different interfaces from a single directory, which is something I had a need for while porting tests over from src/tools/regression/. Additionally, this change makes it possible to perform some other requested changes to bsd.test.mk in an easier manner. Coming soon.
2014-03-14 09:56:19 +01:00
# You must include bsd.test.mk instead of this file from your Makefile.
#
Clearly split the logic to build ATF and plain tests apart. This change introduces a new plain.test.mk file that provides the build infrastructure to build test programs that don't use any framework. Most of the code previously in bsd.test.mk moves to plain.test.mk and atf.test.mk is extended with the missing pieces. In doing so, this change pushes all test program building logic to the various *.test.mk files instead of trying to reuse some tiny bits. In fact, this attempt to reuse some definitions makes the code harder to read and harder to extend. The clear benefit of this is that the interface of bsd.test.mk is now clearly delimited. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:48:49 +02:00
# Logic to build and install ATF test programs; i.e. test programs linked
# against the ATF libraries.
Enable ATF testing. Submitted by: Garrett Cooper Approved by: marcel (mentor)
2012-11-07 23:02:02 +01:00
Make bsd.test.mk the only public mk fragment for the building of tests. Change {atf,plain,tap}.test.mk to be internal implementation details of bsd.test.mk. Makefiles that build tests should now only include bsd.test.mk and declaratively specify what they want to build, without worrying about the internal implementation of the mk files. The reason for this change is to permit building test programs of different interfaces from a single directory, which is something I had a need for while porting tests over from src/tools/regression/. Additionally, this change makes it possible to perform some other requested changes to bsd.test.mk in an easier manner. Coming soon.
2014-03-14 09:56:19 +01:00
.if !target(__<bsd.test.mk>__)
.error atf.test.mk cannot be included directly.
.endif
Enable ATF testing. Submitted by: Garrett Cooper Approved by: marcel (mentor)
2012-11-07 23:02:02 +01:00
Clearly split the logic to build ATF and plain tests apart. This change introduces a new plain.test.mk file that provides the build infrastructure to build test programs that don't use any framework. Most of the code previously in bsd.test.mk moves to plain.test.mk and atf.test.mk is extended with the missing pieces. In doing so, this change pushes all test program building logic to the various *.test.mk files instead of trying to reuse some tiny bits. In fact, this attempt to reuse some definitions makes the code harder to read and harder to extend. The clear benefit of this is that the interface of bsd.test.mk is now clearly delimited. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:48:49 +02:00
# List of C, C++ and shell test programs to build.
#
# Programs listed here are built using PROGS, PROGS_CXX and SCRIPTS,
# respectively, from bsd.prog.mk. However, the build rules are tweaked to
# require the ATF libraries.
#
# Test programs registered in this manner are set to be installed into TESTSDIR
Feex a cuple of small typos
2018-07-27 12:44:38 +02:00
# (which should be overridden by the Makefile) and are not required to provide a
Clearly split the logic to build ATF and plain tests apart. This change introduces a new plain.test.mk file that provides the build infrastructure to build test programs that don't use any framework. Most of the code previously in bsd.test.mk moves to plain.test.mk and atf.test.mk is extended with the missing pieces. In doing so, this change pushes all test program building logic to the various *.test.mk files instead of trying to reuse some tiny bits. In fact, this attempt to reuse some definitions makes the code harder to read and harder to extend. The clear benefit of this is that the interface of bsd.test.mk is now clearly delimited. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:48:49 +02:00
# manpage.
ATF_TESTS_C?=
ATF_TESTS_CXX?=
ATF_TESTS_SH?=
Add the ZFS test suite It was originally written by Sun as part of the STF (Solaris test framework). They open sourced it in OpenSolaris, then HighCloud partially ported it to FreeBSD, and Spectra Logic finished the port. We also added many testcases, fixed many broken ones, and converted them all to the ATF framework. We've had help along the way from avg, araujo, smh, and brd. By default most of the tests are disabled. Set the disks Kyua variable to enable them. Submitted by: asomers, will, justing, ken, brd, avg, araujo, smh Sponsored by: Spectra Logic Corp, HighCloud
2018-02-23 17:31:00 +01:00
ATF_TESTS_KSH93?=
Clearly split the logic to build ATF and plain tests apart. This change introduces a new plain.test.mk file that provides the build infrastructure to build test programs that don't use any framework. Most of the code previously in bsd.test.mk moves to plain.test.mk and atf.test.mk is extended with the missing pieces. In doing so, this change pushes all test program building logic to the various *.test.mk files instead of trying to reuse some tiny bits. In fact, this attempt to reuse some definitions makes the code harder to read and harder to extend. The clear benefit of this is that the interface of bsd.test.mk is now clearly delimited. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:48:49 +02:00
.if !empty(ATF_TESTS_C)
PROGS+= ${ATF_TESTS_C}
Add the automatic generation of Kyuafile files. These files are generated from bsd.test.mk because kyua is able to run test programs implemented using different libraries/frameworks. In order to make this possible, this change also extends the various *.test.mk file to explicitly indicate the interface of every test program. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:50:56 +02:00
_TESTS+= ${ATF_TESTS_C}
Clearly split the logic to build ATF and plain tests apart. This change introduces a new plain.test.mk file that provides the build infrastructure to build test programs that don't use any framework. Most of the code previously in bsd.test.mk moves to plain.test.mk and atf.test.mk is extended with the missing pieces. In doing so, this change pushes all test program building logic to the various *.test.mk files instead of trying to reuse some tiny bits. In fact, this attempt to reuse some definitions makes the code harder to read and harder to extend. The clear benefit of this is that the interface of bsd.test.mk is now clearly delimited. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:48:49 +02:00
.for _T in ${ATF_TESTS_C}
BINDIR.${_T}= ${TESTSDIR}
MAN.${_T}?= # empty
Enable ATF testing. Submitted by: Garrett Cooper Approved by: marcel (mentor)
2012-11-07 23:02:02 +01:00
SRCS.${_T}?= ${_T}.c
DPADD.${_T}+= ${LIBATF_C}
Fix linking static test binaries with atf.test.mk Check for -static in LDFLAGS or LDFLAGS.<test>, then pass in the appropriate dependency (LIBATF or LDATF) This unbreaks the build with some of the NetBSD tests that need to be compiled statically Reviewed by: imp, jmmv Phabric: D991 Sponsored by: EMC / Isilon Storage Division
2014-10-22 20:11:10 +02:00
.if empty(LDFLAGS:M-static) && empty(LDFLAGS.${_T}:M-static)
Rework PRIVATELIB Now when a lib is marked as PRIVATELIB it is renamed into libprivate$foo instead of being installed in /usr/lib/private and playing with rpath. Also allow to install headers for PRIVATELIBS in that case the headers will be installed in /usr/include/private/$foo Keep the headers under a private namespace to prevent third party build system to easily find them to ensure they are only used on purpose. This allows for non base applications to statically link against a library in base which is linked to a privatelib Treating PRIVATELIBS as regular libraries allows to push them into our current compatX packages if needed. While here finish promotion of libevent as PRIVATELIB Install header for bsdstat and libucl Differential Revision: https://reviews.freebsd.org/D2365 Reviewed by: brooks, des Discussed with: imp
2015-05-04 18:28:54 +02:00
LDADD.${_T}+= ${LDADD_atf_c}
Fix linking static test binaries with atf.test.mk Check for -static in LDFLAGS or LDFLAGS.<test>, then pass in the appropriate dependency (LIBATF or LDATF) This unbreaks the build with some of the NetBSD tests that need to be compiled statically Reviewed by: imp, jmmv Phabric: D991 Sponsored by: EMC / Isilon Storage Division
2014-10-22 20:11:10 +02:00
.else
LDADD.${_T}+= ${LIBATF_C}
.endif
Add the automatic generation of Kyuafile files. These files are generated from bsd.test.mk because kyua is able to run test programs implemented using different libraries/frameworks. In order to make this possible, this change also extends the various *.test.mk file to explicitly indicate the interface of every test program. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:50:56 +02:00
TEST_INTERFACE.${_T}= atf
Enable ATF testing. Submitted by: Garrett Cooper Approved by: marcel (mentor)
2012-11-07 23:02:02 +01:00
.endfor
.endif
Clearly split the logic to build ATF and plain tests apart. This change introduces a new plain.test.mk file that provides the build infrastructure to build test programs that don't use any framework. Most of the code previously in bsd.test.mk moves to plain.test.mk and atf.test.mk is extended with the missing pieces. In doing so, this change pushes all test program building logic to the various *.test.mk files instead of trying to reuse some tiny bits. In fact, this attempt to reuse some definitions makes the code harder to read and harder to extend. The clear benefit of this is that the interface of bsd.test.mk is now clearly delimited. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:48:49 +02:00
.if !empty(ATF_TESTS_CXX)
PROGS_CXX+= ${ATF_TESTS_CXX}
Add the automatic generation of Kyuafile files. These files are generated from bsd.test.mk because kyua is able to run test programs implemented using different libraries/frameworks. In order to make this possible, this change also extends the various *.test.mk file to explicitly indicate the interface of every test program. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:50:56 +02:00
_TESTS+= ${ATF_TESTS_CXX}
Clearly split the logic to build ATF and plain tests apart. This change introduces a new plain.test.mk file that provides the build infrastructure to build test programs that don't use any framework. Most of the code previously in bsd.test.mk moves to plain.test.mk and atf.test.mk is extended with the missing pieces. In doing so, this change pushes all test program building logic to the various *.test.mk files instead of trying to reuse some tiny bits. In fact, this attempt to reuse some definitions makes the code harder to read and harder to extend. The clear benefit of this is that the interface of bsd.test.mk is now clearly delimited. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:48:49 +02:00
.for _T in ${ATF_TESTS_CXX}
BINDIR.${_T}= ${TESTSDIR}
MAN.${_T}?= # empty
Enable ATF testing. Submitted by: Garrett Cooper Approved by: marcel (mentor)
2012-11-07 23:02:02 +01:00
SRCS.${_T}?= ${_T}${CXX_SUFFIX:U.cc}
DPADD.${_T}+= ${LIBATF_CXX} ${LIBATF_C}
Fix linking static test binaries with atf.test.mk Check for -static in LDFLAGS or LDFLAGS.<test>, then pass in the appropriate dependency (LIBATF or LDATF) This unbreaks the build with some of the NetBSD tests that need to be compiled statically Reviewed by: imp, jmmv Phabric: D991 Sponsored by: EMC / Isilon Storage Division
2014-10-22 20:11:10 +02:00
.if empty(LDFLAGS:M-static) && empty(LDFLAGS.${_T}:M-static)
Rework PRIVATELIB Now when a lib is marked as PRIVATELIB it is renamed into libprivate$foo instead of being installed in /usr/lib/private and playing with rpath. Also allow to install headers for PRIVATELIBS in that case the headers will be installed in /usr/include/private/$foo Keep the headers under a private namespace to prevent third party build system to easily find them to ensure they are only used on purpose. This allows for non base applications to statically link against a library in base which is linked to a privatelib Treating PRIVATELIBS as regular libraries allows to push them into our current compatX packages if needed. While here finish promotion of libevent as PRIVATELIB Install header for bsdstat and libucl Differential Revision: https://reviews.freebsd.org/D2365 Reviewed by: brooks, des Discussed with: imp
2015-05-04 18:28:54 +02:00
LDADD.${_T}+= ${LDADD_atf_cxx} ${LDADD_atf_c}
Fix linking static test binaries with atf.test.mk Check for -static in LDFLAGS or LDFLAGS.<test>, then pass in the appropriate dependency (LIBATF or LDATF) This unbreaks the build with some of the NetBSD tests that need to be compiled statically Reviewed by: imp, jmmv Phabric: D991 Sponsored by: EMC / Isilon Storage Division
2014-10-22 20:11:10 +02:00
.else
LDADD.${_T}+= ${LIBATF_CXX} ${LIBATF_C}
.endif
Add the automatic generation of Kyuafile files. These files are generated from bsd.test.mk because kyua is able to run test programs implemented using different libraries/frameworks. In order to make this possible, this change also extends the various *.test.mk file to explicitly indicate the interface of every test program. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:50:56 +02:00
TEST_INTERFACE.${_T}= atf
Enable ATF testing. Submitted by: Garrett Cooper Approved by: marcel (mentor)
2012-11-07 23:02:02 +01:00
.endfor
Another round of attempting to squelch -Wdeprecated-declarations, which has become very trigger-happy with libc++ 9.0.0. It does not help that gcc's implementation of this warning is even more trigger-happy, in the sense that it already warns on the declaration itself, not when you are using it. This is very annoying with our use of -Wsystem-headers. That should really be disabled for gcc.
2019-09-17 08:07:08 +02:00
# Silence warnings about usage of deprecated std::auto_ptr
CXXWARNFLAGS+= -Wno-deprecated-declarations
Enable ATF testing. Submitted by: Garrett Cooper Approved by: marcel (mentor)
2012-11-07 23:02:02 +01:00
.endif
Clearly split the logic to build ATF and plain tests apart. This change introduces a new plain.test.mk file that provides the build infrastructure to build test programs that don't use any framework. Most of the code previously in bsd.test.mk moves to plain.test.mk and atf.test.mk is extended with the missing pieces. In doing so, this change pushes all test program building logic to the various *.test.mk files instead of trying to reuse some tiny bits. In fact, this attempt to reuse some definitions makes the code harder to read and harder to extend. The clear benefit of this is that the interface of bsd.test.mk is now clearly delimited. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:48:49 +02:00
.if !empty(ATF_TESTS_SH)
SCRIPTS+= ${ATF_TESTS_SH}
Add the automatic generation of Kyuafile files. These files are generated from bsd.test.mk because kyua is able to run test programs implemented using different libraries/frameworks. In order to make this possible, this change also extends the various *.test.mk file to explicitly indicate the interface of every test program. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:50:56 +02:00
_TESTS+= ${ATF_TESTS_SH}
Clearly split the logic to build ATF and plain tests apart. This change introduces a new plain.test.mk file that provides the build infrastructure to build test programs that don't use any framework. Most of the code previously in bsd.test.mk moves to plain.test.mk and atf.test.mk is extended with the missing pieces. In doing so, this change pushes all test program building logic to the various *.test.mk files instead of trying to reuse some tiny bits. In fact, this attempt to reuse some definitions makes the code harder to read and harder to extend. The clear benefit of this is that the interface of bsd.test.mk is now clearly delimited. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:48:49 +02:00
.for _T in ${ATF_TESTS_SH}
SCRIPTSDIR_${_T}= ${TESTSDIR}
Add the automatic generation of Kyuafile files. These files are generated from bsd.test.mk because kyua is able to run test programs implemented using different libraries/frameworks. In order to make this possible, this change also extends the various *.test.mk file to explicitly indicate the interface of every test program. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:50:56 +02:00
TEST_INTERFACE.${_T}= atf
Enable ATF testing. Submitted by: Garrett Cooper Approved by: marcel (mentor)
2012-11-07 23:02:02 +01:00
CLEANFILES+= ${_T} ${_T}.tmp
Add the *_TESTS_SH_SED_* functionality to atf.test.mk. This exists already in plain.test.mk and tap.test.mk and should have been added to atf.test.mk too when the feature was first introduced. (It is probably time to address the related TODOs but I will do that separately.)
2014-06-06 19:53:05 +02:00
# TODO(jmmv): It seems to me that this SED and SRC functionality should
# exist in bsd.prog.mk along the support for SCRIPTS. Move it there if
# this proves to be useful within the tests.
ATF_TESTS_SH_SED_${_T}?= # empty
Clearly split the logic to build ATF and plain tests apart. This change introduces a new plain.test.mk file that provides the build infrastructure to build test programs that don't use any framework. Most of the code previously in bsd.test.mk moves to plain.test.mk and atf.test.mk is extended with the missing pieces. In doing so, this change pushes all test program building logic to the various *.test.mk files instead of trying to reuse some tiny bits. In fact, this attempt to reuse some definitions makes the code harder to read and harder to extend. The clear benefit of this is that the interface of bsd.test.mk is now clearly delimited. Submitted by: Julio Merino jmmv google.com MFC after: 2 weeks
2013-10-19 08:48:49 +02:00
ATF_TESTS_SH_SRC_${_T}?= ${_T}.sh
${_T}: ${ATF_TESTS_SH_SRC_${_T}}
Move atf-sh from /usr/bin/ to /usr/libexec/ In r266650, we made libatf-c and libatf-c++ private libraries so that no components outside of the source tree could unintendedly depend on them. This change does the same for the "atf-sh library" by moving the atf-sh interpreter from its public location in /usr/bin/ to the private location in /usr/libexec/. Our build system will ensure that our own test programs use the right binary, but users won't be able to depend on atf-sh by "mistake". Committing this now to ride the UPDATING notice added with r267172 today.
2014-06-06 20:58:06 +02:00
echo '#! /usr/libexec/atf-sh' > ${.TARGET}.tmp
As an optimization (in simple, well used cases) only call cat ${.ALLSRC} when generating files from various sources instead of calling cat ${.ALLSRC} | sed The perl case was skipped because it's not being used in the tree at this time
2014-11-04 02:57:31 +01:00
.if empty(ATF_TESTS_SH_SED_${_T})
cat ${.ALLSRC:N*Makefile*} >>${.TARGET}.tmp
.else
Fix atf-sh's integration_test With the move of atf-sh into /usr/libexec in r267181, some of the tests in the integration_test program broke because they could not execute atf-sh from the path any longer. This slipped through because I do have a local atf installation in my home directory that appears in my path, hence the tests could still execute my own version. Fix this by forcing /usr/libexec to appear at the beginning of the path when attempting to execute atf-sh. To make upgrading easy (and to avoid an unnecessary entry in UPDATING), make integration_test depend on the Makefile so that a rebuild of the shell script is triggered. This requires a hack in the *.test.mk files to ensure the Makefile is not treated as a source to the generated program. Ugly, I know, but I don't have a better way of doing this at the moment. Will think of one once I address the TODO in the *.test.mk files that suggests generalizing the file generation functionality. PR: 191052 Reviewed by: Garrett Cooper
2014-07-09 02:55:50 +02:00
cat ${.ALLSRC:N*Makefile*} \
| sed ${ATF_TESTS_SH_SED_${_T}} >>${.TARGET}.tmp
As an optimization (in simple, well used cases) only call cat ${.ALLSRC} when generating files from various sources instead of calling cat ${.ALLSRC} | sed The perl case was skipped because it's not being used in the tree at this time
2014-11-04 02:57:31 +01:00
.endif
Enable ATF testing. Submitted by: Garrett Cooper Approved by: marcel (mentor)
2012-11-07 23:02:02 +01:00
chmod +x ${.TARGET}.tmp
mv ${.TARGET}.tmp ${.TARGET}
.endfor
.endif
Add the ZFS test suite It was originally written by Sun as part of the STF (Solaris test framework). They open sourced it in OpenSolaris, then HighCloud partially ported it to FreeBSD, and Spectra Logic finished the port. We also added many testcases, fixed many broken ones, and converted them all to the ATF framework. We've had help along the way from avg, araujo, smh, and brd. By default most of the tests are disabled. Set the disks Kyua variable to enable them. Submitted by: asomers, will, justing, ken, brd, avg, araujo, smh Sponsored by: Spectra Logic Corp, HighCloud
2018-02-23 17:31:00 +01:00
.if !empty(ATF_TESTS_KSH93)
SCRIPTS+= ${ATF_TESTS_KSH93}
_TESTS+= ${ATF_TESTS_KSH93}
.for _T in ${ATF_TESTS_KSH93}
SCRIPTSDIR_${_T}= ${TESTSDIR}
TEST_INTERFACE.${_T}= atf
tests: require ksh93 in the Kyuafiles for all atf-ksh93 test programs
2018-02-24 16:13:20 +01:00
TEST_METADATA.${_T}+= required_programs="ksh93"
Add the ZFS test suite It was originally written by Sun as part of the STF (Solaris test framework). They open sourced it in OpenSolaris, then HighCloud partially ported it to FreeBSD, and Spectra Logic finished the port. We also added many testcases, fixed many broken ones, and converted them all to the ATF framework. We've had help along the way from avg, araujo, smh, and brd. By default most of the tests are disabled. Set the disks Kyua variable to enable them. Submitted by: asomers, will, justing, ken, brd, avg, araujo, smh Sponsored by: Spectra Logic Corp, HighCloud
2018-02-23 17:31:00 +01:00
CLEANFILES+= ${_T} ${_T}.tmp
# TODO(jmmv): It seems to me that this SED and SRC functionality should
# exist in bsd.prog.mk along the support for SCRIPTS. Move it there if
# this proves to be useful within the tests.
ATF_TESTS_KSH93_SED_${_T}?= # empty
ATF_TESTS_KSH93_SRC_${_T}?= ${_T}.sh
${_T}: ${ATF_TESTS_KSH93_SRC_${_T}}
echo '#! /usr/libexec/atf-sh -s/usr/local/bin/ksh93' > ${.TARGET}.tmp
.if empty(ATF_TESTS_KSH93_SED_${_T})
cat ${.ALLSRC:N*Makefile*} >>${.TARGET}.tmp
.else
cat ${.ALLSRC:N*Makefile*} \
| sed ${ATF_TESTS_KSH93_SED_${_T}} >>${.TARGET}.tmp
.endif
chmod +x ${.TARGET}.tmp
mv ${.TARGET}.tmp ${.TARGET}
.endfor
.endif
Reference in New Issue Copy Permalink