commit b19a54d2f9d8a3e83366c638aace4cf3b5daa96e Author: nosamad Date: Mon May 25 23:24:23 2020 +0200 Squashed 'build/dobuild/' content from commit b017db7 git-subtree-dir: build/dobuild git-subtree-split: b017db7c22e7c0ca6cee3f0f5ac1dd50f3b68eb9 diff --git a/.env b/.env new file mode 100644 index 0000000..654f256 --- /dev/null +++ b/.env @@ -0,0 +1,10 @@ +COMPOSEENV_DOCKER_VERSION=18.09.6 +COMPOSEENV_DOCKER_DOWNLOAD=https://download.docker.com +COMPOSEENV_BATS_DOWNLOAD=https://github.com/bats-core +COMPOSEENV_DUMB_INIT_DOWNLOAD=https://github.com/Yelp/dumb-init +COMPOSEENV_LIBFAKETIME_DOWNLOAD=https://github.com/wolfcw/libfaketime +COMPOSEENV_REGISTRY_PREFIX= +COMPOSEENV_PROJECT_VERSION=0.0.1 +COMPOSEENV_PROJECTPATH=/var/tmp +COMPOSEENV_CAPATH=./assets/ca-certificates +DOCKERSOCK_GROUP=100 diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..6a1b23f --- /dev/null +++ b/.gitignore @@ -0,0 +1,3 @@ +/assets/ca-certificates/*.crt +.build +.deps \ No newline at end of file diff --git a/.project b/.project new file mode 100644 index 0000000..3e4423f --- /dev/null +++ b/.project @@ -0,0 +1,11 @@ + + + dobuild + + + + + + + + diff --git a/AUTHORS b/AUTHORS new file mode 100644 index 0000000..c6f79bb --- /dev/null +++ b/AUTHORS @@ -0,0 +1,7 @@ +# This is the list of dobuild authors for copyright purposes. +# +# This does not necessarily list everyone who has contributed code, since in +# some cases, their employer may be the copyright holder. To see the full list +# of contributors, see the revision history in source control. +Falk Werner +Osama El Hosami diff --git a/LICENSE.txt b/LICENSE.txt new file mode 100644 index 0000000..14e2f77 --- /dev/null +++ b/LICENSE.txt @@ -0,0 +1,373 @@ +Mozilla Public License Version 2.0 +================================== + +1. Definitions +-------------- + +1.1. "Contributor" + means each individual or legal entity that creates, contributes to + the creation of, or owns Covered Software. + +1.2. "Contributor Version" + means the combination of the Contributions of others (if any) used + by a Contributor and that particular Contributor's Contribution. + +1.3. "Contribution" + means Covered Software of a particular Contributor. + +1.4. "Covered Software" + means Source Code Form to which the initial Contributor has attached + the notice in Exhibit A, the Executable Form of such Source Code + Form, and Modifications of such Source Code Form, in each case + including portions thereof. + +1.5. "Incompatible With Secondary Licenses" + means + + (a) that the initial Contributor has attached the notice described + in Exhibit B to the Covered Software; or + + (b) that the Covered Software was made available under the terms of + version 1.1 or earlier of the License, but not also under the + terms of a Secondary License. + +1.6. "Executable Form" + means any form of the work other than Source Code Form. + +1.7. "Larger Work" + means a work that combines Covered Software with other material, in + a separate file or files, that is not Covered Software. + +1.8. "License" + means this document. + +1.9. "Licensable" + means having the right to grant, to the maximum extent possible, + whether at the time of the initial grant or subsequently, any and + all of the rights conveyed by this License. + +1.10. "Modifications" + means any of the following: + + (a) any file in Source Code Form that results from an addition to, + deletion from, or modification of the contents of Covered + Software; or + + (b) any new file in Source Code Form that contains any Covered + Software. + +1.11. "Patent Claims" of a Contributor + means any patent claim(s), including without limitation, method, + process, and apparatus claims, in any patent Licensable by such + Contributor that would be infringed, but for the grant of the + License, by the making, using, selling, offering for sale, having + made, import, or transfer of either its Contributions or its + Contributor Version. + +1.12. "Secondary License" + means either the GNU General Public License, Version 2.0, the GNU + Lesser General Public License, Version 2.1, the GNU Affero General + Public License, Version 3.0, or any later versions of those + licenses. + +1.13. "Source Code Form" + means the form of the work preferred for making modifications. + +1.14. "You" (or "Your") + means an individual or a legal entity exercising rights under this + License. For legal entities, "You" includes any entity that + controls, is controlled by, or is under common control with You. For + purposes of this definition, "control" means (a) the power, direct + or indirect, to cause the direction or management of such entity, + whether by contract or otherwise, or (b) ownership of more than + fifty percent (50%) of the outstanding shares or beneficial + ownership of such entity. + +2. License Grants and Conditions +-------------------------------- + +2.1. Grants + +Each Contributor hereby grants You a world-wide, royalty-free, +non-exclusive license: + +(a) under intellectual property rights (other than patent or trademark) + Licensable by such Contributor to use, reproduce, make available, + modify, display, perform, distribute, and otherwise exploit its + Contributions, either on an unmodified basis, with Modifications, or + as part of a Larger Work; and + +(b) under Patent Claims of such Contributor to make, use, sell, offer + for sale, have made, import, and otherwise transfer either its + Contributions or its Contributor Version. + +2.2. Effective Date + +The licenses granted in Section 2.1 with respect to any Contribution +become effective for each Contribution on the date the Contributor first +distributes such Contribution. + +2.3. Limitations on Grant Scope + +The licenses granted in this Section 2 are the only rights granted under +this License. No additional rights or licenses will be implied from the +distribution or licensing of Covered Software under this License. +Notwithstanding Section 2.1(b) above, no patent license is granted by a +Contributor: + +(a) for any code that a Contributor has removed from Covered Software; + or + +(b) for infringements caused by: (i) Your and any other third party's + modifications of Covered Software, or (ii) the combination of its + Contributions with other software (except as part of its Contributor + Version); or + +(c) under Patent Claims infringed by Covered Software in the absence of + its Contributions. + +This License does not grant any rights in the trademarks, service marks, +or logos of any Contributor (except as may be necessary to comply with +the notice requirements in Section 3.4). + +2.4. Subsequent Licenses + +No Contributor makes additional grants as a result of Your choice to +distribute the Covered Software under a subsequent version of this +License (see Section 10.2) or under the terms of a Secondary License (if +permitted under the terms of Section 3.3). + +2.5. Representation + +Each Contributor represents that the Contributor believes its +Contributions are its original creation(s) or it has sufficient rights +to grant the rights to its Contributions conveyed by this License. + +2.6. Fair Use + +This License is not intended to limit any rights You have under +applicable copyright doctrines of fair use, fair dealing, or other +equivalents. + +2.7. Conditions + +Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted +in Section 2.1. + +3. Responsibilities +------------------- + +3.1. Distribution of Source Form + +All distribution of Covered Software in Source Code Form, including any +Modifications that You create or to which You contribute, must be under +the terms of this License. You must inform recipients that the Source +Code Form of the Covered Software is governed by the terms of this +License, and how they can obtain a copy of this License. You may not +attempt to alter or restrict the recipients' rights in the Source Code +Form. + +3.2. Distribution of Executable Form + +If You distribute Covered Software in Executable Form then: + +(a) such Covered Software must also be made available in Source Code + Form, as described in Section 3.1, and You must inform recipients of + the Executable Form how they can obtain a copy of such Source Code + Form by reasonable means in a timely manner, at a charge no more + than the cost of distribution to the recipient; and + +(b) You may distribute such Executable Form under the terms of this + License, or sublicense it under different terms, provided that the + license for the Executable Form does not attempt to limit or alter + the recipients' rights in the Source Code Form under this License. + +3.3. Distribution of a Larger Work + +You may create and distribute a Larger Work under terms of Your choice, +provided that You also comply with the requirements of this License for +the Covered Software. If the Larger Work is a combination of Covered +Software with a work governed by one or more Secondary Licenses, and the +Covered Software is not Incompatible With Secondary Licenses, this +License permits You to additionally distribute such Covered Software +under the terms of such Secondary License(s), so that the recipient of +the Larger Work may, at their option, further distribute the Covered +Software under the terms of either this License or such Secondary +License(s). + +3.4. Notices + +You may not remove or alter the substance of any license notices +(including copyright notices, patent notices, disclaimers of warranty, +or limitations of liability) contained within the Source Code Form of +the Covered Software, except that You may alter any license notices to +the extent required to remedy known factual inaccuracies. + +3.5. Application of Additional Terms + +You may choose to offer, and to charge a fee for, warranty, support, +indemnity or liability obligations to one or more recipients of Covered +Software. However, You may do so only on Your own behalf, and not on +behalf of any Contributor. You must make it absolutely clear that any +such warranty, support, indemnity, or liability obligation is offered by +You alone, and You hereby agree to indemnify every Contributor for any +liability incurred by such Contributor as a result of warranty, support, +indemnity or liability terms You offer. You may include additional +disclaimers of warranty and limitations of liability specific to any +jurisdiction. + +4. Inability to Comply Due to Statute or Regulation +--------------------------------------------------- + +If it is impossible for You to comply with any of the terms of this +License with respect to some or all of the Covered Software due to +statute, judicial order, or regulation then You must: (a) comply with +the terms of this License to the maximum extent possible; and (b) +describe the limitations and the code they affect. Such description must +be placed in a text file included with all distributions of the Covered +Software under this License. Except to the extent prohibited by statute +or regulation, such description must be sufficiently detailed for a +recipient of ordinary skill to be able to understand it. + +5. Termination +-------------- + +5.1. The rights granted under this License will terminate automatically +if You fail to comply with any of its terms. However, if You become +compliant, then the rights granted under this License from a particular +Contributor are reinstated (a) provisionally, unless and until such +Contributor explicitly and finally terminates Your grants, and (b) on an +ongoing basis, if such Contributor fails to notify You of the +non-compliance by some reasonable means prior to 60 days after You have +come back into compliance. Moreover, Your grants from a particular +Contributor are reinstated on an ongoing basis if such Contributor +notifies You of the non-compliance by some reasonable means, this is the +first time You have received notice of non-compliance with this License +from such Contributor, and You become compliant prior to 30 days after +Your receipt of the notice. + +5.2. If You initiate litigation against any entity by asserting a patent +infringement claim (excluding declaratory judgment actions, +counter-claims, and cross-claims) alleging that a Contributor Version +directly or indirectly infringes any patent, then the rights granted to +You by any and all Contributors for the Covered Software under Section +2.1 of this License shall terminate. + +5.3. In the event of termination under Sections 5.1 or 5.2 above, all +end user license agreements (excluding distributors and resellers) which +have been validly granted by You or Your distributors under this License +prior to termination shall survive termination. + +************************************************************************ +* * +* 6. Disclaimer of Warranty * +* ------------------------- * +* * +* Covered Software is provided under this License on an "as is" * +* basis, without warranty of any kind, either expressed, implied, or * +* statutory, including, without limitation, warranties that the * +* Covered Software is free of defects, merchantable, fit for a * +* particular purpose or non-infringing. The entire risk as to the * +* quality and performance of the Covered Software is with You. * +* Should any Covered Software prove defective in any respect, You * +* (not any Contributor) assume the cost of any necessary servicing, * +* repair, or correction. This disclaimer of warranty constitutes an * +* essential part of this License. No use of any Covered Software is * +* authorized under this License except under this disclaimer. * +* * +************************************************************************ + +************************************************************************ +* * +* 7. Limitation of Liability * +* -------------------------- * +* * +* Under no circumstances and under no legal theory, whether tort * +* (including negligence), contract, or otherwise, shall any * +* Contributor, or anyone who distributes Covered Software as * +* permitted above, be liable to You for any direct, indirect, * +* special, incidental, or consequential damages of any character * +* including, without limitation, damages for lost profits, loss of * +* goodwill, work stoppage, computer failure or malfunction, or any * +* and all other commercial damages or losses, even if such party * +* shall have been informed of the possibility of such damages. This * +* limitation of liability shall not apply to liability for death or * +* personal injury resulting from such party's negligence to the * +* extent applicable law prohibits such limitation. Some * +* jurisdictions do not allow the exclusion or limitation of * +* incidental or consequential damages, so this exclusion and * +* limitation may not apply to You. * +* * +************************************************************************ + +8. Litigation +------------- + +Any litigation relating to this License may be brought only in the +courts of a jurisdiction where the defendant maintains its principal +place of business and such litigation shall be governed by laws of that +jurisdiction, without reference to its conflict-of-law provisions. +Nothing in this Section shall prevent a party's ability to bring +cross-claims or counter-claims. + +9. Miscellaneous +---------------- + +This License represents the complete agreement concerning the subject +matter hereof. If any provision of this License is held to be +unenforceable, such provision shall be reformed only to the extent +necessary to make it enforceable. Any law or regulation which provides +that the language of a contract shall be construed against the drafter +shall not be used to construe this License against a Contributor. + +10. Versions of the License +--------------------------- + +10.1. New Versions + +Mozilla Foundation is the license steward. Except as provided in Section +10.3, no one other than the license steward has the right to modify or +publish new versions of this License. Each version will be given a +distinguishing version number. + +10.2. Effect of New Versions + +You may distribute the Covered Software under the terms of the version +of the License under which You originally received the Covered Software, +or under the terms of any subsequent version published by the license +steward. + +10.3. Modified Versions + +If you create software not governed by this License, and you want to +create a new license for such software, you may create and use a +modified version of this License if you rename the license and remove +any references to the name of the license steward (except to note that +such modified license differs from this License). + +10.4. Distributing Source Code Form that is Incompatible With Secondary +Licenses + +If You choose to distribute Source Code Form that is Incompatible With +Secondary Licenses under the terms of this version of the License, the +notice described in Exhibit B of this License must be attached. + +Exhibit A - Source Code Form License Notice +------------------------------------------- + + This Source Code Form is subject to the terms of the Mozilla Public + License, v. 2.0. If a copy of the MPL was not distributed with this + file, You can obtain one at http://mozilla.org/MPL/2.0/. + +If it is not possible or desirable to put the notice in a particular +file, then You may include the notice in a location (such as a LICENSE +file in a relevant directory) where a recipient would be likely to look +for such a notice. + +You may add additional accurate notices of copyright ownership. + +Exhibit B - "Incompatible With Secondary Licenses" Notice +--------------------------------------------------------- + + This Source Code Form is "Incompatible With Secondary Licenses", as + defined by the Mozilla Public License, v. 2.0. diff --git a/Makefile b/Makefile new file mode 100644 index 0000000..6d68cb9 --- /dev/null +++ b/Makefile @@ -0,0 +1,86 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +MAKEFLAGS += --no-builtin-rules +MAKEFILE := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)) +SHELL := /bin/sh + +.SUFFIXES: + +.PHONY: default +default: all + +####################################################################################################################### +# Overridable defaults + +DOBUILD_PROJECTDIR ?= $(patsubst %/,%,$(dir $(MAKEFILE))) + +FETCHDIR = $(DOBUILD_PROJECTDIR)/.deps +CMAKE_BUILDER_OPTS += -f '$(DOBUILD_PROJECTDIR)/assets/projects/cmake/builder-template.mk' +DOBUILD_BUILDER_OPTS += -f '$(DOBUILD_PROJECTDIR)/assets/projects/dobuild/builder-template.mk' +GRADLE_BUILDER_OPTS += -f '$(DOBUILD_PROJECTDIR)/assets/projects/gradle/builder-template.mk' + +####################################################################################################################### +# Standard rule target configuration + +BUILD_TARGETS += cmake-builder-template-all +BUILD_TARGETS += dobuild-builder-template-all +BUILD_TARGETS += gradle-builder-template-all + +CLEAN_TARGETS += cmake-builder-template-clean +CLEAN_TARGETS += dobuild-builder-template-clean +CLEAN_TARGETS += gradle-builder-template-clean + +DISTCLEAN_TARGETS += cmake-builder-template-distclean +DISTCLEAN_TARGETS += dobuild-builder-template-distclean +DISTCLEAN_TARGETS += gradle-builder-template-distclean + +DIST_TARGETS += cmake-builder-template-dist +DIST_TARGETS += dobuild-builder-template-dist +DIST_TARGETS += gradle-builder-template-dist + +CHECK_TARGETS += + +MAKEOVERRIDES := FETCHDIR=$(FETCHDIR) $(MAKEOVERRIDES) + +####################################################################################################################### +# Shell exports + +####################################################################################################################### +# Standard rule targets + +.PHONY: all +all: $(BUILD_TARGETS); + +.PHONY: check +check: $(CHECK_TARGETS) + $(DOBUILD_PROJECTDIR)/run_tests + +.PHONY: run +run: + -$(DOBUILD_PROJECTDIR)/run_tests bash + +.PHONY: clean +clean: $(CLEAN_TARGETS); + +.PHONY: dist +dist: $(DIST_TARGETS); + +.PHONY: distclean +distclean: $(DISTCLEAN_TARGETS); + +cmake-builder-template-%: + $(MAKE) $(MFLAGS) $(CMAKE_BUILDER_OPTS) $* $(MAKEOVERRIDES) + +dobuild-builder-template-%: + $(MAKE) $(MFLAGS) $(DOBUILD_BUILDER_OPTS) $* $(MAKEOVERRIDES) + +gradle-builder-template-%: + $(MAKE) $(MFLAGS) $(GRADLE_BUILDER_OPTS) $* $(MAKEOVERRIDES) diff --git a/README.md b/README.md new file mode 100644 index 0000000..0487ce3 --- /dev/null +++ b/README.md @@ -0,0 +1,89 @@ +# dobuild + +## Introduction + +A major concern when developing and maintaining software, is to achieve reproducibility of builds. This can be +especially necessary, when product service contract or warranty requirements enforce backports of bug fixes into +previously released versions. + +As noted by [reproducible-builds.org](https://reproducible-builds.org/docs/): + +> Getting reproducible builds for your software might be easier than you think! But it might require small changes +> to your build system and a strategy on how to enable others to recreate an environment in which the builds can be +> reproduced. + +The goal of this project is to support users, editing, building, testing and distributing software, by providing an +*easy to install* and *auditable* deterministic build environment based on GNU Make and OCI Containers. + +## Overview + +TODO + +## Prerequisites + +TODO + +## Embed dobuild in your project using git + +A simple way of referencing dobuild in your standalone project is to use it as git subtree. Using git subtree allows to nest +one repository inside another as a sub-directory. It is one of several ways git can manage dependencies between repositories +(or projects). For further details refer to [Git subtree: the alternative to Git submodule](https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree). + +### Initial Setup + +Change to the root of your standalone project git repository. Ensure repository has no outstanding changes, or reset them +(git reset --hard). Adding the subtree as a remote allows you to refer to it in shorter form (dobuild). After adding +the remote, add the subtree to a folder e.g. build/dobuild/ selecting a tag or branch e.g. master to clone the content of +repository. + +```bash + + git remote add -f dobuild https://github.com/nosamad/dobuild.git \ +&& git subtree add --squash --prefix build/dobuild/ dobuild master + +``` + +### Update changes + +To update changes in sub-project dobuild, pull the subtree (build/dobuild/) selecting the branch or tag e.g. master again. + +```bash + + git fetch dobuild master \ +&& git subtree pull --squash --prefix build/dobuild/ dobuild master + +``` + +Afterwards changes can be freely committed as part of your standalone project repository. + +### Contribute changes back upstream + +To contribute changes back to the upstream project (dobuild), your need to fork the project and add it as another remote: + +```bash + +git remote add dobuild-fork https://github.com//dobuild.git + +``` + +Afterwards you can use a subtree push to get your changes into your forked repository, in order to open a pull-request. + +```bash + +git subtree push --prefix=build/dobuild/ dobuild-fork master + +``` + +### Purge + +Purging a git subtree is also fairly easy - just remove the directory and commit the change. There are no additional config settings to purge unlike known from git submodules. + +## Roadmap + +TODO + +## License + +Copyright (c) 2019 Contributors as noted in the AUTHORS file. + +Licensed under the [MPL-2](LICENSE.txt) License. diff --git a/VERSION b/VERSION new file mode 100644 index 0000000..8a9ecc2 --- /dev/null +++ b/VERSION @@ -0,0 +1 @@ +0.0.1 \ No newline at end of file diff --git a/assets/adapters/cmake/assemble b/assets/adapters/cmake/assemble new file mode 100755 index 0000000..a45665b --- /dev/null +++ b/assets/adapters/cmake/assemble @@ -0,0 +1,3 @@ +#!/bin/sh + +exec cmake_adapter "$@" diff --git a/assets/adapters/cmake/check b/assets/adapters/cmake/check new file mode 100755 index 0000000..b309c51 --- /dev/null +++ b/assets/adapters/cmake/check @@ -0,0 +1,15 @@ +#!/bin/sh -e + +supports_check_target() { + [ -f "$1" ] && grep -q '^check$' "$1" +} + +if supports_check_target 'DoBuildFiles/targets.txt'; then + set -- check "$@" +else + assemble + set -- test "$@" +fi + +exec cmake_adapter "$@" + diff --git a/assets/adapters/cmake/check-memcheck b/assets/adapters/cmake/check-memcheck new file mode 100755 index 0000000..1d8ceb6 --- /dev/null +++ b/assets/adapters/cmake/check-memcheck @@ -0,0 +1,15 @@ +#!/bin/sh -e + +supports_memcheck_target() { + [ -f "$1" ] && grep -q '^memcheck$' "$1" +} + +if supports_memcheck_target 'DoBuildFiles/targets.txt'; then + set -- cmake_adapter memcheck "$@" +else + assemble + set -- ctest_adapter -T memcheck "$@" +fi + +exec "$@" + diff --git a/assets/adapters/cmake/cmake_adapter b/assets/adapters/cmake/cmake_adapter new file mode 100755 index 0000000..ec5df27 --- /dev/null +++ b/assets/adapters/cmake/cmake_adapter @@ -0,0 +1,12 @@ +#!/bin/sh + +. DoBuildFiles/generic.properties +. DoBuildFiles/cmake.properties +. cmake_helper.sh + +export CTEST_OUTPUT_ON_FAILURE=1 + +{ [ -z "$BUILDVERBOSE" ] && [ -z "$TESTVERBOSE" ]; } || set -- "$(cmake_generator_verbose_opt "$CMAKE_GENERATOR_NAME")" "$@" +[ -z "$VERBOSE" ] || set -x + +exec "$CMAKE_GENERATOR_CMD" -j"$NPROC" "$@" diff --git a/assets/adapters/cmake/cmake_helper.sh b/assets/adapters/cmake/cmake_helper.sh new file mode 100644 index 0000000..a067fd6 --- /dev/null +++ b/assets/adapters/cmake/cmake_helper.sh @@ -0,0 +1,13 @@ +#!/bin/sh + +cmake_ninja_verbose_opt() { + echo '-v' +} + +cmake_make_verbose_opt() { + echo 'VERBOSE=1' +} + +cmake_generator_verbose_opt() { + "cmake_${1}_verbose_opt" +} diff --git a/assets/adapters/cmake/ctest_adapter b/assets/adapters/cmake/ctest_adapter new file mode 100755 index 0000000..77ca7c6 --- /dev/null +++ b/assets/adapters/cmake/ctest_adapter @@ -0,0 +1,10 @@ +#!/bin/sh + +. DoBuildFiles/generic.properties +. DoBuildFiles/cmake.properties +. cmake_helper.sh + +{ [ -z "$BUILDVERBOSE" ] && [ -z "$TESTVERBOSE" ]; } || set -- "$(cmake_generator_verbose_opt "$CMAKE_GENERATOR_NAME")" "$@" +[ -z "$VERBOSE" ] || set -x + +exec ctest --output-on-failure -j"$NPROC" "$@" diff --git a/assets/adapters/cmake/delegate b/assets/adapters/cmake/delegate new file mode 100755 index 0000000..a45665b --- /dev/null +++ b/assets/adapters/cmake/delegate @@ -0,0 +1,3 @@ +#!/bin/sh + +exec cmake_adapter "$@" diff --git a/assets/adapters/cmake/package b/assets/adapters/cmake/package new file mode 100755 index 0000000..eeb84b5 --- /dev/null +++ b/assets/adapters/cmake/package @@ -0,0 +1,3 @@ +#!/bin/sh + +exec cmake_adapter package "$@" diff --git a/assets/adapters/cmake/package-install b/assets/adapters/cmake/package-install new file mode 100755 index 0000000..a168a01 --- /dev/null +++ b/assets/adapters/cmake/package-install @@ -0,0 +1,5 @@ +#!/bin/sh + +export DESTDIR="$1" + +exec cmake_adapter install "$@" diff --git a/assets/adapters/cmake/prepare b/assets/adapters/cmake/prepare new file mode 100755 index 0000000..f4fee0f --- /dev/null +++ b/assets/adapters/cmake/prepare @@ -0,0 +1,19 @@ +#!/bin/sh + +. DoBuildFiles/generic.properties +. DoBuildFiles/cmake.properties + +camelcase() { + echo "$@" | sed -r 's/(^|_)([a-z])/\U\2/g' +} + +if [ -f "${CMAKE_TOOLCHAIN_FILE}" ]; then + set -- "$@" "-DCMAKE_TOOLCHAIN_FILE=${CMAKE_TOOLCHAIN_FILE}" +fi +if [ -n "$BUILD_TESTING" ]; then + set -- "$@" "-DBUILD_TESTING=ON" +fi +set -- "$@" -DCMAKE_EXPORT_COMPILE_COMMANDS=ON "-DCMAKE_BUILD_TYPE=$(camelcase "$VARIANT")" "-G${CMAKE_GENERATOR}" "-B${PWD}" "$PROJECTDIR" +[ -z "$VERBOSE" ] || set -x + +exec cmake "$@" diff --git a/assets/adapters/cmake/save-artifacts b/assets/adapters/cmake/save-artifacts new file mode 100755 index 0000000..cd0c7b3 --- /dev/null +++ b/assets/adapters/cmake/save-artifacts @@ -0,0 +1,13 @@ +#!/bin/sh + +. DoBuildFiles/generic.properties +. DoBuildFiles/cmake.properties + +[ -z "$VERBOSE" ] || set -x + +exec "$DOBUILDDIR/bin/tar_cc_settings" \ + --project-root "$PROJECTDIR" \ + --build-system "cmake-${CMAKE_GENERATOR_NAME}" \ + --include-prefix "$TARGET/DoBuildFiles" \ + compile_commands.json \ + CMakeCache.txt diff --git a/assets/adapters/gradle/assemble b/assets/adapters/gradle/assemble new file mode 100755 index 0000000..15cdda3 --- /dev/null +++ b/assets/adapters/gradle/assemble @@ -0,0 +1,9 @@ +#!/bin/sh + +. DoBuildFiles/generic.properties + +if [ -n "$BUILD_TESTING" ]; then + set -- testClasses "$@" +fi + +exec gradle_adapter classes "$@" diff --git a/assets/adapters/gradle/check b/assets/adapters/gradle/check new file mode 100755 index 0000000..f4d958b --- /dev/null +++ b/assets/adapters/gradle/check @@ -0,0 +1,3 @@ +#!/bin/sh + +exec gradle_adapter cleanTest test "$@" diff --git a/assets/adapters/gradle/delegate b/assets/adapters/gradle/delegate new file mode 100755 index 0000000..8f3664f --- /dev/null +++ b/assets/adapters/gradle/delegate @@ -0,0 +1,3 @@ +#!/bin/sh + +exec gradle_adapter "$@" diff --git a/assets/adapters/gradle/gradle_adapter b/assets/adapters/gradle/gradle_adapter new file mode 100755 index 0000000..d6d153b --- /dev/null +++ b/assets/adapters/gradle/gradle_adapter @@ -0,0 +1,17 @@ +#!/bin/sh + +. DoBuildFiles/generic.properties + +{ [ -z "$BUILDVERBOSE" ] && [ -z "$TESTVERBOSE" ]; } || set -- --info "$@" +[ -z "$VERBOSE" ] || set -x + +exec gradle \ + --no-daemon \ + --parallel \ + --console plain \ + --max-workers "$NPROC" \ + --gradle-user-home "$PWD/gradle" \ + --project-cache-dir "$PWD/gradle-cache" \ + --project-dir "$PROJECTDIR" \ + "-Dorg.gradle.project.buildDir=$PWD/build" \ + "$@" diff --git a/assets/ca-certificates/.gitignore b/assets/ca-certificates/.gitignore new file mode 100644 index 0000000..e69de29 diff --git a/assets/projects/cmake/arm32v7-alpine-builder-template.dockerfile b/assets/projects/cmake/arm32v7-alpine-builder-template.dockerfile new file mode 100644 index 0000000..12a4f7c --- /dev/null +++ b/assets/projects/cmake/arm32v7-alpine-builder-template.dockerfile @@ -0,0 +1,89 @@ +ARG REGISTRY_PREFIX='' +ARG DISTRIB_VERSION=3.9 + +FROM ${REGISTRY_PREFIX}arm32v7/alpine:${DISTRIB_VERSION} as builder + +ARG QEMU_VERSION_=v3.1.0-2 +COPY bin/qemu-arm-static-$QEMU_VERSION_ /usr/bin/qemu-arm-static + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +RUN set -x \ + && installdeps="tzdata" \ + && apk add --no-cache --virtual .install-deps $installdeps \ + && ls /usr/share/zoneinfo \ + && cp -H --remove-destination "/usr/share/zoneinfo/$TZ" /tmp/localtime \ + && apk del .install-deps \ + && mv /tmp/localtime /etc/localtime \ + && echo "$TZ" > /etc/timezone \ + && apk add --no-cache \ + bash \ + make \ + gcc \ + musl-dev + +COPY src /usr/local/src + +ARG PARALLELMFLAGS=-j2 + +ARG DUMB_INIT_VERSION=1.2.2 +ARG DUMB_INIT_MTIME= +RUN set -x \ + && builddeps="vim" \ + && apk add --no-cache --virtual .build-deps $builddeps \ + && [ -n "$DUMB_INIT_MTIME" ] && export SOURCE_DATE_EPOCH="$DUMB_INIT_MTIME" \ + && builddir="/tmp/out" \ + && mkdir -p "$builddir" \ + && cd "$builddir" \ + && cp -R "/usr/local/src/dumb-init-$DUMB_INIT_VERSION" . \ + && cd "dumb-init-$DUMB_INIT_VERSION" \ + && make "$PARALLELMFLAGS" \ + && chmod +x dumb-init \ + && mv dumb-init /usr/local/bin/dumb-init \ + && dumb-init --version \ + && rm -rf "$builddir" \ + && apk del .build-deps + +FROM ${REGISTRY_PREFIX}arm32v7/alpine:${DISTRIB_VERSION} + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +COPY --from=builder /usr/bin/qemu-arm-static /usr/bin/qemu-arm-static +COPY --from=builder /usr/local /usr/local +COPY --from=builder /etc/localtime /etc/localtime +COPY --from=builder /etc/timezone /etc/timezone + +RUN set -x \ + && apk add --no-cache \ + coreutils \ + gcc \ + g++ \ + make \ + cmake \ + ninja \ + pkgconf \ + gdb \ + valgrind + +ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib" +ENV PKG_CONFIG_PATH=/usr/local/lib32/pkgconfig + +ENTRYPOINT ["dumb-init", "--"] +CMD [ "/bin/sh" ] + +ONBUILD ARG USERID=1000 +ONBUILD RUN set -x \ + && adduser -u "$USERID" -s /bin/bash -D user + +# unused +ARG DISTRIB_ID= +ARG SYS= +ARG ABI= +ARG MARCH= +ARG HOSTMARCH= +ARG ID= +ARG VARIANT= diff --git a/assets/projects/cmake/arm32v7-ubuntu-builder-template.dockerfile b/assets/projects/cmake/arm32v7-ubuntu-builder-template.dockerfile new file mode 100644 index 0000000..7cbcdbd --- /dev/null +++ b/assets/projects/cmake/arm32v7-ubuntu-builder-template.dockerfile @@ -0,0 +1,91 @@ +ARG REGISTRY_PREFIX='' +ARG DISTRIB_VERSION=bionic + +FROM ${REGISTRY_PREFIX}arm32v7/ubuntu:${DISTRIB_VERSION} as builder + +ARG QEMU_VERSION_=v3.1.0-2 +COPY bin/qemu-arm-static-$QEMU_VERSION_ /usr/bin/qemu-arm-static + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +RUN set -x \ + && installdeps="tzdata" \ + && apt-get update \ + && apt-get install --yes --no-install-recommends $installdeps \ + && ls /usr/share/zoneinfo \ + && cp -H --remove-destination "/usr/share/zoneinfo/$TZ" /tmp/localtime \ + && { apt-get purge -y $installdeps || true; } \ + && mv /tmp/localtime /etc/localtime \ + && echo "$TZ" > /etc/timezone \ + && apt-get update \ + && apt-get install --yes --no-install-recommends \ + build-essential \ + && rm -rf /var/lib/apt/lists/* + +COPY src /usr/local/src + +ARG PARALLELMFLAGS=-j2 + +ARG DUMB_INIT_VERSION=1.2.2 +ARG DUMB_INIT_MTIME= +RUN set -x \ + && builddeps="vim-common" \ + && apt-get update \ + && apt-get install --yes --no-install-recommends $builddeps \ + && rm -rf /var/lib/apt/lists/* \ + && [ -n "$DUMB_INIT_MTIME" ] && export SOURCE_DATE_EPOCH="$DUMB_INIT_MTIME" \ + && builddir="/tmp/out" \ + && mkdir -p "$builddir" \ + && cd "$builddir" \ + && cp -R "/usr/local/src/dumb-init-$DUMB_INIT_VERSION" . \ + && cd "dumb-init-$DUMB_INIT_VERSION" \ + && make "$PARALLELMFLAGS" \ + && chmod +x dumb-init \ + && mv dumb-init /usr/local/bin/dumb-init \ + && dumb-init --version \ + && rm -rf "$builddir" \ + && apt-get purge -y $builddeps + +FROM ${REGISTRY_PREFIX}arm32v7/ubuntu:${DISTRIB_VERSION} + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +COPY --from=builder /usr/bin/qemu-arm-static /usr/bin/qemu-arm-static +COPY --from=builder /usr/local /usr/local +COPY --from=builder /etc/localtime /etc/localtime +COPY --from=builder /etc/timezone /etc/timezone +COPY --from=builder /etc/apt/sources.list.d /etc/apt/sources.list.d + +RUN set -x \ + && apt-get update \ + && apt-get install --yes --no-install-recommends \ + build-essential \ + cmake \ + ninja-build \ + pkg-config \ + gdb \ + gdbserver \ + valgrind \ + && rm -rf /var/lib/apt/lists/* + +ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib" + +ENTRYPOINT ["dumb-init", "--"] +CMD [ "/bin/bash" ] + +ONBUILD ARG USERID=1000 +ONBUILD RUN set -x \ + && useradd -u "$USERID" -ms /bin/bash user + +# unused +ARG DISTRIB_ID= +ARG SYS= +ARG ABI= +ARG MARCH= +ARG HOSTMARCH= +ARG ID= +ARG VARIANT= diff --git a/assets/projects/cmake/builder-template.mk b/assets/projects/cmake/builder-template.mk new file mode 100644 index 0000000..0cdc8db --- /dev/null +++ b/assets/projects/cmake/builder-template.mk @@ -0,0 +1,106 @@ +SHELL := /bin/sh +MAKEFILE := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)) +MAKEFLAGS += --no-builtin-rules + +.SUFFIXES: + +.PHONY: default +default: all + +####################################################################################################################### +# Overridable project defaults + +DOBUILD_TOPDIR ?= $(DOBUILDDIR) +DOBUILD_PROJECTDIR ?= $(patsubst %/,%,$(dir $(MAKEFILE))) +DOBUILD_PROJECTVERSIONFILE ?= $(DOBUILDDIR)/VERSION + +PROJECTDIR = $(DOBUILD_PROJECTDIR) +DOBUILDDIR ?= $(PROJECTDIR)/../../.. + +include $(DOBUILDDIR)/defaults.mk + +####################################################################################################################### +# Project defaults and macros + +DEFAULTTARGET = x86_64-alpine@3.10+builder-template + +FETCH_TARGETS = +FETCHDIR = $(BUILDDIR)/.deps + +####################################################################################################################### +# Project dependencies + +DUMB_INIT_VERSION = 1.2.2 +IMAGE_BUILDARGS += DUMB_INIT_VERSION=$(DUMB_INIT_VERSION) +IMAGE_BUILDARGS += DUMB_INIT_MTIME=$(call mtime,$(FETCHDIR)/dumb-init-$(DUMB_INIT_VERSION).tar.gz) +FETCH_TARGETS += $(FETCHDIR)/dumb-init-$(DUMB_INIT_VERSION).tar.gz +$(FETCHDIR)/dumb-init-$(DUMB_INIT_VERSION).tar.gz: URL = https://github.com/Yelp/dumb-init/archive/v$(DUMB_INIT_VERSION).tar.gz +$(SKIP_MD5SUM)$(FETCHDIR)/dumb-init-$(DUMB_INIT_VERSION).tar.gz: MD5 = 6166084b05772cdcf615a762c6f3b32e + +QEMU_VERSION = v3.1.0-2 +IMAGE_BUILDARGS += QEMU_VERSION_=$(QEMU_VERSION) +FETCH_TARGETS += $(FETCHDIR)/qemu-arm-static-$(QEMU_VERSION) +$(FETCHDIR)/qemu-arm-static-$(QEMU_VERSION): URL = https://github.com/multiarch/qemu-user-static/releases/download/$(QEMU_VERSION)/qemu-arm-static +$(SKIP_MD5SUM)$(FETCHDIR)/qemu-arm-static-$(QEMU_VERSION): MD5 = 8ebd24e63fdfa07c557d45373bd831b1 + +####################################################################################################################### +# Architecture-specific rule target configuration + +DOCKER_TARGETS += $(call target_properties_combine,\ + ,\ + x86_64 arm32v7,\ + ubuntu@xenial ubuntu@bionic alpine@3.9 alpine@3.10,\ + ,\ + ,\ + builder-template,\ + ,\ + \ + ) + +####################################################################################################################### +# Common rule target configuration + +CURLFLAGS += -s + +OUTDIRS += $(OUTDIR)/src +OUTDIRS += $(OUTDIR)/bin + +EXTRACT_TARGETS += $(OUTDIR)/bin/qemu-arm-static-$(QEMU_VERSION) +EXTRACT_TARGETS += $(patsubst $(FETCHDIR)/%.tar.gz,$(OUTDIR)/src/%,$(FETCH_TARGETS)) + +####################################################################################################################### +# Makefile dependencies + +MAKEFILE_DEPS += gzip +MAKEFILE_DEPS += tar +MAKEFILE_DEPS += chmod +MAKEFILE_DEPS += touch +MAKEFILE_DEPS += cp +MAKEFILE_DEPS += mkdir + +####################################################################################################################### +# Rules + +include $(DOBUILDDIR)/docker.mk +include $(DOBUILDDIR)/standardrules.mk + +$(FETCH_TARGETS): | $(FETCHDIR) + $(SILENT)$(call curl,$@,$(URL),$(MD5)) + +$(OUTDIR)/bin/qemu-arm-static-$(QEMU_VERSION) : $(FETCHDIR)/qemu-arm-static-$(QEMU_VERSION) | $(OUTDIRS) + $(SILENT) \ + $(call echo_if_silent_cmd,cp $< $@) \ + && cp $< $@ \ + && chmod +x $@ + +$(OUTDIR)/src/%: $(FETCHDIR)/%.tar.gz | $(OUTDIRS) + $(SILENT) \ + $(call echo_if_silent_cmd,tar -C $(dir $@) -xf $<) \ + && tar -I 'gzip -n' -C $(dir $@) -xf $< \ + && touch $@ + +$(FETCHDIR): + $(SILENT)mkdir -p $@ + +.DELETE_ON_ERROR: $(FETCH_TARGETS) + diff --git a/assets/projects/cmake/x86_64-alpine-builder-template.dockerfile b/assets/projects/cmake/x86_64-alpine-builder-template.dockerfile new file mode 100644 index 0000000..4ecb8f5 --- /dev/null +++ b/assets/projects/cmake/x86_64-alpine-builder-template.dockerfile @@ -0,0 +1,86 @@ +ARG REGISTRY_PREFIX='' +ARG DISTRIB_VERSION=3.9 + +FROM ${REGISTRY_PREFIX}alpine:${DISTRIB_VERSION} as builder + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +RUN set -x \ + && installdeps="tzdata" \ + && apk add --no-cache --virtual .install-deps $installdeps \ + && ls /usr/share/zoneinfo \ + && cp -H --remove-destination "/usr/share/zoneinfo/$TZ" /tmp/localtime \ + && apk del .install-deps \ + && mv /tmp/localtime /etc/localtime \ + && echo "$TZ" > /etc/timezone \ + && apk add --no-cache \ + bash \ + make \ + gcc \ + musl-dev + +COPY src /usr/local/src + +ARG PARALLELMFLAGS=-j2 + +ARG DUMB_INIT_VERSION=1.2.2 +ARG DUMB_INIT_MTIME= +RUN set -x \ + && builddeps="vim" \ + && apk add --no-cache --virtual .build-deps $builddeps \ + && [ -n "$DUMB_INIT_MTIME" ] && export SOURCE_DATE_EPOCH="$DUMB_INIT_MTIME" \ + && builddir="/tmp/out" \ + && mkdir -p "$builddir" \ + && cd "$builddir" \ + && cp -R "/usr/local/src/dumb-init-$DUMB_INIT_VERSION" . \ + && cd "dumb-init-$DUMB_INIT_VERSION" \ + && make "$PARALLELMFLAGS" \ + && chmod +x dumb-init \ + && mv dumb-init /usr/local/bin/dumb-init \ + && dumb-init --version \ + && rm -rf "$builddir" \ + && apk del .build-deps + +FROM ${REGISTRY_PREFIX}alpine:${DISTRIB_VERSION} + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +COPY --from=builder /usr/local /usr/local +COPY --from=builder /etc/localtime /etc/localtime +COPY --from=builder /etc/timezone /etc/timezone + +RUN set -x \ + && apk add --no-cache \ + coreutils \ + gcc \ + g++ \ + make \ + cmake \ + ninja \ + pkgconf \ + gdb \ + valgrind + +ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib" +ENV PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig + +ENTRYPOINT ["dumb-init", "--"] +CMD [ "/bin/sh" ] + +ONBUILD ARG USERID=1000 +ONBUILD RUN set -x \ + && adduser -u "$USERID" -s /bin/bash -D user + +# unused +ARG DISTRIB_ID= +ARG SYS= +ARG ABI= +ARG MARCH= +ARG HOSTMARCH= +ARG ID= +ARG VARIANT= +ARG QEMU_VERSION_= diff --git a/assets/projects/cmake/x86_64-ubuntu-builder-template.dockerfile b/assets/projects/cmake/x86_64-ubuntu-builder-template.dockerfile new file mode 100644 index 0000000..08f8425 --- /dev/null +++ b/assets/projects/cmake/x86_64-ubuntu-builder-template.dockerfile @@ -0,0 +1,88 @@ +ARG REGISTRY_PREFIX='' +ARG DISTRIB_VERSION=bionic + +FROM ${REGISTRY_PREFIX}ubuntu:${DISTRIB_VERSION} as builder + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +RUN set -x \ + && installdeps="tzdata" \ + && apt-get update \ + && apt-get install --yes --no-install-recommends $installdeps \ + && ls /usr/share/zoneinfo \ + && cp -H --remove-destination "/usr/share/zoneinfo/$TZ" /tmp/localtime \ + && { apt-get purge -y $installdeps || true; } \ + && mv /tmp/localtime /etc/localtime \ + && echo "$TZ" > /etc/timezone \ + && apt-get update \ + && apt-get install --yes --no-install-recommends \ + build-essential \ + && rm -rf /var/lib/apt/lists/* + +COPY src /usr/local/src + +ARG PARALLELMFLAGS=-j2 + +ARG DUMB_INIT_VERSION=1.2.2 +ARG DUMB_INIT_MTIME= +RUN set -x \ + && builddeps="vim-common" \ + && apt-get update \ + && apt-get install --yes --no-install-recommends $builddeps \ + && rm -rf /var/lib/apt/lists/* \ + && [ -n "$DUMB_INIT_MTIME" ] && export SOURCE_DATE_EPOCH="$DUMB_INIT_MTIME" \ + && builddir="/tmp/out" \ + && mkdir -p "$builddir" \ + && cd "$builddir" \ + && cp -R "/usr/local/src/dumb-init-$DUMB_INIT_VERSION" . \ + && cd "dumb-init-$DUMB_INIT_VERSION" \ + && make "$PARALLELMFLAGS" \ + && chmod +x dumb-init \ + && mv dumb-init /usr/local/bin/dumb-init \ + && dumb-init --version \ + && rm -rf "$builddir" \ + && apt-get purge -y $builddeps + +FROM ${REGISTRY_PREFIX}ubuntu:${DISTRIB_VERSION} + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +COPY --from=builder /usr/local /usr/local +COPY --from=builder /etc/localtime /etc/localtime +COPY --from=builder /etc/timezone /etc/timezone +COPY --from=builder /etc/apt/sources.list.d /etc/apt/sources.list.d + +RUN set -x \ + && apt-get update \ + && apt-get install --yes --no-install-recommends \ + build-essential \ + cmake \ + ninja-build \ + pkg-config \ + gdb \ + gdbserver \ + valgrind \ + && rm -rf /var/lib/apt/lists/* + +ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib" + +ENTRYPOINT ["dumb-init", "--"] +CMD [ "/bin/bash" ] + +ONBUILD ARG USERID=1000 +ONBUILD RUN set -x \ + && useradd -u "$USERID" -ms /bin/bash user + +# unused +ARG DISTRIB_ID= +ARG SYS= +ARG ABI= +ARG MARCH= +ARG HOSTMARCH= +ARG VARIANT= +ARG ID= +ARG QEMU_VERSION_= diff --git a/assets/projects/dobuild/builder-template.mk b/assets/projects/dobuild/builder-template.mk new file mode 100644 index 0000000..38837c8 --- /dev/null +++ b/assets/projects/dobuild/builder-template.mk @@ -0,0 +1,120 @@ +SHELL := /bin/sh +MAKEFILE := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)) +MAKEFLAGS += --no-builtin-rules + +.SUFFIXES: + +.PHONY: default +default: all + +####################################################################################################################### +# Overridable project defaults + +DOBUILD_TOPDIR ?= $(DOBUILDDIR) +DOBUILD_PROJECTDIR ?= $(patsubst %/,%,$(dir $(MAKEFILE))) +DOBUILD_PROJECTVERSIONFILE ?= $(DOBUILDDIR)/VERSION + +PROJECTDIR = $(DOBUILD_PROJECTDIR) +DOBUILDDIR ?= $(PROJECTDIR)/../../.. + +include $(DOBUILDDIR)/defaults.mk + +####################################################################################################################### +# Project defaults and macros + +DEFAULTTARGET = x86_64-alpine@3.10+builder-template + +FETCH_TARGETS = +FETCHDIR = $(BUILDDIR)/.deps + +####################################################################################################################### +# Project dependencies + +DUMB_INIT_VERSION = 1.2.2 +IMAGE_BUILDARGS += DUMB_INIT_VERSION=$(DUMB_INIT_VERSION) +IMAGE_BUILDARGS += DUMB_INIT_MTIME=$(call mtime,$(FETCHDIR)/dumb-init-$(DUMB_INIT_VERSION).tar.gz) +FETCH_TARGETS += $(FETCHDIR)/dumb-init-$(DUMB_INIT_VERSION).tar.gz +$(FETCHDIR)/dumb-init-$(DUMB_INIT_VERSION).tar.gz: URL = https://github.com/Yelp/dumb-init/archive/v$(DUMB_INIT_VERSION).tar.gz +$(SKIP_MD5SUM)$(FETCHDIR)/dumb-init-$(DUMB_INIT_VERSION).tar.gz: MD5 = 6166084b05772cdcf615a762c6f3b32e + +DOCKER_VERSION = 18.09.6 +IMAGE_BUILDARGS += DOCKER_VERSION=$(DOCKER_VERSION) +FETCH_TARGETS += $(FETCHDIR)/docker-$(DOCKER_VERSION)-x86_64.tgz +$(FETCHDIR)/docker-$(DOCKER_VERSION)-x86_64.tgz: URL = https://download.docker.com/linux/static/stable/x86_64/docker-$(DOCKER_VERSION).tgz +$(SKIP_MD5SUM)$(FETCHDIR)/docker-$(DOCKER_VERSION)-x86_64.tgz: MD5 = a6be1e734421d05abfc4d3e28997e271 + +####################################################################################################################### +# Architecture-specific rule target configuration + +DOCKER_TARGETS += $(call target_properties_combine,\ + ,\ + x86_64,\ + alpine@3.9 alpine@3.10,\ + ,\ + ,\ + builder-template,\ + ,\ + \ + ) + +####################################################################################################################### +# Common rule target configuration + +CURLFLAGS += -s + +OUTDIRS += $(OUTDIR)/src +OUTDIRS += $(OUTDIR)/bin +OUTDIRS += $(OUTDIR)/src/docker-$(DOCKER_VERSION)-x86_64 +OUTDIRS += $(OUTDIR)/assets/ca-certificates + +EXTRACT_TARGETS += $(patsubst $(FETCHDIR)/%.tar.gz,$(OUTDIR)/src/%,$(FETCH_TARGETS)) +EXTRACT_TARGETS += $(patsubst $(DOBUILDDIR)/%,$(OUTDIR)/%,$(wildcard $(DOBUILDDIR)/assets/ca-certificates/*.crt)) +EXTRACT_TARGETS += $(OUTDIR)/bin/docker-$(DOCKER_VERSION)-x86_64 + +####################################################################################################################### +# Makefile dependencies + +MAKEFILE_DEPS += gzip +MAKEFILE_DEPS += tar +MAKEFILE_DEPS += chmod +MAKEFILE_DEPS += touch +MAKEFILE_DEPS += cp +MAKEFILE_DEPS += mkdir + +####################################################################################################################### +# Rules + +include $(DOBUILDDIR)/docker.mk +include $(DOBUILDDIR)/standardrules.mk + +$(FETCH_TARGETS): | $(FETCHDIR) + $(SILENT)$(call curl,$@,$(URL),$(MD5)) + +$(OUTDIR)/assets/ca-certificates/%.crt : $(DOBUILDDIR)/assets/ca-certificates/%.crt | $(OUTDIRS) + cp $< $@ + +$(OUTDIR)/bin/docker-$(DOCKER_VERSION)-x86_64 : $(OUTDIR)/src/docker-$(DOCKER_VERSION)-x86_64/docker/docker + $(SILENT) \ + $(call echo_if_silent_cmd,cp $< $@) \ + && cp $< $@ \ + && chmod +x $@ + +$(OUTDIR)/src/docker-$(DOCKER_VERSION)-x86_64/docker/docker: $(OUTDIR)/src/docker-$(DOCKER_VERSION)-x86_64/docker + +$(OUTDIR)/src/docker-$(DOCKER_VERSION)-x86_64/docker: $(FETCHDIR)/docker-$(DOCKER_VERSION)-x86_64.tgz | $(OUTDIRS) + $(SILENT) \ + $(call echo_if_silent_cmd,tar -C $(dir $@) -xf $<) \ + && tar -I 'gzip -n' -C $(dir $@) -xf $< \ + && touch $@ + +$(OUTDIR)/src/%: $(FETCHDIR)/%.tar.gz | $(OUTDIRS) + $(SILENT) \ + $(call echo_if_silent_cmd,tar -C $(dir $@) -xf $<) \ + && tar -I 'gzip -n' -C $(dir $@) -xf $< \ + && touch $@ + +$(FETCHDIR): + $(SILENT)mkdir -p $@ + +.DELETE_ON_ERROR: $(FETCH_TARGETS) + diff --git a/assets/projects/dobuild/x86_64-alpine-builder-template.dockerfile b/assets/projects/dobuild/x86_64-alpine-builder-template.dockerfile new file mode 100644 index 0000000..d319e08 --- /dev/null +++ b/assets/projects/dobuild/x86_64-alpine-builder-template.dockerfile @@ -0,0 +1,90 @@ +ARG REGISTRY_PREFIX='' +ARG DISTRIB_VERSION=3.9 + +FROM ${REGISTRY_PREFIX}alpine:${DISTRIB_VERSION} as builder + +ARG MARCH=x86_64 + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +RUN set -x \ + && installdeps="tzdata" \ + && apk add --no-cache --virtual .install-deps $installdeps \ + && ls /usr/share/zoneinfo \ + && cp -H --remove-destination "/usr/share/zoneinfo/$TZ" /tmp/localtime \ + && apk del .install-deps \ + && mv /tmp/localtime /etc/localtime \ + && echo "$TZ" > /etc/timezone \ + && apk add --no-cache \ + bash \ + make \ + gcc \ + musl-dev + +COPY src /usr/local/src + +ARG PARALLELMFLAGS=-j2 + +ARG DUMB_INIT_VERSION=1.2.2 +ARG DUMB_INIT_MTIME= +RUN set -x \ + && builddeps="vim" \ + && apk add --no-cache --virtual .build-deps $builddeps \ + && [ -n "$DUMB_INIT_MTIME" ] && export SOURCE_DATE_EPOCH="$DUMB_INIT_MTIME" \ + && builddir="/tmp/out" \ + && mkdir -p "$builddir" \ + && cd "$builddir" \ + && cp -R "/usr/local/src/dumb-init-$DUMB_INIT_VERSION" . \ + && cd "dumb-init-$DUMB_INIT_VERSION" \ + && make "$PARALLELMFLAGS" \ + && chmod +x dumb-init \ + && mv dumb-init /usr/local/bin/dumb-init \ + && dumb-init --version \ + && rm -rf "$builddir" \ + && apk del .build-deps + +ARG DOCKER_VERSION=17.06.0-ce +ARG DOCKER_MARCH=$MARCH +COPY bin/docker-$DOCKER_VERSION-$DOCKER_MARCH /usr/local/bin/docker + +FROM ${REGISTRY_PREFIX}alpine:${DISTRIB_VERSION} + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +ARG CAPATH="./assets/ca-certificates" +COPY $CAPATH /usr/local/share/ca-certificates +COPY --from=builder /usr/local /usr/local +COPY --from=builder /etc/localtime /etc/localtime +COPY --from=builder /etc/timezone /etc/timezone + +RUN set -x \ + && apk add --no-cache \ + make \ + gzip \ + git \ + curl \ + openssl \ + ca-certificates \ + && update-ca-certificates + +ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib" + +ENTRYPOINT ["dumb-init", "--"] +CMD [ "/bin/sh" ] + +ONBUILD ARG USERID=1000 +ONBUILD RUN set -x \ + && adduser -u "$USERID" -s /bin/bash -D user + +# unused +ARG DISTRIB_ID= +ARG SYS= +ARG ABI= +ARG MARCH= +ARG HOSTMARCH= +ARG ID= +ARG VARIANT= \ No newline at end of file diff --git a/assets/projects/gradle/builder-template.mk b/assets/projects/gradle/builder-template.mk new file mode 100644 index 0000000..b949e88 --- /dev/null +++ b/assets/projects/gradle/builder-template.mk @@ -0,0 +1,88 @@ +SHELL := /bin/sh +MAKEFILE := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)) +MAKEFLAGS += --no-builtin-rules + +.SUFFIXES: + +.PHONY: default +default: all + +####################################################################################################################### +# Overridable project defaults + +DOBUILD_TOPDIR ?= $(DOBUILDDIR) +DOBUILD_PROJECTDIR ?= $(patsubst %/,%,$(dir $(MAKEFILE))) +DOBUILD_PROJECTVERSIONFILE ?= $(DOBUILDDIR)/VERSION +DOBUILD_DOCKERFILE ?= $(PROJECTDIR)/%MARCH%-%DISTRIB_ID%.dockerfile + +PROJECTDIR = $(DOBUILD_PROJECTDIR) +DOBUILDDIR ?= $(PROJECTDIR)/../../.. + +include $(DOBUILDDIR)/defaults.mk + +####################################################################################################################### +# Project defaults and macros + +DEFAULTTARGET = x86_64-graalvm~-ce@20.0.0-linux-java11+6.3 + +FETCH_TARGETS = +FETCHDIR = $(BUILDDIR)/.deps + +dobuild_image_prerequisites = %OUTDIR%/bin/gradle-%ID% + +####################################################################################################################### +# Project dependencies + +FETCH_TARGETS += $(FETCHDIR)/gradle-6.3.zip +$(FETCHDIR)/gradle-6.3.zip: URL = https://services.gradle.org/distributions/gradle-6.3-bin.zip +$(SKIP_MD5SUM)$(FETCHDIR)/gradle-6.3.zip: MD5 = 737c68904f35e6480fa013b1eb3c9c50 + +FETCH_TARGETS += $(FETCHDIR)/gradle-5.6.zip +$(FETCHDIR)/gradle-5.6.zip: URL = https://services.gradle.org/distributions/gradle-5.6-bin.zip +$(SKIP_MD5SUM)$(FETCHDIR)/gradle-5.6.zip: MD5 = 2dde6806b36fe0832a7438752be6ed36 + +####################################################################################################################### +# Architecture-specific rule target configuration + +DOCKER_TARGETS += $(call target_properties_combine,\ + ,\ + x86_64,\ + graalvm~-ce,\ + linux,\ + java11 java8,\ + 6.3 5.6,\ + 20.0.0,\ + \ + ) + +####################################################################################################################### +# Common rule target configuration + +CURLFLAGS += -s + +OUTDIRS += $(OUTDIR)/bin + +####################################################################################################################### +# Makefile dependencies + +MAKEFILE_DEPS += unzip + +####################################################################################################################### +# Rules + +include $(DOBUILDDIR)/docker.mk +include $(DOBUILDDIR)/standardrules.mk + +$(FETCH_TARGETS): | $(FETCHDIR) + $(SILENT)$(call curl,$@,$(URL),$(MD5)) + +$(OUTDIR)/bin/%: $(FETCHDIR)/%.zip | $(OUTDIRS) + $(SILENT) \ + $(call echo_if_silent_cmd,unzip -d $(dir $@) $<) \ + && unzip -o -d $(dir $@) $< \ + && touch $@ + +$(FETCHDIR): + $(SILENT)mkdir -p $@ + +.DELETE_ON_ERROR: $(FETCH_TARGETS) diff --git a/assets/projects/gradle/x86_64-graalvm-ce.dockerfile b/assets/projects/gradle/x86_64-graalvm-ce.dockerfile new file mode 100644 index 0000000..d258806 --- /dev/null +++ b/assets/projects/gradle/x86_64-graalvm-ce.dockerfile @@ -0,0 +1,33 @@ +ARG REGISTRY_PREFIX='' +ARG SYS +ARG ABI +ARG DISTRIB_ID +ARG DISTRIB_VERSION + +FROM ${REGISTRY_PREFIX}oracle/${DISTRIB_ID}:${DISTRIB_VERSION}-${ABI} + +RUN set -x \ + && gu install native-image + +ARG ID +ARG GRADLE_HOME=/usr/local/lib/gradle-${ID} +COPY bin/gradle-${ID} $GRADLE_HOME +ENV PATH=$PATH:$GRADLE_HOME/bin + +RUN set -x \ + && java -version \ + && javac -version \ + && native-image --version \ + && gradle --version + +ARG TZ=UTC +ENV LANG=C +ENV LC_ALL=${LANG} + +CMD [ "/bin/bash" ] + +# unused +ARG PARALLELMFLAGS= +ARG MARCH= +ARG HOSTMARCH= +ARG VARIANT= diff --git a/assets/run_image.sh.template b/assets/run_image.sh.template new file mode 100644 index 0000000..e532913 --- /dev/null +++ b/assets/run_image.sh.template @@ -0,0 +1,50 @@ +#!/bin/sh + +set -e + +DOCKER="${DOCKER:-%DOCKER%}" +IMAGE="%IMAGE%" +NETWORK="${NETWORK:-host}" +PROJECTDIR="${PROJECTDIR:-%PROJECTDIR%}" +DOBUILDDIR="${DOBUILDDIR:-%DOBUILDDIR%}" +ENTRYPOINT="${ENTRYPOINT:-%RUNCMD%}" +PATH="${DOBUILDDIR}/bin:$PATH" + +set -- --entrypoint "$ENTRYPOINT" "$IMAGE" "$@" +set -- --network "$NETWORK" --workdir "$PWD" "$@" + +if [ -n "$USERID" ]; then + set -- --user "$USERID:$USERID" "$@" +fi + +if [ -n "$PID" ]; then + set -- --pid "$PID" "$@" +fi + +if [ -n "$CONTAINER_CGROUP_PARENT" ]; then + set -- --cgroup-parent "$CONTAINER_CGROUP_PARENT" "$@" +fi + +HOST_CONTAINER="${HOST_CONTAINER:-"$(get_container_id.sh)"}" || true +if [ -n "$HOST_CONTAINER" ]; then + set -- --volumes-from "$HOST_CONTAINER" "$@" +else + set -- --volume "$PROJECTDIR:$PROJECTDIR:cached" "$@" +fi + +set -- %RUNFLAGS% "$@" + +if [ -t 0 ] && ! is_running_in_bg.sh $$; then + set -- --interactive "$@" +fi + +# if STDIN piped or redirected +if [ -p /dev/stdin ] || { [ ! -t 0 ] && [ ! -p /dev/stdin ]; }; then + set -- --interactive "$@" +elif [ -t 1 ]; then + set -- --tty "$@" +fi + +set -- --rm "$@" + +exec "$DOCKER" run "$@" diff --git a/assets/templates/cmake.properties.template b/assets/templates/cmake.properties.template new file mode 100644 index 0000000..5fdabbc --- /dev/null +++ b/assets/templates/cmake.properties.template @@ -0,0 +1,3 @@ +CMAKE_GENERATOR='%CMAKE_GENERATOR%' +CMAKE_GENERATOR_NAME='%CMAKE_GENERATOR_CMD%' +CMAKE_GENERATOR_CMD='%CMAKE_GENERATOR_CMD%' diff --git a/assets/templates/md5sum.txt.template b/assets/templates/md5sum.txt.template new file mode 100644 index 0000000..a7f6df5 --- /dev/null +++ b/assets/templates/md5sum.txt.template @@ -0,0 +1 @@ +%MD5% %FILE% \ No newline at end of file diff --git a/assets/templates/properties.template b/assets/templates/properties.template new file mode 100644 index 0000000..afdc180 --- /dev/null +++ b/assets/templates/properties.template @@ -0,0 +1,17 @@ +TARGET='%TARGET%' +NPROC="$DOBUILD_NPROC" +VERBOSE="$DOBUILD_VERBOSE" +BUILDVERBOSE="$DOBUILD_BUILDVERBOSE" +TESTVERBOSE="$DOBUILD_TESTVERBOSE" +PROJECTDIR='%PROJECTDIR%' +DOBUILDDIR='%DOBUILDDIR%' +HOSTMARCH='%HOSTMARCH%' +MARCH='%MARCH%' +DISTRIB='%DISTRIB%' +DISTRIB_ID='%DISTRIB_ID%' +DISTRIB_VERSION='%DISTRIB_VERSION%' +SYS='%SYS%' +ABI='%ABI%' +ID='%ID%' +VARIANT='%VARIANT%' +BUILD_TESTING='%BUILD_TESTING%' diff --git a/bin/container_run b/bin/container_run new file mode 100755 index 0000000..6ea75e9 --- /dev/null +++ b/bin/container_run @@ -0,0 +1,95 @@ +#!/bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +set -e + +enabled() { + { [ "$1" -ne 0 ] || [ "$1" = 'true' ]; } 2>/dev/null +} + +physical_pwd() { + pwd -P 2>/dev/null || pwd +} + +try_canonicalize() { + readlink -f "$@" 2>/dev/null || realpath "$@" +} + +canonicalize() { + if ! try_canonicalize "$1" 2>/dev/null; then + echo "$(cd "$(dirname "$1")" && physical_pwd)/$(basename "$1")" + fi +} + +scriptdir() { + dirname "$(canonicalize "${BASH_SOURCE:-$1}")" +} + +DOBUILDDIR="${DOBUILDDIR:-"$(dirname "$(scriptdir "$0")")"}" +PATH="$DOBUILDDIR/bin:$PATH" + +DOCKER="${DOCKER:-docker}" +USERID="${DOBUILD_USERID:-$(id -u)}" +HOSTENVFILTER="^DOCKER_\|_proxy=\|^SOURCE_DATE_EPOCH=${DOBUILD_HOSTENVFILTER:+\|$DOBUILD_HOSTENVFILTER}" + +ENVFLAGS="$(printenv | grep -e "$HOSTENVFILTER" | sed -n -e 's/\([^=]*\)=.*/-e \1/p')" || true +#shellcheck disable=SC2086 +set -- $ENVFLAGS "$@" + +set -- --user "$USERID:$USERID" "$@" + +if [ -n "$DOBUILD_CGROUPPARENT" ]; then + set -- --cgroup-parent "$DOBUILD_CGROUPPARENT" "$@" +fi + +HOST_TZ="${DOBUILD_HOSTTZ:-1}" +if enabled "$HOST_TZ"; then + set -- -v "/etc/timezone:/etc/timezone:ro" "$@" + set -- -v "/etc/localtime:/etc/localtime:ro" "$@" +fi + +HOST_CONTAINER="${DOBUILD_HOSTCONTAINER:-"$(get_container_id.sh)"}" || true +if [ -n "$HOST_CONTAINER" ]; then + set -- --volumes-from "$HOST_CONTAINER" "$@" +elif [ -n "$DOBUILD_PROJECTDIR" ]; then + PROJECTDIR="$(canonicalize "$DOBUILD_PROJECTDIR")" + set -- --volume "$PROJECTDIR:$PROJECTDIR:cached" "$@" +fi + +# setup options for connection to docker host +DOCKERSOCK="$(echo "$DOCKER_HOST" | sed -ne 's/^unix:\/\/\(.*\)/\1/p')" +if [ -S "$DOCKERSOCK" ]; then + DOCKERSOCK_GROUP="$(stat -c '%g' "$DOCKERSOCK")" + set -- -e DOCKERSOCK_GROUP="$DOCKERSOCK_GROUP" --group-add "$DOCKERSOCK_GROUP" --volume "$DOCKERSOCK:$DOCKERSOCK" "$@" +elif [ -n "$DOCKER_HOST" ]; then + # select the network to reach the daemon + if [ -n "$HOST_CONTAINER" ] && [ -z "${DOBUILD_NETWORK+x}" ]; then + set -- --network "$("$DOCKER" inspect --format='{{.HostConfig.NetworkMode}}' "$HOST_CONTAINER")" "$@" + else + DOBUILD_NETWORK="${DOBUILD_NETWORK:-host}" + set -- --network "$DOBUILD_NETWORK" "$@" + fi +fi + +if [ -t 0 ] && ! is_running_in_bg.sh $$; then + set -- --interactive "$@" +fi + +# if STDIN piped or redirected +if [ -p /dev/stdin ] || { [ ! -t 0 ] && [ ! -p /dev/stdin ]; }; then + set -- --interactive "$@" +elif [ -t 1 ]; then + set -- --tty "$@" +fi + +set -- "$DOCKER" run --rm "$@" + +exec "$@" diff --git a/bin/dobuild b/bin/dobuild new file mode 100755 index 0000000..f932a72 --- /dev/null +++ b/bin/dobuild @@ -0,0 +1,240 @@ +#!/bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +set -e + +physical_pwd() { + pwd -P 2>/dev/null || pwd +} + +try_canonicalize() { + readlink -f "$@" 2>/dev/null || realpath "$@" +} + +canonicalize() { + if ! try_canonicalize "$1" 2>/dev/null; then + echo "$(cd "$(dirname "$1")" && physical_pwd)/$(basename "$1")" + fi +} + +scriptdir() { + dirname "$(canonicalize "${BASH_SOURCE:-$1}")" +} + +template_subst() { + sed \ + -e "s!%MARCH%!${MARCH}!g" \ + -e "s!%DISTRIB%!${DISTRIB}!g" \ + -e "s!%REGISTRY_PREFIX%!${REGISTRY_PREFIX}!g" +} + +mktemp_dockerfile() { + mkdir -p "$1" + echo "$1/dobuild_generated.dockerfile" +} + +findonly_file() { + workingdir="$1" + name="$2" + filter="$3" + + find "$workingdir" -maxdepth 1 -type f -name "$name" \ + | grep "^${workingdir}/${filter}" \ + | { + firstfound= + lastfound= + + while IFS= read -r line; do + firstfound="${firstfound:-$line}" + lastfound="$line" + done + + if [ "$firstfound" != "$lastfound" ]; then + printf 'error: more than one match found for %s in %s using filter %s\n' "$name" "$workingdir" "$filter" 1>&2 + return 3 + fi + + echo "$firstfound" + } +} + +export LANG=C +export LC_ALL=C + +DOBUILDDIR="${DOBUILDDIR:-"$(dirname "$(scriptdir "$0")")"}" +MAKE="${MAKE:-$(command -v make)}" +PATH="$DOBUILDDIR/bin:$PATH" + +set -- "$@" -- + +FILTER="$DOBUILD_FILTER" +WORKINGDIR= +MAKEFILE= +DISTRIB= +MARCH= +IMAGE= + +while :; do + case $1 in + -f|--file|--makefile) + if [ "$2" != '--' ]; then + MAKEFILE="${MAKEFILE:-$2}" + set -- "$@" "$1" "$2" + shift + else + printf 'error: "%s" requires a non-empty option argument.\n' "$1" >&2 + exit 3 + fi + ;; + --file=|--makefile=) + printf 'error: "%s" requires a non-empty option argument.\n' "$1" >&2 + exit 3 + ;; + --file=?*|--makefile=?*) + MAKEFILE="${MAKEFILE:-${1#*=}}" + set -- "$@" "$1" + ;; + -C|--directory) + if [ "$2" != '--' ]; then + WORKINGDIR="$2" + shift + else + printf 'error: "%s" requires a non-empty option argument.\n' "$1" >&2 + exit 3 + fi + ;; + --directory=) + printf 'error: "%s" requires a non-empty option argument.\n' "$1" >&2 + exit 3 + ;; + --directory=?*) + WORKINGDIR="${1#*=}" + ;; + --dockerfile) + if [ "$2" != '--' ]; then + DOCKERFILE="$2" + shift + else + printf 'warning: "%s" ignored, requires a non-empty option argument.\n' "$1" >&2 + fi + ;; + --dockerfile=) + printf 'warning: "%s" ignored, requires a non-empty option argument.\n' "$1" >&2 + ;; + --dockerfile=?*|DOCKERFILE=?*) + DOCKERFILE="${1#*=}" + ;; + --image) + if [ "$2" != '--' ]; then + IMAGE="$2" + shift + else + printf 'error: "%s" requires a non-empty option argument.\n' "$1" >&2 + exit 3 + fi + ;; + --image=) + printf 'error: "%s" requires a non-empty option argument.\n' "$1" >&2 + exit 3 + ;; + --image=?*|IMAGE=?*) + IMAGE="${1#*=}" + ;; + --filter) + if [ "$2" != '--' ]; then + FILTER="$2" + set -- "$@" "FILTER=$FILTER" + shift + else + printf 'warning: "%s" ignored, requires a non-empty option argument.\n' "$1" >&2 + fi + ;; + --filter=) + printf 'warning: "%s" ignored, requires a non-empty option argument.\n' "$1" >&2 + ;; + --filter=?*|FILTER=?*) + FILTER="${1#*=}" + set -- "$@" "FILTER=$FILTER" + ;; + MARCH=?*) + MARCH="${1#*=}" + ;; + DISTRIB=?*) + DISTRIB="${1#*=}" + ;; + PROJECTDIR=?*) + PROJECTDIR="${1#*=}" + ;; + --) + shift + break + ;; + *) + set -- "$@" "$1" + ;; + esac + + shift +done + +if [ -n "$WORKINGDIR" ]; then + cd "$WORKINGDIR" +fi + +PROJECTDIR="${DOBUILD_PROJECTDIR:-.}" +OUTDIR="${DOBUILD_OUTDIR:-$PROJECTDIR/.build}" +DISTRIB_SUFFIX="${DISTRIB:+-${DISTRIB}}" +MARCH_DISTRIB="${MARCH:-[^-]*}${DISTRIB_SUFFIX}" +FILTER="${FILTER:-${MARCH_DISTRIB:+${MARCH_DISTRIB}-}[^.]*}" +BUILD_DOCKERFILE="${DOCKERFILE:-$(findonly_file "$PROJECTDIR" '*.dockerfile' "$FILTER")}" +BUILD_DOCKERFILE="${BUILD_DOCKERFILE:-$(find "$PROJECTDIR" -maxdepth 1 -type f -name 'Dockerfile')}" || true + +if [ -z "$MAKEFILE" ]; then + if [ -f "$PROJECTDIR/CMakeLists.txt" ]; then + set -- --file="$DOBUILDDIR/examples/cmake-gtest-example/Makefile" "$@" + elif [ -f "$PROJECTDIR/Makefile" ] || [ -f "$PROJECTDIR/makefile" ] || [ -f "$PROJECTDIR/GNUmakefile" ]; then + printf 'error: makefile project support not implemented.\n' >&2 + exit 3 + else + printf 'error: unknown project kind.\n' >&2 + exit 3 + fi +fi + +set -- "$@" "PROJECTDIR=$PROJECTDIR" "DOBUILDDIR=$DOBUILDDIR" "OUTDIR=$OUTDIR" + +if [ -n "$IMAGE" ]; then + BUILD_DOCKERFILE="$(mktemp_dockerfile "$OUTDIR")" + + { \ + echo "FROM %REGISTRY_PREFIX%${IMAGE}"; \ + echo; \ + } | template_subst > "$BUILD_DOCKERFILE" +fi + +if [ -n "$BUILD_DOCKERFILE" ]; then + if [ "$BUILD_DOCKERFILE" = '-' ]; then + BUILD_DOCKERFILE="$(mktemp_dockerfile "$OUTDIR")" + template_subst > "$BUILD_DOCKERFILE" + fi + + set -- "$@" "DOCKERFILE=$BUILD_DOCKERFILE" +fi + +if [ -n "$MARCH" ]; then + set -- "$@" "MARCH=$MARCH" +fi + +if [ -n "$DISTRIB" ]; then + set -- "$@" "DISTRIB=$DISTRIB" +fi + +exec "$MAKE" "$@" diff --git a/bin/docker_compose b/bin/docker_compose new file mode 100755 index 0000000..0a32d30 --- /dev/null +++ b/bin/docker_compose @@ -0,0 +1,85 @@ +#!/bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +set -e + +physical_pwd() { + pwd -P 2>/dev/null || pwd +} + +try_canonicalize() { + readlink -f "$@" 2>/dev/null || realpath "$@" +} + +canonicalize() { + if ! try_canonicalize "$1" 2>/dev/null; then + echo "$(cd "$(dirname "$1")" && physical_pwd)/$(basename "$1")" + fi +} + +scriptdir() { + dirname "$(canonicalize "${BASH_SOURCE:-$1}")" +} + +DOBUILDDIR="${DOBUILDDIR:-"$(dirname "$(scriptdir "$0")")"}" +PATH="$DOBUILDDIR/bin:$PATH" + +DOBUILD_COMPOSEVERSION="${DOBUILD_COMPOSEVERSION:-1.24.0}" +DOBUILD_COMPOSEIMAGE="${REGISTRY_PREFIX}docker/compose:${DOBUILD_COMPOSEVERSION}" +DOBUILD_COMPOSEENTRYPOINT="${DOBUILD_COMPOSEENTRYPOINT:-docker-compose}" + +export DOBUILDDIR +export DOBUILD_PROJECTDIR="${DOBUILD_COMPOSEPROJECTDIR:-"$PWD"}" +export DOBUILD_HOSTENVFILTER="${DOBUILD_COMPOSEHOSTENVFILTER:-^COMPOSE}" + +set -- "$@" -- + +WORKINGDIR= + +while :; do + case $1 in + --project-directory) + if [ "$2" != '--' ]; then + WORKINGDIR="$2" + shift + else + printf 'error: "%s" requires a non-empty option argument.\n' "$1" >&2 + exit 3 + fi + ;; + --project-directory=) + printf 'error: "%s" requires a non-empty option argument.\n' "$1" >&2 + exit 3 + ;; + --project-directory=?*) + WORKINGDIR="${1#*=}" + ;; + --) + shift + break + ;; + *) + set -- "$@" "$1" + ;; + esac + + shift +done + +WORKINGDIR="${WORKINGDIR:-"$DOBUILD_PROJECTDIR"}" + +if [ -z "${DOCKER_HOST+x}" ]; then + export DOCKER_HOST='unix:///var/run/docker.sock' +fi + +set -- container_run --workdir "$(canonicalize "$WORKINGDIR")" --entrypoint "$DOBUILD_COMPOSEENTRYPOINT" "$DOBUILD_COMPOSEIMAGE" "$@" + +exec "$@" diff --git a/bin/get_container_id.sh b/bin/get_container_id.sh new file mode 100755 index 0000000..9db1fa6 --- /dev/null +++ b/bin/get_container_id.sh @@ -0,0 +1,39 @@ +#!/bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +set -e + +container_id() { + if [ "$#" -le 0 ]; then + container_id_by_cgroup < /proc/self/cgroup + else + container_id_by_object "$@" + fi +} + +container_id_by_object() { + "$DOCKER" inspect --format='{{.Id}}' "$@" +} + +container_id_by_cgroup() { + while IFS= read -r cmd; do + id="$(echo "$cmd" | sed -n -e 's/[^:]*:[^:]*:.*\/\([0-9a-fA-F]\+\)$/\1/p')" + if container_id_by_object "$id" >/dev/null 2>&1; then + echo "$id" + return 0 + fi + done + + return 3 +} + +DOCKER="${DOCKER:-docker}" +container_id "$@" diff --git a/bin/get_source_date_epoch b/bin/get_source_date_epoch new file mode 100755 index 0000000..62421c0 --- /dev/null +++ b/bin/get_source_date_epoch @@ -0,0 +1,40 @@ +#!/bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +exec 0<&- + +set -e + +export LANG=C +export LC_ALL=C + +TOPDIR="$1" + +[ -z "$TOPDIR" ] || cd "$TOPDIR" + +try_git() { + [ -d .git ] || return 1 + git show -s --format=format:%ct HEAD +} + +try_svn() { + [ -d .svn ] || return 1 + LAST_CHANGED_DATE="$(svn info | sed -n -e 's/^Last Changed Date: //p')" + [ -n "$LAST_CHANGED_DATE" ] || return 2 + SOURCE_DATE_EPOCH="$(date -d "$LAST_CHANGED_DATE" +%s)" +} + +try_mtime() { + stat -c '%Y' "$PWD" +} + +SOURCE_DATE_EPOCH="$(try_git || try_svn || try_mtime)" +echo "$SOURCE_DATE_EPOCH" diff --git a/bin/groovy3 b/bin/groovy3 new file mode 100755 index 0000000..c52f465 --- /dev/null +++ b/bin/groovy3 @@ -0,0 +1,50 @@ +#!/bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +set -e + +physical_pwd() { + pwd -P 2>/dev/null || pwd +} + +try_canonicalize() { + readlink -f "$@" 2>/dev/null || realpath "$@" +} + +canonicalize() { + if ! try_canonicalize "$1" 2>/dev/null; then + echo "$(cd "$(dirname "$1")" && physical_pwd)/$(basename "$1")" + fi +} + +scriptdir() { + dirname "$(canonicalize "${BASH_SOURCE:-$1}")" +} + +DOBUILDDIR="${DOBUILDDIR:-"$(dirname "$(scriptdir "$0")")"}" +PATH="$DOBUILDDIR/bin:$PATH" + +DOBUILD_GROOVYVERSION="${DOBUILD_GROOVYVERSION:-3.0-jre13}" +DOBUILD_GROOVYIMAGE="${REGISTRY_PREFIX}groovy:${DOBUILD_GROOVYVERSION}" +DOBUILD_GROOVYENTRYPOINT="${DOBUILD_GROOVYENTRYPOINT:-groovy}" + +export DOBUILDDIR +export DOBUILD_PROJECTDIR="${DOBUILD_GROOVYPROJECTDIR:-"$PWD"}" + +set -- --entrypoint "$DOBUILD_GROOVYENTRYPOINT" "$DOBUILD_GROOVYIMAGE" "$@" + +if [ -n "$DOBUILD_GRAPESCACHE" ]; then + set -- -v "${DOBUILD_GRAPESCACHE}:/home/groovy/.groovy/grapes:delegated" "$@" +fi + +set -- container_run --workdir "$(canonicalize "$DOBUILD_PROJECTDIR")" "$@" + +exec "$@" diff --git a/bin/is_running_in_bg.sh b/bin/is_running_in_bg.sh new file mode 100755 index 0000000..6ff9601 --- /dev/null +++ b/bin/is_running_in_bg.sh @@ -0,0 +1,21 @@ +#!/bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +set -e + +running_in_background() { + pid="$1" + { "$PS" -o stat= -p "$pid" 2>/dev/null || echo '+'; } | grep -vq -e '+$' +} + +PS="${PS:-ps}" + +running_in_background "${@:-$$}" diff --git a/bin/parse_make_targets.sh b/bin/parse_make_targets.sh new file mode 100755 index 0000000..8794c6f --- /dev/null +++ b/bin/parse_make_targets.sh @@ -0,0 +1,67 @@ +#!/bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +set -e + +# Derived based on https://stackoverflow.com/questions/4219255/how-do-you-get-the-list-of-targets-in-a-makefile +# +# Note: This relies on -p printing the database nonetheless, which is the case as of GNU make 3.82. +# Sadly, GNU make offers no direct option to just print the database. +# +# Invokes make in order to print the database derived from a makefile: +# -p prints the database +# -Rr suppresses inclusion of built-in rules and variables +# -q only tests the up-to-date-status of a target (without remaking anything), but that by itself doesn't +# prevent execution of recipe commands in all cases +# "$@" list of separate commandline parameters (additional make parameters) +# : is a deliberately invalid target that is meant to ensure that no commands are executed; +# 2>&1 1>&3 +# redirect stderr to stdout and stdout to fd 3 to allow filtering unwanted error message +# grep -v '.*\s\+No rule to make target\s\+.*:.*' 1>&2 +# suppresses the resulting error message - "make: *** No rule to make target ':'. Stop." and redirect +# other output to stderr +# 3>&1 redirect fd 3 back to stdout +print_make_db() { + { "$MAKE" -pRrq "$@" : 2>&1 1>&3 | grep -v '.*\s\+No rule to make target\s\+.*:.*' 1>&2; } 3>&1 +} + +export LANG=C +export LC_ALL=C + +MAKE="${MAKE:-make}" + +print_make_db "$@" \ + | awk -v RS= -F: '/^# File/,/^# Finished Make database/ {if ($1 !~ "^[#.]") {print $1}}' \ + | sort \ + | grep -v -e '^[^[:alnum:]]' -e '[%]$' + +# Pipe-chain explanation: +# awk -v RS= -F: '/^# File/,/^# Finished Make database/ {if ($1 !~ "^[#.]") {print $1}}' +# v RS= this is an awk idiom that breaks the input into blocks of contiguous non-empty lines. +# -F: input field speparator +# /^# File/,/^# Finished Make database/ +# matches the range of lines in the output that contains all targets (true as of GNU make 3.82) - by limiting +# parsing to this range, there is no need to deal with false positives from other output sections. +# if ($$1 !~ "^[#.]") +# selectively ignores blocks: +# # ... ignores non-targets, whose blocks start with # Not a target: +# . ... ignores special targets +# all other blocks should each start with a line containing only the name of an explicitly defined target +# followed by : +# sort +# sorts the resulting list of targets, which is the best option, since not sorting doesn't produce a helpful +# ordering in that the order in which the targets appear in the makefile is not preserved. +# grep -v -e '^[^[:alnum:]]' -e '[%]$$' removes unwanted targets from the output: +# -v revert machtes +# -e '^[^[:alnum:]]' +# ... hidden targets, which - by convention - are targets that start neither with a letter nor a digit. +# -e '[%]$$' +# ... targets ending with % (pattern targets) diff --git a/bin/parse_target_properties.sh b/bin/parse_target_properties.sh new file mode 100755 index 0000000..c89ca98 --- /dev/null +++ b/bin/parse_target_properties.sh @@ -0,0 +1,147 @@ +#!/bin/sh -e + +# +---+ +# +# arch = x86_64, i386, arm, thumb, mips, etc. +# sub = for ex. on ARM: v5, v6m, v7a, v7m, etc. +# vendor = pc, apple, nvidia, ibm etc. +# sys = none, linux, win32, darwin, cuda, etc. +# abi = eabi, gnu, android, macho, elf, etc. + +sh_escape() { + sed \ + -e "s/\\([;\\*&\\|\\'\\\"\\\$\\(\\)]\\)/\\\\\\1/g" \ + -e "s/\\([^$ESCCHAR]\\)$ESCCHAR\\(\\s\\)/\\1$ESCCHAR\\\\\\2/g" \ + -e "s/$ESCCHAR/\\\\$ESCCHAR/" +} + +split() { + delimiter="$1" + shift + printf -- '%s\n' "$@" | sed -e "s/\\([^$ESCCHAR]\\)$delimiter/\\1\\n/g" +} + +unescape() { + printf -- '%s\n' "$@" | sed -e "s/[$ESCCHAR]\\(.\\)\\?/\\1/g" +} + +join() { + paste -s -d"$1" - +} + +tail() { + skip 1 "$@" +} + +skip() { + toskip="$1" + shift + + if [ "$toskip" -gt "$#" ]; then + toskip="$#" + fi + + while [ "$toskip" -gt '0' ] + do + shift + toskip="$((toskip - 1))" + done + + printf -- '%s\n' "$@" +} + +host_march() { + uname -m +} + +if [ "$#" -eq 0 ]; then + { + printf 'Usage:\n%s \n\n' "$0" + printf 'Format of target-name:\n' + printf '[+][+]\n' + printf 'or\n' + printf '[] [] \n\n' + printf 'Format of target-triple:\n' + printf '[-][-][-]\n' + printf 'or\n' + printf '[-[@distrib-version]][-][-]\n\n' + printf 'Special characters: %s,+,-,@\n' "$ESCCHAR" + printf 'Escape character: %s\n\n' "$ESCCHAR" + printf 'Examples:\n' + printf '%s myapp\n' "$0" + printf '%s x86_64+myapp\n' "$0" + printf '%s x86_64 myapp\n' "$0" + printf '%s x86_64-alpine@3.9-linux-gnueabihf+myapp\n' "$0" + printf '%s x86_64-alpine@3.9-linux-gnueabihf myapp\n' "$0" + printf '%s x86_64+arm32v7-alpine@3.9-linux-gnueabihf+myapp\n' "$0" + printf '%s x86_64 arm32v7-alpine@3.9-linux-gnueabihf myapp\n' "$0" + printf '%s x86_64+arm32v7-alpine@3.9-linux-gnueabihf+myapp+id\n' "$0" + printf '%s x86_64 arm32v7-alpine@3.9-linux-gnueabihf myapp id\n\n' "$0" + } >&2 + return 1 +fi + +ESCCHAR='~' +TARGET="$(echo "$@" | join '+')" + +# TODO: remove ugly hack +# necessary while posix shell can not handle arrays except the argument array +# maybe re-implement using a more suitable language! + +#shellcheck disable=SC2046 +eval set -- $(split '+' "$@" | sh_escape) + +if [ "$#" -gt 2 ]; then + HOST_ARCH="$1" + shift +fi + +if [ "$#" -eq 1 ]; then + set -- "$(host_march)" "$@" +fi + +ID_VARIANT="$(tail "$@" | join '+')" +TARGET_TRIPLE="$1" + +#shellcheck disable=SC2046 +eval set -- $(split '@' "$ID_VARIANT" | sh_escape) + +ID="$1" +if [ -z "$ID" ]; then + printf 'error: failed to parse "%s" aka "%s", unsupported target name\n' "$*" "$TARGET" >&2 + return 2 +fi + +VARIANT="$(tail "$@" | join '@')" +VARIANT="${VARIANT:-release}" + +#shellcheck disable=SC2046 +eval set -- $(split '-' "$TARGET_TRIPLE" | sh_escape) + +TARGET_ARCH_SUB="$1" +HOST_ARCH="${HOST_ARCH:-$TARGET_ARCH_SUB}" +VENDOR="${2:-unknown}" +TARGET_SYS="${3:-linux}" + +TARGET_ABI="$(skip 3 "$@" | join '-')" +TARGET_ABI="${TARGET_ABI:-gnu}" + +#shellcheck disable=SC2046 +eval set -- $(split '@' "$VENDOR" | sh_escape) + +VENDOR_OR_DISTRIB_ID="$1" + +DISTRIB_VERSION="$(tail "$@" | join '@')" +DISTRIB_VERSION="${DISTRIB_VERSION:-latest}" + +set -- \ + "$HOST_ARCH" \ + "$TARGET_ARCH_SUB" \ + "$VENDOR_OR_DISTRIB_ID" \ + "$TARGET_SYS" \ + "$TARGET_ABI" \ + "$ID" \ + "$DISTRIB_VERSION" \ + "$VARIANT" + +unescape "$@" | cat diff --git a/bin/shellcheck b/bin/shellcheck new file mode 100755 index 0000000..4c23a9a --- /dev/null +++ b/bin/shellcheck @@ -0,0 +1,44 @@ +#!/bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +set -e + +physical_pwd() { + pwd -P 2>/dev/null || pwd +} + +try_canonicalize() { + readlink -f "$@" 2>/dev/null || realpath "$@" +} + +canonicalize() { + if ! try_canonicalize "$1" 2>/dev/null; then + echo "$(cd "$(dirname "$1")" && physical_pwd)/$(basename "$1")" + fi +} + +scriptdir() { + dirname "$(canonicalize "${BASH_SOURCE:-$1}")" +} + +DOBUILDDIR="${DOBUILDDIR:-"$(dirname "$(scriptdir "$0")")"}" +PATH="$DOBUILDDIR/bin:$PATH" + +DOBUILD_SHELLCHECKVERSION="${DOBUILD_SHELLCHECKVERSION:-v0.6.0}" +DOBUILD_SHELLCHECKIMAGE="${REGISTRY_PREFIX}koalaman/shellcheck:${DOBUILD_SHELLCHECKVERSION}" + +export DOBUILDDIR +export DOBUILD_PROJECTDIR="${DOBUILD_SHELLCHECKPROJECTDIR:-"$PWD"}" +export DOBUILD_HOSTENVFILTER="${DOBUILD_SHELLCHECKHOSTENVFILTER:-^SHELLCHECK_}" + +set -- container_run --workdir "$(canonicalize "$DOBUILD_PROJECTDIR")" "$DOBUILD_SHELLCHECKIMAGE" "$@" + +exec "$@" diff --git a/bin/tar_cc_settings b/bin/tar_cc_settings new file mode 100755 index 0000000..5575819 --- /dev/null +++ b/bin/tar_cc_settings @@ -0,0 +1,290 @@ +#!/bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +set -e + +# FIXME: don't depend on cmake internals +discover_cmake_includes() { + if [ $# -gt 0 ]; then + sed -n -e 's/.*_INCLUDE_DIRS:INTERNAL=\(.*\)$/\1/p' "$@"; + fi +} + +discover_system_includes() { + case "$BUILDSYSTEM" in + cmake-?*) + discover_cmake_includes "$@" + ;; + *) + ;; + esac +} + +# FIXME: handle different compilers and language standards +discover_cc_includes() { + { + echo | "$CXX" -xc++ -E -Wp,-v - 2>&1 + echo | "$CC" -xc -E -Wp,-v - 2>&1 + } \ + | sed -n -e 's/\s\+\(\/.*$\).*/\1/p' +} + +discover_includes() { + { + discover_cc_includes; + discover_system_includes "$@" + } \ + | sed 's/;/\n/g' | sed '/^$/d' | sort -u 2>/dev/null \ + | while IFS= read -r file; do + if [ -e "$file" ]; then + canonicalize "$file" + fi + done +} + +detect_toolchain() { + case "$BUILDSYSTEM" in + cmake-?*) + cmake -LA -N "$(dirname "$1")" > "$CACHED_VALUES_TMPFILE" + CXX="$(sed -n -e 's/CMAKE_CXX_COMPILER:FILEPATH=\(.*\)$/\1/p' -e 's/CMAKE_CXX_COMPILER:STRING=\(.*\)$/\1/p' "$CACHED_VALUES_TMPFILE")" + CC="$(sed -n -e 's/CMAKE_C_COMPILER:FILEPATH=\(.*\)$/\1/p' -e 's/CMAKE_C_COMPILER:STRING=\(.*\)$/\1/p' "$CACHED_VALUES_TMPFILE")" + ;; + *) + ;; + esac +} + +parse_make_targets() { + parse_make_targets.sh -C "$WORKINGDIR" +} + +parse_ninja_targets() { + "$NINJA" -C "$WORKINGDIR" -t targets all | sed -n -e 's/\([^:]*\):.*/\1/p' | sort 2>/dev/null +} + +discover_targets() { + case "$BUILDSYSTEM" in + cmake-make|make) + parse_make_targets + ;; + cmake-ninja|ninja) + parse_ninja_targets + ;; + *) + ;; + esac +} + +prefix_dir() { + while IFS= read -r dir; do + printf '%s%s\n' "$1" "$dir" + done +} + +sed_keyword() { + printf '%s\n' "$@" | sed -e 's/[]\/$*.^[]/\\&/g' +} + +sed_replacement() { + printf '%s\n' "$@" | sed -e 's/[\/&]/\\&/g' +} + +format_json() { + i=0 + printf '[\n' + while IFS= read -r line; do + if [ "$i" -gt 0 ]; then + printf ',\n' + fi + printf '"%s"' "$line" + i="$((i+1))" + done + printf '\n]\n' +} + +physical_pwd() { + pwd -P 2>/dev/null || pwd +} + +try_canonicalize() { + readlink -f "$@" 2>/dev/null || realpath "$@" +} + +canonicalize() { + if ! try_canonicalize "$1" 2>/dev/null; then + echo "$(cd "$(dirname "$1")" && physical_pwd)/$(basename "$1")" + fi +} + +scriptdir() { + dirname "$(canonicalize "${BASH_SOURCE:-$1}")" +} + +cleanup() { + EXITCODE="${EXITCODE:-$?}" + rm -f "$INCLUDE_DIRS_TMPFILE" "$CACHED_VALUES_TMPFILE" + return "$EXITCODE" +} + +trap cleanup TERM INT EXIT + +export LANG=C +export LC_ALL=C + +DOBUILDDIR="${DOBUILDDIR:-"$(dirname "$(scriptdir "$0")")"}" +NINJA="${NINJA:-ninja}" +PATH="$DOBUILDDIR/bin:$PATH" + +set -- "$@" -- + +CXX= +CC= +BUILDSYSTEM= +WORKINGDIR="$PWD" +PROJECTDIR= +INCLUDE_PREFIX="$WORKINGDIR" +COMPILE_COMMANDS_JSON_FILE= + +while :; do + case $1 in + -C|--directory) + if [ "$2" != '--' ]; then + WORKINGDIR="$2" + shift + else + printf 'error: "%s" requires a non-empty option argument.\n' "$1" >&2 + exit 3 + fi + ;; + --directory=) + printf 'error: "%s" requires a non-empty option argument.\n' "$1" >&2 + exit 3 + ;; + --directory=?*) + WORKINGDIR="${1#*=}" + ;; + --project-root) + if [ "$2" != '--' ]; then + PROJECTDIR="$2" + shift + else + PROJECTDIR= + fi + ;; + --project-root=) + PROJECTDIR= + ;; + --project-root=?*) + PROJECTDIR="${1#*=}" + ;; + --build-system) + if [ "$2" != '--' ]; then + BUILDSYSTEM="$2" + shift + else + BUILDSYSTEM= + fi + ;; + --build-system=) + BUILDSYSTEM= + ;; + --build-system=?*) + BUILDSYSTEM="${1#*=}" + ;; + -p|--include-prefix) + if [ "$2" != '--' ]; then + INCLUDE_PREFIX="$2" + shift + else + INCLUDE_PREFIX= + fi + ;; + --include-prefix=) + INCLUDE_PREFIX= + ;; + --include-prefix=?*) + INCLUDE_PREFIX="${1#*=}" + ;; + compile_commands.json|?*/compile_commands.json) + COMPILE_COMMANDS_JSON_FILE="$1" + ;; + --help) + { + printf 'Usage: %s [option...] [file...]\n' "$(basename "$0")" + printf 'Options:\n' + printf '\t-C|--directory\t\tWorking directory\n' + printf '\t--build-system\t\tBuildsystem kind e.g. cmake-make,cmake-ninja\n' + printf '\t--project-root\t\tProject root directory\n' + printf '\t-p|--include-prefix\tInclude prefix appended to all discovered include dirs\n' + printf 'Files:\n' + printf '\tBuildsystem specific configurations files e.g. CMakeCache.txt\n' + } >&2 + exit 0 + ;; + --) + shift + break + ;; + *) + set -- "$@" "$1" + ;; + esac + + shift +done + +OUTDIR="$WORKINGDIR/DoBuildFiles" +mkdir -p "$OUTDIR" + +INCLUDE_PREFIX_REPLACEMENT="%OUTDIR%/$INCLUDE_PREFIX" + +CACHED_VALUES_TMPFILE="$(mktemp -p "$OUTDIR" cached_values_XXXXXXXXXX.txt)" +INCLUDE_DIRS_TMPFILE="$(mktemp -p "$OUTDIR" include_dirs_XXXXXXXXXX.txt)" +INCLUDE_DIRS_TMPL_FILE="$OUTDIR/include_dirs.json.template" +TARGETS_JSON_FILE="$OUTDIR/targets.json" +TARGETS_TXT_FILE="$OUTDIR/targets.txt" +COMPILE_COMMANDS_TMPL_FILE="$OUTDIR/compile_commands.json.template" +C_BUILTIN_FILE="$OUTDIR/builtins.h" +CXX_BUILTIN_FILE="$OUTDIR/builtins.hpp" + +detect_toolchain "$@" + +discover_targets "$@" | tee "$TARGETS_TXT_FILE" | format_json > "$TARGETS_JSON_FILE" +discover_includes "$@" | tee "$INCLUDE_DIRS_TMPFILE" | prefix_dir "$INCLUDE_PREFIX_REPLACEMENT" | format_json > "$INCLUDE_DIRS_TMPL_FILE" + +# FIXME: handle different compilers and language standards +"$CC" -xc -dM -E - < /dev/null > "$C_BUILTIN_FILE" +"$CXX" -xc++ -dM -E - < /dev/null > "$CXX_BUILTIN_FILE" + +if [ -n "$COMPILE_COMMANDS_JSON_FILE" ]; then + set -- + while IFS= read -r line; do + set -- "$@" -e "s/$(sed_keyword "${line}")/$(sed_replacement "${INCLUDE_PREFIX_REPLACEMENT}${line}")/g" + done < "$INCLUDE_DIRS_TMPFILE" + set -- "$@" -e "s/$(sed_keyword "$WORKINGDIR")/$(sed_replacement "%OUTDIR%")/g" + if [ -n "$PROJECTDIR" ]; then + set -- "$@" -e "s/$(sed_keyword "$PROJECTDIR")/$(sed_replacement "%PROJECTDIR%")/g" + fi + set -- "$@" "$COMPILE_COMMANDS_JSON_FILE" + sed "$@" > "$COMPILE_COMMANDS_TMPL_FILE" +fi + +set -- + +if [ -n "$INCLUDE_PREFIX" ]; then + while IFS= read -r file; do + set -- "$@" "$file" + done < "$INCLUDE_DIRS_TMPFILE" +fi + +cleanup + +exec tar cf - -C / "$@" diff --git a/cmake.mk b/cmake.mk new file mode 100644 index 0000000..a451ce3 --- /dev/null +++ b/cmake.mk @@ -0,0 +1,213 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +ifndef cmake_include_guard +cmake_include_guard := 1 + +current_makefile := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)) + +ifndef defaults_include_guard + include $(patsubst %/,%,$(dir $(current_makefile)))/defaults.mk +endif + +####################################################################################################################### +# Overridable CMake defaults + +DOBUILD_CMAKE_GENERATOR ?= make + +####################################################################################################################### +# Overridable cmake macros, hooks to customize target default values + +# hook called to retrieve the target cmake extension directory, +# may return an empty value +# $(call dobuild_cmake_extdir,target-name) +dobuild_cmake_extdir ?= $(call dobuild_generic_extdir,$1) + +# hook called to retrieve the target cmake adapter name +# $(call dobuild_cmake_adapter,target-name) +dobuild_cmake_adapter ?= cmake + +# hook called to retrieve the target cmake prerequisites, +# may return an empty list +# $(call dobuild_cmake_prerequisites,target-name) +dobuild_cmake_prerequisites ?= $(call dobuild_generic_prerequisites,$1) + +# hook called to retrieve the target cmake order-only prerequisites, +# may return an empty list +# $(call dobuild_cmake_orderonly_prerequisites,target-name) +dobuild_cmake_orderonly_prerequisites ?= $(call dobuild_generic_orderonly_prerequisites,$1) + +# hook called to retrieve the target cmake prepare step, +# may return an empty value +# $(call dobuild_cmake_prepare,target-name) +dobuild_cmake_prepare ?= $(call dobuild_generic_prepare,$1) + +# hook called to retrieve the target cmake assemble step +# $(call dobuild_cmake_assemble,target-name) +dobuild_cmake_assemble ?= $(call dobuild_generic_assemble,$1) + +# hook called to retrieve the target cmake save artifacts step, +# may return an empty value +# $(call dobuild_cmake_saveartifacts,target-name) +dobuild_cmake_saveartifacts ?= $(call dobuild_generic_saveartifacts,$1) + +# hook called to retrieve the target cmake lint step, +# may return an empty value +# $(call dobuild_cmake_lint,target-name) +dobuild_cmake_lint ?= $(call dobuild_generic_lint,$1) + +# hook called to retrieve the target cmake check step, +# may return an empty value +# $(call dobuild_cmake_check,target-name) +dobuild_cmake_check ?= $(call dobuild_generic_check,$1) + +# hook called to retrieve the target cmake memcheck step, +# may return an empty value +# $(call dobuild_cmake_memcheck,target-name) +dobuild_cmake_memcheck ?= $(call dobuild_generic_memcheck,$1) + +# hook called to retrieve the target cmake package step, +# may return an empty value +# $(call dobuild_cmake_package,target-name) +dobuild_cmake_package ?= $(call dobuild_generic_package,$1) + +# hook called to retrieve the target cmake install step, +# may return an empty value +# $(call dobuild_cmake_install,target-name) +dobuild_cmake_install ?= $(call dobuild_generic_install,$1) + +# hook called to retrieve the target cmake delegate step +# $(call dobuild_cmake_delegate,target-name) +dobuild_cmake_delegate ?= $(call dobuild_generic_delegate,$1) + +# hook called to retrieve the target cmake build testing option, +# may return an empty value in which case BUILD_TESTING is used +# $(call dobuild_cmake_buildtesting,target-name) +dobuild_cmake_buildtesting ?= $(call dobuild_generic_buildtesting,$1) + +# hook called to retrieve the target cmake generator name, +# may return an empty value in which case CMAKE_GENERATOR is used +# $(call dobuild_cmake_generator,target-name) +dobuild_cmake_generator ?= + +####################################################################################################################### +# CMake macros + +# retrieves the target cmake default prerequisites +# $(call cmake_default_prerequisites,target-name) +cmake_default_prerequisites = \ + $(OUTDIR)/$1/DoBuildFiles/cmake.properties \ + $(wildcard $(PROJECTDIR)/CMakeLists.txt) + +# retrieves the target cmake generator +# $(call cmake_generator,target-name) +cmake_generator = $(cmake_generator_$(cache.$1.cmake_generator)) + +# retrieves the target cmake generator command +# $(call cmake_generator_cmd,target-name) +cmake_generator_cmd = $(cache.$1.cmake_generator) + +# creates a cmake properties generation cmd +# $(call cmake_props_cmd,target-name,input-file) +cmake_props_cmd = sed \ + -e 's!%CMAKE_GENERATOR_CMD%!$(call escape,$(call cmake_generator_cmd,$1),!)!g' \ + -e 's!%CMAKE_GENERATOR%!$(call escape,$(call cmake_generator,$1),!)!g' \ + $2 + +# creates a cmake properties generation rule +# $(call cmake_props_rule,target-name) +cmake_props_rule = \ + $$(OUTDIR)/$1/DoBuildFiles/cmake.properties: $$(DOBUILDDIR)/assets/templates/cmake.properties.template $$(MAKEFILE_LIST); \ + $$(SILENT)mkdir -p $$(dir $$@); $$(call cmake_props_cmd,$1,$$<) > $$@ + +####################################################################################################################### +# CMake rule target configuration + +CMAKE_GENERATOR = $(call memorize,CMAKE_GENERATOR,$(DOBUILD_CMAKE_GENERATOR)) + +docker_runflags += --env NINJA_STATUS + +cmake_generator_ninja = Ninja +cmake_generator_make = Unix Makefiles + +cmake_rule_targets = $(addsuffix /cmakerules.mk,$(OUTDIR)) + +GENERIC_TARGETS += $(CMAKE_TARGETS) +RULE_TARGETS += $(cmake_rule_targets) + +####################################################################################################################### +# Makefile dependencies + +MAKEFILE_DEPS += touch +MAKEFILE_DEPS += echo +MAKEFILE_DEPS += sed + +####################################################################################################################### +# CMake rules + +$(cmake_rule_targets): + $(SILENT) \ + { \ + echo '$(\#) generated file - do not edit!!!'; \ + echo; \ + ID='$(call id,$@)'; \ + echo "ifndef $${ID}_include_guard"; \ + echo "$${ID}_include_guard := 1"; \ + echo; \ + $(foreach target,$(CMAKE_TARGETS),\ + echo '$(\#)$(\#) BEGIN of cmake $(target) configuration'; \ + echo; \ + echo '$(\#)$(\#)$(\#) defaults'; \ + echo '$(target) ?= $$(call memorize,$(target),$(call target_properties_parse,$(target)))'; \ + echo '$(target).generic_adapter = $$(call $$1.cmake_adapter,$$1)'; \ + echo '$(target).generic_extdir = $$(call $$1.cmake_extdir,$$1)'; \ + echo '$(target).generic_prerequisites = $$(call $$1.cmake_prerequisites,$$1) $$(call cmake_default_prerequisites,$$1)'; \ + echo '$(target).generic_orderonly_prerequisites ?= $$(call $$1.cmake_orderonly_prerequisites,$$1)'; \ + echo '$(target).generic_prepare = $$(call $$1.cmake_prepare,$$1)'; \ + echo '$(target).generic_assemble = $$(call $$1.cmake_assemble,$$1)'; \ + echo '$(target).generic_saveartifacts = $$(call $$1.cmake_saveartifacts,$$1)'; \ + echo '$(target).generic_lint = $$(call $$1.cmake_lint,$$1)'; \ + echo '$(target).generic_check = $$(call $$1.cmake_check,$$1)'; \ + echo '$(target).generic_memcheck = $$(call $$1.cmake_memcheck,$$1)'; \ + echo '$(target).generic_package = $$(call $$1.cmake_package,$$1)'; \ + echo '$(target).generic_install = $$(call $$1.cmake_install,$$1)'; \ + echo '$(target).generic_delegate = $$(call $$1.cmake_delegate,$$1)'; \ + echo '$(target).generic_buildtesting = $$(call $$1.cmake_buildtesting,$$1)'; \ + echo '$(target).cmake_adapter ?= $$(call dobuild_cmake_adapter,$$1)'; \ + echo '$(target).cmake_extdir ?= $$(call dobuild_cmake_extdir,$$1)'; \ + echo '$(target).cmake_prerequisites ?= $$(call dobuild_cmake_prerequisites,$$1)'; \ + echo '$(target).cmake_orderonly_prerequisites ?= $$(call dobuild_cmake_orderonly_prerequisites,$$1)'; \ + echo '$(target).cmake_prepare ?= $$(call dobuild_cmake_prepare,$$1)'; \ + echo '$(target).cmake_assemble ?= $$(call dobuild_cmake_assemble,$$1)'; \ + echo '$(target).cmake_saveartifacts ?= $$(call dobuild_cmake_saveartifacts,$$1)'; \ + echo '$(target).cmake_lint ?= $$(call dobuild_cmake_lint,$$1)'; \ + echo '$(target).cmake_check ?= $$(call dobuild_cmake_check,$$1)'; \ + echo '$(target).cmake_memcheck ?= $$(call dobuild_cmake_memcheck,$$1)'; \ + echo '$(target).cmake_package ?= $$(call dobuild_cmake_package,$$1)'; \ + echo '$(target).cmake_install ?= $$(call dobuild_cmake_install,$$1)'; \ + echo '$(target).cmake_delegate ?= $$(call dobuild_cmake_delegate,$$1)'; \ + echo '$(target).cmake_buildtesting ?= $$(call dobuild_cmake_buildtesting,$$1)'; \ + echo '$(target).cmake_generator ?= $$(call dobuild_cmake_generator,$$1)'; \ + echo; \ + echo '$(\#)$(\#)$(\#) cached values'; \ + echo 'cache.$(target).cmake_generator = $$(call memorize,cache.$(target).cmake_generator,$$(or $$(call $(target).cmake_generator,$(target)),$$(CMAKE_GENERATOR)))'; \ + echo; \ + echo '$(\#)$(\#)$(\#) rules'; \ + echo '$(call cmake_props_rule,$(target))'; \ + echo; \ + echo '$(\#)$(\#) END of cmake $(target) configuration'; \ + echo; \ + ) \ + echo 'endif'; \ + echo; \ + } > $@ + +endif + diff --git a/defaults.mk b/defaults.mk new file mode 100644 index 0000000..37447c7 --- /dev/null +++ b/defaults.mk @@ -0,0 +1,681 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +ifndef defaults_include_guard +defaults_include_guard := 1 + +current_makefile := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)) + +####################################################################################################################### +# Overridable common defaults + +# NOTE: default assumes first loaded makefile is located in root directory of project +MAKEFILE ?= $(firstword $(MAKEFILE_LIST)) +MAKEFILE := $(MAKEFILE) + +DOBUILD_TOPDIR ?= $(PROJECTDIR) +DOBUILD_PROJECTDIR ?= $(patsubst %/,%,$(dir $(MAKEFILE))) +DOBUILD_PROJECTNAME ?= $(notdir $(shell cd $(PROJECTDIR) && pwd)) +DOBUILD_PROJECTVERSIONFILE ?= $(PROJECTDIR)/VERSION +DOBUILD_PROJECTVERSION ?= $(shell cat '$(VERSIONFILE)' 2>/dev/null) +DOBUILD_OUTDIR ?= $(BUILDDIR)/.build +DOBUILD_BUILDDIR ?= $(PROJECTDIR) +DOBUILD_BUILDVERBOSE ?= +DOBUILD_TESTVERBOSE ?= +DOBUILD_BUILDTESTING ?= 1 +DOBUILD_USERID ?= $(shell id -u) +DOBUILD_HOSTCONTAINER ?= $(shell $(DOBUILDDIR)/bin/get_container_id.sh) +DOBUILD_JOBSLOTS ?= $(call jobslots,$(make_cmdline)) +DOBUILD_SKIPMD5SUM ?= $(call filter_not_found,md5sum) +DOBUILD_SKIPCURL ?= $(call filter_not_found,curl) +DOBUILD_SKIPDEFAULTTARGET ?= $(or $(HOSTMARCH),$(MARCH),$(DISTRIB_ID),$(DISTRIB_VERSION),$(SYS),$(ABI),$(ID),$(VARIANT),$(call not,$(DEFAULTTARGET))) +DOBUILD_SKIPEXTERNSYNC ?= +DOBUILD_FILTER ?= $(call target_properties_combine, \ + $(or $(HOSTMARCH),%), \ + $(or $(MARCH),%), \ + $(or $(DISTRIB_ID),%), \ + $(or $(SYS),%), \ + $(or $(ABI),%), \ + $(or $(ID),%), \ + $(or $(DISTRIB_VERSION),%), \ + $(or $(VARIANT),%) \ + ) +DOBUILD_EXCLUDEFILTER ?= $(if $(strip \ + $(or $(EXCLUDE_HOSTMARCH),$(EXCLUDE_MARCH),$(EXCLUDE_DISTRIB_ID),$(EXCLUDE_SYS),$(EXCLUDE_ABI),$(EXCLUDE_ID),$(EXCLUDE_DISTRIB_VERSION),$(EXCLUDE_VARIANT)) \ + ), \ + $(call target_properties_combine, \ + $(or $(EXCLUDE_HOSTMARCH),%), \ + $(or $(EXCLUDE_MARCH),%), \ + $(or $(EXCLUDE_DISTRIB_ID),%), \ + $(or $(EXCLUDE_SYS),%), \ + $(or $(EXCLUDE_ABI),%), \ + $(or $(EXCLUDE_ID),%), \ + $(or $(EXCLUDE_DISTRIB_VERSION),%), \ + $(or $(EXCLUDE_VARIANT),%)\ + ) \ + ) +DOBUILD_FILTERMEMCHECK ?= $(call target_properties_format_args,$(or $(machine),%),$(or $(machine),%),%,%,%,%,%,%) +DOBUILD_CGROUPPARENT ?= +DOBUILD_COLOR ?= $(shell { command -v tput >/dev/null 2>&1 && test "$$(tput $(addprefix -T,$(TERM)) colors)" -ge 8 2>/dev/null; } && echo '1') +DOBUILD_EXTDIR ?= $(PROJECTDIR)/dobuild-extensions + +BUILDKIT_PROGRESS ?= plain +DOCKER_BUILDKIT ?= 1 +DESTDIR ?= +TERM ?= linux +REGISTRY_PREFIX ?= +VERBOSE ?= + +COLOR = $(call memorize,COLOR,$(DOBUILD_COLOR)) + +PREPARE_TARGETS = $(EXTRACT_TARGETS) +RULE_TARGETS = +BUILD_TARGETS = +CHECK_TARGETS = +MEMCHECK_TARGETS = +LINT_TARGETS = +RUN_TARGETS = +CLEAN_TARGETS = +EXTRACT_TARGETS = +DIST_TARGETS = +DISTCLEAN_TARGETS = +INSTALL_TARGETS = +TARGETS = +OUTDIRS = +BUILDTARGET = $(warning: deprecated use DEFAULTTARGET instead) + +DOCKER_RUNFLAGS = +DOCKER_BUILDFLAGS = +IMAGE_BUILDARGS = + +CURLFLAGS = + +MAKEFILE_DEPS = + +####################################################################################################################### +# Common macros + +# caches a value lazily - at it's first evaluation +# $(call memorize,VAR,value...) +memorize = $(eval $1 := $$2)$2 + +# asserts value is not empty, evaluates to value or reports an error message on failure and aborts build +# $(call assert,value,message) +assert = $(or $1,$(error assertion failed: $2)) + +# asserts value is not empty, evaluates to value or reports a warning message on failure and continues build +# $(call expect,value,message) +expect = $(or $1,$(warning expectation failed: $2)) + +# asserts value is of scalar type, evaluates to value or reports an error message on failure +# $(call assert_scalar,value...,message) +assert_scalar = $(if $(call eq_s,1,$(words $1)),$1,$(error assertion failed: $2)) + +# filters out all commands found on host, resulting in a list of unavailable commands +# $(call filter_not_found,command...) +filter_not_found = $(filter $1,$(foreach cmd,$1,$(shell command -v $(cmd) >/dev/null 2>&1 || echo $(cmd)))) + +# escapes a character in string +# $(call escape,string,char) +escape = $(subst $2,$(\\)$2,$1) + +# filters all elements matching any of the given regular expressions +# $(call filter_by_regex,regex...,element...) +filter_by_regex = $(shell printf '%s\n' $2 | sed -n$(foreach regex,$1, -e 's!$(call escape,$(regex),!)!\0!p')) + +# parses the target properties [host-arch arch_sub vendor_or_distrib_id sys abi id distrib_version variant] for given name +# $(call target_properties_parse,target-name) +target_properties_parse = $(shell $(DOBUILDDIR)/bin/parse_target_properties.sh $1) + +# retrieves the target properties +# $(call target_properties,target-name) +target_properties = $(or $($1),$(call target_properties_parse,$1)) + +# formats a list of target properties as string +# $(call target_properties_format,property...) +target_properties_format = $(call target_properties_format_args, \ + $(call target_property_host_arch,$1), \ + $(call target_property_arch_sub,$1), \ + $(call target_property_vendor_or_distrib_id,$1), \ + $(call target_property_sys,$1), \ + $(call target_property_abi,$1), \ + $(call target_property_id,$1), \ + $(call target_property_distrib_version,$1), \ + $(call target_property_variant,$1) \ + ) + +# formats target name by its properties as separated arguments - each may be empty +# $(call target_properties_format_args,host_arch,arch_sub,distrib,sys,abi,id,distrib_version,variant) +target_properties_format_args = $(call join_s,$1 $(call join_s,$2 $(call join_s,$3 $7,@) $4 $5,-) $(call join_s,$6 $8,@),+) + +# tests each target property matches associated pattern +# $(call target_properties_matches,pattern...,property...) +target_properties_matches = $(call eq,$1,$2,filter) + +# retrieves the target property host-arch from the list of properties +# $(call target_property_host_arch,property...) +target_property_host_arch = $(word 1,$1) + +# retrieves the target property arch_sub from the list of properties +# $(call target_property_arch_sub,property...) +target_property_arch_sub = $(word 2,$1) + +# retrieves the target property vendor_or_distrib_id from the list of properties +# $(call target_property_vendor_or_distrib_id,property...) +target_property_vendor_or_distrib_id = $(word 3,$1) + +# retrieves the target property sys from the list of properties +# $(call target_property_sys,property...) +target_property_sys = $(word 4,$1) + +# retrieves the target property abi from the list of properties +# $(call target_property_abi,property...) +target_property_abi = $(word 5,$1) + +# retrieves the target property id from the list of properties +# $(call target_property_id,property...) +target_property_id = $(word 6,$1) + +# retrieves the target property distrib_version from the list of properties +# $(call target_property_distrib_version,property...) +target_property_distrib_version = $(word 7,$1) + +# retrieves the target property variant from the list of properties +# $(call target_property_variant,property...) +target_property_variant = $(word 8,$1) + +# retrieves the target host-arch +# $(call target_host_arch,target-name) +target_host_arch = $(call target_property_host_arch,$(call target_properties,$1)) + +# retrieves the target arch_sub +# $(call target_arch_sub,target-name) +target_arch_sub = $(call target_property_arch_sub,$(call target_properties,$1)) + +# retrieves the target vendor_or_distrib_id +# $(call target_vendor_or_distrib_id,target-name) +target_vendor_or_distrib_id = $(call target_property_vendor_or_distrib_id,$(call target_properties,$1)) + +# retrieves the target sys +# $(call target_sys,target-name) +target_sys = $(call target_property_sys,$(call target_properties,$1)) + +# retrieves the target abi +# $(call target_abi,target-name) +target_abi = $(call target_property_abi,$(call target_properties,$1)) + +# retrieves the target id +# $(call target_id,target-name) +target_id = $(call target_property_id,$(call target_properties,$1)) + +# retrieves the target id +# $(call target_variant,target-name) +target_variant = $(call target_property_variant,$(call target_properties,$1)) + +# retrieves the target distribution version +# $(call target_distrib_version,target-name) +target_distrib_version = $(call target_property_distrib_version,$(call target_properties,$1)) + +# retrieves the target distribution id and version +# $(call target_distrib,target-name) +target_distrib = $(addsuffix $(addprefix @,$(call target_distrib_version,$1)),$(call target_vendor_or_distrib_id,$1)) + +# retrieves the target distribution id +# $(call target_distrib_id,target-name) +target_distrib_id = $(call target_vendor_or_distrib_id,$1) + +# tests target properties matches given pattern +# $(call target_matches,target-name,pattern...) +target_matches = $(call target_properties_matches,$2,$(call target_properties,$1)) + +# filters all targets which properties matches given patterns +# $(call target_filter_by_properties,pattern...,target-name...) +target_filter_by_properties = $(call filter_p,$2,target_matches,$1) + +# filters all targets which properties matches given include-pattern and not the given exclude-pattern strings +# $(call target_filter,inlcude-pattern-string...,target-name...[,exclude-pattern-string...]) +target_filter = \ + $(filter-out \ + $(foreach pattern,$3,$(call target_filter_by_properties,$(call target_properties_parse,$(pattern)),$2)),\ + $(foreach pattern,$1,$(call target_filter_by_properties,$(call target_properties_parse,$(pattern)),$2))\ + ) + +# replaces template values in string list with target properties +# $(call target_subst,target-name[,values...,]) +target_subst = $(strip \ + $(subst %TARGET%,$1, \ + $(subst %OUTDIR%,$(OUTDIR), \ + $(subst %HOSTMARCH%,$(call target_host_arch,$1), \ + $(subst %MARCH%,$(call target_arch_sub,$1), \ + $(subst %DISTRIB%,$(call target_distrib,$1), \ + $(subst %DISTRIB_ID%,$(call target_distrib_id,$1), \ + $(subst %DISTRIB_VERSION%,$(call target_distrib_version,$1), \ + $(subst %SYS%,$(call target_sys,$1), \ + $(subst %ABI%,$(call target_abi,$1), \ + $(subst %ID%,$(call target_id,$1), \ + $(subst %VARIANT%,$(call target_variant,$1),$2) \ + ) \ + ) \ + ) \ + ) \ + ) \ + ) \ + ) \ + ) \ + ) \ + ) \ + ) + +# replaces template values of target field with target properties +# $(call target_subst_cmd,target-name,field[,default-value]) +target_get_and_subst = $(call target_subst,$1,$(or $(call $1.$2,$1),$3)) + +# replaces template values of target field with target properties and resolves result against given search +# paths, returning a list of existing resolved paths +# $(call target_subst_and_resolve,target-name,field,path...) +target_subst_and_resolve = $(wildcard $(addsuffix /$(call target_get_and_subst,$1,$2,),$3)) + +# generates target names by combining elements of list properties +# $(call target_properties_combine,host_arch1...,arch_sub1..,distrib1...,sys1...,abi1...,id1...,distrib_version1...,variant1...) +target_properties_combine = $(call combine,target_properties_format_args,$1,$2,$3,$4,$5,$6,$7,$8) + +# tests scalar values on equality, returns the value or empty if non-equal +# $(call eq_s,value1,value2) +eq_s = $(and $(findstring $1,$2),$(findstring $2,$1)) + +# tests scalar values on inequality, returns non empty value if values are unequal +# $(call ne_s,value1,value2) +ne_s = $(call not,$(call eq_s,$1,$2)) + +# negates value, returns empty for non-empty value and vice versa +# $(call not,value) +not = $(if $(strip $1),,1) + +# compares number values, returns negative value if num1 is less than num2, +# a positive value if num1 is greather than 2 and 0 if both are equal +# $(call compare,num1,num2) +compare = $(call bc,$1-$2) + +# tests if num1 is less than num2 +# $(call lt,num1,num2) +lt = $(if $(findstring -,$(call compare,$1,$2)),1) + +# tests if num1 is greater than num2 +# $(call gt,num1,num2) +gt = $(call not,$(or $(call lt,$1,$2),$(call eq_s,$1,$2))) + +# tests if num1 is greater than or equal to num2 +# $(call ge,num1,num2) +ge = $(call not,$(call lt,$1,$2)) + +# tests if version parts (major, minor, patch) are greater or equal to required version parts +# $(call ge_version,[major minor patch],[required-major required-minor required-patch]) +ge_version = $(strip \ + $(or \ + $(call gt,$(or $(firstword $1),0),$(or $(firstword $2),0)), \ + $(if $(call eq_s,$(or $(firstword $1),0),$(or $(firstword $2),0)), \ + $(or \ + $(call gt,$(or $(word 2,$1),0),$(or $(word 2,$2),0)) \ + $(if $(call eq_s,$(or $(word 2,$1),0),$(or $(word 2,$2),0)), \ + $(call ge,$(or $(word 3,$1),0),$(or $(word 3,$2),0)) \ + ) \ + ) \ + ) \ + ) \ + ) + +# sorts numbers +# $(call sort_n,value...) +sort_n = $(shell LANG=C LC_ALL=C printf '%s\n' $1 | sort -n) + +# caluclates minimum of numeric values +# $(call min,values...) +min = $(firstword $(call sort_n,$1)) + +# caluclates maximum of numeric values +# $(call max,values...) +max = $(call lastword,$(call sort_n,$1)) + +# caluclates the value of an arithmetic expression +# $(call bc,expr...) +bc = $(shell echo "$$(($1))") + +# returns the tail of a list, removing its first element +# $(call tail,element...) +tail = $(wordlist 2,$(words $1),$1) + +# returns the last element of a list (implemented for backward compatiblity with gnu make 3.8) +# $(call lastword,element...) +ifneq ($(lastword 0 1),1) + lastword = $(if $1,$(word $(words $1),$1),) +endif + +# removes whitespace characters between positional argument and its value +# $(call normalize_args,arg...,cmdline) +normalize_args = $(if $1,$(call normalize_args,$(call tail,$1),$(subst $(firstword $1)$( ),$(firstword $1),$2)),$2) + +# parses the number of available job slots (-jx | --jobs X) +# $(call jobslots,mflags...) +jobslots = $(call lastword,$(shell printf '%s\n' $(call normalize_args,-j --jobs,$1) | sed -n -e 's@^\(--jobs=\?\|-j\)\([0-9]\+\)@\2@p')) + +# creates a colored message +# $(call color,color-name,value...) +color = '$(tc_$1)'$2'$(tc_reset)' +$(COLOR)color = $2 + +# creates a command that prints a message when VERBOSE is disabled +# $(call echo_if_silent_cmd,value...) +echo_if_silent_cmd = VERBOSE=1 +$(VERBOSE)echo_if_silent_cmd = { printf '$(DEFAULT_FMT) ' $1 && printf '\n'; } + +# creates a command which is printed before it is executed +# $(call echo_cmd,command[,args...][,output-command]) +echo_cmd = $(call echo_if_silent_cmd,$(or $3,$1) $2) && $1 $2 + +# creates a command which changes its working directory before it is executed +# $(call chdir_cmd,target-name,workdir,command[,args...][,output-command]) +chdir_cmd = \ + $(call echo_if_silent_cmd,$(call color,green,$(addprefix TARGET=,$1) WORKDIR='$2') $(or $5,$3) $4) \ + && (cd '$2' && PATH='$(cache.$1.env_path)' $3 $4) + +# overridable helper to create a command which is executed as part of a target rule +# $(call run_cmd,target-name,command[,args...][,output-command]) +run_cmd = $(call chdir_cmd,$1,$(OUTDIR)/$1,$2,$3,$4) + +# retrieves the host path environment variable value with optional target specific extensions +# $(call env_host_path[,target-name]) +env_host_path = $(call join_s,\ + $(realpath \ + $(cache.$1.extdir) \ + $(addprefix $(DOBUILDDIR)/assets/adapters/,$(cache.$1.adapter)) \ + ) \ + $(PATH),\ + : \ + ) + +# overridable helper to retrieve the target default path environment variable value +# $(call env_default_path,target-name) +env_default_path = $(call env_host_path,$1) + +# retrieves the last modification time of each files in seconds since epoch +# $(call mtime,file...) +mtime = $(if $1,$(shell stat -c '%Y' $1)) + +# joins a list to a string +# $(call join_s,element...[,separator]) +join_s = $(subst $( ),$(strip $2),$(strip $1)) + +# splits a string to a list +# $(call split_s,element...[,separator]) +split_s = $(subst $(strip $2),$( ),$(strip $1)) + +# identity function +# $(call identity,element...) +identity = $1 + +# filters all elements matching given predicate +# $(call filter_p,element...,predicate[,predicate-context]) +filter_p = $(foreach elm,$1,$(if $(strip $(call $2,$(elm),$3)),$(elm),)) + +# tests inequality of list 1 and list 2 +# $(call ne,elem1...,elem2...[,predicate]) +ne = $(call not,$(call eq,$1,$2,$3)) + +# tests equality of list 1 and list 2 +# $(call eq,elem1...,elem2...[,predicate]) +eq = $(strip \ + $(if \ + $(call eq_s,$(words $1),$(words $2)), \ + $(call eq_each,$1,$2,$(or $3,eq_s)) \ + ) \ + ) + +# tests equality of elements in list 1 and list 2 +# $(call eq_each,list1_elem...,list2_elem...,predicate) +eq_each = $(strip \ + $(if $(strip $1),\ + $(if \ + $(call $3,$(firstword $1),$(firstword $2)), \ + $(call eq_each,$(call tail,$1),$(call tail,$2),$3), \ + ), \ + 1 \ + ) \ + ) + +# encodes string to a valid make identifier +# $(call id,string...) +id = $(subst $( ),~,$(subst $(=),_,$(call base64_encode,$1))) + +# encodes string to base64 +# $(call base64_encode,string...) +base64_encode = $(shell echo $1 | base64) + +# decodes string from base64 +# $(call base64_decode,string...) +base64_decode = $(shell echo $1 | base64 -d) + +# replaces all elements which equals oldvalue with newvalue +# $(call replace_all,element...,oldvalue,newvalue) +replace_all = $(foreach elm,$1,$(if $(call eq_s,$(elm),$2),$3,$(elm))) + +# internal helper to generate the cartesian product of list1 to list8 using mapping function fn +# $(call cartesian_product_helper,fn,emptylist_placeholder,elem1...,elem2..,elem3...,elem4...,elem5...,elem6...,elem7...,elem8...) +cartesian_product_helper = $(strip \ + $(foreach f1,$3,\ + $(foreach f2,$4,\ + $(foreach f3,$5,\ + $(foreach f4,$6,\ + $(foreach f5,$7,\ + $(foreach f6,$8,\ + $(foreach f7,$9,\ + $(foreach f8,$(10),\ + $(call $1,\ + $(call replace_all,$(f1),$2,),\ + $(call replace_all,$(f2),$2,),\ + $(call replace_all,$(f3),$2,),\ + $(call replace_all,$(f4),$2,),\ + $(call replace_all,$(f5),$2,),\ + $(call replace_all,$(f6),$2,),\ + $(call replace_all,$(f7),$2,),\ + $(call replace_all,$(f8),$2,)\ + ) \ + ) \ + ) \ + ) \ + ) \ + ) \ + ) \ + ) \ + ) \ + ) + +# generates the cartesian product of list1 to list8 using mapping function fn, mapping empty list to place holder value +# $(call cartesian_product,fn,emptylist_placeholder,elem1...,elem2..,elem3...,elem4...,elem5...,elem6...,elem7...,elem8...) +cartesian_product = $(call cartesian_product_helper,$1,$2,\ + $(or $(strip $3),$2),\ + $(or $(strip $4),$2),\ + $(or $(strip $5),$2),\ + $(or $(strip $6),$2),\ + $(or $(strip $7),$2),\ + $(or $(strip $8),$2),\ + $(or $(strip $9),$2),\ + $(or $(strip $(10)),$2)\ + ) + +# combines elements of list1 to list8 using mapping function fn +# $(call combine,fn,elem1...,elem2..,elem3...,elem4...,elem5...,elem6...,elem7...,elem8...) +combine = $(call cartesian_product,$1,~EmptyList~,$2,$3,$4,$5,$6,$7,$8,$9) + +# retrieves the version parts (major, minor) of a make version string +# $(call make_version_parts,version-string) +make_version_parts = $(shell echo '$1' | sed -ne 's![0]*\([^.]\+\).[0]*\([^.]\+\).*!\1 \2!p') + +# tests if make version is greater or equal to required version +# $(call make_version_ge,required-version-string) +make_version_ge = \ + $(call ge_version, \ + $(call make_version_parts,$(make_version)),\ + $(call make_version_parts,$1)\ + ) + +####################################################################################################################### +# Common defaults + +TOPDIR = $(call memorize,TOPDIR,$(DOBUILD_TOPDIR)) +PROJECTDIR = $(call memorize,PROJECTDIR,$(DOBUILD_PROJECTDIR)) +PROJECTNAME = $(call memorize,PROJECTNAME,$(DOBUILD_PROJECTNAME)) +DOBUILDDIR := $(patsubst %/,%,$(dir $(current_makefile))) +OUTDIR = $(call memorize,OUTDIR,$(DOBUILD_OUTDIR)) +BUILDDIR = $(call memorize,BUILDDIR,$(DOBUILD_BUILDDIR)) +EXTDIR = $(call memorize,EXTDIR,$(DOBUILD_EXTDIR)) + +BUILDVERBOSE = $(call memorize,BUILDVERBOSE,$(DOBUILD_BUILDVERBOSE)) +TESTVERBOSE = $(call memorize,TESTVERBOSE,$(DOBUILD_TESTVERBOSE)) +BUILDSILENT = $(if $(BUILDVERBOSE),,1) +TESTSILENT = $(if $(TESTVERBOSE),,1) +BUILD_TESTING = $(call memorize,BUILD_TESTING,$(DOBUILD_BUILDTESTING)) + +VERSIONFILE = $(call memorize,VERSIONFILE,$(DOBUILD_PROJECTVERSIONFILE)) +VERSION = $(call memorize,VERSION,$(DOBUILD_PROJECTVERSION)) + +USERID = $(call memorize,USERID,$(DOBUILD_USERID)) + +HOST_CONTAINER = $(call memorize,HOST_CONTAINER,$(DOBUILD_HOSTCONTAINER)) + +SOURCE_DATE_EPOCH ?= $(call memorize,SOURCE_DATE_EPOCH,$(shell $(DOBUILDDIR)/bin/get_source_date_epoch $(TOPDIR))) +BUILDTIME = $(call memorize,BUILDTIME,$(shell date -u -d '@$(SOURCE_DATE_EPOCH)' --rfc-3339 ns 2>/dev/null)) + +JOBSLOTS_DEFAULT = $(if $(findstring --jobserver-,$(make_cmdline)),2,$(shell nproc 2>/dev/null || echo '2')) +JOBSLOTS = $(call memorize,JOBSLOTS,$(or $(DOBUILD_JOBSLOTS),$(JOBSLOTS_DEFAULT))) + +SKIP_MD5SUM = $(call memorize,SKIP_MD5SUM,$(DOBUILD_SKIPMD5SUM)) +SKIP_CURL = $(call memorize,SKIP_CURL,$(DOBUILD_SKIPCURL)) +SKIP_DEFAULTTARGET = $(call memorize,SKIP_DEFAULTTARGET,$(DOBUILD_SKIPDEFAULTTARGET)) +SKIP_EXTERNSYNC = $(call memorize,SKIP_EXTERNSYNC,$(DOBUILD_SKIPEXTERNSYNC)) + +DEFAULTTARGET = $(BUILDTARGET) +HOSTMARCH = +MARCH = +VENDOR = +DISTRIB_ID = $(VENDOR) +SYS = +ABI = +DISTRIB_VERSION = +ID = +VARIANT = + +EXCLUDE_HOSTMARCH = +EXCLUDE_MARCH = +EXCLUDE_VENDOR = +EXCLUDE_DISTRIB_ID = $(EXCLUDE_VENDOR) +EXCLUDE_SYS = +EXCLUDE_ABI = +EXCLUDE_DISTRIB_VERSION = +EXCLUDE_ID = +EXCLUDE_VARIANT = + +FILTER = $(call memorize,FILTER,$(DOBUILD_FILTER)) +EXCLUDEFILTER = $(call memorize,EXCLUDEFILTER,$(DOBUILD_EXCLUDEFILTER)) +MEMCHECKFILTER = $(call memorize,MEMCHECKFILTER,$(DOBUILD_FILTERMEMCHECK)) + +$(VERBOSE)SILENT := @ + +make_pid = $(call memorize,make_pid,$(shell echo "$$PPID")) +make_cmdline = $(call memorize,make_cmdline,$(shell set -- $$(ps T 2>/dev/null | sed -n -e 's!^\s*$(make_pid)\s\+.*\($(call escape,$(MAKE),!)\s\+.*\)!\1!p') && echo "$$@")) + +machine = $(call memorize,machine,$(shell uname -m 2>/dev/null)) + +make_version = $(MAKE_VERSION) +make_version_req = 3.81 + +# defines $, $; $% $= $: $(\#) $(\\) $($$) $( ) variables. +# NOTE: $(,) has to be used within macros to avoid conflict with argument spliting operator , +, := , +; := ; +% := % +esc_slash := \\ +$(esc_slash) := \\ +esc_hash := \\\# +$(esc_hash) := \# +esc_equal := = +$(esc_equal) := = +esc_colon := : +$(esc_colon) := : +esc_dollar := $$ +$(esc_dollar) := $$ +esc_space := +esc_space += +$(esc_space) := +$(esc_space) += +define n + + +endef + +DEFAULT_FMT := %s +ifneq ($(COLOR),) + DEFAULT_FMT := %b + tc_red := \033[0;31m + tc_green := \033[0;32m + tc_brown := \033[0;33m + tc_blue := \033[0;34m + tc_purple := \033[0;35m + tc_cyan := \033[0;36m + tc_light_gray := \033[0;37m + tc_dark_gray := \033[1;30m + tc_light_green := \033[1;32m + tc_yellow := \033[1;33m + tc_light_blue := \033[1;34m + tc_light_purple := \033[1;35m + tc_light_cyan := \033[1;36m + tc_reset := \033[0m +endif + +docker_runflags = +docker_buildflags = +image_buildargs = +project_targets = + +####################################################################################################################### +# Makefile dependencies + +MAKEFILE_DEPS += cat +MAKEFILE_DEPS += id +MAKEFILE_DEPS += echo +MAKEFILE_DEPS += sed +MAKEFILE_DEPS += pwd +MAKEFILE_DEPS += cd +MAKEFILE_DEPS += sort +MAKEFILE_DEPS += printf +MAKEFILE_DEPS += test +MAKEFILE_DEPS += base64 +MAKEFILE_DEPS += stat + +####################################################################################################################### +# Basic assertions + +ASSERTIONS += $(call assert_scalar,$(PROJECTDIR),Project directory PROJECTDIR='$(PROJECTDIR)' should not contain whitespaces) +ASSERTIONS += $(call assert_scalar,$(OUTDIR),Output directory OUTDIR='$(OUTDIR)' should not contain whitespaces) +ASSERTIONS += $(call assert_scalar,$(DOBUILDDIR),Script directory DOBUILDDIR='$(DOBUILDDIR)' should not contain whitespaces) +ASSERTIONS += $(call assert,$(call not,$(filter $(abspath $(PROJECTDIR)),$(abspath $(sort $(OUTDIR) $(OUTDIRS))))),Project \ +location PROJECTDIR='$(abspath $(PROJECTDIR))' should not point to one of the output locations:$n \ +$(addsuffix $n,$(abspath $(sort $(OUTDIR) $(OUTDIRS))))) +ASSERTIONS += $(call assert,$(SOURCE_DATE_EPOCH),Value of variable SOURCE_DATE_EPOCH should not be empty) +ASSERTIONS += $(call assert,$(or $(SKIP_DEFAULTTARGET),$(filter $(DEFAULTTARGET),$(project_targets))),Default \ +target TARGET='$(DEFAULTTARGET)' is not contained in the list of project targets:$n \ +$(addsuffix $n,$(sort $(project_targets)))) +ASSERTIONS += $(call assert,$(or $(call not,$(project_targets)),$(strip $(TARGETS))),No target matches$(,) with include \ +FILTER=[$(call join_s,$(FILTER),$(,))] and exclude filter EXCLUDEFILTER=[$(call join_s,$(EXCLUDEFILTER),$(,))]$(,) \ +one of the project targets:$n $(addsuffix $n,$(sort $(project_targets)))) + +EXPECTATIONS += $(call expect,$(call make_version_ge,$(make_version_req)),Using old make version=$(make_version)$(,) \ +consider upgrading to a newer version >= $(make_version_req)) + +endif diff --git a/docker-compose.yml b/docker-compose.yml new file mode 100644 index 0000000..1fb5941 --- /dev/null +++ b/docker-compose.yml @@ -0,0 +1,73 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +version: "2.4" + +services: + check: + build: + context: . + args: + - DOCKER_VERSION=$COMPOSEENV_DOCKER_VERSION + - REGISTRY_PREFIX=$COMPOSEENV_REGISTRY_PREFIX + - DOCKER_DOWNLOAD=$COMPOSEENV_DOCKER_DOWNLOAD + - BATS_DOWNLOAD=$COMPOSEENV_BATS_DOWNLOAD + - DUMB_INIT_DOWNLOAD=$COMPOSEENV_DUMB_INIT_DOWNLOAD + - LIBFAKETIME_DOWNLOAD=$COMPOSEENV_LIBFAKETIME_DOWNLOAD + - CAPATH=$COMPOSEENV_CAPATH + dockerfile: tests/runners/bats.dockerfile + image: dobuild_bats:${COMPOSEENV_PROJECT_VERSION} + depends_on: + - dind + environment: + DOCKER_HOST: tcp://dind:2375 + DOCKER_VERSION: $COMPOSEENV_DOCKER_VERSION + REGISTRY_PREFIX: $COMPOSEENV_REGISTRY_PREFIX + TMPDIR: /var/tmp + volumes: + - tmp:/var/tmp + volumes_from: + - dind:ro + networks: + - service + stdin_open: true + tty: true + working_dir: ${COMPOSEENV_PROJECTPATH} + command: ["bash"] + + dind: + build: + context: . + args: + - DOCKER_VERSION=$COMPOSEENV_DOCKER_VERSION + - REGISTRY_PREFIX=$COMPOSEENV_REGISTRY_PREFIX + - CAPATH=$COMPOSEENV_CAPATH + dockerfile: tests/runners/dind.dockerfile + image: dobuild_dind:${COMPOSEENV_PROJECT_VERSION} + environment: + DOCKER_HOST: tcp://localhost:2375 + volumes: + - /etc/timezone:/etc/timezone:ro + - /etc/localtime:/etc/localtime:ro + - docker:/var/lib/docker + - tmp:/var/tmp + networks: + - service + privileged: true + command: ["dockerd", "--host=tcp://0.0.0.0:2375", "--storage-driver", "overlay2"] + +volumes: + tmp: + docker: + +networks: + service: + driver: bridge + diff --git a/docker.mk b/docker.mk new file mode 100644 index 0000000..099faea --- /dev/null +++ b/docker.mk @@ -0,0 +1,499 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +ifndef _DOBUILD_INCLUDE_GUARD_DOCKER +_DOBUILD_INCLUDE_GUARD_DOCKER := 1 + +current_makefile := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)) + +ifndef defaults_include_guard + include $(patsubst %/,%,$(dir $(current_makefile)))/defaults.mk +endif + +####################################################################################################################### +# Overridable docker defaults + +DOCKER ?= docker +DOBUILD_DOCKERFILE ?= $(PROJECTDIR)/%MARCH%-%DISTRIB_ID%-%ID%.dockerfile +DOBUILD_RUNCMD ?= + +####################################################################################################################### +# Overridable docker macros, hooks to customize target default values + +# hook called to retrieve the target dockerfile name, +# may return an empty value in which case the default dockerfile is used +# $(call dobuild_dockerfile,target-name) +dobuild_dockerfile ?= $(DOCKERFILE) + +# hook called to retrieve the target image build context, +# may return an empty value in which case OUTDIR is used +# $(call dobuild_image_context,target-name) +dobuild_image_context ?= + +# hook called to retrieve the target image archive file name, +# may return an empty value +# $(call dobuild_image_archivefile,target-name) +dobuild_image_archivefile ?= $(call image_default_archivefile,$1) + +# hook called to retrieve the target image prerequisites, +# may return an empty list +# $(call dobuild_image_prerequisites,target-name) +dobuild_image_prerequisites ?= + +# hook called to retrieve the target image names, +# may return an empty list in which case the name is derived from the build target name +# $(call dobuild_image_names,target-name) +dobuild_image_names ?= + +# hook called to retrieve the target container user, +# may return an empty value in which case CONTAINER_USER is used +# $(call dobuild_container_user,target-name) +dobuild_container_user ?= + +# hook called to retrieve the target container group, +# may return an empty value in which case CONTAINER_GROUP is used +# $(call dobuild_container_group,target-name) +dobuild_container_group ?= + +# hook called to retrieve the target container run command, +# may return an empty value in which case CONTAINER_CMD is used +# $(call dobuild_container_cmd,target-name) +dobuild_container_cmd ?= + +# hook called to retrieve the target image build arguments, +# may return an empty list +# $(call dobuild_image_buildargs,target-name) +dobuild_image_buildargs ?= \ + $(addprefix HOSTMARCH=,%HOSTMARCH%) \ + $(addprefix MARCH=,%MARCH%) \ + $(addprefix DISTRIB_ID=,%DISTRIB_ID%) \ + $(addprefix DISTRIB_VERSION=,%DISTRIB_VERSION%) \ + $(addprefix SYS=,%SYS%) \ + $(addprefix ABI=,%ABI%) \ + $(addprefix ID=,%ID%) \ + $(addprefix VARIANT=,%VARIANT%) + +# hook called to retrieve the target docker build flags, +# may return an empty list +# $(call dobuild_docker_buildflags,target-name) +dobuild_docker_buildflags ?= + +# hook called to retrieve the target docker run flags, +# may return an empty list +# $(call dobuild_docker_runflags,target-name) +dobuild_docker_runflags ?= + +####################################################################################################################### +# Docker macros + +# retrieves the version parts (major, minor) of a docker version string +# $(call docker_version_parts,version-string) +docker_version_parts = $(shell echo '$1' | sed -ne 's![^0-9]*[0]*\([^.]\+\).[0]*\([^.]\+\).*!\1 \2!p') + +# tests if docker version is greater or equal to required version +# $(call docker_version_ge,required-version-string) +docker_version_ge = \ + $(call ge_version, \ + $(call docker_version_parts,$(docker_version)),\ + $(call docker_version_parts,$1)\ + ) + +# retrieves the target default docker identifier file +# $(call docker_default_idfile,target-name...) +docker_default_idfile = $(addsuffix /DoBuildFiles/docker.idfile,$(addprefix $(OUTDIR)/,$1)) + +# retrieves the target default dockerfile name +# $(call image_default_dockerfile,target-name...) +image_default_dockerfile = $(addsuffix /Dockerfile,$(addprefix $(OUTDIR)/,$1)) + +# retrieves the target default image archive file name +# $(call image_default_archivefile,target-name...) +image_default_archivefile = $(addsuffix /image-root$(addprefix -,$(VERSION)).tar,$(addprefix $(OUTDIR)/,$1)) + +# retrieves the target default image prerequisites +# $(call image_default_prerequisites,target-name) +image_default_prerequisites = + +# retrieves the target default image build flags +# $(call image_default_buildflags,target-name) +image_default_buildflags = $(addprefix --build-arg ,$(strip $(cache.$1.image_buildargs))) + +# retrieves the target default environment path variable value +# $(call env_default_path,target-name) +env_default_path = $(call join_s,\ + $(abspath \ + $(cache.$1.container_extdir) \ + $(addprefix $(container_dobuilddir)/assets/adapters/,$(cache.$1.adapter)) \ + ) \ + $(call image_env_path,$1),\ + : \ + ) + +# retrieves the target default extension directory value +# $(call container_default_exdir,target-name) +container_default_extdir = $(if $(docker_portable_workspace),$(container_extdir),$(cache.$1.extdir)) + +# retrieves the target default run flags +# $(call docker_default_runflags,target-name) +docker_default_runflags = \ + $(if $(and $(docker_portable_workspace),$(realpath $(cache.$1.extdir))),--volume '$(realpath $(cache.$1.extdir)):$(abspath $(cache.$1.container_extdir)):cached') + +# retrieves the target image path env variable value +# $(call image_env_path,target-name) +image_env_path = $(shell $(DOCKER) inspect -f '{{range $$i, $$var := .Config.Env}}{{println $$var}}{{end}}' '$(call image_name,$1)' | sed -n -e 's!^PATH=\(.*\)!\1!p') + +# formats a list of target properties as string +# $(call target_properties_format,property...) +image_target_properties_format = \ + $(call join_s, \ + $(call target_property_host_arch,$1) \ + $(call target_property_arch_sub,$1) \ + $(call target_property_vendor_or_distrib_id,$1) \ + $(call target_property_distrib_version,$1) \ + $(call target_property_sys,$1) \ + $(call target_property_abi,$1) \ + $(PROJECTNAME) \ + $(call target_property_id,$1) \ + $(call replace_all,$(call target_property_variant,$1),release,), \ + /) + +# retrieves the target default image qualified name (registry/repository:tag-version) +# $(call image_default_name,target-name[,tag-version][,registry-prefix]) +image_default_name = \ + $(call join_s, \ + $3 \ + $(call image_target_properties_format,$(call target_properties,$1)) \ + $(addprefix :,$2) \ + ) + +# retrieves the target image archive file name +# $(call image_archivefile,target-name...) +image_archivefile = $(foreach target,$1,$(cache.$(target).image_archivefile)) + +# retrieves the target image build target name +# $(call image_buildtarget,target-name...) +image_buildtarget = $(call image_idfile,$(filter $1,$(DOCKER_TARGETS))) + +# retrieves the target image identifier file +# $(call image_idfile,target-name...) +image_idfile = $(addsuffix -image.idfile,$(addprefix $(OUTDIR)/,$1)) + +# retrieves the unique local identifier from id file +# $(call image_id,id-file...) +image_id = $(shell cat $1 2>/dev/null) + +# retrieves the image hash value from id file +# $(call image_hash,id-file...) +image_hash = $(patsubst sha256:%,%,$(call image_id,$1)) + +# retrieves the target image name +# $(call image_name,target-name...) +image_name = $(foreach target,$1,$(cache.$(target).image_name)) + +# creates an image build command +# $(call image_build_cmd,dockerfile,tag...,context[,docker-build-flag...][,output-prefix]) +image_build_cmd = $(call echo_cmd,$(DOCKER),build --rm $4 --file '$1' $(addprefix --tag ,$2) '$3',$(addprefix $5 ,docker)) + +# creates an image build rule command for given target configuration, assuming the first prerequisite is the dockerfile +# $(call image_build_rule_cmd,target-name,idfile) +image_build_rule_cmd = { \ + mkdir -p $(dir $2); \ + $(call image_build_cmd,$<, \ + $(cache.$1.image_tags),$(cache.$1.image_context),$(docker_buildflags) $(cache.$1.docker_buildflags) --iidfile '$2',\ + $(call color,green,TARGET=$1)); \ + } + +# creates an image build rule +# $(call image_build_rule,target-name) +image_build_rule = \ + build/image/$1 $$(cache.$1.docker_idfile): $$(cache.$1.dockerfile) $$(cache.$1.image_prerequisites) $$(EXTRACT_TARGETS) $$(MAKEFILE_LIST); \ + $$(SILENT)$$(call image_build_rule_cmd,$1,$$(cache.$1.docker_idfile)) + +# creates an image tag rule command for given target configuration +# $(call image_tag_rule_cmd,target-name,oci-idfile,docker-idfile) +# NOTE: workaround for moby/moby issue #39796 +# referencing images with local sha256:id does not work when referenced as base image using BuildKit, +# therefore we tag the image additionally with its local unique identifier +image_tag_rule_cmd = { \ + mkdir -p $(dir $2); \ + name='$(call image_default_name,$1,$(call image_hash,$3),)'; \ + $(call echo_cmd,$(DOCKER),tag '$(call image_id,$3)' "$$name",docker) \ + && echo "$$name" > '$2'; \ + } + +# creates an image tag rule +# $(call image_tag_rule,target-name) +image_tag_rule = \ + $$(cache.$1.image_idfile): $$(cache.$1.docker_idfile); \ + $$(SILENT)$$(call image_tag_rule_cmd,$1,$$(cache.$1.image_idfile),$$(cache.$1.docker_idfile)) + +# creates an image save command +# $(call image_save_cmd,image-name,output-file[,output-prefix]) +image_save_cmd = $(call echo_cmd,$(DOCKER),save '$1' $(addprefix --output ,$2),$(addprefix $3 ,docker)) + +# creates an image save rule command for given target configuration +# $(call image_save_rule_cmd,target-name,archivefile) +image_save_rule_cmd = $(call image_save_cmd,$(call image_name,$1),$2,$(call color,green,TARGET=$1)) + +# creates an image save rule +# $(call image_save_rule,target-name) +image_save_rule = \ + package/image/$1 $$(cache.$1.image_archivefile): $$(cache.$1.image_idfile); \ + $$(SILENT)$$(call image_save_rule_cmd,$1,$$(cache.$1.image_archivefile)) + +# creates an image load command +# $(call image_load_cmd,image-name,input-file) +image_load_cmd = $(call echo_if_silent_cmd,docker image load $2) \ + && $(DOCKER) tag "$$($(DOCKER) load --quiet --input '$2' | sed -n -e 's!^Loaded image ID:\s\+\(.*\)!\1!p')" $1 + +# creates an image load rule command for given target configuration, assuming the first prerequisite is the image +# archive file to load +# $(call image_load_rule_cmd,image-name) +image_load_rule_cmd = \ + $(call image_load_cmd,$1,$<) + +# creates an image remove command +# $(call image_rm_cmd,image-name...[,output-prefix]) +image_rm_cmd = $(if $(strip $1),$(call echo_cmd,$(DOCKER),rmi -f $1,$(addprefix $2 ,docker)),true) + +# creates an image clean command +# $(call image_clean_cmd,target-name[,output-prefix]) +image_clean_cmd = { \ + $(call image_rm_cmd,$(cache.$1.image_tags),$2) 2>/dev/null; \ + rm -f '$(cache.$1.docker_idfile)'; \ + } + +# creates an image clean rule command +# $(call image_clean_rule_cmd,target-name) +image_clean_rule_cmd = $(call image_clean_cmd,$1,$(call color,green,TARGET=$1)) + +# creates an image clean rule +# $(call image_clean_rule,target-name) +image_clean_rule = \ + clean/image/$1: ; \ + $$(SILENT)-$$(call image_clean_rule_cmd,$1) + +# creates an image dist clean rule command +# $(call image_distclean_rule_cmd,target-name) +image_distclean_rule_cmd = { \ + $(call echo_if_silent_cmd,$(call color,green,TARGET=$1) distclean image); \ + $(call image_clean_cmd,$1,); \ + $(call image_rm_cmd,$(call image_name,$1),); \ + rm -f '$(cache.$1.image_idfile)'; \ + } + +# creates an image dist clean rule +# $(call image_distclean_rule,target-name) +image_distclean_rule = \ + distclean/image/$1: ; \ + $$(SILENT)-$$(call image_distclean_rule_cmd,$1) + +# retrieves the target container default working directory +# $(call container_default_workdir,target-name...) +container_default_workdir = $(addprefix $(container_outdir)/,$1) + +# creates a command which is executed in a container +# $(call container_run_cmd,[target-name],image-name,workdir,command...[,docker-run-flag...][,output-command]) +container_run_cmd = \ + $(call echo_if_silent_cmd,$(call color,green,$(addprefix TARGET=,$1)) container_run $(or $6,$4)) \ + && $(DOBUILDDIR)/bin/container_run $5 --workdir '$3' '$2' $4 + +# creates a container run rule +# $(call container_run_rule,target-name) +container_run_rule = \ + run/container/$1: $$(cache.$1.image_idfile); \ + $$(SILENT)-$$(call run_cmd,$1,$$(cache.$1.container_cmd)) + +# creates a container run rule command +# $(call run_cmd,target-name,command[,args...][,output-command]) +run_cmd = $(call container_run_cmd,$1,$(call image_name,$1),$(cache.$1.container_workdir),$2 $3,\ + $(docker_runflags) -e 'PATH=$(cache.$1.env_path)' \ + --user '$(cache.$1.container_user):$(cache.$1.container_group)' $(cache.$1.docker_runflags),$(or $4,$2) $3) + +####################################################################################################################### +# Docker rule target configuration + +DOCKERFILE = $(call memorize,DOCKERFILE,$(DOBUILD_DOCKERFILE)) + +CONTAINER_CMD = $(call memorize,CONTAINER_CMD,$(DOBUILD_RUNCMD)) +CONTAINER_USER = $(USERID) +CONTAINER_GROUP = $(USERID) + +docker_version = $(call memorize,docker_version,$(shell $(DOCKER) version --format '{{.Client.Version}}')) +docker_version_req = 18.09 + +container_dobuilddir = /mnt/dobuild +container_projectdir = $(container_dobuilddir)/workspace/src +container_outdir = $(container_dobuilddir)/workspace/out +container_extdir = $(container_dobuilddir)/workspace/extension +container_destdir = $(if $(DESTDIR),$(container_dobuilddir)/workspace/stage) + +docker_portable_workspace = $(call not,$(HOST_CONTAINER)) + +ifeq ($(docker_portable_workspace),) + container_projectdir = $(abspath $(PROJECTDIR)) + container_dobuilddir = $(abspath $(DOBUILDDIR)) + container_outdir = $(abspath $(OUTDIR)) + container_destdir = $(abspath $(DESTDIR)) +endif + +container_cpus = $(INTERNPARALLEL) +container_cpuperiod = 100000 +container_quota = $(call bc,($(container_cpus)*$(container_cpuperiod))) +# allow twice as much parallel executions, while container are already limited by cgroup +container_nproc = $(call bc,(2*$(container_cpus))) + +docker_runflags += $(DOCKER_RUNFLAGS) +docker_runflags += $(addprefix --cpus ,$(container_cpus)) +docker_runflags += --env SOURCE_DATE_EPOCH +docker_runflags += --env BUILDTIME +docker_runflags += --env DOBUILD_VERBOSE +docker_runflags += --env DOBUILD_BUILDVERBOSE +docker_runflags += --env DOBUILD_TESTVERBOSE +docker_runflags += --env DOBUILD_NPROC$(addprefix $(=),$(container_nproc)) + +ifeq ($(HOST_CONTAINER),) + docker_runflags += --volume '$(realpath $(PROJECTDIR)):$(container_projectdir):cached' + docker_runflags += --volume '$(realpath $(DOBUILDDIR)):$(container_dobuilddir):cached' + docker_runflags += --volume '$(realpath $(OUTDIR)):$(container_outdir):delegated' + ifneq ($(container_destdir),) + ASSERTIONS += $(call assert,$(realpath $(DESTDIR)),Staging directory DESTDIR='$(DESTDIR)' doesn't exist) + docker_runflags += --volume '$(realpath $(DESTDIR)):$(container_destdir):delegated' + endif +endif + +image_buildargs += $(IMAGE_BUILDARGS) +image_buildargs += 'PARALLELMFLAGS=$(addprefix -j,$(container_nproc))' +image_buildargs += 'REGISTRY_PREFIX=$(REGISTRY_PREFIX)' + +docker_buildflags += $(DOCKER_BUILDFLAGS) +docker_buildflags += $(addprefix --cpu-period ,$(container_cpuperiod)) +docker_buildflags += $(addprefix --cpu-quota ,$(container_quota)) +docker_buildflags += $(addprefix --build-arg ,$(image_buildargs)) + +docker_defaulttarget = $(if $(SKIP_DEFAULTTARGET),,$(DEFAULTTARGET)) +docker_targets = $(filter $(docker_defaulttarget),$(DOCKER_TARGETS)) +docker_selected_targets = $(if $(SKIP_DEFAULTTARGET),$(DOCKER_TARGETS),$(docker_targets)) +docker_active_targets = $(call memorize,docker_active_targets,$(call target_filter,$(FILTER),$(docker_selected_targets),$(EXCLUDEFILTER))) + +docker_build_targets = $(call image_idfile,$(docker_active_targets)) +docker_run_targets = $(addprefix run/container/,$(docker_active_targets)) +docker_clean_targets = $(addprefix clean/image/,$(docker_active_targets)) +docker_dist_targets = +docker_distclean_targets = $(addprefix distclean/image/,$(docker_active_targets)) +docker_rule_targets = $(addsuffix /dockerrules.mk,$(OUTDIR)) +docker_outdirs = $(addprefix $(OUTDIR)/,$(docker_active_targets)) + +project_targets += $(DOCKER_TARGETS) + +BUILD_TARGETS += $(docker_build_targets) +DIST_TARGETS += $(docker_dist_targets) +DISTCLEAN_TARGETS += $(docker_distclean_targets) +CLEAN_TARGETS += $(docker_clean_targets) +RUN_TARGETS += $(docker_run_targets) +RULE_TARGETS += $(docker_rule_targets) +TARGETS += $(docker_active_targets) +OUTDIRS += $(docker_outdirs) + +####################################################################################################################### +# Makefile dependencies + +MAKEFILE_DEPS += $(DOCKER) +MAKEFILE_DEPS += cat +MAKEFILE_DEPS += chmod +MAKEFILE_DEPS += rm +MAKEFILE_DEPS += mkdir + +####################################################################################################################### +# Docker rules + +$(docker_rule_targets): + $(SILENT) \ + { \ + echo '$(\#) generated file - do not edit!!!'; \ + echo; \ + ID='$(call id,$@)'; \ + echo "ifndef $${ID}_include_guard"; \ + echo "$${ID}_include_guard := 1"; \ + echo; \ + $(foreach target,$(DOCKER_TARGETS),\ + echo '$(\#)$(\#) BEGIN of docker $(target) configuration'; \ + echo; \ + echo '$(\#)$(\#)$(\#) defaults'; \ + echo '$(target) ?= $$(call memorize,$(target),$(call target_properties_parse,$(target)))'; \ + echo '$(target).env_path = $$(call env_default_path,$$1)'; \ + echo '$(target).adapter ?= '; \ + echo '$(target).extdir ?= '; \ + echo '$(target).dockerfile ?= $$(call dobuild_dockerfile,$$1)'; \ + echo '$(target).image_context ?= $$(call dobuild_image_context,$$1)'; \ + echo '$(target).image_archivefile ?= $$(call dobuild_image_archivefile,$$1)'; \ + echo '$(target).image_prerequisites ?= $$(call dobuild_image_prerequisites,$$1)'; \ + echo '$(target).image_tags ?= $$(call dobuild_image_names,$$1)'; \ + echo '$(target).image_buildargs ?= $$(call dobuild_image_buildargs,$$1)'; \ + echo '$(target).image_name = $$(call image_id,$$(cache.$(target).image_idfile))'; \ + echo '$(target).container_workdir = $$(call container_default_workdir,$$1)'; \ + echo '$(target).container_extdir = $$(call container_default_extdir,$$1)'; \ + echo '$(target).container_user ?= $$(call dobuild_container_user,$$1)'; \ + echo '$(target).container_group ?= $$(call dobuild_container_group,$$1)'; \ + echo '$(target).container_cmd ?= $$(call dobuild_container_cmd,$$1)'; \ + echo '$(target).docker_buildflags ?= $$(call dobuild_docker_buildflags,$$1)'; \ + echo '$(target).docker_runflags ?= $$(call dobuild_docker_runflags,$$1)'; \ + echo; \ + echo '$(\#)$(\#)$(\#) cached values'; \ + echo 'cache.$(target).image_idfile = $$(call memorize,cache.$(target).image_idfile,$$(call image_idfile,$(target)))'; \ + echo 'cache.$(target).docker_idfile = $$(call memorize,cache.$(target).docker_idfile,$$(call docker_default_idfile,$(target)))'; \ + echo 'cache.$(target).env_path = $$(call memorize,cache.$(target).env_path,$$(call $(target).env_path,$(target)))'; \ + echo 'cache.$(target).adapter = $$(call memorize,cache.$(target).adapter,$$(call target_get_and_subst,$(target),adapter,))'; \ + echo 'cache.$(target).extdir = $$(call memorize,cache.$(target).extdir,$$(call target_get_and_subst,$(target),extdir,$$(EXTDIR)))'; \ + echo 'cache.$(target).dockerfile = $$(call memorize,cache.$(target).dockerfile,$$(call target_get_and_subst,$(target),dockerfile,$$(call image_default_dockerfile,$(target))))'; \ + echo 'cache.$(target).image_context = $$(call memorize,cache.$(target).image_context,$$(call target_get_and_subst,$(target),image_context,$$(OUTDIR)))'; \ + echo 'cache.$(target).image_archivefile = $$(call memorize,cache.$(target).image_archivefile,$$(call target_get_and_subst,$(target),image_archivefile,))'; \ + echo 'cache.$(target).image_prerequisites = $$(call memorize,cache.$(target).image_prerequisites,$$(call target_get_and_subst,$(target),image_prerequisites,) $$(call image_default_prerequisites,$(target)))'; \ + echo 'cache.$(target).image_tags = $$(call memorize,cache.$(target).image_tags,$$(call target_get_and_subst,$(target),image_tags,$$(call image_default_name,$(target),$$(VERSION),$$(REGISTRY_PREFIX))))'; \ + echo 'cache.$(target).image_buildargs = $$(call memorize,cache.$(target).image_buildargs,$$(call target_get_and_subst,$(target),image_buildargs,))'; \ + echo 'cache.$(target).image_name = $$(call memorize,cache.$(target).image_name,$$(call $(target).image_name,$(target)))'; \ + echo 'cache.$(target).container_workdir = $$(call memorize,cache.$(target).container_workdir,$$(call $(target).container_workdir,$(target)))'; \ + echo 'cache.$(target).container_extdir = $$(call memorize,cache.$(target).container_extdir,$$(call $(target).container_extdir,$(target)))'; \ + echo 'cache.$(target).container_user = $$(call memorize,cache.$(target).container_user,$$(or $$(call $(target).container_user,$(target)),$$(CONTAINER_USER)))'; \ + echo 'cache.$(target).container_group = $$(call memorize,cache.$(target).container_group,$$(or $$(call $(target).container_group,$(target)),$$(CONTAINER_GROUP)))'; \ + echo 'cache.$(target).container_cmd = $$(call memorize,cache.$(target).container_cmd,$$(or $$(call $(target).container_cmd,$(target)),$$(CONTAINER_CMD)))'; \ + echo 'cache.$(target).docker_buildflags = $$(call memorize,cache.$(target).docker_buildflags,$$(call target_get_and_subst,$(target),docker_buildflags,) $$(call image_default_buildflags,$(target)))'; \ + echo 'cache.$(target).docker_runflags = $$(call memorize,cache.$(target).docker_runflags,$$(call $(target).docker_runflags,$(target)) $$(call docker_default_runflags,$(target)))'; \ + echo; \ + echo '$(\#)$(\#)$(\#) rules'; \ + echo '$(call image_build_rule,$(target))'; \ + echo; \ + echo '$(call image_tag_rule,$(target))'; \ + echo; \ + echo '$(call image_save_rule,$(target))'; \ + echo; \ + echo '$(call image_clean_rule,$(target))'; \ + echo; \ + echo '$(call image_distclean_rule,$(target))'; \ + echo; \ + echo '$(call container_run_rule,$(target))'; \ + echo; \ + echo '$(\#)$(\#) END of docker $(target) configuration'; \ + echo; \ + ) \ + echo 'endif'; \ + echo; \ + } > $@ + +####################################################################################################################### +# Docker assertions + +ASSERTIONS += $(call assert,$(DOCKER),Value of variable DOCKER should not be empty) + +EXPECTATIONS += $(call expect,$(call docker_version_ge,$(docker_version_req)),Using old docker version=$(docker_version)$(,) \ +consider upgrading to a newer version >= $(docker_version_req)) + +endif + diff --git a/examples/cmake-gtest-example/CMakeLists.txt b/examples/cmake-gtest-example/CMakeLists.txt new file mode 100644 index 0000000..ea6e6af --- /dev/null +++ b/examples/cmake-gtest-example/CMakeLists.txt @@ -0,0 +1,31 @@ +cmake_minimum_required(VERSION 3.10) + +project(cmake-gtest-example VERSION 0.0.1 DESCRIPTION "CMake gtest example") + +option(THREADS_PREFER_PTHREAD_FLAG "If the use of the -pthread compiler and linker flag is prefered then the caller can set" ON) +option(POSITION_INDEPENDENT_CODE "This variable is used to initialize the POSITION_INDEPENDENT_CODE property on all the targets" ON) + +find_package(PkgConfig REQUIRED) +find_package(Threads REQUIRED) + +pkg_check_modules(GTEST gtest_main gtest) + +include(CTest) +include(GoogleTest) + +add_executable(alltests + test_stringcompare.cpp +) + +target_include_directories(alltests PUBLIC ${GTEST_INCLUDE_DIRS}) +target_compile_options(alltests PUBLIC -Wall -Wextra ${GTEST_CFLAGS}) +target_link_libraries(alltests PUBLIC Threads::Threads ${GTEST_LIBRARIES} gtest_main) + +enable_testing() +gtest_discover_tests(alltests TEST_PREFIX alltests:) + +add_custom_target(check COMMAND ${CMAKE_CTEST_COMMAND}) +add_dependencies(check alltests) + +add_custom_target(memcheck COMMAND ${CMAKE_CTEST_COMMAND} -T memcheck) +add_dependencies(memcheck alltests) diff --git a/examples/cmake-gtest-example/Makefile b/examples/cmake-gtest-example/Makefile new file mode 100644 index 0000000..2ade478 --- /dev/null +++ b/examples/cmake-gtest-example/Makefile @@ -0,0 +1,94 @@ +SHELL := /bin/sh +MAKEFILE := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)) +MAKEFLAGS += --no-builtin-rules + +.SUFFIXES: + +.PHONY: default +default: all + +####################################################################################################################### +# Overridable project defaults + +DOBUILD_TOPDIR ?= $(DOBUILDDIR) +DOBUILD_PROJECTDIR ?= $(patsubst %/,%,$(dir $(MAKEFILE))) +DOBUILD_DOCKERFILE ?= $(PROJECTDIR)/%ID%.dockerfile + +PROJECTDIR = $(DOBUILD_PROJECTDIR) +DOBUILDDIR ?= $(PROJECTDIR)/../.. + +include $(DOBUILDDIR)/defaults.mk + +####################################################################################################################### +# Project defaults and macros + +DEFAULTTARGET = x86_64-ubuntu@bionic+builder@debug + +FETCHDIR = $(BUILDDIR)/.deps + +####################################################################################################################### +# Project dependencies + +GTEST_VERSION = 1.8.1 +IMAGE_BUILDARGS += GTEST_VERSION=$(GTEST_VERSION) +IMAGE_BUILDARGS += GTEST_MTIME=$(call mtime,$(FETCHDIR)/googletest-release-$(GTEST_VERSION).tar.gz) +FETCH_TARGETS += $(FETCHDIR)/googletest-release-$(GTEST_VERSION).tar.gz +$(FETCHDIR)/googletest-release-$(GTEST_VERSION).tar.gz: URL := https://github.com/google/googletest/archive/release-$(GTEST_VERSION).tar.gz +$(SKIP_MD5SUM)$(FETCHDIR)/googletest-release-$(GTEST_VERSION).tar.gz: MD5 := 2e6fbeb6a91310a16efe181886c59596 + +####################################################################################################################### +# Architecture-specific rule target configuration + +CMAKE_TARGETS += $(call target_properties_combine,\ + ,\ + x86_64 arm32v7,\ + ubuntu@bionic alpine@3.10,\ + ,\ + ,\ + builder,\ + ,\ + debug release \ + ) +DOCKER_TARGETS += $(CMAKE_TARGETS) + +####################################################################################################################### +# Common rule target configuration + +CURLFLAGS += -s + +DOCKER_RUNFLAGS += --cap-add SYS_PTRACE +DOCKER_RUNFLAGS += --security-opt seccomp=unconfined + +OUTDIRS += $(OUTDIR)/src + +EXTRACT_TARGETS += $(patsubst $(FETCHDIR)/%.tar.gz,$(OUTDIR)/src/%,$(FETCH_TARGETS)) + +####################################################################################################################### +# Makefile dependencies + +MAKEFILE_DEPS += gzip +MAKEFILE_DEPS += tar +MAKEFILE_DEPS += touch +MAKEFILE_DEPS += mkdir + +####################################################################################################################### +# Rules + +include $(DOBUILDDIR)/cmake.mk +include $(DOBUILDDIR)/docker.mk +include $(DOBUILDDIR)/standardrules.mk + +$(OUTDIR)/src/%: $(FETCHDIR)/%.tar.gz | $(OUTDIRS) + $(SILENT) \ + $(call echo_if_silent_cmd,tar -C $(dir $@) -xf $<) \ + && tar -I 'gzip -n' -C $(dir $@) -xf $< \ + && touch $@ + +$(FETCH_TARGETS): | $(FETCHDIR) + $(SILENT)$(call curl,$@,$(URL),$(MD5)) + +$(FETCHDIR): + $(SILENT)mkdir -p $@ + +.DELETE_ON_ERROR: $(FETCH_TARGETS) + diff --git a/examples/cmake-gtest-example/builder.dockerfile b/examples/cmake-gtest-example/builder.dockerfile new file mode 100644 index 0000000..7c4ecc0 --- /dev/null +++ b/examples/cmake-gtest-example/builder.dockerfile @@ -0,0 +1,30 @@ +ARG REGISTRY_PREFIX='' +ARG BUILDER_TAG=0.0.1 + +ARG USERID=1000 +ARG PARALLELMFLAGS=-j2 +ARG HOSTMARCH=x86_64 +ARG MARCH=$HOSTMARCH +ARG DISTRIB_ID=ubuntu +ARG DISTRIB_VERSION=bionic +ARG SYS=linux +ARG ABI=gnu + +FROM ${REGISTRY_PREFIX}${HOSTMARCH}/${MARCH}/${DISTRIB_ID}/${DISTRIB_VERSION}/${SYS}/${ABI}/cmake/builder-template:${BUILDER_TAG} + +COPY src /usr/local/src + +ARG PARALLELMFLAGS=-j2 + +ARG GTEST_VERSION=1.8.1 +ARG GTEST_MTIME= + +RUN set -x \ + && [ -n "$GTEST_MTIME" ] && export SOURCE_DATE_EPOCH="$GTEST_MTIME" \ + && builddir="/tmp/out" \ + && mkdir -p "$builddir" \ + && cd "$builddir" \ + && cmake "/usr/local/src/googletest-release-$GTEST_VERSION" \ + && make "$PARALLELMFLAGS" install \ + && rm -rf "$builddir" + diff --git a/examples/cmake-gtest-example/test_stringcompare.cpp b/examples/cmake-gtest-example/test_stringcompare.cpp new file mode 100644 index 0000000..6f7e7fd --- /dev/null +++ b/examples/cmake-gtest-example/test_stringcompare.cpp @@ -0,0 +1,18 @@ +#include + +#include +using std::string; + +char const actualValTrue[] = "hello gtest"; +char const actualValFalse[] = "hello world"; +char const expectVal[] = "hello gtest"; + +TEST(StrCompare, CStrEqual) +{ + EXPECT_STREQ(expectVal, actualValTrue); +} + +TEST(StrCompare, CStrNotEqual) +{ + EXPECT_STRNE(expectVal, actualValFalse); +} diff --git a/examples/gradle-junit5-example/Makefile b/examples/gradle-junit5-example/Makefile new file mode 100644 index 0000000..249adeb --- /dev/null +++ b/examples/gradle-junit5-example/Makefile @@ -0,0 +1,57 @@ +SHELL := /bin/sh +MAKEFILE := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)) +MAKEFLAGS += --no-builtin-rules + +.SUFFIXES: + +.PHONY: default +default: all + +####################################################################################################################### +# Overridable project defaults + +DOBUILD_TOPDIR ?= $(DOBUILDDIR) +DOBUILD_PROJECTDIR ?= $(patsubst %/,%,$(dir $(MAKEFILE))) +DOBUILD_DOCKERFILE ?= $(PROJECTDIR)/%ID%.dockerfile + +PROJECTDIR = $(DOBUILD_PROJECTDIR) +DOBUILDDIR ?= $(PROJECTDIR)/../.. + +include $(DOBUILDDIR)/defaults.mk + +####################################################################################################################### +# Project defaults and macros + +DEFAULTTARGET = x86_64-gradle@6.3-linux-java11+builder + +####################################################################################################################### +# Project dependencies + +####################################################################################################################### +# Architecture-specific rule target configuration + +GRADLE_TARGETS += $(call target_properties_combine,\ + ,\ + x86_64,\ + gradle,\ + linux,\ + java11,\ + builder,\ + 6.3,\ + \ + ) +DOCKER_TARGETS += $(GRADLE_TARGETS) + +####################################################################################################################### +# Common rule target configuration + +####################################################################################################################### +# Makefile dependencies + +####################################################################################################################### +# Rules + +include $(DOBUILDDIR)/gradle.mk +include $(DOBUILDDIR)/docker.mk +include $(DOBUILDDIR)/standardrules.mk + diff --git a/examples/gradle-junit5-example/build.gradle b/examples/gradle-junit5-example/build.gradle new file mode 100644 index 0000000..7c6fa4b --- /dev/null +++ b/examples/gradle-junit5-example/build.gradle @@ -0,0 +1,19 @@ +plugins { + id 'java' + id 'eclipse' +} + +repositories { + mavenCentral() +} + +dependencies { + testImplementation('org.junit.jupiter:junit-jupiter:5.6.2') +} + +test { + useJUnitPlatform() + testLogging { + events "passed", "skipped", "failed" + } +} diff --git a/examples/gradle-junit5-example/builder.dockerfile b/examples/gradle-junit5-example/builder.dockerfile new file mode 100644 index 0000000..d5ccbb3 --- /dev/null +++ b/examples/gradle-junit5-example/builder.dockerfile @@ -0,0 +1,15 @@ +ARG REGISTRY_PREFIX='' +ARG BUILDER_TAG=0.0.1 + +ARG HOSTMARCH +ARG MARCH +ARG DISTRIB_ID +ARG DISTRIB_VERSION +ARG SYS +ARG ABI + +FROM ${REGISTRY_PREFIX}${HOSTMARCH}/${MARCH}/graalvm-ce/20.0.0/${SYS}/${ABI}/${DISTRIB_ID}/${DISTRIB_VERSION}:${BUILDER_TAG} + +ARG ID= +ARG VARIANT= +ARG PARALLELMFLAGS= diff --git a/examples/gradle-junit5-example/src/main/java/com/github/nosamad/dobuild/example/Calculator.java b/examples/gradle-junit5-example/src/main/java/com/github/nosamad/dobuild/example/Calculator.java new file mode 100644 index 0000000..1e5c31a --- /dev/null +++ b/examples/gradle-junit5-example/src/main/java/com/github/nosamad/dobuild/example/Calculator.java @@ -0,0 +1,9 @@ +package com.github.nosamad.dobuild.example; + +public class Calculator { + + public int add(int a, int b) { + return a + b; + } + +} \ No newline at end of file diff --git a/examples/gradle-junit5-example/src/test/java/com/github/nosamad/dobuild/example/CalculatorTest.java b/examples/gradle-junit5-example/src/test/java/com/github/nosamad/dobuild/example/CalculatorTest.java new file mode 100644 index 0000000..7426ab8 --- /dev/null +++ b/examples/gradle-junit5-example/src/test/java/com/github/nosamad/dobuild/example/CalculatorTest.java @@ -0,0 +1,31 @@ +package com.github.nosamad.dobuild.example; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +import org.junit.jupiter.api.DisplayName; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.CsvSource; + +class CalculatorTests { + + @Test + @DisplayName("1 + 1 = 2") + void addsTwoNumbers() { + Calculator calculator = new Calculator(); + assertEquals(2, calculator.add(1, 1), "1 + 1 should equal 2"); + } + + @ParameterizedTest(name = "{0} + {1} = {2}") + @CsvSource({ + "0, 1, 1", + "1, 2, 3", + "49, 51, 100", + "1, 100, 101" + }) + void add(int first, int second, int expectedResult) { + Calculator calculator = new Calculator(); + assertEquals(expectedResult, calculator.add(first, second), + () -> first + " + " + second + " should equal " + expectedResult); + } +} diff --git a/generic.mk b/generic.mk new file mode 100644 index 0000000..cc7b912 --- /dev/null +++ b/generic.mk @@ -0,0 +1,406 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +current_makefile := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)) + +# Guard from user inclusion +ifndef standardrules_include_guard + $(error $(current_makefile) is not intended to be included directly - include standardrules.mk instead) +endif + +ifndef generic_include_guard +generic_include_guard := 1 + +ifndef defaults_include_guard + include $(patsubst %/,%,$(dir $(current_makefile)))/defaults.mk +endif + +####################################################################################################################### +# Overridable generic defaults + +DOBUILD_GENERIC_ADAPTER ?= generic + +####################################################################################################################### +# Overridable generic macros, hooks to customize target default values + +# hook called to retrieve the target generic extension directory, may return an empty value +# $(call dobuild_generic_extdir,target-name) +dobuild_generic_extdir ?= + +# hook called to retrieve the target generic adapter name, may return an empty value +# $(call dobuild_generic_adapter,target-name) +dobuild_generic_adapter ?= + +# hook called to retrieve the target generic prerequisites, may return an empty list in which case the default generic +# prerequisites are used +# $(call dobuild_generic_prerequisites,target-name) +dobuild_generic_prerequisites ?= + +# hook called to retrieve the target generic order only prerequisites, may return an empty list in which case the +# default generic order only prerequisites are used +# $(call dobuild_generic_orderonly_prerequisites,target-name) +dobuild_generic_orderonly_prerequisites ?= + +# hook called to retrieve the target generic perpare step, may return an empty value +# $(call dobuild_generic_prepare,target-name) +dobuild_generic_prepare ?= prepare + +# hook called to retrieve the target generic assemble step +# $(call dobuild_generic_assemble,target-name) +dobuild_generic_assemble ?= assemble + +# hook called to retrieve the target generic save artifacts step, may return an empty value +# $(call dobuild_generic_saveartifacts,target-name) +dobuild_generic_saveartifacts ?= save-artifacts + +# hook called to retrieve the target generic lint step, may return an empty value +# $(call dobuild_generic_lint,target-name) +dobuild_generic_lint ?= lint + +# hook called to retrieve the target generic check step, may return an empty value +# $(call dobuild_generic_check,target-name) +dobuild_generic_check ?= check + +# hook called to retrieve the target generic memcheck step, may return an empty value +# $(call dobuild_generic_memcheck,target-name) +dobuild_generic_memcheck ?= check-memcheck + +# hook called to retrieve the target generic package step, may return an empty value +# $(call dobuild_generic_package,target-name) +dobuild_generic_package ?= package + +# hook called to retrieve the target generic install step, may return an empty value +# $(call dobuild_generic_install,target-name) +dobuild_generic_install ?= package-install + +# hook called to retrieve the target generic delegate step +# $(call dobuild_generic_delegate,target-name) +dobuild_generic_delegate ?= delegate + +# hook called to retrieve the target generic build testing option, +# may return an empty value in which case BUILD_TESTING is used +# $(call dobuild_generic_buildtesting,target-name) +dobuild_generic_buildtesting ?= + +####################################################################################################################### +# Generic macros + +# replaces template values of target field with target properties and resolves result against extension search +# directories, returning the first existing resolved path +# $(call generic_resolve_ext,target-name,field) +generic_resolve_ext = $(firstword $(call target_subst_and_resolve,$1,$2,$(cache.$1.ext_search_paths))) + +# retrieves the target generic default prerequisites +# $(call generic_default_prerequisites,target-name) +generic_default_prerequisites = \ + $(call image_buildtarget,$1) \ + $(OUTDIR)/$1/DoBuildFiles/generic.properties + +# retrieves the target generic default extension directory +# $(call generic_default_extdir,target-name) +generic_default_extdir = $(addprefix $(PROJECTDIR)/dobuild-extensions/,$(cache.$1.adapter)) + +# creates a generic run rule command +# $(call generic_run_cmd,target-name,command[,args],alternative-command[,output-command]) +generic_run_cmd = $(call run_cmd,$1,$(or $(strip $(notdir $(wildcard $2))),$(strip $4)),$3,$5) + +# create a generic perpare command +# $(call generic_prepare_cmd,target-name) +generic_prepare_cmd = $(call generic_run_cmd,$1,$(cache.$1.generic_prepare),,true,prepare) + +# create a generic assemble command +# $(call generic_assemble_cmd,target-name) +generic_assemble_cmd = $(call generic_run_cmd,$1,$(cache.$1.generic_assemble),,$(call dobuild_generic_assemble,$1),assemble) + +# create a generic save artifacts command +# $(call generic_saveartifacts_cmd,target-name) +generic_saveartifacts_cmd = { $(call generic_run_cmd,$1,$(cache.$1.generic_saveartifacts),,tar -cf - -T /dev/null,save-artifacts) | tar xvf - -C '$(OUTDIR)/$1/DoBuildFiles'; } + +# create a generic lint command +# $(call generic_lint_cmd,target-name) +generic_lint_cmd = $(call generic_run_cmd,$1,$(cache.$1.generic_lint),,true,lint) + +# create a generic check command +# $(call generic_check_cmd,target-name) +generic_check_cmd = $(call generic_run_cmd,$1,$(cache.$1.generic_check),,true,check) + +# create a generic memcheck command +# $(call generic_memcheck_cmd,target-name) +generic_memcheck_cmd = $(call generic_run_cmd,$1,$(cache.$1.generic_memcheck),,true,memcheck) + +# create a generic package command +# $(call generic_package_cmd,target-name) +generic_package_cmd = $(call generic_run_cmd,$1,$(cache.$1.generic_package),,true,package) + +# create a generic install command +# $(call generic_install_cmd,target-name) +generic_install_cmd = $(call generic_run_cmd,$1,$(cache.$1.generic_install),$(generic_destdir),true,install) + +# create a generic clean command +# $(call generic_clean_cmd,target-name) +generic_clean_cmd = rm -rf '$(OUTDIR)/$1' + +# create a generic delegate command +# $(call generic_delegate_cmd,target-name,delegate-target-name) +generic_delegate_cmd = $(call generic_run_cmd,$1,$(cache.$1.generic_delegate),$2,$(call dobuild_generic_delegate,$1),delegate) + +# creates a generic properties generation cmd +# $(call generic_props_cmd,target-name,input-file) +generic_props_cmd = sed \ + -e 's!%TARGET%!$(call escape,$1,!)!g' \ + -e 's!%DOBUILDDIR%!$(call escape,$(generic_dobuilddir),!)!g' \ + -e 's!%PROJECTDIR%!$(call escape,$(generic_projectdir),!)!g' \ + -e 's!%HOSTMARCH%!$(call escape,$(call target_host_arch,$1),!)!g' \ + -e 's!%MARCH%!$(call escape,$(call target_arch_sub,$1),!)!g' \ + -e 's!%DISTRIB%!$(call escape,$(call target_distrib,$1),!)!g' \ + -e 's!%DISTRIB_ID%!$(call escape,$(call target_distrib_id,$1),!)!g' \ + -e 's!%DISTRIB_VERSION%!$(call escape,$(call target_distrib_version,$1),!)!g' \ + -e 's!%SYS%!$(call escape,$(call target_sys,$1),!)!g' \ + -e 's!%ABI%!$(call escape,$(call target_abi,$1),!)!g' \ + -e 's!%ID%!$(call escape,$(call target_id,$1),!)!g' \ + -e 's!%VARIANT%!$(call escape,$(call target_variant,$1),!)!g' \ + -e 's!%BUILD_TESTING%!$(call escape,$(cache.$1.generic_buildtesting),!)!g' \ + $2 + +# creates a generic properties generation rule +# $(call generic_props_rule,target-name) +generic_props_rule = \ + $$(OUTDIR)/$1/DoBuildFiles/generic.properties: $$(DOBUILDDIR)/assets/templates/properties.template $$(MAKEFILE_LIST); \ + $$(SILENT)mkdir -p $$(dir $$@); $$(call generic_props_cmd,$1,$$<) > $$@ + +# creates a generic prepare rule +# $(call generic_prepare_rule,target-name) +generic_prepare_rule = \ + $$(OUTDIR)/$1/DoBuildFiles/prepare.stage: $$(cache.$1.generic_prepare) $$(cache.$1.generic_prerequisites) | $$(cache.$1.generic_orderonly_prerequisites); \ + $$(SILENT)$$(call generic_prepare_cmd,$1) && touch $$@ + +# creates a generic assemble rule +# $(call generic_assemble_rule,target-name) +generic_assemble_rule = \ + build/$1 $$(OUTDIR)/$1/DoBuildFiles/assemble.stage: $$(cache.$1.generic_assemble) $$(OUTDIR)/$1/DoBuildFiles/prepare.stage | $$(OUTDIR)/$1/DoBuildFiles/save-artifacts.stage; \ + $$(SILENT)$$(call generic_assemble_cmd,$1) && touch $$(OUTDIR)/$1/DoBuildFiles/assemble.stage + +# creates a generic save artifacts rule +# $(call generic_saveartifacts_rule,target-name) +generic_saveartifacts_rule = \ + $$(OUTDIR)/$1/DoBuildFiles/save-artifacts.stage: $$(cache.$1.generic_saveartifacts) $$(OUTDIR)/$1/DoBuildFiles/prepare.stage; \ + $$(SILENT)$$(call generic_saveartifacts_cmd,$1) && touch $$@ + +# creates a generic lint rule +# $(call generic_lint_rule,target-name) +generic_lint_rule = \ + lint/$1 $$(OUTDIR)/$1/DoBuildFiles/lint.stage: $$(cache.$1.generic_lint) $$(OUTDIR)/$1/DoBuildFiles/assemble.stage; \ + $$(SILENT)$$(call generic_lint_cmd,$1) && touch $$(OUTDIR)/$1/DoBuildFiles/lint.stage + +# creates a generic check rule +# $(call generic_check_rule,target-name) +generic_check_rule = \ + check/$1 $$(OUTDIR)/$1/DoBuildFiles/check.stage: $$(cache.$1.generic_check) $$(OUTDIR)/$1/DoBuildFiles/assemble.stage; \ + $$(SILENT)$$(call generic_check_cmd,$1) && touch $$(OUTDIR)/$1/DoBuildFiles/check.stage + +# creates a generic memcheck rule +# $(call generic_memcheck_rule,target-name) +generic_memcheck_rule = \ + memcheck/$1 $$(OUTDIR)/$1/DoBuildFiles/memcheck.stage: $$(cache.$1.generic_memcheck) $$(OUTDIR)/$1/DoBuildFiles/assemble.stage; \ + $$(SILENT)$$(call generic_memcheck_cmd,$1) && touch $$(OUTDIR)/$1/DoBuildFiles/memcheck.stage + +# creates a generic package rule +# $(call generic_package_rule,target-name) +generic_package_rule = \ + package/$1 $$(OUTDIR)/$1/DoBuildFiles/package.stage: $$(cache.$1.generic_package) $$(OUTDIR)/$1/DoBuildFiles/assemble.stage; \ + $$(SILENT)$$(call generic_package_cmd,$1) && touch $$(OUTDIR)/$1/DoBuildFiles/package.stage + +# creates a generic install rule +# $(call generic_install_rule,target-name) +generic_install_rule = \ + install/$1: $$(wildcard $$(cache.$1.generic_install)) $$(OUTDIR)/$1/DoBuildFiles/assemble.stage; \ + $$(SILENT)$$(call generic_install_cmd,$1) + +# creates a generic clean rule +# $(call generic_clean_rule,target-name) +generic_clean_rule = \ + clean/$1: ; \ + $$(SILENT)-$$(call generic_clean_cmd,$1) + +# creates a generic delegate rule +# $(call generic_delegate_rule,adapter,target-name,delegate-target-name) +generic_delegate_rule = \ + $1/$2/$3: $$(OUTDIR)/$2/DoBuildFiles/prepare.stage; \ + $$(SILENT)$$(call generic_delegate_cmd,$2,$3) + +# creates a generic delegate pattern rule +# $(call generic_delegate_pattern_rule,adapter,target-name) +generic_delegate_pattern_rule = \ + $1/$2/%: $$(OUTDIR)/$2/DoBuildFiles/prepare.stage; \ + $$(SILENT)$$(call generic_delegate_cmd,$2,$$*) + +# creates a generic delegate makefile generation command, assuming target is the makefile to create and +# first prerequisite a file containing the delegate target names +# $(call generic_delegate_makefile_rule_cmd,target-name) +generic_delegate_makefile_rule_cmd = \ + { \ + echo '$(\\\#) generated file - do not edit!!!'; \ + echo; \ + ID='$(call id,$@)'; \ + echo "ifndef $${ID}_include_guard"; \ + echo "$${ID}_include_guard := 1"; \ + echo; \ + $(foreach target,$(shell cat '$<' 2>/dev/null),\ + echo '$(call generic_delegate_rule,$$(cache.$1.adapter),$1,$(target))'; \ + echo; \ + ) \ + echo 'endif'; \ + echo; \ + } > $@ + +# creates a generic delegate makefile generation rule +# $(call generic_delegate_makefile_rule,target-name) +generic_delegate_makefile_rule = \ + $$(OUTDIR)/$1/DoBuildFiles/generic_delegaterules.mk: $$(wildcard $$(OUTDIR)/$1/DoBuildFiles/targets.txt) | $$(OUTDIR)/$1/DoBuildFiles; \ + $$(SILENT)$$(call generic_delegate_makefile_rule_cmd,$1) + +####################################################################################################################### +# Generic rule target configuration + +GENERIC_ADAPTER = $(call memorize,GENERIC_ADAPTER,$(DOBUILD_GENERIC_ADAPTER)) + +generic_projectdir = $(or $(container_projectdir),$(PROJECTDIR)) +generic_dobuilddir = $(or $(container_dobuilddir),$(DOBUILDIR)) +generic_destdir = $(or $(container_destdir),$(DESTDIR)) + +generic_defaulttarget = $(if $(SKIP_DEFAULTTARGET),,$(DEFAULTTARGET)) +generic_targets = $(filter $(generic_defaulttarget),$(GENERIC_TARGETS)) +generic_selected_targets = $(if $(SKIP_DEFAULTTARGET),$(GENERIC_TARGETS),$(generic_targets)) +generic_active_targets = $(call memorize,generic_active_targets,$(call target_filter,$(FILTER),$(generic_selected_targets),$(EXCLUDEFILTER))) + +generic_prepare_targets = $(addsuffix /DoBuildFiles/prepare.stage,$(addprefix $(OUTDIR)/,$(generic_active_targets))) +generic_build_targets = $(addprefix build/,$(generic_active_targets)) +generic_clean_targets = $(addprefix clean/,$(generic_active_targets)) +generic_check_targets = $(addprefix check/,$(generic_active_targets)) +generic_memcheck_targets = $(addprefix memcheck/,$(call target_filter,$(MEMCHECKFILTER),$(generic_active_targets),)) +generic_lint_targets = $(addprefix lint/,$(generic_active_targets)) +generic_dist_targets = $(addprefix package/,$(generic_active_targets)) +generic_install_targets = $(addprefix install/,$(generic_active_targets)) +generic_rule_targets = $(addsuffix /genericrules.mk,$(OUTDIR)) +generic_outdirs = $(addprefix $(OUTDIR)/,$(generic_active_targets)) + +project_targets += $(GENERIC_TARGETS) + +PREPARE_TARGETS += $(generic_prepare_targets) +BUILD_TARGETS += $(generic_build_targets) +LINT_TARGETS += $(generic_lint_targets) +CHECK_TARGETS += $(generic_check_targets) +MEMCHECK_TARGETS += $(generic_memcheck_targets) +CLEAN_TARGETS += $(generic_clean_targets) +INSTALL_TARGETS += $(generic_install_targets) +DIST_TARGETS += $(generic_dist_targets) +RULE_TARGETS += $(generic_rule_targets) +TARGETS += $(generic_active_targets) +OUTDIRS += $(generic_outdirs) + +####################################################################################################################### +# Makefile dependencies + +MAKEFILE_DEPS += touch +MAKEFILE_DEPS += echo +MAKEFILE_DEPS += sed +MAKEFILE_DEPS += tar + +####################################################################################################################### +# Generic rules + +$(generic_rule_targets): + $(SILENT) \ + { \ + echo '$(\#) generated file - do not edit!!!'; \ + echo; \ + ID='$(call id,$@)'; \ + echo "ifndef $${ID}_include_guard"; \ + echo "$${ID}_include_guard := 1"; \ + echo; \ + $(foreach target,$(GENERIC_TARGETS),\ + echo '$(\#)$(\#) BEGIN of generic $(target) configuration'; \ + echo; \ + echo '$(\#)$(\#)$(\#) defaults'; \ + echo '$(target) ?= $$(call memorize,$(target),$(call target_properties_parse,$(target)))'; \ + echo '$(target).env_path = $$(call env_default_path,$$1)'; \ + echo '$(target).adapter = $$(call $$1.generic_adapter,$$1)'; \ + echo '$(target).extdir = $$(call $$1.generic_extdir,$$1)'; \ + echo '$(target).generic_adapter ?= $$(call dobuild_generic_adapter,$$1)'; \ + echo '$(target).generic_extdir ?= $$(call dobuild_generic_extdir,$$1)'; \ + echo '$(target).generic_prerequisites ?= $$(call dobuild_generic_prerequisites,$$1)'; \ + echo '$(target).generic_orderonly_prerequisites ?= $$(call dobuild_generic_orderonly_prerequisites,$$1)'; \ + echo '$(target).generic_prepare ?= $$(call dobuild_generic_prepare,$$1)'; \ + echo '$(target).generic_assemble ?= $$(call dobuild_generic_assemble,$$1)'; \ + echo '$(target).generic_saveartifacts ?= $$(call dobuild_generic_saveartifacts,$$1)'; \ + echo '$(target).generic_lint ?= $$(call dobuild_generic_lint,$$1)'; \ + echo '$(target).generic_check ?= $$(call dobuild_generic_check,$$1)'; \ + echo '$(target).generic_memcheck ?= $$(call dobuild_generic_memcheck,$$1)'; \ + echo '$(target).generic_package ?= $$(call dobuild_generic_package,$$1)'; \ + echo '$(target).generic_install ?= $$(call dobuild_generic_install,$$1)'; \ + echo '$(target).generic_delegate ?= $$(call dobuild_generic_delegate,$$1)'; \ + echo '$(target).generic_buildtesting ?= $$(call dobuild_generic_buildtesting,$$1)'; \ + echo; \ + echo '$(\#)$(\#)$(\#) cached values'; \ + echo 'cache.$(target).ext_search_paths = $$(call memorize,cache.$(target).ext_search_paths,$$(call split_s,$$(call env_host_path,$(target)),:))'; \ + echo 'cache.$(target).env_path = $$(call memorize,cache.$(target).env_path,$$(call $(target).env_path,$(target)))'; \ + echo 'cache.$(target).adapter = $$(call memorize,cache.$(target).adapter,$$(call target_get_and_subst,$(target),adapter,$(GENERIC_ADAPTER)))'; \ + echo 'cache.$(target).extdir = $$(call memorize,cache.$(target).extdir,$$(call target_get_and_subst,$(target),extdir,$$(call generic_default_extdir,$(target))))'; \ + echo 'cache.$(target).generic_prerequisites = $$(call memorize,cache.$(target).generic_prerequisites,$$(call target_get_and_subst,$(target),generic_prerequisites,) $$(call generic_default_prerequisites,$(target)))'; \ + echo 'cache.$(target).generic_orderonly_prerequisites = $$(call memorize,cache.$(target).generic_orderonly_prerequisites,$$(call target_get_and_subst,$(target),generic_orderonly_prerequisites,))'; \ + echo 'cache.$(target).generic_prepare = $$(call memorize,cache.$(target).generic_prepare,$$(call generic_resolve_ext,$(target),generic_prepare))'; \ + echo 'cache.$(target).generic_assemble = $$(call memorize,cache.$(target).generic_assemble,$$(call generic_resolve_ext,$(target),generic_assemble))'; \ + echo 'cache.$(target).generic_saveartifacts = $$(call memorize,cache.$(target).generic_saveartifacts,$$(call generic_resolve_ext,$(target),generic_saveartifacts))'; \ + echo 'cache.$(target).generic_lint = $$(call memorize,cache.$(target).generic_lint,$$(call generic_resolve_ext,$(target),generic_lint))'; \ + echo 'cache.$(target).generic_check = $$(call memorize,cache.$(target).generic_check,$$(call generic_resolve_ext,$(target),generic_check))'; \ + echo 'cache.$(target).generic_memcheck = $$(call memorize,cache.$(target).generic_memcheck,$$(call generic_resolve_ext,$(target),generic_memcheck))'; \ + echo 'cache.$(target).generic_package = $$(call memorize,cache.$(target).generic_package,$$(call generic_resolve_ext,$(target),generic_package))'; \ + echo 'cache.$(target).generic_install = $$(call memorize,cache.$(target).generic_install,$$(call generic_resolve_ext,$(target),generic_install))'; \ + echo 'cache.$(target).generic_delegate = $$(call memorize,cache.$(target).generic_delegate,$$(call generic_resolve_ext,$(target),generic_delegate))'; \ + echo 'cache.$(target).generic_buildtesting = $$(call memorize,cache.$(target).generic_buildtesting,$$(or $$(call $(target).generic_buildtesting,$(target)),$$(BUILD_TESTING)))'; \ + echo; \ + echo '$(\#)$(\#)$(\#) rules'; \ + echo '$(call generic_props_rule,$(target))'; \ + echo; \ + echo '$(call generic_prepare_rule,$(target))'; \ + echo; \ + echo '$(call generic_assemble_rule,$(target))'; \ + echo; \ + echo '$(call generic_saveartifacts_rule,$(target))'; \ + echo; \ + echo '$(call generic_lint_rule,$(target))'; \ + echo; \ + echo '$(call generic_check_rule,$(target))'; \ + echo; \ + echo '$(call generic_memcheck_rule,$(target))'; \ + echo; \ + echo '$(call generic_package_rule,$(target))'; \ + echo; \ + echo '$(call generic_install_rule,$(target))'; \ + echo; \ + echo '$(call generic_clean_rule,$(target))'; \ + echo; \ + echo '$(call generic_delegate_makefile_rule,$(target))'; \ + echo; \ + echo '$$(cache.$(target).adapter)/$(target): build/$(target)'; \ + echo; \ + echo '-include $$(OUTDIR)/$(target)/DoBuildFiles/generic_delegaterules.mk'; \ + echo; \ + echo '$(call generic_delegate_pattern_rule,$$(cache.$(target).adapter),$(target))'; \ + echo; \ + echo '$(\#)$(\#) END of generic $(target) configuration'; \ + echo; \ + ) \ + echo 'endif'; \ + echo; \ + } > $@ + +endif + diff --git a/gradle.mk b/gradle.mk new file mode 100644 index 0000000..3f3fca9 --- /dev/null +++ b/gradle.mk @@ -0,0 +1,170 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +ifndef gradle_include_guard +gradle_include_guard := 1 + +current_makefile := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)) + +ifndef defaults_include_guard + include $(patsubst %/,%,$(dir $(current_makefile)))/defaults.mk +endif + +####################################################################################################################### +# Overridable Gradle defaults + +####################################################################################################################### +# Overridable gradle macros, hooks to customize target default values + +# hook called to retrieve the target gradle extension directory, +# may return an empty value +# $(call dobuild_gradle_extdir,target-name) +dobuild_gradle_extdir ?= $(call dobuild_generic_extdir,$1) + +# hook called to retrieve the target gradle adapter name +# $(call dobuild_gradle_adapter,target-name) +dobuild_gradle_adapter ?= gradle + +# hook called to retrieve the target gradle prerequisites, +# may return an empty list +# $(call dobuild_gradle_prerequisites,target-name) +dobuild_gradle_prerequisites ?= $(call dobuild_generic_prerequisites,$1) + +# hook called to retrieve the target gradle order-only prerequisites, +# may return an empty list +# $(call dobuild_gradle_orderonly_prerequisites,target-name) +dobuild_gradle_orderonly_prerequisites ?= $(call dobuild_generic_orderonly_prerequisites,$1) + +# hook called to retrieve the target gradle prepare step, +# may return an empty value +# $(call dobuild_gradle_prepare,target-name) +dobuild_gradle_prepare ?= $(call dobuild_generic_prepare,$1) + +# hook called to retrieve the target gradle assemble step +# $(call dobuild_gradle_assemble,target-name) +dobuild_gradle_assemble ?= $(call dobuild_generic_assemble,$1) + +# hook called to retrieve the target gradle save artifacts step, +# may return an empty value +# $(call dobuild_gradle_saveartifacts,target-name) +dobuild_gradle_saveartifacts ?= $(call dobuild_generic_saveartifacts,$1) + +# hook called to retrieve the target gradle lint step, +# may return an empty value +# $(call dobuild_gradle_lint,target-name) +dobuild_gradle_lint ?= $(call dobuild_generic_lint,$1) + +# hook called to retrieve the target gradle check step, +# may return an empty value +# $(call dobuild_gradle_check,target-name) +dobuild_gradle_check ?= $(call dobuild_generic_check,$1) + +# hook called to retrieve the target gradle memcheck step, +# may return an empty value +# $(call dobuild_gradle_memcheck,target-name) +dobuild_gradle_memcheck ?= $(call dobuild_generic_memcheck,$1) + +# hook called to retrieve the target gradle package step, +# may return an empty value +# $(call dobuild_gradle_package,target-name) +dobuild_gradle_package ?= $(call dobuild_generic_package,$1) + +# hook called to retrieve the target gradle install step, +# may return an empty value +# $(call dobuild_gradle_install,target-name) +dobuild_gradle_install ?= $(call dobuild_generic_install,$1) + +# hook called to retrieve the target gradle delegate step +# $(call dobuild_gradle_delegate,target-name) +dobuild_gradle_delegate ?= $(call dobuild_generic_delegate,$1) + +# hook called to retrieve the target gradle build testing option, +# may return an empty value in which case BUILD_TESTING is used +# $(call dobuild_gradle_buildtesting,target-name) +dobuild_gradle_buildtesting ?= $(call dobuild_generic_buildtesting,$1) + +####################################################################################################################### +# Gradle macros + +# retrieves the target generic default prerequisites +# $(call gradle_default_prerequisites,target-name) +gradle_default_prerequisites = \ + $(wildcard $(PROJECTDIR)/*.gradle) + +####################################################################################################################### +# Gradle rule target configuration + +gradle_rule_targets = $(addsuffix /gradlerules.mk,$(OUTDIR)) + +GENERIC_TARGETS += $(GRADLE_TARGETS) +RULE_TARGETS += $(gradle_rule_targets) + +####################################################################################################################### +# Makefile dependencies + +####################################################################################################################### +# CMake rules + +$(gradle_rule_targets): + $(SILENT) \ + { \ + echo '$(\#) generated file - do not edit!!!'; \ + echo; \ + ID='$(call id,$@)'; \ + echo "ifndef $${ID}_include_guard"; \ + echo "$${ID}_include_guard := 1"; \ + echo; \ + $(foreach target,$(GRADLE_TARGETS),\ + echo '$(\#)$(\#) BEGIN of gradle $(target) configuration'; \ + echo; \ + echo '$(\#)$(\#)$(\#) defaults'; \ + echo '$(target) ?= $$(call memorize,$(target),$(call target_properties_parse,$(target)))'; \ + echo '$(target).generic_adapter = $$(call $$1.gradle_adapter,$$1)'; \ + echo '$(target).generic_extdir = $$(call $$1.gradle_extdir,$$1)'; \ + echo '$(target).generic_prerequisites = $$(call $$1.gradle_prerequisites,$$1) $$(call gradle_default_prerequisites,$$1)'; \ + echo '$(target).generic_orderonly_prerequisites ?= $$(call $$1.gradle_orderonly_prerequisites,$$1)'; \ + echo '$(target).generic_prepare = $$(call $$1.gradle_prepare,$$1)'; \ + echo '$(target).generic_assemble = $$(call $$1.gradle_assemble,$$1)'; \ + echo '$(target).generic_saveartifacts = $$(call $$1.gradle_saveartifacts,$$1)'; \ + echo '$(target).generic_lint = $$(call $$1.gradle_lint,$$1)'; \ + echo '$(target).generic_check = $$(call $$1.gradle_check,$$1)'; \ + echo '$(target).generic_memcheck = $$(call $$1.gradle_memcheck,$$1)'; \ + echo '$(target).generic_package = $$(call $$1.gradle_package,$$1)'; \ + echo '$(target).generic_install = $$(call $$1.gradle_install,$$1)'; \ + echo '$(target).generic_delegate = $$(call $$1.gradle_delegate,$$1)'; \ + echo '$(target).generic_buildtesting = $$(call $$1.gradle_buildtesting,$$1)'; \ + echo '$(target).gradle_adapter ?= $$(call dobuild_gradle_adapter,$$1)'; \ + echo '$(target).gradle_extdir ?= $$(call dobuild_gradle_extdir,$$1)'; \ + echo '$(target).gradle_prerequisites ?= $$(call dobuild_gradle_prerequisites,$$1)'; \ + echo '$(target).gradle_orderonly_prerequisites ?= $$(call dobuild_gradle_orderonly_prerequisites,$$1)'; \ + echo '$(target).gradle_prepare ?= $$(call dobuild_gradle_prepare,$$1)'; \ + echo '$(target).gradle_assemble ?= $$(call dobuild_gradle_assemble,$$1)'; \ + echo '$(target).gradle_saveartifacts ?= $$(call dobuild_gradle_saveartifacts,$$1)'; \ + echo '$(target).gradle_lint ?= $$(call dobuild_gradle_lint,$$1)'; \ + echo '$(target).gradle_check ?= $$(call dobuild_gradle_check,$$1)'; \ + echo '$(target).gradle_memcheck ?= $$(call dobuild_gradle_memcheck,$$1)'; \ + echo '$(target).gradle_package ?= $$(call dobuild_gradle_package,$$1)'; \ + echo '$(target).gradle_install ?= $$(call dobuild_gradle_install,$$1)'; \ + echo '$(target).gradle_delegate ?= $$(call dobuild_gradle_delegate,$$1)'; \ + echo '$(target).gradle_buildtesting ?= $$(call dobuild_gradle_buildtesting,$$1)'; \ + echo; \ + echo '$(\#)$(\#)$(\#) cached values'; \ + echo; \ + echo '$(\#)$(\#)$(\#) rules'; \ + echo; \ + echo '$(\#)$(\#) END of gradle $(target) configuration'; \ + echo; \ + ) \ + echo 'endif'; \ + echo; \ + } > $@ + +endif + diff --git a/run_tests b/run_tests new file mode 100755 index 0000000..a760ccc --- /dev/null +++ b/run_tests @@ -0,0 +1,63 @@ +#!/bin/sh +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +set -e + +enabled() { + { [ "$1" -ne 0 ] || [ "$1" = 'true' ]; } 2>/dev/null +} + +physical_pwd() { + pwd -P 2>/dev/null || pwd +} + +try_canonicalize() { + readlink -f "$@" 2>/dev/null || realpath "$@" +} + +canonicalize() { + if ! try_canonicalize "$1" 2>/dev/null; then + echo "$(cd "$(dirname "$1")" && physical_pwd)/$(basename "$1")" + fi +} + +scriptdir() { + dirname "$(canonicalize "${BASH_SOURCE:-$1}")" +} + +if [ $# -eq 0 ]; then + set -- 'bats' 'tests/' +fi + +DOBUILDDIR="${DOBUILDDIR:-"$(scriptdir "$0")"}" +PATH="${DOBUILDDIR}/bin:$PATH" +ENABLE_BUILD="${ENABLE_BUILD:-0}" +PROJECTDIR="${DOBUILD_COMPOSEPROJECTDIR:-"$PWD"}" +DOBUILD_HOSTCONTAINER="${DOBUILD_HOSTCONTAINER:-"$(get_container_id.sh)"}" || true +COMPOSEENV_PROJECTPATH="$(canonicalize "${COMPOSEENV_PROJECTPATH:-$PROJECTDIR}")" + +export DOBUILDDIR +export COMPOSEENV_VOLUMESFROM="${COMPOSEENV_VOLUMESFROM:-$DOBUILD_HOSTCONTAINER}" +export COMPOSEENV_PROJECTPATH + +if [ -n "$COMPOSEENV_VOLUMESFROM" ]; then + DIND_VOLUME_METHOD='dind-volumes_from.yml' +else + DIND_VOLUME_METHOD='dind-bind_mount.yml' +fi + +set -- -f docker-compose.yml -f "tests/runners/$DIND_VOLUME_METHOD" run --rm check "$@" + +if enabled "${ENABLE_BUILD}"; then + docker_compose build 0<&- || exit $? +fi + +exec docker_compose "$@" diff --git a/standardrules.mk b/standardrules.mk new file mode 100644 index 0000000..ca8b76c --- /dev/null +++ b/standardrules.mk @@ -0,0 +1,148 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +ifndef standardrules_include_guard +standardrules_include_guard := 1 + +.PHONY: all +all: + +current_makefile := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)) +include $(patsubst %/,%,$(dir $(current_makefile)))/generic.mk + +####################################################################################################################### +# Standard macros + +# md5sum command +# $(call md5sum,file,checksum) +md5sum = { \ + if [ -n "$2" ]; then \ + sed -e 's!%MD5%!$2!g' -e 's!%FILE%!$(call escape,$1,!)!g' $(DOBUILDDIR)/assets/templates/md5sum.txt.template > '$1.md5'; \ + md5sum -c '$1.md5'; \ + else \ + echo 'warning:$1: no md5 skipping verification' 1>&2; \ + fi; \ +} + +# curl command +# $(call curl,file,url) +$(SKIP_CURL)curl = { \ + $(call echo_if_silent_cmd,curl -fSL $(CURLFLAGS) -o '$1' '$2') \ + && curl -fSL $(CURLFLAGS) -o '$1' '$2' \ + && $(call md5sum,$1,$3); \ +} +curl ?= echo 'warning:$1: download skipped (SKIP_CURL=$(SKIP_CURL))' + +####################################################################################################################### +# Makefile dependencies + +MAKEFILE_DEPS += rm +MAKEFILE_DEPS += mkdir +MAKEFILE_DEPS += cat +$(SKIP_MD5SUM)MAKEFILE_DEPS += md5sum +$(SKIP_CURL)MAKEFILE_DEPS += curl + +####################################################################################################################### +# Standard rule target configuration + +OUTDIRS := $(sort $(OUTDIRS)) + +standardrules_unsatisfied_deps = $(call filter_not_found,$(sort $(MAKEFILE_DEPS))) +standardrules_rule_targets_deps := $(filter-out $(RULE_TARGETS) $(addprefix $(PROJECTDIR),$(RULE_TARGETS)),$(MAKEFILE_LIST)) + +-include $(RULE_TARGETS) + +JOBS = $(call memorize,JOBS,$(words $(or $(sort $(TARGETS)),_))) + +# uses golden cut to manage relation between intern and external parallelization (az = phi^z) +# phi^-3 ~= 0,236, phi^-2 ~= 0,382, phi^-1 ~= 0,618 +EXTERNPARALLEL = $(call memorize,EXTERNPARALLEL,$(call min,$(call bc,((($(JOBSLOTS) - 1) * 382) / 1000) + 1) $(JOBS))) +INTERNPARALLEL = $(call memorize,INTERNPARALLEL,$(call bc,($(JOBSLOTS) - 1) / $(call max,$(EXTERNPARALLEL) 1) + 1)) + +ifeq ($(filter 1,$(EXTERNPARALLEL)),) + $(SKIP_EXTERNSYNC)GNUMAKEFLAGS += --output-sync + MAKEFLAGS += $(addprefix -j,$(EXTERNPARALLEL)) +endif + +####################################################################################################################### +# Standard rule assertions + +ASSERTIONS += $(call assert,$(call not,$(standardrules_unsatisfied_deps)),Required commands $(standardrules_unsatisfied_deps) \ +not found; install appropriate packages e.g. docker-ce$(,) busybox and curl) + +# evaluate only once, while make restarts because of generated makefiles +ifeq ($(MAKE_RESTARTS),) + ASSERTIONS := $(ASSERTIONS) + EXPECTATIONS := $(EXPECTATIONS) +endif + +####################################################################################################################### +# Shell exports + +export DOBUILD_HOSTCONTAINER := $(HOST_CONTAINER) +export DOBUILD_VERBOSE := $(VERBOSE) +export DOBUILD_BUILDVERBOSE := $(BUILDVERBOSE) +export DOBUILD_TESTVERBOSE := $(TESTVERBOSE) +export DOBUILD_NPROC := $(INTERNPARALLEL) +export SOURCE_DATE_EPOCH +export BUILDTIME +export DOCKER +export DOCKER_BUILDKIT +export BUILDKIT_PROGRESS + +####################################################################################################################### +# Standard rules + +.PHONY: all +all: $(BUILD_TARGETS); + +.PHONY: check +check: $(CHECK_TARGETS); + +.PHONY: memcheck +memcheck: $(MEMCHECK_TARGETS); + +.PHONY: lint +lint: $(LINT_TARGETS); + +.PHONY: run +run: $(firstword $(RUN_TARGETS)); + +.PHONY: clean +clean: $(CLEAN_TARGETS); + +.PHONY: prepare +prepare: $(PREPARE_TARGETS); + +.PHONY: dist +dist: $(DIST_TARGETS); + +.PHONY: distclean +distclean: $(DISTCLEAN_TARGETS) + $(SILENT)-$(if $(strip $(OUTDIRS)),rm -rf $(OUTDIRS)) + +.PHONY: install +install: $(INSTALL_TARGETS); + +.PHONY: print-targets +print-targets: + $(SILENT)MAKE=$(MAKE) $(DOBUILDDIR)/bin/parse_make_targets.sh $(MFLAGS) $(addprefix -f ,$(MAKEFILE_LIST)) $(MAKEOVERRIDES) + +.PHONY: debug-print-% +debug-print-%: + @printf '%s\n' '$*:' 1>&2; \ + printf '%s\n' $($*) + +$(RULE_TARGETS): $(standardrules_rule_targets_deps) | $(OUTDIRS) + +$(OUTDIRS): + $(SILENT)mkdir -p $@ + +endif diff --git a/tests/10_get_container_id.bats b/tests/10_get_container_id.bats new file mode 100644 index 0000000..6cccbed --- /dev/null +++ b/tests/10_get_container_id.bats @@ -0,0 +1,95 @@ +#!/usr/bin/env bats +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +load test_helper + +setup() { + BINPATH="$(readlink -f "$BATS_TEST_DIRNAME/../bin")" + PATH="$BINPATH:$PATH" +} + +print_cgroup_container() { + cat <&2 + + set -- "$(cat "$IDDFILE")" "$@" + + debug_trace container_run "$@" +} + +setup() { + PROJECTPATH="$(readlink -f "$BATS_TEST_DIRNAME/..")" + BINPATH="$PROJECTPATH/bin" + PATH="$BINPATH:$PATH" + IDDFILE="$(mktemp --tmpdir="$BATS_TMPDIR" idd_XXXXXXXXXX.txt)" + DOCKERFILE="$BATS_TEST_DIRNAME/get_container_id.dockerfile" + + export PROJECTPATH + export IDDFILE + export DOCKERFILE +} + +teardown() { + "$DOCKER" rmi "$(cat "$IDDFILE")" + rm -f "$IDDFILE" +} + +@test "get_container_id has required dep docker" { + "$DOCKER" --version + echo "docker unreachable!?" + echo "DOCKER_HOST=$DOCKER_HOST" + "$DOCKER" info +} + +@test "get_container_id self-id within container" { + ID="$(build_and_run ./get_container_id.sh)" + [ -n "$ID" ] +} diff --git a/tests/31_shellcheck.bats b/tests/31_shellcheck.bats new file mode 100644 index 0000000..94edd1d --- /dev/null +++ b/tests/31_shellcheck.bats @@ -0,0 +1,37 @@ +#!/usr/bin/env bats +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +load test_helper + +setup() { + PROJECTPATH="$(readlink -f "$BATS_TEST_DIRNAME/..")" + BINPATH="$PROJECTPATH/bin" + PATH="$BINPATH:$PATH" + + SHELLCHECK=( shellcheck --color=never --format=gcc -x ) +} + +@test "shellcheck has required deps" { + "${SHELLCHECK[@]}" --version +} + +@test "shellcheck run_tests" { + "${SHELLCHECK[@]}" "$PROJECTPATH/run_tests" +} + +@test "shellcheck helper scripts" { + find "$BINPATH" -type f -print0 | parallel --keep-order -0 "${SHELLCHECK[@]}" {} +} + +@test "shellcheck test helper scripts" { + find "$BATS_TEST_DIRNAME" -type f -name '*.bash' -print0 | parallel --keep-order -0 "${SHELLCHECK[@]}" {} +} + diff --git a/tests/32_groovy3.bats b/tests/32_groovy3.bats new file mode 100644 index 0000000..b2b82cc --- /dev/null +++ b/tests/32_groovy3.bats @@ -0,0 +1,70 @@ +#!/usr/bin/env bats +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +load test_helper + +setup() { + PROJECTPATH="$(readlink -f "$BATS_TEST_DIRNAME/..")" + BINPATH="$PROJECTPATH/bin" + PATH="$BINPATH:$PATH" + + GROOVY=( groovy3 ) +} + +@test "groovy3 has required deps" { + "${GROOVY[@]}" --version +} + +print_fixture_ymlslurper() { + cat <<- EOF + import groovy.yaml.YamlSlurper + + def configYaml = '''\ + |--- + |application: "Sample App" + |users: + |- name: "mrhaki" + | likes: + | - Groovy + | - Clojure + | - Java + |- name: "Hubert" + | likes: + | - Apples + | - Bananas + |connections: + |- "WS1" + |- "WS2" + ''' + + // Parse the YAML. + def config = new YamlSlurper().parseText(configYaml.stripMargin()) + + assert config.application == 'Sample App' + + assert config.users.size() == 2 + assert config.users[0] == [name: 'mrhaki', likes: ['Groovy', 'Clojure', 'Java']] + assert config.users[1] == [name: 'Hubert', likes: ['Apples', 'Bananas']] + + assert config.connections == ['WS1', 'WS2'] +EOF +} + +@test "groovy3 ymlslurper test" { + run "${GROOVY[@]}" -e "$(print_fixture_ymlslurper)" + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [ -z "${lines[@]}" ] +} + + + diff --git a/tests/40_parse_make_targets.bats b/tests/40_parse_make_targets.bats new file mode 100644 index 0000000..0e516be --- /dev/null +++ b/tests/40_parse_make_targets.bats @@ -0,0 +1,90 @@ +#!/usr/bin/env bats +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +load test_helper + +setup() { + BINPATH="$(readlink -f "$BATS_TEST_DIRNAME/../bin")" + PATH="$BINPATH:$PATH" +} + +@test "parse_make_targets has required dep awk" { + awk 'BEGIN {print "Hello from awk"}' +} + +@test "parse_make_targets has required dep grep" { + echo 'hello' | grep -e 'hello' +} + +@test "parse_make_targets has required dep sort" { + sort --version +} + +@test "parse_make_targets has required dep make" { + make --version +} + +print_simple_makefile() { + cat <&1 + + echo "$output" + [ "$status" -ne 0 ] + [ -n "$output" ] +} + +@test "parse_make_targets should fail when makefile contains errors" { + run parse_make_targets.sh -f -< <(print_error_makefile) + + echo "$output" + [ "$status" -ne 0 ] + [ -n "$output" ] +} + +@test "parse_make_targets should parse simple makefile from stdin" { + run parse_make_targets.sh -f -< <(print_simple_makefile) + + echo "$output" + [ "$status" -eq 0 ] + [ "${lines[0]}" = 'all' ] + [ "${lines[1]}" = 'clean' ] + [ "${lines[2]}" = 'default' ] + [ "${#lines[@]}" -eq 3 ] +} + +@test "parse_make_targets should parse a more advanced makefile" { + run parse_make_targets.sh -C "$BATS_TEST_DIRNAME/fixtures/make-gtest-example" + + echo "$output" + [ "$status" -eq 0 ] + [ "${lines[0]}" = 'all' ] + [ "${lines[1]}" = 'alltests' ] + [ "${lines[2]}" = 'clean' ] + [ "${#lines[@]}" -eq 3 ] +} + + diff --git a/tests/40_parse_target_properties.bats b/tests/40_parse_target_properties.bats new file mode 100644 index 0000000..8b7ebe1 --- /dev/null +++ b/tests/40_parse_target_properties.bats @@ -0,0 +1,230 @@ +#!/usr/bin/env bats +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +load test_helper + +setup() { + BINPATH="$(readlink -f "$BATS_TEST_DIRNAME/../bin")" + PATH="$BINPATH:$PATH" + HOST_ARCH="$(uname -m)" +} + +@test "parse_target_properties.sh has required deps" { + sed --version + paste --version + uname --version + cat --version +} + +@test "parse_target_properties.sh fails when called without target name" { + run parse_target_properties.sh + + echo "$output" + [ "$status" -ne 0 ] + [ -n "$output" ] +} + +@test "parse_target_properties.sh should parse shortest target name" { + mapfile -t PROPERTIES < <( parse_target_properties.sh app-id ) + + [ "${PROPERTIES[0]}" = "$HOST_ARCH" ] + [ "${PROPERTIES[1]}" = "$HOST_ARCH" ] + [ "${PROPERTIES[2]}" = 'unknown' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu' ] + [ "${PROPERTIES[5]}" = 'app-id' ] + [ "${PROPERTIES[6]}" = 'latest' ] + [ "${PROPERTIES[7]}" = 'release' ] +} + +@test "parse_target_properties.sh should parse host-march and id target name" { + mapfile -t PROPERTIES < <( parse_target_properties.sh x86_64+builder-template ) + + [ "${PROPERTIES[0]}" = 'x86_64' ] + [ "${PROPERTIES[1]}" = 'x86_64' ] + [ "${PROPERTIES[2]}" = 'unknown' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu' ] + [ "${PROPERTIES[5]}" = 'builder-template' ] + [ "${PROPERTIES[6]}" = 'latest' ] + + mapfile -t PROPERTIES < <( parse_target_properties.sh x86_64 builder-template ) + + [ "${PROPERTIES[0]}" = 'x86_64' ] + [ "${PROPERTIES[1]}" = 'x86_64' ] + [ "${PROPERTIES[2]}" = 'unknown' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu' ] + [ "${PROPERTIES[5]}" = 'builder-template' ] + [ "${PROPERTIES[6]}" = 'latest' ] +} + +@test "parse_target_properties.sh should parse longest target name" { + mapfile -t PROPERTIES < <( parse_target_properties.sh i386+arm32v7-alpine@3.9-none-eabihf+builder-template ) + + [ "${PROPERTIES[0]}" = 'i386' ] + [ "${PROPERTIES[1]}" = 'arm32v7' ] + [ "${PROPERTIES[2]}" = 'alpine' ] + [ "${PROPERTIES[3]}" = 'none' ] + [ "${PROPERTIES[4]}" = 'eabihf' ] + [ "${PROPERTIES[5]}" = 'builder-template' ] + [ "${PROPERTIES[6]}" = '3.9' ] + + mapfile -t PROPERTIES < <( parse_target_properties.sh i386 arm32v7-alpine@3.9-none-eabihf builder-template ) + + [ "${PROPERTIES[0]}" = 'i386' ] + [ "${PROPERTIES[1]}" = 'arm32v7' ] + [ "${PROPERTIES[2]}" = 'alpine' ] + [ "${PROPERTIES[3]}" = 'none' ] + [ "${PROPERTIES[4]}" = 'eabihf' ] + [ "${PROPERTIES[5]}" = 'builder-template' ] + [ "${PROPERTIES[6]}" = '3.9' ] +} + +@test "parse_target_properties.sh should parse distribution version containing unescaped special character [@]" { + mapfile -t PROPERTIES < <( parse_target_properties.sh i386+arm32v7-alpine@3.9@345+builder-template~@y ) + + [ "${PROPERTIES[0]}" = 'i386' ] + [ "${PROPERTIES[1]}" = 'arm32v7' ] + [ "${PROPERTIES[2]}" = 'alpine' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu' ] + [ "${PROPERTIES[5]}" = 'builder-template@y' ] + [ "${PROPERTIES[6]}" = '3.9@345' ] + + mapfile -t PROPERTIES < <( parse_target_properties.sh i386 arm32v7-alpine@3.9@345 builder-template~@y ) + + [ "${PROPERTIES[0]}" = 'i386' ] + [ "${PROPERTIES[1]}" = 'arm32v7' ] + [ "${PROPERTIES[2]}" = 'alpine' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu' ] + [ "${PROPERTIES[5]}" = 'builder-template@y' ] + [ "${PROPERTIES[6]}" = '3.9@345' ] +} + +@test "parse_target_properties.sh should parse target abi containing unescaped special character [-]" { + mapfile -t PROPERTIES < <( parse_target_properties.sh arm32v7-alpine@3.9-linux-gnu-x+builder-template ) + + [ "${PROPERTIES[0]}" = 'arm32v7' ] + [ "${PROPERTIES[1]}" = 'arm32v7' ] + [ "${PROPERTIES[2]}" = 'alpine' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu-x' ] + [ "${PROPERTIES[5]}" = 'builder-template' ] + [ "${PROPERTIES[6]}" = '3.9' ] + + mapfile -t PROPERTIES < <( parse_target_properties.sh arm32v7-alpine@3.9-linux-gnu-x builder-template ) + + [ "${PROPERTIES[0]}" = 'arm32v7' ] + [ "${PROPERTIES[1]}" = 'arm32v7' ] + [ "${PROPERTIES[2]}" = 'alpine' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu-x' ] + [ "${PROPERTIES[5]}" = 'builder-template' ] + [ "${PROPERTIES[6]}" = '3.9' ] +} + +@test "parse_target_properties.sh should parse application id containing unescaped special characters [-]" { + mapfile -t PROPERTIES < <( parse_target_properties.sh i386+builder-template@y-z ) + + [ "${PROPERTIES[0]}" = 'i386' ] + [ "${PROPERTIES[1]}" = 'i386' ] + [ "${PROPERTIES[2]}" = 'unknown' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu' ] + [ "${PROPERTIES[5]}" = 'builder-template' ] + [ "${PROPERTIES[6]}" = 'latest' ] + [ "${PROPERTIES[7]}" = 'y-z' ] + + mapfile -t PROPERTIES < <( parse_target_properties.sh i386 builder-template@y-z ) + + [ "${PROPERTIES[0]}" = 'i386' ] + [ "${PROPERTIES[1]}" = 'i386' ] + [ "${PROPERTIES[2]}" = 'unknown' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu' ] + [ "${PROPERTIES[5]}" = 'builder-template' ] + [ "${PROPERTIES[6]}" = 'latest' ] + [ "${PROPERTIES[7]}" = 'y-z' ] +} + +@test "parse_target_properties.sh should parse application id containing unescaped special character [+], when host arch is given" { + mapfile -t PROPERTIES < <( parse_target_properties.sh + ) + + [ "${PROPERTIES[0]}" = "$HOST_ARCH" ] + [ "${PROPERTIES[1]}" = "$HOST_ARCH" ] + [ "${PROPERTIES[2]}" = 'unknown' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu' ] + [ "${PROPERTIES[5]}" = '+' ] + [ "${PROPERTIES[6]}" = 'latest' ] + + mapfile -t PROPERTIES < <( parse_target_properties.sh i386+arm32v7+builder-template@x+y-z ) + + [ "${PROPERTIES[0]}" = 'i386' ] + [ "${PROPERTIES[1]}" = 'arm32v7' ] + [ "${PROPERTIES[2]}" = 'unknown' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu' ] + [ "${PROPERTIES[5]}" = 'builder-template' ] + [ "${PROPERTIES[6]}" = 'latest' ] + [ "${PROPERTIES[7]}" = 'x+y-z' ] + + mapfile -t PROPERTIES < <( parse_target_properties.sh i386 arm32v7 builder-template@x y-z ) + + [ "${PROPERTIES[0]}" = 'i386' ] + [ "${PROPERTIES[1]}" = 'arm32v7' ] + [ "${PROPERTIES[2]}" = 'unknown' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu' ] + [ "${PROPERTIES[5]}" = 'builder-template' ] + [ "${PROPERTIES[6]}" = 'latest' ] + [ "${PROPERTIES[7]}" = 'x+y-z' ] +} + +@test "parse_target_properties.sh should parse escaped chars [+|\\s] in target name" { + mapfile -t PROPERTIES < <( parse_target_properties.sh 'x86~+arm32~ ~~-x-alpine~+app' ) + + [ "${PROPERTIES[0]}" = "$HOST_ARCH" ] + [ "${PROPERTIES[1]}" = "$HOST_ARCH" ] + [ "${PROPERTIES[2]}" = 'unknown' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu' ] + [ "${PROPERTIES[5]}" = 'x86+arm32 ~-x-alpine+app' ] + [ "${PROPERTIES[6]}" = 'latest' ] +} + +@test "parse_target_properties.sh should parse escaped char [@] in target name" { + mapfile -t PROPERTIES < <( parse_target_properties.sh arm32v7-alpine~@3.9-linux-gnu-x+builder-template ) + + [ "${PROPERTIES[0]}" = 'arm32v7' ] + [ "${PROPERTIES[1]}" = 'arm32v7' ] + [ "${PROPERTIES[2]}" = 'alpine@3.9' ] + [ "${PROPERTIES[3]}" = 'linux' ] + [ "${PROPERTIES[4]}" = 'gnu-x' ] + [ "${PROPERTIES[5]}" = 'builder-template' ] + [ "${PROPERTIES[6]}" = 'latest' ] +} + +@test "parse_target_properties.sh should parse escaped char [-] in target name" { + mapfile -t PROPERTIES < <( parse_target_properties.sh arm32v7-linaro~-gcc-none-eabihf+builder-template ) + + [ "${PROPERTIES[0]}" = 'arm32v7' ] + [ "${PROPERTIES[1]}" = 'arm32v7' ] + [ "${PROPERTIES[2]}" = 'linaro-gcc' ] + [ "${PROPERTIES[3]}" = 'none' ] + [ "${PROPERTIES[4]}" = 'eabihf' ] + [ "${PROPERTIES[5]}" = 'builder-template' ] + [ "${PROPERTIES[6]}" = 'latest' ] + +} + diff --git a/tests/41_dobuild_opts.bats b/tests/41_dobuild_opts.bats new file mode 100644 index 0000000..7e360c3 --- /dev/null +++ b/tests/41_dobuild_opts.bats @@ -0,0 +1,310 @@ +#!/usr/bin/env bats +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +load test_helper + +setup() { + BINPATH="$(readlink -f "$BATS_TEST_DIRNAME/../bin")" + PATH="$BINPATH:$PATH" + TEMP="$(mktemp --directory --tmpdir="$BATS_TMPDIR" dobuild_XXXXXXXXXX)" +} + +teardown() { + rm -rf "$TEMP" +} + +@test "dobuild has required dep find" { + find --version +} + +@test "dobuild has required dep dirname" { + dirname --version +} + +@test "dobuild has required dep mkdir" { + mkdir --version +} + +@test "dobuild has required dep grep" { + grep --version +} + +@test "dobuild has required dep sed" { + sed --version +} + +@test "dobuild should fail when -f, --file or --makefile argument is empty" { + export MAKE='echo' + + run dobuild -f 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ '"-f" requires a non-empty option argument' ]] + + run dobuild --file 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ '"--file" requires a non-empty option argument' ]] + + run dobuild --file= 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ '"--file=" requires a non-empty option argument' ]] + + run dobuild --makefile 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ '"--makefile" requires a non-empty option argument' ]] + + run dobuild --makefile= 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ '"--makefile=" requires a non-empty option argument' ]] +} + +@test "dobuild should pass -f, --file or --makefile argument to make" { + export MAKE='echo' + + run dobuild -f Makefile + + echo "$output" + [ "$status" -eq 0 ] + [[ "$output" =~ '-f Makefile' ]] + + run dobuild --file Makefile + + echo "$output" + [ "$status" -eq 0 ] + [[ "$output" =~ '--file Makefile' ]] + + run dobuild --file=Makefile + + echo "$output" + [ "$status" -eq 0 ] + [[ "$output" =~ '--file=Makefile' ]] +} + +@test "dobuild should pass all -f, --file or --makefile arguments to make and use first to determine project kind" { + export MAKE='echo' + + run dobuild -f Makefile -f other.mk + + echo "$output" + [ "$status" -eq 0 ] + [[ "$output" =~ '-f Makefile -f other.mk' ]] +} + +@test "dobuild should fail when -C, --directory argument is empty" { + export MAKE='echo' + + run dobuild -C 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ '"-C" requires a non-empty option argument' ]] + + run dobuild --directory 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ '"--directory" requires a non-empty option argument' ]] + + run dobuild --directory= 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ '"--directory=" requires a non-empty option argument' ]] +} + +@test "dobuild should not pass -C, --directory arguments to make" { + export MAKE='echo' + + run dobuild -C "$TEMP" -f Makefile + + echo "$output" + [ "$status" -eq 0 ] + [[ ! "$output" =~ "-C $TEMP" ]] + + run dobuild --directory "$TEMP" -f Makefile + + echo "$output" + [ "$status" -eq 0 ] + [[ ! "$output" =~ "--directory $TEMP" ]] + + run dobuild --directory="$TEMP" -f Makefile + + echo "$output" + [ "$status" -eq 0 ] + [[ ! "$output" =~ "--directory=$TEMP" ]] +} + +@test "dobuild should fail when directory denoted by -C, --directory does not exist" { + export MAKE='echo' + + run dobuild -C non-existent -f Makefile 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ "can't cd to non-existent" ]] + + run dobuild --directory non-existent -f Makefile 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ "can't cd to non-existent" ]] + + run dobuild --directory=non-existent -f Makefile 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ "can't cd to non-existent" ]] +} + +@test "dobuild should fail when --image argument is empty" { + export MAKE='echo' + + run dobuild --image 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ '"--image" requires a non-empty option argument' ]] + + run dobuild --image= 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ '"--image=" requires a non-empty option argument' ]] +} + +@test "dobuild should print warning when --dockerfile argument is empty and fail while project kind is unknown" { + export MAKE='echo' + + run dobuild -C "$TEMP" --dockerfile 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ 'warning: "--dockerfile" ignored, requires a non-empty option argument' ]] + [[ "$output" =~ 'error: unknown project kind' ]] + + run dobuild -C "$TEMP" --dockerfile= 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ 'warning: "--dockerfile=" ignored, requires a non-empty option argument' ]] + [[ "$output" =~ 'error: unknown project kind' ]] +} + +@test "dobuild should pass --dockerfile argument to make as DOCKERFILE variable" { + export MAKE='echo' + + run dobuild -f Makefile --dockerfile=Dockerfile + + echo "$output" + [ "$status" -eq 0 ] + [[ "$output" =~ 'DOCKERFILE=Dockerfile' ]] +} + +@test "dobuild should try to find dockerfile by FILTER in project by default and fails while no exact match can be found" { + export MAKE='echo' + + touch "$TEMP/Dockerfile" + touch "$TEMP/arm32v7-alpine-x.dockerfile" + touch "$TEMP/x86_64-alpine-x.dockerfile" + + run dobuild -C "$TEMP" -f Makefile 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ 'more than one match found for *.dockerfile in . using filter [^-]*-[^.]*' ]] +} + +@test "dobuild should find exact match for dockerfile using MARCH and DISTRIB" { + export MAKE='echo' + + touch "$TEMP/Dockerfile" + touch "$TEMP/arm32v7-alpine-x.dockerfile" + touch "$TEMP/x86_64-alpine-x.dockerfile" + touch "$TEMP/arm32v7-ubuntu-x.dockerfile" + touch "$TEMP/x86_64-ubuntu-x.dockerfile" + + run dobuild -C "$TEMP" -f Makefile MARCH=arm32v7 DISTRIB=ubuntu 2>&1 + + echo "$output" + [ "$status" -eq 0 ] + [[ "$output" =~ 'DOCKERFILE=./arm32v7-ubuntu-x.dockerfile' ]] + + run dobuild -C "$TEMP" -f Makefile MARCH=arm32v7 DISTRIB=alpine 2>&1 + + echo "$output" + [ "$status" -eq 0 ] + [[ "$output" =~ 'DOCKERFILE=./arm32v7-alpine-x.dockerfile' ]] +} + +@test "dobuild should find exact match for dockerfile using FILTER" { + export MAKE='echo' + + touch "$TEMP/Dockerfile" + touch "$TEMP/arm32v7-alpine-x.dockerfile" + touch "$TEMP/x86_64-alpine-x.dockerfile" + touch "$TEMP/arm32v7-ubuntu-x.dockerfile" + touch "$TEMP/x86_64-ubuntu-x.dockerfile" + + run dobuild -C "$TEMP" -f Makefile --filter 'arm.\?.\?v7-alpine-x' 2>&1 + + echo "$output" + [ "$status" -eq 0 ] + [[ "$output" =~ 'DOCKERFILE=./arm32v7-alpine-x.dockerfile' ]] + [[ "$output" =~ 'FILTER=arm.\?.\?v7-alpine-x' ]] + + run dobuild -C "$TEMP" -f Makefile 'FILTER=arm.\?.\?v7-alpine-x' 2>&1 + + echo "$output" + [ "$status" -eq 0 ] + [[ "$output" =~ 'DOCKERFILE=./arm32v7-alpine-x.dockerfile' ]] + [[ "$output" =~ 'FILTER=arm.\?.\?v7-alpine-x' ]] +} + +@test "dobuild should fallback to Dockerfile in project by default" { + export MAKE='echo' + + touch "$TEMP/Dockerfile" + + run dobuild -C "$TEMP" -f Makefile + + echo "$output" + [ "$status" -eq 0 ] + [[ "$output" =~ "DOCKERFILE=./Dockerfile" ]] +} + +@test "dobuild should print warning when --filter argument is empty and fail while project kind is unknown" { + export MAKE='echo' + + run dobuild -C "$TEMP" --filter 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ 'warning: "--filter" ignored, requires a non-empty option argument' ]] + [[ "$output" =~ 'error: unknown project kind' ]] + + run dobuild -C "$TEMP" --filter= 2>&1 + + echo "$output" + [ "$status" -ne 0 ] + [[ "$output" =~ 'warning: "--filter=" ignored, requires a non-empty option argument' ]] + [[ "$output" =~ 'error: unknown project kind' ]] +} + + diff --git a/tests/50_defaults_mk.bats b/tests/50_defaults_mk.bats new file mode 100644 index 0000000..4cb5503 --- /dev/null +++ b/tests/50_defaults_mk.bats @@ -0,0 +1,554 @@ +#!/usr/bin/env bats +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +load test_helper + +setup() { + PROJECTPATH="$(readlink -f "$BATS_TEST_DIRNAME/..")" + MAKE=("make" --no-print-directory -C "$PROJECTPATH" -f defaults.mk -f "$BATS_TEST_DIRNAME/test_helper.mk") + VERSIONFILE="$(mktemp --tmpdir="$BATS_TMPDIR" version_XXXXXXXXXX.txt)" + + export DOBUILD_PROJECTVERSIONFILE="$VERSIONFILE" +} + +teardown() { + rm -f "$VERSIONFILE" +} + +@test "defaults_mk has required deps" { + "${MAKE[@]}" --version +} + +@test "defaults_mk has required makefile deps" { + "${MAKE[@]}" testhelper-print-MAKEFILE_DEPS | sort -u | parallel --keep-order command -V +} + +@test "defaults_mk can be included multiple times" { + run "${MAKE[@]}" -f defaults.mk 2>&1 + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ 'make: Nothing to be done' ]] +} + +print_fixture_or() { + cat < "$VERSIONFILE" + VALUE="$("${MAKE[@]}" testhelper-print-VERSION)" + echo "$VALUE != $EXPECTED" + [ "$VALUE" = "$EXPECTED" ] +} + +@test "defaults_mk HOST_CONTAINER can be overridden by env var DOBUILD_HOSTCONTAINER" { + EXPECTED="1234" + export DOBUILD_HOSTCONTAINER="$EXPECTED" + VALUE="$("${MAKE[@]}" testhelper-print-HOST_CONTAINER)" + echo "$VALUE != $EXPECTED" + [ "$VALUE" = "$EXPECTED" ] +} + +@test "defaults_mk SOURCE_DATE_EPOCH is number" { + VALUE="$("${MAKE[@]}" testhelper-print-SOURCE_DATE_EPOCH)" + [ "$VALUE" -ge 0 ] +} + +@test "defaults_mk SOURCE_DATE_EPOCH can be overridden" { + EXPECTED="1234" + VALUE="$("${MAKE[@]}" testhelper-print-SOURCE_DATE_EPOCH SOURCE_DATE_EPOCH="$EXPECTED")" + echo "$VALUE != $EXPECTED" + [ "$VALUE" = "$EXPECTED" ] +} + +@test "defaults_mk BUILDTIME is not empty" { + VALUE="$("${MAKE[@]}" testhelper-print-BUILDTIME)" + [ -n "$VALUE" ] +} + +@test "defaults_mk VERBOSE is off by default and can be overridden by env var" { + VALUE="$("${MAKE[@]}" testhelper-default-silent)" + echo "$VALUE" + [ -z "$VALUE" ] + + export VERBOSE=1 + VALUE="$("${MAKE[@]}" testhelper-default-silent)" + echo "$VALUE" + [ "$VALUE" = 'true' ] +} + +@test "defaults_mk JOBSLOTS defaults to system available processors" { + EXPECTED="$(nproc)" + VALUE="$("${MAKE[@]}" testhelper-print-JOBSLOTS)" + echo "$VALUE != $EXPECTED" + [ "$VALUE" -eq "$EXPECTED" ] +} + +@test "defaults_mk JOBSLOTS defaults to system available processors when make called recursive without -j" { + EXPECTED="$(nproc)" + VALUE="$("${MAKE[@]}" testhelper-recursive-print-JOBSLOTS)" + echo "$VALUE != $EXPECTED" + [ "$VALUE" -eq "$EXPECTED" ] +} + +@test "defaults_mk JOBSLOTS defaults to system available processors when make called recursive with unlimited job slots" { + EXPECTED="$(nproc)" + VALUE="$("${MAKE[@]}" -j testhelper-recursive-print-JOBSLOTS)" + echo "$VALUE != $EXPECTED" + [ "$VALUE" -eq "$EXPECTED" ] +} + +@test "defaults_mk JOBSLOTS defaults to 2 when make called recursive with limited job slots" { + EXPECTED="2" + VALUE="$("${MAKE[@]}" -j10 testhelper-recursive-print-JOBSLOTS)" + echo "$VALUE != $EXPECTED" + [ "$VALUE" -eq "$EXPECTED" ] +} + +@test "defaults_mk JOBSLOTS should be equal to last make command line argument" { + VALUE="$("${MAKE[@]}" -j10 -j 42 testhelper-print-JOBSLOTS)" + echo "$VALUE != 42" + [ "$VALUE" -eq "42" ] + + VALUE="$("${MAKE[@]}" -j14 testhelper-print-JOBSLOTS)" + echo "$VALUE != 14" + [ "$VALUE" -eq "14" ] + + VALUE="$("${MAKE[@]}" -j11 testhelper-print-JOBSLOTS)" + echo "$VALUE != 11" + [ "$VALUE" -eq "11" ] + + VALUE="$("${MAKE[@]}" --jobs 11 --jobs 10 testhelper-print-JOBSLOTS)" + echo "$VALUE != 10" + [ "$VALUE" -eq "10" ] + + VALUE="$("${MAKE[@]}" -j10 --jobs=88 testhelper-print-JOBSLOTS)" + echo "$VALUE != 88" + [ "$VALUE" -eq "88" ] +} + +@test "defaults_mk MAKE_VERSION should be greater or equal to 3.81 warning" { + run "${MAKE[@]}" make_version=4.1 testhelper-print-EXPECTATIONS 2>&1 + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ ! "${lines[@]}" =~ "Using old make version" ]] + + run "${MAKE[@]}" make_version=3.80 testhelper-print-EXPECTATIONS 2>&1 + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ "Using old make version" ]] +} + diff --git a/tests/60_standardrules_mk.bats b/tests/60_standardrules_mk.bats new file mode 100644 index 0000000..0991d2b --- /dev/null +++ b/tests/60_standardrules_mk.bats @@ -0,0 +1,237 @@ +#!/usr/bin/env bats +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +load test_helper + +setup() { + PROJECTPATH="$(readlink -f "$BATS_TEST_DIRNAME/..")" + BINPATH="$PROJECTPATH/bin" + PATH="$BINPATH:$PATH" + OUTDIR="$(mktemp --directory --tmpdir="$BATS_TMPDIR" standardrules_XXXXXXXXXX)" + MAKE=("make" --no-print-directory -C "$PROJECTPATH" -f standardrules.mk -f "$BATS_TEST_DIRNAME/test_helper.mk") + + export DOBUILD_OUTDIR="$OUTDIR" +} + +teardown() { + rm -rf "$OUTDIR" +} + +has_required_version() { + local major=${1:-0} + local minor=${2:-0} + local bugfix=${3:-0} + + set -- $(sed -n -e 's/.* Make \([0-9]\+\)[.]\([0-9]\+\)\([.]\([0-9]\+\)\)\?/\1 \2 \4/p') + + if [ "$1" -gt "$major" ]; then + return 0 + fi + if [ "$1" -lt "$major" ]; then + return -1 + fi + + if [ "$2" -gt "$minor" ]; then + return 0 + fi + if [ "$2" -lt "$minor" ]; then + return -2 + fi + + if [ "${3:-0}" -ge "$bugfix" ]; then + return 0 + fi + + return -3 +} + +make_has_required_version() { + "${MAKE[@]}" --version | has_required_version "$@" +} + +@test "standardrules_mk has required deps" { + "${MAKE[@]}" --version +} + +@test "standardrules_mk has required makefile deps" { + "${MAKE[@]}" testhelper-print-MAKEFILE_DEPS | sort -u | parallel --keep-order command -V +} + +@test "standardrules_mk PROJECTDIR should not contain whitespace" { + run "${MAKE[@]}" PROJECTDIR='my project' + printf -- '%s\n' "${lines[@]}" + [ "$status" -ne 0 ] + [[ "${lines[@]}" =~ "Project directory PROJECTDIR='my project' should not contain whitespaces" ]] +} + +@test "standardrules_mk DOBUILDDIR should not contain whitespace" { + run "${MAKE[@]}" DOBUILDDIR='my scriptdir' + printf -- '%s\n' "${lines[@]}" + [ "$status" -ne 0 ] + [[ "${lines[@]}" =~ "Script directory DOBUILDDIR='my scriptdir' should not contain whitespaces" ]] +} + +@test "standardrules_mk OUTDIRS should not contain PROJECTDIR" { + run "${MAKE[@]}" PROJECTDIR="$PROJECTPATH" OUTDIR='.' + printf -- '%s\n' "${lines[@]}" + [ "$status" -ne 0 ] + [[ "${lines[@]}" =~ "assertion failed: Project location PROJECTDIR" ]] + [[ "${lines[@]}" =~ "should not point to one of the output locations" ]] +} + +@test "standardrules_mk all targets are tested" { + run "${MAKE[@]}" print-targets 2>&1 + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [ "${lines[0]}" = 'all' ] + [ "${lines[1]}" = 'check' ] + [ "${lines[2]}" = 'clean' ] + [ "${lines[3]}" = 'dist' ] + [ "${lines[4]}" = 'distclean' ] + [ "${lines[5]}" = 'install' ] + [ "${lines[6]}" = 'lint' ] + [ "${lines[7]}" = 'memcheck' ] + [ "${lines[8]}" = 'prepare' ] + [ "${lines[9]}" = 'print-targets' ] + [ "${lines[10]}" = 'run' ] + # do not forget to add a test :) + [ "${#lines[@]}" -ge 11 ] +} + +@test "standardrules_mk should run default target 'all' by default" { + run "${MAKE[@]}" 2>&1 + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ 'make: Nothing to be done for '[\`|\']"all'." ]] +} + +@test "standardrules_mk returns success when empty check target is triggered" { + run "${MAKE[@]}" check 2>&1 + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ 'make: Nothing to be done for '[\`|\']"check'." ]] +} + +@test "standardrules_mk returns success when empty clean target is triggered" { + run "${MAKE[@]}" clean 2>&1 + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ 'make: Nothing to be done for '[\`|\']"clean'." ]] +} + +@test "standardrules_mk returns success when empty distclean target is triggered" { + run "${MAKE[@]}" distclean 2>&1 + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ 'make: Nothing to be done for '[\`|\']"distclean'." ]] +} + +@test "standardrules_mk returns success when empty prepare target is triggered " { + run "${MAKE[@]}" prepare 2>&1 + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ 'make: Nothing to be done for '[\`|\']"prepare'." ]] +} + +@test "standardrules_mk returns success when empty lint target is triggered" { + run "${MAKE[@]}" lint 2>&1 + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ 'make: Nothing to be done for '[\`|\']"lint'." ]] +} + +@test "standardrules_mk returns success when empty memcheck target is triggered" { + run "${MAKE[@]}" memcheck 2>&1 + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ 'make: Nothing to be done for '[\`|\']"memcheck'." ]] +} + +@test "standardrules_mk returns success when empty run target is triggered" { + run "${MAKE[@]}" run 2>&1 + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ 'make: Nothing to be done for '[\`|\']"run'." ]] +} + +@test "standardrules_mk returns error when makefile dependency are unsatisfied" { + run "${MAKE[@]}" MAKEFILE_DEPS='not-found1 not-found2 cat' not-found1 2>&1 + + printf -- '%s\n' "${lines[@]}" + [ "$status" -ne 0 ] + [[ "${lines[@]}" =~ "Required commands not-found1 not-found2 not found; install appropriate packages" ]] +} + +@test "standardrules_mk should print PROJECTNAME when target triggered" { + EXPECTED="$(basename "$PROJECTPATH")" + VALUE="$("${MAKE[@]}" debug-print-PROJECTNAME)" + + echo "$VALUE != $EXPECTED" + [ "$VALUE" = "$EXPECTED" ] +} + +@test "standardrules_mk EXTERNPARALLEL should default to 1 when make called recursive with limited job slots" { + EXPECTED="1" + VALUE="$("${MAKE[@]}" -j10 testhelper-recursive-print-EXTERNPARALLEL)" + echo "$VALUE != $EXPECTED" + [ "$VALUE" -eq "$EXPECTED" ] +} + +@test "standardrules_mk INTERNPARALLEL should default to 2 when make called recursive with limited job slots" { + EXPECTED="2" + VALUE="$("${MAKE[@]}" -j10 testhelper-recursive-print-INTERNPARALLEL)" + echo "$VALUE != $EXPECTED" + [ "$VALUE" -eq "$EXPECTED" ] +} + +@test "standardrules_mk EXTERNPARALLEL * INTERNPARALLEL should not exceed provided job slots" { + PARALLELMFLAGS='-j10' + EXPECTED='10' + EXTERNPARALLEL="$("${MAKE[@]}" "$PARALLELMFLAGS" testhelper-print-EXTERNPARALLEL)" + INTERNPARALLEL="$("${MAKE[@]}" "$PARALLELMFLAGS" testhelper-print-INTERNPARALLEL)" + VALUE="$(($EXTERNPARALLEL * $INTERNPARALLEL))" + echo "$VALUE != $EXPECTED" + [ "$VALUE" -eq "$EXPECTED" ] +} + +@test "standardrules_mk EXTERNPARALLEL should not exceed active targets" { + EXPECTED="$("${MAKE[@]}" testhelper-print-JOBS)" + VALUE="$("${MAKE[@]}" testhelper-print-EXTERNPARALLEL)" + echo "$VALUE <= $EXPECTED" + [ "$VALUE" -le "$EXPECTED" ] +} + +@test "standardrules_mk MAKEFLAGS should contain output-sync option when job slots greater than 1" { + if ! make_has_required_version '4'; then + skip "make version too old '$("${MAKE[@]}" --version)'" + fi + + run "${MAKE[@]}" testhelper-print-MAKEFLAGS EXTERNPARALLEL="32" + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ "-Otarget" ]] +} + +@test "standardrules_mk MAKEFLAGS should not contain output-sync option when job slots equal to 1" { + run "${MAKE[@]}" testhelper-print-MAKEFLAGS EXTERNPARALLEL="1" + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ ! "${lines[@]}" =~ "-Otarget" ]] +} diff --git a/tests/70_cmake_docker_mk.bats b/tests/70_cmake_docker_mk.bats new file mode 100644 index 0000000..a038065 --- /dev/null +++ b/tests/70_cmake_docker_mk.bats @@ -0,0 +1,164 @@ +#!/usr/bin/env bats +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +load test_helper + +setup() { + PROJECTPATH="$(readlink -f "$BATS_TEST_DIRNAME/..")" + FIXTUREPATH="$(readlink -f "$BATS_TEST_DIRNAME/fixtures/cmake-gtest-example")" + BUILDDIR="$(mktemp --directory --tmpdir="$BATS_TMPDIR" cmake_XXXXXXXXXX)" + MAKE=(timeout --preserve-status --signal=SIGTERM 300 "make" --no-print-directory -C "$FIXTUREPATH" -f Makefile -f "$BATS_TEST_DIRNAME/test_helper.mk") + + export DOBUILD_BUILDDIR="$BUILDDIR" +} + +teardown() { + rm -rf "$BUILDDIR" +} + +@test "cmake_docker_mk has required deps" { + "${MAKE[@]}" --version +} + +@test "cmake_docker_mk has required makefile deps" { + "${MAKE[@]}" testhelper-print-MAKEFILE_DEPS | sort -u | parallel --keep-order command -V +} + +@test "cmake_docker_mk should run default target 'all' with make generator by default" { + run "${MAKE[@]}" + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ "Configuring done" ]] + [[ "${lines[@]}" =~ "Generating done" ]] + [[ "${lines[@]}" =~ "Linking CXX executable alltests" ]] + + run "${MAKE[@]}" + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ ! "${lines[@]}" =~ "Configuring done" ]] + [[ ! "${lines[@]}" =~ "Generating done" ]] + [[ ! "${lines[@]}" =~ "Linking CXX executable alltests" ]] +} + +@test "cmake_docker_mk build should be reproducible using make generator" { + run "${MAKE[@]}" + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ "Configuring done" ]] + [[ "${lines[@]}" =~ "Generating done" ]] + [[ "${lines[@]}" =~ "Linking CXX executable alltests" ]] + + mv "$BUILDDIR/.build/"*/alltests "$BUILDDIR" + rm -rf "$BUILDDIR/.build" + + run "${MAKE[@]}" + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ "Linking CXX executable alltests" ]] + + diff <(xxd "$BUILDDIR/.build/"*/alltests) <(xxd "$BUILDDIR/alltests") +} + +@test "cmake_docker_mk should run default target 'all' with ninja generator" { + run "${MAKE[@]}" CMAKE_GENERATOR=ninja + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ "Configuring done" ]] + [[ "${lines[@]}" =~ "Generating done" ]] + [[ "${lines[@]}" =~ "Linking CXX executable alltests" ]] + + run "${MAKE[@]}" CMAKE_GENERATOR=ninja + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ ! "${lines[@]}" =~ "Configuring done" ]] + [[ ! "${lines[@]}" =~ "Generating done" ]] + [[ "${lines[@]}" =~ "ninja: no work to do" ]] +} + +@test "cmake_docker_mk build should be reproducible using ninja generator" { + run "${MAKE[@]}" CMAKE_GENERATOR=ninja + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ "Configuring done" ]] + [[ "${lines[@]}" =~ "Generating done" ]] + [[ "${lines[@]}" =~ "Linking CXX executable alltests" ]] + + mv "$BUILDDIR/.build/"*/alltests "$BUILDDIR" + rm -rf "$BUILDDIR/.build" + + run "${MAKE[@]}" CMAKE_GENERATOR=ninja + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ "Linking CXX executable alltests" ]] + + diff <(xxd "$BUILDDIR/.build/"*/alltests) <(xxd "$BUILDDIR/alltests") +} + +@test "cmake_docker_mk should run tests when check target is triggered after a successful built" { + run "${MAKE[@]}" + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ "Linking CXX executable alltests" ]] + + run "${MAKE[@]}" check + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ ! "${lines[@]}" =~ "Configuring done" ]] + [[ ! "${lines[@]}" =~ "Generating done" ]] + [[ "${lines[@]}" =~ "100% tests passed, 0 tests failed out of 1" ]] + + run "${MAKE[@]}" check + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ ! "${lines[@]}" =~ "Configuring done" ]] + [[ ! "${lines[@]}" =~ "Generating done" ]] + [[ "${lines[@]}" =~ "100% tests passed, 0 tests failed out of 1" ]] +} + +@test "cmake_docker_mk should run tests with memcheck when memcheck target is triggered after a successful built" { + run debug_trace "${MAKE[@]}" + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ "${lines[@]}" =~ "Linking CXX executable alltests" ]] + + "${MAKE[@]}" debug-print-MEMCHECKFILTER + run debug_trace "${MAKE[@]}" memcheck + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ ! "${lines[@]}" =~ "Configuring done" ]] + [[ ! "${lines[@]}" =~ "Generating done" ]] + [[ "${lines[@]}" =~ "Memory check" ]] + [[ "${lines[@]}" =~ "100% tests passed, 0 tests failed out of 1" ]] + + "${MAKE[@]}" debug-print-MEMCHECKFILTER + run debug_trace "${MAKE[@]}" memcheck + + printf -- '%s\n' "${lines[@]}" + [ "$status" -eq 0 ] + [[ ! "${lines[@]}" =~ "Configuring done" ]] + [[ ! "${lines[@]}" =~ "Generating done" ]] + [[ "${lines[@]}" =~ "Memory check" ]] + [[ "${lines[@]}" =~ "100% tests passed, 0 tests failed out of 1" ]] +} + diff --git a/tests/fixtures/cmake-gtest-example/CMakeLists.txt b/tests/fixtures/cmake-gtest-example/CMakeLists.txt new file mode 100644 index 0000000..2ddb4a0 --- /dev/null +++ b/tests/fixtures/cmake-gtest-example/CMakeLists.txt @@ -0,0 +1,19 @@ +cmake_minimum_required(VERSION 2.8) + +project(cmake-gtest-example) + +option(POSITION_INDEPENDENT_CODE "This variable is used to initialize the POSITION_INDEPENDENT_CODE property on all the targets" ON) + +include(CTest) + +add_executable(alltests + test_stringcompare.cpp +) + +target_include_directories(alltests PUBLIC ${GTEST_INCLUDE_DIRS}) +target_compile_options(alltests PUBLIC -Wall -Wextra ${GTEST_CFLAGS}) +target_link_libraries(alltests PUBLIC -pthread ${GTEST_LIBRARIES} gtest_main gtest) + +enable_testing() +add_test(NAME alltests COMMAND alltests) + diff --git a/tests/fixtures/cmake-gtest-example/Makefile b/tests/fixtures/cmake-gtest-example/Makefile new file mode 100644 index 0000000..7e31bc6 --- /dev/null +++ b/tests/fixtures/cmake-gtest-example/Makefile @@ -0,0 +1,95 @@ +SHELL := /bin/sh +MAKEFILE := $(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST)) +MAKEFLAGS += --no-builtin-rules + +.SUFFIXES: + +.PHONY: default +default: all + +####################################################################################################################### +# Overridable project defaults + +DOBUILD_TOPDIR ?= $(DOBUILDDIR) +DOBUILD_PROJECTDIR ?= $(patsubst %/,%,$(dir $(MAKEFILE))) + +PROJECTDIR = $(DOBUILD_PROJECTDIR) +DOBUILDDIR ?= $(PROJECTDIR)/../../.. + +include $(DOBUILDDIR)/defaults.mk + +####################################################################################################################### +# Project defaults and macros + +DEFAULTTARGET = x86_64-ubuntu@bionic+builder + +FETCHDIR = $(BUILDDIR)/.deps + +####################################################################################################################### +# Project dependencies + +DUMB_INIT_VERSION = 1.2.2 +IMAGE_BUILDARGS += DUMB_INIT_VERSION=$(DUMB_INIT_VERSION) +IMAGE_BUILDARGS += DUMB_INIT_MTIME=$(call mtime,$(FETCHDIR)/dumb-init-$(DUMB_INIT_VERSION).tar.gz) +FETCH_TARGETS += $(FETCHDIR)/dumb-init-$(DUMB_INIT_VERSION).tar.gz +$(FETCHDIR)/dumb-init-$(DUMB_INIT_VERSION).tar.gz: URL = https://github.com/Yelp/dumb-init/archive/v$(DUMB_INIT_VERSION).tar.gz +$(SKIP_MD5SUM)$(FETCHDIR)/dumb-init-$(DUMB_INIT_VERSION).tar.gz: MD5 = 6166084b05772cdcf615a762c6f3b32e + +GTEST_VERSION = 1.8.0 +IMAGE_BUILDARGS += GTEST_VERSION=$(GTEST_VERSION) +IMAGE_BUILDARGS += GTEST_MTIME=$(call mtime,$(FETCHDIR)/googletest-release-$(GTEST_VERSION).tar.gz) +FETCH_TARGETS += $(FETCHDIR)/googletest-release-$(GTEST_VERSION).tar.gz +$(FETCHDIR)/googletest-release-$(GTEST_VERSION).tar.gz: URL := https://github.com/google/googletest/archive/release-$(GTEST_VERSION).tar.gz +$(SKIP_MD5SUM)$(FETCHDIR)/googletest-release-$(GTEST_VERSION).tar.gz: MD5 := 16877098823401d1bf2ed7891d7dce36 + +####################################################################################################################### +# Architecture-specific rule target configuration + +CMAKE_TARGETS += x86_64-ubuntu@bionic+builder +DOCKER_TARGETS += $(CMAKE_TARGETS) + +####################################################################################################################### +# Common rule target configuration + +CURLFLAGS += -s + +DOCKER_RUNFLAGS += --cap-add SYS_PTRACE +DOCKER_RUNFLAGS += --security-opt seccomp=unconfined + +OUTDIRS += $(OUTDIR)/src + +EXTRACT_TARGETS += $(patsubst $(FETCHDIR)/%.tar.gz,$(OUTDIR)/src/%,$(FETCH_TARGETS)) + +####################################################################################################################### +# Makefile dependencies + +MAKEFILE_DEPS += gzip +MAKEFILE_DEPS += tar +MAKEFILE_DEPS += touch +MAKEFILE_DEPS += cp +MAKEFILE_DEPS += mkdir + +####################################################################################################################### +# Rules + +include $(DOBUILDDIR)/cmake.mk +include $(DOBUILDDIR)/docker.mk +include $(DOBUILDDIR)/standardrules.mk + +$(OUTDIR)/src/%: $(FETCHDIR)/%.tar.gz | $(OUTDIRS) + $(SILENT) \ + $(call echo_if_silent_cmd,tar -C $(dir $@) -xf $<) \ + && tar -I 'gzip -n' -C $(dir $@) -xf $< \ + && touch $@ + +$(OUTDIR)/docker/%.dockerfile : $(PROJECTDIR)/%.dockerfile | $(OUTDIRS) + cp $< $@ + +$(FETCH_TARGETS): | $(FETCHDIR) + $(SILENT)$(call curl,$@,$(URL),$(MD5)) + +$(FETCHDIR): + $(SILENT)mkdir -p $@ + +.DELETE_ON_ERROR: $(FETCH_TARGETS) + diff --git a/tests/fixtures/cmake-gtest-example/test_stringcompare.cpp b/tests/fixtures/cmake-gtest-example/test_stringcompare.cpp new file mode 100644 index 0000000..f08be82 --- /dev/null +++ b/tests/fixtures/cmake-gtest-example/test_stringcompare.cpp @@ -0,0 +1,24 @@ +#include + +#include +using std::string; + +char const actualValTrue[] = "hello gtest"; +char const actualValFalse[] = "hello world"; +char const expectVal[] = "hello gtest"; + +TEST(StrCompare, Equal) +{ + EXPECT_STREQ(expectVal, actualValTrue); +} + +TEST(StrCompare, NotEqual) +{ + EXPECT_STRNE(expectVal, actualValFalse); +} + +TEST(StrCompare, WithNonReproducibleValues) +{ + EXPECT_STREQ(__DATE__, __DATE__); + EXPECT_STREQ(__TIME__, __TIME__); +} diff --git a/tests/fixtures/cmake-gtest-example/x86_64-ubuntu-builder.dockerfile b/tests/fixtures/cmake-gtest-example/x86_64-ubuntu-builder.dockerfile new file mode 100644 index 0000000..80a8fd7 --- /dev/null +++ b/tests/fixtures/cmake-gtest-example/x86_64-ubuntu-builder.dockerfile @@ -0,0 +1,94 @@ +ARG REGISTRY_PREFIX='' +ARG CODENAME=bionic + +FROM ${REGISTRY_PREFIX}ubuntu:${CODENAME} as builder + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +RUN set -x \ + && installdeps="tzdata" \ + && rm -f /etc/apt/sources.list.d/*.list \ + && apt-get update \ + && apt-get install --yes --no-install-recommends $installdeps \ + && ls /usr/share/zoneinfo \ + && cp -H --remove-destination "/usr/share/zoneinfo/$TZ" /tmp/localtime \ + && { apt-get purge -y $installdeps || true; } \ + && mv /tmp/localtime /etc/localtime \ + && echo "$TZ" > /etc/timezone \ + && apt-get update \ + && apt-get install --yes --no-install-recommends \ + build-essential \ + cmake \ + && rm -rf /var/lib/apt/lists/* + +COPY src /usr/local/src + +ARG PARALLELMFLAGS=-j2 + +ARG DUMB_INIT_VERSION=1.2.2 +ARG DUMB_INIT_MTIME= +RUN set -x \ + && builddeps="vim-common" \ + && apt-get update \ + && apt-get install --yes --no-install-recommends $builddeps \ + && rm -rf /var/lib/apt/lists/* \ + && [ -n "$DUMB_INIT_MTIME" ] && export SOURCE_DATE_EPOCH="$DUMB_INIT_MTIME" \ + && builddir="/tmp/out" \ + && mkdir -p "$builddir" \ + && cd "$builddir" \ + && cp -R "/usr/local/src/dumb-init-$DUMB_INIT_VERSION" . \ + && cd "dumb-init-$DUMB_INIT_VERSION" \ + && make "$PARALLELMFLAGS" \ + && chmod +x dumb-init \ + && mv dumb-init /usr/local/bin/dumb-init \ + && dumb-init --version \ + && rm -rf "$builddir" \ + && apt-get purge -y $builddeps + +ARG GTEST_VERSION=1.8.1 +ARG GTEST_MTIME= + +RUN set -x \ + && [ -n "$GTEST_MTIME" ] && export SOURCE_DATE_EPOCH="$GTEST_MTIME" \ + && builddir="/tmp/out" \ + && mkdir -p "$builddir" \ + && cd "$builddir" \ + && cmake "/usr/local/src/googletest-release-$GTEST_VERSION" \ + && make "$PARALLELMFLAGS" install \ + && rm -rf "$builddir" + +FROM ${REGISTRY_PREFIX}ubuntu:${CODENAME} + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +COPY --from=builder /usr/local /usr/local +COPY --from=builder /etc/localtime /etc/localtime +COPY --from=builder /etc/timezone /etc/timezone +COPY --from=builder /etc/apt/sources.list.d /etc/apt/sources.list.d + +RUN set -x \ + && rm -f /etc/apt/sources.list.d/*.list \ + && apt-get update \ + && apt-get install --yes --no-install-recommends \ + build-essential \ + cmake \ + ninja-build \ + pkg-config \ + gdb \ + gdbserver \ + valgrind \ + && rm -rf /var/lib/apt/lists/* + +ENV LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib" + +ARG USERID=1000 +RUN set -x \ + && useradd -u "$USERID" -ms /bin/bash user + +ENTRYPOINT ["dumb-init", "--"] +CMD [ "/bin/bash" ] + diff --git a/tests/fixtures/make-gtest-example/Makefile b/tests/fixtures/make-gtest-example/Makefile new file mode 100644 index 0000000..9f1e174 --- /dev/null +++ b/tests/fixtures/make-gtest-example/Makefile @@ -0,0 +1,18 @@ +# Makefile for gtest example + +CPPFLAGS = -I /usr/local/include +CXXFLAGS = -c -Wall +LDFLAGS = -L /usr/local/lib -l gtest_main -l gtest -l pthread + +TARGET = alltests +OBJECTS = test_stringcompare.o + +all: $(TARGET) + +$(TARGET): $(OBJECTS) + $(CXX) -o $@ $^ $(LD_FLAGS) + +clean: + rm -f $(TARGET) $(OBJECTS) + +.PHONY: all clean diff --git a/tests/fixtures/make-gtest-example/test_stringcompare.cpp b/tests/fixtures/make-gtest-example/test_stringcompare.cpp new file mode 100644 index 0000000..748c91b --- /dev/null +++ b/tests/fixtures/make-gtest-example/test_stringcompare.cpp @@ -0,0 +1,18 @@ +#include // googletest header file + +#include +using std::string; + +char const actualValTrue[] = "hello gtest"; +char const actualValFalse[] = "hello world"; +char const expectVal[] = "hello gtest"; + +TEST(StrCompare, CStrEqual) +{ + EXPECT_STREQ(expectVal, actualValTrue); +} + +TEST(StrCompare, CStrNotEqual) +{ + EXPECT_STRNE(expectVal, actualValFalse); +} diff --git a/tests/get_container_id.dockerfile b/tests/get_container_id.dockerfile new file mode 100644 index 0000000..176a712 --- /dev/null +++ b/tests/get_container_id.dockerfile @@ -0,0 +1,27 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +ARG REGISTRY_PREFIX='' +ARG DOCKER_VERSION=18.09.6 + +FROM ${REGISTRY_PREFIX}docker:${DOCKER_VERSION}-dind + +ARG USERID=1000 +ARG PROJECTPATH=/home/user/dobuild + +RUN set -x \ + && adduser -u "$USERID" -s /bin/sh -D user \ + && mkdir -p "$PROJECTPATH" \ + && chown user:user "$PROJECTPATH" + +COPY --chown=user:user . "$PROJECTPATH" +WORKDIR "$PROJECTPATH/bin" + +VOLUME "$PROJECTPATH" diff --git a/tests/runners/bats.dockerfile b/tests/runners/bats.dockerfile new file mode 100644 index 0000000..f835e26 --- /dev/null +++ b/tests/runners/bats.dockerfile @@ -0,0 +1,167 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +ARG REGISTRY_PREFIX='' +ARG CODENAME=trusty + +FROM ${REGISTRY_PREFIX}ubuntu:${CODENAME} as builder + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +ARG PARALLELMFLAGS= + +ARG DOCKER_CHANNEL=stable +ARG DOCKER_VERSION=18.09.6 +ARG DOCKER_MD5=a6be1e734421d05abfc4d3e28997e271 +ARG DOCKER_DOWNLOAD=https://download.docker.com + +ARG DOCKER_HOME="/usr/local/lib/docker-$DOCKER_VERSION" + +ARG CAPATH="./ca-certificates" +COPY $CAPATH /usr/local/share/ca-certificates + +RUN set -x \ + && { update-ca-certificates || true; } \ + && installdeps="tzdata" \ + && apt-get update \ + && apt-get install --yes --no-install-recommends $installdeps \ + && ls /usr/share/zoneinfo \ + && cp -H --remove-destination "/usr/share/zoneinfo/$TZ" /tmp/localtime \ + && { apt-get purge -y $installdeps || true; } \ + && mv /tmp/localtime /etc/localtime \ + && echo "$TZ" > /etc/timezone \ + && apt-get install --yes --no-install-recommends \ + ca-certificates \ + curl \ + openssl \ + build-essential \ + make \ + && rm -rf /var/lib/apt/lists/* \ + && update-ca-certificates + +RUN set -x \ + && curl -fSL -s "$DOCKER_DOWNLOAD/linux/static/${DOCKER_CHANNEL}/$(uname -m)/docker-${DOCKER_VERSION}.tgz" -o docker.tgz \ + && md5sum docker.tgz \ + && echo "$DOCKER_MD5 docker.tgz" | md5sum -c - \ + && tar -xzvf docker.tgz \ + && mkdir -p "$DOCKER_HOME" \ + && mv docker/docker "$DOCKER_HOME" \ + && rm -rf docker \ + && rm docker.tgz \ + && ln -s "$DOCKER_HOME/docker" /usr/local/bin/docker + +ARG BATS_VERSION=1.1.0 +ARG BATS_MD5=0cb16021aa8f75a29240434c5aaae0a1 +ARG BATS_DOWNLOAD=https://github.com/bats-core + +RUN set -x \ + && builddeps="coreutils ncurses-bin bash" \ + && apt-get update \ + && apt-get install --yes --no-install-recommends $builddeps \ + && rm -rf /var/lib/apt/lists/* \ + && curl -fSL -s "$BATS_DOWNLOAD/bats-core/archive/v$BATS_VERSION.tar.gz" -o bats.tar.gz \ + && md5sum bats.tar.gz \ + && echo "$BATS_MD5 bats.tar.gz" | md5sum -c - \ + && tar -xzvf bats.tar.gz \ + && bash "bats-core-$BATS_VERSION/install.sh" /usr/local \ + && rm -rf "bats-core-$BATS_VERSION" \ + && rm bats.tar.gz + +ARG DUMB_INIT_VERSION=1.2.2 +ARG DUMB_INIT_MD5=6166084b05772cdcf615a762c6f3b32e +ARG DUMB_INIT_DOWNLOAD=https://github.com/Yelp/dumb-init + +RUN set -x \ + && parallelMFlags="$PARALLELMFLAGS" \ + && [ -n "$parallelMFlags" ] || parallelMFlags="-j$(nproc)" \ + && builddeps="vim-common" \ + && apt-get update \ + && apt-get install --yes --no-install-recommends $builddeps \ + && rm -rf /var/lib/apt/lists/* \ + && curl -fSL -s "$DUMB_INIT_DOWNLOAD/archive/v${DUMB_INIT_VERSION}.tar.gz" -o dumb-init.tar.gz \ + && md5sum dumb-init.tar.gz \ + && echo "$DUMB_INIT_MD5 dumb-init.tar.gz" | md5sum -c - \ + && export SOURCE_DATE_EPOCH="$(stat -c '%Y' dumb-init.tar.gz)" \ + && tar -xzvf dumb-init.tar.gz \ + && cd "dumb-init-$DUMB_INIT_VERSION" \ + && make "$parallelMFlags" \ + && chmod +x dumb-init \ + && cp dumb-init /usr/local/bin/dumb-init \ + && cd .. \ + && rm -rf "dumb-init-$DUMB_INIT_VERSION" \ + && rm dumb-init.tar.gz \ + && apt-get purge -y $builddeps + +ARG LIBFAKETIME_VERSION=0.9.7 +ARG LIBFAKETIME_MD5=8617e2c6caf0977b3ce9a271f867302c +ARG LIBFAKETIME_DOWNLOAD=https://github.com/wolfcw/libfaketime + +RUN set -x \ + && parallelMFlags="$PARALLELMFLAGS" \ + && [ -n "$parallelMFlags" ] || parallelMFlags="-j$(nproc)" \ + && curl -fSL -s "$LIBFAKETIME_DOWNLOAD/archive/v${LIBFAKETIME_VERSION}.tar.gz" -o libfaketime.tar.gz \ + && md5sum libfaketime.tar.gz \ + && echo "$LIBFAKETIME_MD5 libfaketime.tar.gz" | md5sum -c - \ + && export SOURCE_DATE_EPOCH="$(stat -c '%Y' libfaketime.tar.gz)" \ + && tar -xzvf libfaketime.tar.gz \ + && cd "libfaketime-$LIBFAKETIME_VERSION" \ + && prefix='/usr/local' \ + && make "$parallelMFlags" \ + CFLAGS="-fpic -lpthread -Wno-error -D'PREFIX=\"${prefix}\"' -D'LIBDIRNAME=\"lib\"'" \ + LDFLAGS='-fpic -lpthread' \ + && make install \ + && cd .. \ + && rm -rf "libfaketime-$LIBFAKETIME_VERSION" \ + && rm libfaketime.tar.gz + +FROM ${REGISTRY_PREFIX}ubuntu:${CODENAME} + +ARG TZ=UTC +ENV LANG=C.UTF-8 +ENV LC_ALL=${LANG} + +COPY --from=builder /usr/local /usr/local +COPY --from=builder /etc/localtime /etc/localtime +COPY --from=builder /etc/timezone /etc/timezone +COPY --from=builder /etc/apt/sources.list.d /etc/apt/sources.list.d + +RUN set -x \ + && { update-ca-certificates || true; } \ + && apt-get update \ + && apt-get install --yes --no-install-recommends \ + coreutils \ + ncurses-bin \ + bash \ + procps \ + parallel \ + ca-certificates \ + make \ + git \ + curl \ + && rm -rf /var/lib/apt/lists/* \ + && update-ca-certificates \ + && useradd -ms /bin/bash user + +USER user + +# Accept the citation notice of GNU parallel (we aren't using this in a +# context where it make sense to cite GNU Parallel). +RUN set -x \ + && mkdir -p ~/.parallel \ + && touch ~/.parallel/will-cite + +RUN dumb-init --version +RUN parallel --version +RUN docker --version +RUN bats --version + +ENTRYPOINT ["dumb-init", "--"] diff --git a/tests/runners/dind-bind_mount.yml b/tests/runners/dind-bind_mount.yml new file mode 100644 index 0000000..a842172 --- /dev/null +++ b/tests/runners/dind-bind_mount.yml @@ -0,0 +1,17 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +version: "2.4" + +services: + dind: + volumes: + - .:${COMPOSEENV_PROJECTPATH}:ro + diff --git a/tests/runners/dind-volumes_from.yml b/tests/runners/dind-volumes_from.yml new file mode 100644 index 0000000..522d0f9 --- /dev/null +++ b/tests/runners/dind-volumes_from.yml @@ -0,0 +1,17 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +version: "2.4" + +services: + dind: + volumes_from: + - container:${COMPOSEENV_VOLUMESFROM}:ro + diff --git a/tests/runners/dind.dockerfile b/tests/runners/dind.dockerfile new file mode 100644 index 0000000..3244065 --- /dev/null +++ b/tests/runners/dind.dockerfile @@ -0,0 +1,24 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +ARG REGISTRY_PREFIX='' +ARG DOCKER_VERSION=18.09.6 + +FROM ${REGISTRY_PREFIX}docker:${DOCKER_VERSION}-dind + +ARG CAPATH="./ca-certificates" +COPY $CAPATH /usr/local/share/ca-certificates + +RUN set -x \ + && { update-ca-certificates || true; } \ + && apk add --no-cache \ + ca-certificates \ + && update-ca-certificates + diff --git a/tests/test_helper.bash b/tests/test_helper.bash new file mode 100644 index 0000000..6c92f39 --- /dev/null +++ b/tests/test_helper.bash @@ -0,0 +1,29 @@ +#!/usr/bin/env bash +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +DOCKER="${DOCKER:-docker}" +export DOCKER + +USERID="${USERID:-$(id -u)}" + +_cancel() { + if [[ -t 3 ]]; then + exec 3<&- + fi +} + +debug_trace() { + echo "${@}" 1>&2 + "${@}" +} + +# workaround to avoid deadlocks on signal termination +trap _cancel SIGINT SIGTERM diff --git a/tests/test_helper.mk b/tests/test_helper.mk new file mode 100644 index 0000000..86daf7f --- /dev/null +++ b/tests/test_helper.mk @@ -0,0 +1,24 @@ +# +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this +# file, You can obtain one at http://mozilla.org/MPL/2.0/. +# +# This file is part of dobuild. +# Copyright (c) 2019 Contributors as noted in the AUTHORS file. +# +# SPDX-License-Identifier: MPL-2.0 + +.PHONY: testhelper-default +testhelper-default: + +.PHONY: testhelper-print-% +testhelper-print-%: + @printf '%s\n' $($*) + +.PHONY: testhelper-default-silent +testhelper-default-silent: + $(SILENT)true + +.PHONY: testhelper-recursive-print-% +testhelper-recursive-print-%: + @$(MAKE) $(MFLAGS) $(addprefix -f,$(MAKEFILE_LIST)) testhelper-print-$* $(MAKEOVERRIDES) diff --git a/todo.txt b/todo.txt new file mode 100644 index 0000000..4dc314b --- /dev/null +++ b/todo.txt @@ -0,0 +1,29 @@ +handle snapshot version for continue integration +add target for validation of reproducible build +extract bootstrap build environment +finalize dobuild wrapper +add tests for dobuild wrapper +fix/ add debugging support +fix test fixture to enable reuse of builder templates (docker import!?) +add compose configuration for jenkins-slave - minimal env, cron cleanup service, dind +check for common security vulnerabilities +add support for a validatable configuration (groovy-makefile-generator?) +generalize default makefile builders to allow zero configuration for common use cases +add package and download caching services to support offline builds (Apt-Cacher-ng?) +add support for docker based packing (runtime dockerfile) +add support for docker caching (cache-from) +add setup support for eclipse cdt +add setup support for visual studio code +add example for coverage with cmake +add example for clang-tidy with cmake +add example for clang-format with cmake +add support for podman +document implemented features +add support for upload of artifacts +add support for gradle based build projects +add support for npm based build projects +add support for make based build projects +add support for debian based packaging +add support for dependency download manager +tests in kubernetes context +publish version 1.0.0 diff --git a/workspace/extension/.gitignore b/workspace/extension/.gitignore new file mode 100644 index 0000000..e69de29 diff --git a/workspace/out/.gitignore b/workspace/out/.gitignore new file mode 100644 index 0000000..e69de29 diff --git a/workspace/src/.gitignore b/workspace/src/.gitignore new file mode 100644 index 0000000..e69de29 diff --git a/workspace/stage/.gitignore b/workspace/stage/.gitignore new file mode 100644 index 0000000..e69de29