1. 23 Nov, 2017 3 commits
  2. 18 Nov, 2017 3 commits
  3. 16 Nov, 2017 2 commits
  4. 13 Nov, 2017 3 commits
    • Masahiro Yamada's avatar
      kbuild: do not call cc-option before KBUILD_CFLAGS initialization · 433dc2eb
      Masahiro Yamada authored
      Some $(call cc-option,...) are invoked very early, even before
      KBUILD_CFLAGS, etc. are initialized.
      The returned string from $(call cc-option,...) depends on
      Since they are exported, they are not empty when the top Makefile
      is recursively invoked.
      The recursion occurs in several places.  For example, the top
      Makefile invokes itself for silentoldconfig.  "make tinyconfig",
      "make rpm-pkg" are the cases, too.
      In those cases, the second call of cc-option from the same line
      runs a different shell command due to non-pristine KBUILD_CFLAGS.
      To get the same result all the time, KBUILD_* and GCC_PLUGINS_CFLAGS
      must be initialized before any call of cc-option.  This avoids
      garbage data in the .cache.mk file.
      Move all calls of cc-option below the config targets because target
      compiler flags are unnecessary for Kconfig.
      Signed-off-by: Masahiro Yamada's avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      Reviewed-by: default avatarDouglas Anderson <dianders@chromium.org>
    • Douglas Anderson's avatar
      kbuild: Cache a few more calls to the compiler · 4e562071
      Douglas Anderson authored and Masahiro Yamada's avatar Masahiro Yamada committed
      These are a few stragglers that I left out of the original patch to
      cache calls to the C compiler ("kbuild: Add a cache for generated
      variables") because they bleed out into the main Makefile and thus
      uglify things a little bit.  The idea is the same here, though.
      Signed-off-by: default avatarDouglas Anderson <dianders@chromium.org>
      Tested-by: default avatarIngo Molnar <mingo@kernel.org>
      Tested-by: default avatarGuenter Roeck <linux@roeck-us.net>
      Signed-off-by: Masahiro Yamada's avatarMasahiro Yamada <yamada.masahiro@socionext.com>
    • Douglas Anderson's avatar
      kbuild: Add a cache for generated variables · 3298b690
      Douglas Anderson authored and Masahiro Yamada's avatar Masahiro Yamada committed
      While timing a "no-op" build of the kernel (incrementally building the
      kernel even though nothing changed) in the Chrome OS build system I
      found that it was much slower than I expected.
      Digging into things a bit, I found that quite a bit of the time was
      spent invoking the C compiler even though we weren't actually building
      anything.  Currently in the Chrome OS build system the C compiler is
      called through a number of wrappers (one of which is written in
      python!) and can take upwards of 100 ms to invoke even if we're not
      doing anything difficult, so these invocations of the compiler were
      taking a lot of time.  Worse the invocations couldn't seem to take
      advantage of the multiple cores on my system.
      Certainly it seems like we could make the compiler invocations in the
      Chrome OS build system faster, but only to a point.  Inherently
      invoking a program as big as a C compiler is a fairly heavy
      operation.  Thus even if we can speed the compiler calls it made sense
      to track down what was happening.
      It turned out that all the compiler invocations were coming from
      usages like this in the kernel's Makefile:
      KBUILD_CFLAGS += $(call cc-option,-fno-delete-null-pointer-checks,)
      Due to the way cc-option and similar statements work the above
      contains an implicit call to the C compiler.  ...and due to the fact
      that we're storing the result in KBUILD_CFLAGS, a simply expanded
      variable, the call will happen every time the Makefile is parsed, even
      if there are no users of KBUILD_CFLAGS.
      Rather than redoing this computation every time, it makes a lot of
      sense to cache the result of all of the Makefile's compiler calls just
      like we do when we compile a ".c" file to a ".o" file.  Conceptually
      this is quite a simple idea.  ...and since the calls to invoke the
      compiler and similar tools are centrally located in the Kbuild.include
      file this doesn't even need to be super invasive.
      Implementing the cache in a simple-to-use and efficient way is not
      quite as simple as it first sounds, though.  To get maximum speed we
      really want the cache in a format that make can natively understand
      and make doesn't really have an ability to load/parse files. ...but
      make _can_ import other Makefiles, so the solution is to store the
      cache in Makefile format.  This requires coming up with a valid/unique
      Makefile variable name for each value to be cached, but that's
      solvable with some cleverness.
      After this change, we'll automatically create a ".cache.mk" file that
      will contain our cached variables.  We'll load this on each invocation
      of make and will avoid recomputing anything that's already in our
      cache.  The cache is stored in a format that it shouldn't need any
      invalidation since anything that might change should affect the "key"
      and any old cached value won't be used.
      Signed-off-by: default avatarDouglas Anderson <dianders@chromium.org>
      Tested-by: default avatarIngo Molnar <mingo@kernel.org>
      Tested-by: default avatarGuenter Roeck <linux@roeck-us.net>
      Signed-off-by: Masahiro Yamada's avatarMasahiro Yamada <yamada.masahiro@socionext.com>
  5. 12 Nov, 2017 1 commit
  6. 08 Nov, 2017 1 commit
  7. 05 Nov, 2017 1 commit
  8. 02 Nov, 2017 1 commit
    • Greg Kroah-Hartman's avatar
      License cleanup: add SPDX GPL-2.0 license identifier to files with no license · b2441318
      Greg Kroah-Hartman authored
      Many source files in the tree are missing licensing information, which
      makes it harder for compliance tools to determine the correct license.
      By default all files without license information are under the default
      license of the kernel, which is GPL version 2.
      Update the files which contain no license information with the 'GPL-2.0'
      SPDX license identifier.  The SPDX identifier is a legally binding
      shorthand, which can be used instead of the full boiler plate text.
      This patch is based on work done by Thomas Gleixner and Kate Stewart and
      Philippe Ombredanne.
      How this work was done:
      Patches were generated and checked against linux-4.14-rc6 for a subset of
      the use cases:
       - file had no licensing information it it.
       - file was a */uapi/* one with no licensing information in it,
       - file was a */uapi/* one with existing licensing information,
      Further patches will be generated in subsequent months to fix up cases
      where non-standard license headers were used, and references to license
      had to be inferred by heuristics based on keywords.
      The analysis to determine which SPDX License Identifier to be applied to
      a file was done in a spreadsheet of side by side results from of the
      output of two independent scanners (ScanCode & Windriver) producing SPDX
      tag:value files created by Philippe Ombredanne.  Philippe prepared the
      base worksheet, and did an initial spot review of a few 1000 files.
      The 4.13 kernel was the starting point of the analysis with 60,537 files
      assessed.  Kate Stewart did a file by file comparison of the scanner
      results in the spreadsheet to determine which SPDX license identifier(s)
      to be applied to the file. She confirmed any determination that was not
      immediately clear with lawyers working with the Linux Foundation.
      Criteria used to select files for SPDX license identifier tagging was:
       - Files considered eligible had to be source code files.
       - Make and config files were included as candidates if they contained >5
         lines of source
       - File already had some variant of a license header in it (even if <5
      All documentation files were explicitly excluded.
      The following heuristics were used to determine which SPDX license
      identifiers to apply.
       - when both scanners couldn't find any license traces, file was
         considered to have no license information in it, and the top level
         COPYING file license applied.
         For non */uapi/* files that summary was:
         SPDX license identifier                            # files
         GPL-2.0                                              11139
         and resulted in the first patch in this series.
         If that file was a */uapi/* path one, it was "GPL-2.0 WITH
         Linux-syscall-note" otherwise it was "GPL-2.0".  Results of that was:
         SPDX license identifier                            # files
         GPL-2.0 WITH Linux-syscall-note                        930
         and resulted in the second patch in this series.
       - if a file had some form of licensing information in it, and was one
         of the */uapi/* ones, it was denoted with the Linux-syscall-note if
         any GPL family license was found in the file or had no licensing in
         it (per prior point).  Results summary:
         SPDX license identifier                            # files
         GPL-2.0 WITH Linux-syscall-note                       270
         GPL-2.0+ WITH Linux-syscall-note                      169
         ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause)    21
         ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause)    17
         LGPL-2.1+ WITH Linux-syscall-note                      15
         GPL-1.0+ WITH Linux-syscall-note                       14
         ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause)    5
         LGPL-2.0+ WITH Linux-syscall-note                       4
         LGPL-2.1 WITH Linux-syscall-note                        3
         ((GPL-2.0 WITH Linux-syscall-note) OR MIT)              3
         ((GPL-2.0 WITH Linux-syscall-note) AND MIT)             1
         and that resulted in the third patch in this series.
       - when the two scanners agreed on the detected license(s), that became
         the concluded license(s).
       - when there was disagreement between the two scanners (one detected a
         license but the other didn't, or they both detected different
         licenses) a manual inspection of the file occurred.
       - In most cases a manual inspection of the information in the file
         resulted in a clear resolution of the license that should apply (and
         which scanner probably needed to revisit its heuristics).
       - When it was not immediately clear, the license identifier was
         confirmed with lawyers working with the Linux Foundation.
       - If there was any question as to the appropriate license identifier,
         the file was flagged for further research and to be revisited later
         in time.
      In total, over 70 hours of logged manual review was done on the
      spreadsheet to determine the SPDX license identifiers to apply to the
      source files by Kate, Philippe, Thomas and, in some cases, confirmation
      by lawyers working with the Linux Foundation.
      Kate also obtained a third independent scan of the 4.13 code base from
      FOSSology, and compared selected files where the other two scanners
      disagreed against that SPDX file, to see if there was new insights.  The
      Windriver scanner is based on an older version of FOSSology in part, so
      they are related.
      Thomas did random spot checks in about 500 files from the spreadsheets
      for the uapi headers and agreed with SPDX license identifier in the
      files he inspected. For the non-uapi files Thomas did random spot checks
      in about 15000 files.
      In initial set of patches against 4.14-rc6, 3 files were found to have
      copy/paste license identifier errors, and have been fixed to reflect the
      correct identifier.
      Additionally Philippe spent 10 hours this week doing a detailed manual
      inspection and review of the 12,461 patched files from the initial patch
      version early this week with:
       - a full scancode scan run, collecting the matched texts, detected
         license ids and scores
       - reviewing anything where there was a license detected (about 500+
         files) to ensure that the applied SPDX license was correct
       - reviewing anything where there was no detection but the patch license
         was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied
         SPDX license was correct
      This produced a worksheet with 20 files needing minor correction.  This
      worksheet was then exported into 3 different .csv files for the
      different types of files to be modified.
      These .csv files were then reviewed by Greg.  Thomas wrote a script to
      parse the csv files and add the proper SPDX tag to the file, in the
      format that the file expected.  This script was further refined by Greg
      based on the output to detect more types of files automatically and to
      distinguish between header and source .c files (which need different
      comment types.)  Finally Greg ran the script using the .csv files to
      generate the patches.
      Reviewed-by: default avatarKate Stewart <kstewart@linuxfoundation.org>
      Reviewed-by: default avatarPhilippe Ombredanne <pombredanne@nexb.com>
      Reviewed-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
  9. 29 Oct, 2017 1 commit
  10. 26 Oct, 2017 1 commit
  11. 24 Oct, 2017 1 commit
  12. 23 Oct, 2017 1 commit
  13. 16 Oct, 2017 1 commit
  14. 14 Oct, 2017 1 commit
  15. 12 Oct, 2017 2 commits
  16. 10 Oct, 2017 3 commits
  17. 09 Oct, 2017 2 commits
  18. 07 Oct, 2017 2 commits
  19. 04 Oct, 2017 1 commit
  20. 01 Oct, 2017 1 commit
  21. 26 Sep, 2017 1 commit
    • Palmer Dabbelt's avatar
      RISC-V: Build Infrastructure · fbe934d6
      Palmer Dabbelt authored
      This patch contains all the build infrastructure that actually enables
      the RISC-V port.  This includes Makefiles, linker scripts, and Kconfig
      files.  It also contains the only top-level change, which adds RISC-V to
      the list of architectures that need a sed run to produce the ARCH
      variable when building locally.
      Signed-off-by: default avatarPalmer Dabbelt <palmer@dabbelt.com>
  22. 24 Sep, 2017 1 commit
  23. 21 Sep, 2017 1 commit
    • Shuah Khan's avatar
      Makefile: kselftest and kselftest-clean fail for make O=dir case · 2bc84526
      Shuah Khan authored
      kselftest and kselftest-clean targets fail when object directory is
      specified to relocate objects. Fix it so it can find the source tree
      to build from.
      make O=/tmp/kselftest_top kselftest
      make[1]: Entering directory '/tmp/kselftest_top'
      make[2]: Entering directory '/tmp/kselftest_top'
      make[2]: *** tools/testing/selftests: No such file or directory.  Stop.
      make[2]: Leaving directory '/tmp/kselftest_top'
      ./linux-kselftest/Makefile:1185: recipe for target
      'kselftest' failed
      make[1]: *** [kselftest] Error 2
      make[1]: Leaving directory '/tmp/kselftest_top'
      Makefile:145: recipe for target 'sub-make' failed
      make: *** [sub-make] Error 2
      Signed-off-by: default avatarShuah Khan <shuahkh@osg.samsung.com>
      Acked-by: Masahiro Yamada's avatarMasahiro Yamada <yamada.masahiro@socionext.com>
  24. 16 Sep, 2017 2 commits
  25. 14 Sep, 2017 1 commit
    • Greg Kroah-Hartman's avatar
      firmware: delete in-kernel firmware · 5620a0d1
      Greg Kroah-Hartman authored
      The last firmware change for the in-kernel firmware source code was back
      in 2013.  Everyone has been relying on the out-of-tree linux-firmware
      package for a long long time.
      So let's drop it, it's baggage we don't need to keep dragging around
      (and having to fix random kbuild issues over time...)
      Cc: Kyle McMartin <kyle@kernel.org>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Michal Marek <mmarek@suse.com>
      Cc: Masahiro Yamada <yamada.masahiro@socionext.com>
      Acked-by: default avatarDavid Woodhouse <dwmw2@infradead.org>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
  26. 03 Sep, 2017 1 commit
  27. 31 Aug, 2017 1 commit
    • Masahiro Yamada's avatar
      kbuild: use $(abspath ...) instead of $(shell cd ... && /bin/pwd) · 8e9b4667
      Masahiro Yamada authored
      Kbuild conventionally uses $(shell cd ... && /bin/pwd) idiom to get
      the absolute path of the directory because GNU Make 3.80, the minimal
      supported version at that time, did not support $(abspath ...) or
      $(realpath ...).
      Commit 37d69ee3
       ("docs: bump minimal GNU Make version to 3.81")
      dropped the GNU Make 3.80 support, so we are now allowed to use those
      make-builtin helpers.
      This conversion will provide better portability without relying on
      the pwd command or its location /bin/pwd.
      I am intentionally using $(realpath ...) instead $(abspath ...) in
      some places.  The difference between the two is $(realpath ...)
      returns an empty string if the given path does not exist.  It is
      convenient in places where we need to error-out if the makefile fails
      to create an output directory.
      Signed-off-by: Masahiro Yamada's avatarMasahiro Yamada <yamada.masahiro@socionext.com>
      Acked-by: default avatarThierry Reding <treding@nvidia.com>