Minki/linux
1
0
Fork 1
mirror of https://github.com/torvalds/linux.git synced 2024-11-22 04:02:20 +00:00
Code Issues Packages Projects Releases Wiki Activity
87c7ee67de
linux/scripts
History
Masahiro Yamada 87c7ee67de scripts: handle BrokenPipeError for python scripts 
In the follow-up of commit fb3041d61f ("kbuild: fix SIGPIPE error
message for AR=gcc-ar and AR=llvm-ar"), Kees Cook pointed out that
tools should _not_ catch their own SIGPIPEs [1] [2].

Based on his feedback, LLVM was fixed [3].

However, Python's default behavior is to show noisy bracktrace when
SIGPIPE is sent. So, scripts written in Python are basically in the
same situation as the buggy llvm tools.

Example:

  $ make -s allnoconfig
  $ make -s allmodconfig
  $ scripts/diffconfig .config.old .config | head -n1
  -ALIX n
  Traceback (most recent call last):
    File "/home/masahiro/linux/scripts/diffconfig", line 132, in <module>
      main()
    File "/home/masahiro/linux/scripts/diffconfig", line 130, in main
      print_config("+", config, None, b[config])
    File "/home/masahiro/linux/scripts/diffconfig", line 64, in print_config
      print("+%s %s" % (config, new_value))
  BrokenPipeError: [Errno 32] Broken pipe

Python documentation [4] notes how to make scripts die immediately and
silently:

  """
  Piping output of your program to tools like head(1) will cause a
  SIGPIPE signal to be sent to your process when the receiver of its
  standard output closes early. This results in an exception like
  BrokenPipeError: [Errno 32] Broken pipe. To handle this case,
  wrap your entry point to catch this exception as follows:

    import os
    import sys

    def main():
        try:
            # simulate large output (your code replaces this loop)
            for x in range(10000):
                print("y")
            # flush output here to force SIGPIPE to be triggered
            # while inside this try block.
            sys.stdout.flush()
        except BrokenPipeError:
            # Python flushes standard streams on exit; redirect remaining output
            # to devnull to avoid another BrokenPipeError at shutdown
            devnull = os.open(os.devnull, os.O_WRONLY)
            os.dup2(devnull, sys.stdout.fileno())
            sys.exit(1)  # Python exits with error code 1 on EPIPE

    if __name__ == '__main__':
        main()

  Do not set SIGPIPE’s disposition to SIG_DFL in order to avoid
  BrokenPipeError. Doing that would cause your program to exit
  unexpectedly whenever any socket connection is interrupted while
  your program is still writing to it.
  """

Currently, tools/perf/scripts/python/intel-pt-events.py seems to be the
only script that fixes the issue that way.

tools/perf/scripts/python/compaction-times.py uses another approach
signal.signal(signal.SIGPIPE, signal.SIG_DFL) but the Python
documentation clearly says "Don't do it".

I cannot fix all Python scripts since there are so many.
I fixed some in the scripts/ directory.

[1]: https://lore.kernel.org/all/202211161056.1B9611A@keescook/
[2]: https://github.com/llvm/llvm-project/issues/59037
[3]: 4787efa380
[4]: https://docs.python.org/3/library/signal.html#note-on-sigpipe

Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
Reviewed-by: Nick Desaulniers <ndesaulniers@google.com>
Reviewed-by: Nicolas Schier <nicolas@fjasle.eu>
2023-01-26 12:43:33 +09:00
..
atomic kbuild: check sha1sum just once for each atomic header 2022-09-29 04:40:14 +09:00
basic fixdep: do not parse *.rlib, *.rmeta, *.so 2023-01-22 23:43:33 +09:00
clang-tools scripts: handle BrokenPipeError for python scripts 2023-01-26 12:43:33 +09:00
coccinelle update Coccinelle URL 2022-08-07 21:30:36 +02:00
dtc scripts/dtc: Update to upstream version v1.6.1-63-g55778a03df61 2022-11-10 08:37:19 -06:00
dummy-tools kbuild: dummy-tools: pretend we understand __LONG_DOUBLE_128__ 2022-08-21 02:47:48 +09:00
gcc-plugins treewide: Replace GPLv2 boilerplate/reference with SPDX - gpl-2.0_385.RULE 2022-06-10 14:51:36 +02:00
gdb Updates to various subsystems which I help look after. lib, ocfs2, 2022-08-07 10:03:24 -07:00
genksyms genksyms: adjust the output format to modpost 2022-05-24 16:33:20 +09:00
kconfig kconfig: Update all declared targets 2023-01-13 16:29:52 +09:00
ksymoops
mod Kbuild updates for v6.2 2022-12-19 12:33:32 -06:00
package scripts: rpm: make clear that mkspec script contains 4.13 feature 2023-01-11 20:42:34 +09:00
selinux selinux: remove runtime disable message in the install_policy.sh script 2022-09-20 14:12:25 -04:00
tracing scripts/tracing: Fix typo 'the the' in comment 2022-08-02 15:17:00 -04:00
.gitignore kbuild: rust: move rust/target.json to scripts/ 2023-01-22 23:43:33 +09:00
adjust_autoksyms.sh kbuild: split the second line of *.mod into *.usyms 2022-05-08 03:16:59 +09:00
as-version.sh kbuild: Update assembler calls to use proper flags and language target 2023-01-26 12:41:38 +09:00
asn1_compiler.c kbuild: do not print extra logs for V=2 2023-01-22 23:43:32 +09:00
bin2c.c
bloat-o-meter scripts/bloat-o-meter: add -p argument 2022-07-17 17:31:40 -07:00
bootgraph.pl
bpf_doc.py bpf: Rework process_dynptr_func 2022-12-08 18:25:31 -08:00
cc-can-link.sh
cc-version.sh Kbuild: add Rust support 2022-09-28 09:02:20 +02:00
check_extable.sh scripts: check_extable: fix typo in user error message 2021-09-08 11:50:28 -07:00
check-local-export kbuild: rewrite check-local-export in sh/awk 2022-09-29 04:40:15 +09:00
check-sysctl-docs
checkdeclares.pl scripts: make some scripts executable 2021-08-10 09:13:25 +09:00
checkincludes.pl
checkkconfigsymbols.py scripts: handle BrokenPipeError for python scripts 2023-01-26 12:43:33 +09:00
checkpatch.pl checkpatch: add check for array allocator family argument order 2022-11-30 16:13:16 -08:00
checkstack.pl checkstack: add riscv support for scripts/checkstack.pl 2022-07-27 21:18:00 +09:00
checksyscalls.sh checksyscalls: ignore -Wunused-macros 2022-05-08 03:16:59 +09:00
checkversion.pl scripts: checkversion: modernize linux/version.h search strings 2021-08-05 20:55:39 +09:00
cleanfile
cleanpatch
coccicheck scripts: coccicheck: use "grep -E" instead of "egrep" 2022-09-21 21:23:56 +02:00
config kconfig: config script: add a little user help 2021-01-04 10:38:11 +09:00
const_structs.checkpatch const_structs.checkpatch: add frequently used ops structs 2022-01-20 08:52:54 +02:00
decode_stacktrace.sh scripts: decode_stacktrace: demangle Rust symbols 2022-09-28 09:01:40 +02:00
decodecode scripts/decodecode: improve faulting line determination 2022-09-11 21:55:05 -07:00
depmod.sh depmod: handle the case of /sbin/depmod without /sbin in PATH 2021-01-01 12:26:39 -08:00
dev-needs.sh scripts/dev-needs: Add script to list device dependencies 2020-09-04 18:19:37 +02:00
diffconfig scripts: handle BrokenPipeError for python scripts 2023-01-26 12:43:33 +09:00
documentation-file-ref-check scripts: documentation-file-ref-check: fix bpf selftests path 2021-10-26 09:42:29 -06:00
export_report.pl
extract_xc3028.pl
extract-ikconfig scripts/extract-ikconfig: add zstd compression support 2022-08-29 13:58:47 +09:00
extract-module-sig.pl
extract-sys-certs.pl
extract-vmlinux
faddr2line scripts/faddr2line: Fix regression in name resolution on ppc64le 2022-11-16 10:42:10 +01:00
file-size.sh
find-unused-docs.sh
gcc-x86_32-has-stack-protector.sh x86/stackprotector/32: Make the canary into a regular percpu variable 2021-03-08 13:19:05 +01:00
gcc-x86_64-has-stack-protector.sh
gen_autoksyms.sh kbuild: change module.order to list *.o instead of *.ko 2022-12-14 15:42:40 +09:00
gen_ksymdeps.sh kbuild: redo fake deps at include/ksym/*.h 2021-09-03 08:17:21 +09:00
gen-randstruct-seed.sh randstruct: Move seed generation into scripts/basic/ 2022-05-08 01:33:07 -07:00
generate_initcall_order.pl init: lto: ensure initcall ordering 2021-01-14 08:21:09 -08:00
generate_rust_analyzer.py rust: add build_error crate 2022-12-04 01:59:16 +01:00
generate_rust_target.rs x86: enable initial Rust support 2022-09-28 09:02:45 +02:00
get_abi.pl scripts/get_abi: Fix wrong script file name in the help message 2022-04-24 10:38:44 +02:00
get_dvb_firmware
get_feat.pl scripts: get_feat.pl: use /usr/bin/env to find perl 2022-06-30 12:22:17 -06:00
get_maintainer.pl get_maintainer: Honor mailmap for in file emails 2022-04-29 14:38:00 -07:00
gfp-translate
head-object-list.txt RISC-V Patches for the 6.2 Merge Window, Part 1 2022-12-14 15:23:49 -08:00
headerdep.pl
headers_install.sh scripts: headers_install.sh: Update config leak ignore entries 2022-07-27 21:18:00 +09:00
insert-sys-cert.c
install.sh kbuild: factor out the common installation code into scripts/install.sh 2022-05-11 21:45:53 +09:00
is_rust_module.sh scripts: add is_rust_module.sh 2022-09-28 09:02:06 +02:00
jobserver-exec scripts: support GNU make 4.4 in jobserver-exec 2023-01-16 20:15:20 +09:00
kallsyms.c Char/Misc driver changes for 6.2-rc1 2022-12-16 03:49:24 -08:00
Kbuild.include kbuild: replace $(dot-target).tmp in filechk with $(tmp-target) 2023-01-22 23:43:33 +09:00
Kconfig.include kbuild: Update assembler calls to use proper flags and language target 2023-01-26 12:41:38 +09:00
kernel-doc kbuild: do not print extra logs for V=2 2023-01-22 23:43:32 +09:00
ld-version.sh kbuild: collect minimum tool versions into scripts/min-tool-version.sh 2021-04-25 05:14:26 +09:00
leaking_addresses.pl leaking_addresses: Always print a trailing newline 2021-10-15 11:25:13 +02:00
Lindent
link-vmlinux.sh kallsyms: Correctly sequence symbols when CONFIG_LTO_CLANG=y 2022-11-12 18:47:36 -08:00
Makefile kbuild: rust: move rust/target.json to scripts/ 2023-01-22 23:43:33 +09:00
Makefile.asm-generic kbuild: add kbuild-file macro 2022-11-22 23:40:02 +09:00
Makefile.build kbuild: remove sed commands after rustc rules 2023-01-22 23:43:33 +09:00
Makefile.clang kbuild: Turn a couple more of clang's unused option warnings into errors 2023-01-26 12:43:19 +09:00
Makefile.clean kbuild: add kbuild-file macro 2022-11-22 23:40:02 +09:00
Makefile.compiler kbuild: Update assembler calls to use proper flags and language target 2023-01-26 12:41:38 +09:00
Makefile.debug Makefile.debug: support for -gz=zstd 2022-11-21 10:18:39 +09:00
Makefile.dtbinst kbuild: add kbuild-file macro 2022-11-22 23:40:02 +09:00
Makefile.extrawarn kbuild: add -Wundef to KBUILD_CPPFLAGS for W=1 builds 2022-12-11 17:28:32 +09:00
Makefile.gcc-plugins gcc-plugins: Undefine LATENT_ENTROPY_PLUGIN when plugin disabled for a file 2022-08-16 12:25:53 -07:00
Makefile.headersinst kbuild: prefix $(srctree)/ to some included Makefiles 2021-03-15 19:20:48 +09:00
Makefile.host kbuild: remove sed commands after rustc rules 2023-01-22 23:43:33 +09:00
Makefile.kasan kasan: always respect CONFIG_KASAN_STACK 2021-09-24 16:13:35 -07:00
Makefile.kcov kbuild: include scripts/Makefile.* only when relevant CONFIG is enabled 2020-08-10 01:32:59 +09:00
Makefile.kcsan kcsan: Ignore GCC 11+ warnings about TSan runtime support 2021-12-09 16:42:27 -08:00
Makefile.kmsan kmsan: add KMSAN runtime core 2022-10-03 14:03:19 -07:00
Makefile.lib kbuild: unify cmd_dt_S_dtb and cmd_dt_S_dtbo 2023-01-22 23:43:33 +09:00
Makefile.modfinal kbuild: rename cmd_$@ to savedcmd_$@ in *.cmd files 2023-01-22 23:43:33 +09:00
Makefile.modinst kbuild: change module.order to list *.o instead of *.ko 2022-12-14 15:42:40 +09:00
Makefile.modpost kbuild: readd -w option when vmlinux.o or Module.symver is missing 2023-01-05 16:53:48 +09:00
Makefile.package kbuild: add a missing line for help message 2022-12-30 17:22:14 +09:00
Makefile.randstruct randstruct: Enable Clang support 2022-05-08 01:33:07 -07:00
Makefile.ubsan ubsan: remove CONFIG_UBSAN_OBJECT_SIZE 2022-01-20 08:52:55 +02:00
Makefile.userprogs
Makefile.vmlinux kbuild: Fix CFI hash randomization with KASAN 2023-01-13 15:22:03 -08:00
Makefile.vmlinux_o kbuild: move modules.builtin(.modinfo) rules to Makefile.vmlinux_o 2022-10-03 03:52:58 +09:00
makelst
markup_oops.pl
min-tool-version.sh Documentation: raise minimum supported version of binutils to 2.25 2022-12-13 22:21:14 +09:00
misc-check kbuild: make W=1 warn files that are tracked but ignored by git 2023-01-22 23:43:33 +09:00
mkcompile_h Revert "kbuild: Make scripts/compile.h when sh != bash" 2022-09-29 04:40:15 +09:00
mksysmap kallsyms: ignore __kstrtab_* and __kstrtabns_* symbols 2022-10-03 03:51:58 +09:00
mkuboot.sh
module.lds.S arm64: unwind: add asynchronous unwind tables to kernel and modules 2022-11-09 18:06:35 +00:00
modules-check.sh kbuild: change module.order to list *.o instead of *.ko 2022-12-14 15:42:40 +09:00
nsdeps scripts/nsdeps: adjust to the format change of *.mod files 2022-06-08 20:14:13 +09:00
objdiff kbuild: clean .tmp_* pattern by make clean 2022-06-05 06:20:57 +09:00
objdump-func scripts: Create objdump-func helper script 2022-05-12 10:08:43 -07:00
pahole-flags.sh scripts/pahole-flags.sh: Parse DWARF and generate BTF with multithreading. 2022-02-22 14:32:44 -08:00
pahole-version.sh kbuild: Add CONFIG_PAHOLE_VERSION 2022-02-02 11:19:33 +01:00
parse-maintainers.pl
patch-kernel
profile2linkerlist.pl
prune-kernel scripts/prune-kernel: Use kernel-install if available 2022-05-11 21:46:38 +09:00
recordmcount.c LoongArch/ftrace: Add recordmcount support 2022-12-14 08:41:53 +08:00
recordmcount.h recordmcount: Correct st_shndx handling 2021-06-18 09:09:17 -04:00
recordmcount.pl nds32: Remove the architecture 2022-03-07 13:54:59 +01:00
remove-stale-files kbuild: rust: move rust/target.json to scripts/ 2023-01-22 23:43:33 +09:00
rust_is_available_bindgen_libclang.h scripts: add rust_is_available.sh 2022-09-28 09:02:06 +02:00
rust_is_available.sh scripts: add rust_is_available.sh 2022-09-28 09:02:06 +02:00
setlocalversion kbuild: do not quote string values in include/config/auto.conf 2022-01-08 18:03:57 +09:00
show_delta tweewide: Fix most Shebang lines 2020-12-08 23:30:04 +09:00
sign-file.c sign-file: Fix confusing error messages 2022-08-03 23:56:20 +03:00
sorttable.c LoongArch: extable: Add type and data fields 2022-12-14 08:36:11 +08:00
sorttable.h script/sorttable: Fix some initialization problems 2022-01-18 10:17:18 -05:00
spdxcheck-test.sh docs: move Linux logo into a new images folder 2022-06-01 09:32:45 -06:00
spdxcheck.py scripts/spdxcheck: Put excluded files and directories into a separate file 2022-05-18 15:34:33 +02:00
spdxexclude scripts/spdxcheck: Exclude top-level README 2022-05-18 15:35:42 +02:00
spelling.txt scripts/spelling.txt: add more spellings to spelling.txt 2022-11-18 13:55:09 -08:00
sphinx-pre-install docs: sphinx-pre-install: don't require the RTD theme 2022-10-13 11:14:43 -06:00
split-man.pl tweewide: Fix most Shebang lines 2020-12-08 23:30:04 +09:00
stackdelta
stackusage
subarch.include LoongArch: Add build infrastructure 2022-06-03 20:09:27 +08:00
syscallhdr.sh scripts: check duplicated syscall number in syscall table 2021-07-09 04:00:39 +09:00
syscallnr.sh scripts: make some scripts executable 2021-08-10 09:13:25 +09:00
syscalltbl.sh scripts: check duplicated syscall number in syscall table 2021-07-09 04:00:39 +09:00
tags.sh kbuild: allow to combine multiple V= levels 2023-01-22 23:43:32 +09:00
test_fortify.sh fortify: Update compile-time tests for Clang 14 2022-02-13 16:50:06 -08:00
tools-support-relr.sh Makefile: fix GDB warning with CONFIG_RELR 2021-06-08 13:09:34 +01:00
unifdef.c
ver_linux Removed the oprofiled version option 2021-05-03 17:23:06 -06:00
xen-hypercalls.sh scripts: make some scripts executable 2021-08-10 09:13:25 +09:00
xz_wrap.sh