Discussion:
Changing compilation flags
(too old to reply)
Pablo Roberto Lezaeta Reyes via arch-general
2017-07-01 09:54:25 UTC
Permalink
Raw Message
*> 1) building gcc to enable PIE by default
*>
I am in the middle of rebuilding gcc with --enable-default-pie. When it
finishes, I will start a todo for rebuilding packages with static libraries.
I also enabled --enable-default-ssp, which means that
-fstack-protector-strong will be dropped from our CFLAGS (as it will be
enforced by gcc) on the next opportunity.
Bartłomiej
Does the -enable-default-ssp enforce also -fstack-check=specific to protect
from stack clash [1], gentoo do it (except on vlc and tcl which not build
but those are upstream bugs) [2]

[1] https://blog.qualys.com/securitylabs/2017/06/19/the-stack-clash
[2] https://wiki.gentoo.org/wiki/Hardened/Gentoo_Hardened_and_Stack_Clash

*Pablo Lezaeta*
Alexander Harrigan
2017-07-01 10:49:16 UTC
Permalink
Raw Message
On On Sat, Jul 1, 2017 at 09:54 AM, arch-general <arch-
***@archlinux.org> wrote:

> >On 2016-10-24 05:56, Allan McRae wrote:

> >*> 1) building gcc to enable PIE by default

> *>

> >I am in the middle of rebuilding gcc with --enable-default-pie. When
it

> >finishes, I will start a todo for rebuilding packages with static
libraries.

> >

> >I also enabled --enable-default-ssp, which means that

> >-fstack-protector-strong will be dropped from our CFLAGS (as it will
be

> >enforced by gcc) on the next opportunity.

> >

> >Bartłomiej

>

> Does the -enable-default-ssp enforce also -fstack-check=specific to
protect

> from stack clash [1], gentoo do it (except on vlc and tcl which not build

> but those are upstream bugs) [2]

>

> [1] https://blog.qualys.com/securitylabs/2017/06/19/the-stack-clash

> [2] https://wiki.gentoo.org/wiki/Hardened/Gentoo_Hardened_and_Stack_Clash

>

> *Pablo Lezaeta*

>

No it doesn't but original plan [1] was to enable -fstack-check, -fno-plt and
-z,now to default flags in makepkg.conf. I hope Pacman maintainer will add
those before mass rebuild started so everythig will be done at once.

[1] https://lists.archlinux.org/pipermail/arch-dev-
public/2016-October/028405.html

\-- Sent using MsgSafe.io's Free Plan Private, encrypted, online communication
For everyone.
Eli Schwartz via arch-general
2017-07-02 02:10:32 UTC
Permalink
Raw Message
Post by Alexander Harrigan
On On Sat, Jul 1, 2017 at 09:54 AM, arch-general <arch-
> >*> 1) building gcc to enable PIE by default
> *>
> >I am in the middle of rebuilding gcc with --enable-default-pie. When
it
> >finishes, I will start a todo for rebuilding packages with static
libraries.
> >
> >I also enabled --enable-default-ssp, which means that
> >-fstack-protector-strong will be dropped from our CFLAGS (as it will
be
> >enforced by gcc) on the next opportunity.
> >
> >Bartłomiej
>
> Does the -enable-default-ssp enforce also -fstack-check=specific to
protect
> from stack clash [1], gentoo do it (except on vlc and tcl which not build
> but those are upstream bugs) [2]
>
> [1] https://blog.qualys.com/securitylabs/2017/06/19/the-stack-clash
> [2] https://wiki.gentoo.org/wiki/Hardened/Gentoo_Hardened_and_Stack_Clash
>
> *Pablo Lezaeta*
>
No it doesn't but original plan [1] was to enable -fstack-check, -fno-plt and
-z,now to default flags in makepkg.conf. I hope Pacman maintainer will add
those before mass rebuild started so everythig will be done at once.
[1] https://lists.archlinux.org/pipermail/arch-dev-
public/2016-October/028405.html
\-- Sent using MsgSafe.io's Free Plan Private, encrypted, online communication
For everyone. https://www.msgsafe.io
It is extremely hard to keep track of what you wrote here, and what you
are quoting from elsewhere (and who and where those quotes come from).
Can you please use an email client that actually works?

Thanks.
--
Eli Schwartz
Jordan Glover via arch-general
2017-07-06 14:27:47 UTC
Permalink
Raw Message
I just looked into it and created simple patch. Anyone could test it and/or submit upstream?

Index: include/clang/Driver/Options.td
--- include/clang/Driver/Options.td
+++ include/clang/Driver/Options.td
@@ -2497,6 +2497,7 @@
defm non_call_exceptions : BooleanFFlag<"non-call-exceptions">, Group<clang_ignored_f_Group>;
defm peel_loops : BooleanFFlag<"peel-loops">, Group<clang_ignored_gcc_optimization_f_Group>;
defm permissive : BooleanFFlag<"permissive">, Group<clang_ignored_f_Group>;
+defm plt : BooleanFFlag<"plt">, Group<clang_ignored_f_Group>;
defm prefetch_loop_arrays : BooleanFFlag<"prefetch-loop-arrays">, Group<clang_ignored_gcc_optimization_f_Group>;
defm printf : BooleanFFlag<"printf">, Group<clang_ignored_f_Group>;
defm profile : BooleanFFlag<"profile">, Group<clang_ignored_f_Group>;

Index: test/Driver/clang_f_opts.c
--- test/Driver/clang_f_opts.c
+++ test/Driver/clang_f_opts.c
@@ -275,12 +275,14 @@
// RUN: -fno-fat-lto-objects -ffat-lto-objects \
// RUN: -fno-merge-constants -fmerge-constants \
// RUN: -fno-caller-saves -fcaller-saves \
+// RUN: -fno-plt \
// RUN: -fno-reorder-blocks -freorder-blocks \
// RUN: -fno-schedule-insns2 -fschedule-insns2 \
// RUN: -fno-stack-check \
// RUN: -fno-check-new -fcheck-new \
// RUN: -ffriend-injection \
// RUN: -fno-implement-inlines -fimplement-inlines \
+// RUN: -fplt \
// RUN: -fstack-check \
// RUN: -fforce-addr \
// RUN: -malign-functions=100 \
-------- Original Message --------
Subject: Re: [arch-dev-public] Changing compilation flags
So I think it would be a good idea to flip the default to -z,now in
the
linker if we"re going to use -fno-plt. I think they"d take a patch for
that upstream. Clang issue could be avoided with a 1 line patch adding
another no-op flag and they"d take that upstream. It"s harmless to use
the slower lazy linking calling convention when -fno-plt is passed.
This is literally just +1 LOC for Clang b/c it has a system for adding
no-op flags already, which is mostly used for GCC compatibility.
It even uses it in cases that are quite dubious like making -fstack-
check into a no-op, despite it not just being an optional optimization /
code gen
Jordan Glover via arch-general
2017-07-07 13:39:27 UTC
Permalink
Raw Message
FYI clang devs don't want to take 1 line patch adding another no-op flag upstream.
https://lists.llvm.org/pipermail/cfe-dev/2017-July/054588.html
-------- Original Message --------
Subject: Re: [arch-dev-public] Changing compilation flags
So I think it would be a good idea to flip the default to -z,now in the
linker if we"re going to use -fno-plt. I think they"d take a patch for
that upstream. Clang issue could be avoided with a 1 line patch adding
another no-op flag and they"d take that upstream. It"s harmless to use
the slower lazy linking calling conven
Evangelos Foutras via arch-general
2017-07-07 14:41:13 UTC
Permalink
Raw Message
Post by Jordan Glover via arch-general
FYI clang devs don't want to take 1 line patch adding another no-op flag upstream.
https://lists.llvm.org/pipermail/cfe-dev/2017-July/054588.html
Thanks for trying to push the change upstream. I'm sorry they weren't
very receptive of it; your responses were on point and further
clarified our intended use case.

I'm not sure what Joerg meant by saying "Arch Linux can't do
compiler-specific flag definitions". He could be thinking of a regular
project that can check for supported compiler flags, but AFAIK that
concept is not readily transferable to distro-wide flags. (Even if it
was, it would also apply to stack-check and friends so it's a weak
argument in my opinion.)

At the end of the day, it's not an issue to carry this patch
downstream in Arch. :-)
Jordan Glover via arch-general
2017-07-07 16:17:27 UTC
Permalink
Raw Message
I'm surprised as it seemed to me that Daniel took it for granted that patch like that will get accepted. Anyway it's hard for an outsider to successfully submit anything to big upstream project. I hope you'll be more lucky if/when you decide to upstream your pie/ssp patches.
It would be nice if makepkg have some conditionals i.e. :
if cc=clang then
cflags="-march=x86-64 -mtune=generic -O2 -pipe -fsanitize=safe-stack -fsanitize=cfi -fvisibility=hidden
if cc=gcc then
cflags="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fstack-check
-------- Original Message --------
Subject: Re: [arch-dev-public] Changing compilation flags
Thanks for trying to push the change upstream. I"m sorry they weren"t
very receptive of it; your responses were on point and further
clarified our intended use case.
I"m not sure what Joerg meant by saying "Arch Linux can"t do
compiler-specific flag definitions". He could be thinking of a regular
project that can check for supported compiler flags, but AFAIK that
concept is not readily transferable to distro-wide flags. (Even if it
was, it would also apply to stack-check and friends so it"s a weak
argument in my opinion.)
At the end of the day, it"s not an issue to carry this patch
downstream i
Evangelos Foutras via arch-general
2017-07-08 02:31:08 UTC
Permalink
Raw Message
Post by Jordan Glover via arch-general
I'm surprised as it seemed to me that Daniel took it for granted that
patch like that will get accepted. Anyway it's hard for an outsider to
successfully submit anything to big upstream project. I hope you'll be more
lucky if/when you decide to upstream your pie/ssp patches.
The SSP/PIE patch is a bit of a hack and is specific to Arch and the
architectures we support. As mentioned in the commit message, it's a
temporary fix until upstream makes those parameters configurable at
compile-time (or a better solution is found). This means it's not
upstreamable.
Post by Jordan Glover via arch-general
if cc=clang then
cflags="-march=x86-64 -mtune=generic -O2 -pipe -fsanitize=safe-stack
-fsanitize=cfi -fvisibility=hidden
if cc=gcc then
cflags="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fstack-check
There are multiple ways the compiler can be selected; two of them are: 1)
exporting CC/CXX in the PKGBUILD and 2) the project's configure script
picking one of the available compilers. makepkg can't realistically know
which compiler is going to be used and thus must have only one set of flags
that is supported by both GCC and Clang.
Post by Jordan Glover via arch-general
I'm surprised as it seemed to me that Daniel took it for granted that
patch like that will get accepted. Anyway it's hard for an outsider to
successfully submit anything to big upstream project. I hope you'll be more
lucky if/when you decide to upstream your pie/ssp patches.
if cc=clang then
cflags="-march=x86-64 -mtune=generic -O2 -pipe -fsanitize=safe-stack
-fsanitize=cfi -fvisibility=hidden
if cc=gcc then
cflags="-march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fstack-check
-------- Original Message --------
Subject: Re: [arch-dev-public] Changing compilation flags
Thanks for trying to push the change upstream. I"m sorry they weren"t
very receptive of it; your responses were on point and further
clarified our intended use case.
I"m not sure what Joerg meant by saying "Arch Linux can"t do
compiler-specific flag definitions". He could be thinking of a regular
project that can check for supported compiler flags, but AFAIK that
concept is not readily transferable to distro-wide flags. (Even if it
was, it would also apply to stack-check and friends so it"s a weak
argument in my opinion.)
At the end of the day, it"s not an issue to carry this patch
downstream in Arch. :-)
LoneVVolf
2017-07-08 10:53:34 UTC
Permalink
Raw Message
Post by Evangelos Foutras via arch-general
There are multiple ways the compiler can be selected; two of them are: 1)
exporting CC/CXX in the PKGBUILD and 2) the project's configure script
picking one of the available compilers. makepkg can't realistically know
which compiler is going to be used and thus must have only one set of flags
that is supported by both GCC and Clang.
An alternative solution would be to tailor things to the compiler used
by the majority (gcc) and to avoid clang as much as possible.
That would ocourse require special settings for packages compiled with
clang.

NOTE : whether that is a good idea is another matter, but it IS an option.
Eli Schwartz via arch-general
2017-07-09 03:44:13 UTC
Permalink
Raw Message
Post by LoneVVolf
Post by Evangelos Foutras via arch-general
There are multiple ways the compiler can be selected; two of them are: 1)
exporting CC/CXX in the PKGBUILD and 2) the project's configure script
picking one of the available compilers. makepkg can't realistically know
which compiler is going to be used and thus must have only one set of flags
that is supported by both GCC and Clang.
An alternative solution would be to tailor things to the compiler used
by the majority (gcc) and to avoid clang as much as possible.
That would ocourse require special settings for packages compiled with
clang.
NOTE : whether that is a good idea is another matter, but it IS an option.
AIUI that is already what we do, in the sense that gcc is the assumed
default everywhere, and our (C|CXX|LD)FLAGS are written with the
assumption that gcc is the default compiler.

Any packages that need to deviate from the default makepkg.conf flags
(because they *already* have special settings to use clang, those
settings are in the project build configuration files) can currently
modify those flags on a PKGBUILD-by-PKGBUILD basis, or alternatively we
can try to get clang to deal with basic gcc compatibility in a graceful
manner.
--
Eli Schwartz
Jordan Glover via arch-general
2017-07-09 18:26:12 UTC
Permalink
Raw Message
My idea was to have general CC=, CXX=... options set in makepkg.conf, then makepkg would use:
CFLAGS_GCC when CC=gcc and CFLAGS_CLANG when CC=clang
This way instead of setting common minimal compatible flags we could enable all features that compiler have and maintainer/user would only need to set different CC= option in PKGBUILD or globally in makepkg.conf (it's already possible but flags have to be also manually changed).
-------- Original Message --------
Subject: Re: [arch-general] [arch-dev-public] Changing compilation flags
Post by LoneVVolf
Post by Evangelos Foutras via arch-general
There are multiple ways the compiler can be selected; two of them are: 1)
exporting CC/CXX in the PKGBUILD and 2) the project"s configure script
picking one of the available compilers. makepkg can"t realistically know
which compiler is going to be used and thus must have only one set of flags
that is supported by both GCC and Clang.
An alternative solution would be to tailor things to the compiler used
by the majority (gcc) and to avoid clang as much as possible.
That would ocourse require special settings for packages compiled with
clang.
NOTE : whether that is a good idea is another matter, but it IS an option.
AIUI that is already what we do, in the sense that gcc is the assumed
default everywhere, and our (C|CXX|LD)FLAGS are written with the
assumption that gcc is the default compiler.
Any packages that need to deviate from the default makepkg.conf flags
(because they *already* have special settings to use clang, those
settings are in the project build configuration files) can currently
modify those flags on a PKGBUILD-by-PKGBUILD basis, or alternatively we
can try to get clang to deal with basic gcc
Jordan Glover via arch-general
2017-07-10 20:45:18 UTC
Permalink
Raw Message
I see [1] -fstack-check is dropped and -fstack-protector-strong kept while being redundant. Anyone know what happened?
[1] https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/pacman&id=0cd22d4454e0e1b3ae589b95274f808001465c15

On 2017-06-30 23:44, Allan McRae wrote:
On 30/06/17 19:07, Bartłomiej Piotrowski wrote:
On 2016-10-24 05:56, Allan McRae wrote:
1) building gcc to enable PIE by default
I am in the middle of rebuilding gcc with --enable-default-pie. When it
finishes, I will start a todo for rebuilding packages with static libraries.
I also enabled --enable-default-ssp, which means that
-fstack-protector-strong will be dropped from our CFLAGS (as it will be
enforced by gcc) on the next opportunity.
Are you adding full RELRO + no-plt at the same time?
A
Yes, and -fstack-check=specific too, although I might drop no-plt if it
will cause too many builders.

Bartłom
Jordan Glover via arch-general
2017-07-12 16:25:36 UTC
Permalink
Raw Message
I testes some rebuilded binaries and BINDNOW isn't always enabled:
checksec -f /usr/bin/unrar
RELRO
Partial RELRO
checksec -f /usr/bin/qml (qt5-declarative)
RELRO
Partial RELRO
I don't know if -fno-plt was correctly passed but it's possible that build process doesn't work as intended. Maybe we need to patch binutils to enable z,now by default as Daniel advised?
There"s no loss of compatibility from only some code using it. The only
issue with it is that immediate binding *must* be used to support it, so
if CFLAGS is respected then LDFLAGS *must* be respected, or immediate
binding needs to be set as the default in the linker(s).
So I think it would be a good idea to flip the default to -z,now in the linker
if we're going to use -fno-plt. I think they'd take a patch for that upstream.
Clang issue could be avoided with a 1 line patch adding another no-op flag
and they'd take that upstream. It's harmless to use the slower lazy linking
calling convention when -fno-plt is passed. -fno-plt code is fully compatible
with non -fno-plt code, the only requirement is that -fno-plt code is linked
with -Wl,-z,now which works fine for non -fno-plt code too and is desirable
for security eithe
Bartłomiej Piotrowski
2017-07-13 06:29:27 UTC
Permalink
Raw Message
Post by Jordan Glover via arch-general
checksec -f /usr/bin/unrar
RELRO
Partial RELRO
checksec -f /usr/bin/qml (qt5-declarative)
RELRO
Partial RELRO
I don't know if -fno-plt was correctly passed but it's possible that build process doesn't work as intended. Maybe we need to patch binutils to enable z,now by default as Daniel advised?
There"s no loss of compatibility from only some code using it. The only
issue with it is that immediate binding *must* be used to support it, so
if CFLAGS is respected then LDFLAGS *must* be respected, or immediate
binding needs to be set as the default in the linker(s).
So I think it would be a good idea to flip the default to -z,now in the linker
if we're going to use -fno-plt. I think they'd take a patch for that upstream.
Clang issue could be avoided with a 1 line patch adding another no-op flag
and they'd take that upstream. It's harmless to use the slower lazy linking
calling convention when -fno-plt is passed. -fno-plt code is fully compatible
with non -fno-plt code, the only requirement is that -fno-plt code is linked
with -Wl,-z,now which works fine for non -fno-plt code too and is desirable
for security either way.
The ongoing rebuild isn't about getting full RELRO, but recompiling
static libraries with PIE enabled. Another one that aims at removing
hardening-wrapper from repositories requires more attention to upstream
projects ignoring our compilation flags.

The plan is to modify binutils to enable BIND_NOW (for which Daniel
already provided me a patch), but it also requires some changes to make
test suite happy, so most likely I will work on it next Friday.
Post by Jordan Glover via arch-general
I see [1] -fstack-check is dropped and -fstack-protector-strong kept while being redundant. Anyone know what happened?
Current -fstack-check implementation is flawed[1] and after discussion
with Allan and Daniel, we have decided not to include it now. As even
now it would require entire repository to be rebuild, I don't see a
reason to do it now. I will get back to it when it's fixed.

[1] http://www.openwall.com/lists/oss-security/2017/06/19/9

Alex Xu via arch-general
2017-07-12 23:19:44 UTC
Permalink
Raw Message
spec file? only supported by gcc, but that seems to be kinda the point
here. I think this is what most distros do (and possibly also how
--enable-default-* works).
Loading...