Discussion:
gcc broken - Arch specific, applying optimization incorrectly - may explain unexplained problems
(too old to reply)
David C. Rankin
2018-03-13 06:36:40 UTC
Permalink
Houston -- we have a problem.

The problem is gcc is mis-applying optimizations, and it seems Arch
specific. See discussion:

[Why does not execute printf inside function?]
https://stackoverflow.com/questions/49248787/why-does-not-execute-printf-inside-function/49249533#49249533

The basic issue can be summarized in this short snippet:

#include <stdio.h>

int arr[10];

int func()
{
printf ("In func\n");
return 0;
}

int main (void)
{
if (&arr[func()])
printf("In main\n");
return 0;
}

Compile (with or without optimization) and the output on Archlinux (all
packages current as of 1hr ago) and you get:

$ ./bin/if2
In main

That is wrong. `func()` has been completely optimized out of the procedure
in main(). The assembly generated omits any call to func. (full assembly
posted as part of the answer on SO)

Debian gcc 7.3 does not exhibit this same behavior. I do not know where to
start looking to find why this is happening, but it does not take much of a
stretch to see how this could be the root cause of some of the "unexplained"
errors I and others have seen in the past few days on Arch.

Let me know if I need to file a bug here. Since it "works on Debian" and not
here, I suspect the bug should start here and so the default optimization we
package with can be eliminated.

What say the devs?
--
David C. Rankin, J.D.,P.E.
PkmX via arch-general
2018-03-13 07:17:47 UTC
Permalink
Hi,

AFAIK this is the exact case of gcc bugzilla #84607:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607

As this is an upstream bug this should affect all distributions, maybe
the commenter on Debian is using 7.3.1 so he can't reproduce the
issue?

On Tue, Mar 13, 2018 at 2:36 PM, David C. Rankin
Post by David C. Rankin
Houston -- we have a problem.
The problem is gcc is mis-applying optimizations, and it seems Arch
[Why does not execute printf inside function?]
https://stackoverflow.com/questions/49248787/why-does-not-execute-printf-inside-function/49249533#49249533
#include <stdio.h>
int arr[10];
int func()
{
printf ("In func\n");
return 0;
}
int main (void)
{
if (&arr[func()])
printf("In main\n");
return 0;
}
Compile (with or without optimization) and the output on Archlinux (all
$ ./bin/if2
In main
That is wrong. `func()` has been completely optimized out of the procedure
in main(). The assembly generated omits any call to func. (full assembly
posted as part of the answer on SO)
Debian gcc 7.3 does not exhibit this same behavior. I do not know where to
start looking to find why this is happening, but it does not take much of a
stretch to see how this could be the root cause of some of the "unexplained"
errors I and others have seen in the past few days on Arch.
Let me know if I need to file a bug here. Since it "works on Debian" and not
here, I suspect the bug should start here and so the default optimization we
package with can be eliminated.
What say the devs?
--
David C. Rankin, J.D.,P.E.
Carsten Mattner via arch-general
2018-03-13 07:49:53 UTC
Permalink
Post by PkmX via arch-general
Hi,
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
As this is an upstream bug this should affect all distributions, maybe
the commenter on Debian is using 7.3.1 so he can't reproduce the
issue?
Let's not forget that Debian is patch happy (as is Ubuntu, logically),
especially for gcc. Mostly for good reason but quite a bit is too
many diffs:

https://sources.debian.org/src/gcc-7/7.3.0-11/debian/patches/

It's quite possible that the snapshot plus diffs fixes/hides the bug
on Debian. Fedora is another distro that is very happy to carry
many diffs for some packages. Thinks of the version string like
git-gc-7.3.1+dirty in mainstream distros. It's like a downstream
branch.
Jelle van der Waa
2018-03-13 08:02:31 UTC
Permalink
Post by PkmX via arch-general
Hi,
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
As this is an upstream bug this should affect all distributions, maybe
the commenter on Debian is using 7.3.1 so he can't reproduce the
issue?
BTW, cross checking with clang was also an option :-)

[***@starfighter][/tmp]%clang foo.c
[***@starfighter][/tmp]%./a.out
In func
In main

Feel free to make a bugreport on https://bugs.archlinux.org
David C. Rankin
2018-03-13 08:10:50 UTC
Permalink
Post by PkmX via arch-general
Hi,
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
As this is an upstream bug this should affect all distributions, maybe
the commenter on Debian is using 7.3.1 so he can't reproduce the
issue?
That is good to know that it is being addressed. I can see this current
handling of optimizations causing many of subtle non-traceable and just plain
"wonky" errors that have been see these past few days. I thought the guy who
posted it was nuts... It built and had the correct output on SuSE, but when I
tried it on Arch - WTF? they was right. Who knows in how many subtle ways that
could manifest itself. I've convinced it is as the root of the (Out of IOMMU
space) error I have in the logs, and more than likely it is responsible for my
hardlocks on the big servers. We will be looking for a fix -- hopefully soon.

Thank you for letting me know.
--
David C. Rankin, J.D.,P.E.
mar77i via arch-general
2018-03-13 20:28:43 UTC
Permalink
Post by PkmX via arch-general
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
Pass -std=... to compiling the programi n question. If it's in clear deviation of what the respective standard dictates, it's a bug.


​Sent with ProtonMail Secure Email.​
David C. Rankin
2018-03-14 01:44:16 UTC
Permalink
Post by mar77i via arch-general
Post by PkmX via arch-general
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
Pass -std=... to compiling the programi n question. If it's in clear deviation of what the respective standard dictates, it's a bug.
​Sent with ProtonMail Secure Email.​
Yes, confirmed it was indeed a bug:

$ gcc -Wall -Wextra -pedantic -std=c11 -o bin/infunc2 infunc2.c
$ ./bin/infunc2
In main

And, with Jelle van der Waa's suggestion, I loaded clang (which I never
otherwise install) and it confirms the issue further:

$ clang -Wall -Wextra -pedantic -std=c11 -o infunc2 infunc2.c
$ ./infunc2
In func
In main

The only part I'm unclear on is where in the fix process
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607 is on getting fixed and
when that will trickle down to us.
--
David C. Rankin, J.D.,P.E.
Jelle van der Waa
2018-03-14 08:26:53 UTC
Permalink
Post by David C. Rankin
Post by mar77i via arch-general
Post by PkmX via arch-general
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
Pass -std=... to compiling the programi n question. If it's in clear deviation of what the respective standard dictates, it's a bug.
​Sent with ProtonMail Secure Email.​
$ gcc -Wall -Wextra -pedantic -std=c11 -o bin/infunc2 infunc2.c
$ ./bin/infunc2
In main
And, with Jelle van der Waa's suggestion, I loaded clang (which I never
$ clang -Wall -Wextra -pedantic -std=c11 -o infunc2 infunc2.c
$ ./infunc2
In func
In main
The only part I'm unclear on is where in the fix process
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607 is on getting fixed and
when that will trickle down to us.
In the comments it reads: "Fixed on the trunk." In commit https://gcc.gnu.org/viewcvs/gcc?view=revision&revision=258061

As I suggested, create a bugtracker issue for gcc on our bugtracker
otherwise you'll have to wait on a new gcc minor release.
--
Jelle van der Waa
Eli Schwartz via arch-general
2018-03-14 11:08:54 UTC
Permalink
Post by Jelle van der Waa
As I suggested, create a bugtracker issue for gcc on our bugtracker
otherwise you'll have to wait on a new gcc minor release.
Someone already did.

https://bugs.archlinux.org/task/57823
--
Eli Schwartz
Bug Wrangler and Trusted User
Bastian Beischer
2018-03-14 08:44:57 UTC
Permalink
Hi David,

On Wed, Mar 14, 2018 at 2:44 AM, David C. Rankin
Post by David C. Rankin
Post by mar77i via arch-general
Post by PkmX via arch-general
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607
Pass -std=... to compiling the programi n question. If it's in clear deviation of what the respective standard dictates, it's a bug.
Sent with ProtonMail Secure Email.
$ gcc -Wall -Wextra -pedantic -std=c11 -o bin/infunc2 infunc2.c
$ ./bin/infunc2
In main
And, with Jelle van der Waa's suggestion, I loaded clang (which I never
$ clang -Wall -Wextra -pedantic -std=c11 -o infunc2 infunc2.c
$ ./infunc2
In func
In main
The only part I'm unclear on is where in the fix process
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=84607 is on getting fixed and
when that will trickle down to us.
The fix is available in the trunk and the gcc7 branch, but there
hasn't been a release yet. It will be in GCC 7.4.0 and GCC 8.1.0 when
they come. If you need the fix before that you need to open a ticket
on bugs.archlinux.org and ask for a backport of that particular
bugfix.

However, whether this will actually solve any real world problems in
your case: I am doubtful. Do you have any specific evidence which
links this particular bug to your symptoms? Please have a look around
the gcc bug tracker: You will see that there are hundreds of compiler
bugs, many of which more severe than the one you have quoted, but that
doesn't mean that Arch actually backports all of these individual
fixes all the time. I'm not saying it's impossible that the fix will
help you, I'm just saying it is not very likely. And if indeed your
symptoms are due to a gcc miscompilation, you would need recompiled
binaries of whatever software causes your problems as well.
Post by David C. Rankin
--
David C. Rankin, J.D.,P.E.
Cheers
Bastian
David C. Rankin
2018-03-14 23:03:00 UTC
Permalink
Post by Bastian Beischer
The fix is available in the trunk and the gcc7 branch, but there
hasn't been a release yet. It will be in GCC 7.4.0 and GCC 8.1.0 when
they come. If you need the fix before that you need to open a ticket
on bugs.archlinux.org and ask for a backport of that particular
bugfix.
Yes, thanks, I see that somebody beat me to it. Thank you.

https://bugs.archlinux.org/index.php?do=details&action=details.addvote&task_id=57823
--
David C. Rankin, J.D.,P.E.
Eli Schwartz via arch-general
2018-03-14 23:52:28 UTC
Permalink
Post by David C. Rankin
Post by Bastian Beischer
The fix is available in the trunk and the gcc7 branch, but there
hasn't been a release yet. It will be in GCC 7.4.0 and GCC 8.1.0 when
they come. If you need the fix before that you need to open a ticket
on bugs.archlinux.org and ask for a backport of that particular
bugfix.
Yes, thanks, I see that somebody beat me to it. Thank you.
https://bugs.archlinux.org/index.php?do=details&action=details. ohnoes addvote&task_id=57823
You just gave the exact same link as I gave 12 hours ago, except your
link (which I have broken in my quote by adding the string " ohnoes ")
will cause people who are logged in, to inadvertently interact with the
bugtracker by adding their vote for the bug.

This was *probably* an accident (flyspray links are sort of weird for
reloading the page and replacing the current url with this at all);
nevertheless, please don't do this again...
--
Eli Schwartz
Bug Wrangler and Trusted User
David C. Rankin
2018-03-15 00:40:40 UTC
Permalink
On 03/14/2018 06:52 PM, Eli Schwartz via arch-general wrote:

<quote>
This was *probably* an accident (flyspray links are sort of weird for
reloading the page and replacing the current url with this at all);
nevertheless, please don't do this again...
</quote>

Oh, sorry, and yes this was definitely an accident. While I see the logic
behind the *probably* comment, I assure you that wasn't intention, I simply
double-clicked the content of the address window and pasted -- will make sure
that does not happen again....
--
David C. Rankin, J.D.,P.E.
Eli Schwartz via arch-general
2018-03-15 01:41:06 UTC
Permalink
Post by David C. Rankin
<quote>
This was *probably* an accident (flyspray links are sort of weird for
reloading the page and replacing the current url with this at all);
nevertheless, please don't do this again...
</quote>
Oh, sorry, and yes this was definitely an accident. While I see the logic
behind the *probably* comment, I assure you that wasn't intention, I simply
double-clicked the content of the address window and pasted -- will make sure
that does not happen again....
:p

Good news, I guess, is that we will almost certainly switch to bugzilla
as soon as Harmony brings stable psgi support (Soon™).
--
Eli Schwartz
Bug Wrangler and Trusted User
Carsten Mattner via arch-general
2018-03-15 21:29:19 UTC
Permalink
Post by Eli Schwartz via arch-general
Good news, I guess, is that we will almost certainly switch to bugzilla
as soon as Harmony brings stable psgi support (Soon™).
Harmony?
PSGI?

I like Bugzilla's features, and I don't use arch's tracker enough to have
a voice, or maintain it to have a say, but I would still like to provide
an opinion. It's just an opinion. Having used large bugzilla installations
like mozilla.org, kernel.org, freedesktop.org, it doesn't look like Bugzilla
can be fast. I've found Mantis and to some extend Redmine to be fast and
lean trackers. These may or may not suit Arch's needs, and I still have
hope Bugzilla will at some point become snappy.

Just an opinion based on experience, the arch admins/devs will know why
bugzilla is the right choice. I trust them.
Carsten Mattner via arch-general
2018-03-16 07:38:49 UTC
Permalink
I should note that Jira takes the position of being the slowest bugtracker.
Nobody would use git if it was slow like Jira or Bugzilla.
Jelle van der Waa
2018-03-16 09:47:35 UTC
Permalink
Post by Carsten Mattner via arch-general
Post by Eli Schwartz via arch-general
Good news, I guess, is that we will almost certainly switch to bugzilla
as soon as Harmony brings stable psgi support (Soon™).
Harmony?
PSGI?
I like Bugzilla's features, and I don't use arch's tracker enough to have
a voice, or maintain it to have a say, but I would still like to provide
an opinion. It's just an opinion. Having used large bugzilla installations
like mozilla.org, kernel.org, freedesktop.org, it doesn't look like Bugzilla
can be fast. I've found Mantis and to some extend Redmine to be fast and
lean trackers. These may or may not suit Arch's needs, and I still have
hope Bugzilla will at some point become snappy.
Just an opinion based on experience, the arch admins/devs will know why
bugzilla is the right choice. I trust them.
Please don't de-rail the thread. (Obviously the Arch devs also want a
performing bugtracker, but Flyspray has to go either way)
--
Jelle van der Waa
Loading...