Discussion:
Kernel source URL change
(too old to reply)
Andrey Vihrov via arch-general
2018-08-01 20:41:12 UTC
Permalink
Raw Message
Hi,

Recently the way kernel sources are retrieved was changed in the linux
package [1]. Now the sources are fetched from
https://github.com/archlinux/linux.

I see a few problems with this:

- Previously the list of applied patches was very transparent. You could
immediately see that the kernel and kernel patch tarballs come from
kernel.org, and view individual extra patches. Now the code comes from a
non-kernel source, and cannot be verified as easily.

- Previously, if a new kernel version is released and is not yet in the
repos, you could more or less take the official linux PKGBUILD, change
one number and build it yourself. With the new layout it is not clear
how to achieve this.

- An often cited Arch policy is to use software as released by upstream
with minimal patching. What becomes of this policy if one of the core
packages builds from a technical fork instead of upstream?


If the patches from kernel.org will no longer be signed, as announced in
[2], then an alternative would be git tags from [3] and [4]. It's
understandable if it may make development harder, nonetheless it would
allow for better transparency and follow upstream closer — just one
user's opinion.


[1]
https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/linux&id=d0c4ab0716e0ae1fc058a83ccb02bde92885ced6
[2] https://www.kernel.org/minor-changes-to-tarball-release-format.html
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/
[4] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git

--
Regards,
Andrey
Tharre via arch-general
2018-08-01 22:05:25 UTC
Permalink
Raw Message
Post by Andrey Vihrov via arch-general
- Previously the list of applied patches was very transparent. You could
immediately see that the kernel and kernel patch tarballs come from
kernel.org, and view individual extra patches. Now the code comes from a
non-kernel source, and cannot be verified as easily.
Just run `git log v$_srcver`. That will show you all the patches that
have been applied to the upstream release.

That the sources do no longer come from kernel.org is irrelevant, you
should _always_ verify the gpg signature for auditing purposes.
Post by Andrey Vihrov via arch-general
- Previously, if a new kernel version is released and is not yet in the
repos, you could more or less take the official linux PKGBUILD, change
one number and build it yourself. With the new layout it is not clear
how to achieve this.
git-rebase(1) will accomplish that.
Post by Andrey Vihrov via arch-general
- An often cited Arch policy is to use software as released by upstream
with minimal patching. What becomes of this policy if one of the core
packages builds from a technical fork instead of upstream?
Nobody forked the linux kernel, and no new patches have been added. The
resulting package is exactly the same, so nothing changed in that
regard.

Regards,
Tharre
--
PGP fingerprint: 42CE 7698 D6A0 6129 AA16 EF5C 5431 BDE2 C8F0 B2F4
Leonidas Spyropoulos via arch-general
2018-08-02 05:28:18 UTC
Permalink
Raw Message
Post by Andrey Vihrov via arch-general
Hi,
Recently the way kernel sources are retrieved was changed in the linux
package [1]. Now the sources are fetched from
https://github.com/archlinux/linux.
- Previously the list of applied patches was very transparent. You could
immediately see that the kernel and kernel patch tarballs come from
kernel.org, and view individual extra patches. Now the code comes from a
non-kernel source, and cannot be verified as easily.
- Previously, if a new kernel version is released and is not yet in the
repos, you could more or less take the official linux PKGBUILD, change
one number and build it yourself. With the new layout it is not clear
how to achieve this.
- An often cited Arch policy is to use software as released by upstream
with minimal patching. What becomes of this policy if one of the core
packages builds from a technical fork instead of upstream?
If the patches from kernel.org will no longer be signed, as announced in
[2], then an alternative would be git tags from [3] and [4]. It's
understandable if it may make development harder, nonetheless it would
allow for better transparency and follow upstream closer — just one
user's opinion.
[1]
https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/linux&id=d0c4ab0716e0ae1fc058a83ccb02bde92885ced6
[2] https://www.kernel.org/minor-changes-to-tarball-release-format.html
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/
[4] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
--
Regards,
Andrey
Also now to build the package locally you download the whole repository
(~2 Gb compared to the ~110 Mb previously).

What's the reasoning behind this change?

Regards,
--
Leonidas Spyropoulos

A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
Joan Aymà via arch-general
2018-08-02 07:11:05 UTC
Permalink
Raw Message
The size problem can be solved using hollow clone.

Regards.

On Thu, 2 Aug 2018, 07:28 Leonidas Spyropoulos via arch-general, <
Post by Andrey Vihrov via arch-general
Post by Andrey Vihrov via arch-general
Hi,
Recently the way kernel sources are retrieved was changed in the linux
package [1]. Now the sources are fetched from
https://github.com/archlinux/linux.
- Previously the list of applied patches was very transparent. You could
immediately see that the kernel and kernel patch tarballs come from
kernel.org, and view individual extra patches. Now the code comes from a
non-kernel source, and cannot be verified as easily.
- Previously, if a new kernel version is released and is not yet in the
repos, you could more or less take the official linux PKGBUILD, change
one number and build it yourself. With the new layout it is not clear
how to achieve this.
- An often cited Arch policy is to use software as released by upstream
with minimal patching. What becomes of this policy if one of the core
packages builds from a technical fork instead of upstream?
If the patches from kernel.org will no longer be signed, as announced in
[2], then an alternative would be git tags from [3] and [4]. It's
understandable if it may make development harder, nonetheless it would
allow for better transparency and follow upstream closer — just one
user's opinion.
[1]
https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/linux&id=d0c4ab0716e0ae1fc058a83ccb02bde92885ced6
Post by Andrey Vihrov via arch-general
[2] https://www.kernel.org/minor-changes-to-tarball-release-format.html
[3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/
[4] https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git
--
Regards,
Andrey
Also now to build the package locally you download the whole repository
(~2 Gb compared to the ~110 Mb previously).
What's the reasoning behind this change?
Regards,
--
Leonidas Spyropoulos
A: Because it messes up the order in which people normally read text.
Q: Why is it such a bad thing?
A: Top-posting.
Q: What is the most annoying thing on usenet and in e-mail?
Jens John
2018-08-02 17:05:13 UTC
Permalink
Raw Message
Post by Joan Aymà via arch-general
The size problem can be solved using hollow clone.
Not in a well-formed PKGBUILD. There's ample discussion on this topic available. Of course you're welcome to hack together whatever you want.
Konstantin Shalygin via arch-general
2018-08-07 05:31:55 UTC
Permalink
Raw Message
Post by Leonidas Spyropoulos via arch-general
Also now to build the package locally you download the whole repository
(~2 Gb compared to the ~110 Mb previously).
Oh...


Cloning into bare repository '/tmp/4.17-arch/archlinux-linux'...
remote: Counting objects: 6135977, done.
remote: Compressing objects: 100% (1099/1099), done.
remote: Total 6135977 (delta 984), reused 385 (delta 385), pack-reused
6134493
Receiving objects: 100% (6135977/6135977), 2.07 GiB | 1.03 MiB/s, done.
Resolving deltas: 100% (5111928/5111928), done.


33 minutes to fetch kernel.

If this changeset is good for arch infrastructure builds, may be tar.xz
will be also provided, via github releases for example.




k
W B via arch-general
2018-08-07 23:31:12 UTC
Permalink
Raw Message
Now we know that the change is a solution in search of a problem.

The fact that they won't sign the patch tarballs anymore isn't a problem.

All that heftig must do, is to make it download the complete tarball and its signature.

Here you go:
https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.13.tar.xz
https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.13.tar.sign

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
Post by Konstantin Shalygin via arch-general
Post by Leonidas Spyropoulos via arch-general
Also now to build the package locally you download the whole repository
(~2 Gb compared to the ~110 Mb previously).
Oh...
Cloning into bare repository '/tmp/4.17-arch/archlinux-linux'...
remote: Counting objects: 6135977, done.
remote: Compressing objects: 100% (1099/1099), done.
remote: Total 6135977 (delta 984), reused 385 (delta 385), pack-reused
6134493
Receiving objects: 100% (6135977/6135977), 2.07 GiB | 1.03 MiB/s, done.
Resolving deltas: 100% (5111928/5111928), done.
33 minutes to fetch kernel.
If this changeset is good for arch infrastructure builds, may be tar.xz
will be also provided, via github releases for example.
k
Eli Schwartz via arch-general
2018-08-08 00:11:06 UTC
Permalink
Raw Message
Post by W B via arch-general
Now we know that the change is a solution in search of a problem.
The fact that they won't sign the patch tarballs anymore isn't a problem.
All that heftig must do, is to make it download the complete tarball and its signature.
https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.13.tar.xz
https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.13.tar.sign
We do not listen to your orders.
--
Eli Schwartz
Bug Wrangler and Trusted User
W B via arch-general
2018-08-08 02:31:22 UTC
Permalink
Raw Message
It isn't an order.

Can you tell us why this change was required, please?

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
Post by Eli Schwartz via arch-general
Post by W B via arch-general
Now we know that the change is a solution in search of a problem.
The fact that they won't sign the patch tarballs anymore isn't a problem.
All that heftig must do, is to make it download the complete tarball and its signature.
https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.13.tar.xz
https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.17.13.tar.sign
We do not listen to your orders.
-----------------------------------
Eli Schwartz
Bug Wrangler and Trusted User
Giancarlo Razzolini via arch-general
2018-08-08 02:54:11 UTC
Permalink
Raw Message
Post by W B via arch-general
It isn't an order.
Can you tell us why this change was required, please?
Have you read the original post to the list? Specially this [0]?

Those tar files you just linked are not signed by Linus anymore, they are signed
instead by Greg Kroah-Hartman. You would have known this if you bothered to actually
download them and check the signature.

Another reason for this move is to apply our patches as commits. You can use any other
kernel if you want.

[0] https://www.kernel.org/minor-changes-to-tarball-release-format.html

Cheers,
Giancarlo Razzolini
Eli Schwartz via arch-general
2018-08-08 02:55:55 UTC
Permalink
Raw Message
Post by W B via arch-general
It isn't an order.
Can you tell us why this change was required, please?
Because heftig decided it was easier *for him* to do it this way.

Because downloading 100 MB for every single patchlevel release quickly
builds up to just as much as a full git clone.

Can you tell us why you believe the move to git was "a solution in
search of a problem"?
Can you tell us why you replied to this list with the message that there
are full tarballs, as though you think heftig did not know this and
reject this already? Who are you trying to convince here?
--
Eli Schwartz
Bug Wrangler and Trusted User
Joakim Hernberg
2018-08-08 10:03:33 UTC
Permalink
Raw Message
On Tue, 7 Aug 2018 22:55:55 -0400
Post by Eli Schwartz via arch-general
Because heftig decided it was easier *for him* to do it this way.
Because downloading 100 MB for every single patchlevel release quickly
builds up to just as much as a full git clone.
Can you tell us why you believe the move to git was "a solution in
search of a problem"?
I wonder if this isn't going to complicate my life maintaining the
linux-rt kernel on AUR. So far I've tried to track the linux package
closely, but I wonder what repercussions it will have. Many times it's
also out of sync with the main distro kernel, but I always used to pick
up the patches etc used by the main kernel.

Oh well, nor really a complaint, let's see how it works out in practice.
--
Joakim
W B via arch-general
2018-08-03 15:36:34 UTC
Permalink
Raw Message
This looks like a solution in search of a problem.

hefti
Geo Kozey via arch-general
2018-08-08 11:43:08 UTC
Permalink
Raw Message
Post by Giancarlo Razzolini via arch-general
Post by W B via arch-general
It isn't an order.
Post by W B via arch-general
Can you tell us why this change was required, please?
Have you read the original post to the list? Specially this [0]?
The author of original post was only speculating about possible reasons for the recent
changes. He also asked few questions which weren't answered.
Post by Giancarlo Razzolini via arch-general
Those tar files you just linked are not signed by Linus anymore, they are signed
instead by Greg Kroah-Hartman. You would have known this if you bothered to actually
download them and check the signature.
Greg Kroah-Hartman PGP key was already included as validpkgkey inside PKGBUILD so there
is no real argument here.
Post by Giancarlo Razzolini via arch-general
Another reason for this move is to apply our patches as commits. You can use any other
kernel if you want.
There is no tradition in Arch to self-host package sources as Debian does unless upstream has
completely broken release process. This can impose security risks on Arch as we now have to
trust their github infra rather than kernel.org (we all know what happened to gentoo recently).
I'm aware that Barthalion made an effort to hardenize Arch github infra but still this is a new risk
which didn't exist before.

Is it general Arch move to self-host sources and applying patches as commits or will linux kernel
package stay as outlier?
Post by Giancarlo Razzolini via arch-general
[0] https://www.kernel.org/minor-changes-to-tarball-release-format.html
Cheers,
Giancarlo Razzolini
Yours sincerely

G. K.
Ralf Mardorf
2018-08-08 12:05:22 UTC
Permalink
Raw Message
The author of original post [snip] asked few questions which weren't
answered.
Hi,

the OP did ask how to build a custom kernel based on the official linux
package [1]. Perhaps somebody with unobjectionable knowledge could
correct related Wiki pages, at least
https://wiki.archlinux.org/index.php/Kernels/Arch_Build_System .

TIA,
Ralf

[1]
Previously, if a new kernel version is released and is not yet in the
repos, you could more or less take the official linux PKGBUILD, change
one number and build it yourself. With the new layout it is not clear
how to achieve this.
Jonathon Fernyhough
2018-08-08 16:09:30 UTC
Permalink
Raw Message
Post by Geo Kozey via arch-general
This can impose security risks on Arch as we now have to
trust their github infra rather than kernel.org (we all know what happened to gentoo recently)
Just to provide some perspective, kernel.org itself had a major issue a
few years back [1][2][3]. kernel.org was down for several weeks after
that incident, and IIRC this prompted them to start using GitHub (at
least as a mirror; my memory is fuzzy as I wasn't paying all that much
attention to that sort of thing seven years ago).

If you don't trust the Arch-run/administered infrastructure you can't
really trust any of the packages in the repos either.

[1] https://www.theregister.co.uk/2011/08/31/linux_kernel_security_breach/
[2] https://en.wikipedia.org/wiki/Kernel.org
[3] https://www.linuxfoundation.org/blog/2011/08/the-cracking-of-kernel-org/
Geo Kozey via arch-general
2018-08-08 16:47:42 UTC
Permalink
Raw Message
Sent: Wed Aug 08 18:09:30 CEST 2018
Subject: Re: [arch-general] Kernel source URL change
Post by Geo Kozey via arch-general
This can impose security risks on Arch as we now have to
trust their github infra rather than kernel.org (we all know what happened to gentoo recently)
Just to provide some perspective, kernel.org itself had a major issue a
few years back [1][2][3]. kernel.org was down for several weeks after
that incident, and IIRC this prompted them to start using GitHub (at
least as a mirror; my memory is fuzzy as I wasn't paying all that much
attention to that sort of thing seven years ago).
IIRC in 2011 Arch didn't even used gpg for signing packages so it's quite ancient time.
If you don't trust the Arch-run/administered infrastructure you can't
really trust any of the packages in the repos either.
The point was that before changes no user had to care about https://github.com/Archlinux
and now it's critical infrastructure for self-hosting package sources.
[1] https://www.theregister.co.uk/2011/08/31/linux_kernel_security_breach/
[2] https://en.wikipedia.org/wiki/Kernel.org
[3] https://www.linuxfoundation.org/blog/2011/08/the-cracking-of-kernel-org/
Yours sincerely

G. K.
Tharre via arch-general
2018-08-08 20:11:23 UTC
Permalink
Raw Message
Post by Geo Kozey via arch-general
There is no tradition in Arch to self-host package sources as Debian does unless upstream has
completely broken release process. This can impose security risks on Arch as we now have to
trust their github infra rather than kernel.org (we all know what happened to gentoo recently).
I'm aware that Barthalion made an effort to hardenize Arch github infra but still this is a new risk
which didn't exist before.
[...]
Post by Geo Kozey via arch-general
The point was that before changes no user had to care about https://github.com/Archlinux
and now it's critical infrastructure for self-hosting package sources.
No, nobody has to trust github or for that fact kernel.org. The
commits/tags are *signed* and thus makepkg will check if that signature
matches one of those specified in the validpgpkeys array.

From a security standpoint, it's irrelevant if the sources come from
arch hosted infra, from github, or from kernel.org.

Regards,
Tharre
--
PGP fingerprint: 42CE 7698 D6A0 6129 AA16 EF5C 5431 BDE2 C8F0 B2F4
Eli Schwartz via arch-general
2018-08-08 20:17:09 UTC
Permalink
Raw Message
Post by Tharre via arch-general
Post by Geo Kozey via arch-general
There is no tradition in Arch to self-host package sources as Debian does unless upstream has
completely broken release process. This can impose security risks on Arch as we now have to
trust their github infra rather than kernel.org (we all know what happened to gentoo recently).
I'm aware that Barthalion made an effort to hardenize Arch github infra but still this is a new risk
which didn't exist before.
[...]
Post by Geo Kozey via arch-general
The point was that before changes no user had to care about https://github.com/Archlinux
and now it's critical infrastructure for self-hosting package sources.
No, nobody has to trust github or for that fact kernel.org. The
commits/tags are *signed* and thus makepkg will check if that signature
matches one of those specified in the validpgpkeys array.
From a security standpoint, it's irrelevant if the sources come from
arch hosted infra, from github, or from kernel.org.
I'm all for hosting it through bittorrent TBH.
--
Eli Schwartz
Bug Wrangler and Trusted User
Loading...