Discussion:
About rebuild of pandoc
(too old to reply)
Óscar García Amor
2017-06-26 06:45:46 UTC
Permalink
Raw Message
Hello all,

Some days ago the pandoc mantainer [1] do a rebuild of it [2] where
add a lot of haskell package dependencies. I think that the build
changes the binary from statically linked to dinamically linked, but
IMHO, I prefer the static one (55,08 MiB of package) over the dinamic
(more than 666 MB in libraries).

What do you think about this?

Other solution can be have other package "pandoc-static", that
maintains the previous method of package.

Greetings.

[1]: https://www.archlinux.org/packages/community/x86_64/pandoc/
[2]: https://git.archlinux.org/svntogit/community.git/commit/trunk?h=packages/pandoc&id=d340c92f8cf5686509551c08bcdaa0b5e66760b0
--
Óscar García Amor | ogarcia at moire.org | http://ogarcia.me
Eli Schwartz via arch-general
2017-06-26 12:23:16 UTC
Permalink
Raw Message
Post by Óscar García Amor
Hello all,
Some days ago the pandoc mantainer [1] do a rebuild of it [2] where
add a lot of haskell package dependencies. I think that the build
changes the binary from statically linked to dinamically linked, but
IMHO, I prefer the static one (55,08 MiB of package) over the dinamic
(more than 666 MB in libraries).
What do you think about this?
Other solution can be have other package "pandoc-static", that
maintains the previous method of package.
Greetings.
[1]: https://www.archlinux.org/packages/community/x86_64/pandoc/
[2]: https://git.archlinux.org/svntogit/community.git/commit/trunk?h=packages/pandoc&id=d340c92f8cf5686509551c08bcdaa0b5e66760b0
And same with shellcheck -- the general issue is that *all*
haskell-based packages now build dynamically linked against the haskell
runtime (which is huge, and few people have more than one or two
packages that need it).

https://bbs.archlinux.org/viewtopic.php?id=227621
https://bbs.archlinux.org/viewtopic.php?id=227477
https://bbs.archlinux.org/viewtopic.php?id=227574
https://bugs.archlinux.org/task/54564
https://bugs.archlinux.org/task/54590
https://bugs.archlinux.org/task/54588
https://bugs.archlinux.org/task/54580

Seems like the official response is "just live with it, no one cares
what you say".

Which, to be fair, has some justification in that technically speaking,
statically-compiled haskell programs were an ugly bug. It's just a pity
haskell is such a terribly bloated ecosystem. :p

That being said, there are pandoc-lite and shellcheck-static packages in
the AUR which use upstream's prebuilt binaries and don't require the
whole haskell ecosystem as a dependency. Which seems fairly reasonable
to me.
--
Eli Schwartz
Felix Yan
2017-06-26 12:48:45 UTC
Permalink
Raw Message
Post by Eli Schwartz via arch-general
Which, to be fair, has some justification in that technically speaking,
statically-compiled haskell programs were an ugly bug. It's just a pity
haskell is such a terribly bloated ecosystem. :p
Half a year ago we have a discussion here at arch-general about the
dynamic linked future of haskell packages:

https://lists.archlinux.org/pipermail/arch-general/2017-January/042925.html

The basic idea is to make the haskell libraries in our official repos
less bloated and more friendly to end users, instead of providing all
development oriented features. Unfortunately I didn't get enough
feedback on that topic. As the ghc package costs more than 1 GiB at that
time, I planned to do the switch at 8.0.2 (with also the help of newly
added dynlibdir).

We still have something to do with the documentation. The HTML docs take
some disk space and waste time to build indexes even with the help of a
hook. Furthermore, the "ghc" library uses 129 MiB and is not needed if
we are going to use ghc-pkg (for library registering) only. (haddock
needs the ghc library and we use it to build doc indexes.)
--
Regards,
Felix Yan
Sebastian Reuße via arch-general
2017-06-26 13:42:27 UTC
Permalink
Raw Message
Hello Felix,
Post by Felix Yan
The basic idea is to make the haskell libraries in our official repos
less bloated and more friendly to end users, instead of providing all
development oriented features. Unfortunately I didn't get enough
feedback on that topic. As the ghc package costs more than 1 GiB at
that time, I planned to do the switch at 8.0.2 (with also the help of
newly added dynlibdir).
Can you advise whether it’s currently possible to do sandboxed static
builds at all? The issue I’m running into is that Cabal won’t install
sandboxed dependencies if it sees that the corresponding packages are
already registered within the global package DB; the build will proceed
but will then error out during static linking.

It’s possible to force Cabal/GHC to ignore the global package database
altogether, but this won’t help much since it will cause the boot
libraries to be ignored as well.

Kind regards,

Sebastian
--
Insane cobra split the wood
Trader of the lowland breed
Call a jittney, drive away
In the slipstream we will stay
Felix Yan
2017-06-26 17:57:17 UTC
Permalink
Raw Message
Can you advise whether it’s currently possible to do sandboxed static
builds at all? The issue I’m running into is that Cabal won’t install
sandboxed dependencies if it sees that the corresponding packages are
already registered within the global package DB; the build will proceed
but will then error out during static linking.
It’s possible to force Cabal/GHC to ignore the global package database
altogether, but this won’t help much since it will cause the boot
libraries to be ignored as well.
With the current package set the only way I have in mind is to install
stack and start from there. This is unfortunately the same for C
libraries since our removal of their static libs.

An idea is to provide an alternative package database in the ghc-static
package that only contains the boot libraries. You will need to ignore
global package database and specify that alternative path to use it.
Does this sound like useful?
--
Regards,
Felix Yan
Sebastian Reuße via arch-general
2017-06-27 06:35:18 UTC
Permalink
Raw Message
Hello Felix,
Post by Felix Yan
Post by Sebastian Reuße via arch-general
Can you advise whether it’s currently possible to do sandboxed static
builds at all? The issue I’m running into is that Cabal won’t install
sandboxed dependencies if it sees that the corresponding packages are
already registered within the global package DB; the build will proceed
but will then error out during static linking.
With the current package set the only way I have in mind is to install
stack and start from there. This is unfortunately the same for C
libraries since our removal of their static libs.
An idea is to provide an alternative package database in the ghc-static
package that only contains the boot libraries. You will need to ignore
global package database and specify that alternative path to use it.
Does this sound like useful?
That does sound preferable to not being able to do sandboxed static
builds at all. I’ll also try pinging the Cabal folks when I get a
chance; maybe it would make sense for cabal-install to be able to
enforce reinstallation of dependencies without listing all packages
explicitly.

Kind regards,
SR
--
Insane cobra split the wood
Trader of the lowland breed
Call a jittney, drive away
In the slipstream we will stay
Sebastian Reuße via arch-general
2017-07-03 06:48:33 UTC
Permalink
Raw Message
Post by Sebastian Reuße via arch-general
Post by Felix Yan
An idea is to provide an alternative package database in the ghc-static
package that only contains the boot libraries. You will need to ignore
global package database and specify that alternative path to use it.
Does this sound like useful?
That does sound preferable to not being able to do sandboxed static
builds at all. I’ll also try pinging the Cabal folks when I get a
chance; maybe it would make sense for cabal-install to be able to
enforce reinstallation of dependencies without listing all packages
explicitly.
As a follow-up, Cabal currently tracks issue #1175 [1]. There seems to
be a disinclination against implementing «cabal install --reinstall
--dependencies-only» in preference to enabling Cabal to track installed
files and detect when registered packages have files missing. Since this
is tied up with improving Cabal’s package management facilities, my
guess is that this isn’t something that will see the light of day too
soon; so Felix’ proposed workaround of using an alternative global
package DB might be required in the meantime.

Kind regards,
SR

[1] <https://github.com/haskell/cabal/issues/1175>
--
Insane cobra split the wood
Trader of the lowland breed
Call a jittney, drive away
In the slipstream we will stay
Felix Yan
2017-07-03 10:10:48 UTC
Permalink
Raw Message
Post by Sebastian Reuße via arch-general
Post by Sebastian Reuße via arch-general
Post by Felix Yan
An idea is to provide an alternative package database in the ghc-static
package that only contains the boot libraries. You will need to ignore
global package database and specify that alternative path to use it.
Does this sound like useful?
That does sound preferable to not being able to do sandboxed static
builds at all. I’ll also try pinging the Cabal folks when I get a
chance; maybe it would make sense for cabal-install to be able to
enforce reinstallation of dependencies without listing all packages
explicitly.
As a follow-up, Cabal currently tracks issue #1175 [1]. There seems to
be a disinclination against implementing «cabal install --reinstall
--dependencies-only» in preference to enabling Cabal to track installed
files and detect when registered packages have files missing. Since this
is tied up with improving Cabal’s package management facilities, my
guess is that this isn’t something that will see the light of day too
soon; so Felix’ proposed workaround of using an alternative global
package DB might be required in the meantime.
Thanks for the info. Please try out ghc{,-static} 8.0.2-2 which is in
[community-testing] now.

This is an example of how it works for me:

$ cabal sandbox init
$ cabal install
--ghc-pkg-option="--global-package-db=/usr/lib/ghc-8.0.2/static-package.conf.d"
alex
--
Regards,
Felix Yan
Ricardo Catalinas Jiménez via arch-general
2017-08-29 19:52:25 UTC
Permalink
Raw Message
Without trying to create a flame war, is there any chance to
reconsider going back to use static linking with Cabal and Haskell
programs?

I ask for this because in my case, it has been pretty painful trying
to use Cabal to build my code pulling multiple dependencies from
Hackage, as I suffered many issues with the linking. In my case most
of the issues have come with using the new feature "new-build", which
is great in how it solves dependency management but it fails pretty
often if you try to use it with dynamic linking.

I recently added a section on the wiki
(https://wiki.archlinux.org/index.php/Haskell#Building_statically_linked_packages_with_Cabal_.28without_using_shared_libraries.29)
explaining how to get your own cabal-install so that you can use
static linking back again, which is working as expected for me,
specially with "new-build".

I understand there are workarounds like you explained before on how to
ignore the global package DB, but not sure it's worth the trouble.
Also, I understand going back to static linking would create another
trade-off with fatter binaries, but IMHO static linking have important
benefits for us the devs and not so many drawbacks for the users of
packages statically compiled in the repos.

You mentioned before that when you asked to gather some feedback
before switching to dynamic linking you didn't get enough responses. I
understand that maybe not many Haskell devs on Arch reacted about it,
but now talking with other Haskell Arch users (specially on #haskell
IRC room), looks like everyone is having some friction with dynamic
linking and cabal for your own projects.

It feels to me that, as other communities such as the Golang one, have
reached the conclusion that dynamic linking isn't worth the trouble
with the extra complexity it brings in. Some people say that with
distros such as NixOS the problem goes away, no opinion on this, but
what matters to me is the current status is in Arch.

So guys, opinions on this?

—Ricardo
Post by Felix Yan
Post by Sebastian Reuße via arch-general
Post by Sebastian Reuße via arch-general
Post by Felix Yan
An idea is to provide an alternative package database in the ghc-static
package that only contains the boot libraries. You will need to ignore
global package database and specify that alternative path to use it.
Does this sound like useful?
That does sound preferable to not being able to do sandboxed static
builds at all. I’ll also try pinging the Cabal folks when I get a
chance; maybe it would make sense for cabal-install to be able to
enforce reinstallation of dependencies without listing all packages
explicitly.
As a follow-up, Cabal currently tracks issue #1175 [1]. There seems to
be a disinclination against implementing «cabal install --reinstall
--dependencies-only» in preference to enabling Cabal to track installed
files and detect when registered packages have files missing. Since this
is tied up with improving Cabal’s package management facilities, my
guess is that this isn’t something that will see the light of day too
soon; so Felix’ proposed workaround of using an alternative global
package DB might be required in the meantime.
Thanks for the info. Please try out ghc{,-static} 8.0.2-2 which is in
[community-testing] now.
$ cabal sandbox init
$ cabal install
--ghc-pkg-option="--global-package-db=/usr/lib/ghc-8.0.2/static-package.conf.d"
alex
--
Regards,
Felix Yan
João Miguel via arch-general
2017-08-29 23:08:38 UTC
Permalink
Raw Message
Hello!

I hate every Haskell package related thing, so take what I say with a
grain of salt. Have you considered using cblrepo? It's working fine for
me, even for packages they didn't have yet.

They have binary packages in an unofficial repository
(https://wiki.archlinux.org/index.php/Unofficial_user_repositories#haskell-core).
I have haskell-pandoc from there, and it seems they agree with you:

$ ldd /bin/pandoc
linux-vdso.so.1 (0x00007fff96912000)
libz.so.1 => /usr/lib/libz.so.1 (0x00007f7fdc07c000)
librt.so.1 => /usr/lib/librt.so.1 (0x00007f7fdbe74000)
libdl.so.2 => /usr/lib/libdl.so.2 (0x00007f7fdbc70000)
libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f7fdba52000)
libgmp.so.10 => /usr/lib/libgmp.so.10 (0x00007f7fdb7bf000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007f7fdb4ad000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f7fdb107000)
/lib64/ld-linux-x86-64.so.2 (0x00007f7fdc293000)

9 lines, to contrast with the binary in community/pandoc (136 lines, I'm
not listing that here, plus half of them point to "not found").

Best luck with your compiling,
João Miguel

Magnus Therning
2017-06-27 21:46:43 UTC
Permalink
Raw Message
Post by Eli Schwartz via arch-general
Post by Óscar García Amor
Hello all,
Some days ago the pandoc mantainer [1] do a rebuild of it [2] where
add a lot of haskell package dependencies. I think that the build
changes the binary from statically linked to dinamically linked, but
IMHO, I prefer the static one (55,08 MiB of package) over the dinamic
(more than 666 MB in libraries).
What do you think about this?
Other solution can be have other package "pandoc-static", that
maintains the previous method of package.
Greetings.
[1]: https://www.archlinux.org/packages/community/x86_64/pandoc/
[2]: https://git.archlinux.org/svntogit/community.git/commit/trunk?h=packages/pandoc&id=d340c92f8cf5686509551c08bcdaa0b5e66760b0
And same with shellcheck -- the general issue is that *all*
haskell-based packages now build dynamically linked against the haskell
runtime (which is huge, and few people have more than one or two
packages that need it).
https://bbs.archlinux.org/viewtopic.php?id=227621
https://bbs.archlinux.org/viewtopic.php?id=227477
https://bbs.archlinux.org/viewtopic.php?id=227574
https://bugs.archlinux.org/task/54564
https://bugs.archlinux.org/task/54590
https://bugs.archlinux.org/task/54588
https://bugs.archlinux.org/task/54580
Seems like the official response is "just live with it, no one cares
what you say".
Which, to be fair, has some justification in that technically speaking,
statically-compiled haskell programs were an ugly bug. It's just a pity
haskell is such a terribly bloated ecosystem. :p
That being said, there are pandoc-lite and shellcheck-static packages in
the AUR which use upstream's prebuilt binaries and don't require the
whole haskell ecosystem as a dependency. Which seems fairly reasonable
to me.
There's also the ArchHaskell repo:
https://wiki.archlinux.org/index.php/ArchHaskell

Some, but not all, binaries are split out into separate packages. Look
for *-tool packages

/M

--
Magnus Therning OpenPGP: 0x927912051716CE39
email: ***@therning.org jabber: ***@therning.org
twitter: magthe http://therning.org/magnus

The right to search for truth implies also a duty; one must not
conceal any part of what one has recognized to be true.
— Albert Einstein
Loading...