Discussion:
Installation: How to get HDD > LUKS > GPT working in a clean way
(too old to reply)
Merlin Büge
2016-11-27 15:48:18 UTC
Permalink
Hey everybody!


I'm currently installing Arch on my laptop (Thinkpad T400), and have decided
for a rather unusual partition scheme: A single LUKS container directly on
the disk (SSD) with a GPT partition table and two partitions inside it: one for
SWAP, the other one for the system and everyting else, formatted with Btrfs.

The laptop runs libreboot, so I have GRUB2 as a payload inside the flash chip
which I use to decrypt the LUKS container and load a GRUB configfile
located at /boot/grub/grub.cfg (generated by grub-mkconfig). This works fine.


While experimenting with GPT inside LUKS before the installation I've noticed
two issues, with at least one of them being present also after installation:

First, after unlocking the LUKS containter the two GPT partitions don't become
visible to the kernel automatically. I have to manually do
partprobe /dev/mapper/<dmname>
to inform the kernel about the two new partitions. partprobe is part of parted.
My idea was to create a custom hook just after the 'encrypt' hook, which would
simply run the above command. I tested this and it seems to work.

Question:
Is there an even simpler solution to that problem? For example an alternative
to partprobe which is already in 'base'?


The second issue was that I could not (after unmounting the Btrfs partition and
deactivating the swap partition of course) directly close the LUKS mapping via
cryptsetup luksClose <dmname>
It gave me:
device-mapper: remove ioctl on <dmname> failed: Device or resource busy
[...]
device-mapper: remove ioctl on <dmname> failed: Device or resource busy
Device <dmname> is still in use.

Instead, I had to remove the partition mappings first via
dmsetup remove <dmname>1 <dmname>2
This was getting me rid of the aforementioned error messages.

As expected, I get these error messages also during system shutdown -- but only
whith the shutdown hook in initramfs. Without it, I presume the system does not
even try to close the LUKS container (which would make sense, since there is no
initramfs created by default for shutdown afaik), therefore also resulting in
no error messages being shown.

What could I do about this?
I'd like to have my system closing the LUKS container correctly -- therefore I
need to remove the partition mappings before that.


I've read a lot in the last days and weeks about Btrfs, SSDs, coreboot, etc. to
make sure I don't run into many issues. Though these two don't come unexpected,
I don't know how to solve the latter one, because systemd shutdown and shutdown
initramfs are still a little miracle to me...

I'd really appreciate any help!

This is all on an up-to-date vanilla 4.8.10-1-ARCH.
I attached two shutdown logs with debugging enabled: one with and one without
the shutdown hook applied. They look very similar though. (I made sure to reboot
twice after building the initramfs before taking the shutdown log.)
My HOOKS array is:
HOOKS="base udev autodetect modconf block keyboard keymap encrypt pp \
filesystems fsck shutdown"
(pp being the hook which runs partprobe against the mapped LUKS container)


Best Regards,

Merlin
--
Merlin Büge <***@bluenox07.de>
Jeremy Brown
2016-11-27 20:30:31 UTC
Permalink
Post by Merlin Büge
Hey everybody!
I'm currently installing Arch on my laptop (Thinkpad T400), and have decided
for a rather unusual partition scheme: A single LUKS container directly on
the disk (SSD) with a GPT partition table and two partitions inside it: one for
SWAP, the other one for the system and everyting else, formatted with Btrfs.
I'm not really answering your question here, but just out of curiosity what's the appeal of this approach over doing one of the options described at https://wiki.archlinux.org/index.php/Dm-crypt/Swap_encryption:

(1) dm-encrypted swap with a new key generated each boot (what I generally use, is the most secure, and works in any case where you don't need to hibernate)
(2) The same thing you're trying to do but with LVM instead of a GPT table
(3) a swap file on your Btrfs volume itself

I've done all these at various points and they all work well.

Jeremy
Merlin Büge
2016-11-27 20:43:57 UTC
Permalink
Hi,
Post by Jeremy Brown
Post by Merlin Büge
I'm currently installing Arch on my laptop (Thinkpad T400), and have decided
for a rather unusual partition scheme: A single LUKS container directly on
the disk (SSD) with a GPT partition table and two partitions inside it: one for
SWAP, the other one for the system and everyting else, formatted with Btrfs.
I'm not really answering your question here, but just out of curiosity what's the
appeal of this approach over doing one of the options described at
(1) dm-encrypted swap with a new key generated each boot (what I generally use, is
the most secure, and works in any case where you don't need to hibernate)
Hibernation :P
Post by Jeremy Brown
(2) The same thing you're trying to do but with LVM instead of a GPT table
I don't need any LVM functionality and would consider it a little bit overkill
for just dividing available space into two partitions.
Post by Jeremy Brown
(3) a swap file on your Btrfs volume itself
That's (currently) not possible, see the link you posted.

By the way, please try to add line breaks at about 75 characters, see [0].

Regards,

Merlin

[0] https://wiki.archlinux.org/index.php/Code_of_conduct#Mailing_lists
--
Merlin Büge <***@bluenox07.de>
Jeremy Brown
2016-11-27 21:13:34 UTC
Permalink
Post by Merlin Büge
Post by Jeremy Brown
(2) The same thing you're trying to do but with LVM instead of a GPT table
I don't need any LVM functionality and would consider it a little bit overkill
for just dividing available space into two partitions.
I completely disagree. As another poster said in their response, LVM is
really the canonical/supported/more flexible way to do this. Even on
systems where I don't need encryption, I generally use LVM to divide up
the space instead of traditional partitions.
Post by Merlin Büge
Post by Jeremy Brown
(3) a swap file on your Btrfs volume itself
That's (currently) not possible, see the link you posted.
Gah. Sorry. You are correct. Should have read my own link all the way
through before sharing it. I usually use good ole ext4.
Post by Merlin Büge
By the way, please try to add line breaks at about 75 characters, see [0].
[0] https://wiki.archlinux.org/index.php/Code_of_conduct#Mailing_lists
Ah, got it. Haven't posted to this list much, but noted for this and
future posts :)

Cheers,

Jeremy
Yaro Kasear
2016-11-27 20:41:41 UTC
Permalink
Post by Merlin Büge
Hey everybody!
I'm currently installing Arch on my laptop (Thinkpad T400), and have decided
for a rather unusual partition scheme: A single LUKS container directly on
the disk (SSD) with a GPT partition table and two partitions inside it: one for
SWAP, the other one for the system and everyting else, formatted with Btrfs.
The laptop runs libreboot, so I have GRUB2 as a payload inside the flash chip
which I use to decrypt the LUKS container and load a GRUB configfile
located at /boot/grub/grub.cfg (generated by grub-mkconfig). This works fine.
While experimenting with GPT inside LUKS before the installation I've noticed
First, after unlocking the LUKS containter the two GPT partitions don't become
visible to the kernel automatically. I have to manually do
partprobe /dev/mapper/<dmname>
to inform the kernel about the two new partitions. partprobe is part of parted.
My idea was to create a custom hook just after the 'encrypt' hook, which would
simply run the above command. I tested this and it seems to work.
Is there an even simpler solution to that problem? For example an alternative
to partprobe which is already in 'base'?
The second issue was that I could not (after unmounting the Btrfs partition and
deactivating the swap partition of course) directly close the LUKS mapping via
cryptsetup luksClose <dmname>
device-mapper: remove ioctl on <dmname> failed: Device or resource busy
[...]
device-mapper: remove ioctl on <dmname> failed: Device or resource busy
Device <dmname> is still in use.
Instead, I had to remove the partition mappings first via
dmsetup remove <dmname>1 <dmname>2
This was getting me rid of the aforementioned error messages.
As expected, I get these error messages also during system shutdown -- but only
whith the shutdown hook in initramfs. Without it, I presume the system does not
even try to close the LUKS container (which would make sense, since there is no
initramfs created by default for shutdown afaik), therefore also resulting in
no error messages being shown.
What could I do about this?
I'd like to have my system closing the LUKS container correctly -- therefore I
need to remove the partition mappings before that.
I've read a lot in the last days and weeks about Btrfs, SSDs, coreboot, etc. to
make sure I don't run into many issues. Though these two don't come unexpected,
I don't know how to solve the latter one, because systemd shutdown and shutdown
initramfs are still a little miracle to me...
I'd really appreciate any help!
This is all on an up-to-date vanilla 4.8.10-1-ARCH.
I attached two shutdown logs with debugging enabled: one with and one without
the shutdown hook applied. They look very similar though. (I made sure to reboot
twice after building the initramfs before taking the shutdown log.)
HOOKS="base udev autodetect modconf block keyboard keymap encrypt pp \
filesystems fsck shutdown"
(pp being the hook which runs partprobe against the mapped LUKS container)
Best Regards,
Merlin
Hi Merlin,

I'd set up two partitions: Your EFI system partition and the LUKS
container. Then inside LUKS, format the whole thing as LVM and then set
up from there, rather than make the LUKS container another GPT "disk."
Then you just use the crypt and lvm2 hooks.

You should only really use partition tables on a physical disk, in my
opinion, not a LUKS container.

The reason for this is that LVM works with a lot more flexibility and is
more readily automated than trying to get the system to re-read
partition tables.

If you were on a system where you could add disks, I'd even suggest
reversing it: LVM on the metal and LUKS on a logical volume spanning the
whole VG, that way you could grow the whole thing across multiple disks
pretty easily without having to make any dramatic changes. I know btrfs
can do multiple disks, but I've always preferred how LVM does it, honestly.

Yaro
Merlin Büge
2016-11-28 02:09:01 UTC
Permalink
Hi Yaro,


thanks for your answer!
Post by Yaro Kasear
I'd set up two partitions: Your EFI system partition and the LUKS
container. Then inside LUKS, format the whole thing as LVM and then set
up from there, rather than make the LUKS container another GPT "disk."
Then you just use the crypt and lvm2 hooks.
I have no EFI system partition because I don't need one.
Post by Yaro Kasear
You should only really use partition tables on a physical disk, in my
opinion, not a LUKS container.
The reason for this is that LVM works with a lot more flexibility and is
more readily automated than trying to get the system to re-read
partition tables.
Hm. I can see your points. But I don't need the flexibility LVM provides,
I have enough flexibility through Btrfs.
And yeah, it's readily automated, and that's indeed practical for many
people. Personally, I'd rather modify the start-up process a tiny bit
so that GPT inside LUKS gets parsed. I just try to strip off unnecessary
'overhead' / layers of my system.
Post by Yaro Kasear
If you were on a system where you could add disks [...]
Since it's on my laptop, I don't need that functionality :)


Best Regards,

Merlin
--
Merlin Büge <***@bluenox07.de>
Yaro Kasear
2016-11-28 02:23:26 UTC
Permalink
Post by Merlin Büge
Hi Yaro,
thanks for your answer!
Post by Yaro Kasear
I'd set up two partitions: Your EFI system partition and the LUKS
container. Then inside LUKS, format the whole thing as LVM and then set
up from there, rather than make the LUKS container another GPT "disk."
Then you just use the crypt and lvm2 hooks.
I have no EFI system partition because I don't need one.
My mistake. I assumed you had a typical UEFI setup, which would require one.
Post by Merlin Büge
Post by Yaro Kasear
You should only really use partition tables on a physical disk, in my
opinion, not a LUKS container.
The reason for this is that LVM works with a lot more flexibility and is
more readily automated than trying to get the system to re-read
partition tables.
Hm. I can see your points. But I don't need the flexibility LVM provides,
I have enough flexibility through Btrfs.
And yeah, it's readily automated, and that's indeed practical for many
people. Personally, I'd rather modify the start-up process a tiny bit
so that GPT inside LUKS gets parsed. I just try to strip off unnecessary
'overhead' / layers of my system.
Okay, then.

Here's my opinion on this approach.

If you have 8 GiB or more and not hibernating, don't bother with swap,
it'd be a waste of disk space. In that case you could just put a btrfs
volume straight on the LUKS container without the GPT. Problem solved as
you don't need any more volume management than opening LUKS containers.

Otherwise WITH swap: Unfortunately btrfs (still) doesn't support swap
files properly, otherwise I'd suggest using them. You can write a custom
hook. Unless you plan to share it, I'd make it a dead simple shell
script that simply reruns the command to scan for added GPT partitions
for your specific setup. Make sure you have a setup hook that gets the
dependencies in there.

Personally, I still think you should just use LVM, for the simple reason
you're having trouble with GPT, which is not meant for being used like
this, since it can work as a more flexible "partition table" inside the
LUKS container and is better supported all around. btrfs really doesn't
act as a good replacement for logical volumes, in my experience. Having
something with more features than you need is better than trying to
coerce something into working ways it's not really intended.
Post by Merlin Büge
Post by Yaro Kasear
If you were on a system where you could add disks [...]
Since it's on my laptop, I don't need that functionality :)
Best Regards,
Merlin
Merlin Büge
2016-12-02 22:29:37 UTC
Permalink
Hi,
Post by Yaro Kasear
Post by Merlin Büge
Hm. I can see your points. But I don't need the flexibility LVM provides,
I have enough flexibility through Btrfs.
And yeah, it's readily automated, and that's indeed practical for many
people. Personally, I'd rather modify the start-up process a tiny bit
so that GPT inside LUKS gets parsed. I just try to strip off unnecessary
'overhead' / layers of my system.
Okay, then.
Here's my opinion on this approach.
If you have 8 GiB or more and not hibernating, don't bother with swap,
it'd be a waste of disk space. In that case you could just put a btrfs
volume straight on the LUKS container without the GPT. Problem solved as
you don't need any more volume management than opening LUKS containers.
Otherwise WITH swap: Unfortunately btrfs (still) doesn't support swap
files properly, otherwise I'd suggest using them. You can write a custom
hook. Unless you plan to share it, I'd make it a dead simple shell
script that simply reruns the command to scan for added GPT partitions
for your specific setup. Make sure you have a setup hook that gets the
dependencies in there.
Personally, I still think you should just use LVM, for the simple reason
you're having trouble with GPT, which is not meant for being used like
this, since it can work as a more flexible "partition table" inside the
LUKS container and is better supported all around. btrfs really doesn't
act as a good replacement for logical volumes, in my experience. Having
something with more features than you need is better than trying to
coerce something into working ways it's not really intended.
Thanks for your input!

I've reconsidered LVM twice now but still feel better without it.


Cheers,

Merlin
--
Merlin Büge <***@bluenox07.de>
Paul Gideon Dann via arch-general
2016-12-05 09:09:59 UTC
Permalink
Post by Yaro Kasear
Personally, I'd rather modify the start-up process a tiny bit so that
GPT inside LUKS gets parsed. I just try to strip off unnecessary
'overhead' / layers of my system.
If you have 8 GiB or more and not hibernating, don't bother with swap,
it'd be a waste of disk space. In that case you could just put a btrfs
volume straight on the LUKS container without the GPT. Problem solved as
you don't need any more volume management than opening LUKS containers.
If another opinion helps, I've done some funky disk layouts at various
times, and I also think that if you need partitioning above the LUKS layer,
you'd do better to use LVM than GPT. GPT is intended to be used at the
lowest level of the stack, whereas LVM is well-supported at pretty much any
level. If you do go ahead with it, double-check that you won't get block
alignment issues in that stack that could affect IO performance.

However, if you say that you don't need the flexibility of LVM, I'd
certainly first try btrfs directly on top of LUKS.

Final consideration: if you want GRUB to open a LUKS container and then
load stage 2 from btrfs, you'll need a decent amount of storage for the
GRUB 1st stage, which on a traditional setup goes in free space you need
to account for after the MBR (or on the EFI partition for UEFI setups). In
your case, as the whole disk is LUKS and you have no partition table, have
you considered where the GRUB 1st stage will be stored? I use a USB stick
to boot GRUB stage 1 on my encrypted machines, and that may work for you
too.

Paul
Merlin Büge
2016-12-05 22:52:36 UTC
Permalink
Hi Paul,
Post by Paul Gideon Dann via arch-general
If another opinion helps, I've done some funky disk layouts at various
times, and I also think that if you need partitioning above the LUKS layer,
you'd do better to use LVM than GPT. GPT is intended to be used at the
lowest level of the stack, whereas LVM is well-supported at pretty much any
level. If you do go ahead with it, double-check that you won't get block
alignment issues in that stack that could affect IO performance.
Thanks for your input.
(I already checked alignment of my setup.)
Post by Paul Gideon Dann via arch-general
However, if you say that you don't need the flexibility of LVM, I'd
certainly first try btrfs directly on top of LUKS.
If I did not want to have a swap partition, I'd to that for sure.
Another possible layout which just comes to mind is

GPT
+-LUKS
| +-Btrfs
+-LUKS
+-SWAP

I think that should work with hibernation, too, and GPT would be on the right
place + still no LVM :)
Maye I'll just try different layouts over time, haven't experimented much yet.
Post by Paul Gideon Dann via arch-general
Final consideration: if you want GRUB to open a LUKS container and then
load stage 2 from btrfs, you'll need a decent amount of storage for the
GRUB 1st stage, which on a traditional setup goes in free space you need
to account for after the MBR (or on the EFI partition for UEFI setups). In
your case, as the whole disk is LUKS and you have no partition table, have
you considered where the GRUB 1st stage will be stored? I use a USB stick
to boot GRUB stage 1 on my encrypted machines, and that may work for you
too.
As mentioned in my initial post, I have GRUB2 along with (deblobbed) coreboot
stored in the SPI flash chip (so no BIOS here). It's very convenient :)


Regards,

Merlin
--
Merlin Büge <***@bluenox07.de>
Leonid Isaev
2016-11-28 01:52:10 UTC
Permalink
Post by Merlin Büge
I'm currently installing Arch on my laptop (Thinkpad T400), and have decided
for a rather unusual partition scheme: A single LUKS container directly on
the disk (SSD) with a GPT partition table and two partitions inside it: one
for SWAP, the other one for the system and everyting else, formatted with
Btrfs.
Why is it unusual? Everyone does this: LUKS on a raw block device, then LVM or
btrfs subvolumes inside it.
Post by Merlin Büge
The laptop runs libreboot, so I have GRUB2 as a payload inside the flash chip
which I use to decrypt the LUKS container and load a GRUB configfile
located at /boot/grub/grub.cfg (generated by grub-mkconfig). This works fine.
Where is /boot physically located? Can grub2 boot from LV these days?

Cheers,
--
Leonid Isaev
GPG fingerprints: DA92 034D B4A8 EC51 7EA6 20DF 9291 EE8A 043C B8C4
C0DF 20D0 C075 C3F1 E1BE 775A A7AE F6CB 164B 5A6D
Merlin Büge
2016-11-28 02:18:54 UTC
Permalink
Hi,
Post by Leonid Isaev
Post by Merlin Büge
I'm currently installing Arch on my laptop (Thinkpad T400), and have decided
for a rather unusual partition scheme: A single LUKS container directly on
the disk (SSD) with a GPT partition table and two partitions inside it: one
for SWAP, the other one for the system and everyting else, formatted with
Btrfs.
Why is it unusual? Everyone does this: LUKS on a raw block device, then LVM or
btrfs subvolumes inside it.
Because in my case LUKS occupies the hole physical disk. I think not everyone
does this. Or, because I have GPT inside a LUKS container, which obviously
does not run out-of-the-box.
Post by Leonid Isaev
Post by Merlin Büge
The laptop runs libreboot, so I have GRUB2 as a payload inside the flash chip
which I use to decrypt the LUKS container and load a GRUB configfile
located at /boot/grub/grub.cfg (generated by grub-mkconfig). This works fine.
Where is /boot physically located? Can grub2 boot from LV these days?
/boot is physically located on my only storage drive in the laptop.
It's not a seperate partition, just on the Btrfs filesystem.
Post by Leonid Isaev
Can grub2 boot from LV these days?
I don't know (nor do I need).


Regards
--
Merlin Büge <***@bluenox07.de>
Yaro Kasear
2016-11-28 02:25:57 UTC
Permalink
Post by Merlin Büge
Post by Leonid Isaev
Where is /boot physically located? Can grub2 boot from LV these days?
/boot is physically located on my only storage drive in the laptop.
It's not a seperate partition, just on the Btrfs filesystem.
Post by Leonid Isaev
Can grub2 boot from LV these days?
I don't know (nor do I need).
Just to answer: Yes, grub2 can boot from LVM and LUKS. UEFI typically
requires an open boot partition, but that could just be used to launch
grub2 which could in turn read things from an encrypted logical volume.

My current computer is a legacy BIOS system (Ugh, I prefer UEFI.) so I
can put the whole thing in LVM/LUKS no issues, since BIOS doesn't use a
system partition. Arch's wiki should provide adequate instructions, I
believe.
Post by Merlin Büge
Regards
Loading...