Discussion:
Dualboot and ZFS
(too old to reply)
Victor Sudakov
2018-01-15 05:13:08 UTC
Permalink
Dear Colleagues,

I have a box with dual boot: Windows and FreeBSD/UFS:

$ gpart show ada0
=> 63 976773105 ada0 MBR (466G)
63 1985 - free - (993K)
2048 293029888 1 ntfs [active] (140G)
293031936 598958080 2 ntfs (286G)
891990016 83886080 3 freebsd (40G)
975876096 897072 - free - (438M)

$ gpart show ada0s3
=> 0 83886080 ada0s3 BSD (40G)
0 75497472 1 freebsd-ufs (36G)
75497472 8388607 2 freebsd-swap (4.0G)
83886079 1 - free - (512B)

$

boot0 handles the OS selection, everything works just fine.

Can I install FreeBSD with root-on-zfs into ada0s3 somehow?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Manish Jain
2018-01-15 09:50:32 UTC
Permalink
Post by Victor Sudakov
Dear Colleagues,
$ gpart show ada0
=> 63 976773105 ada0 MBR (466G)
63 1985 - free - (993K)
2048 293029888 1 ntfs [active] (140G)
293031936 598958080 2 ntfs (286G)
891990016 83886080 3 freebsd (40G)
975876096 897072 - free - (438M)
$ gpart show ada0s3
=> 0 83886080 ada0s3 BSD (40G)
0 75497472 1 freebsd-ufs (36G)
75497472 8388607 2 freebsd-swap (4.0G)
83886079 1 - free - (512B)
$
boot0 handles the OS selection, everything works just fine.
Can I install FreeBSD with root-on-zfs into ada0s3 somehow?
Hi Victor,

Yes, you can. But you will need a spare slice somewhere - another disk
(SATA / USB). If the used potion of your ada0s3a is small, you can even
use a USB pen drive.

You can rsync your current installation to the spare slice, and then
mirror the sync back to a newly created ada0s3-zfs.

Let's say your spare disk is da0 with 40G spare (or at least the used
size of ada0s3a).

gpart destroy -F da0 # only if needed
gpart create -s MBR da0 # only if needed

If da0 already has a spare slice, the 2 steps above are not needed, but
then you will need to replace the slice number instead of da0s1 in the
steps below.

gpart add -t freebsd da0 # adds da0s1; note the slice number here
gpart create -s BSD da0s1
gpart set -a active -i 1 da0
gpart add -t freebsd-zfs da0s1
zpool create zroot /dev/da0s1a
zpool set bootfs=zroot zroot
gpart bootcode -b /boot/boot0 da0
zpool export zroot
dd if=/boot/zfsboot of=/tmp/zfsboot1 count=1
gpart bootcode -b /tmp/zfsboot1 /dev/da0s1
dd if=/boot/zfsboot of=/dev/da0s1a skip=1 seek=1024
zpool import zroot
zfs set checksum=fletcher4 zroot


{
cat <<- EOF

/dev/*
/proc/*
/sys/*
/tmp/*
/mnt/*
/media/*
/lost+found
/usr/ports
/usr/src/*
/zroot
} > /root/excl

rsync -aAHXv --delete --exclude-from /root/excl / /zroot/


echo 'zfs_load=YES' > /zroot/boot/loader.conf
echo 'vfs.root.mountfrom="zfs:zroot"' >> /zroot/boot/loader.conf
echo 'kern.geom.label.disk_ident.enable=0' >> /zroot/boot/loader.conf

Boot into da0, delete ada0s3 ('gpart delete -i 3 ada0; gpart destroy -F
ada0s3'), and carry out the reverse transaction from da0s1 to ada0s3,
using a new pool name zmirror in place of zroot.

Hope this helps.

Regards,
Manish Jain
Manish Jain
2018-01-15 11:21:12 UTC
Permalink
Post by Manish Jain
Post by Victor Sudakov
Dear Colleagues,
$ gpart show ada0
=> 63 976773105 ada0 MBR (466G)
63 1985 - free - (993K)
2048 293029888 1 ntfs [active] (140G)
293031936 598958080 2 ntfs (286G)
891990016 83886080 3 freebsd (40G)
975876096 897072 - free - (438M)
$ gpart show ada0s3
=> 0 83886080 ada0s3 BSD (40G)
0 75497472 1 freebsd-ufs (36G)
75497472 8388607 2 freebsd-swap (4.0G)
83886079 1 - free - (512B)
$
boot0 handles the OS selection, everything works just fine.
Can I install FreeBSD with root-on-zfs into ada0s3 somehow?
Hi Victor,
Yes, you can. But you will need a spare slice somewhere - another disk
(SATA / USB). If the used potion of your ada0s3a is small, you can even
use a USB pen drive.
Actually, if you don't care about your data on ada0s3a (or are willing
to back it up to another disk), you can even use ada0s4 as the spare slice.

The strategy for this would be to install FreeBSD-UFS afresh again, this
time creating two slices ada0s3 and ada0s4 (both freebsd). Size ada0s3
as 36G and s4 as 4G. Install into s4, not s3.

Boot into s4 and install rsync via pkg.

Then use the steps previously mentioned to rsync from ada0s4a-ufs to
ad0s3a-zfs.

Once the sync to s3 is done, boot into s3. And then you can use s4a as a
swap partition.

Regards,
Manish Jain
Victor Sudakov
2018-01-15 12:52:41 UTC
Permalink
Post by Manish Jain
Post by Victor Sudakov
$ gpart show ada0
=> 63 976773105 ada0 MBR (466G)
63 1985 - free - (993K)
2048 293029888 1 ntfs [active] (140G)
293031936 598958080 2 ntfs (286G)
891990016 83886080 3 freebsd (40G)
975876096 897072 - free - (438M)
$ gpart show ada0s3
=> 0 83886080 ada0s3 BSD (40G)
0 75497472 1 freebsd-ufs (36G)
75497472 8388607 2 freebsd-swap (4.0G)
83886079 1 - free - (512B)
$
boot0 handles the OS selection, everything works just fine.
Can I install FreeBSD with root-on-zfs into ada0s3 somehow?
Hi Victor,
Yes, you can. But you will need a spare slice somewhere - another disk
(SATA / USB). If the used potion of your ada0s3a is small, you can even
use a USB pen drive.
You can rsync your current installation to the spare slice, and then
mirror the sync back to a newly created ada0s3-zfs.
Sorry if my question was misleading, I did not mean converting the
existing system from UFS to ZFS at ada0s3. Suppose ada0s3 is an unused
slice, I can probably create a zpool thereon. But a few questions
remain:

1. Can I install a root-on-zfs system on ada0s3 with bsdinstall? I
suppose not?

2. In case of rsync/restore/installworld, what do I do about the VBR
in ada0s3? This VBR should support booting from ZFS. Looks like
zfsboot(8) allows this.

3. Please see the other questions below, step by step.
Post by Manish Jain
Let's say your spare disk is da0 with 40G spare (or at least the used
size of ada0s3a).
gpart destroy -F da0 # only if needed
gpart create -s MBR da0 # only if needed
If da0 already has a spare slice, the 2 steps above are not needed, but
then you will need to replace the slice number instead of da0s1 in the
steps below.
gpart add -t freebsd da0 # adds da0s1; note the slice number here
gpart create -s BSD da0s1
gpart set -a active -i 1 da0
gpart add -t freebsd-zfs da0s1
zpool create zroot /dev/da0s1a
Why are you creating a zpool within a BSD "a" partition, and not in
da0s1 directly? I have never seen such a configuration. Is it even
possible to set the "freebsd-zfs" type on a BSD partition?

And a pool created this way would be non-compatible with beadm. If
anyone has a script to create a beadm-compatible set of datasets,
please share it. But it's already a different matter.
Post by Manish Jain
zpool set bootfs=zroot zroot
gpart bootcode -b /boot/boot0 da0
zpool export zroot
So you suggest using boot0 in MBR with zfsboot in VBR? Will that work?
Post by Manish Jain
dd if=/boot/zfsboot of=/tmp/zfsboot1 count=1
gpart bootcode -b /tmp/zfsboot1 /dev/da0s1
dd if=/boot/zfsboot of=/dev/da0s1a skip=1 seek=1024
It looks like the procedure described in zfsboot(8) but the da0s1a
(the "a" partition) beats me. Should not the pool be in da0s1 ?
Post by Manish Jain
zpool import zroot
zfs set checksum=fletcher4 zroot
{
cat <<- EOF
/dev/*
/proc/*
/sys/*
/tmp/*
/mnt/*
/media/*
/lost+found
/usr/ports
/usr/src/*
/zroot
} > /root/excl
rsync -aAHXv --delete --exclude-from /root/excl / /zroot/
echo 'zfs_load=YES' > /zroot/boot/loader.conf
echo 'vfs.root.mountfrom="zfs:zroot"' >> /zroot/boot/loader.conf
echo 'kern.geom.label.disk_ident.enable=0' >> /zroot/boot/loader.conf
^^^^^^^^^^^^^^^^^^^^^^^^^^^ why would I need to set this sysctl to 0?
It's "1" by default on my root-on-zfs boxes.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Manish Jain
2018-01-15 14:23:53 UTC
Permalink
Post by Victor Sudakov
Sorry if my question was misleading, I did not mean converting the
existing system from UFS to ZFS at ada0s3. Suppose ada0s3 is an unused
slice, I can probably create a zpool thereon. But a few questions
1. Can I install a root-on-zfs system on ada0s3 with bsdinstall? I
suppose not?
That's right. You cannot install ZFS onto an MBR slice. ZFS by default
needs an entire disk, partitioned GPT.
Post by Victor Sudakov
Why are you creating a zpool within a BSD "a" partition, and not in
da0s1 directly? I have never seen such a configuration. Is it even
possible to set the "freebsd-zfs" type on a BSD partition?
I presume you are using MBR partitioning. Under MBR, freebsd slices are
always nested. The slice contains partitions denoted with a, b and so on.

I do not think it is viable to set freebsd-zfs on an MBR slice. It just
will not work. ZFS needs an entire disk. If you wish to install on an
MBR slice, you must use rsync after installing freebsd-UFS.
Post by Victor Sudakov
Post by Manish Jain
echo 'kern.geom.label.disk_ident.enable=0' >> /zroot/boot/loader.conf
It's "1" by default on my root-on-zfs boxes.
I think this is not necessary. You can leave out this line.

Regards,
Manish Jain
Victor Sudakov
2018-01-15 14:47:48 UTC
Permalink
Post by Manish Jain
Post by Victor Sudakov
Sorry if my question was misleading, I did not mean converting the
existing system from UFS to ZFS at ada0s3. Suppose ada0s3 is an unused
slice, I can probably create a zpool thereon. But a few questions
1. Can I install a root-on-zfs system on ada0s3 with bsdinstall? I
suppose not?
That's right. You cannot install ZFS onto an MBR slice. ZFS by default
needs an entire disk, partitioned GPT.
However, later you suggest it's possible to install freebsd-zfs on a
BSD slice. How is that?
Post by Manish Jain
Post by Victor Sudakov
Why are you creating a zpool within a BSD "a" partition, and not in
da0s1 directly? I have never seen such a configuration. Is it even
possible to set the "freebsd-zfs" type on a BSD partition?
I presume you are using MBR partitioning. Under MBR, freebsd slices are
always nested. The slice contains partitions denoted with a, b and so on.
I do not think it is viable to set freebsd-zfs on an MBR slice. It just
will not work. ZFS needs an entire disk. If you wish to install on an
MBR slice, you must use rsync after installing freebsd-UFS.
Do you mean to say it's viable to install freebsd-zfs on ada0s3a but
not viable on ada0s3 ? I don't see logic here.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Manish Jain
2018-01-15 14:57:09 UTC
Permalink
Post by Victor Sudakov
However, later you suggest it's possible to install freebsd-zfs on a
BSD slice. How is that?
I never suggested installing freebsd-zfs on a MBR/BSD slice, unless I
made a typo. freebsd-zfs can only be installed on an entire disk,
partitioned GPT.

The steps mentioned in my first message target moving an existing UFS to
ZFS.
Post by Victor Sudakov
Do you mean to say it's viable to install freebsd-zfs on ada0s3a but
not viable on ada0s3 ? I don't see logic here.
You have to recreate the s3 slice and its nested s3a partition during rsync.


Thanks & Regards,
Manish Jain
Victor Sudakov
2018-01-15 15:15:26 UTC
Permalink
Post by Manish Jain
Post by Victor Sudakov
However, later you suggest it's possible to install freebsd-zfs on a
BSD slice. How is that?
I never suggested installing freebsd-zfs on a MBR/BSD slice, unless I
made a typo. freebsd-zfs can only be installed on an entire disk,
partitioned GPT.
The steps mentioned in my first message target moving an existing UFS to
ZFS.
Do we perhaps have a different understanding of "installing"? If by
"installing" you mean only that by the regular bsdinstall, then I
understand you.

For me, however, running "make installworld DESTDIR=xxx" is also
installing. Can DESTDIR be a zpool on a MBR/BSD slice?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Manish Jain
2018-01-15 15:31:37 UTC
Permalink
Post by Victor Sudakov
Do we perhaps have a different understanding of "installing"? If by
"installing" you mean only that by the regular bsdinstall, then I
understand you.
For me, however, running "make installworld DESTDIR=xxx" is also
installing. Can DESTDIR be a zpool on a MBR/BSD slice?
I think this might be a key difference. When I say installing, it
usually means -- pick up your optical CD installer, and install it onto
your hard disk.

I never bother about buildworld / installworld.

Thanks & Regards,
Manish Jain
Dean E. Weimer
2018-01-15 15:42:00 UTC
Permalink
Post by Victor Sudakov
Post by Manish Jain
Post by Victor Sudakov
However, later you suggest it's possible to install freebsd-zfs on a
BSD slice. How is that?
I never suggested installing freebsd-zfs on a MBR/BSD slice, unless I
made a typo. freebsd-zfs can only be installed on an entire disk,
partitioned GPT.
The steps mentioned in my first message target moving an existing UFS to
ZFS.
Do we perhaps have a different understanding of "installing"? If by
"installing" you mean only that by the regular bsdinstall, then I
understand you.
For me, however, running "make installworld DESTDIR=xxx" is also
installing. Can DESTDIR be a zpool on a MBR/BSD slice?
You can do an install from source to a zfs dataset no matter where it
exists as that just shows up as any other mount point, but if you want
to be able to boot from it I would suggest making it on a GPT disk. I
checked and there is a section on the wiki
<https://wiki.freebsd.org/RootOnZFS/ZFSBootSlice> about installing to
MBR, but its flagged as will not boot.

By the way the setup I sent was installed by using the live CD, creating
the zpool, and zfs datasets, and extracting the data manually. I do
updates by building from source and installing to clones of root dataset
then change zpools bootfs and reboot. With the new dataset.
--
Thanks,
Dean E. Weimer
http://www.dweimer.net/
Victor Sudakov
2018-01-16 03:49:29 UTC
Permalink
Dean E. Weimer wrote:

[dd]
Post by Dean E. Weimer
Post by Victor Sudakov
For me, however, running "make installworld DESTDIR=xxx" is also
installing. Can DESTDIR be a zpool on a MBR/BSD slice?
You can do an install from source to a zfs dataset no matter where it
exists as that just shows up as any other mount point, but if you want
to be able to boot from it I would suggest making it on a GPT disk.
The whole point of my question was to install FreeBSD/ZFS as a second
system to a box where Windows/NTFS already occupies one MBR partition and
there is another MBR partition for FreeBSD. Do you think there is no
such luck? I
Post by Dean E. Weimer
checked and there is a section on the wiki
<https://wiki.freebsd.org/RootOnZFS/ZFSBootSlice> about installing
to MBR, but its flagged as will not boot.
"Will not boot"? That is odd because:

1. That's basically the setup Manish suggested (using ada0s3a for
zpool), and I think it works for him if he gave this advice.

2. zfsboot(8) is designed for MBR so why would it fail to boot?
Post by Dean E. Weimer
By the way the setup I sent was installed by using the live CD, creating
But your setup is not dualboot which I was asking about.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Trond Endrestøl
2018-01-16 08:52:12 UTC
Permalink
Post by Victor Sudakov
[dd]
Post by Dean E. Weimer
Post by Victor Sudakov
For me, however, running "make installworld DESTDIR=xxx" is also
installing. Can DESTDIR be a zpool on a MBR/BSD slice?
You can do an install from source to a zfs dataset no matter where it
exists as that just shows up as any other mount point, but if you want
to be able to boot from it I would suggest making it on a GPT disk.
The whole point of my question was to install FreeBSD/ZFS as a second
system to a box where Windows/NTFS already occupies one MBR partition and
there is another MBR partition for FreeBSD. Do you think there is no
such luck? I
Post by Dean E. Weimer
checked and there is a section on the wiki
<https://wiki.freebsd.org/RootOnZFS/ZFSBootSlice> about installing
to MBR, but its flagged as will not boot.
1. That's basically the setup Manish suggested (using ada0s3a for
zpool), and I think it works for him if he gave this advice.
2. zfsboot(8) is designed for MBR so why would it fail to boot?
Post by Dean E. Weimer
By the way the setup I sent was installed by using the live CD, creating
But your setup is not dualboot which I was asking about.
I couldn't resist attempting a proof of concept, so here it is.

The main motivation for using a BSD disklabel within a MBR slice is to
have a ZFS partition and a swap partition while only consuming one of
the four available MBR slices. You also avoid using a zvol for
swap/dump.

See https://wiki.freebsd.org/RootOnZFS/ZFSBootPartition for more
details.

Script started on Tue Jan 16 08:40:32 2018
# PS1='[script]# '

!!! These are the two MBR slices created by Windows 10, half of ada0 is available for FreeBSD !!!

[script]# gpart show ada0
=> 63 134217665 ada0 MBR (64G)
63 1985 - free - (993K)
2048 1024000 1 ntfs [active] (500M)
1026048 66082816 2 ntfs (32G)
67108864 67108864 - free - (32G)

!!! Now, add our FreeBSD slice !!!

[script]# gpart add -t freebsd ada0
ada0s3 added

!!! Here it is !!!

[script]# gpart show ada0
=> 63 134217665 ada0 MBR (64G)
63 1985 - free - (993K)
2048 1024000 1 ntfs [active] (500M)
1026048 66082816 2 ntfs (32G)
67108864 67108864 3 freebsd (32G)

!!! Create a BSD disklabel within the FreeBSD slice !!!

[script]# gpart create -s BSD ada0s3
ada0s3 created
[script]# gpart show ada0
=> 63 134217665 ada0 MBR (64G)
63 1985 - free - (993K)
2048 1024000 1 ntfs [active] (500M)
1026048 66082816 2 ntfs (32G)
67108864 67108864 3 freebsd (32G)

!!! Yes, the disklabel is in place !!!

[script]# gpart show ada0s3
=> 0 67108864 ada0s3 BSD (32G)
0 67108864 - free - (32G)

!!! Add our ZFS partition !!!

[script]# gpart add -s 28G -t freebsd-zfs ada0s3
ada0s3a added

!!! Add our swap partition !!!

[script]# gpart add -t swap ada0s3
ada0s3b added

!!! Show the resulting disklabel !!!

[script]# gpart show ada0s3
=> 0 67108864 ada0s3 BSD (32G)
0 58720256 1 freebsd-zfs (28G)
58720256 8388608 2 freebsd-swap (4.0G)

!!! Load the ZFS module, and optionally set the ashift property to 12 !!!

[script]# kldload zfs
[script]# sysctl vfs.zfs.min_auto_ashift=12
vfs.zfs.min_auto_ashift: 9 -> 12

!!! Create our zpool, YMMV !!!

[script]# zpool create -O mountpoint=legacy zroot /dev/ada0s3a

!!! Create our initial BE, YMMV !!!

[script]# uname -a
FreeBSD 11.1-STABLE FreeBSD 11.1-STABLE #0 r327786: Wed Jan 10 22:26:38 UTC 2018 ***@releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
[script]# zfs create zroot/ROOT
[script]# zfs create zroot/ROOT/20180110-r327786
[script]# zfs set mountpoint=/tmp/zroot zroot/ROOT/20180110-r327786

!!! The BE is temporary mounted at /tmp/zroot !!!

[script]# df -ah
Filesystem Size Used Avail Capacity Mounted on
/dev/iso9660/11_1_STABLE_AMD64_CD 701M 701M 0B 100% /
devfs 1.0K 1.0K 0B 100% /dev
tmpfs 32M 124K 32M 0% /var
tmpfs 20M 68K 20M 0% /tmp
zroot/ROOT/20180110-r327786 27G 88K 27G 0% /tmp/zroot

!!! Extract a minimum installation !!!

[script]# cd /tmp/zroot
[script]# tar xf /usr/freebsd-dist/base.txz
[script]# tar xf /usr/freebsd-dist/kernel.txz
[script]# tar xf /usr/freebsd-dist/lib32.txz

!!! Ensure the ZFS module will be loaded by the loader !!!

[script]# echo 'zfs_load="YES"' >> boot/loader.conf

!!! Make a note of our swap partition !!!

[script]# echo '/dev/ada0s3b none swap sw 0 0' >> etc/fstab

!! Prepare our pool for export !!!

[script]# zfs unmount -a
[script]# zfs inherit mountpoint zroot/ROOT/20180110-r327786
[script]# zpool set bootfs=zroot/ROOT/20180110-r327786 zroot
[script]# df -ah
Filesystem Size Used Avail Capacity Mounted on
/dev/iso9660/11_1_STABLE_AMD64_CD 701M 701M 0B 100% /
devfs 1.0K 1.0K 0B 100% /dev
tmpfs 32M 128K 32M 0% /var
tmpfs 20M 68K 20M 0% /tmp
[script]# zpool export zroot

!!! Write the boot manager to the boot block in the MBR !!!

[script]# gpart bootcode -b /boot/boot0 /dev/ada0
bootcode written to ada0

!!! Write /boot/zfsboot as two separate pieces to /dev/ada0s3{,a} !!!

[script]# dd if=/boot/zfsboot of=/tmp/zfsboot1 count=1
1+0 records in
1+0 records out
512 bytes transferred in 0.000383 secs (1337758 bytes/sec)
[script]# gpart bootcode -b /tmp/zfsboot1 /dev/ada0s3
bootcode written to ada0s3

[script]# dd if=/boot/zfsboot of=/dev/ada0s3a skip=1 seek=1024
128+0 records in
128+0 records out
65536 bytes transferred in 0.043591 secs (1503439 bytes/sec)

!!! Import our pool !!!

[script]# zpool import zroot
[script]# zfs list -rt all
NAME USED AVAIL REFER MOUNTPOINT
zroot 742M 26.2G 88K legacy
zroot/ROOT 742M 26.2G 88K legacy
zroot/ROOT/20180110-r327786 742M 26.2G 742M legacy

!!! Everything checks out, it's time for a reboot to assess the damage !!!

[script]# exit

Script done on Tue Jan 16 08:53:07 2018

Both Windows 10 and FreeBSD boots, so I would call this a success.
--
Trond.
Victor Sudakov
2018-01-16 11:28:14 UTC
Permalink
Post by Trond Endrestøl
I couldn't resist attempting a proof of concept, so here it is.
!!! Show the resulting disklabel !!!
[script]# gpart show ada0s3
=> 0 67108864 ada0s3 BSD (32G)
0 58720256 1 freebsd-zfs (28G)
58720256 8388608 2 freebsd-swap (4.0G)
How funny! I did not even know that fstype in the disklabel can be "ZFS". I
have only seen "swap" and "4.2BSD" so far.

$ gpart add -t freebsd-zfs md0s1 && disklabel md0s1
md0s1a added
# /dev/md0s1:
8 partitions:
# size offset fstype [fsize bsize bps/cpg]
a: 4095 0 ZFS
c: 4095 0 unused 0 0 # "raw" part, don't edit
$

[dd]
Post by Trond Endrestøl
!!! Create our zpool, YMMV !!!
!!! Create our initial BE, YMMV !!!
Do you know how to create a beadm-friendly zroot manually (like the one
created automatically by bsdinstall)?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Trond Endrestøl
2018-01-16 12:11:54 UTC
Permalink
Post by Victor Sudakov
Post by Trond Endrestøl
I couldn't resist attempting a proof of concept, so here it is.
!!! Show the resulting disklabel !!!
[script]# gpart show ada0s3
=> 0 67108864 ada0s3 BSD (32G)
0 58720256 1 freebsd-zfs (28G)
58720256 8388608 2 freebsd-swap (4.0G)
How funny! I did not even know that fstype in the disklabel can be "ZFS". I
have only seen "swap" and "4.2BSD" so far.
$ gpart add -t freebsd-zfs md0s1 && disklabel md0s1
md0s1a added
# size offset fstype [fsize bsize bps/cpg]
a: 4095 0 ZFS
c: 4095 0 unused 0 0 # "raw" part, don't edit
$
[dd]
Post by Trond Endrestøl
!!! Create our zpool, YMMV !!!
!!! Create our initial BE, YMMV !!!
Do you know how to create a beadm-friendly zroot manually (like the one
created automatically by bsdinstall)?
I have created my own recipe based on the guides published elsewhere,
including those on the FreeBSD wiki, and as usual I have applied
thoughts from my own lurid mind.

Have a look at my files at
https://ximalas.info/~trond/create-zfs/canmount/

I create the disk layout manually, see 00-create-gpart-layout-UEFI.txt
and 00-create-gpart-layout.txt for some ideas.

I use a SysV approach when creating the ZFS filesystem layout and
installing the system, i.e. lots and lots of environment variables.

See 01-create-zfs-layout.sh, 02-temp-mountpoints.sh,
03b-install-stable-9-10-11-or-head.sh, and 04-final-mountpoints.sh.

For special cases such as my mail server, I edited
01-create-zfs-layout.sh to suit the two pools, one for the system and
another one for user data. All mail related filesystems ended up in
the data pool.

Between steps 3 and 4, I edit various files, set the root password and
the timezone, ensuring sendmail's files are up & running in /etc/mail,
all done from within chroot $DESTDIR.

I know some like to use snapshots and clones as a safety belt before
they upgrade their main BE.

I do the opposite, I create a snapshot and a clone, install the new
world and kernel into the clone, merge config files, update the bootfs
pool property, and reboot into the new clone. To me, this saves time
while giving me plenty of seatbelts to boot from should I need to.

Running -CURRENT on some of my VMs has forced me to use my old clones
to recover from clang bugs, etc.

On VMs I restrain the number of snapshots/clones/BE to 3. The current
BE and the 2 previous ones, and the snapshots that tie them all
together. Physical systems usually have more than enough storage, and
I clean up the long list of BEs about once a year (zfs promote, zfs
destroy -Rv).

Here's a good execise on creating snapshots and clones, and how to
clean them up:
https://ximalas.info/2015/06/23/an-exercise-on-zfs-clones/
--
Trond.
Matthew Seaman
2018-01-16 12:15:27 UTC
Permalink
Post by Victor Sudakov
Do you know how to create a beadm-friendly zroot manually (like the one
created automatically by bsdinstall)?
This is not particularly difficult. First, you need to consider the
standard directory structure and mentally divide the ZFSes containing
this content into two classes:

* The Kernel, kernel modules, binaries, shared libraries, application
and system configuration files that would be affected by a system
upgrade.

* Working data, temporary files, documents, log files etc. which
would not be affected by a system upgrade.

The first type should be part of the boot environment files, the second
type outside it. If necessary you may need to create additional ZFSes
to divide content appropriately.

For my systems this works out to a breakdown like this:

/ --> BE
(This includes /boot, /etc)

/tmp --> not. Not a ZFS. I usually use a tmpfs for /tmp
/dev --> not. Not a ZFS. Uses devfs

/home --> not. I generally remove the symlink from /usr/home and
make /home a first class ZFS directly.

/usr --> BE
/usr/local --> BE
/usr/obj --> BE
/usr/src --> BE
/usr/ports --> not
/usr/ports/distfiles --> not

/var --> BE
/var/crash --> not
/var/db --> BE
/var/db/pkg --> BE
/var/db/postgres --> not
/var/empty --> not
/var/log --> not
/var/run --> not
/var/tmp --> not
/var/mail --> not

Plus various other ZFSes for jails and poudriere etc. Now, assuming
your root zpool is called zroot, and the BE name is '11.1-RELEASE-p6'
you will want to create ZFSes like so:

ZFS Mountpoint Canmount
zroot / no
zroot/ROOT /ROOT no

zroot/ROOT/11.1-RELEASE-p6 / yes
zroot/ROOT/11.1-RELEASE-p6/usr /usr yes
zroot/ROOT/11.1-RELEASE-p6/var /var yes
zroot/ROOT/11.1-RELEASE-p6/var/db /var/db yes

...etc...

zroot/home /home yes
zroot/var /var no
zroot/var/log /var/log yes
zroot/var/db /var/db no
zroot/var/db/postgres /var/db/postgres yes

...etc...

Note the trick of creating two different ZFSes with the same mountpoint,
where one is actually mounted and contains files, and the other is never
mounted, contains nothing and only serves to provide the inheritance of
values for child ZFSes. This allows mounting a filesystem from outside
the BE (eg. /var/log) beneath a directory from within the BE (eg. /var)

If this seems overly complex, well, it is. That's the price you pay for
taking the familiar Unix filesystem hierarchy and adapting it to
filesystem semantics developed decades later. If we were starting from
scratch laying out an OS based on ZFS we'd certainly arrange things a
bit more conveniently for use with boot environments.

Cheers,

Matthew
Victor Sudakov
2018-01-16 14:35:59 UTC
Permalink
Matthew Seaman wrote:

[dd]
Post by Matthew Seaman
Post by Victor Sudakov
Do you know how to create a beadm-friendly zroot manually (like the one
created automatically by bsdinstall)?
This is not particularly difficult. First, you need to consider the
standard directory structure and mentally divide the ZFSes containing
* The Kernel, kernel modules, binaries, shared libraries, application
and system configuration files that would be affected by a system
upgrade.
* Working data, temporary files, documents, log files etc. which
would not be affected by a system upgrade.
The first type should be part of the boot environment files, the second
type outside it. If necessary you may need to create additional ZFSes
to divide content appropriately.
This is clear, but...
Post by Matthew Seaman
Plus various other ZFSes for jails and poudriere etc. Now, assuming
your root zpool is called zroot, and the BE name is '11.1-RELEASE-p6'
ZFS Mountpoint Canmount
zroot / no
zroot/ROOT /ROOT no
zroot/ROOT/11.1-RELEASE-p6 / yes
zroot/ROOT/11.1-RELEASE-p6/usr /usr yes
zroot/ROOT/11.1-RELEASE-p6/var /var yes
zroot/ROOT/11.1-RELEASE-p6/var/db /var/db yes
...etc...
zroot/home /home yes
zroot/var /var no
zroot/var/log /var/log yes
zroot/var/db /var/db no
zroot/var/db/postgres /var/db/postgres yes
Your layout is different from that created automatically by
bsdinstall. There is a principal difference.

The default layout does not have zroot/ROOT/${BE}/usr,
zroot/ROOT/${BE}/var etc. datasets like yours does. Still somehow when
I create a BE with beadm, the files in /usr/bin etc end up being
included in the BE. I don't understand how this works:

zroot 33,3G 13,0G 96K /zroot
zroot/ROOT 9,37G 13,0G 96K none
zroot/ROOT/default 9,37G 13,0G 8,81G /
zroot/ROOT/install 8K 13,0G 585M /
zroot/ROOT/some_test_be 8K 13,0G 8,81G /
zroot/tmp 56,9M 13,0G 47,6M /tmp
zroot/usr 23,7G 13,0G 96K /usr
zroot/usr/home 23,7G 13,0G 22,3G /usr/home
zroot/usr/ports 96K 13,0G 96K /usr/ports
zroot/usr/src 96K 13,0G 96K /usr/src
zroot/var 126M 13,0G 96K /var
zroot/var/audit 96K 13,0G 96K /var/audit
zroot/var/crash 96K 13,0G 96K /var/crash
zroot/var/log 2,21M 13,0G 1,06M /var/log
zroot/var/mail 15,1M 13,0G 15,0M /var/mail
zroot/var/tmp 109M 13,0G 109M /var/tmp

***@vas:~ # beadm list
BE Active Mountpoint Space Created
default NR / 9.4G 2016-08-13 22:50
install - - 391.0M 2016-08-13 23:01
some_test_be - - 192.0K 2018-01-16 21:20
***@vas:~ # beadm mount some_test_be
Mounted successfully on '/tmp/BE-some_test_be.WjrL7kVi'
***@vas:~ # ls /tmp/BE-some_test_be.WjrL7kVi/usr/bin/mail
/tmp/BE-some_test_be.WjrL7kVi/usr/bin/mail
***@vas:~ #

zroot/usr is not below zroot/ROOT/some_test_be, but somehow it's
there in the BE. Can you please explain this?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Marko Turk
2018-01-16 14:44:58 UTC
Permalink
Post by Victor Sudakov
zroot/usr is not below zroot/ROOT/some_test_be, but somehow it's
there in the BE. Can you please explain this?
Hi,

that's because zroot/usr is not mounted (check 'zfs get all zroot/usr').

/Marko
Victor Sudakov
2018-01-16 15:54:31 UTC
Permalink
Post by Marko Turk
Post by Victor Sudakov
zroot/usr is not below zroot/ROOT/some_test_be, but somehow it's
there in the BE. Can you please explain this?
Hi,
that's because zroot/usr is not mounted (check 'zfs get all zroot/usr').
Yes, I know it's not mounted (canmount=off), but it's not a child
dataset of zroot/ROOT/some_test_be, and yet it's somehow within
zroot/ROOT/some_test_be when the BE was cloned by beadm.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Marko Turk
2018-01-16 15:57:19 UTC
Permalink
Post by Victor Sudakov
Post by Marko Turk
Post by Victor Sudakov
zroot/usr is not below zroot/ROOT/some_test_be, but somehow it's
there in the BE. Can you please explain this?
Hi,
that's because zroot/usr is not mounted (check 'zfs get all zroot/usr').
Yes, I know it's not mounted (canmount=off), but it's not a child
dataset of zroot/ROOT/some_test_be, and yet it's somehow within
zroot/ROOT/some_test_be when the BE was cloned by beadm.
Hi,

since zroot/usr is not mounted, path /usr/bin/* in the filesystem
belongs to dataset zroot/ROOT/some_test_be so that's why the files are
there.

E.g.:
$ df -h /usr/bin/
Filesystem Size Used Avail Capacity Mounted on
zroot/ROOT/default 504G 19G 485G 4% /

/Marko
Victor Sudakov
2018-01-16 16:52:06 UTC
Permalink
Post by Marko Turk
Post by Victor Sudakov
Post by Marko Turk
Post by Victor Sudakov
zroot/usr is not below zroot/ROOT/some_test_be, but somehow it's
there in the BE. Can you please explain this?
Hi,
that's because zroot/usr is not mounted (check 'zfs get all zroot/usr').
Yes, I know it's not mounted (canmount=off), but it's not a child
dataset of zroot/ROOT/some_test_be, and yet it's somehow within
zroot/ROOT/some_test_be when the BE was cloned by beadm.
Hi,
since zroot/usr is not mounted, path /usr/bin/* in the filesystem
belongs to dataset zroot/ROOT/some_test_be
But why is that? This must be witchcraft :-)

Why does it not belong to dataset "zroot", or dataset
"zroot/ROOT/some_other_great_be"?

Is it because zroot/ROOT/some_test_be is mounted on / ?
Post by Marko Turk
so that's why the files are
there.
$ df -h /usr/bin/
Filesystem Size Used Avail Capacity Mounted on
zroot/ROOT/default 504G 19G 485G 4% /
Very demonstrative, thank you, but what makes "zroot/ROOT/default" and
not any other dataset the container for the files in /usr/bin/ ?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Marko Turk
2018-01-17 07:10:06 UTC
Permalink
Hi,
Post by Victor Sudakov
Post by Marko Turk
since zroot/usr is not mounted, path /usr/bin/* in the filesystem
belongs to dataset zroot/ROOT/some_test_be
But why is that? This must be witchcraft :-)
Why does it not belong to dataset "zroot", or dataset
"zroot/ROOT/some_other_great_be"?
Is it because zroot/ROOT/some_test_be is mounted on / ?
Yes, exactly.
Post by Victor Sudakov
Post by Marko Turk
so that's why the files are
there.
$ df -h /usr/bin/
Filesystem Size Used Avail Capacity Mounted on
zroot/ROOT/default 504G 19G 485G 4% /
Very demonstrative, thank you, but what makes "zroot/ROOT/default" and
not any other dataset the container for the files in /usr/bin/ ?
Because of the mountpoints.

zroot/usr exists so you can create children of it which will not be in a
BE.

For example, zroot/usr/src is not a part of a BE:

$ df -h /usr/src/
Filesystem Size Used Avail Capacity Mounted on
zroot/usr/src 487G 1.6G 485G 0% /usr/src

/Marko
Victor Sudakov
2018-01-18 05:14:23 UTC
Permalink
Post by Trond Endrestøl
[script]# sysctl vfs.zfs.min_auto_ashift=12
vfs.zfs.min_auto_ashift: 9 -> 12
Why do you change this?

"Min ashift used when creating new top-level vdevs", hmm. You probably
know something I don't.
[dd]
Post by Trond Endrestøl
!!! Write the boot manager to the boot block in the MBR !!!
[script]# gpart bootcode -b /boot/boot0 /dev/ada0
bootcode written to ada0
Fine, here you install the bootmanager into the MBR. However I have a
couple of questions below regarding the VBR.
Post by Trond Endrestøl
!!! Write /boot/zfsboot as two separate pieces to /dev/ada0s3{,a} !!!
[script]# dd if=/boot/zfsboot of=/tmp/zfsboot1 count=1
1+0 records in
1+0 records out
512 bytes transferred in 0.000383 secs (1337758 bytes/sec)
[script]# gpart bootcode -b /tmp/zfsboot1 /dev/ada0s3
bootcode written to ada0s3
Why do you need this trick with /tmp/zfsboot1 ? Why not

sysctl kern.geom.debugflags=0x10
dd if=/boot/zfsboot of=/dev/ada0s3 count=1

The example in zfsboot(8) is even weirder. They create a BSD label
within an MBR slice, but never any partitions within. Why do they do
it like that? Quoting the manual page:

gpart create -s BSD ada0s1
gpart bootcode -b /boot/boot0 ada0
gpart set -a active -i 1 ada0
dd if=/boot/zfsboot of=/dev/ada0s1 count=1
dd if=/boot/zfsboot of=/dev/ada0s1 iseek=1 oseek=1024
Post by Trond Endrestøl
[script]# dd if=/boot/zfsboot of=/dev/ada0s3a skip=1 seek=1024
128+0 records in
128+0 records out
65536 bytes transferred in 0.043591 secs (1503439 bytes/sec)
Should the "of=" target here be ada0s3a or just ada0s3 ?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Trond Endrestøl
2018-01-18 08:48:38 UTC
Permalink
Post by Victor Sudakov
Post by Trond Endrestøl
[script]# sysctl vfs.zfs.min_auto_ashift=12
vfs.zfs.min_auto_ashift: 9 -> 12
Why do you change this?
"Min ashift used when creating new top-level vdevs", hmm. You probably
know something I don't.
[dd]
If you know you will be using 4Kn (AF) drives now or in the near
future, you might as well prepare ZFS. Otherwise leave ashift at 9.
Post by Victor Sudakov
Post by Trond Endrestøl
!!! Write /boot/zfsboot as two separate pieces to /dev/ada0s3{,a} !!!
[script]# dd if=/boot/zfsboot of=/tmp/zfsboot1 count=1
1+0 records in
1+0 records out
512 bytes transferred in 0.000383 secs (1337758 bytes/sec)
[script]# gpart bootcode -b /tmp/zfsboot1 /dev/ada0s3
bootcode written to ada0s3
Why do you need this trick with /tmp/zfsboot1 ? Why not
sysctl kern.geom.debugflags=0x10
dd if=/boot/zfsboot of=/dev/ada0s3 count=1
The example in zfsboot(8) is even weirder. They create a BSD label
within an MBR slice, but never any partitions within. Why do they do
gpart create -s BSD ada0s1
gpart bootcode -b /boot/boot0 ada0
gpart set -a active -i 1 ada0
dd if=/boot/zfsboot of=/dev/ada0s1 count=1
dd if=/boot/zfsboot of=/dev/ada0s1 iseek=1 oseek=1024
Post by Trond Endrestøl
[script]# dd if=/boot/zfsboot of=/dev/ada0s3a skip=1 seek=1024
128+0 records in
128+0 records out
65536 bytes transferred in 0.043591 secs (1503439 bytes/sec)
Should the "of=" target here be ada0s3a or just ada0s3 ?
I followed step 10 of
https://wiki.freebsd.org/RootOnZFS/ZFSBootPartition.
I'm sure you can optimize away the drudgery.

The key is to place some of the bootcode in the boot block of the
disklabel and the remainder in the boot block of the zpool.
--
Trond.
Victor Sudakov
2018-01-19 04:05:45 UTC
Permalink
Trond Endrest?l wrote:

[dd]
Post by Trond Endrestøl
The key is to place some of the bootcode in the boot block of the
disklabel and the remainder in the boot block of the zpool.
OK, I'm experimenting in bhyve anyway, so I can put all that bootblock
hassle for later. bhyve does not use the bootblocks anyway.

There is another issue that I don't understand. After creating the
root zpool like the howtos (and you) recommend, I get a bootable
system, but for some reason all the children datasets of the BE
are not mounted after boot. Like this:

***@test1:~ # mount
zroot3/ROOT/default on / (zfs, local, nfsv4acls)
devfs on /dev (devfs, local, multilabel)
***@test1:~ #

***@test1:~ # zfs list -o name,used,avail,refer,mountpoint,canmount,mounted
NAME USED AVAIL REFER MOUNTPOINT CANMOUNT MOUNTED
zroot3 1.05G 6.64G 23K /zroot3 on no
zroot3/ROOT 1.05G 6.64G 23K none on no
zroot3/ROOT/default 1.05G 6.64G 1.05G / on yes
zroot3/tmp 24K 6.64G 24K /tmp on no
zroot3/usr 92K 6.64G 23K /usr off no
zroot3/usr/home 23K 6.64G 23K /usr/home on no
zroot3/usr/ports 23K 6.64G 23K /usr/ports on no
zroot3/usr/src 23K 6.64G 23K /usr/src on no
zroot3/var 266K 6.64G 23K /var off no
zroot3/var/audit 23K 6.64G 23K /var/audit on no
zroot3/var/crash 23.5K 6.64G 23.5K /var/crash on no
zroot3/var/log 150K 6.64G 150K /var/log on no
zroot3/var/mail 23K 6.64G 23K /var/mail on no
zroot3/var/tmp 24K 6.64G 24K /var/tmp on no
***@test1:~ #

What have I forgotten when creating the pools?
This is how I did it: http://termbin.com/awuy
Based on https://wiki.freebsd.org/RootOnZFS/ZFSBootPartition

I can mount all those datasets manually though:

***@test1:~ # zfs mount zroot3/var/mail
***@test1:~ # mount
zroot3/ROOT/default on / (zfs, local, nfsv4acls)
devfs on /dev (devfs, local, multilabel)
zroot3/var/mail on /var/mail (zfs, local, nfsv4acls)
***@test1:~ #

But for some reason it's not happening on boot.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Marko Turk
2018-01-19 07:14:36 UTC
Permalink
Post by Victor Sudakov
There is another issue that I don't understand. After creating the
root zpool like the howtos (and you) recommend, I get a bootable
system, but for some reason all the children datasets of the BE
zroot3/ROOT/default on / (zfs, local, nfsv4acls)
devfs on /dev (devfs, local, multilabel)
NAME USED AVAIL REFER MOUNTPOINT CANMOUNT MOUNTED
zroot3 1.05G 6.64G 23K /zroot3 on no
zroot3/ROOT 1.05G 6.64G 23K none on no
zroot3/ROOT/default 1.05G 6.64G 1.05G / on yes
zroot3/tmp 24K 6.64G 24K /tmp on no
zroot3/usr 92K 6.64G 23K /usr off no
zroot3/usr/home 23K 6.64G 23K /usr/home on no
zroot3/usr/ports 23K 6.64G 23K /usr/ports on no
zroot3/usr/src 23K 6.64G 23K /usr/src on no
zroot3/var 266K 6.64G 23K /var off no
zroot3/var/audit 23K 6.64G 23K /var/audit on no
zroot3/var/crash 23.5K 6.64G 23.5K /var/crash on no
zroot3/var/log 150K 6.64G 150K /var/log on no
zroot3/var/mail 23K 6.64G 23K /var/mail on no
zroot3/var/tmp 24K 6.64G 24K /var/tmp on no
What have I forgotten when creating the pools?
This is how I did it: http://termbin.com/awuy
Based on https://wiki.freebsd.org/RootOnZFS/ZFSBootPartition
zroot3/ROOT/default on / (zfs, local, nfsv4acls)
devfs on /dev (devfs, local, multilabel)
zroot3/var/mail on /var/mail (zfs, local, nfsv4acls)
But for some reason it's not happening on boot.
Hi,

take a look at:
https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/zfs-quickstart.html

/Marko
Trond Endrestøl
2018-01-19 08:14:00 UTC
Permalink
Post by Victor Sudakov
There is another issue that I don't understand. After creating the
root zpool like the howtos (and you) recommend, I get a bootable
system, but for some reason all the children datasets of the BE
are not mounted after boot.
You are probably missing
zfs_enable="YES"
in /etc/rc.conf.

You should also consider enabling zfsd(8), i.e.
zfsd_enable="YES"
--
Trond.
Victor Sudakov
2018-01-22 01:29:17 UTC
Permalink
Post by Trond Endrestøl
Post by Victor Sudakov
There is another issue that I don't understand. After creating the
root zpool like the howtos (and you) recommend, I get a bootable
system, but for some reason all the children datasets of the BE
are not mounted after boot.
You are probably missing
zfs_enable="YES"
in /etc/rc.conf.
I new I was missing something obvious. Thank you. However,
zfs_enable="YES" could be implicit if($vfs.root.mountfrom =~ m/^zfs/)
Post by Trond Endrestøl
You should also consider enabling zfsd(8), i.e. zfsd_enable="YES"
Not in 10.4 I'm experimenting with.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Victor Sudakov
2018-01-31 08:27:38 UTC
Permalink
My pool works successfully in a bhyve environment, but in VirtualBox 5.0.14
(to test the real boot sequence) I get this message from zfsboot:

zfsboot: No ZFS pools located, can't boot

However, if I boot from LiveCD, the pool is there, it's healthy and
can be imported.

Why is it that zfsboot is not finding the pool? Is it because the pool
has been created in /dev/ada0s1a instead of /dev/ada0s1 ?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Frank Leonhardt
2018-02-05 15:23:43 UTC
Permalink
Post by Victor Sudakov
My pool works successfully in a bhyve environment, but in VirtualBox 5.0.14
zfsboot: No ZFS pools located, can't boot
However, if I boot from LiveCD, the pool is there, it's healthy and
can be imported.
Why is it that zfsboot is not finding the pool? Is it because the pool
has been created in /dev/ada0s1a instead of /dev/ada0s1 ?
Seems likely!

zfsboot will attempt to get a pool together from BIOS-visible disks.
That probably means MBR but your BIOS may vary - especially one on a
virtual environment. gptzfsboot might do better. But I'd look at the
virtual machine BIOS configuration first.

Once you've booted (e.g. from a LiveCD), disks and slices not visible to
the BIOS become fair game for forming a zpool.

Regards, Frank.
Victor Sudakov
2018-03-19 02:59:14 UTC
Permalink
Post by Victor Sudakov
My pool works successfully in a bhyve environment, but in VirtualBox 5.0.14
zfsboot: No ZFS pools located, can't boot
However, if I boot from LiveCD, the pool is there, it's healthy and
can be imported.
Finally, and thanks to Eugene Grossbein, the problem is solved.

The gist of the problem was the presence of a BSD label in the slice
where the bootable zfs pool was located. It turns out that zfsboot
ignores the slice if it finds a BSD label therein. The bootable zpool
should be directly on the slice (like on da1s1, and not da1s1a).

This means that both zfsboot(8) and the articles in the FreeBSD wiki
are erroneous because they all suggest "gpart create -s BSD ada0s1" -
in fact, it's fatal.


Below is the final working example:

================================================
#!/bin/sh

sysctl kern.geom.debugflags=0x10

DISK=da1
POOL="zroot-test3"
NEWSYSTEM=newsystem
SLICE=2

partition() {
gpart create -s mbr ${DISK}
gpart add -t fat32 -s 1G ${DISK}
gpart add -t freebsd -s 2G -i ${SLICE} ${DISK}
gpart set -a active -i ${SLICE} ${DISK}
}

bootcode() {
gpart bootcode -b /boot/boot0 ${DISK}
dd if=/boot/zfsboot of=/dev/${DISK}s${SLICE} count=1
dd if=/boot/zfsboot of=/dev/${DISK}s${SLICE} iseek=1 oseek=1024

}

zfscreate() {
zpool create -m none -R /${NEWSYSTEM} ${POOL} ${DISK}s${SLICE}

zfs create -o mountpoint=none ${POOL}/ROOT
zfs create -o mountpoint=/ ${POOL}/ROOT/default

zfs create -o mountpoint=/usr -o canmount=off ${POOL}/usr
zfs create -o mountpoint=/var -o canmount=off ${POOL}/var

zfs create -o mountpoint=/tmp ${POOL}/tmp
zfs create -o mountpoint=/usr/home ${POOL}/usr/home
zfs create -o mountpoint=/var/audit ${POOL}/var/audit
zfs create -o mountpoint=/var/crash ${POOL}/var/crash
zfs create -o mountpoint=/var/log ${POOL}/var/log
zfs create -o mountpoint=/var/mail ${POOL}/var/mail
zfs create -o mountpoint=/var/tmp ${POOL}/var/tmp

zfs create -V 1G -o org.freebsd:swap=on ${POOL}/swap

zpool set bootfs=${POOL}/ROOT/default ${POOL}
}

clone() {
cd /${NEWSYSTEM} || exit 3
#sleep 10
dump -0af - / | restore -ryf - || exit 3
mv etc/fstab etc/fstab.bak
echo '# empty' > etc/fstab
echo 'zfs_enable="YES"' >> etc/rc.conf.local
echo 'zfs_load="YES"' >> boot/loader.conf
cd /root
}

zfsexport() {
zpool export ${POOL}
}

partition
bootcode
zfscreate
clone
zfsexport
================================================
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Matthias Fechner
2018-03-19 06:14:02 UTC
Permalink
Post by Victor Sudakov
partition() {
gpart create -s mbr ${DISK}
is there a special reason why you use mbr instead of gpt?


Gruß,
Matthias
--
"Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs, and the universe trying to
produce bigger and better idiots. So far, the universe is winning." --
Rich Cook
Victor Sudakov
2018-03-19 17:10:28 UTC
Permalink
Post by Matthias Fechner
Post by Victor Sudakov
partition() {
gpart create -s mbr ${DISK}
is there a special reason why you use mbr instead of gpt?
MS-DOS and Windows XP compatibility, for example. I think even Windows 7
uses MBR by default. For me, MBR is preferred for a dualboot
configuration for compatibility reasons.

Besides, I don't know of a small simple boot manager (like boot0) for
GPT. Can you advise one?

PS if I install a dedicated FreeBSD system, I usually go for GPT.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Matthias Fechner
2018-03-20 06:47:49 UTC
Permalink
Hi Victor,
Post by Victor Sudakov
Besides, I don't know of a small simple boot manager (like boot0) for
GPT. Can you advise one?
PS if I install a dedicated FreeBSD system, I usually go for GPT.
hm, I used already an EFI partition with gpt to boot Freebsd, but I
never used FreeBSD on a computer with dual boot, so sry I cannot give
you any advice here.
But Win7 should be able to use GPT as standard partition schema.

Gruß,
Matthias
--
"Programming today is a race between software engineers striving to
build bigger and better idiot-proof programs, and the universe trying to
produce bigger and better idiots. So far, the universe is winning." --
Rich Cook
Victor Sudakov
2018-03-20 07:12:14 UTC
Permalink
Post by Matthias Fechner
Post by Victor Sudakov
Besides, I don't know of a small simple boot manager (like boot0) for
GPT. Can you advise one?
PS if I install a dedicated FreeBSD system, I usually go for GPT.
hm, I used already an EFI partition with gpt to boot Freebsd, but I
never used FreeBSD on a computer with dual boot, so sry I cannot give
you any advice here.
But Win7 should be able to use GPT as standard partition schema.
Win7 can use GPT, but by default the Win7 installer creates an MBR
schema, especially when it does not detect a UEFI boot. That's what I
have been told by our Windows admins. They say Win7 can be forced to
create a GPT schema, but you have to resort to manual disk
partitioning.

FreeBSD being the second or third OS in the dualboot, you know, I have
to deal with what I already have.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Trond Endrestøl
2018-03-20 07:21:39 UTC
Permalink
Post by Matthias Fechner
But Win7 should be able to use GPT as standard partition schema.
Not for booting. That became possible with Win8 onwards.
--
Trond.
Trond Endrestøl
2018-03-20 07:13:05 UTC
Permalink
Post by Victor Sudakov
Besides, I don't know of a small simple boot manager (like boot0) for
GPT. Can you advise one?
Try GRUB2 or rEFInd.
My laptop is pretty happy about the latter.

https://www.gnu.org/software/grub/
http://www.rodsbooks.com/refind/
--
Trond.
Victor Sudakov
2018-03-20 09:46:25 UTC
Permalink
Post by Trond Endrestøl
Post by Victor Sudakov
Besides, I don't know of a small simple boot manager (like boot0) for
GPT. Can you advise one?
Try GRUB2 or rEFInd.
My laptop is pretty happy about the latter.
https://www.gnu.org/software/grub/
http://www.rodsbooks.com/refind/
I will most certainly look at rEFInd, thank you.

To boot FreeBSD with it, do you follow the procedure described in
https://wiki.freebsd.org/UEFI ?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
krad
2018-03-20 09:58:36 UTC
Permalink
Dual booting makes me shudder these days, I would need really solid reasons
for having to do it, but everyone is different..
Post by Victor Sudakov
Post by Trond Endrestøl
Post by Victor Sudakov
Besides, I don't know of a small simple boot manager (like boot0) for
GPT. Can you advise one?
Try GRUB2 or rEFInd.
My laptop is pretty happy about the latter.
https://www.gnu.org/software/grub/
http://www.rodsbooks.com/refind/
I will most certainly look at rEFInd, thank you.
To boot FreeBSD with it, do you follow the procedure described in
https://wiki.freebsd.org/UEFI ?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
_______________________________________________
https://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to "freebsd-questions-
Victor Sudakov
2018-03-21 02:21:40 UTC
Permalink
Post by krad
Dual booting makes me shudder these days, I would need really solid reasons
for having to do it, but everyone is different..
Dualboot is very convenient to have on an engineer's notebook because
some tasks are better done in Unix while others in Windows. I can
elaborate on that if you are interested.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Victor Sudakov
2018-03-26 09:47:03 UTC
Permalink
Post by Victor Sudakov
Post by krad
Dual booting makes me shudder these days, I would need really
solid reasons for having to do it, but everyone is different..
Dualboot is very convenient to have on an engineer's notebook because
some tasks are better done in Unix while others in Windows. I can
elaborate on that if you are interested.
I agree, but i virtualize as much as possible. Some things wont do for that
though, which is why i have an external drive to boot off if needed.
Slightly more physically cumbersome, but a lot less pain in general.
I also keep a couple of bootable USB sticks (with a regular FreeBSD
and with mfsBSD) but having a system installed on a HDD is less
cumbersome, especially if it's a graphical environment like mate.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Trond Endrestøl
2018-03-20 11:43:20 UTC
Permalink
Post by Victor Sudakov
Post by Trond Endrestøl
Post by Victor Sudakov
Besides, I don't know of a small simple boot manager (like boot0) for
GPT. Can you advise one?
Try GRUB2 or rEFInd.
My laptop is pretty happy about the latter.
https://www.gnu.org/software/grub/
http://www.rodsbooks.com/refind/
I will most certainly look at rEFInd, thank you.
To boot FreeBSD with it, do you follow the procedure described in
https://wiki.freebsd.org/UEFI ?
No, I took matters into my own hands.

I first installed the least cooperative OS, Win10.

I manually installed FreeBSD with ZFS.

I extracted rEFInd to the ESP.

I copied the boot1.efi bootloader to a \FreeBSD directory on the ESP.

I configured rEFInd to suit my needs.

I configured some boot entries in the UEFI environment of my laptop,
one for each of rEFInd, Windows 10, and FreeBSD, and made the entry
for rEFInd the preferred one.

You don't need to read the entire UEFI specification (2575 pages
long), but I do recommend browsing through chapter 5 (GUID Partition
Table (GPT) Disk Layout), and the entire section 13.3 (File System
Format).

http://uefi.org/specifications
--
Trond.
Victor Sudakov
2018-03-21 06:12:31 UTC
Permalink
Post by Trond Endrestøl
Post by Victor Sudakov
Post by Trond Endrestøl
Post by Victor Sudakov
Besides, I don't know of a small simple boot manager (like boot0) for
GPT. Can you advise one?
Try GRUB2 or rEFInd.
My laptop is pretty happy about the latter.
https://www.gnu.org/software/grub/
http://www.rodsbooks.com/refind/
I will most certainly look at rEFInd, thank you.
To boot FreeBSD with it, do you follow the procedure described in
https://wiki.freebsd.org/UEFI ?
No, I took matters into my own hands.
I first installed the least cooperative OS, Win10.
I manually installed FreeBSD with ZFS.
I extracted rEFInd to the ESP.
Who created the ESP partition for you?
Post by Trond Endrestøl
I copied the boot1.efi bootloader to a \FreeBSD directory on the ESP.
I configured rEFInd to suit my needs.
I configured some boot entries in the UEFI environment of my laptop,
one for each of rEFInd, Windows 10, and FreeBSD, and made the entry
for rEFInd the preferred one.
Why do you need rEFInd at all if your laptop's UEFI environment can
choose which OS to boot?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Trond Endrestøl
2018-03-21 08:55:10 UTC
Permalink
Post by Victor Sudakov
Post by Trond Endrestøl
I extracted rEFInd to the ESP.
Who created the ESP partition for you?
Windows 10.

The UEFI spec allows for additional ESPs and the FreeBSD installer
creates one of its own, but I reckon one is sufficient. Besides, I
installed FreeBSD entirely on my own, due to my long list of
specialised ZFS filesystems.
Post by Victor Sudakov
Post by Trond Endrestøl
I configured some boot entries in the UEFI environment of my laptop,
one for each of rEFInd, Windows 10, and FreeBSD, and made the entry
for rEFInd the preferred one.
Why do you need rEFInd at all if your laptop's UEFI environment can
choose which OS to boot?
Unless I want to boot the default, I have to hit F12 every time I want
to boot something else. This is on a Dell Latitude E5530. If I could
persuade the UEFI boot firmware to always present its menu, I wouldn't
need rEFInd at all. It's nice to have something that presents me with
a menu and remembers what I chose the last time and selects that after
a configureable timeout. It's also nice to have additional means
readily available should rEFInd implode.
--
Trond.
Victor Sudakov
2018-03-22 01:50:43 UTC
Permalink
Post by Trond Endrestøl
Post by Victor Sudakov
Post by Trond Endrestøl
I configured some boot entries in the UEFI environment of my laptop,
one for each of rEFInd, Windows 10, and FreeBSD, and made the entry
for rEFInd the preferred one.
Why do you need rEFInd at all if your laptop's UEFI environment can
choose which OS to boot?
Unless I want to boot the default, I have to hit F12 every time I want
to boot something else. This is on a Dell Latitude E5530. If I could
persuade the UEFI boot firmware to always present its menu, I wouldn't
need rEFInd at all. It's nice to have something that presents me with
a menu and remembers what I chose the last time and selects that after
a configureable timeout. It's also nice to have additional means
readily available should rEFInd implode.
Trond, thanks for the enlightenment, I think I grasp the idea. Two
more questions if you please:

1. If I have more than one freebsd-ufs partition, how can I configure
boot1.efi (AKA /EFI/BOOT/BOOTX64.EFI) to boot the loader from a
particular partition, not from the first one?

2. Why are /boot/boot1.efi and /EFI/BOOT/BOOTX64.EFI (extracted from
boot1.efifat) so different, even different length? They are supposed
to be one and the same file, aren't they?
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Victor Sudakov
2018-03-22 04:05:01 UTC
Permalink
Post by Victor Sudakov
2. Why are /boot/boot1.efi and /EFI/BOOT/BOOTX64.EFI (extracted from
boot1.efifat) so different, even different length? They are supposed
to be one and the same file, aren't they?
hexdump shows that boot1.efifat:/EFI/BOOT/BOOTX64.EFI is padded with spaces (ascii 0x20)
at the end, and otherwise they are identical.

Why, I wonder.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Trond Endrestøl
2018-03-23 10:33:29 UTC
Permalink
Post by Victor Sudakov
Post by Trond Endrestøl
Post by Victor Sudakov
Why do you need rEFInd at all if your laptop's UEFI environment can
choose which OS to boot?
Unless I want to boot the default, I have to hit F12 every time I want
to boot something else. This is on a Dell Latitude E5530. If I could
persuade the UEFI boot firmware to always present its menu, I wouldn't
need rEFInd at all. It's nice to have something that presents me with
a menu and remembers what I chose the last time and selects that after
a configureable timeout. It's also nice to have additional means
readily available should rEFInd implode.
Trond, thanks for the enlightenment, I think I grasp the idea. Two
1. If I have more than one freebsd-ufs partition, how can I configure
boot1.efi (AKA /EFI/BOOT/BOOTX64.EFI) to boot the loader from a
particular partition, not from the first one?
I doubt you can without modifying the source code. Such has never been
an issue at my end. Maybe someone else will chime in.
Post by Victor Sudakov
2. Why are /boot/boot1.efi and /EFI/BOOT/BOOTX64.EFI (extracted from
boot1.efifat) so different, even different length? They are supposed
to be one and the same file, aren't they?
It's done this way to simplify the creation of boot1.efifat which is
the image for the 800K FAT filesystem. Have a look at the Makefiles in
stand/efi/boot1 in the source tree.
--
Trond.
Trond Endrestøl
2018-03-23 11:13:18 UTC
Permalink
Post by Trond Endrestøl
Post by Victor Sudakov
1. If I have more than one freebsd-ufs partition, how can I configure
boot1.efi (AKA /EFI/BOOT/BOOTX64.EFI) to boot the loader from a
particular partition, not from the first one?
I doubt you can without modifying the source code. Such has never been
an issue at my end. Maybe someone else will chime in.
This is from uefi(8) on fairly recent stable/11:

2. boot1.efi reads boot configuration from /boot.config or
/boot/config. Unlike other first-stage boot loaders,
boot1.efi passes the configuration to the next stage boot
loader and does not itself act on the contents of the file.
3. boot1.efi searches partitions of type freebsd-ufs and
freebsd-zfs for loader.efi. The search begins with partitions
on the device from which boot1.efi was loaded, and continues
with other available partitions. If both freebsd-ufs and
freebsd-zfs partitions exist on the same device the
freebsd-zfs partition is preferred. boot1.efi then loads and
executes loader.efi.

Apparently, there are some options. Using /boot.config or /boot/config
on the ESP. Using /boot/loader.conf whereever /boot/loader.efi
resides. Relocating loader.efi to wanted the UFS partition. Note the
preference towards ZFS when both are present. I don't have any
experience using either option.
--
Trond.
Victor Sudakov
2018-03-25 04:42:52 UTC
Permalink
Post by Trond Endrestøl
Post by Trond Endrestøl
Post by Victor Sudakov
1. If I have more than one freebsd-ufs partition, how can I configure
boot1.efi (AKA /EFI/BOOT/BOOTX64.EFI) to boot the loader from a
particular partition, not from the first one?
I doubt you can without modifying the source code. Such has never been
an issue at my end. Maybe someone else will chime in.
2. boot1.efi reads boot configuration from /boot.config or
/boot/config. Unlike other first-stage boot loaders,
boot1.efi passes the configuration to the next stage boot
loader and does not itself act on the contents of the file.
3. boot1.efi searches partitions of type freebsd-ufs and
freebsd-zfs for loader.efi. The search begins with partitions
on the device from which boot1.efi was loaded, and continues
with other available partitions. If both freebsd-ufs and
freebsd-zfs partitions exist on the same device the
freebsd-zfs partition is preferred. boot1.efi then loads and
executes loader.efi.
Apparently, there are some options. Using /boot.config or /boot/config
on the ESP. Using /boot/loader.conf whereever /boot/loader.efi
resides. Relocating loader.efi to wanted the UFS partition. Note the
preference towards ZFS when both are present. I don't have any
experience using either option.
IMHO boot1.efi could at least honor the "bootme" flag if there are
several freebsd GPT partitions, the way gptboot does.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Victor Sudakov
2018-03-26 05:19:38 UTC
Permalink
Post by Trond Endrestøl
Post by Victor Sudakov
Post by Trond Endrestøl
I configured some boot entries in the UEFI environment of my laptop,
one for each of rEFInd, Windows 10, and FreeBSD, and made the entry
for rEFInd the preferred one.
Why do you need rEFInd at all if your laptop's UEFI environment can
choose which OS to boot?
Unless I want to boot the default, I have to hit F12 every time I want
to boot something else. This is on a Dell Latitude E5530. If I could
persuade the UEFI boot firmware to always present its menu, I wouldn't
need rEFInd at all. It's nice to have something that presents me with
a menu and remembers what I chose the last time and selects that after
a configureable timeout. It's also nice to have additional means
readily available should rEFInd implode.
Dear Trond,

Can you please show me the contents of your EFI partition with rEFInd?

mkdir /efi ; mount_msdos /dev/ada0p1 /efi ; ls -alR /efi

I've tried to install rEFInd manually, it sorta works but menu items
and nice graphics are missing, I guess rEFInd cannot find its config
or libs because I've misplaced its binaries.

My refind.conf http://termbin.com/4vo7n
"ls -alR /efi" http://termbin.com/oz92
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Trond Endrestøl
2018-03-26 08:03:09 UTC
Permalink
Post by Victor Sudakov
Can you please show me the contents of your EFI partition with rEFInd?
I already have an entry for the ESP in my fstab:

/dev/ada0p2 /esp msdosfs rw,-l,-m=664,-M=775,noauto 0 0

Here's the directory listing:

/esp:
total 58
drwxrwxr-x 1 root wheel 1024 Dec 31 1979 .
drwxr-xr-x 23 root wheel 41 Mar 26 09:27 ..
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 EFI
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 freebsd
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 refind-bin-0.11.2

/esp/EFI:
total 4
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 1024 Dec 31 1979 ..
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 Boot
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 Microsoft

/esp/EFI/Boot:
total 1226
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 1252768 Feb 10 06:31 bootx64.efi

/esp/EFI/Microsoft:
total 8
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 ..
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 Boot
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 Recovery

/esp/EFI/Microsoft/Boot:
total 4959
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 .
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 61440 Mar 26 09:26 BCD
-rw-rw-r-- 1 root wheel 114688 Aug 12 2015 BCD.LOG
-rw-rw-r-- 1 root wheel 0 Aug 12 2015 BCD.LOG1
-rw-rw-r-- 1 root wheel 0 Aug 12 2015 BCD.LOG2
-rw-rw-r-- 1 root wheel 65536 Nov 22 15:39 BOOTSTAT.DAT
drwxrwxr-x 1 root wheel 3072 Aug 12 2015 Fonts
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 Resources
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 bg-BG
-rw-rw-r-- 1 root wheel 4669 Sep 29 14:41 boot.stl
-rw-rw-r-- 1 root wheel 1252768 Feb 10 06:31 bootmgfw.efi
-rw-rw-r-- 1 root wheel 1237912 Feb 10 06:54 bootmgr.efi
-rw-rw-r-- 1 root wheel 115616 Mar 18 2017 bootspaces.dll
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 cs-CZ
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 da-DK
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 de-DE
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 el-GR
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 en-GB
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 en-US
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 es-ES
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 es-MX
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 et-EE
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 fi-FI
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 fr-CA
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 fr-FR
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 hr-HR
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 hu-HU
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 it-IT
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 ja-JP
-rw-rw-r-- 1 root wheel 31128 Sep 29 14:37 kd_02_10df.dll
-rw-rw-r-- 1 root wheel 329112 Sep 29 14:37 kd_02_10ec.dll
-rw-rw-r-- 1 root wheel 26520 Sep 29 14:37 kd_02_1137.dll
-rw-rw-r-- 1 root wheel 216472 Sep 29 14:37 kd_02_14e4.dll
-rw-rw-r-- 1 root wheel 44440 Sep 29 14:37 kd_02_15b3.dll
-rw-rw-r-- 1 root wheel 42392 Sep 29 14:37 kd_02_1969.dll
-rw-rw-r-- 1 root wheel 30240 Sep 29 14:37 kd_02_19a2.dll
-rw-rw-r-- 1 root wheel 20376 Sep 29 14:37 kd_02_1af4.dll
-rw-rw-r-- 1 root wheel 248728 Sep 29 14:37 kd_02_8086.dll
-rw-rw-r-- 1 root wheel 18840 Sep 29 14:37 kd_07_1415.dll
-rw-rw-r-- 1 root wheel 40344 Sep 29 14:37 kd_0C_8086.dll
-rw-rw-r-- 1 root wheel 23448 Sep 29 14:37 kdstub.dll
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 ko-KR
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 lt-LT
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 lv-LV
-rw-rw-r-- 1 root wheel 1083808 Mar 1 10:27 memtest.efi
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 nb-NO
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 nl-NL
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 pl-PL
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 pt-BR
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 pt-PT
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 qps-ploc
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 ro-RO
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 ru-RU
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 sk-SK
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 sl-SI
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 sr-Latn-CS
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 sr-Latn-RS
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 sv-SE
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 tr-TR
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 uk-UA
-rw-rw-r-- 1 root wheel 4662 Mar 18 2017 updaterevokesipolicy.p7b
-rw-rw-r-- 1 root wheel 7678 Sep 29 14:41 winsipolicy.p7b
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 zh-CN
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 zh-HK
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 zh-TW

/esp/EFI/Microsoft/Boot/Fonts:
total 13177
drwxrwxr-x 1 root wheel 3072 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 3695714 Sep 29 14:37 chs_boot.ttf
-rw-rw-r-- 1 root wheel 3878407 Sep 29 14:37 cht_boot.ttf
-rw-rw-r-- 1 root wheel 1985861 Sep 29 14:37 jpn_boot.ttf
-rw-rw-r-- 1 root wheel 2372994 Sep 29 14:37 kor_boot.ttf
-rw-rw-r-- 1 root wheel 177411 Sep 29 14:37 malgun_boot.ttf
-rw-rw-r-- 1 root wheel 174954 Sep 29 14:37 malgunn_boot.ttf
-rw-rw-r-- 1 root wheel 145414 Sep 29 14:37 meiryo_boot.ttf
-rw-rw-r-- 1 root wheel 143750 Sep 29 14:37 meiryon_boot.ttf
-rw-rw-r-- 1 root wheel 164341 Sep 29 14:37 msjh_boot.ttf
-rw-rw-r-- 1 root wheel 162323 Sep 29 14:37 msjhn_boot.ttf
-rw-rw-r-- 1 root wheel 155350 Sep 29 14:37 msyh_boot.ttf
-rw-rw-r-- 1 root wheel 154422 Sep 29 14:37 msyhn_boot.ttf
-rw-rw-r-- 1 root wheel 44854 Sep 29 14:37 segmono_boot.ttf
-rw-rw-r-- 1 root wheel 86174 Sep 29 14:37 segoe_slboot.ttf
-rw-rw-r-- 1 root wheel 85858 Sep 29 14:37 segoen_slboot.ttf
-rw-rw-r-- 1 root wheel 49086 Sep 29 14:37 wgl4_boot.ttf

/esp/EFI/Microsoft/Boot/Resources:
total 98
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 92568 Sep 29 14:37 bootres.dll
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 en-US

/esp/EFI/Microsoft/Boot/Resources/en-US:
total 14
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 12184 Sep 29 14:39 bootres.dll.mui

/esp/EFI/Microsoft/Boot/bg-BG:
total 158
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 77720 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 77720 Sep 29 14:41 bootmgr.efi.mui

/esp/EFI/Microsoft/Boot/cs-CZ:
total 203
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 77208 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 77208 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45464 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/da-DK:
total 201
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 76184 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 76184 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45464 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/de-DE:
total 207
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 79768 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 79768 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45976 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/el-GR:
total 209
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 79392 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 80280 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 46488 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/en-GB:
total 152
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 74648 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 74648 Sep 29 14:41 bootmgr.efi.mui

/esp/EFI/Microsoft/Boot/en-US:
total 196
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 73760 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 74648 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 44952 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/es-ES:
total 203
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 77720 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 77720 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45976 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/es-MX:
total 158
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 77720 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 77720 Sep 29 14:41 bootmgr.efi.mui

/esp/EFI/Microsoft/Boot/et-EE:
total 154
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 74784 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 75672 Sep 29 14:41 bootmgr.efi.mui

/esp/EFI/Microsoft/Boot/fi-FI:
total 202
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 76320 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 77208 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45464 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/fr-CA:
total 161
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 79256 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 78368 Sep 29 14:41 bootmgr.efi.mui

/esp/EFI/Microsoft/Boot/fr-FR:
total 207
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 79768 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 79768 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45976 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/hr-HR:
total 156
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 76696 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 76696 Sep 29 14:41 bootmgr.efi.mui

/esp/EFI/Microsoft/Boot/hu-HU:
total 207
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 79256 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 79256 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45976 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/it-IT:
total 203
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 77208 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 77208 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45464 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/ja-JP:
total 181
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 67104 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 67992 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 42904 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/ko-KR:
total 180
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 67480 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 67480 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 42904 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/lt-LT:
total 156
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 76184 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 76184 Sep 29 14:41 bootmgr.efi.mui

/esp/EFI/Microsoft/Boot/lv-LV:
total 156
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 76184 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 76184 Sep 29 14:41 bootmgr.efi.mui

/esp/EFI/Microsoft/Boot/nb-NO:
total 201
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 76184 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 76184 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45464 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/nl-NL:
total 205
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 78232 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 78232 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45464 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/pl-PL:
total 205
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 78232 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 78232 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45976 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/pt-BR:
total 203
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 77208 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 77208 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45464 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/pt-PT:
total 203
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 77208 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 77208 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45976 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/qps-ploc:
total 205
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 74080 Jul 10 2015 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 74080 Jul 10 2015 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 54168 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/ro-RO:
total 156
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 76696 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 76696 Sep 29 14:41 bootmgr.efi.mui

/esp/EFI/Microsoft/Boot/ru-RU:
total 202
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 77208 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 77208 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 44952 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/sk-SK:
total 158
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 77720 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 77720 Sep 29 14:41 bootmgr.efi.mui

/esp/EFI/Microsoft/Boot/sl-SI:
total 157
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 77208 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 76320 Sep 29 14:41 bootmgr.efi.mui

/esp/EFI/Microsoft/Boot/sr-Latn-CS:
total 203
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 77152 Jul 10 2015 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 77152 Jul 10 2015 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 45400 Apr 28 2017 memtest.efi.mui

/esp/EFI/Microsoft/Boot/sr-Latn-RS:
total 157
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 76320 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 77208 Sep 29 14:41 bootmgr.efi.mui

/esp/EFI/Microsoft/Boot/sv-SE:
total 200
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 76696 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 76696 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 44064 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/tr-TR:
total 198
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 75672 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 75672 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 44576 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/uk-UA:
total 158
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 77720 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 77720 Sep 29 14:41 bootmgr.efi.mui

/esp/EFI/Microsoft/Boot/zh-CN:
total 174
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 63896 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 63896 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 42392 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Boot/zh-HK:
total 174
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 63840 Aug 5 2016 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 63840 Aug 5 2016 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 42336 Aug 6 2016 memtest.efi.mui

/esp/EFI/Microsoft/Boot/zh-TW:
total 173
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 5120 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 64408 Sep 29 14:41 bootmgfw.efi.mui
-rw-rw-r-- 1 root wheel 64408 Sep 29 14:41 bootmgr.efi.mui
-rw-rw-r-- 1 root wheel 41504 Sep 29 14:41 memtest.efi.mui

/esp/EFI/Microsoft/Recovery:
total 62
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 ..
-rw-rw-r-- 1 root wheel 28672 Dec 4 22:07 BCD
-rw-rw-r-- 1 root wheel 32768 Aug 12 2015 BCD.LOG
-rw-rw-r-- 1 root wheel 0 Aug 12 2015 BCD.LOG1
-rw-rw-r-- 1 root wheel 0 Aug 12 2015 BCD.LOG2

/esp/freebsd:
total 329
drwxrwxr-x 1 root wheel 1024 Aug 12 2015 .
drwxrwxr-x 1 root wheel 1024 Dec 31 1979 ..
-rw-rw-r-- 1 root wheel 75776 Jan 13 2017 boot1.efi
-rw-rw-r-- 1 root wheel 33811 Aug 12 2015 boot1.efi.old0
-rw-rw-r-- 1 root wheel 72139 Jan 29 2016 boot1.efi.old1
-rw-rw-r-- 1 root wheel 74833 Mar 18 2016 boot1.efi.old2
-rw-rw-r-- 1 root wheel 75776 Dec 29 2016 boot1.efi.old3

/esp/refind-bin-0.11.2:
total 233
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 .
drwxrwxr-x 1 root wheel 1024 Dec 31 1979 ..
-rw-rw-r-- 1 root wheel 35147 Nov 29 2015 COPYING.txt
-rw-rw-r-- 1 root wheel 4891 Aug 8 2017 CREDITS.txt
-rw-rw-r-- 1 root wheel 2204 Nov 29 2015 LICENSE.txt
-rw-rw-r-- 1 root wheel 96583 Oct 22 21:03 NEWS.txt
-rw-rw-r-- 1 root wheel 4010 May 26 2017 README.txt
drwxrwxr-x 1 root wheel 1024 Nov 29 2015 banners
drwxrwxr-x 1 root wheel 1024 Dec 3 2015 docs
drwxrwxr-x 1 root wheel 2048 Jul 29 2017 fonts
drwxrwxr-x 1 root wheel 2048 Jan 20 2016 keys
-rw-rw-r-- 1 root wheel 2428 Nov 29 2015 mkrlconf
-rw-rw-r-- 1 root wheel 2564 Nov 29 2015 mountesp
-rw-rw-r-- 1 root wheel 10303 Aug 3 2017 mvrefind
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 refind
-rw-rw-r-- 1 root wheel 56263 Oct 22 21:36 refind-install
-rw-rw-r-- 1 root wheel 7173 Apr 24 2016 refind-mkdefault

/esp/refind-bin-0.11.2/banners:
total 42
drwxrwxr-x 1 root wheel 1024 Nov 29 2015 .
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 ..
-rw-rw-r-- 1 root wheel 11274 Nov 29 2015 refind_banner-alpha.png
-rw-rw-r-- 1 root wheel 11960 Nov 29 2015 refind_banner.png
-rw-rw-r-- 1 root wheel 14483 Nov 29 2015 refind_banner.svg

/esp/refind-bin-0.11.2/docs:
total 9
drwxrwxr-x 1 root wheel 1024 Dec 3 2015 .
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 ..
drwxrwxr-x 1 root wheel 1024 Nov 29 2015 Styles
drwxrwxr-x 1 root wheel 1024 Aug 8 2017 man
drwxrwxr-x 1 root wheel 4096 Oct 22 21:32 refind

/esp/refind-bin-0.11.2/docs/Styles:
total 4
drwxrwxr-x 1 root wheel 1024 Nov 29 2015 .
drwxrwxr-x 1 root wheel 1024 Dec 3 2015 ..
-rw-rw-r-- 1 root wheel 1322 Nov 29 2015 styles.css

/esp/refind-bin-0.11.2/docs/man:
total 27
drwxrwxr-x 1 root wheel 1024 Aug 8 2017 .
drwxrwxr-x 1 root wheel 1024 Dec 3 2015 ..
-rw-rw-r-- 1 root wheel 2028 Oct 22 21:36 mkrlconf.8
-rw-rw-r-- 1 root wheel 3693 Oct 22 21:36 mvrefind.8
-rw-rw-r-- 1 root wheel 14259 Oct 22 21:36 refind-install.8
-rw-rw-r-- 1 root wheel 4716 Oct 22 21:36 refind-mkdefault.8

/esp/refind-bin-0.11.2/docs/refind:
total 3446
drwxrwxr-x 1 root wheel 4096 Oct 22 21:32 .
drwxrwxr-x 1 root wheel 1024 Dec 3 2015 ..
-rw-rw-r-- 1 root wheel 22962 Nov 29 2015 FDL-1.3.txt
-rw-rw-r-- 1 root wheel 1667 Nov 29 2015 HashTool1.png
-rw-rw-r-- 1 root wheel 2629 Nov 29 2015 HashTool2.png
-rw-rw-r-- 1 root wheel 1975 Nov 29 2015 MokManager1.png
-rw-rw-r-- 1 root wheel 27758 Nov 29 2015 MokManager2.png
-rw-rw-r-- 1 root wheel 812483 Feb 20 2016 Screenshot (3).png
-rw-rw-r-- 1 root wheel 66474 Nov 29 2015 about.png
-rw-rw-r-- 1 root wheel 71471 Nov 29 2015 about.svg
-rw-rw-r-- 1 root wheel 241341 Jun 11 2017 ambience.png
-rw-rw-r-- 1 root wheel 30136 Feb 23 2016 asus-bootmanager.jpg
-rw-rw-r-- 1 root wheel 99730 Feb 23 2016 asus-bootorder.jpg
-rw-rw-r-- 1 root wheel 29095 Nov 29 2015 automatic-submenu.png
-rw-rw-r-- 1 root wheel 54542 Oct 22 21:13 bootcoup.html
-rw-rw-r-- 1 root wheel 18258 Oct 22 21:11 bootmode.html
-rw-rw-r-- 1 root wheel 77716 Oct 22 21:14 configfile.html
-rw-rw-r-- 1 root wheel 3023 Dec 3 2015 donate.png
-rw-rw-r-- 1 root wheel 4120 Dec 3 2015 donate.svg
-rw-rw-r-- 1 root wheel 35867 Oct 22 21:28 drivers.html
-rw-rw-r-- 1 root wheel 40805 Apr 24 2016 easyuefi.png
-rw-rw-r-- 1 root wheel 1667 Nov 29 2015 editor.png
-rw-rw-r-- 1 root wheel 17231 Oct 22 21:11 features.html
-rw-rw-r-- 1 root wheel 1730 Nov 29 2015 func_csr_rotate.png
-rw-rw-r-- 1 root wheel 18174 Oct 22 21:12 getting.html
-rw-rw-r-- 1 root wheel 18441 Oct 22 21:10 index.html
-rw-rw-r-- 1 root wheel 85723 Oct 22 21:12 installing.html
-rw-rw-r-- 1 root wheel 39402 Oct 22 21:28 linux.html
-rw-rw-r-- 1 root wheel 36645 Jun 11 2017 mac-theme.png
-rw-rw-r-- 1 root wheel 16140 Nov 29 2015 manual-submenu.png
-rw-rw-r-- 1 root wheel 2980 Oct 22 21:40 mkrlconf.html
-rw-rw-r-- 1 root wheel 29983 Oct 21 2016 msinfo32.png
-rw-rw-r-- 1 root wheel 4735 Oct 22 21:40 mvrefind.html
-rw-rw-r-- 1 root wheel 4114 Nov 29 2015 os_legacy.png
-rw-rw-r-- 1 root wheel 20772 Apr 16 2017 rEFInd-Metro.png
-rw-rw-r-- 1 root wheel 369782 Nov 29 2015 refind-background-snowy.png
-rw-rw-r-- 1 root wheel 690377 Nov 29 2015 refind-background.png
-rw-rw-r-- 1 root wheel 15506 Oct 22 21:40 refind-install.html
-rw-rw-r-- 1 root wheel 6070 Oct 22 21:40 refind-mkdefault.html
-rw-rw-r-- 1 root wheel 85526 Nov 29 2015 refind.png
-rw-rw-r-- 1 root wheel 93566 Oct 22 21:32 revisions.html
-rw-rw-r-- 1 root wheel 41287 Oct 22 21:29 secureboot.html
-rw-rw-r-- 1 root wheel 27874 Oct 22 21:13 sip.html
-rw-rw-r-- 1 root wheel 53649 Feb 21 2016 startup-disk.png
-rw-rw-r-- 1 root wheel 23991 Nov 29 2015 submenu.png
-rw-rw-r-- 1 root wheel 26237 Oct 22 21:28 themes.html
-rw-rw-r-- 1 root wheel 23280 Oct 22 21:28 todo.html
-rw-rw-r-- 1 root wheel 20556 Apr 16 2017 tux-refind-theme.png
-rw-rw-r-- 1 root wheel 8 Aug 1 2017 ucs2.txt
-rw-rw-r-- 1 root wheel 30209 Oct 22 21:13 using.html
-rw-rw-r-- 1 root wheel 10 Aug 1 2017 utf16.txt
-rw-rw-r-- 1 root wheel 36284 Nov 29 2015 windows-gpt.png
-rw-rw-r-- 1 root wheel 265 Nov 29 2015 windows8-on-mac-efi.txt
-rw-rw-r-- 1 root wheel 11832 May 21 2017 yosemite.html

/esp/refind-bin-0.11.2/fonts:
total 117
drwxrwxr-x 1 root wheel 2048 Jul 29 2017 .
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 ..
-rw-rw-r-- 1 root wheel 1901 Jul 29 2017 README.txt
-rw-rw-r-- 1 root wheel 6048 Nov 29 2015 liberation-mono-regular-12.png
-rw-rw-r-- 1 root wheel 7432 Nov 29 2015 liberation-mono-regular-14.png
-rw-rw-r-- 1 root wheel 13994 Nov 29 2015 liberation-mono-regular-24.png
-rw-rw-r-- 1 root wheel 13669 Jul 29 2017 liberation-mono-regular-28.png
-rw-rw-r-- 1 root wheel 1880 Nov 29 2015 mkfont.sh
-rw-rw-r-- 1 root wheel 3850 Nov 29 2015 nimbus-mono-12.png
-rw-rw-r-- 1 root wheel 4260 Nov 29 2015 nimbus-mono-14.png
-rw-rw-r-- 1 root wheel 5064 Nov 29 2015 nimbus-mono-16.png
-rw-rw-r-- 1 root wheel 8302 Nov 29 2015 nimbus-mono-24.png
-rw-rw-r-- 1 root wheel 11759 Jul 29 2017 nimbus-mono-28.png
-rw-rw-r-- 1 root wheel 2984 Nov 29 2015 ubuntu-mono-12.png
-rw-rw-r-- 1 root wheel 3931 Nov 29 2015 ubuntu-mono-14.png
-rw-rw-r-- 1 root wheel 4567 Nov 29 2015 ubuntu-mono-16.png
-rw-rw-r-- 1 root wheel 8137 Nov 29 2015 ubuntu-mono-24.png
-rw-rw-r-- 1 root wheel 11826 Jul 29 2017 ubuntu-mono-28.png

/esp/refind-bin-0.11.2/keys:
total 46
drwxrwxr-x 1 root wheel 2048 Jan 20 2016 .
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 ..
-rw-rw-r-- 1 root wheel 3724 Dec 14 2015 README.txt
-rw-rw-r-- 1 root wheel 1257 Dec 3 2015 SLES-UEFI-CA-Certificate.cer
-rw-rw-r-- 1 root wheel 1757 Dec 3 2015 SLES-UEFI-CA-Certificate.crt
-rw-rw-r-- 1 root wheel 767 Nov 29 2015 altlinux.cer
-rw-rw-r-- 1 root wheel 1517 Nov 29 2015 canonical-uefi-ca.crt
-rw-rw-r-- 1 root wheel 1080 Nov 29 2015 canonical-uefi-ca.der
-rw-rw-r-- 1 root wheel 1500 Dec 13 2015 centos.cer
-rw-rw-r-- 1 root wheel 2086 Dec 13 2015 centos.crt
-rw-rw-r-- 1 root wheel 876 Nov 29 2015 fedora-ca.cer
-rw-rw-r-- 1 root wheel 1241 Nov 29 2015 fedora-ca.crt
-rw-rw-r-- 1 root wheel 1516 Nov 29 2015 microsoft-kekca-public.der
-rw-rw-r-- 1 root wheel 1499 Nov 29 2015 microsoft-pca-public.der
-rw-rw-r-- 1 root wheel 2163 Nov 29 2015 microsoft-uefica-public.crt
-rw-rw-r-- 1 root wheel 1556 Nov 29 2015 microsoft-uefica-public.der
-rw-rw-r-- 1 root wheel 1656 Dec 3 2015 openSUSE-UEFI-CA-Certificate-4096.cer
-rw-rw-r-- 1 root wheel 2297 Dec 3 2015 openSUSE-UEFI-CA-Certificate-4096.crt
-rw-rw-r-- 1 root wheel 1144 Dec 3 2015 openSUSE-UEFI-CA-Certificate.cer
-rw-rw-r-- 1 root wheel 1606 Dec 3 2015 openSUSE-UEFI-CA-Certificate.crt
-rw-rw-r-- 1 root wheel 831 Nov 29 2015 refind.cer
-rw-rw-r-- 1 root wheel 1180 Nov 29 2015 refind.crt

/esp/refind-bin-0.11.2/refind:
total 710
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 .
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 ..
drwxrwxr-x 1 root wheel 1024 Oct 22 21:41 drivers_aa64
drwxrwxr-x 1 root wheel 1024 Oct 22 21:41 drivers_ia32
drwxrwxr-x 1 root wheel 1024 Oct 22 21:42 drivers_x64
drwxrwxr-x 1 root wheel 6144 Oct 22 21:40 icons
-rw-rw-r-- 1 root wheel 29980 Dec 26 20:59 refind.conf
-rw-rw-r-- 1 root wheel 29637 Oct 15 14:36 refind.conf-sample
-rw-rw-r-- 1 root wheel 1228 Dec 26 21:00 refind.conf.diff
-rw-rw-r-- 1 root wheel 216288 Oct 22 21:41 refind_aa64.efi
-rw-rw-r-- 1 root wheel 208704 Oct 22 21:41 refind_ia32.efi
-rw-rw-r-- 1 root wheel 222024 Oct 22 21:42 refind_x64.efi
drwxrwxr-x 1 root wheel 1024 Oct 22 21:41 tools_aa64
drwxrwxr-x 1 root wheel 1024 Oct 22 21:41 tools_ia32
drwxrwxr-x 1 root wheel 1024 Oct 22 21:42 tools_x64

/esp/refind-bin-0.11.2/refind/drivers_aa64:
total 276
drwxrwxr-x 1 root wheel 1024 Oct 22 21:41 .
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 ..
-rw-rw-r-- 1 root wheel 2094 Nov 29 2015 LICENSE.txt
-rw-rw-r-- 1 root wheel 17992 Nov 29 2015 LICENSE_GPL.txt
-rw-rw-r-- 1 root wheel 46592 Oct 22 21:41 btrfs_aa64.efi
-rw-rw-r-- 1 root wheel 32480 Oct 22 21:41 ext2_aa64.efi
-rw-rw-r-- 1 root wheel 33024 Oct 22 21:41 ext4_aa64.efi
-rw-rw-r-- 1 root wheel 35232 Oct 22 21:41 hfs_aa64.efi
-rw-rw-r-- 1 root wheel 31776 Oct 22 21:41 iso9660_aa64.efi
-rw-rw-r-- 1 root wheel 39488 Oct 22 21:41 ntfs_aa64.efi
-rw-rw-r-- 1 root wheel 35072 Oct 22 21:41 reiserfs_aa64.efi

/esp/refind-bin-0.11.2/refind/drivers_ia32:
total 260
drwxrwxr-x 1 root wheel 1024 Oct 22 21:41 .
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 ..
-rw-rw-r-- 1 root wheel 2094 Nov 29 2015 LICENSE.txt
-rw-rw-r-- 1 root wheel 17992 Nov 29 2015 LICENSE_GPL.txt
-rw-rw-r-- 1 root wheel 45312 Oct 22 21:41 btrfs_ia32.efi
-rw-rw-r-- 1 root wheel 29952 Oct 22 21:41 ext2_ia32.efi
-rw-rw-r-- 1 root wheel 30464 Oct 22 21:41 ext4_ia32.efi
-rw-rw-r-- 1 root wheel 33088 Oct 22 21:41 hfs_ia32.efi
-rw-rw-r-- 1 root wheel 29440 Oct 22 21:41 iso9660_ia32.efi
-rw-rw-r-- 1 root wheel 37696 Oct 22 21:41 ntfs_ia32.efi
-rw-rw-r-- 1 root wheel 32704 Oct 22 21:41 reiserfs_ia32.efi

/esp/refind-bin-0.11.2/refind/drivers_x64:
total 289
drwxrwxr-x 1 root wheel 1024 Oct 22 21:42 .
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 ..
-rw-rw-r-- 1 root wheel 2094 Nov 29 2015 LICENSE.txt
-rw-rw-r-- 1 root wheel 17992 Nov 29 2015 LICENSE_GPL.txt
-rw-rw-r-- 1 root wheel 49160 Oct 22 21:42 btrfs_x64.efi
-rw-rw-r-- 1 root wheel 33928 Oct 22 21:42 ext2_x64.efi
-rw-rw-r-- 1 root wheel 34440 Oct 22 21:42 ext4_x64.efi
-rw-rw-r-- 1 root wheel 37064 Oct 22 21:42 hfs_x64.efi
-rw-rw-r-- 1 root wheel 33544 Oct 22 21:42 iso9660_x64.efi
-rw-rw-r-- 1 root wheel 41608 Oct 22 21:42 ntfs_x64.efi
-rw-rw-r-- 1 root wheel 36872 Oct 22 21:42 reiserfs_x64.efi

/esp/refind-bin-0.11.2/refind/icons:
total 509
drwxrwxr-x 1 root wheel 6144 Oct 22 21:40 .
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 ..
-rw-rw-r-- 1 root wheel 5569 Aug 12 2017 README
-rw-rw-r-- 1 root wheel 1392 Nov 29 2015 arrow_left.png
-rw-rw-r-- 1 root wheel 1346 Nov 29 2015 arrow_right.png
-rw-rw-r-- 1 root wheel 9077 Nov 29 2015 boot_linux.png
-rw-rw-r-- 1 root wheel 3766 Nov 29 2015 boot_win.png
-rw-rw-r-- 1 root wheel 2479 Nov 29 2015 func_about.png
-rw-rw-r-- 1 root wheel 1761 Nov 29 2015 func_csr_rotate.png
-rw-rw-r-- 1 root wheel 2208 Nov 29 2015 func_exit.png
-rw-rw-r-- 1 root wheel 3772 Nov 29 2015 func_firmware.png
-rw-rw-r-- 1 root wheel 7161 Aug 3 2017 func_hidden.png
-rw-rw-r-- 1 root wheel 2602 Nov 29 2015 func_reset.png
-rw-rw-r-- 1 root wheel 2760 Nov 29 2015 func_shutdown.png
drwxrwxr-x 1 root wheel 1024 Oct 22 21:40 licenses
-rw-rw-r-- 1 root wheel 4846 Aug 12 2017 mouse.png
-rw-rw-r-- 1 root wheel 5555 Nov 29 2015 os_arch.png
-rw-rw-r-- 1 root wheel 15008 Nov 29 2015 os_centos.png
-rw-rw-r-- 1 root wheel 13776 Nov 29 2015 os_chakra.png
-rw-rw-r-- 1 root wheel 10391 Nov 29 2015 os_chrome.png
-rw-rw-r-- 1 root wheel 7229 Nov 29 2015 os_clover.png
-rw-rw-r-- 1 root wheel 8480 Nov 29 2015 os_crunchbang.png
-rw-rw-r-- 1 root wheel 9377 Nov 29 2015 os_debian.png
-rw-rw-r-- 1 root wheel 5072 May 8 2017 os_devuan.png
-rw-rw-r-- 1 root wheel 16482 Nov 29 2015 os_elementary.png
-rw-rw-r-- 1 root wheel 8763 Nov 29 2015 os_fedora.png
-rw-rw-r-- 1 root wheel 5980 Nov 29 2015 os_freebsd.png
-rw-rw-r-- 1 root wheel 11658 Nov 29 2015 os_frugalware.png
-rw-rw-r-- 1 root wheel 7549 Nov 29 2015 os_gentoo.png
-rw-rw-r-- 1 root wheel 7668 Nov 29 2015 os_gummiboot.png
-rw-rw-r-- 1 root wheel 5830 Nov 29 2015 os_haiku.png
-rw-rw-r-- 1 root wheel 14399 Nov 29 2015 os_hwtest.png
-rw-rw-r-- 1 root wheel 12873 Nov 29 2015 os_kubuntu.png
-rw-rw-r-- 1 root wheel 5620 Nov 29 2015 os_legacy.png
-rw-rw-r-- 1 root wheel 10097 Nov 29 2015 os_linux.png
-rw-rw-r-- 1 root wheel 6246 Nov 29 2015 os_linuxmint.png
-rw-rw-r-- 1 root wheel 11232 Nov 29 2015 os_lubuntu.png
-rw-rw-r-- 1 root wheel 10574 Nov 29 2015 os_mac.png
-rw-rw-r-- 1 root wheel 7783 Nov 29 2015 os_mageia.png
-rw-rw-r-- 1 root wheel 12854 Nov 29 2015 os_mandriva.png
-rw-rw-r-- 1 root wheel 12379 Nov 29 2015 os_netbsd.png
-rw-rw-r-- 1 root wheel 14836 Nov 29 2015 os_network.png
-rw-rw-r-- 1 root wheel 6572 Nov 29 2015 os_opensuse.png
-rw-rw-r-- 1 root wheel 10041 Nov 29 2015 os_redhat.png
-rw-rw-r-- 1 root wheel 7231 Nov 29 2015 os_refind.png
-rw-rw-r-- 1 root wheel 8484 Nov 29 2015 os_refit.png
-rw-rw-r-- 1 root wheel 9202 Nov 29 2015 os_slackware.png
-rw-rw-r-- 1 root wheel 6572 Nov 29 2015 os_suse.png
-rw-rw-r-- 1 root wheel 14608 Jun 13 2017 os_trusty.png
-rw-rw-r-- 1 root wheel 12270 Nov 29 2015 os_ubuntu.png
-rw-rw-r-- 1 root wheel 11686 Nov 29 2015 os_unknown.png
-rw-rw-r-- 1 root wheel 6565 Nov 29 2015 os_win.png
-rw-rw-r-- 1 root wheel 3872 Nov 29 2015 os_win8.png
-rw-rw-r-- 1 root wheel 9025 Jun 13 2017 os_xenial.png
-rw-rw-r-- 1 root wheel 10547 Nov 29 2015 os_xubuntu.png
-rw-rw-r-- 1 root wheel 5371 Jun 13 2017 os_zesty.png
-rw-rw-r-- 1 root wheel 4507 Nov 29 2015 tool_apple_rescue.png
-rw-rw-r-- 1 root wheel 5177 Apr 24 2016 tool_fwupdate.png
-rw-rw-r-- 1 root wheel 3018 Nov 29 2015 tool_memtest.png
-rw-rw-r-- 1 root wheel 1974 Nov 29 2015 tool_mok_tool.png
-rw-rw-r-- 1 root wheel 3785 Nov 29 2015 tool_netboot.png
-rw-rw-r-- 1 root wheel 3446 Nov 29 2015 tool_part.png
-rw-rw-r-- 1 root wheel 13817 Nov 29 2015 tool_rescue.png
-rw-rw-r-- 1 root wheel 2010 Nov 29 2015 tool_shell.png
-rw-rw-r-- 1 root wheel 4313 Nov 29 2015 tool_windows_rescue.png
-rw-rw-r-- 1 root wheel 213 Nov 29 2015 transparent.png
-rw-rw-r-- 1 root wheel 1344 Nov 29 2015 vol_external.png
-rw-rw-r-- 1 root wheel 1418 Nov 29 2015 vol_internal.png
-rw-rw-r-- 1 root wheel 1892 Nov 29 2015 vol_net.png
-rw-rw-r-- 1 root wheel 2423 Nov 29 2015 vol_optical.png

/esp/refind-bin-0.11.2/refind/icons/licenses:
total 75
drwxrwxr-x 1 root wheel 1024 Oct 22 21:40 .
drwxrwxr-x 1 root wheel 6144 Oct 22 21:40 ..
-rw-rw-r-- 1 root wheel 22240 Dec 1 2015 cc-3.0.txt
-rw-rw-r-- 1 root wheel 20133 May 8 2017 cc-by-sa-4.0.txt
-rw-rw-r-- 1 root wheel 18092 Nov 29 2015 gpl-2.0.txt
-rw-rw-r-- 1 root wheel 7651 Nov 29 2015 lgpl-3.0.txt

/esp/refind-bin-0.11.2/refind/tools_aa64:
total 3
drwxrwxr-x 1 root wheel 1024 Oct 22 21:41 .
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 ..

/esp/refind-bin-0.11.2/refind/tools_ia32:
total 39
drwxrwxr-x 1 root wheel 1024 Oct 22 21:41 .
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 ..
-rw-rw-r-- 1 root wheel 36736 Oct 22 21:41 gptsync_ia32.efi

/esp/refind-bin-0.11.2/refind/tools_x64:
total 44
drwxrwxr-x 1 root wheel 1024 Oct 22 21:42 .
drwxrwxr-x 1 root wheel 2048 Oct 22 21:42 ..
-rw-rw-r-- 1 root wheel 41608 Oct 22 21:42 gptsync_x64.efi
Post by Victor Sudakov
I've tried to install rEFInd manually, it sorta works but menu items
and nice graphics are missing, I guess rEFInd cannot find its config
or libs because I've misplaced its binaries.
Here's a condensed version of my
/esp/refind-bin-0.11.2/refind/refind.conf:

timeout 120

scanfor manual,netboot,optical

menuentry "Windows 10 Enterprise 1709 x64 English MS Imagine" {
icon \refind-bin-0.11.2\refind\icons\os_win.png
loader \EFI\Boot\bootx64.efi
}

menuentry "FreeBSD/amd64 stable/11 UEFI ZFS" {
icon \refind-bin-0.11.2\refind\icons\os_freebsd.png
loader \freebsd\boot1.efi
}

The UEFI environment has been told to boot
FS0:\refind-bin-0.11.2\refind\refind_x64.efi.
--
Trond.
Victor Sudakov
2018-03-27 04:33:38 UTC
Permalink
Post by Trond Endrestøl
Post by Victor Sudakov
Can you please show me the contents of your EFI partition with rEFInd?
/dev/ada0p2 /esp msdosfs rw,-l,-m=664,-M=775,noauto 0 0
Thank you. The crazy thing is my rEFInd cannot find its config. I've
placed copies of the config virtually everywhere:


$ find /esp/ -iname refind.conf*

/esp/EFI/BOOT/refind.conf
/esp/EFI/refind.conf
/esp/refind-bin-0.11.2/refind/refind.conf-sample
/esp/refind-bin-0.11.2/refind/refind.conf
/esp/refind-bin-0.11.2/refind.conf
/esp/refind.conf

and still it does not see it (even the timeout value remains 20s
instead of 500s I've put into the config).

My refind.conf (in many identical copies):

timeout 800
scanfor manual,netboot,optical
menuentry FreeBSD {
loader \freebsd\boot1.efi
icon \refind-bin-0.11.2\refind\icons\os_freebsd.png
}
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Victor Sudakov
2018-04-05 02:02:24 UTC
Permalink
I think both the articles
https://wiki.freebsd.org/RootOnZFS/ZFSBootPartition and
https://wiki.freebsd.org/RootOnZFS/ZFSBootSlice are incomplete and excessive.

I suggest merging and rewriting them. Below is my attepmt of a rewrite:
https://bitbucket.org/victor_sudakov/faq/src/tip/FreeBSD/Installing_FreeBSD_Root_on_ZFS_using_FreeBSD-ZFS_partition_in_a_FreeBSD_MBR_Slice.txt

If anybody cares please have a look at it.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Warren Block
2018-03-21 12:27:35 UTC
Permalink
Post by Victor Sudakov
Post by Victor Sudakov
My pool works successfully in a bhyve environment, but in VirtualBox 5.0.14
zfsboot: No ZFS pools located, can't boot
However, if I boot from LiveCD, the pool is there, it's healthy and
can be imported.
Finally, and thanks to Eugene Grossbein, the problem is solved.
The gist of the problem was the presence of a BSD label in the slice
where the bootable zfs pool was located. It turns out that zfsboot
ignores the slice if it finds a BSD label therein. The bootable zpool
should be directly on the slice (like on da1s1, and not da1s1a).
This means that both zfsboot(8) and the articles in the FreeBSD wiki
are erroneous because they all suggest "gpart create -s BSD ada0s1" -
in fact, it's fatal.
================================================
#!/bin/sh
sysctl kern.geom.debugflags=0x10
Please reconsider doing this without it being required. It means "allow
destroying mounted partitions" and is usually not necessary.
Victor Sudakov
2018-03-22 01:26:21 UTC
Permalink
Post by Warren Block
Post by Victor Sudakov
sysctl kern.geom.debugflags=0x10
Please reconsider doing this without it being required. It means "allow
destroying mounted partitions" and is usually not necessary.
It is recommended, or even required ("must") by the zfsboot(8) man page:

EXAMPLES
zfsboot is typically installed using dd(1). To install zfsboot on the
ada0 drive:

dd if=/boot/zfsboot of=/dev/ada0 count=1
dd if=/boot/zfsboot of=/dev/ada0 iseek=1 oseek=1024

If the drive is currently in use, the GEOM safety will prevent writes and
must be disabled before running the above commands:

sysctl kern.geom.debugflags=0x10

zfsboot can also be installed in an MBR slice:

gpart create -s mbr ada0
gpart add -t freebsd ada0
gpart create -s BSD ada0s1
gpart bootcode -b /boot/boot0 ada0
gpart set -a active -i 1 ada0
dd if=/boot/zfsboot of=/dev/ada0s1 count=1
dd if=/boot/zfsboot of=/dev/ada0s1 iseek=1 oseek=1024
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Warren Block
2018-03-22 13:28:14 UTC
Permalink
Post by Victor Sudakov
Post by Warren Block
Post by Victor Sudakov
sysctl kern.geom.debugflags=0x10
Please reconsider doing this without it being required. It means "allow
destroying mounted partitions" and is usually not necessary.
The vagueness here is from the "If the drive is currently in use". It
could easily be replaced with "If the user wants to destroy data from a
drive that is currently being used". I would argue that we should
remove or rewrite that, as it is misleading. Overwriting a drive that
is "in use" is generally exactly the opposite of what people want.
Post by Victor Sudakov
EXAMPLES
zfsboot is typically installed using dd(1). To install zfsboot on the
dd if=/boot/zfsboot of=/dev/ada0 count=1
dd if=/boot/zfsboot of=/dev/ada0 iseek=1 oseek=1024
If the drive is currently in use, the GEOM safety will prevent writes and
sysctl kern.geom.debugflags=0x10
gpart create -s mbr ada0
gpart add -t freebsd ada0
gpart create -s BSD ada0s1
gpart bootcode -b /boot/boot0 ada0
gpart set -a active -i 1 ada0
dd if=/boot/zfsboot of=/dev/ada0s1 count=1
dd if=/boot/zfsboot of=/dev/ada0s1 iseek=1 oseek=1024
Victor Sudakov
2018-03-23 02:32:02 UTC
Permalink
Post by Warren Block
Post by Warren Block
Post by Victor Sudakov
sysctl kern.geom.debugflags=0x10
Please reconsider doing this without it being required. It means "allow
destroying mounted partitions" and is usually not necessary.
The vagueness here is from the "If the drive is currently in use". It
could easily be replaced with "If the user wants to destroy data from a
drive that is currently being used".
In my script, the ${DISK} is supposed to be a new disk onto which a
system is installed (cloned actually), so any data thereon would be
destroyed anyway.
Post by Warren Block
I would argue that we should
remove or rewrite that, as it is misleading. Overwriting a drive that
is "in use" is generally exactly the opposite of what people want.
I'm afraid your concern is a bit misdirected. If you can suggest a way
to install zfsboot without dd, and document it in zfsboot(8) then of course
kern.geom.debugflags=0x10 is better avoided.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Victor Sudakov
2018-03-23 05:06:42 UTC
Permalink
Post by Victor Sudakov
In my script, the ${DISK} is supposed to be a new disk onto which a
system is installed (cloned actually), so any data thereon would be
destroyed anyway.
This reminds me. As the new system is a clone, there must be two
important lines in the script:

rm ${NEWSYSTEM}/etc/hostid
rm ${NEWSYSTEM}/etc/ssh/*key*
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Warren Block
2018-03-23 17:42:37 UTC
Permalink
Post by Victor Sudakov
Post by Warren Block
Post by Warren Block
Post by Victor Sudakov
sysctl kern.geom.debugflags=0x10
Please reconsider doing this without it being required. It means "allow
destroying mounted partitions" and is usually not necessary.
The vagueness here is from the "If the drive is currently in use". It
could easily be replaced with "If the user wants to destroy data from a
drive that is currently being used".
In my script, the ${DISK} is supposed to be a new disk onto which a
system is installed (cloned actually), so any data thereon would be
destroyed anyway.
Post by Warren Block
I would argue that we should
remove or rewrite that, as it is misleading. Overwriting a drive that
is "in use" is generally exactly the opposite of what people want.
I'm afraid your concern is a bit misdirected. If you can suggest a way
to install zfsboot without dd, and document it in zfsboot(8) then of course
kern.geom.debugflags=0x10 is better avoided.
That's the thing, dd should work without the debugflags setting. If it
does not, it means you are trying to overwrite something in use.
Victor Sudakov
2018-03-25 04:33:51 UTC
Permalink
Post by Warren Block
Post by Victor Sudakov
Post by Warren Block
Post by Warren Block
Post by Victor Sudakov
sysctl kern.geom.debugflags=0x10
Please reconsider doing this without it being required. It means "allow
destroying mounted partitions" and is usually not necessary.
The vagueness here is from the "If the drive is currently in use". It
could easily be replaced with "If the user wants to destroy data from a
drive that is currently being used".
In my script, the ${DISK} is supposed to be a new disk onto which a
system is installed (cloned actually), so any data thereon would be
destroyed anyway.
Post by Warren Block
I would argue that we should
remove or rewrite that, as it is misleading. Overwriting a drive that
is "in use" is generally exactly the opposite of what people want.
I'm afraid your concern is a bit misdirected. If you can suggest a way
to install zfsboot without dd, and document it in zfsboot(8) then of course
kern.geom.debugflags=0x10 is better avoided.
That's the thing, dd should work without the debugflags setting. If it
does not, it means you are trying to overwrite something in use.
To my surprise, my script works just fine without sysctl kern.geom.debugflags=0x10.
This may mean that this requirement in man zfsboot is superfluous.
Thanks for pointing this out, I've updated my bug report
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=226714

Still I'm surprised that geom permits my overwriting of a VBR with dd.
--
Victor Sudakov, VAS4-RIPE, VAS47-RIPN
AS43859
Dean E. Weimer
2018-01-15 14:58:46 UTC
Permalink
Post by Victor Sudakov
Post by Manish Jain
Post by Victor Sudakov
Sorry if my question was misleading, I did not mean converting the
existing system from UFS to ZFS at ada0s3. Suppose ada0s3 is an unused
slice, I can probably create a zpool thereon. But a few questions
1. Can I install a root-on-zfs system on ada0s3 with bsdinstall? I
suppose not?
That's right. You cannot install ZFS onto an MBR slice. ZFS by default
needs an entire disk, partitioned GPT.
However, later you suggest it's possible to install freebsd-zfs on a
BSD slice. How is that?
Post by Manish Jain
Post by Victor Sudakov
Why are you creating a zpool within a BSD "a" partition, and not in
da0s1 directly? I have never seen such a configuration. Is it even
possible to set the "freebsd-zfs" type on a BSD partition?
I presume you are using MBR partitioning. Under MBR, freebsd slices are
always nested. The slice contains partitions denoted with a, b and so on.
I do not think it is viable to set freebsd-zfs on an MBR slice. It just
will not work. ZFS needs an entire disk. If you wish to install on an
MBR slice, you must use rsync after installing freebsd-UFS.
Do you mean to say it's viable to install freebsd-zfs on ada0s3a but
not viable on ada0s3 ? I don't see logic here.
It can be installed on a partition, but it has to be GPT, not MBR, from
one of my servers I have this GPT partition setup.

# gpart show -p
=> 40 1953525088 ada0 GPT (932G)
40 1024 ada0p1 freebsd-boot (512K)
1064 33554432 ada0p2 freebsd-swap (16G)
33555496 1919969624 ada0p3 freebsd-zfs (916G)
1953525120 8 - free - (4.0K)

ZFS is designed to use a full raw disk, but you need to have a boot
partition, and in this case I setup swap as well outside of ZFS. This
server never hits swap though, so I don't have any real world metrics on
whether or not this is better that running swap on a zvol inside of ZFS.
There is a second drive duplicating this and GEOM mirror used for the
swap, along with ZFS mirror for redundancy and a second boot partition.
--
Thanks,
Dean E. Weimer
http://www.dweimer.net/
Loading...