Discussion:
ZFS boot: 3 smaller glitches with console, /etc/dfs/sharetab and /dev/random
(too old to reply)
Constantin Gonzalez Schmitz
2007-04-19 13:22:54 UTC
Permalink
Hi,

I've now gone through both the opensolaris instructions:

http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/

and Tim Foster's script:

http://blogs.sun.com/timf/entry/zfs_bootable_datasets_happily_rumbling

for making my laptop ZFS bootable.


Both work well and here's a big THANK YOU to the ZFS boot team!


There seem to be 3 smaller glitches with these approaches:

1. The instructions on opensolaris.org assume that one wants console output
to show up in /dev/tty. This may be true for a server, but it isn't for a
laptop or workstation user. Therefore, I suggest someone explains them to be
optional as not everybody knows that these can be left out.

2. After going through the zfs-bootification, Solaris complains on reboot that
/etc/dfs/sharetab is missing. Somehow this seems to have been fallen through
the cracks of the find command. Well, touching /etc/dfs/sharetab just fixes
the issue.

3. But here's a more serious one: While booting, Solaris complains:

Apr 19 15:00:37 foeni kcf: [ID 415456 kern.warning] WARNING: No randomness
provider enabled for /dev/random. Use cryptoadm(1M) to enable a provider.

Somehow, /dev/random and/or it's counterpart in /devices seems to have
suffered from the migration procedure.

Does anybody know how to fix the /dev/random issue? I'm not very fluent in
cryptoadm(1M) and some superficial reading of it's manpage did not enlighten
me too much (cryptoadm list -p claims all is well...).

Best regards and again, congratulations to the ZFS boot team!

Constantin
--
Constantin Gonzalez Sun Microsystems GmbH, Germany
Platform Technology Group, Global Systems Engineering http://www.sun.de/
Tel.: +49 89/4 60 08-25 91 http://blogs.sun.com/constantin/

Sitz d. Ges.: Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Marcel Schneider, Wolfgang Engels, Dr. Roland Boemer
Vorsitzender des Aufsichtsrates: Martin Haering
Darren J Moffat
2007-04-19 13:29:40 UTC
Permalink
Post by Constantin Gonzalez Schmitz
2. After going through the zfs-bootification, Solaris complains on reboot that
/etc/dfs/sharetab is missing. Somehow this seems to have been fallen through
the cracks of the find command. Well, touching /etc/dfs/sharetab just fixes
the issue.
sharetab is now an in kernel filesystem.
Post by Constantin Gonzalez Schmitz
Apr 19 15:00:37 foeni kcf: [ID 415456 kern.warning] WARNING: No randomness
provider enabled for /dev/random. Use cryptoadm(1M) to enable a provider.
Somehow, /dev/random and/or it's counterpart in /devices seems to have
suffered from the migration procedure.
Does anybody know how to fix the /dev/random issue? I'm not very fluent in
cryptoadm(1M) and some superficial reading of it's manpage did not enlighten
me too much (cryptoadm list -p claims all is well...).
Send the output of the following:

cryptoadm list -p
cat /etc/crypto/kcf.conf
dd if=/dev/random bs=10 count=1 | od -x (basically is it working now)

Can you also send the messages file so we can see when during boot this
happened since it might indicate a race condition with the availability
of /dev/random that only shows up with a ZFS root.
--
Darren J Moffat
Nicolas Linkert
2007-06-03 19:31:04 UTC
Permalink
Post by Constantin Gonzalez Schmitz
Post by Constantin Gonzalez Schmitz
2. After going through the zfs-bootification,
Solaris complains on reboot that
Post by Constantin Gonzalez Schmitz
/etc/dfs/sharetab is missing. Somehow this seems
to have been fallen through
Post by Constantin Gonzalez Schmitz
the cracks of the find command. Well, touching
/etc/dfs/sharetab just fixes
Post by Constantin Gonzalez Schmitz
the issue.
sharetab is now an in kernel filesystem.
Post by Constantin Gonzalez Schmitz
3. But here's a more serious one: While booting,
Apr 19 15:00:37 foeni kcf: [ID 415456
kern.warning] WARNING: No randomness
Post by Constantin Gonzalez Schmitz
provider enabled for /dev/random. Use
cryptoadm(1M) to enable a provider.
Post by Constantin Gonzalez Schmitz
Somehow, /dev/random and/or it's counterpart in
/devices seems to have
Post by Constantin Gonzalez Schmitz
suffered from the migration procedure.
Does anybody know how to fix the /dev/random issue?
I'm not very fluent in
Post by Constantin Gonzalez Schmitz
cryptoadm(1M) and some superficial reading of it's
manpage did not enlighten
Post by Constantin Gonzalez Schmitz
me too much (cryptoadm list -p claims all is
well...).
cryptoadm list -p
cat /etc/crypto/kcf.conf
dd if=/dev/random bs=10 count=1 | od -x (basically
is it working now)
***************************************************************

# cryptoadm list -p

Provider at user level:
=====================
/usr/lib/security/$ISA/pkcs11_kernel.so: All Mechanisms are activated.
/usr/lib/security/$ISA/pkcs11_softtoken.so: All Mechanisms are activated. random is activated.

Kernel Software provider:
==========================
des: All Mechanisms are activated..
aes: All Mechanisms are activated..
arcfour: All Mechanisms are activated..
blowfish: All Mechanisms are activated..
sha1: All Mechanisms are activated..
sha2: All Mechanisms are activated..
md4: All Mechanisms are activated..
md5: All Mechanisms are activated..
rsa: All Mechanisms are activated..
swrand: random is activated.

Kernel-Hardware provider:
==========================

****************************************************************

# cat /etc/crypto/kcf.conf
#
# CDDL HEADER START
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License (the "License").
# You may not use this file except in compliance with the License.
#
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# or http://www.opensolaris.org/os/licensing.
# See the License for the specific language governing permissions
# and limitations under the License.
#
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
#
# CDDL HEADER END
#
#
# Copyright 2007 Sun Microsystems, Inc. All rights reserved.
# Use is subject to license terms.
#
# ident "@(#)kcf.conf 1.10 07/04/10 SMI"
#
# /etc/crypto/kcf.conf
#
# Do NOT edit this file by hand. An administrator should use cryptoadm(1m)
# to administer the cryptographic framework. A developer for a kernel software
# provider package or a cryptographic provider device driver(s) package should
# provide an input file and use the {i,r}.kcfconf class action scripts to
# update this file during the installation and removal of the package.
#
# This document does not constitute an API. The /etc/crypto/kcf.conf file may
# not exist or may have a different content or interpretation in a future
# release. The existence of this notice does not imply that any other
# documentation that lacks this notice constitutes an API.
#
#
#
# Start SUNWcsr
des:supportedlist=CKM_DES_CBC,CKM_DES_ECB,CKM_DES3_CBC,CKM_DES3_ECB
aes:supportedlist=CKM_AES_ECB,CKM_AES_CBC,CKM_AES_CTR
arcfour:supportedlist=CKM_RC4
blowfish:supportedlist=CKM_BLOWFISH_ECB,CKM_BLOWFISH_CBC
sha1:supportedlist=CKM_SHA_1,CKM_SHA_1_HMAC_GENERAL,CKM_SHA_1_HMAC
sha2:supportedlist=CKM_SHA256,CKM_SHA256_HMAC,CKM_SHA256_HMAC_GENERAL,CKM_SHA384,CKM_SHA384_HMAC,CKM_SHA384_HMAC_GENERAL,CKM_SHA512,CKM_SHA512_HMAC,CKM_SHA512_HMAC_GENERAL
md4:supportedlist=CKM_MD4
md5:supportedlist=CKM_MD5,CKM_MD5_HMAC_GENERAL,CKM_MD5_HMAC
rsa:supportedlist=CKM_RSA_PKCS,CKM_RSA_X_509,CKM_MD5_RSA_PKCS,CKM_SHA1_RSA_PKCS,CKM_SHA256_RSA_PKCS,CKM_SHA384_RSA_PKCS,CKM_SHA512_RSA_PKCS
swrand:supportedlist=random
# End SUNWcsr
# Start SUNWdcar driver_names=dca
# End SUNWdcar

******************************************************************************************

# dd if=/dev/random bs=10 count=1 | od -x
1+0 datasets in
1+0 datasets out
0000000 285a 16c1 e017 b6ee 05eb
0000012

*******************************************************************************************


This message posted from opensolaris.org
Yannick Robert
2007-08-09 09:27:03 UTC
Permalink
Hello

it seems i have the same problem after zfs boot installation (following this setup on a snv_69 release http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/ ). The outputs from the requested command are similar to the outputs posted by dev2006.

Reading this page, i found no solution concerning the /dev/random problem. Is there somewhere a procedure to repair my install ?

Thanks in advance


This message posted from opensolaris.org
Yannick Robert
2007-08-09 09:39:23 UTC
Permalink
forgot to specify some details :

in my setup i do not install the ufsroot.

i have 2 disks
-c0d0 for the ufs install
-c1d0s0 which is my zfs root i want to exploit

my idea is to remove the c0d0 disk when the system will be ok


This message posted from opensolaris.org
Jürgen Keil
2007-08-09 11:25:00 UTC
Permalink
Post by Yannick Robert
in my setup i do not install the ufsroot.
i have 2 disks
-c0d0 for the ufs install
-c1d0s0 which is my zfs root i want to exploit
my idea is to remove the c0d0 disk when the system will be ok
Btw. if you're trying to pull the ufs disk c0d0 from the system, and
physically move the zfs root disk from c1d0 -> c0d0 and use that as
the only disk (= boot disk) in the system, you'll probably run into the
problem that zfs root becomes unbootable, because in the
etc/zfs/zpool.cache file the c1d0 name is still recorded for the
zpool containing the rootfs.

To fix it you probably have to boot a failsafe kernel from somewhere,
zpool import the pool from the disk's new location, and copy the
updated /etc/zfs/zpool.cache into the zfs root filesystem and build
new boot archives there...


This message posted from opensolaris.org
Darren J Moffat
2007-08-09 10:05:03 UTC
Permalink
Post by Yannick Robert
Hello
it seems i have the same problem after zfs boot installation (following this setup on a snv_69 release http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/ ). The outputs from the requested command are similar to the outputs posted by dev2006.
Reading this page, i found no solution concerning the /dev/random problem. Is there somewhere a procedure to repair my install ?
I've been thinking about the /dev/random problem recently and I think I
know what the problem is, but not yet how to fix it. I've cc'd the rest
of the crypto team.

With a UFS boot there is no attempted use of randomness until after
svc://system/cryptosvc has run. That service does two important things,
first it starts kcfd to put in place the kernel thread pool for async
crypto (not relevant to randomness) and secondly it runs 'cryptoadm
refresh' which pushes the (private) /etc/crypto/kcf.conf into the kernel.

When /dev/random was initially integrated it was monolithic, that is the
randomness pool and the entropy gatherer we combined. Later on when KCF
came along we split appart the pool (drv/random) from the software
entropy provider (crypto/swrand).

Unlike UFS when we do a ZFS boot we do use the in kernel interface to
/dev/random (random_get_bytes) before svc://system/cryptosvc has run.
The message you are seeing is from KCF saying that it has a random pool
but nothing providing entropy to it. This is because swrand hasn't yet
registered with kcf.

Now this was all done prior to newboot and SMF and part of the goal of
why KCF works this way with software providers is was to ensure no boot
time performance regressions by doing load on demand rather than forcing
the loading of all modules at boot time. With newboot on x86, and soon
on SPARC, the swrand module will be in the boot archive anyway.
--
Darren J Moffat
Krishna Yenduri
2007-08-09 21:36:33 UTC
Permalink
Post by Darren J Moffat
Post by Yannick Robert
Hello
it seems i have the same problem after zfs boot installation (following this setup on a snv_69 release http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/ ). The outputs from the requested command are similar to the outputs posted by dev2006.
Reading this page, i found no solution concerning the /dev/random problem. Is there somewhere a procedure to repair my install ?
To answer Yannick's question, the /dev/random warning message does not
indicate
any problem with the install and can be ignored.
Post by Darren J Moffat
...
Unlike UFS when we do a ZFS boot we do use the in kernel interface to
/dev/random (random_get_bytes) before svc://system/cryptosvc has run.
To be exact, the API used by ZFS kernel module is
random_get_pseudo_bytes().
Post by Darren J Moffat
The message you are seeing is from KCF saying that it has a random pool
but nothing providing entropy to it. This is because swrand hasn't yet
registered with kcf.
We had a similar issue with SCTP where in it uses the kernel API
random_get_pseudo_bytes() before swrand could register.

The solution we had there was to load swrand directly. From
uts/sparc/ip/Makefile
78 #
79 # Depends on md5 and swrand (for SCTP). SCTP needs to depend on
80 # swrand as it needs random numbers early on during boot before
81 # kCF subsystem can load swrand.
82 #
83 LDFLAGS += -dy -Nmisc/md5 -Ncrypto/swrand -Nmisc/hook
-Nmisc/neti


I think we can do a similar thing here. The zfs (or is it zfs-root ?)
kernel module
can have crypto/swrand as a dependency. I see that uts/sparc/zfs/Makefile
lists drv/random as a dependency. This is not needed because the
API is in modstubs now and it is not implemented in drv/random any more.
That can be replaced with crypto/swrand.

swrand does not need any crypto signature verification. So, it can
safely be loaded
early on during boot.
Post by Darren J Moffat
Now this was all done prior to newboot and SMF and part of the goal of
why KCF works this way with software providers is was to ensure no boot
time performance regressions by doing load on demand rather than forcing
the loading of all modules at boot time.
Yes. This requirement added a lot of complexity to KCF.
Post by Darren J Moffat
With newboot on x86, and soon
on SPARC, the swrand module will be in the boot archive anyway.
That would be great. It is cleaner and will remove the need for ad hoc
solutions like above.

-Krishna

Jürgen Keil
2007-08-09 11:13:23 UTC
Permalink
Post by Yannick Robert
it seems i have the same problem after zfs boot
installation (following this setup on a snv_69 release
http://www.opensolaris.org/os/community/zfs/boot/zfsboot-manual/ ).
Hmm, in step 4., wouldn't it be better to use ufsdump / ufsrestore
instead of find / cpio to clone the ufs root into the zfs root pool?

cd /zfsroot
ufsdump 0f - / | ufsrestore -xf -


Advantages:

- it copies the mountpoint for the /etc/dfs/dfstab filesystem
(and all the other mountpoints, like /tmp, /proc, /etc/mnttab, ...)


- it does not mess up the /lib/libc.so.1 shared library

I think the procedure at the above url could copy the wrong
version of the shared libc.so.1 into the zfsroot /lib/libc.so.1;
this might explain bugs like 6423745,
Synopsis: zfs root pool created while booted 64 bit can not be booted 32 bit


- the files hidden by the /devices mount are copied,too
Post by Yannick Robert
The outputs from the requested command
are similar to the outputs posted by dev2006.
Reading this page, i found no solution concerning the
/dev/random problem. Is there somewhere a procedure
to repair my install ?
AFAICT, there's nothing you can do to avoid the
"WARNING: No randomness provider enabled for /dev/random."
message with zfs root at this time. It seems that zfs mountroot
needs some random numbers for mounting the zfs root filesystem,
and at that point early during the bootstrap there isn't a fully initialized
random device available. This fact is remembered by the random
device and is reported later on, when the system is fully booted.

I think when the system is fully booted from zfs root, the random
device should work just fine.


This message posted from opensolaris.org
Robert Thurlow
2007-04-19 17:44:44 UTC
Permalink
Post by Constantin Gonzalez Schmitz
2. After going through the zfs-bootification, Solaris complains on reboot that
/etc/dfs/sharetab is missing. Somehow this seems to have been fallen through
the cracks of the find command. Well, touching /etc/dfs/sharetab just fixes
the issue.
This is unrelated to ZFS boot issues, and sounds like this bug:

6542481 No sharetab after BFU from snv_55

It's fixed in build 62.

Rob T
Constantin Gonzalez
2007-04-20 08:03:07 UTC
Permalink
Hi,
Post by Robert Thurlow
Post by Constantin Gonzalez Schmitz
2. After going through the zfs-bootification, Solaris complains on reboot that
/etc/dfs/sharetab is missing. Somehow this seems to have been fallen through
the cracks of the find command. Well, touching /etc/dfs/sharetab just fixes
the issue.
6542481 No sharetab after BFU from snv_55
It's fixed in build 62.
hmm, that doesn't fit what I saw:

- Upgraded from snv_61 to snv_62
- snv_62 booted with not problems (other than the t_optmgmt bug)
- Then migrated to ZFS boot
- Now the sharetab issues shows up.

So why did the sharetab issue only show up after the ZFSification of the
boot process?

Best regards,
Constantin
--
Constantin Gonzalez Sun Microsystems GmbH, Germany
Platform Technology Group, Global Systems Engineering http://www.sun.de/
Tel.: +49 89/4 60 08-25 91 http://blogs.sun.com/constantin/

Sitz d. Ges.: Sun Microsystems GmbH, Sonnenallee 1, 85551 Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Marcel Schneider, Wolfgang Engels, Dr. Roland Boemer
Vorsitzender des Aufsichtsrates: Martin Haering
Loading...