Test Solaris Root Mirror

Here's the situation. Being the good UNIX SysAdmin that you are, one of the first things you do is mirror the rootpool. You do something like:

zpool attach -f rpool c0t5000CCA03C5A7C00d0 c0t5000CCA03C5C19CCd0

...wait for the mirror to finish resilvering...

installboot -f -F zfs /usr/platform/`uname -i`/lib/fs/zfs/bootblk /dev/rdsk/c0t5000CCA03C5C19CCd0

(or better use - see comments below for why - :bootadm install-bootloader)

(Notice that my disk devices don't use slices - there'd be an "s0" at the end of the disk names - older ZFS systems needed to the root disk to be on a slice - this has fallen away)

So to test that you boot off the root disk - you go to ok prompt and try to boot off the second disk

shutdown -y -i0 -g0
...
ok> boot disk1
Boot device: /pci@3c0/pci@1/pci@0/pci@2/scsi@0/disk@p0  File and args:
ERROR: /packages/deblocker: Last Trap: Fast Data Access MMU Miss

So that's a bit of a bitch. Luckily, this is only a test. Start up your machine normally and then shut down with an init 0. Somehow rebooting with an init, sorts this out.

(If it wasn't a test, you can try to specify the path old school. Your path you can figure out - though I've had hit and miss success - by running devalias and scsi-probe-all and doing a path similar to /pci@400/pci@1/pci@0/pci@0/LSI,sas@0/disk@w5000cca02584ad19,0:a. - Sidenote: If that doesn't work I've had limited success by adding a to the last number before the comma).

Either way, once you've got a booted system. You can check which disk you're booted from by running prtconf -vp |grep bootpath.

This post is a little neither here nor there - but that's because my testing has brought various results and was done whil I was changing from a sas root disk to an ssd root disk. I'll update it as I retest.

VLAN tagging in Solaris

If you want to have zones in multiple subnets but using the same physical port, you have to use VLAN tagging. VLAN tagging is pretty easy to configure on the zones (point 7), less so on the global zone.

1. The Network guys have to do a few things for you:
• set the network ports your nic connects to as "trunked"
• give you the vlan id of the vlans you want to connect to (digits)
• for aggregated NICs, set LACP to active (rather than auto)
• set the default vlan-id of the ports to 1 
2. NOTE: Configuring the ports as trunked, obsoletes any traffic that isn't vlan tagged. All or nothing baby. 
3. Your aggregate needs LACP activity to be active
• dladm modify-aggr -L active -T short aggr0
4. I use aggregates, but I think most of the same steps below applies for IPMP.
5. I wish you could add a default vlan ID to the aggregate when you create it but you can't (and I get the feeling if I think really hard about it, I'll be able to see the logic in why). Instead you have to create a vnic on the aggregate that uses that vlan ID:
• dladm create-vnic -v 10 -l aggr0 vnic10
6. Now create an address on that vnic
• ipadm create-ip vnic10
• ipadm create-addr -T static -a 196.0.10.15/24 vnic10
7. That sorts out the global zone. For the zones its pretty easy. Just set the vlan-id attribute (under anet) on the zone config.

NOTES:

• The active LACP is not something I'm sure needs to be there but it worked so I'm leaving it.
• IPMP in zones - if I recall correctly - needs vnics created for you to do IPMP within the zone. Just make sure you assign the correct vlan ID to those vnics and you should be fine.