always remember

Nothing is foolproof to a sufficiently talented fool... Make something
idiot proof, and the world will simply make a bigger idiot.

How To: Configure multiple VLAN interfaces in SolusVM

There may be times when you wish to give VM’s on one of your SolusVM nodes access to IP resrouces that are segmented into discrete VLAN’s at network level. If this is the case, you need to create network bridge interfaces on the node, and suply them with your VLAN interfaces. This is explained below.

  1. Configure the base interface, in this example, we ahve trunked eno2 with vlan’s 220 and 221, as we have group of VM’s that require to bind IP’s within this VLAN.
  2. [root@solus-node01]# cat ifcfg-eno2
    DEVICE=eno2
    BOOTPROTO=none
    UUID=xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    ONBOOT=yes
    TYPE=Ethernet
    NM_CONTROLLED=no
  3. Configure your VLAN sub interfaces, note that we designate each interface to its own new bridge interface, this is required.
  4. [root@solus-node01]# cat ifcfg-en02.220
    VLAN=yes
    BRIDGE=br2
    DEVICE=eno2.220
    BOOTPROTO=none
    ONBOOT=yes
    TYPE=Ethernet
    NM_CONTROLLED=no
    [root@solus-node01]# cat ifcfg-eno2.221
    VLAN=yes
    BRIDGE=br1
    DEVICE=eno2.221
    BOOTPROTO=none
    ONBOOT=yes
    TYPE=Ethernet
    NM_CONTROLLED=no
  5. Configure your bridge interfaces.
  6. [root@solus-node01]# cat ifcfg-br2
    DEVICE=br2
    TYPE=Bridge
    BOOTPROTO=static
    ONBOOT=yes
    [root@solus-node01]# cat ifcfg-br1
    DEVICE=br1
    TYPE=Bridge
    BOOTPROTO=static
    ONBOOT=yes

    At this point, if you want the host node to have an IP in this VLAN, you would bind it to the bridge interface directly, you can use the usual IPADDR, PREFIX, GATEWAY etc..

  7. ‘UP’ your interfaces.
  8. [root@solus-node01]# ifup eno2.220
    [root@solus-node01]# ifup eno2.221
    [root@solus-node01]# ifup br2
    [root@solus-node01]# ifup br1
  9. Check the state of your bridges.
  10. [root@solus-node01]# brctl show
    <some info redacted>
    br1             8000.0cc47xxxxxxx       no              eno2.221
    br2             8000.0cc47xxxxxxx       no              eno2.220

    Note you should see your 2 new bridges with the relevant vlan interface attached to it, I’ve removed some data here as I use some odd custom work on br0 that would confuse this article.

    Now that you have bridges available, you can begin assigning these to VM’s that need access to it. In my case, I ahd to use KVM Custom Config in SolusVM to be able to A) specifiy the right bridge and B) create a second interface inside the VM.

  11. Custom config for a sample VM.
  12. <domain type='kvm'>
    	<name>kvmXXX</name>
     <uuid>xxxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx</uuid>
     <memory>1048576</memory>
     <currentMemory>1048576</currentMemory>
      <memtune>
       <hard_limit>1099776</hard_limit>
      </memtune>
     <vcpu>4</vcpu>
    <cpu>
     </cpu>
      <os>
       <type machine='pc'>hvm</type>
       <boot dev='hd'/>
       <boot dev='cdrom'/>
      </os>
     <clock sync='localtime'/>
      <devices>
       <emulator>/usr/libexec/qemu-kvm</emulator>
        <graphics type='vnc' port='xxxx' passwd='xxxxxxxx' listen='0.0.0.0'/>
        <disk type='file' device='disk'>
         <source file='/dev/vg_xxxxxxxx/kvmXXX_img'/>
         <target dev='hda' bus='virtio'/>
        </disk>
        <disk type='file' device='cdrom'>
         <target dev='hdc'/>
         <readonly/>
        </disk>
        <interface type='bridge'>
         <source bridge='br1'/>
         <target dev='kvmXXX.0'/>
         <mac address='00:16:3c:xx:xx:xx'/>
        </interface>
        <interface type='bridge'>
         <source bridge='br2'/>
         <target dev='kvmXXX.1'/>
        </interface>
        <input type='tablet'/>
        <input type='mouse'/>
        </devices>
        <features>
        <acpi/>
        <apic/>
        </features>
        <on_poweroff>destroy</on_poweroff>
        <on_reboot>restart</on_reboot>
    </domain>

    Note that this is heavily edited, the main focus is the duplicate “interface” section, and that the duplicate has no MAC address specified. You can also see that br1 and br2 have been specified. Make a mental note of which one is which so that in your VM, you can assign IP’s in the relevant VLAN.

    Save the custom config and reboot the VM. Assign IP’s once booted into the VM.

  13. Checking your bridge status now should show the VM interface active within it.
  14. [root@solus-node01]# brctl show
    <some info redacted>
    br1             8000.0cc47axxxxxx       no              eno2.221
                                                            kvmXXX.0
    br2             8000.0cc47xxxxxxx       no              eno2.220
                                                            kvmXXX.1

dave / October 24, 2018 / Code, Guide

OGG-00730 – No minimum Supplemental Logging is enabled

This issue was encountered whilst shipping an Oracle 12c schema to an MSSQL Server 2014 instance using OGG 12.3.

During the Change Data Capture configuration and EXTRACT setup and start processes, you may find your EXTRACT abends with:

OGG-00730  No minimum supplemental logging is enabled.

There are 2 reasons this may occur, the first is that you actually don’t have any supplemental logging enabled… The second is a documented Oracle bug, in which the GoldenGate process detects the presence of LOG DATA, but reports back on it incorrectly. Both scenarios are explained below.

CHECK TO SEE IF DATABASE LEVEL SUPPLEMENTAL LOGGING IS ENABLED OR NOT:

SQL> SELECT force_logging, supplemental_log_data_min FROM v$database;

FORCE_LOGGING             SUPPLEME
------------------------- --------
NO                        NO

SQL>

In this case, there is no logging, so OGG is correct. We can enable it with:

SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
Database altered.

Switch the LOG FILE in DB:

SQL> ALTER SYSTEM SWITCH LOGFILE;
System altered.

Modify your EXTRACT process to begin at a point where SUPPLEMENTAL LOGGING is enabled:

GGSCI> ALTER EXTRACT, EORA9001 BEGIN NOW
EXTRACT altered.

IF SUPPLEMENTAL LOGGING IS ENABLED, AND THE ERROR IS STILL PRESENT:

If the above check reported supplemental logging was already enabled, you can get around the OGG bug by instructing your EXTRACT to ignore the results of the SUPP LOG CHECK. You can do this by editing your Change Capture EXTRACT PARAM file and adding the line “TRANLOGOPTIONS DISABLESUPPLOGCHECK”.

Note that the above parameter is not documented piublicly by Oracle. You may wish to remove this once the change capture process has started successfully.

dave / July 27, 2018 / Code, Oracle

OGG-01194 – Oracle Golden Gate CHARSET mismatch

When entertaining the loathsome idea of shipping an established Oracle data set to MSSQL (SQL Server 2014, Oracle 12c, and OGG 12.3 in this case), you may run into an issue that presents itself in the following form in your EXTRACT report:

WARNING OGG-01194
EXTRACT task RINI9001 abended : Conversion from character set UTF-8 of source column <COLUMN_NAME> to character set windows-1252 of target column <COLUMN_NAME> failed because the source column contains a character 'ef 81 8a' at offset 123 that is not available in the target character set.

THE PROBLEM?:

Essentially, the issue is that you are trying to have your REPLICAT process convert Unicode data into a CHARSET where that Unicode character doesn’t exist. This is the default behaviour of REPLICAT, it will always try to convert source data charsets to the target machine native.

RESOLUTION:

This can be controlled with “SOURCECHARSET” parameter in your REPLICAT task param file. Specifically “SOURCECHARSET PASSTHRU”. Using this parameter will force REPLICAT to blindly import the source data and not try to convert it to the native charset of the target machine.

More information on SOURCECHARSET here

dave / July 26, 2018 / Code, Oracle

Monitor Pending Connections – Zen/Zevenet Load Balancers

In my working environment, we use (rather extensively) ZenLB (or as they are now know, Zevenet) Load Balancers. In production systems, sometimes the back-ends of an infrastructure, or the “real servers” behind the load balancers, can become unresponsive for whatever reason. A typical one that I see quite often is when using clustered MS Exchange Client Access servers behind a load balanced pool. IIS may lock up on one or multiple CAS’s causing the connections coming in from clients to be stored at LB level as “pending”.

This is fine, but in my experience, once the Zevenet LB racks up 1500+ pending connections on one of its farms, it quickly exhausts it’s available memory.

The following check is called by the Nagios NRPE agent installed locally on the LB (It’s just Debian 8 afterall)

#!/bin/bash
#
# ZenLB Pending/Established Connection Tracking v1.0 - Dave Byrne
#
hour=`date +%H`
pending=`cat /proc/net/nf_conntrack |grep SYN_SENT |grep dport='443|80' |wc -l`
established=`cat /proc/net/nf_conntrack |grep ESTABLISHED |grep dport='443|80' |wc -l`

if [ $pending -gt 5 ]
   then
      printf "CRITICAL - Pending connections above threshold! Pending: $pending -- Established: $establishedn"
   exit 2
elif [ $established -eq 0 ] && [ $hour -ge 8 ] && [ $hour -le 23 ];
   then
      printf "CRITICAL - No established connections! Pending: $pending -- Established: $establishedn"
   exit 2
else
      printf "OK - Pending connections at acceptable level. Pending: $pending -- Established: $establishedn"
   exit 0
fi

The check will go CRITICAL if pending connections across ANY of the farms goes above 5. It will also go CRITICAL is the established connections drops to 0 (probably bad). But I have limited this to a certain time frame, as I appreciate that there may well be 0 established connections at 4am!!

-Dave

dave / August 21, 2017 / Code, Nagios Monitoring

dave / July 27, 2016 / Code, Linux Bash, Nagios Monitoring

dave / April 6, 2016 / Code, Nagios Monitoring