always remember

Nothing is foolproof to a sufficiently talented fool... Make something
idiot proof, and the world will simply make a bigger idiot.

Using LogDump & RBA’s to Reposition a REPLICAT Process

When a REPLICAT process ABEND’s it can be dificult to pinpoint why, and trying to get over the error and allow the REPLICAT to continue can be tricky.

In this example, my source is an Oracle Linux 12c machine running EXTRACT, my target is a Windows Server 2016 machine running MSSQL/REPLICAT.

The Error:

In this example, we are presented with the following errors in the REPLCIAT report after it has ABENDED

2019-08-15 18:21:20  WARNING OGG-03014  Source column COLUMN_NAME has more characters than target column COLUMN_NAME can hold. Some source characters will not be mapped during conversion from source character set UTF-8 to target character set UTF-16.

2019-08-16 09:28:56  ERROR   OGG-01163  Bad column length (357) specified for column COLUMN_NAME in table TABLE.X, maximum allowable length is 255.

From the same report, we need to garner what RBA the fault occured at, you can see this here:

Last log location read:
     FILE:      C:GoldenGatedirdat/1p000003921
     SEQNO:     3921
     RBA:       4982061
     TIMESTAMP: 2019-08-16 09:28:53.053432
     EOF:       NO
     READERR:   0

With this information, open LogDump, prepare the application, and move to your RBA:

Logdump 1 > open C:GoldenGatedirdat/1p000003921 (use your filename here)
Logdump 2 > ghdr on
logdump 3 > detail on
logdump 4 > detail data
logdump 5 > ggstoken on
logdump 6 > ggstoken detail
logdump 7 > pos 4982061 (use your RBA here)

Read On… ->

dave / September 3, 2019 / Code, Oracle

How To: Configure multiple VLAN interfaces in SolusVM

There may be times when you wish to give VM’s on one of your SolusVM nodes access to IP resrouces that are segmented into discrete VLAN’s at network level. If this is the case, you need to create network bridge interfaces on the node, and suply them with your VLAN interfaces. This is explained below.

  1. Configure the base interface, in this example, we ahve trunked eno2 with vlan’s 220 and 221, as we have group of VM’s that require to bind IP’s within this VLAN.
  2. [root@solus-node01]# cat ifcfg-eno2
    DEVICE=eno2
    BOOTPROTO=none
    UUID=xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    ONBOOT=yes
    TYPE=Ethernet
    NM_CONTROLLED=no
  3. Configure your VLAN sub interfaces, note that we designate each interface to its own new bridge interface, this is required.
  4. Read On… ->

dave / October 24, 2018 / Code, Guide

OGG-00730 – No minimum Supplemental Logging is enabled

This issue was encountered whilst shipping an Oracle 12c schema to an MSSQL Server 2014 instance using OGG 12.3.

During the Change Data Capture configuration and EXTRACT setup and start processes, you may find your EXTRACT abends with:

OGG-00730  No minimum supplemental logging is enabled.

There are 2 reasons this may occur, the first is that you actually don’t have any supplemental logging enabled… The second is a documented Oracle bug, in which the GoldenGate process detects the presence of LOG DATA, but reports back on it incorrectly. Both scenarios are explained below.

CHECK TO SEE IF DATABASE LEVEL SUPPLEMENTAL LOGGING IS ENABLED OR NOT:

SQL> SELECT force_logging, supplemental_log_data_min FROM v$database;

FORCE_LOGGING             SUPPLEME
------------------------- --------
NO                        NO

SQL>

In this case, there is no logging, so OGG is correct. We can enable it with:

SQL> ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;
Database altered.

Read On… ->

dave / July 27, 2018 / Code, Oracle

OGG-01194 – Oracle Golden Gate CHARSET mismatch

When entertaining the loathsome idea of shipping an established Oracle data set to MSSQL (SQL Server 2014, Oracle 12c, and OGG 12.3 in this case), you may run into an issue that presents itself in the following form in your EXTRACT report:

WARNING OGG-01194
EXTRACT task RINI9001 abended : Conversion from character set UTF-8 of source column <COLUMN_NAME> to character set windows-1252 of target column <COLUMN_NAME> failed because the source column contains a character 'ef 81 8a' at offset 123 that is not available in the target character set.

THE PROBLEM?:

Essentially, the issue is that you are trying to have your REPLICAT process convert Unicode data into a CHARSET where that Unicode character doesn’t exist. This is the default behaviour of REPLICAT, it will always try to convert source data charsets to the target machine native.

RESOLUTION:

This can be controlled with “SOURCECHARSET” parameter in your REPLICAT task param file. Specifically “SOURCECHARSET PASSTHRU”. Using this parameter will force REPLICAT to blindly import the source data and not try to convert it to the native charset of the target machine.

More information on SOURCECHARSET here

dave / July 26, 2018 / Code, Oracle

Monitor Pending Connections – Zen/Zevenet Load Balancers

In my working environment, we use (rather extensively) ZenLB (or as they are now know, Zevenet) Load Balancers. In production systems, sometimes the back-ends of an infrastructure, or the “real servers” behind the load balancers, can become unresponsive for whatever reason. A typical one that I see quite often is when using clustered MS Exchange Client Access servers behind a load balanced pool. IIS may lock up on one or multiple CAS’s causing the connections coming in from clients to be stored at LB level as “pending”.

This is fine, but in my experience, once the Zevenet LB racks up 1500+ pending connections on one of its farms, it quickly exhausts it’s available memory.

The following check is called by the Nagios NRPE agent installed locally on the LB (It’s just Debian 8 afterall)

#!/bin/bash
#
# ZenLB Pending/Established Connection Tracking v1.0 - Dave Byrne
#
hour=`date +%H`
pending=`cat /proc/net/nf_conntrack |grep SYN_SENT |grep dport='443|80' |wc -l`
established=`cat /proc/net/nf_conntrack |grep ESTABLISHED |grep dport='443|80' |wc -l`

if [ $pending -gt 5 ]
   then
      printf "CRITICAL - Pending connections above threshold! Pending: $pending -- Established: $establishedn"
   exit 2
elif [ $established -eq 0 ] && [ $hour -ge 8 ] && [ $hour -le 23 ];
   then
      printf "CRITICAL - No established connections! Pending: $pending -- Established: $establishedn"
   exit 2
else
      printf "OK - Pending connections at acceptable level. Pending: $pending -- Established: $establishedn"
   exit 0
fi

The check will go CRITICAL if pending connections across ANY of the farms goes above 5. It will also go CRITICAL is the established connections drops to 0 (probably bad). But I have limited this to a certain time frame, as I appreciate that there may well be 0 established connections at 4am!!

-Dave

dave / August 21, 2017 / Code, Nagios Monitoring

dave / July 27, 2016 / Code, Linux Bash, Nagios Monitoring