Configure CDC for the Replicated Tables
29.4.1 Initial Load Method
The staging tables contain a replica of the structure and data from the source tables. The Oracle GoldenGate processes capture changes on the source tables and apply them to the target. Yet the staging tables must be initially loaded with the original content of the source tables. You can use the following methods to perform the initial load: ■ Using Oracle GoldenGate: A specific GoldenGate process loads the whole content of the source tables into the staging tables. ■ Using Oracle Data Integrator: The Generate Interfaces IN option of Oracle Data Integrators Common Format Designer. This method uses ODI interfaces to transfer the data. ■ Using database backuprestore tools to copy data and structures.29.4.2 Tuning Replication Performances
The following KM options can be used to improve replication performances: ■ COMPATIBLE: This Oracle-specific option affects the use of the PURGE key word and the way statistics using DBMS_STATS or ANALYZE are collected. Set this value to the database version of your staging server. ■ NB_APPLY_PROCESS: Number of Oracle GoldenGate Apply processes created on the staging server. ■ TRAIL_FILE_SIZE: Size of the Oracle GoldenGate trail file in Megabytes. For the NB_APPLY_PROCESS and TRAIL_FILE_SIZE parameters, see the Oracle GoldenGate Documentation on OTN for more information on performance tuning.29.4.3 One Source Multiple Staging Configuration
It is possible to set up a configuration where changes are captured on a single source and replicated to several staging servers. The example below illustrates how to set this up in a typical configuration. Replication should source from source server SRC and replicate in both STG1 and STG2 staging servers. 1. Configure CDC for STG1 with the following configuration: ■ SRC_OGG_OBJECT_GROUP = SRC ■ SRC_SETUP_OGG_PROCESSES = YES ■ STG_OGG_OBJECT_GROUP = STG1 ■ STG_SETUP_OGG_PROCESSES = YES ■ ENABLE_ODI_CDC= YES 2. Start the journal and follow the instructions in the readme to set up the Oracle GoldenGate processes in SRC and STG1. 3. Configure CDC for STG2 with the following configuration: 29-10 Oracle® Fusion Middleware Connectivity and Knowledge Modules Guide for Oracle Data Integrator ■ SRC_OGG_OBJECT_GROUP = SRC Use the same name as for STG1 ■ SRC_SETUP_OGG_PROCESSES = NO The processes have been set up with STG1 ■ STG_OGG_OBJECT_GROUP = STG2 ■ STG_SETUP_OGG_PROCESSES = YES ■ ENABLE_ODI_CDC= YES Start the journal and follow the instructions in the readme to set up the Oracle GoldenGate processes in SRC and STG2. Note that playing the configuration on SRC again will not recreate a capture process, trail files, or definition files. It will simply create a new Oracle GoldenGate Datapump process to push data to STG2. 30 Oracle SOA Suite Cross References 30-1 30 Oracle SOA Suite Cross References This chapter describes how to work with Oracle SOA Suite cross references in Oracle Data Integrator. This chapter includes the following sections: ■ Section 30.1, Introduction ■ Section 30.2, Installation and Configuration ■ Section 30.3, Working with XREF using the SOA Cross References KMs ■ Section 30.4, Knowledge Module Options Reference30.1 Introduction
Oracle Data Integrator features are designed to work best with Oracle SOA Suite cross references, including integration interfaces that load a target table from several source tables and handle cross references.30.1.1 Concepts
Cross-referencing is the Oracle Fusion Middleware Function, available through the Oracle BPEL Process Manager and Oracle Mediator, previously Enterprise Service Bus ESB, and leveraged typically by any loosely coupled integration built on the Service Oriented Architecture. It is used to manage the runtime correlation between the various participating applications of the integration.30.1.1.1 General Principles
The cross-referencing feature of Oracle SOA Suite enables you to associate identifiers for equivalent entities created in different applications. For example, you can use cross references to associate a customer entity created in one application with native id Cust_100 with an entity for the same customer in another application with native id CT_001. Cross-referencing XREF facilitates mapping of native keys for entities across applications. For example, correlate the same order across different ERP systems. The implementation of cross-referencing uses a database schema to store a cross reference information to reference records across systems and data stores. For more information about cross references, see Working with Cross References in the Oracle Fusion Middleware Developers Guide for Oracle SOA Suite. The optional ability to update or delete source table data after the data is loaded into the target table is also a need in integration. This requires that the bulk integrationParts
» Oracle Fusion Middleware Online Documentation Library
» Terminology Using This Guide
» Concepts Knowledge Modules Introduction
» System Requirements and Certifications
» Using External Tables Technology Specific Requirements
» Using Oracle Streams Technology Specific Requirements
» Connectivity Requirements Installation and Configuration
» Creating an Oracle Physical Schema
» Setting Up an Integration Project
» Reverse-engineer an Oracle Model
» Setting up Changed Data Capture
» Designing an ETL-Style Interface
» Troubleshooting Oracle Database Errors Common Problems and Solutions
» System Requirements and Certifications Technology Specific Requirements
» Creating a File Physical Schema
» In the Models accordion, right click your File Model and select New Datastore.
» In the editor toolbar, click Reverse-Engineer.The Columns Setup Wizard is
» Click OK when the columns definition is complete. From the File main menu, select Save.
» In the Definition Tab, enter the following fields:
» Go to the Files tab to describe the type of file. Set the fields as follows:
» In the toolbar menu, click Reverse Engineer COBOL CopyBook.
» Click OK. COBOL Copybook reverse-engineering
» Create an ODBC Datasource for the Excel Spreadsheet
» Define the Data Server, Physical and Logical Schema for the Microsoft Excel
» Run the customized reverse-engineering
» Select the Microsoft Excel Driver .xls driver.
» Name the data source: ODI_EXCEL_FILE_REPO and select the file
» In Topology Navigator, add a Microsoft Excel data server with the following
» From the File main menu, select Save.
» Add a physical schema to this data server. Leave the default values in the
» In the Context tab of the physical schema, click Add.
» In the new line, select the context that will be used for reverse engineering and
» In the Reverse-Engineer Tab, set the following parameters:
» In the toolbar menu, click Reverse-Engineer.
» Technology-Specific Requirements Installation and Configuration
» Reverse-engineer a Data Model
» Loading Data from an ANSI SQL-92 Compliant Technology
» Loading Data to an ANSI SQL-92 Compliant Technology
» Integrating Data in an ANSI SQL-92 Compliant Technology
» System Requirements Installation and Configuration
» Technologic Specific Requirements Installation and Configuration
» Creating a Physical Schema for XML
» Reverse-Engineering an XML Model
» Synchronizing XML File and Schema
» Loading Data from an XML Schema
» Loading Data to an XML Schema
» Detect the Errors Coming from XML Common Errors
» Creating a Complex File Physical Schema
» Designing an Interface Oracle Fusion Middleware Online Documentation Library
» Using the BULK INSERT Command
» Using Linked Servers Technology Specific Requirements
» Creating a Microsoft SQL Server Physical Schema
» Create a Microsoft SQL Server Model
» Reverse-engineer a Microsoft SQL Server Model
» Loading Data from Microsoft SQL Server
» Integrating Data in Microsoft SQL Server
» Creating a Microsoft Excel Data Server
» Creating a Microsoft Excel Physical Schema
» Setting up Data Quality Setting Up an Integration Project
» Create a Microsoft Excel Model
» Reverse-engineer a Microsoft Excel Model
» Loading Data from Microsoft Excel
» Loading Data to Microsoft Excel
» Decoding Error Messages Common Problems and Solutions
» Specific Requirements Oracle Fusion Middleware Online Documentation Library
» Creating a Netezza Physical Schema
» Reverse-engineer a Netezza Model
» Loading Data from Netezza Loading Data to Netezza
» Creating a Teradata Physical Schema
» Reverse-engineer a Teradata Model
» Loading Data from Teradata Loading Data to Teradata
» Integrating Data in Teradata
» Primary Indexes and Statistics
» Support for Teradata Utilities Support for Named Pipes Optimized Management of Temporary Tables
» Creating a Hypersonic SQL Data Server
» Creating a Hypersonic SQL Physical Schema
» Setting up Changed Data Capture Setting up Data Quality Designing an Interface
» Introduction Oracle Fusion Middleware Online Documentation Library
» Concepts Knowledge Modules Oracle Fusion Middleware Online Documentation Library
» Creating a DB2400 Physical Schema
» Reverse-engineer an IBM DB2400 Model
» Setting up Trigger-Based CDC
» CDCRTVJRN Program Details Setting up Log-Based CDC
» Using the CDC with the Native Journals
» Problems While Reading Journals
» Loading Data from IBM DB2 for iSeries
» Loading Data to IBM DB2 for iSeries
» Integrating Data in IBM DB2 for iSeries
» Installing the Run-Time Agent on iSeries
» Using Client Access Alternative Connectivity Methods for iSeries
» Change the driver and URL to your AS400 server with the following information:
» Set the following java properties for the java machine the run-time agent deployed
» Troubleshooting Error messages Troubleshooting
» Connection Errors Common Problems and Solutions
» Integrating Data in Oracle BI
» Extracts the OBIEE Metadata from a OBIEE Instance
» Using the Lineage Lineage Lifecycle
» Installation Overview Installing the Lineage in an OBIEE Server
» Requirements Installing the Lineage in an OBIEE Server
» Post-Installation Tasks Installing the Lineage in an OBIEE Server
» Exporting the OBIEE Repository Documentation to a Text File
» Exporting the OBIEE Web Catalog Report to a Text File
» Refreshing the OBIEE Lineage From Existing Exports
» Configuring the Scripts Automating the Lineage Tasks
» Automating Lineage Deployment Automating Lineage Refresh
» Viewing Execution Statistics Viewing and Filtering Lineage Data
» Using the Dashboard Using the Lineage in OBIEE Dashboards
» Using Lineage and Hierarchy Using Contextual Lineage
» Reverse-engineer an Essbase Model
» Loading Metadata Designing an Interface
» Loading Data Designing an Interface
» Data Extraction Methods for Essbase
» Extracting Essbase Data Extracting Data
» Extracting Members from Metadata
» Creating an Hyperion Financial Management Data Server
» Creating an Hyperion Financial Management Physical Schema
» Create an Financial Management Model
» Reverse-Engineer an Financial Management Model
» Extracting Financial Management Data
» Extracting Members from Member Lists
» Data Store Tables Oracle Fusion Middleware Online Documentation Library
» Creating an Hyperion Planning Data Server
» Creating an Hyperion Planning Physical Schema
» Reverse-engineer a Planning Model
» Log on to Planning Web. Select Administration Data Load Administration.
» Accounts Datastore Tables and Data Load Columns
» Employee Datastore Tables and Data Load Columns
» Entities Datastore Tables and Data Load Columns
» User-Defined Dimensions Datastore Tables and Data Load Columns
» Attribute Dimensions Datastore Tables and Data Load Columns
» JMS Message Structure Concepts
» Creating a JMS Physical Schema
» Create a JMS Model Defining the JMS Datastores
» Loading Data from a JMS Source Integrating Data in a JMS Target
» Declaring JMS Properties Using JMS Properties
» Using Property Values as Source Data
» Setting Properties when Sending a Message
» Creating a JMS XML Physical Schema
» Reverse-Engineering a JMS XML Model
» Loading Data from a JMS XML Source Integrating Data in a JMS XML Target
» Creating a Physical Schema for LDAP
» Reverse-Engineering an LDAP Model
» Loading Data from an LDAP Directory
» Loading Data to an LDAP Directory
» Integrating Data in an LDAP Directory
» Setting Up an Integration Project Troubleshooting
» Creating a TimesTen Physical Schema
» Reverse-engineer a TimesTen Model
» Setting Up an Integration Project Setting up Data Quality
» Integrating Data in TimesTen
» Create an Attunity Stream Model Reverse-engineer an Attunity Stream Model
» Setting Up an Integration Project Designing an Interface Using the LKM Attunity to SQL
» Overview of the GoldeGate CDC Process
» Create the Staging Physical Schema
» Define the Source Data Server
» Create the Source Physical Schema
» Create the Replicated Tables
» Set Up an Integration Project
» Configure CDC for the Replicated Tables
» Configure and Start Oracle GoldenGate Processes
» Design Interfaces Using Replicated Data
» Initial Load Method Advanced Configuration
» Tuning Replication Performances Advanced Configuration
» One Source Multiple Staging Configuration
» Cross Reference Table Structures
» Loading Phase LKM Overview of the SOA XREF KM Process
» Defining the Topology Working with XREF using the SOA Cross References KMs
Show more