Oracle BI Cloud Service (BICS): Data Sync Utility Overview

Author: Michael Bender, Performance Architects

Late last year, Oracle released its Software-as-a-Service (SAAS) business intelligence offering called Business Intelligence Cloud Service, or BICS. One of the requirements for using this tool is that all relational database data must be first uploaded to a cloud version of the Oracle database:

The next obvious question is, “What are the ways in which I can upload data to my cloud Oracle database?” On initial release, BICS offered a few options such as:

1. Data Loader: An Oracle ApEx application that allows for uploading data via a flat file export from a relational database. It is limited in scope, however, and does not allow automation.
2. SQL Developer Carts: An interactive method of creating a cloud connection within SQL Developer. Automation and scheduling are more cumbersome for novice users.
3. REST API: An advanced method that requires API programming knowledge.

More recently, Oracle introduced a new tool for uploading data to BICS: a client utility called “Data Sync.” Data Sync is a free utility that supports loading on premise data from both relational databases and flat file sources, and can be downloaded from Oracle TechNet here.

The download is a simple unzip operation with no executable file. For security reasons, the tool should be downloaded to a server environment and is supported on both Windows and Linux/Unix platforms. A quick note on prerequisites for using Data Sync:

• Java Developer Kit (JDK) 1.7 or later is required – JRE will not suffice
• Prior to running the utility, update the config.bat file to point to your JDK
  
• Databases supported:
  • Oracle
  • MS SQL Server
  • DB2
  • Teradata
  • MySQL
  • Oracle Times Ten
• JDBC Drivers: Data Sync uses JDBC drivers to extract data from source databases. For databases other than Oracle, it may be necessary to copy the .jar drivers from the source database install to the \lib directory in the Data Sync folder

The first time Data Sync starts, it will prompt for environment configuration, a repository name, and a login password. Then, there are three main components of the utility:

1. Connections: Defining source databases / files and target (BICS) databases.
2. Project: Defining source and target database tables.
3. Jobs: Defining groups of tables to upload.

Connections: Source Definition

Connections: Target Definition

Project: Database Tables and Load Strategy Definition

Job: Scheduling Capabilities

Other than standard logging, one of the other useful features of the tool is email notification of success or failure.

For more advanced functionality, such as integration with upstream or downstream ETL processes, Data Sync also includes a command line option. The most recent version of Data Sync is v1.1, and contains a few enhancements:

• Automatic retry: In the event a connection fails during a data upload job, Data Sync will retry up to 10 times. In order to support this, three additional columns are added to each target table to accommodate the reload process. In the case of failed data loads, the utility will automatically clean up any partially loaded records.
• Additional delimiter support for file based data sources (pipe and custom single character).
• Support for table / column names using mixed case and Oracle reserved words.

In our experience at Performance Architects, Data Sync is a lightweight, reliable, and easy to use tool to batch up and schedule data loads into a BICS environment. Contact the Performance Architects team at sales@performancearchitects.com for more info.