Oracle Data Integrator 12c

ODI 12c is out! check for latest version ODI Downloads The main new features are, new flow-based paradigm and the ability to load multiple targets within the same interface…no “interfaces” anymore, they are now called “mappings”!. New mappings: In the new logical tab of mapping, you can drag and drop source datastores & target datastores in the same canvas.

To map source to target and build the logic, a new “component” panel, very similar to OWB has been added on the right hand side of the Studio. From there we can drag and drop join, filter, expression, union and lookup components into your canvas. There is even a distinct component, which means we are done with creating a yellow interface just to select distinct rows from the source.

components_ds

Every datastores and components have an IN and an OUT connector. Dataflows are created by dragging a connector on another one. We can map an OUT connector to multiple IN connectors, and therefore load multiple targets at the same time!

mapping_multi_target_ds

We can now have multiple physical implementations – called deployment specifications – while keeping the same business logic we can for instance create only one mapping for both initial and incremental load by selecting a different IKM in each of these deployment specifications (DS) and select which one to use at run-time.

deployment_specs_ds

Few more useful new features are:

  • In-mapping lineage and impact analysis : When a column in one datastore is selected, all the columns used to load it and all the columns loaded by it are highlighted.
  • Syntax highlighting appears in the expression fields.
  • Autocompletion is available in every expression fields. Columns are suggested based on the few characters already typed.

lineage_auto_completion_small_ds

Reusable Mappings: ODI 12c introduces the concept of reusable mapping, similar to those in OWB. It is designed like a standard mapping except that it uses an input and/or an output signature in replacement of datastores. This allow to reuse it in multiple regular mappings by connecting these signatures to other components. When upgrading from ODI 11g to ODI 12c, yellow (temporary) interfaces will turned to Reusable Mappings.

Debugger: Instead of running a mapping, we can now also debug it. A new pane appears where you can see the blueprints of the mapping. From there, we can set breakpoints or control execution step by step.

debug_add_breakpoint_blog_ds

What about OWB jobs ? ODI 12.1.2 enables the execution and monitoring of OWB jobs within ODI. A new OWB technology is now supported in the topology in order to plug the OWB repository. Once it’s done you can run an OWB mapping or process flow from a package or a procedure which is done with a new ODI Tool called OdiStartOwbJob.

WebLogic Management Framework: Another big change with this new version is the new WebLogic Management Framework used to manage the standalone agent in replacement of OPMN (Oracle Process Manager and Notification). Based on Weblogic Server, it brings a unified management of Fusion Middleware components and it provides a wizard for the configuration – the Fusion Middleware Configuration Wizard. Practically the standalone agent is now installed outside of the ODI installation, in a domain similar to WLS domains. The benefits of it, apart from the centralised management, is that it supports Weblogic Scripting Tool (WLST) and can be monitored through Enterprise Manager. The standalone agent can be launched manually or through the Node Manager.

Component Knowledge Modules: Next to the standard KMs – now called Template KMs -, a new type of KM has been introduced. Going one step further in the reusability, the Component KMs contain steps that are defined once and shared among several of them. Instead of being templates of code with some call to the Susbtitution API at runtime , these KMs are actually compiled libraries that will generate the code based on the components present in a mapping. The Component KMs are provided out-of-the-box and are not editable. You can check the type of a KM in the physical tab of a mapping.

KM Editor: we can still import, edit and create Template KMs, now find them in the folder <odi_home>/odi/sdk/xml-reference.

The KM Editor has been reviewed to avoid us to constantly switch from one tab to another. Now when the focus is set on a KM task, it is directly displayed in the Property Editor. In addition the command field supports syntax highlighting and auto-completion.

KM_editor_ds

In-Session Parallelism: it is now possible to have the extract tasks (LKMs) running in parallel. It’s actually done by default. If two sources are located in the same execution unit on the physical tab, they will run in parallel. If you want a sequential execution, you can drag and drop one of your units onto a blank area. A new execution unit will be created and ODI will choose in which order it will be loaded.

in-session_para3

In the topology, we can now define the number of threads per Session for your physical agents to avoid one session to get all the resources.

maxim_thread_session_ds

Parallel Target Table Load: With ODI 11g, if two interfaces loading data in the same datastore are executed at the same time or if the same interface is executed twice, you can face some problem. For instance a session might delete a worktable in which the other session wants to insert data. With 12c, a new “Use Unique Temporary Object Names” checkbox appears in the Physical tab of your mapping. If you select it, the worktables for every session will have a unique name. You are now sure that another session won’t delete it or insert other data. If the execution of the mapping fails? With these unique name, these tables won’t be deleted the next time we run the mapping.” A task in a KM can now be created under the section “Mapping Cleanup”. It will be executed at the end of the mapping even if it fails.

Cleanup_tasks_ds

What if the agent crashes? A new ODI Tool is available: OdiRemoveTemporaryObjects. It runs every cleanup tasks that have not run normally and by default it is automatically called when a agent restarts.

Session Blueprint: When doing real-time integration with ODI, you can have a lot of jobs running every few seconds or minutes. For each call of a scenario, the ODI 11g agent retrieves all the steps and tasks it should execute from the repository and write it synchronously in the logs before starting to execute anything. After the execution it deletes the logs that shouldn’t be kept when the log level is low. At the end of the day, even if you set the logs to a low level, you end up with a lot of redo logs and archive logs in your database.

Instead of retrieving the scenario for every execution, the ODI 12c agent now only retrieves a Session Blueprint once and keeps it in the cache. As it is cached in the agent, there is no need to get anything from the repository if the job runs again a few minutes later. The agent also write – asynchronous – only the needed logs defined by the log level, as it only relies on the blueprint for it’s execution and not on the logs. The parameters related to blueprints are available in the definition of the physical agents in the topology. In summary, the overhead of executing session is greatly reduced.

Datastore Change Notification: a change notification will be displayed in every mapping using the datastore which has been changed. Wallet: We can now store Login Credentials in a Wallet, itself protected by a password.

For starter follow the Getting Started Guide, & download the Pre-Built VM

Advertisements