Where to Install Standalone Agent in ODI

ODI is a true ELT tool: no middle-tier server is required. Everything runs in the db, all operations can be run by a lightweight agent. Therefore without a dedicated server, where to install the agent?

Source systems are not ideal – they could be dispersed widely… so installing the agent on the target systems appears more appropriate. But in the end, “target” is a convenience.

Agent Connectivity Needs – An agent may need to perform up to 3 tasks for a process to run:
• Connect to the repository (always)
• Connect to the sources and targets (always)
• Provide JDBC access to the data (if needed)

Connection to the repository – agent will connect to the repository to perform following tasks:
• Retrieve the code that must be executed
• Finish the code generation that must be executed based on the context that was selected
• Write the generated code in the operator tables
• After the code is executed by the db, update the operator tables with statistics and if necessary, error messages.

To perform all these operations, the agent will connect to the repository using JDBC. The parameters for the agent to connect are defined when the agent is installed. For a standalone agent, you will find these parameters in the odiparams.sh file (or odiparams.bat on windows).

• What does this mean for the location of the agent?
Since the agent uses JDBC to connect to the repository, the agent does not have to be on the same machine as the repository. The amount of data exchanged with the repository is limited to logs generation and updates, but this can become somewhat consequent in near real time environments. Thus It is recommended that the agent be at least on the same LAN as the repository.

Connection to the sources and targets –  The agent will use JDBC to connect to all database sources and targets at the beginning of a session execution. These connections will be used by the agent to send the DDL and DML to the databases.

• What does this mean for the location of the agent?
It does not have to be physically installed on any of the systems that host the databases it should just be able to connect to all databases.
Conclusion: from an orchestration perspective, the agent could be anywhere in the LAN.

Data Transfer Using JDBC – ODI processes can use multiple techniques to extract from and load data into sources and targets: JDBC is one of these techniques. If the processes executed by the agent use JDBC to move data from source to target, then the agent itself establishes this connection: as a result the data will physically flow through the agent.

• What does this mean for the location of the agent?
This is a case we have to pay more attention to the agent location. In all previous cases, the agent could have been installed pretty much anywhere as the performance impact of moving it was negligible. Now if data physically moves through the agent, placing the agent on either the source server or the target server will in effect limit the number of network hops required for the data.

install the agent on the file server along with the loading utilities – or share the directories where the files and utilities are installed so that the agent can view them remotely.

Beyond databases: Big Data
Namenode is in charge of distributing the execution across all DataNodes. It would be totally counter-productive for the ODI agent to try and bypass the NameNode. From that perspective, the agent would have to be installed on the NameNode.

Note: The Oracle BigData appliance ships with the ODI agent pre-packaged so that the environment is immediately ready to use.

Firewall Considerations
One element that seems pretty obvious is that no matter where you place your agents, you have to make sure that the firewalls in your corporation will let you access the necessary resources.

 

Advertisements