Agents

  • An agent is a service written to deal with a specific type of data source.
  • The Script agent allows you create a service for non-supported data sources.
  • The controller installs agents on your on-premise Financial Planning & Analysis server.
  • You can subscribe to one or more agents and can have agents running in multiple on-premise servers.
  • Agents divide into two general types:

Sources and destinations

  • Most agents can act as both a data source and data destination.
  • Certain agents, such as file movers, can only be used in combination with other file movers.

  • The following table shows all combinations among file mover agents:

    Source Destination
     

    AWS S3

    Buckets

    		 SFTP

    Local

    File
    AWS S3 Buckets
    SFTP
    Local File

  • The following table shows all possible combinations of sources and destinations among all the non-file mover agents (the data source agents):

    Source Destination
      Excel MySQL

    ODBC/

    ODBC32

    		 PowerBI3

    SQL

    Server

    		 Text
    Acumatica GL  
    Dynamics 365 F&O  
    Excel  
    Financial Consolidation  
    MySQL
    Netsuite  
    ODBC/ODBC32  
    Sage Intacct  
    Script  
    SQL Server  
    Text  

Data-source agents

  • Data source agents, as the name suggests, work with data files
    • They open data files.
    • They process data files.

Bulk insert to destination through ODBC is not available to any source; therefore records are added through normal methods. As a result, ODBC performance on large data sets is slower.

ODBC is a generic layer that sits over any data source. Consequently, not all data source/data type combinations have ODBC drivers available. In these cases Prophix DIS makes every attempt to map data types, however some sources may have data types that cannot be mapped correctly, resulting in errors.

 Example:  The SQL Variant and DateTimeOffset data types are not supported by the ODBC driver.

 

File-mover agents

  • File mover agents, as the name suggests, stream files from one location to another:
    • They stream any file type—not only data files but even images, audio, video, and so on.
    • They never open the file.

To use a file-mover agent to get data into a database

 Example:  Your data is in an Excel file in AWS S3 and you want to get it into a MySQL database.

In this situation you can't open the file from the S3 stream directly, because it is a "file-mover" agent.

  1. Make the S3 data file local, using a Data Integration process:
    • For Source, select AWS S3 Bucket.
    • For Destination, select Local File.

  2. Define a connection to the Local File agent.

    • When DIS was installed on your server, your were given a Local File location; you can use the Local File agent as an intermediary between your file-mover agent and your data-source agent.
    • The connection type is either Excel or Text, depending on the type of file you are streaming from S3.
    • If, as in the example above, the data is in an Excel file in AWS S3 and your destination is a MySQL database, use an Excel connection.
  3. Create the second Data Integration process as follows:
    • For Source, select the Excel/Text connection.
    • For Destination, select MySQL.
  4. Pull the data into the database using a second Data Integration process.