This page describes common best practices for ArcESB Flow design.
Begin at Entry and Exit Points
Most Flows have a connector at which data enters the Flow and a connector at which data exits the Flow. For some cyclical Flows, the entry and exit point may be the same connector.
For example, data may enter the Flow at an AS2 Connector when an external trading partner sends a business document over AS2. The data from this business document may need to be imported into a back-end system like a database. In this case, the AS2 Connector and the Database Connector are the entry and exit points for the Flow.
When creating a new Flow, it is often easiest to start with these entry and exit connectors. Then, “work inwards” by adding more connectors to the Flow that fill in the steps between the entry and exit point (e.g. add connectors that transform an inbound AS2 payload into a database insert).
File Transfer Connectors are Partner-Specific
Connectors that send or receive files over the network (e.g. AS2, AS4, FTP, SFTP, OFTP, etc) are bi-directional, but are configured for a single trading partner (i.e. a single remote party). In other words, a single AS2 Connector could both send and receive AS2 messages with Amazon, but it could not also send or receive files with Walmart.
Many MFT (Managed File Transfer) connectors require setting up a Profile to exchange files. The Profile page contains a tab for configuring application-wide settings for each supported protocol. For example, the information configured in the AS2 Profile is used in all AS2 Connectors, and partner-specific AS2 connection details are configured within each AS2 Connector.
Use a Naming Convention for Connectors
Connectors should always be given a descriptive name that communicates the role of the connector within the Flow.
- The name of a Managed File Transfer connector (e.g. AS2, AS4, FTP, SFTP, OFTP, etc) should indicate the remote party with whom files will be exchanged, and the protocol over which the exchange will occur. For example: Walmart_Production_AS2
- The name of an EDI connector (e.g. X12 and EDIFACT) should indicate the name of the partner with whom the EDI document is exchanged, the direction of translation, and the format of the document. For example: Walmart_X12_Outbound
- The name of a Data Mapping connector (e.g. XML Map) should indicate the format/schemas that the connector is mapping from and mapping to. For example: Map_850_to_Orders
- The name of a Back-End Integration connector (e.g. Database, MySQL, CData) should indicate the specific back-end system and the table within that back-end system (if applicable). For example: PostgreSQL_Orders_table
Use XML for Data Transformation
Arc uses XML as an intermediary format for data manipulation and transformation. Many connectors translate files of various formats into XML and vice versa (e.g. the X12 Connector, EDIFACT Connector, CSV Connector etc). Additionally, Arc uses XML to model back-end system input and output (e.g. SELECTS and INSERTS into database-like systems).
Since most source and destination formats can be represented as XML, data transformation Flows often have the following structure:
- A connector that pulls or receives data in its original (source) format
- A connector that converts this source format into XML (if it is not already XML)
- An XML Map Connector that transforms this source XML into a new structure representing the target (destination) format
- A connector that converts this transformed XML into the destination format (if a non-XML format is required)
- A connector that sends the finalized file to its intended destination
The XML Map Connector is the engine that performs the critical work in these data transformation Flows. Becoming comfortable with data transformation is often a matter of familiarizing oneself with the XML Map Connector’s visual designer.
Organize Flows into Workspaces
Multiple separate Flows can be configured within the same Flows canvas without risk of cross-contamination, but dividing Flows into different Workspaces can help reduce clutter and maintain organization.
The cogwheel in the top-right of the Flows page can be used to create new workspaces, import or export workspace configuration settings, and delete the current workspace (except for the Default workspace). Only the connectors from the current workspace are displayed at any given time.
Connectors in different workspaces cannot be connected to each other, and the data and configuration files specific to these connectors are held in separate folder structures.