The SCP Connector transfers files to and from an SSH server via SCP.
Each SCP Connector connects uploads to a specified folder and downloads from a set of folders on the remote SSH server.
Files that reach the SCP Connector in the ArcESB flow are uploaded to the upload target folder, and files found in the download target folder(s) are downloaded and entered into the ArcESB flow. The connector supports upload and download filters to only send and receive specified file names and/or file extensions.
This section contains all of the configurable connector properties.
Settings related to establishing the connection to the remote SSH host.
- Connector Id The static name of the connector. All connector-specific files are held in a folder by the same name within the Data Directory.
- Connector Description An optional field to provide free-form description of the connector and its role in the flow.
- Remote Host The hostname or IP address of the SSH server.
- Remote Port The port on which to connect to the SSH server.
Settings related to authenticating to the SSH server.
- Authentication Mode The type of authentication to use with the SSH server.
- User The username credential for logging in to the SSH server.
- Password The password credential for logging in to the SSH server.
- Client Certificate The certificate with a private key to use during public key authentication.
- Certificate Password The password required to access the Client Certificate.
Settings related to verifying the SSH server’s identity.
- Server Thumbprint The thumbprint of the certificate that identifies the SSH server. This can also be set to a certificate file containing the server’s public key. When testing the connection, the connector will prompt to accept/trust the thumbprint presented by the server.
Settings related to uploading files to the SSH server.
- Remote Path Files processed by the connector will be uploaded to this specified remote folder.
Settings related to downloading from the SSH server.
- Remote Path The remote folder from which files will be downloaded. Multiple paths can be specified in a comma-delimited list.
- File Mask A glob pattern that determines which files within the Remote Path should be downloaded (e.g. *.txt).
- Overwrite local files Whether files that already exist on the local file system should be downloaded (overwritten) or skipped.
- Delete files (after received) Whether files that are successfully downloaded should be deleted from the SSH server afterwards.
Settings related to the automatic processing of files by the connector.
- Upload Whether files arriving at the connector will automatically be uploaded.
- Retry Interval The amount of time before a failed upload is retried.
- Retry Maximum Attempts The maximum number of times a failed upload will be retried.
- Download Whether the connector should automatically poll the remote download path(s) for files to download.
- Download Interval The interval between automatic download attempts.
- Minutes The number of minutes to wait before downloading. Only applicable when Download Interval is set to Minute.
- Minutes Past the Hour The minutes offset for an hourly schedule. Only applicable when Download Interval is set to Hourly. For example, if this value is set to 5, the automation service will download at 1:05, 2:05, 3:05, etc.
- Time The time within a given day that the download should occur. Only applicable when Download Interval is set to Daily, or Weekly, or Monthly.
- Day The day on which the download should occur. Only applicable when Download Interval is set to Weekly or Monthly.
- Cron Expression An arbitrary string representing a cron expression that determines when the download should occur. Only applicable when Download Interval is set to Advanced.
Settings that determine the folder on disk that files will be sent/uploaded from, and the folder that they will be received/downloaded to.
- Input Folder (Send) The connector can send/upload files placed in this folder. If Send Automation is enabled, the connector will automatically poll this location for files to process.
- Output Folder (Receive) The connector will place received/downloaded files in this folder. If the connector is connected to another connector in the flow, files will not remain here and will instead be passed along to the Input/Send folder for the connected connector.
- Processed Folder (Sent) After processing a file, the connector will place a copy of sent/uploaded files in this folder if Save to Sent Folder is enabled.
Settings related to the allocation of resources to the connector.
- Max Workers The maximum number of worker threads that will be consumed from the threadpool to process files on this connector. If set, overrides the default setting from the Profile tab.
- Max Files The maximum number of files that will be processed by the connector each time worker threads are assigned to the connector. If set, overrides the default setting from the Profile tab.
Settings not included in the previous categories.
- Delete Command The command to use when deleting files after download.
- Send Filter A glob pattern filter to determine which files in the Send folder will be uploaded by the connector (e.g. *.txt). Negative patterns may be used to indicate files that should not be uploaded (e.g. -*.tmp). Multiple patterns may be separated by commas, with later filters taking priority except when an exact match is found.
- Local File Scheme A filemask for determining local file names as they are downloaded by the connector. The following macros may be used to reference contextual information:
%ConnectorId%, %Filename%, %FilenameNoExt%, %Ext%, %ShortDate%, %LongDate%, %RegexFilename:%, %DateFormat:%.
As an example: %FilenameNoExt%_%ShortDate%%Ext%
- Log Level The verbosity of logs generated by the connector. When requesting support, it is recommended to set this to Debug.
- Recurse Subdirectories Whether to download files in subfolders of the target remote path.
- SSH Encryption Algorithms A comma-delimited list of algorithms to use during SSH encryption. By default, all algorithms are enabled. The following algorithms are supported:
aes256-ctr, aes256-cbc, aes192-ctr, aes192-cbc, aes128-ctr, aes128-cbc, 3des-ctr, 3des-cbc, cast128-cbc, blowfish-cbc, arcfour, arcfour128, arcfour256, email@example.com, firstname.lastname@example.org, email@example.com
- Parent Connector The connector from which settings should be inherited, unless explicitly overwritten within the existing connector configuration. Must be set to a connector of the same type as the current connector.
- Upload File Permissions The file permissions to set on files uploaded by the connector.
- Timeout The duration the connector will wait for a connection response before throwing a timeout error.
- Log Subfolder Scheme By default, logs for transactions processed by the connector will be stored in the Logs subfolder for the connector. For connectors that process many transactions, it may be desirable to further divide the logs based on the datetime they were generated. When this setting is set to Daily, logs generated on the same day will be grouped in a subfolder;; when this setting is set to Weekly, logs generated in the same week will be grouped in a subfolder; and so on.
- Log Messages Whether the log entry for a processed file will include a copy of the file itself.
- Save to Sent Folder Whether files processed by the connector should be copied to the Sent folder for the connector.
Establishing a Connection
The following settings are required to connect to a remote SSH server:
- Remote Host
- Remote Port
- Authentication Mode
- Password or Client Certificate/Certificate Password
- Server Thumbprint
When testing the connection to the server, the connector will prompt to accept/trust the thumbprint of the server’s public key.
The SCP Connector uploads files from the Input/Send Folder that match the Send Filter set in the Advanced tab. Files are automatically uploaded to the SSH server if Upload Automation is enabled in the connector.
Remote Path should be set to the folder on the remote SSH server where files should be uploaded. The application changes directories to the specified folder after logging in to the server.
If any files could not be uploaded, the connector attempts to send them later after Retry Interval minutes have elapsed. This process continues until the Retry Maximum Attempts setting has been reached, after which the connector will raise an error and the file will no longer be sent automatically.
Downloaded files are displayed in the Output/Receive Folder of the connector, or passed along to the next connector in the flow. Files are automatically downloaded from the SSH server if Download Automation is enabled in the connector, and can be manually downloaded by clicking the Receive button in the Output tab.
Remote Path should be set to the folder on the remote SSH server from which files should be downloaded. The application changes directories to the specified folder after logging in to the server. The File Mask property filters files such that only filenames matching the glob pattern are downloaded from the server.
The Overwrite Local Files setting governs whether files that already exist in the Output/Receive Folder should be overwritten or not. The Delete files (after download) setting can be used to delete the files on the remote server after successfully downloading them. Within the Output tab, click the Receive to test these settings and retrieve the matching files from the server.