Migrating files from local machine to Salesforce Instance as Attachment

Client

A USA based consulting group having offices in Cary, North Carolina which provides variety of services and products that solve the key needs of the entrepreneurs. These needs include: information technology, sales methodology, marketing, organizational development, process improvement and general business strategy. The customer also builds integration App to integrate Salesforce with external system.

Challenges

Client needs were to migrate files from local machine into Salesforce instance as attachment in different object and wanted to map Parent ID, OwnerId dynamically from an external CSV file based on the matched file name.

CSV file contains fileName, AccountID, ContactID, Opportunity ID, fileSize and OwnerId columns.

CSV file contains multiple null values in Account Id, Contact Id and Opportunity ID column and an identifier letter for each object. for e.g. A for Account.

Client wants to migrate more than 470K files from local machine to salesforce which were about 160 GB in size.

Frequency of the job to migrate filed form local machine to salesforce was One Time.

Wants to use excel sheet as database for mapping of Parent Id and Owner Id of the attachments record.

Files were uploaded as attachment in Account, Contact, Opportunity object.

Mapping Parent Id and Owner Id for the attachment was dynamic from Excel sheet based on the file name.

Solutions

Astrea It services have come up with a solution for this requirement with the help of Talend Tool. Astrea has used Talend Open Source Data Integration Tool which is free to use to setup the migration Job. Job will be responsible for pulling the files from local machine and pushing the same as attachment on SFDC Objects. Job also have the error handling and reporting capabilities.

Talend is an Open Source Data Integration Tool which provides the connectors for many external systems.

Results

Configured a talend job which fetches the files from Local Machine Directory and then upload files into salesforce instance as attachment.

Added filter to process only those files from local machine which filename matches with the filename in CSV file.

Added filter for Owner ID.

Added filters to avoid null values and Parent Id column.

Were Sending the Success/Error logs through Email functionality with the details of Success and Error and no of files uploaded/failed into salesforce.

So instead of spending time on custom coding as client wants to run the job only one time, we used Talend Open Source Data Integration Tool. Maintenance is easy in Talend if new changes comes in future.