In this video, you will learn how to extract records from Salesforce and write them into Amazon SQS
Read full transcript
Hi! In this video, I will explain how to extract records from Salesforce and write them to Amazon SQS queues in two ways—using the JMS Producer Snap and using the SQS Producer Snap.
First, I need to create a new Pipeline file;
Then we drag the Salesforce Read Snap, select the account from which I want to read records, and then click Save;
Configure it to read ‘Opportunity’ records in the Salesforce Read Snap Settings
Next, I add a Mapper Snap to convert the schema later.
I will use a Copy Snap to create duplicate records to write these records to a queue in two ways
Then I add the Document to Binary Snaps to convert the data from Document format to Binary format before writing it to the queues.
I open the Mapper Snap settings and see that the Input Schema has been populated based on Salesforce Read output and the Target Schema has been populated based on the input format required by the ‘Document to Binary’ Snap.
I convert the whole incoming document into a String and assign it to the content field and assign the Id to the Content-disposition field;
Now I go to the AWS platform and see an empty queue called ‘SFDC_Opportunities1’ and the region is us-east-1.
Now moving back to SnapLogic, a JMS Producer Snap is added and I select the JMS Account that has permissions to write to the queue.
We open the settings and enter the Queue name. We need to ensure that QUEUE is selected as the Destination Type and TEXT is selected as the Message Type;
Add a Message Property called ‘Record ID’ and specify the Id of the incoming record as its value, and prepend ‘JMS Producer’ to identify the source.
Then I add an SQS Producer Snap and configure settings like we did with the JMS Snap
Then I select us-east-1 as the region; then select the name of the Queue. This is one of the advantages of using SQS producer Snaps when compared to JMS Producer. This Snap provides suggestions whereas we had to manually specify the name in case of JMS Producer;
Then we add a ‘Record ID’ like we did in the previous case. However, this time we prepend ‘SQS Producer’.
I will execute the completed and validated pipeline.
I open Pipeline Execution Statistics. Observe that 51 records were processed by each Producer Snaps. Observe that the JMS Producer consumes more time and resources when compared to the SQS Producer Snap.
Now we return to the AWS Console and refresh the page.
Notice that 102 records have been written to the queue.
I demonstrated how SnapLogic makes it very easy to extract records from endpoints and write them to Queues.
Thank you for watching this demo! Visit docs-snaplogic.atlassian.net to learn about more new features.
The AI Mindset: Getting Started with Self-Service Machine Learning