Amazon Kinesis vs Amazon SQS

The main difference between SQS and Kinesis is that the first is a FIFO queue, whereas the latter is a real time stream that allows processing data posted with minimal delay.

With that been said let us examine the cases. An online web shop and an Order Management System is a case where sensitive data is transferred, for example the order information. To the problem of securing the information sent by the web shop, and ensuring that it is processed from the OMS, a decoupled queue like SQS provides a solid solution. Even in down time the order information would be stored, until requested, in SQS, and later processed by the system when it is restored, by periodically checking for new insertions in the queue.

After creating your SQS Queue you can follow the code below to simply add and receive a message:

var AWS = require('aws-sdk');
AWS.config.loadFromPath(__dirname+'/config.json');
var sqs = new AWS.SQS();

// Adding a message to Queue
function addToSQS(){
  return new Promise(function(resolve,reject){

    // Set up the order as parameters for adding to queue
    var params = {
      DelaySeconds: 10,
      MessageBody: JSON.stringify(myData),
      QueueUrl: "the_Url_to_My_SQS_queue"
    };

    sqs.sendMessage(params, function(error, data) {
      if (error) {
        console.log("Error", error);
        reject(error)
      } else {
        resolve(data.MessageId);
      }
    });
  });
}

// Receive a message
function lookUpSQS() {
  var params = {
    QueueUrl: queueURL,
    VisibilityTimeout: 0,
    WaitTimeSeconds: 0
  };

  sqs.receiveMessage(params, function(err, data) {
    if (err) {
      console.log("Receive Error", err);
      return;
    } else {
      // if empty return
      if (data.Messages==undefined){
       return;
      }

      // Handle your messages 
      for (var i = 0; i < data.Messages.length; i++) {
        ...
      }
    }
  });
}

Notifying third party systems of the new incoming order would also be a responsibility of our OMS. In this case, storing in SQS and later processing and sending the information to these third party systems is not an option. Real time processing is achieved by passing data in a Kinesis stream. Amazon includes in its services the Lambda functions. A Lambda function is an event driven highly scalable code, which perfectly fits the real time nature of a stream.

After having created the Kinesis stream and the Lambda function, configured to receive events from Kinesis, adding Data to the stream is done by pushing "Records" to it. An example of sending a single Record and handling the generated event is the following:

// Insert to kinesis
var AWS = require('aws-sdk');
AWS.config.loadFromPath('./config.json');
var kinesis = new AWS.Kinesis();

function addToKinesis(){
  var params = {
    Data: JSON.stringify(myData),
    PartitionKey: "my_partition_key",
    StreamName:"my_stream_name"
  };

  kinesis.putRecord(params, function(error, data){
    if(error){
      throw(new Error(error))
    } else {
      return;
    }
  });
}

// Handling the event in Lambda
exports.handler = (event, context, callback) => {
  const output = event.Records.map((record) => {

    // Kinesis data is base64 encoded so decode here
    var payload = new Buffer(record.kinesis.data, 'base64').toString('ascii');
    payload = JSON.parse(payload);

    // your payload data handling
  });

  callback(null, { records: output });
};

After taking into consideration the cases I mentioned above, SQS is an option that provides high scalability and reliability to your application. But if real time data processing is needed for your message queue, then I would suggest an event driven architecture based on Kinesis.