Integrations

Once you've created uni, the next step is to hook it up to the rest of your application. Vendia provides a large array of options for integrating your uni with other cloud vendor services, APIs from any vendor, and with your web and mobile clients.

Web, Mobile, and API-based Access

Every Vendia uni comes with built-in support for API-based web, mobile, and cloud integration. Vendia offers a GraphQL-based, HTTPS protocol API that supports your custom data model with queries and updates ("mutations" in GraphQL parlance).

The API key and URL required to interact with this service are included in the resource returned from share get <uni_name>, named graphqlurl and graphqlapikey. A simple example of using this to list a top-level, array-based data type in your model:

curl -XPOST <the client url> \
    -H "Content-Type:application/graphql" \
    -H "x-api-key:<the client api key>" \
    -d "{\"query\": \"query q {listMileageGeos{MileageGeos{vin mileage timestamp position}}}\"}"

Replace <the client url> and <the client api key> in the code fragment above with your uni's resources.

Inbound Service Connectors

Vendia Share supports several additional inbound ("ingress") options beyond GraphQL mutations:

  • Lambda functions and AWS Event Hub. You can easily use AWS Lambda functions to convert any of the dozens of AWS-provided events into GraphQL mutations, including events from S3, Aurora, and many other AWS services into transactions and then submit them to the Vendia-provided SQS queue for processing.

  • SQS queues. Vendia automatically creates an inbound SQS queue to hold pending transactions. Vendia recommends using the GraphQL endpoint when possible because it is often easier and adds a layer of type safety; however, submitting transactions directly to SQS is supported.

  • SNS topics. You can connect an SNS topic to the Vendia-provided SQS queue for automatic ingress via SNS.

Outbound

Uni nodes can be configured to emit events for the following subscriber types:

  • HTTPS webhooks
  • AWS Lambda functions
  • AWS SQS queues
  • Email addresses

Vendia supports the following notification types:

  • Real-time block notifications: Emitted when the node commits a new block. Includes a summary of the transactions included in the block.
  • Dead-letter notifications: Emitted when an asynchronous transaction cannot be committed within the retry policy. Includes the full details of the original transaction.

Configuring a Lambda function subscriber

You can use a Lambda-based subscription to implement custom logic or to emit notifications to other AWS services, such as streaming data services (Amazon Kinesis Streams, Amazon Kinesis Firehose, and Amazon Kafka).

Detailed instructions for configuring Lambda notifications

NOTE The directions below use the AWS Command Line Interface.

NOTE The AWS Lambda function must be deployed to the same AWS region as your node.

  • Determine the ARN of the SNS topic provisioned with your Vendia node.

The ARN is visible from your node's settings page in the web interface. For block notifications, use the topic labelled "Block Notification SNS Topic ARN". For dead-letter notifications, use the topic labelled "Dead-letter SNS Topic ARN".

The ARN will look something like arn:aws:sns:aws-region:your-node-aws-account-number:node-sns-topic-name

  • Create a Lambda function in your AWS account if one does not already exist. For the sake of this example the function will be referred to as lambda-block-function-in-your-account.

  • Attach a Resource-based Policy to allow your node's SNS topic to invoke the Lambda function in your account.

{
  "Version": "2012-10-17",
  "Id": "default",
  "Statement": [
    {
      "Sid": "lambda-block-notification",
      "Effect": "Allow",
      "Principal": {
        "Service": "sns.amazonaws.com"
      },
      "Action": "lambda:InvokeFunction",
      "Resource": "arn:aws:lambda:aws-region:your-aws-account-number:function:lambda-block-function-in-your-account",
      "Condition": {
        "ArnLike": {
          "AWS:SourceArn": "arn:aws:sns:aws-region:your-node-aws-account-number:node-sns-topic-name"
        }
      }
    }
  ]
}
  • Query and record the existing node settings

View existing node settings. When we update settings we will overwrite any existing settings so take care to record any present values.

query getSettings {
  get_Settings {
    aws_DataDogMonitoring {
      ddApiKey
      ddExternalId
      ddLogEndpoint
      ddSendLogs
    }
    aws_LambdaIngressAccounts
    aws_S3ReadAccounts
    aws_SQSIngressAccounts
    aws_blockReportFirehoses
    aws_blockReportLambdas
    aws_blockReportSQSQueues
    blockReportEmails
    blockReportWebhooks
    aws_deadLetterLambdas
    aws_deadLetterSQSQueues
    deadLetterEmails
    deadLetterWebhooks
    graphqlAuth {
      authorizerArn
      authorizerType
    }
  }
}
  • Update the node setting to subscribe the Lambda function

Run the following GraphQL mutation, taking care to use your AWS Lambda function ARN as appropriate.

For block notifications, use the aws_blockReportLambdas setting. For dead-letter notifications, use the aws_deadLetterLambdas setting.

NOTE: Adjust the query to accommodate any additional settings you already have applied.

mutation update_blockReportLambdas {
  update_Settings_async(input: {
        aws_blockReportLambdas: [
            "arn:aws:lambda:aws-region:your-aws-account-number:function:lambda-block-function-in-your-account"
        ],
        aws_deadLetterLambdas: [
            "arn:aws:lambda:aws-region:your-aws-account-number:function:lambda-block-function-in-your-account"
        ]
    }) {
        error
        result {
          id
          tx_id
        }
  }
}
  • Subscribe the Lambda function in your account to your node's SNS topic.
aws sns subscribe --protocol lambda \
--topic-arn arn:aws:sns:aws-region:your-node-aws-account-number:node-sns-topic-name \
--notification-endpoint arn:aws:lambda:aws-region:your-aws-account-number:function:lambda-block-function-in-your-account \
--region aws-region # eg us-east-1, eu-west-2, etc.  Should match Topic and Function region.
  • Run a sample data mutation

Validate a block notification message has been sent to the Lambda function. There should be a full block report available.

For more information on sending Amazon SNS messages to a AWS Lambda function in a different account, please refer to AWS documentation.

Configuring an SQS queue subscriber

Vendia can emit block production and dead-letter events to an SQS queue. The queue is not strictly ordered, so consumers must rely on the block number to serialize events if they are not consumed in real time.

Detailed instructions for configuring SQS Notifications
  • Determine the ARN of the SNS topic provisioned with your Vendia node.

The ARN is visible from your node's settings page in the web interface. For block notifications, use the topic labelled "Block Notification SNS Topic ARN". For dead-letter notifications, use the topic labelled "Dead-letter SNS Topic ARN".

The ARN will look something like arn:aws:sns:aws-region:your-node-aws-account-number:node-sns-topic-name

Make note of the account number embedded in the ARN as well (ie your-node-aws-account-number). We will need both to allow your node's SNS topic to publish to your SQS queue.

  • Create a SQS queue in your AWS account if one does not already exist.

  • Update the SQS queue's Access Policy to ensure the queue can receive both the subscription confirmation message along with subsequent notifications generated by your node's SNS topic.

The policy need to be updated to look something like this:

{
  "Version": "2008-10-17",
  "Id": "__default_policy_ID",
  "Statement": [
    {
      "Sid": "__owner_statement",
      "Effect": "Allow",
      "Principal": {
        "AWS": "your-aws-account-number"
      },
      "Action": [
        "SQS:*"
      ],
      "Resource": "arn:aws:sqs:aws-region:your-aws-account-number:your-queue-receiving-notifications"
    },
    {
      "Sid": "SendMessagesFromMyNode",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::your-node-aws-account-number:root"
      },
      "Action": "SQS:SendMessage",
      "Resource": "arn:aws:sqs:aws-region:your-aws-account-number:your-queue-receiving-notifications"
    },
    {
      "Sid": "AllowSubscriptionConfirmationMessage",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "SQS:SendMessage",
      "Resource": "arn:aws:sqs:aws-region:your-aws-account-number:your-queue-receiving-notifications",
      "Condition": {
        "ArnLike": {
          "aws:SourceArn": "arn:aws:sns:aws-region:your-node-aws-account-number:node-sns-topic-name"
        }
      }
    }
  ]
}
  • Query and record the existing node settings

View existing node settings. When we update settings we will overwrite any existing settings so take care to record any present values.

query getSettings {
  get_Settings {
    aws_DataDogMonitoring {
      ddApiKey
      ddExternalId
      ddLogEndpoint
      ddSendLogs
    }
    aws_LambdaIngressAccounts
    aws_S3ReadAccounts
    aws_SQSIngressAccounts
    aws_blockReportFirehoses
    aws_blockReportLambdas
    aws_blockReportSQSQueues
    blockReportEmails
    blockReportWebhooks
    aws_deadLetterLambdas
    aws_deadLetterSQSQueues
    deadLetterEmails
    deadLetterWebhooks
    graphqlAuth {
      authorizerArn
      authorizerType
    }
  }
}
  • Update the node setting

For block notifications, use the aws_blockReportSQSQueues setting. For dead-letter notifications, use the aws_deadLetterSQSQueues setting.

NOTE: Adjust the query to accommodate any additional settings you already have applied.

Run a GraphQL mutation to subscribe your SQS AWS SQS ARN as appropriate. i.e.

mutation update_blockReportSQSQueues {
  update_Settings_async(input: {
        aws_blockReportSQSQueues: [
            "arn:aws:sqs:aws-region:your-aws-account-number:your-queue-receiving-notifications"
        ],
        aws_deadLetterSQSQueues: [
            "arn:aws:sqs:aws-region:your-aws-account-number:your-dead-letter-queue-receiving-notifications"
        ]
    }) {
        error
  }
}
  • Confirm the SNS subscription

Go to the settings page of your SQS queue. You should see a message available. Poll for messages and open it. The message sent from your Vendia node's SNS topic will contain a SubscribeURL attribute. Copy and paste the value into a web browser to confirm the subscription.

NOTE None of the node's block reports will be sent to the SQS queue until the subscription is confirmed.

  • Run a sample data mutation

Validate a block notification message has been sent to the SQS queue. There should be a full block report available.

For more information on sending Amazon SNS messages to an Amazon SQS queue in a different account, please refer to AWS documentation.

Configuring a webhook subscriber

Notifications can be emitted to webhook URLs via the blockReportWebhooks and deadLetterWebhooks settings.

mutation subscribeWebhooks {
  update_Settings_async(input: {
    blockReportWebhooks: ["https://my.webhook.url"], 
    deadLetterWebhooks: ["https://my.webhook.url"]})
}

Configuring an email subscriber

Vendia provides email notifications for block creation and dead-letter events via the blockReportEmails and deadLetterEmails settings. This is best for development, low-volume tests, and integration with legacy systems that do not support webhooks or the other methods above.

i.e.

mutation subscribeEmails {
  update_Settings_async(input: {
    blockReportEmails: ["blockreports@companyxyz.com"], 
    deadLetterEmails: ["deadletter@companyxyz.com"]})
}

Next Steps

Developing and Using Unis

Defining Your Data Model

Learning More

Terms and Definitions

Show table of contents
Edit this page