sqs2kinesis configuration reference

This is a complete list of the options that can be configured in the sqs2kinesis HOCON config file. The example configs in github show how to prepare an input file.

Common options

input.queue	Required. The url of the SQS topic from which to read collector payloads.
output.good.streamName	Required. The output kinesis stream in which to write successfully parsed messages.
output.bad.streamName	Required. The output kinesis stream in which to write messages that could not be base64-decoded.
monitoring.sentry.dsn	Optional, for tracking uncaught run time exceptions
monitoring.health.host	Optional, default 0.0.0.0, sets the host name the health check endpoint listens on
monitoring.health.port	Optional, default 8080, sets the port the health check endpoint listens on

Advanced options

We believe these advanced options are set to sensible defaults, and hopefully you won't need to ever change them.

input.kinesisKey	Default kinesisKey (for compatibility with snowplow collector). Sets the sqs message attribute that will be used to set the kinesis partition key
input.minBackoff	Default 500 millis. Minimum backoff before retrying after sqs failure.
input.maxBackoff	Default 5 seconds. Maximum backoff before retrying after sqs failure.
input.randomFactor	Default 0.1. Random factor when calculating backoff time after sqs failure.
input.maxRetries	Default 5. Maximum number of retries after failure to ack (delete) a sqs message.
input.maxRetriesWithin	Default 1 minute. Duration for which input.maxRetries are counted, before exiting with failure.
output.good.maxKinesisBytesPerRequest	Default 5000000 (5 MB). The kinesis PutRecordsRequest will not exceed this size.
output.good.maxKinesisBatch	Default 500. The kinesis PutRecordsRequest will have no greater than this many records
output.good.keepAlive	Default 1 second. Maximum time to wait before sending an incomplete PutRecordsRequst
output.good.minBackoff	Default 500 millis. Minimum backoff before retrying after failure
output.good.maxBackoff	Default 1 second. Maximum backoff before retrying after failure
output.good.randomFactor	Default 0.1. Random factor used when calculating backoff time after failure.
output.good.maxRetries	Default 5. Maximum number of retries after failure.
output.bad.maxKinesisBytesPerRequest	Default 5000000 (5 MB). The kinesis PutRecordsRequest will not exceed this size.
output.bad.maxKinesisBatch	Default 500. The kinesis PutRecordsRequest will have no greater than this many records
output.bad.keepAlive	Default 1 second. Maximum time to wait before sending an incomplete PutRecordsRequst
output.bad.minBackoff	Default 500 millis. Minimum backoff before retrying after failure
output.bad.maxBackoff	Default 1 second. Maximum backoff before retrying after failure
output.bad.randomFactor	Default 0.1. Random factor used when calculating backoff time after failure.
output.bad.maxRetries	Default 5. Maximum number of retries after failure.
akka.*	Set any standard akka streams option. For example, akka.loglevel = INFO