tl;dr use Timescale (open source extensions to postgres) – All the power, expressivity and ease of SQL w/horizontal scalability via partitioning. Like your own, Amazon RedShift. Faster and obvs. much cheaper than RedShit.

Other options:

  • InfluxDb (and cloud) – simplicity of ingest
  • ElasticSearch – needs a custom browse component (Kibana too complex)
  • Clickhouse, Vertica, Pilosa and others – “exotic” options, not really worth the effort, esp. given how good TimescaleDb is.

How to ingest:

  • v0: statsD -> influx (fast, cheap, quick, scalable)
  • v2 rsyslog (on each server) => central rsyslog => beanstalkd / kafka => (fan out) postgres (timescaledb) + mongo + influx + es