tl;dr use Timescale (open source extensions to postgres) – All the power, expressivity and ease of SQL w/horizontal scalability via partitioning. Like your own, Amazon RedShift. Faster and obvs. much cheaper than RedShit.
Other options:
- InfluxDb (and cloud) – simplicity of ingest
- ElasticSearch – needs a custom browse component (Kibana too complex)
- Clickhouse, Vertica, Pilosa and others – “exotic” options, not really worth the effort, esp. given how good TimescaleDb is.
How to ingest:
- v0: statsD -> influx (fast, cheap, quick, scalable)
- v2 rsyslog (on each server) => central rsyslog => beanstalkd / kafka => (fan out) postgres (timescaledb) + mongo + influx + es