Over the years I got to work with a lot of tech. While some of it never stopped feeling like work, a few awesome tools earned a place in my heart. So please enjoy my semi-sorted, semi-structured list of things I defaulted on using more than once.
Web Analytics Tools
- Adobe Analytics: This is the obvious one. Not only have I spent most of my Analytics-workdays with AA, but it quickly became my reference point for evaluating a mature, feature complete enterprise analytics tool. It consistently applies a well documented set of logical rules to every existing and newly developed feature. Speaking of new features, those come with every monthly release and keep expanding on the incredible value it delivers. With Analysis Workspace, Adobe managed to build an environment that accommodates both expert analysts and non-technical business folk. On top of that, you can build awesome automated environments to deliver your data in every way possible, no matter if as Excel files, raw data feeds or via APIs.
- Piwik (nowadays known as Matomo): Whenever Adobe Analytics is not an option, I use Piwik (and will continue to call it that). When you know your ways around programming and databases, you can build some cool things with Piwik, because it is published as open source. The greatest appeal for me is that you have the most direct way to access your data by using your normal database. Almost everything you can find in Google Analytics is implemented or hackable in Piwik, which is why I prefer it whenever AA is not an option.
The Elastic Stack
- Elasticsearch: There are two “standard” use cases for Elasticsearch: Indexing files to make them available for real time search (like Ebay does it) and log file analysis. But it has even more to offer! You can feed it almost any kind of data (structured JSON, BLOB files, text, etc.) and do whatever the data types allows (search, word count, math, statistics, etc.) The only thing it isn’t very good at is relational data, which can be tackled by denormalizing data at the expense of some storage overhead. I use it as storage backend whenever there is uncertainty about future requirements for the systems I build. It is also simple to tune it for any of the CAP dimensions.
- Kibana: A few years back, Kibana was mainly used to visualize data stored in Elasticsearch. This is also what it is useful for today, whenever you want to take a quick look at your data or put up some simple graphs or even dashboards. Canvas is the newest addition to that aspect, though there are still some rough edges there. On top of that, Kibana has grown to be a visual administrative interface to your Elasticsearch cluster, allowing you to easily edit settings and perform some maintenance tasks.
- Logstash: Over the years, Logstash provided a very flexible way to get your data into Elasticsearch (or other tools, if you like). With recent advancements, it can be monitored in Kibana and administered if you pay for the enterprise version of ES.
- Beats (Metricbeat, Filebeat, etc.): Beats are comparably new to the stack (and not part of the beloved ELK acronym) and offer an easy way to get some basic functionality running in your stack. Metricbeat ships performance data to ES, Filebeat can tail logfiles, etc. That way I don’t need to rely on the IT department for monitoring my stack 😉
Other tools
- Grafana: My go-to tool for the “nerdy” type of visualization. Lets you build awesome real time applications and supports a load of data sources. With it’s responsive design, it works great on mobile devices as well. Most of the time I’ve been using it with Elasticsearch but also combined it with some RDBMS’ to get an overview across different data sources.
- NiFi: My favorite for building data flows between systems today. It has really grown close to me over the last couple of years. It allows to build visually understandable pipelines without the need to write it in YAML. It can talk to any system in one way or another and can be scaled in every direction.
- Power BI: Whenever Grafana is too nerdy for the audience, Power BI has proven to be a nice way to get your business folk to look at dashboards and data, be it in the browser or on the app.