Apache Storm

Apache Storm is a distributed stream processing computation framework written predominantly in the Clojure programming language. A Storm application is designed as a “topology” in the shape of a directed acyclic graph (DAG). Edges on the graph are named streams and direct data from one node to another. Together, the topology acts as a data transformation pipeline. Storm topologies run indefinitely until “killed”. Storm uses Zookeeper to manage its processes. Storm can read and write files to HDFS.

Architecture: Storm processes the events one at a time.

Performance: millisecond latency.

Advantages: complete stream processing engine with micro-batching support.

Concerns: supports only streaming data, not integrated with Azure platform.

%d bloggers like this: