Running Apache Storm in production requires increasing the nofile and nproc default ulimits.

There is a lot of documentation around Apache Storm and Apache HBase but not so much about how to use the hbase-client inside of storm. In this post, I’ll outline:

1. Information about my dev environment
2. Setting up your Storm project to use the HBase client
3. Managing connections to HBase in Storm
4. Reading one row (Get)
5. Reading many rows (Scan)
6. Writing one row (Put)
7. Writing many rows in a batch of Puts

Please note, this post assumes you already are comfortable with Storm and HBase terminology. If you are just starting out with Storm, take a look at my example project on GitHub: storm-stlhug-demo.

Also, an option to consider when writing to HBase from storm is storm-hbase and it is a great way to start streaming data into hbase. However, if you need to write to multiple tables or get into more advanced scenarios you will need to understand how to write your own HBase bolts.

Logging within storm uses Simple Logging Facade for Java (SLF4J).

Tick tuples are useful when you want to execute some logic within your bolts on a schedule. For example, refreshing a cache every 5 minutes.

Within your bolt, first enable receiving the tick tuples and configure how often you want to receive them:

@Override
public Map<String, Object> getComponentConfiguration() {
    // configure how often a tick tuple will be sent to our bolt
    Config conf = new Config();
    conf.put(Config.TOPOLOGY_TICK_TUPLE_FREQ_SECS, 300);
    return conf;
}

Next create the isTickTuple helper method to determine whether or not we’ve received a tick tuple: