JCache EntryProcessor: nice abstraction for rate limiting!

JCache entry processor is an API of JCache specification. Most of the API is close to a Map but it has one nice addition trick to execute some logic using the cache "lock": the EntryProcessor.

Why is it useful? If you need to be able to rely and mutate cache values it is the way to do it. Then question is why would I mutate cache values? Simply because you need to maintain a volatile state, kind of session.

Then why JCache? Because as a framework or product which is not bound to a deployment stack (can be hosted differently by several customers), JCache brings you the abstraction you need to support local or distributed volatile storage.

Side note: JCache is NOT about distributed caching (more about performances than clustering) but most providers support it, that's why previous statement is true.

In trendy/buzzy world you can think about microservices rate limiting implementation: each time a request is done you need to increment the current window counter to ensure you can execute the request. In JCache world it means you will update the currently cached value incrementing it. The nice thing about this setup is you have automatically with JCache an eviction you can align on window period. Therefore you don't need to handle it yourself! You get for free the state management and eviction! Powerful isn't it?

What does it look like?

First we'll create a key, just to show you can get a complex key and don't need a plain string I'll use that model:

import java.io.Serializable;

public class Key implements Serializable {
    private String id;
    private long version;

    public Key(final String id, final long version) {
        this.id = id;
        this.version = version;
    }

    public String getId() {
        return id;
    }

    public void setId(final String id) {
        this.id = id;
    }

    public long getVersion() {
        return version;
    }

    public void setVersion(final long version) {
        this.version = version;
    }

    @Override
    public boolean equals(final Object o) {
        if (this == o) return true;
        if (o == null || getClass() != o.getClass()) return false;

        final Key key = (Key) o;

        return version == key.version && (id != null ? id.equals(key.id) : key.id == null);
    }

    @Override
    public int hashCode() {
        int result = id != null ? id.hashCode() : 0;
        result = 31 * result + (int) (version ^ (version >>> 32));
        return result;
    }
}

So the key is composed of an identifier and version. Then we need a value, to keep things simple for this post we'll just modelize a counter holding an int but it can hold much more like some policies or rules to apply when the max is reached:

import java.io.Serializable;

public class Counter implements Serializable {
    private int value;
    // you can get other values


    public Counter(final int value) {
        this.value = value;
    }

    public int getValue() {
        return value;
    }

    public void setValue(final int value) {
        this.value = value;
    }

    @Override
    public String toString() {
        return "Counter{" +
                "value=" + value +
                '}';
    }
}

Finally we need the task incrementing the counter which will be our EntryProcessor:

import javax.cache.processor.EntryProcessor;
import javax.cache.processor.EntryProcessorException;
import javax.cache.processor.MutableEntry;

public class Increment implements EntryProcessor<Key, Counter, Counter> {
    public Counter process(final MutableEntry<Key, Counter> entry, final Object... arguments) throws EntryProcessorException {
        if (arguments.length > 1) {
            throw new IllegalArgumentException("Only boolean parameter to create the entry if missing is allowed");
        }

        if (entry.exists()) { // 1
            final Counter counter = entry.getValue();
            entry.setValue(new Counter(counter.getValue() + 1)); // 2
            return counter;
        }

        if (arguments.length == 1 && Boolean.class.cast(arguments[0])) { // 3
            return null;
        }
        // else create the entry
        entry.setValue(new Counter(1));
        return null;
    }
}

1. JCache API allows to check if there is a value or not for the related key
2. we can also update the value, in our case we increment the counter (you can also just increase the int in counter but this was to show you can completely rebuild the value or just create it if it was missing
3. you can also pass parameters to your tasks to reuse them

Then to invoke our processor we just call invoke on our cache:

cache.invoke(new Key("entry1", 1), new Increment(), true); // with parameter
cache.invoke(new Key("entry1", 1), new Increment());

To be complete you can get a cache with:

Cache<Key, Counter> cache = Caching.getCachingProvider()
   .getCacheManager()
   .createCache("test", new MutableConfiguration<Key, Counter>()
       .setTypes(Key.class, Counter.class));

or if you don't use a flat standalone classloader:

CachingProvider cachingProvider = Caching.getCachingProvider();
Cache<Key, Counter> cache = cachingProvider
        .getCacheManager(cachingProvider.getDefaultURI(), Thread.currentThread().getContextClassLoader())
        .createCache("test", new MutableConfiguration<Key, Counter>()
                .setTypes(Key.class, Counter.class));

Note: don't forget to close the manager and/or at least the provider once your app is over

What do we miss for a complete rate limiting? The window management. As mentionned you just need to set an expiration on the cache entries, this is done on the configuration. For a 5mn window you can use:

new MutableConfiguration<Key, Counter>()
    .setExpiryPolicyFactory(CreatedExpiryPolicy.factoryOf(Duration.FIVE_MINUTES))

And here we are, we just need to put the invoke in a servlet or JAX-RS filter and we have some nice rate limiting. Note that you often return some metadata as headers when doing so to say how long remains in the window and how many calls you can still do for instance. Since the processor can return any value (even the Counter in our case) it is the way to handle it: just return the metadata you need as returned value of the processor.

To rephrase the explanation of the beginning of the post, JCache can now - with Hazelcast, Ignite and Infinispan implementations - be used as an abstraction for clustered friendly states with eviction. Therefore it can be a nice alternative to Redis with a simpler infrastructure and no single point of failure :).