Throttle methods with Spring AOP and Guava rate limiter

Tapio RautonenJuly 9th, 2015Last Updated: July 9th, 2015

2 775 4 minutes read

External services or APIs may have usage limits or they just cannot handle loads of requests without failing. This post explains how to create a Spring Framework based aspect that can be used to throttle any adviced method calls with Guava’s rate limiter. The following implementation requires Java 8, Spring AOP and Guava.

Let’s start with an annotation that is used to advice any Spring AOP enabled method call.

@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface RateLimit {
 
    /**
     * @return rate limit in queries per second
     */
    int value();
 
    /**
     * @return rate limiter identifier (optional)
     */
    String key() default "";
 
}

The annotation defines two things: the rate limit as in queries (or permits) per second and an optional key to identify a rate limiter. Multiple methods can use the same rate limiter if the keys are equal. For example when an API is called with different parameters from different methods the desired total queries per second will not exceed.

Next thing is the actual throttling aspect which is implemented as a Spring Framework component. It is fairly simple to use the aspect in any context, with or without Spring Framework.

@Aspect
@Component
public class RateLimiterAspect {
 
    public interface KeyFactory {
        String createKey(JoinPoint jp, RateLimit limit);
    }
 
    private static final Logger LOGGER = LoggerFactory.getLogger(RateLimiterAspect.class);
 
    private static final KeyFactory DEFAULT_KEY_FACTORY = (jp, limit) -> JoinPointToStringHelper.toString(jp);
    
    private final ConcurrentHashMap<String, RateLimiter> limiters;
    private final KeyFactory keyFactory;
 
    @Autowired
    public RateLimiterAspect(Optional<KeyFactory> keyFactory) {
        this.limiters = new ConcurrentHashMap<>();
        this.keyFactory = keyFactory.orElse(DEFAULT_KEY_FACTORY);
    }
 
    @Before("@annotation(limit)")
    public void rateLimit(JoinPoint jp, RateLimit limit) {
        String key = createKey(jp, limit);
        RateLimiter limiter = limiters.computeIfAbsent(key, createLimiter(limit));
        double delay = limiter.acquire();
        LOGGER.debug("Acquired rate limit permission ({} qps) for {} in {} seconds", limiter.getRate(), key, delay);
    }
 
    private Function<String, RateLimiter> createLimiter(RateLimit limit) {
        return name -> RateLimiter.create(limit.value());
    }
 
    private String createKey(JoinPoint jp, RateLimit limit) {
        return Optional.ofNullable(Strings.emptyToNull(limit.key()))
                .orElseGet(() -> keyFactory.createKey(jp, limit));
    }
}

The class defines an additional interface and a default implementation for a key factory that is used if the annotation does not provide an explicit key for a rate limiter. The key factory can use the join point (basically a method call) and the provided annotation to create a suitable key for the rate limiter. The aspect also uses concurrent hashmap to store the rate limiter instances. The aspect is defined as a singleton but the rateLimit method can be called from multiple threads so the concurrent hashmap ensures we allocate only single rate limiter per unique key. Constructor injection in the aspect utilizes the optional injection support of Spring Framework. If there is no KeyFactory bean defined in the context, the default key factory is used.

The class is annotated with @Aspect and @Component so that Spring understands an aspect is defined and enables the @Before advice. @Before advice contains only one pointcut which requires a RateLimit annotation and binds it to the limit parameter of the method. The throttling implementation is quite simple. First a key is created for the rate limiter. Then the key is used to find or create a limiter and finally the limiter is acquired for a permission.

There’s a small gotcha in the rate limiter key creation. The key defined by the annotation is converted to an optional, but optional’s orElse method cannot be used due to performance reasons. Optional’s orElse method takes a value which we need to create in any case, when the optional is present and when it’s not. The other method orElseGet on the other hand takes a supplier which allows lazy evaluation of the value only when the optional is not present. The key factory’s createKey may be an expensive operation so the supplier version is used.

Concurrent hashmap contains a handy method computeIfAbsent that atomically finds or creates a value based on a key and a defined function. This allows simple and concise lazy initialization of the map values. The rate limiters are created on demand and guaranteed to have only single instance per unique limiter key.

The default key factory implementation uses a helper method from JoinPointToStringHelper that converts a join point to textual representation.

public class JoinPointToStringHelper {
 
    public static String toString(JoinPoint jp) {
        StringBuilder sb = new StringBuilder();
        appendType(sb, getType(jp));
        Signature signature = jp.getSignature();
        if (signature instanceof MethodSignature) {
            MethodSignature ms = (MethodSignature) signature;
            sb.append("#");
            sb.append(ms.getMethod().getName());
            sb.append("(");
            appendTypes(sb, ms.getMethod().getParameterTypes());
            sb.append(")");
        }
        return sb.toString();
    }
 
    private static Class<?> getType(JoinPoint jp) {
        return Optional.ofNullable(jp.getSourceLocation())
                .map(SourceLocation::getWithinType)
                .orElse(jp.getSignature().getDeclaringType());
    }
 
    private static void appendTypes(StringBuilder sb, Class<?>[] types) {
        for (int size = types.length, i = 0; i < size; i++) {
            appendType(sb, types[i]);
            if (i < size - 1) {
                sb.append(",");
            }
        }
    }
 
    private static void appendType(StringBuilder sb, Class<?> type) {
        if (type.isArray()) {
            appendType(sb, type.getComponentType());
            sb.append("[]");
        } else {
            sb.append(type.getName());
        }
    }
}

Finally the throttling can be applied to any Spring enabled method by just adding the @RateLimit annotation.

@Service
public class MyService {
 
    ...
 
    @RateLimit(5)
    public String callExternalApi() {
        return restTemplate.getForEntity(url, String.class).getBody();
    }
 
}

One might wonder if this solution scales out very well? No, it really doesn’t. Guava’s rate limiter blocks the current thread so if there’s a burst of asynchronous calls against the throttled service lots of threads will be blocked and might result exhaust of free threads. Another issue raises if the services are replicated in multiple applications or JVM instances. There is no global synchronization of a limiter rate. This implementation works fine for single application living in single JVM with decent load to throttled methods.

Throttle methods with Spring AOP and Guava rate limiter

Thank you!

Tapio Rautonen

Thank you!

Thank you!

Related Articles

Thank you!