#1644 confirmed
Jiri Helmich

CacheFor - hitting memcache key length limit

Reported by Jiri Helmich | February 5th, 2013 @ 02:19 PM | in 1.4.x

When you annotate your action method with the @CacheFor annotation, you may easily hit the limit of the maximum key length for memcache.

I'd like to start a discussion about integrating the following change into the official Play Framework 1.x distribution:
https://github.com/teuzz/play/commit/e1afde832e8d85ac0d39222ec5e4b2...

The ActionInvoker computes the cache key based on request URL if cache key is not explicitly set. If the URL contains a long set of parameters (e.g. because of GA tracking), you can easily hit the limit of 250 chars. Moreover the cache key is prefixed with the "urlcache:" string, which makes it even easier.

Example URL: http://www.example.cz/magazin/zivotni-styl-a-zdravi/nase-zdravi/282... (even not 250 chars and fails, even not (250 - "urlcache:".length) = 241).

I wanted to avoid this by using different memcache implementation (with no key length limit), but the problem is, that the limitation is enforced by key validation before saving the value to the cache server. This is done by the underlying library (net.spy.memcached.MemcachedClient) which has the key maxlength hardcoded (https://github.com/dustin/java-memcached-client/blob/master/src/mai...).

Trace:

play.exceptions.UnexpectedException: Unexpected Error
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:264) at Invocation.HTTP Request(Play!) Caused by: java.lang.IllegalArgumentException: Key is too long (maxlen = 250)
at net.spy.memcached.MemcachedClient.validateKey(MemcachedClient.java:245) at net.spy.memcached.MemcachedClient.addOp(MemcachedClient.java:277) at net.spy.memcached.MemcachedClient.asyncGet(MemcachedClient.java:799) at play.cache.MemcachedImpl.get(MemcachedImpl.java:136) at play.cache.Cache.get(Cache.java:174) at play.mvc.ActionInvoker.invoke(ActionInvoker.java:155) ... 1 more

The solution presented in the changeset is to hash the URL with the Codec.hexSHA1(String) method. This enforces a consistent length of the cache key. I'm pretty sure there's no need to be afraid of SHA1 collisions.

The best way would be to make the user able to set the max key length as a parameter. But that would need patching the Memcache client lib as well as the framework and would not be so easy to implement.

Comments and changes to this ticket

  • Jiri Helmich
  • notalifeform (at gmail)

    notalifeform (at gmail) March 31st, 2013 @ 09:06 PM

    • State changed from “new” to “confirmed”
    • Assigned user set to “notalifeform (at gmail)”
    • Milestone set to 1.3

    Hi,

    I'd think it would be better to only use SHA1 if the key exceeds 250 chars. There might be use-cases where the cache-keys do contain useful information (depending on your cache implementation).

    Don't you think?

    regards,

    Robert

  • Jiri Helmich

    Jiri Helmich August 16th, 2013 @ 01:40 PM

    As you can see, I applied this only to the automatically created key based on current URL, if the key contains a useful information, it is probable, that the developer will assign it manually and therefore, they are able to control its length.

    cacheKey = "urlcache:" + Codec.hexSHA1(request.url + request.querystring);

    Of course, it is possible to check the length and use the SHA1 conditionally, but that would make the caching a bit inconsistent, wouldn't it?

    Sorry for the long time span, did not receive any notification.

  • notalifeform (at gmail)

    notalifeform (at gmail) August 21st, 2013 @ 07:28 PM

    Hi Jiri,

    We did an implementation that did something like

    if key > 250 chars
    key = key.subStr(1,216) key += Codec.hexMD5(key)

    (a bit more complicated, since our keys can also contain illegal chars, like spaces - and cache keys are measured in bytes rather than characters)

    I'll make an alternative pull request for it.

  • notalifeform (at gmail)

    notalifeform (at gmail) April 27th, 2014 @ 12:03 PM

    • Milestone changed from 1.3 to 1.4.x
  • Hard Work

    Hard Work September 7th, 2018 @ 06:33 AM

    If you are having desire of playing free online bejeweled 3 games so you can enjoy the same games here http://bejeweled3.co as this platform is all safe and secure to play these games in high quality at different attracting levels.

Please Sign in or create a free account to add a new ticket.

With your very own profile, you can contribute to projects, track your activity, watch tickets, receive and update tickets through your email and much more.

New-ticket Create new ticket

Create your profile

Help contribute to this project by taking a few moments to create your personal profile. Create your profile »

<h2>Play framework</h2>

Play makes it easier to build Web applications with Java. It is a clean alternative to bloated Enterprise Java stacks. It focuses on developer productivity and targets RESTful architectures. Learn more on the <a href="http://www.playframework.org">http://www.playframework.org</a> website.<br><br>

<h2>Source code is hosted on github</h2>Check out our repository at <a href="http://github.com/playframework/play">http://github.com/playframework/play</a><br><br>

<h2>Contributing, creating a patch</h2> Please read the <a href="http://play.lighthouseapp.com/projects/57987/contributor-guide">contributor guide</a><br><br>

<h2>Reporting Security Vulnerabilities</h2> Since all bug reports are public, please report any security vulnerability directly to <em>guillaume dot bort at gmail dot com</em>.<br><br>

<h2>Creating a bug report</h2> Bug reports are incredibly helpful, so take time to report bugs and request features in our ticket tracker. We’re always grateful for patches to Play’s code. Indeed, bug reports with attached patches will get fixed far quickly than those without any.<br><br>

Please include as much relevant information as possible including the exact framework version you're using and a code snippet that reproduces the problem.<br><br>

Don't have too much expectations. Unless the bug is really a serious "everything is broken" thing, you're creating a ticket to start a discussion. Having a patch (or a branch on Github we can pull from) is better, but then again we'll only pull high quality branches that make sense to be in the core of Play.

Pages