GlassFish - World's first Java EE 7 Application Server


Compilation of Mail Discussion: Consistent hash vs. location transparency - Stetements

This is a mail discussion that spawned from Discussion - Consistent hash vs. location transparency and was initiated by Erik van der Velden 072607 17:22 CET and replied to by Joel Binnquist and Jan Leuhe .

I'm still a bit uncertain about the implications of the things we are/have been discussing.

This is what I came up with (right or wrong?)

  • The @SipApplicationKey mechanism should be used by the application to associate incoming SIP sessions with a specific SIP application session id (encodeURI is deprecated)
    • Reply by Jan 072707 02:04 CET: I think this should be more general, as in: "incoming SIP requests", since the annotated method must take an argument of type "SipServletRequest"
      • Reply by Joel 072707 11.08 CET: Agree (I assumed that is what Erik meant)
    • Reply by Joel 072707 09.47 CET: Correct.
  • This @SipApplicationKey mechanism should use certain fields from the initial request (if this is based on other criteria, like a database lookup, information in the body, or a random number then the correlation with the LB routing, see below, will be a problem).
    • Reply by Jan 072707 02:04 CET: Where in the spec does it mention that encodeURI is deprecated?
      • Reply by Joel 072707 11.08 CET: Section 15.11: last sentence of the introduction text.
    • Reply by Joel 072707 09.47 CET: Correct.
  • The LB plugin should use the same data as the @SipApplicationKey method to generate the hashkey, this will mean that all SIP sessions belonging to the SAS are routed the same.
    • Reply by Joel 072707 09.47 CET: Yes, that is our current proposal how to solve this. If we can find another better way that would be great.
  • All the applications should use the same @SipApplicationKey mechanism (since the LB is not aware of applications and the applicaiton is only selected after the AR has run)
    • Reply by Joel 072707 09.47 CET: Unfortunately, as it looks right now we got this undesired limitation, unless we find a better solution. However, if possible, one might let the applications have different rules and let the load balancer merges the rules.
  • the sipapplicationsession.encodeURL generation should also include the jroute in the header ? This ensures that the encoded URL is routed to the same server as were the SASid is? I'm a bit unsure here.
    • Reply by Jan 072707 02:04 CET: Hmm, you mention URL encoding and header in the same sentence. Normally, URL encoding is used orthogonally to headers, so it's either URL encoding or headers.
      The proxy-id mechanism is used by Sun's HTTP LB and identifies the target instance. I believe the Converged LB team is thinking of a similar header (and corresponding param name) called "bekey", which is supposed to hold the consistent hashkey and which would replace the proxy-id mechanism.
      (I assume "bekey" stands for "backend key"?)
      • Reply by Joel 072707 11.08 CET: Correct, and yes, bekey stands for backend key.
    • Reply by Joel 072707 09.47 CET: Not really, I have proposed in my FSD for SIP Load Balancer to use a different parameter (cookie), bekey, which is the hash key that was used when routing the initial request. As I see it, that parameter replaces the jroute.
  • When the access to the SAS in any other way (JCA, JMS, RMI/remote EJB) then we have no way of guaranteeing that the requests are routed to the server that has the data!
    Therefore we have to take remote access or migration of active sessions into account. Very tricky.
    For the application it should still look like all requests execute in the same JVM, but I do not see how it can. This is an issue for more discussion.
    • Reply by Joel 072707 09.47 CET: Agree, it might be that we have to put a limitation what application developers should expect from Sailfin in this area (at least in the first version).
  • the convergedHttpSession.encodeURL should include both the jsessionid and the jroute? (is this method really useful in practice?)
    • Reply by Jan 072707 02:04 CET: Replace "jroute" with "bekey".
      Currently, HttpServletResponse.encodeURL() includes the jroute-id, but I believe the jroute-id mechanism will be replaced by "bekey" when using the converged LB.
      I believe both SipApplicationSession.encodeURI() and ConvergedHttpSession.encodeURL() will need to include the "bekey" param.
    • Reply by Joel 072707 09.47 CET: I beleive so. However, it should add bekey too.
  • If we start migrating active sessions, we have to remember that we have to invoke the appropriate SipApplicationListener and SipSessionListener callbacks.
    • Reply by Joel 072707 09.47 CET: Hmm, I think you really found an important problem here. What would be the proper event to signal? Session Passivated or Dession Destroyed? When to think of that it hits me: how shall we handle ongoing transactions? JSR-289 does not allow a session to be invalidated when a transaction is ongoing! Actually, this is something that really makes it almost impossible to migrate active sessions (also after the recovery from a fail-over)!
      • Reply by Joel on 072707 17:39 CET: After thinking of it a bit more (see Meeting Minutes - July 26th), I have come to the conclusion that it should be acceptable to lose your transactions at recovery. I mean, we do not give any guarantees that transactions are kept when a node fails, consequently it should be OK to not guarantee that transactions are kept at recovery after fail-over.
  • if the hash key is used as the jroute cookie in HTTP or as the bekey in SIP then sessions, whether ongoing or new are routed according to the current consistent hash setup, which implies that after a restore active sessions must be migrated, but also that ongoing and new sessions that belong to the same SAS (e.g., based on @SipApplicationKey mapping) will still end up together in the same server instance.
    • Reply by Jan 072707 02:04 CET: ... unless we added an additional mapping level. So, instead of mapping hashkey to instance id, we would map hashkey to sasid to instance id.
      • Reply by Joel 072707 11.08 CET: Yes, but that won't work if an application uses the "Session Key Based targeting Mechanism", because then SIP-URI:s will not have cookies encoded in them.
    • Reply by Joel 072707 09.47 CET: Yes, that is the idea.