JavaMail can be evil (and force you to restart your app server)
JavaMail always had an interesting approach when it comes to its configuration. Basically you have to fill an untyped map or Properties structure and hope for the correct interpretation. Countless tutorials on the net show the minimal properties required to make it work (send / receive mails).
However, as we painfully just learned, there are some lesser known properties you should probably take care of, which is timeout settings for socket IO. By default, JavaMail uses an infinite timeout for all socket operations (connect, IO, …)!
Now suppose you have a cluster of SMTP servers which handle outgoing mail, accessed via a DNS round robin. If one of those servers fail, which happens to be the one JavaMail wanted to connect to, your mail sending thread will hang – forever! This is exactly what happened to us and we needed to perform some real nasty magic to avoid tragedy.
Therefore, we now set timeouts for all operations:
String MAIL_SMTP_CONNECTIONTIMEOUT ="mail.smtp.connectiontimeout"; String MAIL_SMTP_TIMEOUT = "mail.smtp.timeout"; String MAIL_SMTP_WRITETIMEOUT = "mail.smtp.writetimeout"; String MAIL_SOCKET_TIMEOUT = "60000"; // Set a fixed timeout of 60s for all operations - // the default timeout is "infinite" props.put(MAIL_SMTP_CONNECTIONTIMEOUT, MAIL_SOCKET_TIMEOUT); props.put(MAIL_SMTP_TIMEOUT, MAIL_SOCKET_TIMEOUT); props.put(MAIL_SMTP_WRITETIMEOUT, MAIL_SOCKET_TIMEOUT);
Also, if you plan to access DNS round robin based services (like amazon S3) or in our case a mail cluster, don’t forget to also configure the DNS cache tiemout of Java (which is also infinite by default):
// Only cache DNS lookups for 10 seconds java.security.Security.setProperty("networkaddress.cache.ttl","10");
And while we’re at it, for us it turned out to be a good idea to set all encodings to UTF-8 (independent of the underlying OS) to provide a stable environment:
System.setProperty("file.encoding", Charsets.UTF_8.name()); System.setProperty("mail.mime.charset", Charsets.UTF_8.name());
…you don’t want to care about stuff like this at all? Feel free to use our open source Java library SIRIUS, which takes care of all that by providing a neat fluet API for sending mails: Sources on GitHub.
An example usage can be found in the cluster manager:
@Part private MailService ms; private void alertClusterFailure() { ... ms.createEmail() .useMailTemplate("system-alert", ctx) .toEmail(receiver).send(); ... }
Reference: | JavaMail can be evil (and force you to restart your app server) from our JCG partner Andreas Haufler at the Andy’s Software Engineering Corner blog. |
I worked at place where many of our sites and applications stopped working without any reason for it. After having fun getting VisualJVM to connect to the apps we all the threads were waiting to connect to the mail server, which was down at the time. Thankfully the issue that was using email was configurable to be turned off but very frustrating and wasted a lot of time trying to work out what it was.
We just cutover our internal smtp relay instance to another datacenter, and since we process about 10,000 messages a day from programmatically sent email, we have a pair of f5 load balancers, and several HT servers. After cutting over our DNS record to use the new IP of the environment in the new datacenter, we slowly watched traffic go to the new site. That is until the dreaded bottom 20%… We had more instances of services caching the IP address than I can count. We literally had to reboot a few, since we couldnt find which process or service was… Read more »