Liferay Unconference: talks we attended and what we learned

Emma from FirelayNews

So, yesterday was the first (unofficial) day of Liferay DevCon. It was the day of the Unconference. The Liferay community came together one day before the big event to discuss several topics. Topics, that they proposed to talk about. Our DevOps Freark and Alwyn and Firelay’s general manager Lex attended. Here’s what they have talked about and what they have learned. If you have any questions or want to comment on their findings, leave a comment :-).

Alwyn’s summary

Performance Monitoring/Analysis

This talk was sparked by a user who was experiencing OOM (Out Of Memory) exceptions in various environments. Quite a few people joined who were interested in ways to trace issues like these, as well as performance tips and tricks.

Core tools such as gcore and the default jmap were discussed: While some (IBM) don’t recommend gcore anymore, it can still be seen as a very quick way to make a heap dump.

The -histo parameter for jmap is very useful; it can generate a histogram of the heap with various info, such as number of objects and memory used, for each class.

GCeasy(.io) was suggested as a more useful way to analyse heap dumps. The discussion moved to performance monitoring, tooling, and general performance talk. Dynatrace, NewRelic, AppDynamics, JavaMelody, and even more generic tools like Nagios and Prometheus were mentioned.

NewRelic is a very decent tool, but – according to some participants – lack the granularity Dynatrace and AppDynamics provide. Dynatrace also comes with excellent live-monitoring and alerting. With this fine granularity, AppDynamics comes with a higher overheard, while Dynatrace manages to keep it low. However, this tooling costs big $$, unless you install the entire server on all environments (needs checking/research).

In short: NewRelic is great monitoring but lacks true live info and granularity, Dynatrace has all this but costs a buck. JavaMelody is a more simple, free and open source tool to monitor the entire java process: usage, requests, threads, caches, etc.

We moved on from analysis and monitoring to enhancements. DXP startup times can be very very long. Most solutions were relatively simple: disable LPKG validation, – note you cannot do this on acceptance or production – disable all the startup properties and filters you don’t need in portal-ext.properties.

One interesting way to prevent slower startup times after clearing osgi/state is to put it into a (git) repository, and simply revert to a known proper state when you want to clean it.

Another thing mentioned was that there is something in development at Liferay with which you can disable/blacklist the loading of individual LPKG’s. So you can disable – for example – the wiki or collaboration portlets.

Security

This talk seemed to be a bit low on actual content… The usual was discussed: firewalling, logging, and rights management.

A recurring topic was that the Liferay administrator account should not be used. Instead, specific roles with rights should be created: the administrator should not have access to edit sites. Editors should not have access to most of the control panel.

A funny topic came up that Liferay’s privacy policy still states that they use “industry standard” encryption algorithms such as “MD5, DES and RSA”. Whoops.

Configuration migration management

Geert van der Ploeg from Finalist has been working on a tool with which you can create easily reproducible environments with custom fields, pages, users…

He took inspiration from the way Flyway handles this.

The idea is that you test changes – be it in content, structure, or configuration – in your local development environment, write (code) them down as a migration using a DSL (domain-specific language) for this tool, and then you can test the migration on a fresh test environment.

This way, you can quickly set up copies of environments without worrying about forgetting some specific setting or making typos.

Freark’s findings

Running production workloads with DXP led by Brett Swaim

Sharing some tips and best practices of operating a high volume Liferay application. For these types of environments, the most critical tip is load testing, making sure there are never any bad regressions. Tuning your EHcache configuration is next in the list of prios which Brett indicated to be of (give or take) 40% importance of the total experience.

Log Analysis: howto and best practices led by Freark van der Bos (Firelay)

Most participants currently don’t have or run as good logging solution at all. Besides logging some advanced debugging techniques were discussed too. Sentry.io was warmly recommended as a tool to investigate using.

Performance issues (+Development best practices + CI + upgrading to DXP) led by “Alex”

A badly coded portlet can defeat all your performance optimisations, otherwise, use caching in Tomcat as well as on the HTTP level, offload asynchronous work out of the request-response cycle and check your database queries.