TL;DR: This article describes the modularity changes in the Ceylon run-time and distribution, in order to make them lighter at run-time. Skip to the Final runtime dependencies section if you just want the outcome.
Ceylon has featured a modular architecture from the start. Not just for Ceylon users who write modules, but also within the Ceylon distribution. Historically we used to have very few modules, that were directly related to separate Git projects. Adding a new module meant a new repository and lots of changes in the build. Naturally, as the project grew, each of those modules also grew, and got new third-party dependencies, and occasionally adding a feature in one module was made tremendously easier by just adding that “one more” dependency between distribution modules, resulting in a big spaghetti graph of distribution modules that is common in older/evolved systems.
As we initially expected most Ceylon users to run their code using the ceylon run command, we figured that since they have the Ceylon distribution installed, it does not matter if they depend on more modules from that distribution than strictly necessary. Those modules had to be there anyway, so it would not save any bandwidth to reduce those dependencies.
Naturally, we were wrong, and between the Ceylon Eclipse or IntelliJ IDEs, running Ceylon on OpenShift, WildFly, Vert.x or on Android, people started running Ceylon without the distribution installed, using just the standard java runner. It became soon apparent that we had to untangle those dependencies to make the runtime requirements lighter.
Historically we had the following modules:
- common module used by other modules
- typechecker (the shared compiler front-end)
- Java compiler back-end
- JavaScript compiler back-end
- module repository system
- JBoss modules runtime
The model module
When we implemented reified generics, we had to add subtyping to the runtime, so that we’d be able to figure out if is T bar was true or not. The easiest thing at the time was to “just” depend on the typechecker (compiler front-end) which dealt with the language model and subtyping, and the Java compiler back-end, which had infrastructure to load a language model from JVM information such as class files, or in this case reflection.
This essentially made the runtime depend on the compiler front-end and back-end, which we realised was not ideal, so during the Ceylon 1.2 development, we extracted all model description, loading and subtyping to a new ceylon-model module, but we did not have enough time to do more and so these dependencies remained due to other causes.
Supporting Java 9
During our work on supporting Java 9 / Jigsaw modules in Ceylon, it became clear that having kept our “fork” of javalang tools (that we use for javac) under its original package name would not work anymore, we renamed its package and used the opportunity to prune away parts of the java tools we did not use. We also extracted the class-file reader part to its own module so we could use it outside of the compiler to remove our dependency tojandex (a class-file scanner).
Finally, when we created the ceylon jigsaw tool (which populates a folder with the jar files required by a Ceylon module, to run it on a Java 9 VM) it became evident that the runtime still depended not just on the compiler front-end and Java back-end, but even on the JavaScript back-end, which frankly made little sense in most JVM executions.
These dependencies were due to the Ceylon Tool Provider API having snuck into the ceylon.language module as a convenience (at the time). Since that allowed you to compile and run Ceylon programmatically for both Java and JavaScript back-ends, it had to depend on the tools.
We decided to split the Ceylon Tool Provider into its own module and got rid of the final dependencies from the language module to the compilers and typechecker, but had no more time to get rid of further dependencies such as JBoss Modules and Aether in time for Ceylon 1.2.2.
Supporting Android
Initial work on running Ceylon on Android revealed that what passes for small dependencies on ordinary JVM executions, or even on Java EE deployments, was not an option on Android where every method counts.
At this point we had to bite the bullet and make every non-required transitive dependency go.
We noticed that the old common module had grew to include the Command-Line Tooling API that makes the ceylon command and its subcommands and plugins work. That in turn depended on a Markdown renderer used by ceylon doc. It was pretty trivial to extract it to its own module because this was never used in Ceylon user programs.
Next in line was our Shrinkwrap Resolver dependency, which our module repository system uses to interoperate with Maven repositories. This was a fat-jar with all its dependencies included, including some Apache Commons modules, and an outdated version of Eclipse Aether. That fat-jar had already been problematic in our Maven module, which already had its version of Aether, so getting rid of the fat-jar was a good idea. We also realised that some of its Apache Commons dependencies were already included outside the fat-jar in our distribution repository, so there was that duplication to fix too.
So what we did was remove the Shrinkwrap Resolver dependency and use Aether directly, by incorporating all its subcomponents into our distribution. It turns out that because the latest version of Aether requires Google Guava, our distribution grew in size rather than shrink (that jar is huge). But to offset that, we made the Aether dependency optional, and made sure it was possible to run Ceylon without it as long as there was some compilation step beforehand that provided all the Maven dependencies that you may use in interop. ceylon fat-jar or ceylon jigsaw would do that for you, for example.
Our module repository system also provided support for writing to WebDAV or Herd repositories, which required some dependencies on Apache Http Client or Sardine, and we made these dependencies optional as well, because at runtime your Ceylon program is very unlikely to write to HTTP repositories. This is something only the compiler and other tools do.
We also removed a dependency to JBoss Modules from the language module using abstraction, since that platform was optional and never used on Android or other flat-classpath runtimes.
Finally, the language module only had one dependency left on the (much slimmer) module repository system via the presence of the Main API in there, and we moved that class to its own module.
Final runtime dependencies
After all this pruning, the language module on the JVM is back down to requiring the following set of transitive dependencies:
- common (small and free of tooling and dependencies)
- model (which depends only on the class-file reader)
- class-file reader
So your Ceylon module will only depend on four jars (these three and the language module), the sum size of which is 2.4 Mb, which is much smaller than initially, and has dramatically less methods, at around 17148 methods. This is still too much, but can be brought down by tooling such as ProGuard to remove unused classes. Remember this includes a runtime for an entire language, so it’s not that big, all things considered.
SDK changes
In order to be able to use Ceylon’s HTTP client on Android, we also split up the ceylon.net module from the Ceylon SDK into client and server modules. Otherwise the HTTP server and its dependencies were too much drag for Android’s method count.
| Reference: | Modularity Changes from our JCG partner Stéphane Épardaud at the Ceylon Team blog blog. |
