Eclipse Vert.x for Scala next steps

TL;DR:

• No Scala 2.13 in Eclipse Vert.x 3.x due to in­creased sup­port bur­den
• New value classes based ap­proach for Vert.x 4

Retrospective

It’s been more than two years since the in­cep­tion of vertx-lang-scala to the Vert.x ecosys­tem. And al­most as long since I wrote my first blog post about it.

A lot has hap­pened since March 2017:

• vertx-lang-scala kept up with new ver­sions of Scala
• all Vert.x-​modules are sup­ported (35 so far)
• a Giter8 based tem­plate was added for eas­ily boot­strap­ping a Vert.x-​Scala-project
• Bugs were squashed

And most re­cently we re­ceived a great con­tri­bu­tion by Niko­laj Leis­chner who was kind enough to port the techempow­ered bench­mark to vert-​lang-scala. Which will be part of the next steps.

The num­bers pro­duced by this bench­mark were very promis­ing and and ad­di­tional mo­ti­va­tion to move to the next phase of Scala sup­port for Vert.x.

Old idea

Be­fore get­ting to the new ideas I want to take a look at the “old” one.

The cur­rent ver­sion of vert-​lang-scala is based around the idea of wrap­ping the Vert.x-API with a ded­i­cated Scala-​layer. That layer is cre­ated using a Freemarker-​based code gen­er­a­tor. I took this idea from the first try by Tim Fox for build­ing that sup­port.

Wrap­ping the ex­ist­ing Java-​API was rather painful but gave me great flex­i­bil­ity to cre­ate an id­iomatic Scala-​API.

But an ap­proach like that comes with a price:

• There are a lot of in­ter­me­di­ate ob­jects being cre­ated.
• Many un­nec­ces­sary con­ver­sions be­tween Java/Scala types

Both in­creased mem­ory con­sump­tion and garbage col­lec­tion ac­tiv­ity quite a bit and has been bug­ging me from the be­gin­ning.

New idea

With Vert.x 4 ap­proach­ing I was fi­nally able to in­vest time into the re­work I had wanted to do for quite a while.

The core idea is to re­place the cur­rent wrap­ping based ap­proach with some­thing more light­weight but na­tive to the Scala-​world.

And that’s where value classes come in.

Value classes allow the ex­ten­sion of ex­ist­ing classes with ad­di­tional meth­ods. They make it easy to con­trol when meth­ods be­come vis­i­ble and do that with a min­i­mum of over­head. To be pre­cise: A wrap­ping class is nor­mally ever only in­stan­ti­ated once.

A good ex­am­ple is the ad­di­tion of meth­ods for wrap­ping the Vert.x ap­proach of Promises with Scala-​Futures. Each method re­turn­ing a Vert.x-​Promise needs to re­ceive an al­ter­na­tive ver­sion which re­turns a Scala-​Future.

In Vert.x 3 I achieved that by adding meth­ods to the wrap­per and giv­ing them a dis­tinct name. A method called lis­ten re­turn­ing a Promise would re­ceive a com­pan­ion called lis­ten­Fu­ture in the Scala layer.

Let’s look at how this looks in the new ap­proach:

package io.vertx.scala
package object core{
   implicit class HttpServerScala(val asJava: io.vertx.core.http.HttpServer) extends AnyVal {
      def listenFuture(port: java.lang.Integer): scala.concurrent.Future[io.vertx.core.http.HttpServer] = {..}
      ..
}

The code above does the fol­low­ing things:

• It cre­ates a pack­age ob­ject for io.vertx.scala.core
• it adds an im­plict class HttpServer­Scala to wrpa HttpServer
• it adds a lis­ten­Fu­ture method

Using this method in code looks like this:

package io.vertx.scala.demo

import io.vertx.lang.scala.VertxExecutionContext
import io.vertx.scala.core._

import scala.util.{Failure, Success}

object Main {
  def main(args: Array[String]): Unit = {
    val vertx = Vertx.vertx()
    implicit val ec = VertxExecutionContext(vertx.getOrCreateContext())
    vertx
      .createHttpServer()
      .requestHandler(r => {
        r.response().end("bye")
      })
      .listenFuture(6667)
      .onComplete {
        case Success(_) => println("Started")
        case Failure(exception) => println("Failure")
      }
  }
}

Im­port­ing the pack­age ob­ject using im­port io.vertx.scala.core._ brings the ex­ten­sion method into scope and makes them avail­able on all in­stances of HttpServer. In the ex­am­ple above cre­ate­HttpServer() re­turn such an in­stance and we can now use the id­iomatic Scala way of han­dling a Fu­ture.

Even more

Ex­tend­ing classes with Future-​methods is only one of the new things to come. On top of that the sup­port for DataOb­jects will be con­sid­er­ably im­proved, both through ex­tend­ing them and by pro­vid­ing type aliases.

I also switched from doing all con­ver­sions for col­lec­tions au­to­mat­i­cally to hand­ing the con­trol back to the user. Some­thing which gets even more im­por­tant for Scala 2.13 and the new col­lec­tion API.

The downside

The clear down­side of this ap­proach is that the Java-​methods will stay vis­i­ble since the java-​classes won’t be wrapped but ex­tended. This might lead to some con­fu­sion but I am pretty sure the ben­e­fits out­weight this down­side.

The big­ger change will be the re­moval of au­to­matic von­ver­sion be­tween Scala types (Long/Int/String and Col­lec­tions) and their Java coun­ter­parts. I spent con­sid­er­able time try­ing to tune that part in the cur­rent ver­sion bbut al­ways ended up hit­ting some edge­case. For now I’ve de­cided to have the user pick the right time to con­vert.

I might still add this fea­ture in a later ver­sion if user feed­back points into that di­rec­tion.

When will I get it?

First for the good news: There is al­ready a branch with a full im­ple­men­ta­tion.

The bad news: It will break until Vert.x 4.0 is fi­nally re­leased.

Vert.x 4 is in ac­tive de­vel­op­ment with most APIs al­ready fi­nal­ized but break­ing changes still hap­pen. So use at your own risk!

What about Scala 2.13?

Scala 2.13 has been re­leased re­cently which prompted ques­tions from the com­mu­nity about when it will be sup­ported by Vert.x.

I haven’t done a good job pro­vid­ing the re­sults of our in­ter­nal dis­cus­sions on that topic to the com­mu­nity. So here we go:

• Vert.x 3 will stay on 2.12 for the fol­low­ing rea­sons:
  • Both are still ac­tively sup­ported
  • Scala ecosys­tems takes some time to do the switch to 2.13
  • We sim­ply don’t have the ca­pac­ity to sup­port both ver­sions AND the up­com­ing new ver­sion
• Vert.x 4 will re­ceive 2.13 sup­port
  • Scala ecosys­tem will have moved closer to 2.13 adop­tion when Vert.x 4 comes out

For the adventure seaker

I ac­tu­ally did a port of vertx-​lang-scala 3.8 to Scala 2.13 and you can grab the work in this branch.

Don’t ex­pect ANY sup­port for this branch. This was only an ex­per­i­ment to see how much I had to change for ini­tial 2.13 sup­port.

Summary

Vert.x 4 will be an evo­lu­tion­ary step for vertx-​lang-scala. Value classes promise to re­duce both com­plex­ity and al­lo­ca­tion rate, two things which have been bug­ging me quite a bit with the cur­rent ap­proach.

I am eager to hear from you all what you think about this new di­rec­tion.