Combine vert.x and mongo to build a giant

This blog post is part of the in­tro­duc­tion to vert.x se­ries. Last time, we have seen how we can use the vertx-jdbc-client to con­nect to a data­base using a JDBC dri­ver. In this post, we are going to re­place this JDBC client by the vertx-mongo-client, and thus con­nect to a Mongo data­base.

You don’t un­der­stand the title, check the mon­goDB web­site.

But be­fore going fur­ther, let’s recap.

Previously in ‘introduction to vert.x’

1. The first post has de­scribed how to build a vert.x ap­pli­ca­tion with Maven and ex­e­cute unit tests.
2. The sec­ond post has de­scribed how this ap­pli­ca­tion can be­come con­fig­urable.
3. The third post has in­tro­duced vertx-​web, and a small col­lec­tion man­age­ment ap­pli­ca­tion has been de­vel­oped. This ap­pli­ca­tion of­fers a REST API used by a HTML/JavaScript fron­tend.
4. The fourth post has pre­sented how you can run in­te­gra­tion tests to en­sure the be­hav­ior of your ap­pli­ca­tion.
5. The last post has pre­sented how you can in­ter­act with a JDBC data­base using the vertx-​jdbc-client.

This post shows an­other client that lets you use Mon­goDB in a vert.x ap­pli­ca­tion. This client pro­vides an vert.x API to ac­cess asyn­chro­nously to the Mongo data­base. We won’t com­pare whether or not JDBC is su­pe­rior to Mongo, they have both pros and cons, and you should use the one that meet your re­quire­ments. Vert.x lets you choose, that’s the point.

The vertx-​mongo-client doc­u­men­ta­tion is avail­able here.

The code de­vel­oped in this blog post is avail­able in the branch post-6. Our start­ing point is the code from the post-5 branch.

Asynchronous data access

One of the vert.x char­ac­ter­is­tics is being asyn­chro­nous. With an asyn­chro­nous API, you don’t wait for a re­sult, but you are no­ti­fied when this re­sult is ready. Thanks to vert.x, this no­ti­fi­ca­tion hap­pens in the same thread (un­der­stand event loop) as the ini­tial re­quest:

Your code (on the left) is going to in­voke the mongo client and pass a call­back that will be in­voked when the re­sult is avail­able. The in­vo­ca­tion to the mongo client is non block­ing and re­turns im­me­di­ately. The client is deal­ing with the mongo data­base and when the re­sult has been com­puted / re­trieved, it in­vokes the call­back in the same event loop as the re­quest.

This model is par­tic­u­larly pow­er­ful as it avoids the syn­chro­niza­tion pit­falls. In­deed, your code is only called by a sin­gle thread, no need to syn­chro­nize any­thing.

As with every Maven project…

… we need to up­date the pom.xml file first.

In the pom.xml file, re­place the vertx-jdbc-client by the vertx-mongo-client:

<dependency>
  <groupId>io.vertx</groupId>
  <artifactId>vertx-mongo-client</artifactId>
  <version>3.1.0</version>
</dependency>

Un­like JDBC where we were in­stan­ti­at­ing a data­base on the fly, here we need to ex­plic­itly starts a Mon­goDB server. In order to launch a Mongo server in our test, we are going to add an­other de­pen­dency:

<dependency>
  <groupId>de.flapdoodle.embed</groupId>
  <artifactId>de.flapdoodle.embed.mongo</artifactId>
  <version>1.50.0</version>
  <scope>test</scope>
</dependency>

This de­pen­dency will be used in our unit tests, as it lets us start a mongo server pro­gram­mat­i­cally. For our in­te­gra­tion tests, we are going to use a Maven plug­in start­ing and stop­ping the mongo server be­fore and after our in­te­gra­tion tests. Add this plug­in to the <plugins/> sec­tion of your pom.xml file.

<plugin>
  <groupId>com.github.joelittlejohn.embedmongo</groupId>
  <artifactId>embedmongo-maven-plugin</artifactId>
  <version>0.2.0</version>
  <executions>
    <execution>
      <id>start</id>
      <goals>
        <goal>start</goal>
      </goals>
      <configuration>
        <port>37017</port>
      </configuration>
    </execution>
    <execution>
      <id>stop</id>
      <goals>
        <goal>stop</goal>
      </goals>
    </execution>
  </executions>
</plugin>

No­tice the port we use here (37017), we will use this port later.

Enough XML for today

Now that we have up­dated our pom.xml file, it’s time to change our ver­ti­cle. The first thing to do is to re­place the jdbc client by the mongo client:

mongo = MongoClient.createShared(vertx, config());

This client is con­fig­ured with the con­fig­u­ra­tion given to the ver­ti­cle (more on this below).

Once done, we need to change how we start the ap­pli­ca­tion. With the mongo client, no need to ac­quire a con­nec­tion, it han­dles this in­ter­nally. So our startup se­quence is a bit more sim­ple:

createSomeData(
    (nothing) -> startWebApp(
        (http) -> completeStartup(http, fut)
    ), fut);

As in the pre­vi­ous post, we need to in­sert some pre­de­fined data if the data­base is empty:

private void createSomeData(Handler<AsyncResult<Void>> next, Future<Void> fut) {
  Whisky bowmore = new Whisky("Bowmore 15 Years Laimrig", "Scotland, Islay");
  Whisky talisker = new Whisky("Talisker 57° North", "Scotland, Island");
  System.out.println(bowmore.toJson());
  // Do we have data in the collection ?
  mongo.count(COLLECTION, new JsonObject(), count -> {
    if (count.succeeded()) {
      if (count.result() == 0) {
        // no whiskies, insert data
        mongo.insert(COLLECTION, bowmore.toJson(), ar -> {
          if (ar.failed()) {
            fut.fail(ar.cause());
          } else {
            mongo.insert(COLLECTION, talisker.toJson(), ar2 -> {
              if (ar2.failed()) {
                fut.failed();
              } else {
                next.handle(Future.<Void>succeededFuture());
              }
            });
          }
        });
      } else {
        next.handle(Future.<Void>succeededFuture());
      }
    } else {
      // report the error
      fut.fail(count.cause());
    }
  });
}

To de­tect whether or not the data­base al­ready con­tains some data, we re­trieve the num­ber of doc­u­ments from the whiskies col­lec­tion. This is done with : mongo.count(COLLECTION, new JsonObject(), count -> {}). The sec­ond pa­ra­me­ter is the query. In our case, we want to count all doc­u­ments. This is done using new JsonObject() that would cre­ate a query ac­cept­ing all doc­u­ments from the col­lec­tion (it’s equiv­a­lent to a SELECT * FROM ...).

Also no­tice the insert calls. Doc­u­ments are passed as JSON ob­ject, so to in­sert an ob­ject, just se­ri­al­ize it to JSON and use mongo.insert(COLLECTION, json, completion handler).

Mongo-ize the REST handlers

Now that the ap­pli­ca­tion boot se­quence has been mi­grated to mongo, it’s time to up­date the code han­dling the REST re­quests.

Let’s start by the getAll method that re­turns all stored prod­ucts. To im­ple­ment this, we use the find method. As we saw for the count method, we pass an empty json ob­ject to de­scribe a query ac­cept­ing all doc­u­ments:

private void getAll(RoutingContext routingContext) {
  mongo.find(COLLECTION, new JsonObject(), results -> {
    List<JsonObject> objects = results.result();
    List<Whisky> whiskies = objects.stream().map(Whisky::new).collect(Collectors.toList());
    routingContext.response()
        .putHeader("content-type", "application/json; charset=utf-8")
        .end(Json.encodePrettily(whiskies));
  });
}

The query re­sults are passed as a list of JSON ob­jects. From this list we can cre­ate our prod­uct in­stances, and fill the HTTP re­sponse with this set.

To delete a spe­cific doc­u­ment we need to se­lect the doc­u­ment using its id:

private void deleteOne(RoutingContext routingContext) {
  String id = routingContext.request().getParam("id");
  if (id == null) {
    routingContext.response().setStatusCode(400).end();
  } else {
    mongo.removeOne(COLLECTION, new JsonObject().put("_id", id),
        ar -> routingContext.response().setStatusCode(204).end());
  }
}

The new JsonObject().put("_id", id) de­scribes a query se­lect­ing a sin­gle doc­u­ment (se­lected by its unique id, so it’s the equiv­a­lent to SELECT * WHERE id=...). No­tice the _id which is a mongo trick to se­lect a doc­u­ment by id.

Up­dat­ing a doc­u­ment is a less triv­ial:

private void updateOne(RoutingContext routingContext) {
  final String id = routingContext.request().getParam("id");
  JsonObject json = routingContext.getBodyAsJson();
  if (id == null || json == null) {
    routingContext.response().setStatusCode(400).end();
  } else {
    mongo.update(COLLECTION,
        new JsonObject().put("_id", id), // Select a unique document
        // The update syntax: {$set, the json object containing the fields to update}
        new JsonObject()
            .put("$set", json),
        v -> {
          if (v.failed()) {
            routingContext.response().setStatusCode(404).end();
          } else {
            routingContext.response()
                .putHeader("content-type", "application/json; charset=utf-8")
                .end(Json.encodePrettily(
                  new Whisky(id, json.getString("name"),
                    json.getString("origin"))));
          }
        });
  }
}

As we can see, the update method takes two JSON ob­jects as pa­ra­me­ter:

1. The first one de­notes the query (here we se­lect a sin­gle doc­u­ment using its id).
2. The sec­ond ob­ject ex­presses the change to apply to the se­lected doc­u­ment. It uses a mongo syn­tax. In our case, we up­date the doc­u­ment using the $set op­er­a­tor.

I def­i­nitely rec­om­mend to have a look to the Mon­goDB doc­u­men­ta­tion, es­pe­cially:

• Query syn­tax doc­u­men­ta­tion
• Up­date syn­tax doc­u­men­ta­tion

Time for configuration

Well, the code is mi­grated, but we still need to up­date the con­fig­u­ra­tion. With JDBC we passed the JDBC url and the dri­ver class in the con­fig­u­ra­tion. With mongo, we need to con­fig­ure the connection_string - the mongo:// url on which the ap­pli­ca­tion is con­nected, and db_name - a name for the data source.

Let’s start by the unit test. Edit the MyFirstVerticleTest file and add the fol­low­ing code:

private static MongodProcess MONGO;
private static int MONGO_PORT = 12345;
@BeforeClass
public static void initialize() throws IOException {
  MongodStarter starter = MongodStarter.getDefaultInstance();
  IMongodConfig mongodConfig = new MongodConfigBuilder()
      .version(Version.Main.PRODUCTION)
      .net(new Net(MONGO_PORT, Network.localhostIsIPv6()))
      .build();
  MongodExecutable mongodExecutable =
      starter.prepare(mongodConfig);
  MONGO = mongodExecutable.start();
}

@AfterClass
public static void shutdown() {  MONGO.stop(); }

Be­fore our tests, we start (pro­gram­mat­i­cally) a mongo data­base on the port 12345. When all our tests have been ex­e­cuted, we shut­down the data­base.

So now that the mongo server is man­aged, we need to to give the right con­fig­u­ra­tion to our ver­ti­cle. Up­date the DeploymentOption in­stance with:

DeploymentOptions options = new DeploymentOptions()
    .setConfig(new JsonObject()
        .put("http.port", port)
        .put("db_name", "whiskies-test")
        .put("connection_string",
            "mongodb://localhost:" + MONGO_PORT)
);

That’s all for the unit tests.

For the integration-​test, we are using an ex­ter­nal­ized json file. Edit the src/test/resources/my-it-config.json with the fol­low­ing con­tent:

{
  "http.port": ${http.port},
  "db_name": "whiskies-it",
  "connection_string": "mongodb://localhost:37017"
}

No­tice the port we are using for the mongo server. This port was con­fig­ured in the pom.xml file.

Last but not least, we still have a con­fig­u­ra­tion file to edit: the con­fig­u­ra­tion you use to launch the ap­pli­ca­tion in production:

{
  "http.port": 8082,
  "db_name": "whiskies",
  "connection_string": "mongodb://localhost:27017"
}

Here you would need to edit the localhost:27017 with the right url for your mongo server.

Time for a run

It’s time to pack­age and run the ap­pli­ca­tion and check that every­thing works as ex­pected. Let’s pack­age the ap­pli­ca­tion using:

mvn clean verify

And then to launch it, start your mongo server and launch:

java -jar target/my-first-app-1.0-SNAPSHOT-fat.jar \
  -conf src/main/conf/my-application-conf.json

If you are, like me, using docker / docker-​machine for al­most every­thing, edit the con­fig­u­ra­tion file to refer to the right host (lo­cal­host for docker, the docker-​machine ip if you use docker-​machine) and then launch:

docker run -d -p 27017:27017 mongo
java -jar target/my-first-app-1.0-SNAPSHOT-fat.jar \
  -conf src/main/conf/my-application-conf.json
# or
java -jar target/my-first-app-1.0-SNAPSHOT-fat.jar \
  -conf src/main/conf/my-application-conf-docker-machine.json

That’s all folks!

We are reach­ing the end of this post. We saw how you can use the vert-​mongo-client to ac­cess asyn­chro­nously data stored in­side a mongo data­base as well as in­sert­ing/up­dat­ing this data. Now you have the choice be­tween JDBC or Mongo. In ad­di­tion, vert.x pro­vides a client for Redis.

Next time, we will see how the ver­ti­cle class can be split in two ver­ti­cles in order to bet­ter or­ga­nize your code. The in­ter­ac­tion be­tween the two ver­ti­cles will uses ser­vices.

Stay tuned & Happy cod­ing!