Real-time bidding with Websockets and Vert.x

The ex­pec­ta­tions of users for in­ter­ac­tiv­ity with web ap­pli­ca­tions have changed over the past few years. Users dur­ing bid­ding in auc­tion no longer want to press the re­fresh but­ton to check if the price has changed or the auc­tion is over. This made bid­ding dif­fi­cult and less fun. In­stead, they ex­pect to see the up­dates in ap­pli­ca­tion in real-​time.

In this ar­ti­cle I want to show how to cre­ate a sim­ple ap­pli­ca­tion that pro­vides real-​time bid­ding. We will use Web­Sock­ets, SockJS and Vert.x.

We will cre­ate a front-​end for fast bid­ding that com­mu­ni­cates with a micro-​service writ­ten in Java and based on Vert.x.

What are Websockets?

Web­Socket is asyn­chro­nous, bidi­rec­tional, full-​duplex pro­to­col that pro­vides a com­mu­ni­ca­tion chan­nel over a sin­gle TCP con­nec­tion. With the Web­Socket API it pro­vides bidi­rec­tional com­mu­ni­ca­tion be­tween the web­site and a re­mote server.

Web­Sock­ets solve many prob­lems which pre­vented the HTTP pro­to­col from being suit­able for use in mod­ern, real-​time ap­pli­ca­tions. Workarounds like polling are no longer needed, which sim­pli­fies ap­pli­ca­tion ar­chi­tec­ture. Web­Sock­ets do not need to open mul­ti­ple HTTP con­nec­tions, they pro­vide a re­duc­tion of un­nec­es­sary net­work traf­fic and re­duce la­tency.

Websocket API vs SockJS

Un­for­tu­nately, Web­Sock­ets are not sup­ported by all web browsers. How­ever, there are li­braries that pro­vide a fall­back when Web­Sock­ets are not avail­able. One such li­brary is SockJS. SockJS starts from try­ing to use the Web­Socket pro­to­col. How­ever, if this is not pos­si­ble, it uses a va­ri­ety of browser-​specific trans­port pro­to­cols. SockJS is a li­brary de­signed to work in all mod­ern browsers and in en­vi­ron­ments that do not sup­port Web­Socket pro­to­col, for in­stance be­hind re­stric­tive cor­po­rate proxy. SockJS pro­vides an API sim­i­lar to the stan­dard Web­Socket API.

Frontend to fast bidding

Auc­tion web page con­tains the bid­ding form and some sim­ple JavaScript which loads cur­rent price from the ser­vice, opens an event bus con­nec­tion to the SockJS server and of­fers bid­ding. HTML source code of sam­ple web page on which we bid might look like this:

<h3>Auction 1</h3>
<div id="error_message"></div>
<form>
    Current price:
    <span id="current_price"></span>
    <div>
        <label for="my_bid_value">Your offer:</label>
        <input id="my_bid_value" type="text">
        <input type="button" onclick="bid();" value="Bid">
    </div>
    <div>
        Feed:
        <textarea id="feed" rows="4" cols="50" readonly></textarea>
    </div>
</form>

We use the vertx-eventbus.js li­brary to cre­ate a con­nec­tion to the event bus. vertx-eventbus.js li­brary is a part of the Vert.x dis­tri­b­u­tion. vertx-eventbus.js in­ter­nally uses SockJS li­brary to send the data to the SockJS server. In the code snip­pet below we cre­ate an in­stance of the event bus. The pa­ra­me­ter to the con­struc­tor is the URI where to con­nect to the event bus. Then we reg­is­ter the han­dler lis­ten­ing on ad­dress auction.<auction_id>. Each client has a pos­si­bil­ity of reg­is­ter­ing at mul­ti­ple ad­dresses e.g. when bid­ding in the auc­tion 1234, they reg­is­ter on the ad­dress auction.1234 etc. When data ar­rives in the han­dler, we change the cur­rent price and the bid­ding feed on the auc­tion’s web page.

function registerHandlerForUpdateCurrentPriceAndFeed() {
    var eventBus = new EventBus('http://localhost:8080/eventbus');
    eventBus.onopen = function () {
        eventBus.registerHandler('auction.' + auction_id, function (error, message) {
            document.getElementById('current_price').innerHTML = JSON.parse(message.body).price;
            document.getElementById('feed').value += 'New offer: ' + JSON.parse(message.body).price + '\n';
        });
    }
};

Any user at­tempt to bid gen­er­ates a PATCH Ajax re­quest to the ser­vice with in­for­ma­tion about the new offer made at auc­tion (see bid() func­tion). On the server side we pub­lish this in­for­ma­tion on the event bus to all clients reg­is­tered to an ad­dress. If you re­ceive an HTTP re­sponse sta­tus code other than 200 (OK), an error mes­sage is dis­played on the web page.

function bid() {
    var newPrice = document.getElementById('my_bid_value').value;

    var xmlhttp = (window.XMLHttpRequest) ? new XMLHttpRequest() : new ActiveXObject("Microsoft.XMLHTTP");
    xmlhttp.onreadystatechange = function () {
        if (xmlhttp.readyState == 4) {
            if (xmlhttp.status != 200) {
                document.getElementById('error_message').innerHTML = 'Sorry, something went wrong.';
            }
        }
    };
    xmlhttp.open("PATCH", "http://localhost:8080/api/auctions/" + auction_id);
    xmlhttp.setRequestHeader("Content-Type", "application/json");
    xmlhttp.send(JSON.stringify({price: newPrice}));
};

Auction Service

SockJS client re­quires the server-​side part. Now we are going to cre­ate a light-​weight REST­ful auc­tion ser­vice. We will send and re­trieve data in JSON for­mat. Let’s start by cre­at­ing a ver­ti­cle. First we need to in­herit from AbstractVerticle and over­ride the start method. Each ver­ti­cle in­stance has a mem­ber vari­able called vertx. This pro­vides ac­cess to the Vert.x core API. For ex­am­ple, to cre­ate an HTTP server you call the createHttpServer method on vertx in­stance. To tell the server to lis­ten on port 8080 for in­com­ing re­quests you use the listen method.

We need a router with routes. A router takes an HTTP re­quest and finds the first match­ing route. The route can have a han­dler as­so­ci­ated with it, which re­ceives the re­quest (e.g. route that matches path /eventbus/* is as­so­ci­ated with eventBusHandler).

We can do some­thing with the re­quest, and then, end it or pass it to the next match­ing han­dler.

If you have a lot of han­dlers it makes sense to split them up into mul­ti­ple routers.

You can do this by mount­ing a router at a mount point in an­other router (see auctionApiRouter that cor­re­sponds to /api mount point in code snip­pet below).

Here’s an ex­am­ple ver­ti­cle:

public class AuctionServiceVerticle extends AbstractVerticle {

    @Override
    public void start() {
        Router router = Router.router(vertx);

        router.route("/eventbus/*").handler(eventBusHandler());
        router.mountSubRouter("/api", auctionApiRouter());
        router.route().failureHandler(errorHandler());
        router.route().handler(staticHandler());

        vertx.createHttpServer().requestHandler(router::accept).listen(8080);
    }

    //…
}

Now we’ll look at things in more de­tail. We’ll dis­cuss Vert.x fea­tures used in ver­ti­cle: error han­dler, SockJS han­dler, body han­dler, shared data, sta­tic han­dler and rout­ing based on method, path etc.

Error handler

As well as set­ting han­dlers to han­dle re­quests you can also set a han­dler for fail­ures in rout­ing. Fail­ure in rout­ing oc­curs if a han­dler throws an ex­cep­tion, or if a han­dler calls fail method. To ren­der error pages we use error han­dler pro­vides by Vert.x:

private ErrorHandler errorHandler() {
    return ErrorHandler.create();
}

SockJS handler

Vert.x pro­vides SockJS han­dler with the event bus bridge which ex­tends the server-​side Vert.x event bus into client side JavaScript.

Con­fig­ur­ing the bridge to tell it which mes­sages should pass through is easy. You can spec­ify which matches you want to allow for in­bound and out­bound traf­fic using the BridgeOptions. If a mes­sage is out­bound, be­fore send­ing it from the server to the client side JavaScript, Vert.x will look through any out­bound per­mit­ted matches. In code snip­pet below we allow any mes­sages from ad­dresses start­ing with “auc­tion.” and end­ing with dig­its (e.g. auction.1, auction.100 etc).

If you want to be no­ti­fied when an event oc­curs on the bridge you can pro­vide a han­dler when call­ing the bridge. For ex­am­ple, SOCKET_CREATED event will occur when a new SockJS socket is cre­ated. The event is an in­stance of Future. When you are fin­ished han­dling the event you can com­plete the fu­ture with “true” to en­able fur­ther pro­cess­ing.

To start the bridge sim­ply call bridge method on the SockJS han­dler:

private SockJSHandler eventBusHandler() {
    BridgeOptions options = new BridgeOptions()
            .addOutboundPermitted(new PermittedOptions().setAddressRegex("auction\\.[0-9]+"));
    return SockJSHandler.create(vertx).bridge(options, event -> {
         if (event.type() == BridgeEventType.SOCKET_CREATED) {
            logger.info("A socket was created");
        }
        event.complete(true);
    });
}

Body handler

The Body­Han­dler al­lows you to re­trieve the re­quest body, limit the body size and to han­dle the file up­load. Body han­dler should be on a match­ing route for any re­quests that re­quire this func­tion­al­ity. We need Body­Han­dler dur­ing the bid­ding process (PATCH method re­quest /auctions/<auction_id> con­tains re­quest body with in­for­ma­tion about a new offer made at auc­tion). Cre­at­ing a new body han­dler is sim­ple:

BodyHandler.create();

If re­quest body is in JSON for­mat, you can get it with getBodyAsJson method.

Shared data

Shared data con­tains func­tion­al­ity that al­lows you to safely share the data be­tween dif­fer­ent ap­pli­ca­tions in the same Vert.x in­stance or across a clus­ter of Vert.x in­stances. Shared data in­cludes local shared maps, dis­trib­uted, cluster-​wide maps, asyn­chro­nous cluster-​wide locks and asyn­chro­nous cluster-​wide coun­ters.

To sim­plify the ap­pli­ca­tion we use the local shared map to save in­for­ma­tion about auc­tions. The local shared map al­lows you to share data be­tween dif­fer­ent ver­ti­cles in the same Vert.x in­stance. Here’s an ex­am­ple of using a shared local map in an auc­tion ser­vice:

public class AuctionRepository {

    //…

    public Optional<Auction> getById(String auctionId) {
        LocalMap<String, String> auctionSharedData = this.sharedData.getLocalMap(auctionId);

        return Optional.of(auctionSharedData)
            .filter(m -> !m.isEmpty())
            .map(this::convertToAuction);
    }

    public void save(Auction auction) {
        LocalMap<String, String> auctionSharedData = this.sharedData.getLocalMap(auction.getId());

        auctionSharedData.put("id", auction.getId());
        auctionSharedData.put("price", auction.getPrice());
    }

    //…
}

If you want to store auc­tion data in a data­base, Vert.x pro­vides a few dif­fer­ent asyn­chro­nous clients for ac­cess­ing var­i­ous data stor­ages (Mon­goDB, Redis or JDBC client).

Auction API

Vert.x lets you route HTTP re­quests to dif­fer­ent han­dlers based on pat­tern match­ing on the re­quest path. It also en­ables you to ex­tract val­ues from the path and use them as pa­ra­me­ters in the re­quest. Cor­re­spond­ing meth­ods exist for each HTTP method. The first match­ing one will re­ceive the re­quest. This func­tion­al­ity is par­tic­u­larly use­ful when de­vel­op­ing REST-​style web ap­pli­ca­tions.

To ex­tract pa­ra­me­ters from the path, you can use the colon char­ac­ter to de­note the name of a pa­ra­me­ter. Reg­u­lar ex­pres­sions can also be used to ex­tract more com­plex matches. Any pa­ra­me­ters ex­tracted by pat­tern match­ing are added to the map of re­quest pa­ra­me­ters.

Consumes de­scribes which MIME types the han­dler can con­sume. By using produces you de­fine which MIME types the route pro­duces. In the code below the routes will match any re­quest with content-type header and accept header that matches application/json.

Let’s look at an ex­am­ple of a sub­router mounted on the main router which was cre­ated in start method in ver­ti­cle:

private Router auctionApiRouter() {
    AuctionRepository repository = new AuctionRepository(vertx.sharedData());
    AuctionValidator validator = new AuctionValidator(repository);
    AuctionHandler handler = new AuctionHandler(repository, validator);

    Router router = Router.router(vertx);
    router.route().handler(BodyHandler.create());

    router.route().consumes("application/json");
    router.route().produces("application/json");

    router.get("/auctions/:id").handler(handler::handleGetAuction);
    router.patch("/auctions/:id").handler(handler::handleChangeAuctionPrice);

    return router;
}

The GET re­quest re­turns auc­tion data, while the PATCH method re­quest al­lows you to bid up in the auc­tion. Let’s focus on the more in­ter­est­ing method, namely handleChangeAuctionPrice. In the sim­plest terms, the method might look like this:

public void handleChangeAuctionPrice(RoutingContext context) {
    String auctionId = context.request().getParam("id");
    Auction auction = new Auction(
        auctionId,
        new BigDecimal(context.getBodyAsJson().getString("price"))
    );

    this.repository.save(auction);
    context.vertx().eventBus().publish("auction." + auctionId, context.getBodyAsString());

    context.response()
        .setStatusCode(200)
        .end();
}

PATCH re­quest to /auctions/1 would re­sult in vari­able auctionId get­ting the value 1. We save a new offer in the auc­tion and then pub­lish this in­for­ma­tion on the event bus to all clients reg­is­tered on the ad­dress on the client side JavaScript. After you have fin­ished with the HTTP re­sponse you must call the end func­tion on it.

Static handler

Vert.x pro­vides the han­dler for serv­ing sta­tic web re­sources. The de­fault di­rec­tory from which sta­tic files are served is webroot, but this can be con­fig­ured. By de­fault the sta­tic han­dler will set cache head­ers to en­able browsers to cache files. Set­ting cache head­ers can be dis­abled with setCachingEnabled method. To serve the auc­tion HTML page, JS files (and other sta­tic files) from auc­tion ser­vice, you can cre­ate a sta­tic han­dler like this:

private StaticHandler staticHandler() {
    return StaticHandler.create()
        .setCachingEnabled(false);
}

Let’s run!

Full ap­pli­ca­tion code is avail­able on github.

Clone the repos­i­tory and run ./gradlew run.

Open one or more browsers and point them to http://localhost:8080. Now you can bid in auc­tion:

Summary

This ar­ti­cle presents the out­line of a sim­ple ap­pli­ca­tion that al­lows real-​time bid­ding. We cre­ated a light­weight, high-​performance and scal­able micro-​service writ­ten in Java and based on Vert.x. We dis­cussed what Vert.x of­fers, among oth­ers, a dis­trib­uted event bus and an el­e­gant API that al­lows you to cre­ate ap­pli­ca­tions in no time.