The Challenge

Previously, on Twootr, Joe had wanted a modern online communication system to be implemented. The previous chapter presented a potential design for Twootr and the implementation of the core business domain was described, including driving out that design through tests. You learned about some of the design and data modeling decisions involved and how to break down the initial problem and structure your solution. That didn’t cover the whole of the Twootr project, so it’s up to this chapter to complete the narrative.

The Goal

This chapter extends and completes the progress made in the previous chapter by helping you understand about the following topics:

• Avoiding coupling with the Dependency Inversion Principle and Dependency Injection

• Persistence with the Repository pattern and the Query Object pattern.

• A brief introduction to functional programming that will show you how you can make use of the ideas from this in a Java-specific context and a real application.

Persistence and the Repository Pattern

So we’ve now got a system that can support much of the core twooting operations. Unfortunately, if we restart the Java process in any way all the twoots and user information is lost. We need a way of persisting the information that we’re storing in order to survive a restart. Earlier in the discussion of software architecture we talked about ports and adapters and how we would like to keep the core of our application agnostic of the storage backend. There’s, in fact, a commonly used pattern that helps us do this: the Repository pattern.

The Repository pattern defines an interface between the domain logic and storage backend. In addition to allowing us to use a different storage backend over time as our application evolves, this approach offers several advantages:

• Centralizing logic for mapping data from our storage backend to the domain model.

• Enables unit testing of core business logic without having to spin up a database. This can speed up the execution of tests.

• Improves maintainability and readability by keeping each class single responsibility.

You can think of a repository as a being like a collection of objects, but instead of just storing the objects in memory, the repository persists them somewhere. When evolving the design of our application we drove the design of the repositories through tests; however, to save time here we will just describe the final implementation. Since a repository is a collection of objects we need two of them in Twootr: one to store User objects and one for Twoot objects. Most repositories have a series of common operations that are implemented:

add()

Stores a new instance of the object into the repository.

get()

Looks up a single object based on an identifier.

delete()

Deletes an instance from the persistence backend.

update()

Ensures that the values saved for this object are equal to the instance fields.

Some people use the acronym CRUD to describe these kind of operations. This stands for Create, Read, Update, and Delete. We’ve used add and get instead of create and read as the naming is more inline with common Java usage, for example, in the collections framework.

public interface UserRepository extends AutoCloseable {
    boolean add(User user);

    Optional<User> get(String userId);

    void update(User user);

    void clear();

    FollowStatus follow(User follower, User userToFollow);
}

public interface TwootRepository {
    Twoot add(String id, String userId, String content);

    Optional<Twoot> get(String id);

    void delete(Twoot twoot);

    void query(TwootQuery twootQuery, Consumer<Twoot> callback);

    void clear();
}

public interface AbstractRepository<T>
{
    void add(T value);

    Optional<T> get(String id);

    void update(T value);

    void delete(T value);
}

List<Twoot> twootsForLogon(User user);

List<Twoot> twootsFromUsersAfterPosition(Set<String> inUsers, Position lastSeenPosition);

List<Twoot> query(TwootQuery query);

public class TwootQuery {
    private Set<String> inUsers;
    private Position lastSeenPosition;

    public Set<String> getInUsers() {
        return inUsers;
    }

    public Position getLastSeenPosition() {
        return lastSeenPosition;
    }


    public TwootQuery inUsers(final Set<String> inUsers) {
        this.inUsers = inUsers;

        return this;
    }

    public TwootQuery inUsers(String... inUsers) {
        return inUsers(new HashSet<>(Arrays.asList(inUsers)));
    }

    public TwootQuery lastSeenPosition(final Position lastSeenPosition) {
        this.lastSeenPosition = lastSeenPosition;

        return this;
    }

    public boolean hasUsers() {
        return inUsers != null && !inUsers.isEmpty();
    }
}

void query(TwootQuery twootQuery, Consumer<Twoot> callback);

twootRepository.query(
    new TwootQuery()
        .inUsers(user.getFollowing())
        .lastSeenPosition(user.getLastSeenPosition()),
    user::receiveTwoot);

Functional Programming

Functional programming is a style of computer programming that treats methods as operating like mathematical functions. This means that it avoids mutable state and changing data. You can program in this style in any language, but some programming languages offer features to help make it easier and better—we call those functional programming languages. Java isn’t a functional programming language, but in version 8, 20 years after it was first released, it started to add a number of features that helped make functional programming in Java a reality. Those features include lambda expressions, the Streams and Collectors API, and the Optional class. In this section we’ll talk a little bit about how those functional programming features can be used and how we use them in Twootr.

There are limits to the level of abstractions that library writers could use in Java before Java 8. A good example of this was the lack of efficient parallel operations over large collections of data. Java from 8 onward allows you to write complex collection-processing algorithms, and simply by changing a single method call you can efficiently execute this code on multicore CPUs. In order to enable writing of these kinds of bulk data parallel libraries, however, Java needed a new language change: lambda expressions.

Of course there’s a cost, in that you must learn to write and read lambda-enabled code, but it’s a good trade-off. It’s easier for programmers to learn a small amount of new syntax and a few new idioms than to have to handwrite a large quantity of complex thread-safe code. Good libraries and frameworks have significantly reduced the cost and time associated with developing enterprise business applications, and any barrier to developing easy-to-use and efficient libraries should be removed.

Abstraction is a concept that is familiar to anyone who does object-oriented programming. The difference is that object-oriented programming is mostly about abstracting over data, while functional programming is mostly about abstracting over behavior. The real world has both of these things, and so do our programs, so we can and should learn from both influences.

There are other benefits to this new abstraction as well. For many of us who aren’t writing performance-critical code all the time, these are more important wins. You can write easier-to-read code—code that spends time expressing the intent of its business logic rather than the mechanics of how it’s achieved. Easier-to-read code is also easier to maintain, more reliable, and less error-prone than code that is more difficult to read.

public interface ReceiverEndPoint {
    void onTwoot(Twoot twoot);
}

public class PrintingEndPoint implements ReceiverEndPoint {
    @Override
    public void onTwoot(final Twoot twoot) {
        System.out.println(twoot.getSenderId() + ": " + twoot.getContent());
    }
}

        final ReceiverEndPoint anonymousClass = new ReceiverEndPoint() {
            @Override
            public void onTwoot(final Twoot twoot) {
                System.out.println(twoot.getSenderId() + ": " + twoot.getContent());
            }
        };

        final ReceiverEndPoint lambda =
            twoot -> System.out.println(twoot.getSenderId() + ": " + twoot.getContent());

twoot -> twoot.getContent()

Twoot::getContent

(user, twootr) -> new SenderEndPoint(user, twootr)

SenderEndPoint::new

Execute Around

The Execute Around pattern is a common functional design pattern. You may encounter a situation where you have common initialization and cleanup code that you always want to do, but parameterize different business logic that runs within the initialization and cleanup code. An example of the general pattern is shown in Figure 7-1. There are a number of example situations in which you can use execute around, for example:

Files

Open a file before you use it, and close it when you’ve finished using the file. You may also want to log an exception when something goes wrong. The parameterized code can read from or write to the file.

Locks

Acquire a lock before your critical section, release the lock after your critical section. The parameterized code is the critical section.

Database connections

Open a connection to a database upon initialization, close it when finished. This is often even more useful if you pool your database connections as it also allows your open logic to also retrieve the connection from your pool.

Figure 7-1. Execute Around pattern

Because the initialization and cleanup logic is being used in many places, it is possible to get into a situation where this logic is duplicated. This means that if you want to modify this common initialization or cleanup code, then you will have to modify multiple different parts of your application. It also exposes the risk that these different code snippets could become inconsistent, introducing potential bugs into your application.

The Execute Around pattern solves this problem by extracting a common method that defines both the initialization and cleanup code. This method takes a parameter containing the behavior that differs between use cases of the same overall pattern. The parameter will use an interface to enable it to be implemented by different blocks of code, usually using lambda expressions.

Example 7-18 shows a concrete example of an extract method. This is used within Twootr in order to run SQL statements against the database. It creates a prepared statement object for a given SQL statement and and then runs our extractor behavior on the statement. The extractor is just a callback that extracts a result, i.e., reads some data from the database, using the PreparedStatement.

Example 7-18. Use of the Execute Around pattern in the extract method

    <R> R extract(final String sql, final Extractor<R> extractor) {
        try (var stmt = conn.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS)) {
            stmt.clearParameters();
            return extractor.run(stmt);
        } catch (SQLException e) {
            throw new IllegalStateException(e);
        }
    }

    private String usersTupleLoop(final Set<String> following) {
        List<String> quotedIds = new ArrayList<>();
        for (String id : following) {
            quotedIds.add("'" + id + "'");
        }
        return '(' + String.join(",", quotedIds) + ')';
    }

    private String usersTuple(final Set<String> following) {
        return following
            .stream()
            .map(id -> "'" + id + "'")
            .collect(Collectors.joining(",", "(", ")"));
    }

    public void queryLoop(final TwootQuery twootQuery, final Consumer<Twoot> callback) {
        if (!twootQuery.hasUsers()) {
            return;
        }

        var lastSeenPosition = twootQuery.getLastSeenPosition();
        var inUsers = twootQuery.getInUsers();

        for (Twoot twoot : twoots) {
            if (inUsers.contains(twoot.getSenderId()) &&
                twoot.isAfter(lastSeenPosition)) {
                callback.accept(twoot);
            }
        }
    }

    @Override
    public void query(final TwootQuery twootQuery, final Consumer<Twoot> callback) {
        if (!twootQuery.hasUsers()) {
            return;
        }

        var lastSeenPosition = twootQuery.getLastSeenPosition();
        var inUsers = twootQuery.getInUsers();

        twoots
            .stream()
            .filter(twoot -> inUsers.contains(twoot.getSenderId()))
            .filter(twoot -> twoot.isAfter(lastSeenPosition))
            .forEach(callback);
    }

Object accumulator = initialValue;
for (Object element : collection) {
 accumulator = combine(accumulator, element);
}

    private final BinaryOperator<Position> maxPosition = maxBy(comparingInt(Position::getValue));

    Twoot combineTwootsBy(final List<Twoot> twoots, final String senderId, final String newId) {
        return twoots
            .stream()
            .reduce(
                new Twoot(newId, senderId, "", INITIAL_POSITION),
                (acc, twoot) -> new Twoot(
                    newId,
                    senderId,
                    twoot.getContent() + acc.getContent(),
                    maxPosition.apply(acc.getPosition(), twoot.getPosition())));
    }

        user.followers()
            .filter(User::isLoggedOn)
            .forEach(follower ->
            {
                follower.receiveTwoot(twoot);
                userRepository.update(follower);
            });

Optional<String> a = Optional.of("a");

assertEquals("a", a.get());

Optional emptyOptional = Optional.empty();
Optional alsoEmpty = Optional.ofNullable(null);

assertFalse(emptyOptional.isPresent());

// a is defined above
assertTrue(a.isPresent());

assertEquals("b", emptyOptional.orElse("b"));
assertEquals("c", emptyOptional.orElseGet(() -> "c"));

        var authenticatedUser = userRepository
            .get(userId)
            .filter(userOfSameId ->
            {
                var hashedPassword = KeyGenerator.hash(password, userOfSameId.getSalt());
                return Arrays.equals(hashedPassword, userOfSameId.getPassword());
            });

        authenticatedUser.ifPresent(user ->
        {
            user.onLogon(receiverEndPoint);
            twootRepository.query(
                new TwootQuery()
                    .inUsers(user.getFollowing())
                    .lastSeenPosition(user.getLastSeenPosition()),
                user::receiveTwoot);
            userRepository.update(user);
        });

        return authenticatedUser.map(user -> new SenderEndPoint(user, this));

User Interface

Throughout this chapter we’ve avoided talking too much about the user interface to this system, because we’re focused on the design of the core problem domain. That said, it’s worth delving a little into what the example project delivers as part of its UI just in order to understand how the event modeling fits together. In our example project we ship a single-page website that uses JavaScript to implement its dynamic functionality. In order to keep things simple and not delve too much into the myriad framework wars, we’ve just used jquery to update the raw HTML page, but kept a simple separation of concerns in the code.

When you browse to the Twootr web page it connects back to the host using WebSockets. These were one of the event communication choices discussed back in “From Events to Design”. All the code for communicating with it lies in the web_adapter subpackage of chapter_06. The WebSocketEndPoint class implements the ReceiverEndPoint and also invokes any needed methods on the SenderEndPoint. For example, when the ReceiverEndPoint receives and parses a message to follow another user it invokes the SenderEndPoint.onFollow(), passing the username through. The returned enum—FollowStatus then gets converted into a wire format response and written down the WebSocket connection.

All communication between the JavaScript frontend and the server is done using the JavaScript Object Notation (JSON) standard. JSON was chosen as it’s very easy for a JavaScript UI to deserialize or serialize.

Within the WebSocketEndPoint we need to map to and from JSON within Java code. There are many libraries that can be used for this purpose, here we’ve chosen the Jackson library, which is commonly used and well maintained. JSON is often used in applications that take a request/response approach rather than an event-driven approach as well. In our case we manually extract the fields from the JSON object to keep things simple, but its also possible to use a higher-level JSON API, such as a binding API.

Dependency Inversion and Dependency Injection

We’ve talked a lot about decoupling patterns in this chapter. Our overall application uses the Ports and Adapters pattern and the Repository pattern to decouple business logic away from implementation details. There is in fact a large, unifying principle that we can think of when we see these patterns—Dependency Inversion. The Dependency Inversion Principle is the final of our five SOLID patterns that we’ve talked about in this book, and like the others was introduced by Robert Martin. It states that:

• High-level modules should not depend upon low-level modules. Both should depend upon abstractions.

• Abstractions should not depend upon details. Details should depend upon abstractions.

The principle is called an inversion because in traditional imperative, structured programming it is often the case that high-level modules compose down to produce low-level modules. It’s often a side effect of the top-down design that we talked about in this chapter. You split up a big problem into different subproblems, write a module to solve each of those subproblems, and then the main problem (the high-level module) depends on the subproblems (the low-level modules).

In the design of Twootr we’ve avoided this problem through the introduction of abstractions. We have a high-level entry point class, called Twootr, and it doesn’t depend upon the low-level modules such as our DataUserRepository. It depends upon the abstraction—the UserRepository interface. We perform the same inversion at the UI port. Twootr doesn’t depend upon the WebSocketEndPoint—it depends upon the ReceiverEndPoint. We program to the interface, not the implementation.

A related term is the concept of Dependency Injection, or DI. To understand what DI is and why we need it, let’s undertake a thought experiment on our design. Our architecture has determined that the main Twootr class needs to depend upon the UserRepository and TwootRepository in order to store User and Twoot objects. We have defined fields inside Twootr to store instances of these objects, as shown in Example 7-30. The question is, how do we instantiate them?

public class Twootr
{

    private final TwootRepository twootRepository;
    private final UserRepository userRepository;

The first strategy that we could use for populating the fields is to try and call constructors using the new keyword, as shown in Example 7-31. Here we’ve hardcoded the use of the database-based repositories into the codebase. Now most of the code in the class still programs to the interface, so we could change the implementation here quite easily without having to replace all our code, but it’s a bit of a hack. We have to always use the database repositories, which means our tests for the Twootr class depend upon the database and run more slowly.

Not only that, but if we want to ship different versions of Twootr to different customers—for example, an in-house Twootr for enterprise customers that uses SQL and a cloud-based version that uses a NoSQL backend—we would have to cut the builds from two different versions of the codebase. It’s not enough to just define interfaces and separate implementation—we also have to have a way of wiring up the right implementation in a way that doesn’t break our abstraction and decoupling approach.

public Twootr()
{
    this.userRepository = new DatabaseUserRepository();
    this.twootRepository = new DatabaseTwootRepository();
}

// How to start Twootr
Twootr twootr = new Twootr();

A commonly used design pattern for instantiating different dependencies is the Abstract Factory Design pattern. Example 7-32 demonstrates this pattern, where we have a factory method that we can use to create an instance of our interface using the getInstance() method. When we want to set up the right implementations to use, we can call a setInstance(). So, for example, we could use setInstance() in tests to create an in-memory implementation, in an on-premise installation to use a SQL database, or in our cloud environment to use a NoSQL database. We’ve decoupled the implementation from the interface and can call this wiring code wherever we want.

public Twootr()
{
    this.userRepository = UserRepository.getInstance();
    this.twootRepository = TwootRepository.getInstance();
}

// How to start Twootr
UserRepository.setInstance(new DatabaseUserRepository());
TwootRepository.setInstance(new DatabaseTwootRepository());
Twootr twootr = new Twootr();

Unfortunately this factory method approach has its downsides as well. For a start, we’ve now created a big ball of shared mutable state. Any situation where we want to run a single JVM with different Twootr instances with different dependencies isn’t possible. We’ve also coupled together lifetimes—perhaps we sometimes want to instantiate a new TwootRepository when we start Twootr, or perhaps we sometimes want to reuse an existing one. The factory method approach won’t let us directly do this. It can also become rather complicated to have a factory for every dependency that we want to create in our application.

This is where Dependency Injection comes in. DI can be thought of as an example of the Hollywood Agent approach—don’t call us, we’ll call you. With DI instead of creating dependencies explicitly or using factories to create them, you simply take a parameter and whatever instantiates your object has the responsibiltiy for passing in the required dependencies. It might be a test class’s setup method passing in a mock. It might be the main() method of your application passing in a SQL database implementation. An example of this in use with the Twootr class is shown in Example 7-33. Dependency Inversion is a strategy; Dependency Injection and the Repository pattern are tactics.

public Twootr(final UserRepository userRepository, final TwootRepository twootRepository)
{
    this.userRepository = userRepository;
    this.twootRepository = twootRepository;
}

// How to start Twootr
Twootr twootr = new Twootr(new DatabaseUserRepository(), new DatabaseTwootRepository());

Taking objects this way not only makes it easier to write tests for your objects, but it has the advantage of externalizing the creation of the objects themselves. This allows your application code or a framework to control when the UserRepository is created and what dependencies are wired into it. Many developers find it convenient to use DI frameworks, such as Spring and Guice, that offer many features on top of basic DI. For example, they define lifecycles for beans that standardize hooks to be called after the objects are instantiated or before they are destroyed if required. They can also offer scopes for objects, such as Singleton objects that are only instantiated once during the lifetime of a process or per-request objects. Furthermore, these DI frameworks often hook nicely into web development frameworks such as Dropwizard or Spring Boot and provide a productive out-of-the-box experience.

Packages and Build Systems

Java allows you to split your codebase into different packages. Throughout this book we’ve put the code for each chapter into its own package and Twootr is the first project where we’ve split out multiple subpackages within the project itself.

Here are the packages can you look at for the different components within the project:

• com.iteratrlearning.shu_book.chapter_06 is the top-level package for the project.

• com.iteratrlearning.shu_book.chapter_06.database contains the adapter for SQL database persistence.

• com.iteratrlearning.shu_book.chapter_06.in_memory contains the adapter for in-memory persistence.

• com.iteratrlearning.shu_book.chapter_06.web_adapter contains the adapter for the WebSockets-based UI.

Splitting out large projects into different packages can be helpful to structure code and make it easier for developers to find. Just in the same way that classes group together related methods and state, packages group together related classes. Packages should follow similar coupling and cohesion rules to your classes. Put classes in the same package when they’re likely to change at the same time and are related to the same structure. For example, in the Twootr project if we want to alter the SQL database persistence code we know we go to the database subpackage.

Packages also enable information hiding. We discussed the idea of having a package-scoped constructor method back in Example 4-3 in order to prevent objects from being instantiated outside of the package. We can also have package scoping for classes and methods. This prevents objects outside of the package from accessing the details of the class and helps us achieve loose coupling. For example, WebSocketEndPoint is package-scoped implementation of the ReceiverEndPoint interface that lives in the web_adapter package. No other code in the project should talk to this class directly—only through the ReceiverEndPoint interface that acts as the port.

Our approach of having a package per adapter in Twootr fits nicely with the hexagonal architectural pattern that we’ve used throughout this module. Not every application is hexagonal, however, and there are two common package structures that you may well encounter in other projects.

One very common approach to structuring packages is to structure them by layer—for example, grouping together all code that generates HTML views in a website into a views package, and all the code that relates to handling web requests into a controller package. Despite being popular, this can be a poor choice of structure as it results in poor coupling and cohesion. If you want to modify an existing web page to add an additional parameter and display a value based upon that parameter, you would end up touching the controller and the view packages, and probably several others as well.

An alternative way of structuring code is to group code by feature. So, for example, if you were writing an ecommerce site you might have a cart package for your shopping cart, a product package for code related to product listings, a payment package code related to taking card payments, etc. This can often be more cohesive. If you want to add support for receiving payment by Mastercard as well as Visa, then you would only need to modify the payment package.

In “Using Maven” we talked about how to set up a basic build structure using the Maven build tool. In the project structure for this book we have one Maven project and the different chapters of the book are different Java packages within that one project. That’s a nice and simple project structure that will work for a wide range of different software projects, but it’s not the only one. Both Maven and Gradle offer project structures that build and output many build artifacts from a single top-level project.

This can make sense if you want to deploy different build artifacts. For example, suppose you’ve got a client/server project where you want to have a single build that builds both the client and the server, but the client and the server are different binaries running on different machines. It’s best not to overthink or over-modularize build scripts, though.

They’re something that you and your team will be running on your machines regularly and the highest priority is for them to be simple, fast, and easy to use. That’s why we went down the route of having one single project for the entire book, rather than submodule per project.

Limitations and Simplifications

You’ve seen how we implement Twootr and learned about our design decisions along the way, but does that mean that the Twootr codebase that we’ve seen so far is the only or the best way to write it? Of course not! In fact, there are a number of limitations to our approach and simplifications that we’ve deliberately taken in order to make the codebase explainable in a single chapter.

For a start we’ve written Twootr as though it will be run on a single thread and completely ignored the issue of concurrency. In practice we may want to have multiple threads responding to and emitting events in our Twootr implementation. That way we can make use of modern multicore CPUs and serve a larger number customers on one box.

In a bigger-picture sense, we’ve also ignored any kind of failover that would allow our service to continue to run if the server that it was hosted on fell over. We’ve also ignored scalability. For example, requiring all our twoots have a single defined order is something that is easy and efficient to implement on a single server but would present a serious scalability/contention bottleneck. Similarly, seeing all the twoots when you log on would cause a bottleneck as well. What if you go on holiday for a week and when you log back on you get 20,000 twoots!

Addressing these issues in detail goes beyond the scope of this chapter. However, these are important topics if you wish to go further with Java, and we plan to address them in greater detail in future books in this series.

Takeaways

• You can now decouple data storage from business logic using the Repository pattern.

• You have seen implementations of two different types of repositories within this approach.

• You were introduced to the ideas of functional programming, including Java 8 Streams.

• You’ve seen how to structure a larger project with different packages.

Iterating on You

If you want to extend and solidify the knowledge from this section you could try one of the following activities.

Suppose that we had taken a pull model for Twootr. Instead of having messages continuously pushed out to a browser-based client over WebSockets, we had used HTTP to poll for the latest messages since a position.

• Brainstorm how our design would have changed. Try drawing a diagram of the different classes and how data would flow between them.

• Implement, using TDD, this alternative model for Twootr. You don’t need to implement the HTTP parts, just the underlying classes following this model.

Completing the Challenge

We built the product and it worked. Unfortunately, Joe realized when he launched that someone called Jack had released a similar product, with a similar name, taking billions in VC funding and with hundreds of millions of users. Jack only got there first by 11 years; it was bad luck for Joe, really.