Fast data access in Java with the Helidon microservices platform

Helidon SE and Helidon MP provide a very diverse array of methods for accessing data sources.

Helidon is a collection of Java libraries for writing microservices. Helidon 2.2.0 is out and provides a very diverse and flexible array of methods for accessing data. In this article, I’ll provide an overview of those data-access methods with references to lower-level material and examples.

First, though, here’s a bit of history.

The techniques and technologies used to access databases are based on various criteria that take into account the best fit for data access needs as well as the platform being run. Figure 1 is a time line of relevant events for Java platforms and data access.

Figure 1. A time line of Java platforms and data access

Jakarta EE is the current and future home of the Java EE platform, and Eclipse MicroProfile was created to extend the enterprise Java environment for developing microservices. Alongside these standards are frameworks such as Spring Boot, Helidon, and Micronaut. Figure 2 shows the wide range of technologies supported by Helidon. You can use these technologies based on the platform you are running or what you are most comfortable with for whatever reason.

Figure 2. Helidon supports a wide range of technologies.

Helidon is an open source project funded by Oracle, so it integrates and aligns with Oracle database technologies and features extremely well. This includes support for the Oracle Universal Connection Pool (UCP) for JDBC.

For Oracle Database, UCP provides performance (via connection pooling and tagging), scalability (via a front end for Database Resident Connection Pool [DRCP], a shared pool for multitenant databases, a swim lane for sharded databases, and sharding data sources), and high availability (via the Transaction Guard, Application Continuity, and Transparent Application Continuity features of Oracle Database).

Helidon SE and Helidon MP

There are Java SE and MicroProfile (MP) versions of Helidon. Helidon SE is designed to be transparent without using contexts and dependency integration (CDI), reactive programming, and functional-style programming. Helidon MP is for those familiar with Java EE; this version uses CDI.

Helidon MP’s CDI integration supports both HikariCP and UCP JDBC connection pools via the following Maven dependencies:

<dependency>

    <groupId>io.helidon.integrations.cdi</groupId>

    <artifactId>helidon-integrations-cdi-datasource-hikaricp</artifactId>

    <scope>runtime</scope>

</dependency> 

<dependency>

    <groupId>io.helidon.integrations.cdi</groupId>

    <artifactId>helidon-integrations-cdi-datasource-ucp</artifactId>

    <scope>runtime</scope>

</dependency>

The following is a sample META-INF/microprofile-config.properties file for a Helidon microservice that connects to an Oracle Autonomous Transaction Processing database. Note the MicroProfile naming convention of [objectype].[objectname].[objectproperty]:

oracle.ucp.jdbc.PoolDataSource.orderpdb.URL = jdbc:oracle:thin:@orderdb2_tp?TNS_ADMIN=/Users/msdataworkshop/Downloads/Wallet_orderdb2

oracle.ucp.jdbc.PoolDataSource.orderpdb.user = orderuser

oracle.ucp.jdbc.PoolDataSource.orderpdb.password = Welcome12345

oracle.ucp.jdbc.PoolDataSource.orderpdb.connectionFactoryClassName = oracle.jdbc.pool.OracleDataSource 

oracle.ucp.jdbc.PoolDataSource.orderpdb.inactiveConnectionTimeout = 60

If you don’t need data source/UCP-specific APIs, you could use the more generic javax.sql.DataSource, javax.sql.DataSource.orderpdb naming convention.

Here’s an example of microservice code where the data source reference configured above is automatically injected.

@Inject

@Named("orderpdb")

PoolDataSource atpOrderPdb;

These simple steps make data source access simple and dynamic and a perfect fit for cloud environments.

Kubernetes deployments

Here is a Kubernetes deployment YAML file for a microservice, which sets the environment variables (with values acquired from Kubernetes secrets, vault, and so on) that will override the defaults set in the Helidon configuration:

containers:

- name: order

  image: %DOCKER_REGISTRY%/order-helidon:0.1

  imagePullPolicy: Always

  env:

  - name: oracle.ucp.jdbc.PoolDataSource.orderpdb.user

    value: "ORDERUSER"

  - name: oracle.ucp.jdbc.PoolDataSource.orderpdb.password

    valueFrom:

      secretKeyRef:

        name: atp-user-cred-orderuser

        key: password

  - name: oracle.ucp.jdbc.PoolDataSource.orderpdb.URL

value: "jdbc:oracle:thin:@%ORDER_PDB_NAME%_tp?TNS_ADMIN=/msdataworkshop/creds"

Java EE/Jakarta EE Persistence API

The Java EE/Jakarta EE Persistence API (JPA), first released in 2009, is still the most widely used API for object-relational mapping. JPA is used not only in Java EE and Jakarta EE applications but also in other frameworks such as Spring Boot and Helidon MP.

Hibernate and Eclipse are the most popular implementations of JPA and are both supported by Helidon MP. Therefore, it is simple to migrate the use of JPA in applications that run JPA (whether they are from an application server, Spring Boot, or some other platform) to the lighter-weight Helidon.

Micronaut Data

Micronaut is a JVM-based, full-stack framework for building modular microservices and serverless applications. The Micronaut framework was released in late 2018, around the same time as Helidon, and has been very successful in providing a smooth transition from Spring Boot to its platform by providing the ability to do the following:

◉ Integrate Spring components into a Micronaut application

◉ Run Spring applications as Micronaut applications

◉ Expose Micronaut beans to a Spring application

Helidon has an integration layer that allows the use of Micronaut features from within a Helidon microservice. These features include Micronaut singleton injection, Micronaut interceptors, Micronaut bean validation and, of particular interest to the current topic, Micronaut Data.

The Micronaut Data database access toolkit precomputes queries and executes them with a thin runtime. Micronaut Data provides a general API for translating a query model into a query at compile time and provides runtime support for JPA/Hibernate and SQL/JDBC back ends.

Inspired by GORM and Spring Data, Micronaut Data improves on these two technologies by eliminating the runtime model that uses reflection, eliminating query translation that uses regular expressions and pattern matching, and adding type safety. The use of reflection in GORM and Spring Data for modeling relationships between entities leads to more memory consumption.

Because Micronaut Data does not perform query translation at runtime—it’s all precomputed—the performance gain can be significant. Micronaut Data JDBC provides nearly 2.5 times the performance of Spring Data; Micronaut Data JPA provides up to 40% better performance than Spring Data JPA. Also, startup times are at least 1.5 times faster than that of Spring Boot.

Micronaut Data supports GraalVM native images for both the JPA and JDBC implementations. The currently supported databases are H2, PostgreSQL, Oracle Database, MariaDB, and Microsoft SQL Server.

Some considerations for the use of direct Micronaut Data JDBC compared to JPA, aside from the performance and memory efficiencies mentioned, include the fact that JDBC has fewer dialects than JPA, is optimized for reads instead of writes (the opposite of JPA), and is better for startup times and, thus, serverless applications.

By integrating with Micronaut in this way, Helidon also inherits the simplicity of porting Spring Boot applications to Helidon. Tomas Langer has written a detailed article on this subject: “Helidon with Micronaut Data repositories.”

Helidon DB Client

Helidon SE is a compact toolkit that embraces the latest Java SE features, such as reactive streams, asynchronous and functional programming, and fluent-style APIs. Helidon DB Client API, designed for Helidon SE, simplifies how you work with databases by abstracting the type of the database. The API can be used for both relational and nonrelational databases.

Helidon DB Client provides

◉ Database configuration abstraction: Using a Helidon configuration allows database implementation-specific configuration options without the need to use database implementation-specific APIs. This allows for seamless switching between databases based on configuration.

◉ Statement configuration abstraction: Using a Helidon configuration allows the use of database-specific statements. This enables the use of different databases on different environments without changing code.

◉ A unified API for data access and querying: Thanks to the statement configuration abstraction, you can invoke a statement against relational or nonrelational databases (such as MySQL and MongoDB) without modifying source code.

◉ Reactive database access with backpressure: Currently the client supports a natively reactive driver for MongoDB and an executor service-wrapped support for any JDBC driver. This allows for seamless use of JDBC drivers in a reactive nonblocking environment, including support for backpressure (the result set is processed as requested by the query subscriber).

◉ Observability: The API offers support for health checks, metrics, and tracing.

Using the API with MongoDB simply requires adding the following Maven dependency:

<dependency>

            <groupId>io.helidon.dbclient</groupId>

                 <artifactId>helidon-dbclient-mongodb</artifactId>

       </dependency>

And a configuration such as this:

db:

  source: "mongoDb"

  connection:

    url: "mongodb://127.0.0.1:27017/pokemon"

  statements:

    # Insert operation contains collection name, operation type and data to be inserted.

    # Name variable is stored as MongoDB primary key attribute _id

    insert2: '{

            "collection": "pokemons",

            "value": {

                "_id": $name,

                "type": $type

            }

        }'

Here’s how you can code the Helidon DB Client and register the endpoints to access the data source:

Config dbConfig = config.get("db");

        DbClient dbClient = DbClient.builder(dbConfig)

                // add an interceptor to named statement(s)

                .addService(DbClientMetrics.counter().statementNames("select-all", "select-one"))

                // add an interceptor to statement type(s)

                .addService(DbClientMetrics.timer()

                                    .statementTypes(DbStatementType.DELETE, DbStatementType.UPDATE, DbStatementType.INSERT))

                // add an interceptor to all statements

                .addService(DbClientTracing.create())

                .build();

        HealthSupport health = HealthSupport.builder()

                .addLiveness(DbClientHealthCheck.create(dbClient))

                .build();

        return Routing.builder()

                .register(health)                   // Health at "/health"

                .register(MetricsSupport.create())  // Metrics at "/metrics"

                .register("/db", new PokemonService(dbClient))

                .build();

The Neo4j graph database

Helidon works with relational and nonrelational databases, SQL and NoSQL databases, and many more databases. These include JDBC, MongoDB via the MongoDB client, and Oracle Database JSON database via Oracle’s Simple Oracle Document Access (SODA) API – and recently, the Helidon project added integration with the graph database Neo4j. The Neo4j integration can be enabled with the following Maven dependencies:

<dependency>

                    <groupId>io.helidon.integrations.neo4j</groupId>

                    <artifactId>helidon-integrations-neo4j</artifactId>

                    <version>${helidon.version}</version>

                </dependency>

                <dependency>

                    <groupId>io.helidon.integrations.neo4j</groupId>

                    <artifactId>helidon-integrations-neo4j-health</artifactId>

                    <version>${helidon.version}</version>

                </dependency>

                <dependency>

                    <groupId>io.helidon.integrations.neo4j</groupId>

                    <artifactId>helidon-integrations-neo4j-metrics</artifactId>

                    <version>${helidon.version}</version>

      </dependency>

As with all Helidon features, configuration may be done in the application.yaml file:

neo4j:

  uri: bolt://localhost:7687

  authentication:

    username: neo4j

    password: secret

  pool:

metricsEnabled: true #should be explicitly enabled in Neo4j driver

Or it can be done via a MicroProfile configuration:

neo4j.uri=bolt://localhost:7687

neo4j.authentication.username=neo4j

neo4j.authentication.password: secret

neo4j.pool.metricsEnabled: true #should be explicitly enabled in Neo4j driver

Here’s how to use Neo4j with Helidon SE:

Neo4JSupport neo4j = Neo4JSupport.builder()

        .config(config)

        .helper(Neo4JMetricsSupport.create()) //optional support for Neo4j Metrics

        .helper(Neo4JHealthSupport.create()) //optional support for Neo4j Health checks

        .build();

 Routing.builder()

        .register(health)                   // Health at "/health"

        .register(metrics)                  // Metrics at "/metrics"

        .register(movieService)

        .build();

Neo4j can be used in Helidon SE by simply injecting the driver, for example:

@Inject

Driver driver;

Coherence Community Edition

Coherence Community Edition (CE) is a reliable and scalable platform for state management. It integrates with Helidon, GraalVM, Oracle Database, and Oracle Database cloud services.

Coherence CE contains the in-memory data grid functionality necessary to write microservices applications. Its features include

◉ Fault-tolerant automatic sharding

◉ Scalable caching, querying, aggregation, transactions, and in-place processing

◉ Polyglot programming on the grid side with GraalVM

◉ Persistence and data source integration

◉ Creating events, sending messages, and streaming

◉ A comprehensive security model

◉ Unlimited clients in polyglot languages and over REST

◉ Docker and Kubernetes support, with Kibana and Prometheus dashboards

Helidon 2.2.0 supports the MicroProfile GraphQL specification, which is an open source data query and manipulation language for APIs. A recent article, “Access Coherence using GraphQL,” by Tim Middleton, shows how to create and use GraphQL endpoints to access data in Coherence CE seamlessly with Helidon MP.

Messaging for Oracle Advanced Queuing

Due to the nature of microservices environments, messaging is often used for interservice communications, and that’s what the MicroProfile Reactive Messaging specification was designed for.

The Oracle Advanced Queuing (AQ) messaging system has been part of Oracle Database since 2002. The system, which supports Java Message Service (JMS), has features that make it perfect for microservices development, including

◉ Transactional queues and an “exactly once” delivery guarantee so you’re not forced to code logic for idempotency

◉ The ability to conduct database work and produce and consume messages within the same local transaction. This facilitates event sourcing, sagas, and general transaction communication patterns used in microservices with atomic (and, again, exactly-once delivery) guarantees not possible with other messaging and database systems

The integration of Oracle AQ with Helidon is powerful and simple to use. Here is an example where an Oracle AQ JMS (“order-placed”) message is received, the underlying JDBC connection is obtained and used to do database work (check inventory), and a response message (“inventory-exists”) is sent.

These three actions are conducted within the same local transaction such that all either fail or succeed, thus relieving an administrator or developer from needing to intervene and rectify a system due to a failure or add logic to a microservice to handle failures such as duplicate deliveries or inconsistent data.

Copy code snippet

Copied to ClipboardError: Could not CopyCopied to Clipboard

@Incoming("orderplaced")

@Outgoing("inventoryexists")

@Acknowledgment(Acknowledgment.Strategy.NONE)

public CompletionStage<Message<String>> reserveInventoryForOrder (AqMessage<String> msg) {

        return CompletableFuture.supplyAsync(() -> {

            Connection jdbcConnection = msg.getDBConnection(); // unique to AQ

   String inventoryStatus = getInventoryForOrder(msg, jdbcConnection);

            return Message.of(inventoryStatus, msg::ack);

        });

}

GraalVM Native Image

All the features mentioned in this article are compatible with GraalVM, which means that Helidon microservices using those features can be built into a GraalVM Native Image, a technology that performs an ahead-of-time compilation of Java code to create a standalone executable.

With the new Oracle Database 21c, GraalVM Native Image support also works with Oracle Universal Connection Pool (UCP) wallets and the Oracle Autonomous Transaction Processing cloud database service.

Integration with sagas and MicroProfile LRA

Applications that require data coordination between multiple microservices create challenges for data consistency and integrity. Those challenges necessitate changes in the transaction processing and data patterns used by them.

Traditional systems rely on two-phase commit or other extended architecture (XA) protocols that use synchronous communication, resource locking, and recovery via rollback or commit. While those protocols provide strong consistency and isolation, they do not scale well in a microservices environment due to the latency of held locks. That means such methods are suitable for only a small subset of microservices use cases—generally those with low throughput requirements.

The saga design pattern, by contrast, uses asynchronous communication and local resources only (thus, no distributed locks) and recovery via compensating actions. The saga pattern scales well, so it is well suited for long running transactions in a microservices environment. Additional application design considerations are necessary, however, for read isolation and compensation logic and debugging can be tricky.

That’s where the MicroProfile Long Running Actions (LRA) API comes in. You can run MicroProfile LRA in Helidon.

Source: oracle.com

Continue reading

5 Best Java Frameworks For Microservices

Microservices are extensively being used to create complex applications with multi-functionality by combining every piece and putting them layer by layer in a single unit. Many of us might not be aware of the fact that Microservices is an approach to crafting a single app in a set of small services where each service runs on its own (process).

In other words, Microservices are more of a service-oriented architecture that enables any app to assemble in small chunks rather than creating a whole single unit. Even today, many organizations and developers love working under this bridge as it enables them to work independently. The primary reason behind this is “Dependency of the same programming language literally ends” here! This clearly saves the boat on cost management and improves efficiency.

So, let’s get started with the 5 Best Java Frameworks For Microservices.

1. Spring Boot

Possibly one of the finest and easy-to-go frameworks in Java for developing microservices. It’s open-source, loaded with massive features and functionality that we might have seen so far. Besides this, it can easily be deployed literally on many platforms (like Docker). It offers a strong backup of a vast community network of developers, you can get each query resolved and that’s for sure. It also enables to provide some fascinating in-built functionality like security, auto-configuration, starter dependency (that boosts rapid app dev.), and a list of other services. Let’s have some key features of using this framework:

◉ Spring Boot helps in monitoring multiple components simultaneously.

◉ It enables maximum throughput and efficiency by using the load balancing method where traffic is being distributed in small chunks.

◉ It also offers the distributed messaging system which follows the Pub-Sub (publish-subscribe) model.

2. Quarkus

It was introduced to create modern yet high functionality java applications to meet the expectations of a cloud-native environment. Besides this, it’s a full-stack Kubernetes-native platform tailored for JVMs (Java Virtual Machine) dedicatedly for containers which enables them to sustain in a purposeful cloud, or serverless kind of environment. It was designed with java frameworks like Eclipse, Kafka, Spring, and so on. It offers the right contextual information to GraaIVM (a high-performance JDK distribution) to enable support in the native compilation of Java applications. Thus, working with Quarkus can be real fun, it also enables some other key features which include:

◉ It is designed to sustain in a low power consumption environment by allowing first-class support for Graal, real-time metadata processing, and so on.

◉ The development model of Quarkus can easily adopt the development pattern of your project and can be a good suit, especially for those who don’t like switching things and this makes it a perfect solution for today’s serverless architecture. 

◉ Quarkus also offer a single unified configuration system which means that with a single configuration file, Quarkus applications can be easily configured at every single extension.

3. Micronaut

If you’re willing to work on AWS then Micronaut is the answer, it’s a perfect blend of full-stack, JVM-based, and that is purely designed to create serverless microservice applications. The best part of using Micronauts is you don’t need to worry about the startup time or memory consumption, though it offers a swift flow of speed despite the code length. It’s not wrong to say that Micronaut is a truly modern developer toolkit, designed for today’s developers that helps with injection dependency, AOP, configure management, and much more and that’s what makes it a simple yet elegant Java Framework. Also, below, we’re mentioning a few more important elements that might be helpful for you to understand:

◉ It offers both HTTP client and server that is built on Netty (client-server framework) which also includes an extensive range of tools that suits the cloud environment.

◉ It also provides AOT compilation (ahead of time – the act of compiling a higher-level programming language into a lower-level language before execution of a program) that promotes low memory, IoT, serverless apps, and much more.

◉ Micronaut also supports an extensive range of support for building applications over Java, Groovy, and Kotlin.

4. Eclipse Vert. x

Formed under the Eclipse foundation, it is a perfect solution for crafting react apps over JVM (Java Virtual Machine). Eclipse Vert.x is also a perfect solution for the execution of all kinds of constrained environments (such as VM and Containers). Besides this, Vert.x is a toolkit that offers high flexibility and accuracy for building blocks for any components. The best about vert.x is the independency of creating any components with all the usual libraries. This makes it interesting to work with Eclipse vert.x in your project. Although there are certain key factors to consider beforehand:

◉ The developer will have the option to use multiple languages in their project by using the basic APIs for writing asynchronous networked applications using polyglot.

◉ It is often known as the I/O threading model where a developer can write code as a single thread app using vert.x

◉ It helps in scaling small or medium segment hardware by handling multiple concurrencies with the help of small kernel threads.

5. Ballerina

Just to be specific, it’s not a framework but a distributed programming language that is being specifically used to code distributed applications and that also enables programmers to develop custom network apps with the help of open-source language. Besides this, Ballerina is a cloud-native programming language that eases the JVM frameworks and it also includes annotations for Kubernetes and Docker which help developers to build apps in a low coding environment. Some other features of using Ballerina are as follows:

◉ It enables language-integrated queries with the help of declarative processing of JSON, tabular data, and XML.

◉ Ballerina is highly reliable and can easily handle errors, concurrency safety with the help of readable syntax

◉ It also offers textual as well as graphical syntax based on sequential diagrams.

The introduction of frameworks is simply to elevate the capabilities and to provide an enriched user experience than ever. The idea is simple, grab the best one and start implementing it in your project, the rest it’s all your requirement and the kind of features you’re looking for. 

Source: geeksforgeeks.org

Continue reading

Microservice, monolith, microlith

A proposal to overcome the limitations of both monolith and microservices applications

Download a PDF of this article

As a training consultant, I often deal with very practical questions about microservices: What are they? What is so special about microservices? What are some of the best-justified and beneficial use cases for microservices?

Often, these questions are answered in quite a partial manner, with answers greatly depending on one’s past experiences and personal preferences. Answers range from “everything should be a microservice” to “one should avoid microservices like the plague,” with various degrees of cautionary approaches in between.

Despite the availability of multiple answers, I’ve found they generally lack scientific precision, representing points of view rather than hard facts. Indeed, many recommendations were essentially personal experience testimonies that describe the success or failure of specific cases of microservice implementations.

This article seeks to present something entirely different from such anecdotal evidence. I’ll explore some hard facts from an ocean of perspectives and points of view about the nature and applicability of microservices.

Defining microservices

Let’s start with a definition of what a microservice actually is. Oh, wait: There is no definition—at least there is no definition that is universally recognized. Instead, there are many competing definitions that appear to share a number of similarities. Instead, here are commonly recognized characteristics to define a microservice.

◉ Microservices are characterized as micro or small in size. This implies a small deployment footprint, making it easier to test, deploy, maintain, and scale a microservice application. Smaller application size is aimed at a shorter and thus cheaper production cycle and more flexible scalability.

◉ Microservices are described as loosely coupled, suggesting that each such application ought to be a self-contained unit of business logic that is not dependent on other applications. Loose coupling also means that a microservice application should be capable of being independently versioned and deployed.

◉ Each microservice should be developed and owned by a small team utilizing a technology stack of its choice. This approach promotes tight development focus on a relatively small subset of business functions, resulting in more precise and capable business logic implementation. (See Figure 1).

Figure 1. Microservice application architecture

Defining monoliths

A microservice does not exist only to satisfy its own requirements but is a part of an extensive collection of services. Together these services meet the business requirements of an organization that owns these services.

Consider large enterprisewide business applications, which are often described as monoliths. Unfortunately, much like a microservice, the term monolith is not strictly defined, so I’ll have to resort to describing the characteristics again.

◉ A monolith is characterized as a large application that implements many different business functions across the enterprise. A similar amount of business logic that many microservices provide can be implemented by a single monolith application, but the development of a larger application would take longer. The monolith will likely be harder to maintain than a group of microservices, and it would be less flexible when it comes to available scalability options.

◉ Components within a monolith application could be tightly coupled, suggesting that internally a monolith application would have many dependencies between its parts. Of course, this does not necessarily have to be the case, because the number of dependencies greatly relies on specific design and architecture choices. Still, it is certainly more likely that internal dependencies would exist, simply because creating such dependencies is less of a hurdle for an application developer when all code belongs to the same application anyway.

◉ Monolith development is a collective effort of many programmers and designers, making the development cycle longer, but it may promote a consistent design approach across many different business functions. Using a common technology stack across the enterprise can simplify maintenance and development. Unlike the polyglot approach promoted by microservice advocates, monolith development does not allow flexibility for design and architecture decisions that would be the best fit for a specific subset of business functions implementations. (See Figure 2.)

Figure 2. Monolith application architecture

Common misconceptions

Here are some common misconceptions regarding microservices and monolith architectures.

Service granularity. The term service granularity describes the distribution of business functions and features across a number of services. For example, consider a business function that needs to create a description of a scientific experiment and to record measurements for this experiment. This business function can be implemented as a single service operation that handles a single large business object combining all the properties of an experiment and all associated measurements. Such an implementation approach is usually described as a coarse-grained service design.

An alternative approach is known as a fine-grained service design. The exact same business function can be implemented as a number of different service operations, separately handling smaller data units such as an experiment or a measurement. Notice that the difference is in the number of service operations it takes to represent a given amount of business functions. In other words, both approaches implement the same unit of logic but expose it as a different number of services.

The problem is that the granularity of the service is often confused with the concept of a microservice: Basically, a fine-grained service design is not necessarily implemented as a microservice, while a coarse-grained service is not necessarily synonymous with a monolith implementation.

The key to understanding why these are not synonymous concepts is linked to one of the most fundamental properties of a service, which is that a service invoker should not be able to tell anything about the service implementation. Therefore, it makes no difference to the service consumer exactly how a service is implemented behind the scenes. Whether it’s a monolith or not, service consumers should not be able to tell the difference anyway.

To resolve this confusion, I propose to use the phrase implementation granularity instead of service granularity, where implementation granularity could be described as either fine-grained or coarse-grained. This focuses on defining the actual complexity and size of the application behind a service interface. The idea of the implementation granularity should be helpful to clarify the confusion. You could essentially describe a microservices approach as based on fine-grained implementation design, and which allows the developers to deliver services of any granularity, if that is convenient.

The issue has to do with the implication that microservices must be implemented as small-sized applications, which could be described as a fine-grained implementation design.

Remember that small size and loose coupling are important microservices characteristics that are considered to be beneficial because of the shorter production cycle, flexible scalability, independent versioning, and deployment. However, these benefits should not be considered automatically granted.

Data fragmentation. One unintended consequence of a fine-grained application implementation is data fragmentation. A loosely coupled design implies that each microservice application has its own data storage that contains information owned by this specific application.

Another implication of the loosely coupled design is that different applications should not use distributed transactions or a two-phase commit to synchronize their data in order to maintain a high degree of separation between microservice applications. This approach introduces the problem of data fragmentation and the potential lack of consistency.

Consider the case when a given microservice application needs information owned by another microservice application. What if the solution is simply to allow one application to invoke another to obtain or synchronize required pieces of information? This could work, but what if a given service experiences performance problems or an outage? This would inevitably have a cascading effect on any other dependent services, leading to larger outages and overall performance degradation. Such an approach may work for a small number of applications, but the larger the set of such applications, the greater the risks to their performance and availability.

Thus, consider another solution that addresses the data consistency and fragmentation implications. For example, what if each microservice application caches information that it needs from other applications?

Caching should provide some degree of autonomy for each application, contributing to its capability to be isolated and self-sustained. However, caching also means that applications would have to be designed, considering that the latest data state may not always be available. Different distributed caching and data-streaming solutions could be utilized to automate the handling of information replication. Finally, each application has to provide data state tracking and undo behaviors instead of the distributed transaction coordination.

Inevitably, these issues lead to design complications, making each microservice application not as simple as it appears at first glance.

Furthermore, each development team that works on a particular microservice cannot really remain in a state of perfect isolation but has to maintain data dependencies with other applications.

In other words, microservices architecture does not appear to actually deliver on the promise of completely solving the dependency issues experienced by monoliths. Instead, data consistency and integrity management are shifted from being an internal concern of a single monolith application to a shared responsibility among many microservice development teams.

Versioning. Another problem arises from the promise of independent versioning capabilities for each microservice application. In more complex service interaction scenarios, functional dependencies had to be considered along with the data dependencies.

Imagine a service in which the internal implementation had been modified. Such modification may not necessarily cause any changes to the service interface or the shape and format of its data. Many developers would not consider such implementation modifications as having any consequences that would require the production of a new version of the service and thus would not notify the dependent application developers of these changes.

However, such modifications may affect the semantics of how the service interprets its data, leading to a discrepancy between microservice applications.

For example, consider the implications of a change in the interpretation of a specific value. Suppose an application that records measurements considers an inch to be a default unit of measure, and other applications may rely on this to be the default value. An internal change may lead to the centimeter being implied to be the default value instead of an inch. This could have a knock-on effect on any other microservice applications, which—all things considered—could even be dangerous. Yet, the chances are that developers of these other systems may not be any wiser about said change.

This shows that in any nontrivial application interaction scenario, microservices characteristics should not be automatically assumed as purely beneficial.

Monoliths and microservices face similar problems

Businesses face the exact same functional and data integration problems regardless of the choice of architecture. Because both monoliths and microservices must address them anyway, the question really is in understanding the benefits and drawbacks of each approach.

◉ A monolith offers data and functional consistency as an integral part of its centralized design and the unified development approach at the cost of scalability and flexibility.

◉ Microservices offer a significant degree of development autonomy yet shift the responsibility to resolve data and functional consistency problems to many different independent development teams, which could be a very precarious coordination task.

Perhaps a balanced approach aiming to embrace benefits and mitigate drawbacks of both microservices and monolith architectures could be the way forward. In my opinion, the most critical factor is the idea of the implementation granularity, as discussed earlier.

REST services are by far the most common form of representing microservices applications, and most of the use case examples for REST services focus on each such service representing a single business entity. This approach results in extremely fine-grained application implementations.

Consider the increase in the number of dependencies between such applications because of the need to maintain data cohesion across so many independently managed business entities.

However, strictly speaking, the microservices architecture does not require such a fine level of implementation granularity. In fact, microservices are usually described as focused on a single business capability, which is not necessarily the same as a single business entity, because a number of business entities can be used to support a single business capability.

Typically, such entities form data groups that exhibit very close ties and a significant number of dependencies. Using these data groupings as a guiding principle to decide on the implementation granularity of applications should result in a smaller number of microservice applications that are better isolated from each other. Each such application would not truly be micro compared to the one-application-per-entity structure, but the application would not be a single monolith that incorporates the entirety of the enterprise functions.

This approach should reduce the need to synchronize information across applications and, in fact, may have a positive effect on the overall system performance and reliability.

Business capability. What constitutes a single business capability? It obviously sounds like a set of commonly used business functions, but that is still a rather vague definition. It’s worth considering the way business functions use data as a grouping principle. For example, data could be grouped as a set of data objects produced in a context of a specific business process and having common ownership.

Common ownership implies that there is a specific business unit that is responsible for a number of business entities. Common ownership also implies that business decisions within this unit define the semantic context for these entities, changes that may affect their data structure and, most importantly, define a set of business functions that are responsible for creating, updating, and deleting this data.

Other business units may wish to read the same information, but they mostly act as consumers of this data rather than producers. Thus, each application would be responsible for its own subset of business entities and would be capable of performing all required transactions locally, without a need for distributed transaction coordination or two-phase commit operations.

Of course, data replication across applications would still be required for data caching purposes to improve the individual application autonomy. However, the overhead of maintaining a set of read-only data replicas is significantly smaller than the overhead of maintaining multidirectional data synchronization. Also, there would be a need to perform fewer data replications because of the overall reduction in the number of applications.

Data ownership. In large enterprises, the question of data ownership could be difficult to resolve and would require some investment into both data and business process analysis. Understanding a larger context of information helps to determine where data originates as well as the possible consumer of this data. This analysis picture has to represent a much broader landscape than that of an individual microservice application.

Practically speaking, in addition to a number of development teams dedicated to the production of specific applications, an extra group of designers and analysts has to be established to produce and maintain an integrated enterprise data model, assist in scoping individual applications, and reconcile any discrepancies between all other development teams.

As you can see, this approach proposes to borrow some monolith application design characteristics but use them differently, not aiming to produce a single enterprisewide application but rather support the integration of many applications, each focused on implementing their specific business capabilities.

Furthermore, this approach aims at ensuring that service application boundaries are well-defined and maintained, and it suggests criteria to establish such boundaries based on data ownership principles. There are even some interesting APIs, such as GraphQL, that can support these integration efforts.

Nomenclature. There is one more problem to resolve: What should we call this architecture? Should it still be called microservices, even though some business capabilities may own a relatively large number of entities, and thus some applications may turn out not to actually be that small?

I want to suggest calling such an approach a microlith architecture to indicate a hybrid nature of the strategy that attempts to combine the benefits of both microservices and monolith architectures. The actual word microlith means a small stone tool such as a prehistoric arrowhead or a needle made of stone. I like the sense of practicality projected by this term. (See Figure 3.)

Figure 3. Microlith application architecture

I’m sure many would agree that the best design and architecture decisions are based on practical cost-benefit analysis rather than blindly following abstract principles.

Source: oracle.com

Continue reading