Pseudorandom numbers in Java, Part 1: The Background

Random and pseudorandom numbers are needed for everything from simulations to cryptography. Here’s how they work.

Computing pioneer John von Neumann, who invented one of the first “arithmetical” computer algorithms for pseudorandom numbers, said “Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin.”

What is randomness? And how does it relate to Java’s random number generators? Let’s ask the question in context. What is the probability of your winning the top prize in the next national lottery? Lottery ticket buyers must believe that the game is fair, that the winning numbers are in fact chosen randomly, and that each ticket has the same probability of winning. They wouldn’t risk their money if they thought the contest was rigged.

But what exactly is randomness? This article is for software developers who may or may not be mathematics whizzes. To those who are, my apologies for what will, with high probability, be perceived as a lack of precision.

Just for fun, I asked ChatGPT for a definition, and it replied as follows:

Randomness is the quality or state of being unpredictable or lacking a definite pattern or order. In other words, randomness is the absence of predictability or regularity in events or processes. For example, the toss of a coin is a random event because it is impossible to know with certainty whether it will land on heads or tails. Randomness is a fundamental concept in many areas of science and mathematics, and it is often used to model complex systems and to generate random numbers for various purposes.

That’s pretty good. You should expect that lottery numbers will be random, revealing no pattern of any digits appearing more or less frequently than others over the long haul. Random numbers to be used for security purposes must be extremely hard for an aggressor to guess.

There are several measures for testing random number generators. One obvious measurement is speed, that is, how quickly random numbers can be generated. Sometimes an application needs a lot of random numbers, but for the common case of an application needing only one or a few random numbers, speed is irrelevant and outweighed by the main work of the application.

More important and still easy to measure is the randomness or consistency, that is, how reliably the values that are generated are spread throughout the range of possible values. In other words, the generator should generate with equal probability all the possible outcomes. An easy way to measure this is to run a given generator many times and visually examine the distribution. I did this in Java Cookbook. There, a program (Random5) creates a java.util.Random instance and calls its nextDouble() and nextGaussian() methods 100,000 times each. The program then invokes an R script to plot the values, producing the plots in Figure 1.

Figure 1. The values of nextDouble() and nextGaussian()

You’d probably intuitively expect the nextDouble() histogram to be relatively flat (meaning all values were picked equally) and the nextGaussian() plot to approximate a bell curve. You can see that both predictions were fulfilled. You can download and run the code yourself to replicate this.

Randomness is not the only measure of goodness for random number generators. Another measure is predictability, that is, how hard it is for an outsider to figure out what value a given call will yield. Low predictability is a requirement for use in cryptography, as explained in John Graham-Cumming’s blog post on why cryptography requires the generation of random numbers.

When it comes to predictability, the period is a measure of how many generations can be requested before the values begin to repeat. If a random number generator repeats values too quickly, an attacker might be able to guess with reasonable accuracy the values used in the encryption.

Random number generators can also be jumpable, meaning that you should be able to skip forward without disrupting the sequence to a point in time that is equivalent to invoking and discarding a number of operations. Similarly, a leapable algorithm is one that can take a huge step without disrupting the sequence.

Randomness has many uses in general computing. For example, in UNIX-like systems, you can create reasonably secure temporary files using the following command:

someprogram > $ mktemp /tmp/scriptXXXXXXXXX

The mktemp utility program uses a random character generator (and characters are just numbers within the range of the ASCII or UTF-8 character set). The mktemp invocation above creates a file with a hard-to-guess name such as /tmp/scriptogWqMcWOygi, and the file is readable and writable only by the owner. Such files provide a reasonably secure way of creating files in a public directory. Even-better systems also randomize the process id numbers given to each program that is run. This makes it harder for an attacker to anticipate the process id number of an upcoming command—and that, in turn, makes it harder to attack the system.

True random numbers versus pseudorandom numbers

Most computers do not have truly random random numbers, just as they don’t have real real numbers; truly random numbers are computationally expensive (and thus time consuming), and generally they aren’t needed unless, for example, you are running a lottery.

Instead, many applications leverage faster, more efficient software that can generate pseudorandom numbers using a pseudorandom number generator (PRNG).

PRNGs use an algorithm that has a starting value, or seed, and a permutation. The permutation part of a commonly used linear congruential generator (LCG) algorithm is based on the following formula:

Xn+1 = (a * Xn + c) mod m

In that equation, X0 is the seed, and each value is based on the previous value multiplied by a constant a; that result is added to another constant called c; and the result of that taken modulo a third constant, m. What if Xn gets large or even overflows? No problem. Whatever’s left will still result in a nice pseudorandom value. And, in practice, many implementations using the LCG algorithm also use bit masking on subsequent calls to remove some of the bits that are not changing on subsequent calls.

The LCG algorithm is comparable to hashCode(), which generates a seemingly random number, with the current state of the given object being the seed. You’ve probably seen a typical IDE-generated hashCode method, such as the one shown in Listing 1.

Listing 1. Datum class with IDE-generated hashCode

public class Datum {

    long id;

    String name;

    int yearJoined;

    @Override

    public int hashCode() {

        int result = (int) (id ^ (id >>> 32));

        result = 31 * result + (name != null ? name.hashCode() : 0);

        result = 31 * result + yearJoined;

        return result;

    }

    ...

}

In mutable objects, changing the state changes the hashCode() value, permitting you to take the comparison a bit farther, as shown in Listing 2. If you run this program, you’ll see it generates random-looking numbers, but they are not very good ones.

Listing 2. Running hashCode() as a bad random number generator

import java.util.stream.IntStream;

public class DatumHash {

    public static void main(String[] args) {

        Datum d = new Datum(123, "Ian", 1999);

        // Loop by 175 just to get a good range of values

        for (int i = 1; i <= 2020; i+=175) {

            d.setId(2020 % i); // just generate some value here

            d.setYearJoined(i);

            System.out.println(d +" hashCode: "+d.hashCode());

        });

    }

}

It’s Knuth time

A definitive treatment of PRNGs can be found in Donald Knuth’s The Art of Computer Programming, volume 2, chapter 3—particularly section 3.2.1, which devotes 16 pages of mathematical discussion to LCGs. Meanwhile, section 3.2.2 dedicates 12 more pages to other PRNG algorithms.

This material will interest mathematics and computer science majors but may reach over the head of those who are not mathematically oriented. Even those who don’t care should note the following, however. Knuth points out that if you exercise really bad judgment in selecting the values for X0, a, m, and c in the formula shown previously, for example, by setting X0, a, and c all to 7 and setting m to 10, the formula will generate the following series as output:

7 6 9 0 7 6 9 0 7 6 9 0 ...

That series has a repeating period of 4, which is completely useless for most purposes. Let that be a caution to you: Do not implement your own LCG algorithm unless you really know what you’re doing! Instead, use existing generators provided in the JDK. (Note: Choosing better values for the various values takes up quite a bit of Knuth’s discussion, and it remains a problem for developers building PRNGs for production use.)

Another problem for generating useful random numbers is that the seed must come from somewhere. In the rare case where you are testing the random number function itself, you probably want the seed to be fixed (you can feed the seed into the constructor or use the setSeed() call), so that the algorithm will generate the same set of numbers each time you run your test.

However, in production use of a random number function, you want the seed to be…wait for it…random. And that would require the use of yet another random number function, which in turn would require its own random seed from yet another random number function. You can see where this will (or won’t) end.

What about true randomness? Only a process that doesn’t follow a fixed algorithm can generate truly random numbers.

Lava lamps for randomness?

One of the first computer-based true random number generators used Lavarand, a lava lamp set in front of a web cam. A lava lamp contains two nonmiscible liquids—wax and a clear liquid—in a clear upright container. The heat from an incandescent bulb causes the wax to rise, where it cools and then falls, in an endless cycle that is slightly different every time.

The idea behind this is that the flow of the wax blobs would be reasonably random and couldn’t be guessed by people who didn’t have a camera focused on the lamp—and even then, they’d not know exactly which part of the lamp the camera was focused on. Since the patent on the original Lavarand has expired, the methodology can be used by anyone and, in fact, a whole wall of lamps is used by internet backbone carrier Cloudflare to secure traffic on the current internet—really.

Most developers probably don’t have lava lamps or another true random number generation system, but they need a random seed value to start. PRNGs tend to use a source of semirandomness such as the low-order bits of the high-precision system clock, which changes every microsecond or nanosecond, making the value almost impossible to guess.

Some operating systems, such as OpenBSD, calculate randomness (termed entropy) from the keyboard, the time, the arrangement of things in the environment, and so on. Such operating systems save some of this randomness to disk when the computer is shut down, so that even when the operating system first boots up, it already has a good source of entropy. Few other operating systems go that far, yet.

Source: oracle.com

Continue reading

Quiz yourself: Use an Optional object when you might have zero data items

Given the following method

public void printSomething(Optional<Integer> o) {

  var p = o.filter(v -> v == null);

  p.ifPresent(System.out::println);

}

and the code fragment

printSomething(Optional.ofNullable(0));    // line 1

printSomething(Optional.ofNullable(null)); // line 2

What is the result? Choose one.

A. There is no output.

B. One blank line is printed.

C. Two blank lines are printed.

D. 0 is printed and then nothing else.

E. 0 is printed followed by a blank line.

Answer. Using a null pointer to indicate that a data item does not exist can be error prone. Another—typically more robust—way to address the possibility of a data item not existing is to use something such as an array. In Java, an array can contain any number of items, and the array knows how many items it contains. If you write code to process all the elements in the array but the array is empty, you simply process nothing, and no consequential errors will occur.

This idea is formalized in the Optional class that was added in Java 8. An Optional object is a kind of wrapper that contains either zero or one data items. The Optional also provides methods that can apply processing to all the elements in the Optional in a manner comparable to what was just outlined for an array, providing another safe way to handle a missing element. This question investigates the behavior of two methods in the Optional API and one of the methods that can create an Optional instance.

The method Optional.ofNullable creates a new Optional object that either contains the provided argument or, if that argument value is null, is empty. So, line 1 in the code snippet above creates an Optional that contains an autoboxed Integer that has a value of zero. Line 2 creates an Optional object representing emptiness.

The Optional.filter() method is similar to the intermediate operation of the Stream method with the same name. It takes a Predicate as an argument and tests elements with that Predicate. The filter method then returns an Optional based on that algorithm, as follows:

◉ If the Optional was empty, filter() returns the same Optional (which is still empty, of course).

◉ If the object contained in the Optional returns false from the Predicate test, an empty Optional is returned.

◉ Otherwise, the same Optional is returned, which still contains the original value.

The filtering operation in the example looks like the following:

var p = o.filter(v -> v == null);

The Predicate shown above returns true only if the argument value is null. In other words, anytime this Predicate is used in a filter operation on an Optional, the result must be an empty Optional. If the original Optional was not empty, the test fails, resulting in an empty result. If the original Optional was actually already empty, the test is never applied and the filter simply returns the original, unchanged—and still empty—Optional.

From the discussion above, you can deduce that both line 1 and line 2 must have the same effect. Option B suggests that one blank line is printed, which could only happen if lines 1 and 2 had different effects. Similarly, options D and E both hinge on different effects for the two lines. You can therefore deduce that options B, D, and E must be incorrect.

Next, consider the behavior of the Optional.ifPresent() method. This method takes a Consumer as its argument. If the Optional contains a value, the Consumer is invoked with the value of the Optional being passed as the argument. If the Optional is empty, ifPresent does nothing.

At this point, you’ve already established that when the code reaches the point of invoking ifPresent, the Optional is empty regardless of the situation prior to that. This in turn shows that no printing is ever invoked. From that, you can determine that the code produces no output and option A is correct, while option C (which suggests that println is called but with an empty string as an argument) is incorrect.

Conclusion. The correct answer is option A.

Source: oracle.com

Continue reading

Curly Braces #9: Was Fred Brooks wrong about late software projects?

If your project is running late, maybe you actually do want to add more developers.

After more than 30 years of professional software development, I’ve learned that not only do you need a lot of code to build software but you also need lots of communication. This is what the late Fred Brooks described as a problem in his famous book, The Mythical Man-Month—especially in regard to adding more software developers to an already late software project. The level of intercommunication between people grows to where it impedes progress, and the project becomes increasingly late with each person added.

That’s where Brooks’ Law comes in, which clearly states: “Adding manpower to a late software project makes it later.”

Things are rarely this straightforward. In my opinion, it may have been inaccurate for Brooks to talk about this human-resource paradox as a generalization. It was a problem specific to his project, his team’s architecture, and his team’s choice of languages and tools. He also assumed that all developers are equal and that tasks cannot be easily worked on independently. Even so, over time, studies on many large software projects have proven him correct—enough that Brooks’ Law is, well, Brooks’ Law.

If you buy into the argument that adding people to an already late software project delays it even more, then what can you do to speed things up? It turns out there are some things you can do. A few of the following suggestions are from Brooks himself and a few are my own (for what they’re worth).

Use the Bermuda Plan

To speed up a software project, the Bermuda Plan—part of Brooks’ Law—may sound cryptic but it’s very simple: Send most of your developers on a nice vacation and let your top people do all the work unabated. That’s not very formulaic, but it’s a guideline that makes sense if communication and distraction are the main impediments to progress. It may not be practical, however.

Want a more practical version? Well, you can move developers to critical nondevelopment tasks. This helps to reduce communication delays and assigns developers to finish tasks that help the remaining developers become more productive. For example, developers can be assigned to

◉ Improve deployment processes (DevOps).

◉ Improve system architecture to support parallel programming teams.

◉ Implement or enhance automated testing.

◉ Build out lab resources to reduce hardware bottlenecks.

◉ Build or identify tools to help coding and debugging.

◉ Improve documentation to help get other developers up to speed.

You might demoralize developers who are removed from mainline development tasks, and you might also create more costs related to additional release cycles, but these downsides can be managed and controlled.

Design the system with proper segmentation

To minimize intercommunication and interdependencies between software teams, carefully segment your system design to allow teams to work independently—that is, in parallel.

For example, a single team working on a client/server application will require a lot of coordination as they work on the message-by-message communication between the two components of the application.

However, if they first decide to use an independent communication protocol (such as HTTP), the client and server teams can work almost completely independently, as long as each adheres to the communication specification. I would suggest with confidence that you could develop a new web browser today without speaking to a single web server developer.

Leverage pair programming

Pair programming, where two individuals are glued together to work side by side on a single task, can reduce communication needs by half (or more). Instead of each person working on different tasks and needing to communicate across the organization, developers are paired, reducing cross talk. Pair programming helps to further reduce communication problems because knowledge sharing occurs organically, especially when you pair a newer developer with a more experienced one.

The benefits from pair programming often improve productivity for each developer, and for the team as a whole, for the following reasons:

◉ Individual programmers can focus on their strengths as part of a pair.

◉ The organization is often more resilient against employee turnover.

◉ There is less schedule impact when people need time off.

◉ Having multiple people working on the same problem and code tends to result in fewer defects to fix later. This is also known as Linus’ Law, named after Linus Torvalds of Linux fame.

◉ With additional people participating in the same conversations, misunderstandings are reduced and communication is often reduced because there is less rehashing.

◉ Best practices and time-saving techniques are easily shared and spread throughout the team.

◉ With rubber ducking, as described in The Pragmatic Programmer by Andrew Hunt and David Thomas, debugging is improved, mainly due to human nature: One person explaining something to another helps to uncover issues very quickly.

◉ When people work together, they tend to stay more focused, reinforce each other’s confidence and strengths, and generally desire to be more productive so as to not let the other person down.

Add more people

Yes; you read that right. I’m suggesting adding even more people to a late project to help speed it up. That completely violates Brooks’ Law. But it can help in situations where less training and overhead are needed for the added people, for example, if they are technology specialists, proven consultants with exceptional skills and expertise, nondevelopers who have exceptional communication skills, or developers who have experience building similar systems.

In the extreme, the use of competition between internal groups can lead to seemingly miraculous results. You can see examples in Tracy Kidder’s The Soul of a New Machine or in other legendary large-scale development efforts written about in books or online articles.

In my experience, an acquisition can make a difference as well. I’ve witnessed multiple examples where a project’s success was so critical that a decision was made to acquire a company with a similar product or technology to enable progress. This can have multiple side effects that do indeed work:

◉ An acquisition can serve as a catalyst for renewed hope and energy that reinvigorates the original team.

◉ There’s an infusion of fresh talent that was not hired by the original team.

◉ Unintended but healthy competition can result.

◉ A new camaraderie between developers can improve teamwork.

◉ New managers who are unafraid to ask questions and suggest changes are infused into the project.

◉ New design patterns and ways of thinking can unlock unrealized time savings.

◉ Additional thought leadership, instead of added developers, can help increase a project’s velocity.

Extend the schedule, if you can

That suggestion may sound like a snarky comment, but in reality, the original schedule may simply be unattainable, as Brooks’ Law also points out. Scheduling mistakes often account for late projects, which is an issue that extends beyond software development. Just ask any homeowner who’s remodeling, and they’re sure to agree.

Whether or not you can extend the schedule, progress can often be improved by performing tasks more often and working out the bottlenecks. This is in line with the Agile development process, as well as with DevOps, where you release smaller increments more often and get better at doing that. If you can define an incremental release plan where each phase has its own schedule and then divide groups of developers across the phases, you may be able to achieve a higher degree of independence and parallelism.

Use Java!

I would never suggest one language is more productive than another, but Java is a complete platform. Java has a robust virtual machine that abstracts the hardware details and a mature set of tools that help in every facet of development and debugging. In addition, there’s a rich set of commercial and open source software to add value to your projects, powerful IDEs, and a vast community to turn to for help.

Source: oracle.com

Continue reading

Quiz yourself: How does a Java finally block handle an exception?

You’ll want to know the difference between abrupt completion and normal completion.

Given these two exception classes

class BatteryException extends Exception { }

class FuelException extends Exception { }

And the following Car class

public class Car {

  public String checkBattery() throws BatteryException {

    // implementation

  }

  public String checkFuel() throws FuelException {

    // implementation

  }

  public String start() {

    try {

      checkBattery();

      checkFuel();

    } catch (BatteryException be) {

      return "BadBattery";

    } finally {

      return "";

    }

  }

}

Which statement is correct about the start() method? Choose one.

A. It may return BadBattery or an empty string.

B. It can only return an empty string.

C. It may throw FuelException.

D. It will cause a compilation error because FuelException is not handled.

Answer. This question investigates a less-frequently used behavior of a finally block.

Looking at the code, notice that there are two domain-specific exceptions related to the battery and the fuel. They’re direct subtypes of Exception, which means that they are checked exceptions. If such an exception might be thrown by a method, it must be declared in the throws clause of that method.

The usual way to handle an exception inside a method is to use a try-catch structure and provide a catch block that names the exception, or a parent type of that exception. In this code, there’s a catch block for the BatteryException but not for the FuelException. Given that the method does not declare throws FuelException, you might expect the compiler to refuse to compile the code.

However, if you think a bit deeper on this, you should realize that the finally block executes return "". This does exactly what it says: No matter how the code reaches the finally block, the result is that the method will return an empty string. No exceptions will be thrown; indeed, any FuelException that might arise will simply be abandoned. In other words, because FuelException is never thrown by the method, it’s not necessary to declare it in a throws clause.

The above examination shows that options C and D are both incorrect, because no FuelException is possible, and the code does not fail to compile due to a missing throws clause.

Digging into this logic a little further, if the code executes return "Bad battery" from inside the try block, it must execute the finally block before control is ultimately passed to the caller. This too causes the method to return the empty string from inside the finally block. This tells you that option A is also incorrect. Further, when you combine this with the earlier discussion, you should see that the code always returns an empty string; therefore, option B is correct.

It might be of interest to follow up on the qualitative descriptions above with some details from the Java Language Specification. First, you need to understand the meaning of the phrase abrupt completion, which is the topic of section 14.1. This section might be paraphrased as follows: If a region of code runs to its end, it completes normally. If, by contrast, it jumps out of that region without completing all the steps, it completes abruptly.

Note that this doesn’t imply (nor does it exclude) exceptions. In particular, a return statement constitutes an abrupt completion of a method, where running off the end—that is, reaching the closing curly brace of the method—is normal completion. (You should read the specification for a formal, and more complete, description.)

With that in mind, let’s continue looking in the specification and now focus on the behavior of catch and finally. In section 14.20.2 you’ll find the following text:

If the catch block completes normally, then the finally block is executed. Then there is a choice:

◉ If the finally block completes normally, then the try statement completes normally.

◉ If the finally block completes abruptly for any reason, then the try statement completes abruptly for the same reason.

If the catch block completes abruptly for reason R, then the finally block is executed. Then there is a choice:

◉ If the finally block completes normally, then the try statement completes abruptly for reason R.

◉ If the finally block completes abruptly for reason S, then the try statement completes abruptly for reason S (and reason R is discarded).

These paragraphs explain that abrupt completion of the finally block means that the try construct completes abruptly for the same reason. In effect, this supersedes any previous mode of completion and any previous reason for abrupt completion.

Note that execution of a return statement constitutes abrupt completion, as explained in the following sentence from section 14.17:

It can be seen, then, that a return statement always completes abruptly.

You know that the return "" in the finally block is always executed because there are no paths through the method that do not enter the try construct. Putting this together with the specification excerpts above, you can see that the only possible result of executing the method is abrupt completion, which returns an empty string.

Conclusion. The correct answer is option B.

Source: oracle.com

Continue reading

Embedded Java: Then and now

The days of J2ME are long gone, but thanks to evolving technologies such as GraalVM Native Image, embedded Java is better than ever.

Oracle Java SE 8 Embedded is the final major release of the Oracle Java SE Embedded product. Starting with JDK 9, Oracle doesn’t plan to offer a separate Java SE Embedded product download.

Mainstream Java is heading towards version 20, so what’s going on with Java in the embedded space?

First, here’s some history. Java (then called Oak) was initially developed by engineers at Sun Microsystems more than 30 years ago. Originally, Oak was developed to provide an object-oriented programming and runtime environment for embedded systems—independent of the underlying hardware and operating system.

The idea was to create a uniform and portable programming platform with automatic memory management and robust execution of code in a virtual environment. By using a virtual machine (VM), programmers would automatically avoid otherwise common problems such as crashes caused by buffer overflows and faulty pointer arithmetic.

Object-oriented programming for embedded systems was a relatively new idea in the 1990s, and the embedded community was skeptical. The Java programming language was designed to be interpreted during runtime and as such, it was initially intrinsically slow and resource hungry. And even though the first Java release had only eight packages and about 200 classes, it was considered too “heavy” by many hardcore C/C++ and assembly language programmers.

Given the limited processing power and very limited memory availability back then, it was understandable that the advantages of object-oriented programming, portability, and secure execution did convince many software engineers.

Set-top box manufacturers were among the earliest adopters of Java in the embedded space even before the Java plugin for web browsers opened the door for an entirely new programming model on the desktop.

Applets—which were small, and usually visual, programs written in Java—could be downloaded into a browser and create an interactive user experience that was previously hard to accomplish with HTML and early scripting languages.

Because the Java code was executed in a VM invoked by the browser, applet authors didn’t have to care much about the variety of target operating systems and underlying CPU architectures. Java grew bigger and stronger and conquered the desktop world. Even back then, PCs had serious computing power and memory, and some technology advances such as just-in-time (JIT) compilers helped increase the acceptance of Java as a mainstream programming language for client applications.

Enter J2ME

Even decades ago, programmers and project managers realized the advantages of object-oriented programming and the benefits of the fast-growing class libraries and functionality included in the Java language. But most embedded systems still weren’t strong and big enough to host a full desktop Java runtime environment (JRE). The Java stakeholders (Sun Microsystems, IBM, Nokia, RIM, Philips, Siemens, Motorola, and others) organized in the Java Community Process approved a Java Specification Request, JSR 68, to specify a Java variant specifically designed for embedded use: Java 2 Micro Edition, also known as J2ME.

Subsetted class libraries and small-footprint JVMs opened the door for widespread use of Java in embedded systems. In particular, mobile phones made use of Java with the Mobile Information Device Profile (MIDP) profile, a configuration targeted to handheld phones with elementary graphics capabilities from the Limited Connected Device User Interface (LCDUI).

In 2001 MicroDoc (the company we work for) was one of the first companies in Europe to begin working on embedded Java. Initial work was done on the infamous PowerPC Red Box with UNIX, followed by a JVM port to Sun’s ChorusOS microkernel operating system for credit card payment terminals. Many of those terminals are still in operation today with their original VM infrastructure.

More and more embedded systems were integrated into communication networks, and implementing complex networking protocols in C or even assembly language turned out to be complicated and error prone. Java offered an integrated network stack and an automatic software distribution mechanism locally and over the network. And embedded JVMs became available for many operating systems and CPU architectures such as SH-4, PowerPC, ARM, MIPS, and x86.

The adoption of Java in the embedded space was still limited by frequent complaints about poor runtime performance and high memory requirements. But new technologies such as tiered garbage collectors and ahead-of-time (AOT) compilation made the execution of Java code more predictable and faster than ever. And the advent of stronger 32-bit processors and affordable memory opened the way for many high-tech use cases such as automotive head units and Global System for Mobile Communications (GSM) network stations and controllers.

Java takes the lead

Java surpassed all other programming languages in popularity for the first time in 2001 when it became the most used programming language according to the TIOBE Index, and it stayed on top until 2019. During that period, embedded Java achieved widespread adoption in devices such as telematics units, Blu-ray players, internet routers, and integrated internet edge devices. MicroDoc ported a VM to Windows CE on AMD’s Geode chipset as part of AMD’s 50x15 initiative. The initiative was founded to accelerate access to the internet with very low-cost devices to enable educational and commercial applications online, even in less-developed countries.

That goal was eventually reached with the advent of feature phones and, of course, smartphones. Beyond that, MicroDoc had its first high-volume deployment in the auto industry in 2009. The company’s engineers cooperated closely with a well-known German tier-one supplier, and they created one of the first aftermarket onboard telematics devices for the trucking industry. The platform, based on 32-bit ARM/Linux, was designed to enable track-and-trace services and to enable third-party applications to be deployed.

What followed was a series of automotive engagements with tier-one suppliers and OEMs. MicroDoc provided an advanced runtime platform for automotive head units on a variety of hardware and software architectures. Starting with 32-bit SH-4 on Windows Automotive, the team ported and optimized VMs for Linux on PowerPC, ARM32, and ARM64, which enabled MicroDoc’s customers to deploy their Java-based applications on whatever hardware generation they chose to deploy.

As a kind of niche market supplier for customized Java VMs, MicroDoc was able to work with customers from a variety of industries, including network infrastructure companies, logistics companies, smart-home device manufacturers, and companies in the healthcare sector.

Besides porting JVMs to numerous target devices, MicroDoc also added valuable reusable components to the standard class libraries. These include Java stacks for the use of many variants of OpenGL, libraries for the open standard communication protocol MQTT, device management protocols, and hardened stacks for the Transport Layer Security (TLS) protocol.

Modern embedded Java

As a typed language, Java is still among the most popular languages today, and embedded applications benefit from the integrated security measures in current Java systems. But it looked like the idea of a truly embedded Java came to an end with Java 8.

To maintain the integrity of the Java language and still allow for customizing Java runtimes for embedded systems, Oracle released Oracle Java SE Embedded, which defined three so-called compact profiles that were strict subsets of the desktop class libraries.

This move was needed since Java 8 had become fairly large and contained many features rarely used in embedded systems. The smallest compact profile class libraries have a footprint of less than 14 Mb compared to the desktop version libraries, which go above 50 MB.

A configurable JIT compiler and a choice of garbage collectors complemented the embedded version and made it a suitable embedded platform for many industries; for example, some of the world’s largest automakers rely on Java 8 technology for their infotainment and telematics systems.

Because the current release of the platform is Java 19, and Java 20 will be released in March 2023, it is fair to ask why there isn’t a more recent Java embedded version. There are several reasons.

The embedded systems market is very complex. There are hundreds, if not thousands, of different CPU variants and operating system dialects on the potential target devices, and it is extremely expensive to maintain a complex codebase such as a VM on so many platforms. The few profitable high-volume embedded applications, such as smartphones, have turned to open source offerings or focus on different languages.

Therefore, it is hard to justify big investments in many VM platforms. Oracle has reduced the number of embedded platforms it supports. What remains is a few smaller software vendors (such as MicroDoc) who specialize in the customization and optimization of JVMs for niche markets.

Java has a new module system. Java 9 introduced a new module system (known as Project Jigsaw). Java’s monolithic class library was rearchitected to allow separation into functional components that can be added, as needed, for a runtime system. Unused components can be left out—thus, the footprint is reduced. Whether this technology lives up to its promises remains to be decided by embedded systems engineers. Some claim that the unbundling was not done thoroughly enough and the essential core modules needed for every application are still too big.

Some people complain about performance issues, in particular startup time. When you launch a VM, you load a big piece of software and classes before any user code can be executed. And then the JIT compiler monitors the application and decides when to interrupt the execution of frequently used methods to compile them into machine code. That helps at runtime later, but it also increases the system’s total startup time.

Help comes from the cloud

Cloud computing has become a mainstream business in recent years. Giant server farms host applications for millions of users. Many services offered in the cloud are based on microservice architectures. Microservices are small functional entities that are invoked and immediately suspended after use. The requirements for cloud computing are fairly in line with what’s needed in the embedded space: a small footprint and fast startup.

Even though today’s servers have abundant horsepower and virtually unlimited memory, when millions of users are served at a time, the resources need to be shared among all users, and the fraction available for a single user can become fairly small. Also, users don’t want to wait for many services to start up; they want an immediate response.

Oracle is one of the major cloud providers and has launched a game-changing project to solve these problems: GraalVM.

GraalVM is a portable VM that can be used to execute a variety of programming languages: Java and also Python, R, Ruby, and JavaScript. And GraalVM offers a unique technology called GraalVM Native Image that can be used to compile Java applications directly into a standalone executable file, called a native image, for a target platform.

Using a native image is different from having a JVM execute AOT code. A native image contains only the ready-to-run machine code of the application and a lightweight memory manager for garbage collection. Most of the other heavyweight JVM components are stripped: No interpreter, no JIT compiler, and no class libraries are part of the native image. This leads to a superslim footprint and blindingly fast startup times.

Does that sound like an embedded platform? It does.

GraalVM Native Image is well suited for bringing new and existing Java applications to embedded devices. It offers the full universe of Java advantages and at the same time saves memory and CPU cycles.

Oracle is focusing on server-based cloud computing with GraalVM, but there is a growing community working on implementations for embedded use. MicroDoc has entered into a contract with Oracle to bring a commercial license offering for GraalVM to the embedded market.

MicroDoc has already implemented a cross-compiler for GraalVM Native Image compilation that can create executables for previously unsupported platforms that are commonly used in the embedded space (such as 32-bit Linux running on ARM). With 30 years of experience in the field of embedded VMs, MicroDoc can bring GraalVM-based solutions to legacy systems and future architectures as well.

In other words, the story of embedded Java is not over. In fact, it has only just begun.

Source: oracle.com

Continue reading

Announcing OCI File Storage replication

The Oracle Cloud Infrastructure (OCI) File Storage service now supports cloud native asynchronous replication as a feature of our highly available, elastic file system. With the launch of this feature, file system replication is available as a fully managed solution for your enterprise workloads.

What is File Storage replication?

File Storage replication allows you to replicate your source file systems to target file systems in different availability domains. These targets can exist across multiple availability domains within a region or across different regions in your tenancy. For example, an Oracle E-Business Suite (EBS) customer with primary operations in availability domain 1 in Phoenix, AZ, can choose to have their backup or recovery site be in availability domain 2 of Ashburn, VA. This functionality is critical for many customers who need disaster recovery solutions to protect critical business data and adhere to compliance requirements. File Storage replication uses snapshots and clones as some of the building blocks for its replication and disaster recovery architecture.

File Storage replication gives you consistent file system replicas. Your application can use the target file system, fully confident of its consistency. The capability of having file system consistency is another OCI first in the industry, where the underlying replication technology doesn’t rely on block-level replication. This setup is unlike what other hyper-scale cloud providers offer, where the filesystem consistency is not provided.

With File Storage replication, the source file system can be replicated to multiple target regions simultaneously. You can select a replication interval that meets your business needs. This flexibility helps you meet your compliance and information life cycle requirements with the following use cases:

◉ Geographically dispersed disaster recovery: Failover and failback

◉ Data migration and data mobility: General data movement (copy and backup), snapshots, and read-write file system clones in other availability domains or regions

Understanding File Storage replication concepts

A replication relationship is established between a file system in the primary (source) region and a file system in the secondary (target) or recovery region. The replication relationship is represented by replication resources, which are tracked by unique Oracle Cloud identifiers (OCIDs) in the source and target regions. Source and target file systems don’t necessarily have to be in different regions. They can be in different availability domains within the same region.

The initial data transfer from the source to the target file system is called the base copy. When the base copy is complete, periodic system-driven snapshots are taken on the source, and the incremental data are securely transferred over to the target file system. These increments are called delta copies. The base copy, snapshots, and delta copies all happen without any intervention from the user.

The frequency of the delta copy is controlled by the replication interval specified by you. For convenience, the replication feature assesses your file system and recommends an appropriate replication interval. You can monitor the health, progress, and performance of the replication by using metrics, alarms, and notifications.

File Storage replication is asynchronous in nature. The source and target file systems have an active and passive role. You can actively use the source file system during replication. The data on the target file system is accessible only when the replication relationship is ended. Alternatively, you can also create a clone from a snapshot in the target filesystem and use that clone with your application.

Get started

With two clicks, you can get replication going! Head over to the Oracle Cloud Console and select the file system that you want to replicate. In the Resources panel, click the Replication link. Then, click the Create Replication button, fill out a few fields, and you’re on your way.

Like any other File Storage feature, you can also use the OCI command line interface (CLI), application programming interface (API), or the software development kit (SDK) to create and manage replications. You can also set up replication using Terraform (resource manager).

When you put together your disaster recovery solution using File Storage replication, you need to understand the starting sizes, the rates of change to your file systems, and the network bandwidth between the source and target regions. For large or rapidly changing file systems, you might find that the replication interval currently supportable is beyond your recovery point objectives.

Replication has a built-in estimator tool that considers these factors and helps you arrive at the recommended replication interval for the target region. It also estimates the completion time for the base copy. With replication metrics and OCI alarms, the replication estimator feature enables you to plan and monitor your recovery point objectives.

Like other File Storage features, using replication has no extra cost. However, as a file system user, you’re billed for the storage that you use. So, when you replicate a file system, you pay for the storage used for the source and the target file systems. If you’re replicating between regions, a common use case for disaster recovery considerations, you’re also charged for the outbound data transfer between regions.

Source: oracle.com

Continue reading