Introducing the Visual Recognition spec for Java machine learning

The VisRec API, JSR 381, addresses common pain points for machine learning in Java.

There are not many machine learning (ML) coding options for Java developers, and the ML libraries currently available have several issues. Many are very complex and designed for data scientists, or they are Java wrappers around C/C++ libraries and don’t feel like Java tools.

Of course, one native Java library is the increasingly popular Tribuo. But that’s not all: For image recognition tasks, there’s JSR 381, Visual Recognition (VisRec) Specification. This JSR was released in early 2022 and was designed to address the following common pain points for ML in Java:

• Many different incompatible data formats
• Many different ML algorithms
• Many confusing configuration parameters
• Lack of a clear task-oriented interface that hides implementation details, such as Collections.sort()
• Lack of simple and portable integration into existing Java applications and devices

The authors of the VisRec specification explained that “there are wide business implications for machine learning capabilities in all applications across many types of devices. VisRec is an important subset of ML. Right now, the primary language for ML is Python. We feel Java needs to play a major role in ML, starting with VisRec.”

The specification authors added that typically, the API will be used in conjunction with an ML engine, package, or set of libraries that “would execute on a server or set of servers (as most ML applications) and callable from either the server side or remotely from a distributed client (JavaFX, web, command-line).”

Common applications of visual ML include pattern recognition and classifications. Imagine training and then processing a series of images and videos.

• How many dogs are in those still images? How many cats? How many lemurs? For videos, how many unique dogs were captured?
• Is there a vehicle in a video, and if so, what type? Is it speeding? Did it go through a stop sign without stopping? Was it speeding through a neighborhood while people were attempting to legally cross the street? Could the registration plate be read?
• Did a security camera show that a shopper at a self-checkout skipped scanning an item?
• Was the product label correctly affixed to a product in an automated assembly facility?

Two of the JSR 381 coauthors, Frank Greco and Zoran Sevarac, had an online chat with Mala Gupta about the VisRec API and its goals. Greco is a senior consultant at Google and chair of the New York Java user group, JavaSIG. Sevarac is an associate professor at the University of Belgrade and a Java and AI deep-learning expert. Mala Gupta is a developer advocate at JetBrains and is a frequent Java Magazine author. All three are Java Champions.

Here are a few highlights from their conversation.

Gupta: What is the most common myth you hear from Java developers about ML in Java applications?

Greco: It’s that you must be a data scientist to do ML. Of course, there will be some experts, but the bulk of the people using ML are not going to be data scientists. The core of ML is recognizing patterns, building models based on that data, and making predictions. Python is the default when it comes to ML, but we wanted to create something for Java, too.

Gupta: What exactly is JSR 381?

Greco: From a high level, it’s a Java-friendly standard API. It’s not only for visual recognition; it’s also a generic ML API. It includes the usual high-level abstractions. It uses Java paradigms. It’s more readable. The bottom line: It was designed for Java developers.

Gupta: What was your primary goal when developing JSR 381?

Sevarac: The main goal was to make visual recognition and ML easy to use by nonexperts. Java developers should be able to use this API intuitively and use their Java skills to the maximum extent, so they don’t have to learn new things. Or, if they do need to learn new things, they can learn them along the way through coding. We were also aware that it was not possible for us to implement all potential use cases in all possible learning algorithms. Therefore, the important point is to stay open, so any existing ML libraries out there can implement this API without much difficulty. We have created a reusable design, which can be applied to other domains, too.

Gupta: Several Java ML libraries already exist. Why develop JSR 381?

Sevarac: There are many libraries with different sets of APIs, but none of them works well with the others. Each library tries to reinvent the wheel. Our idea was to create a standard API collection that would address the most typical use cases and that would be extendable, so any existing ML library could easily implement it.

Also, existing libraries support different specific sets of algorithms and there are no common abstractions, or they are not compatible with each other. When it comes to visual recognition, there are so many Java imaging libraries that use different images and classes, and some of them use native dependencies, which creates portability issues. Most of these libraries do not look like Java and are very complex for the average Java developer, not only because the API is not Java flavored but also because most application developers don’t have a background in data science or know how ML algorithms work and how to configure them. These are all issues we are trying to address with JSR 381.

Gupta: What are some of the common barriers for ML in Java that you tried to address with JSR 381?

Sevarac: We tried to create abstractions where you can specify, with generic parameters, what type of data you’re going to use. There are many different ML algorithms, and we tried to create abstractions for the most common ones. Existing and new libraries should consider implementing some of these abstractions.

Confusing configuration parameters are one of the biggest challenges for every ML algorithm. We tried to keep those to a minimum and provide reasonable defaults—or at least some starting points people can use to create models. Learning these parameters will come with experience.

One of the most important things we addressed is providing a clear, task-oriented interface. It is very simple and straightforward.

Also, we felt it was important to hide implementation details. Developers don’t necessarily want to know details about how an image is being stored in memory and the most efficient way calculations are performed on that image. That might be important for the people who are implementing the algorithms, but end users generally don’t need those details.

Finally, JSR 381 provides simple and portable integration into existing Java applications and devices because the reference implementation is pure Java. It does not have native dependencies. That makes it very portable. You get all the benefits of the Java ecosystem.

Gupta: How would you sum up JSR 381 and its impact on the Java community?

Sevarac: We believe we created something that can evolve into a friendly Java API for developers. We could not solve all problems for everyone, but I think this is a good starting point. I believe that with feedback from the community, we can move this forward.

Source: oracle.com