Monday, May 31, 2021

Go (Con)Figure

Core Java, Oracle Java Tutorial and Material, Oracle Java Certification, Oracle Java Preparation

Another post about Lightweight Config, a library I’ve recently built from the ground up, after creating various versions of it in previous projects.

Where previously, I’d tried to be clever and prepare the library for lots of possibilities, I took the opposite approach this time. I created a simple monolithic library to solve the core problem, and then in a later version added some new ideas that the simple idea opened up for me.

The Basic Challenge

In a service that has no specific injection framework, like Spring or DropWizard, and especially in a test, what’s the nicest way to load some configuration into an object.

E.g.

username: user

password: foo

urls:

  - http://foo.com

  - http://bar.com

We’d like to load this into an object:

public class Config {

    private String user;

    private String password;

    private List<String> urls;

    // getters and setters

}

This is easily achieved in one line with ConfigLoader.loadYmlConfigFromResource("config.yml", Config.class)

This assumes config.yml is in the appropriate resources.

Placeholders

What if we wanted to interpolate runtime values from environment variables or system properties? Well, that’s really the point of this framework. It’s intended to externalise the setting of values:

username: ${USERNAME}

password: ${PASSWORD}

urls:

  - http://foo.com

  - http://bar.com

Imports

If we have some common snippets of configuration to share between various configuration objects, then an import syntax would be nice. Even better if we can drive the import by a placeholder:

# config.yml

#import ${PROFILE}-config.yml

And then:

# dev-config.yml

username: username

password: foo

urls:

   - http://www.dev.com

Core Java, Oracle Java Tutorial and Material, Oracle Java Certification, Oracle Java Preparation
And so on. Now we can load the config.yml but set PROFILE to determine which child config is also loaded. We can even put some common properties in the parent and it’ll all mash together into the load operation.

Plugins

What if we’re using a password/secret manager to load certain values dynamically? Let’s say we want to express that a certain value may be loaded from a secret:

username: !secret ${SECRET_ID}.user

password: !secret ${SECRET_ID}.password

We can add our custom tag – secret – to the loader:

Config myConfig = new ConfigLoader()

    .withTag("secret", secretPath -> secretsManager.load(secretPath))

    .loadAs("config.yml", Config.class);

Source: codingcraftsman.wordpress.com

Wednesday, May 26, 2021

Apache Arrow on the JVM: Streaming Writes

JVM Exam Prep, JVM Preparation, JVM Career, JVM Certification, Oracle Java Certification, Core Java

Previously we went to create some schemas on Arrow.  On this blog we will have a look on writing through streaming API.

Based on the previous post’s Schema we shall create a DTO for our classes.

package com.gkatzioura.arrow;

import lombok.Builder;

import lombok.Data;

@Data

@Builder

public class DefaultArrowEntry {

    private String col1;

    private Integer col2;

}

Our goal would be to transform those Java objects into a Stream of Arrow bytes.

The allocator creates DirectByteBuffer‘s.

Those buffers are off-heap. You do need to free up the memory used, but for the library user this is done by executing the close() operation on the allocator. In our case our class will implement the Closeable interface which shall do the allocator close operation.

By using the stream api, the data will be streamed to the OutPutStream submitted using the Arrow format.

package com.gkatzioura.arrow;

import java.io.Closeable;

import java.io.IOException;

import java.nio.channels.WritableByteChannel;

import java.util.List;

import org.apache.arrow.memory.RootAllocator;

import org.apache.arrow.vector.IntVector;

import org.apache.arrow.vector.VarCharVector;

import org.apache.arrow.vector.VectorSchemaRoot;

import org.apache.arrow.vector.dictionary.DictionaryProvider;

import org.apache.arrow.vector.ipc.ArrowStreamWriter;

import org.apache.arrow.vector.util.Text;

import static com.gkatzioura.arrow.SchemaFactory.DEFAULT_SCHEMA;

public class DefaultEntriesWriter implements Closeable {

    private final RootAllocator rootAllocator;

    private final VectorSchemaRoot vectorSchemaRoot;

    public DefaultEntriesWriter() {

        rootAllocator = new RootAllocator();

        vectorSchemaRoot = VectorSchemaRoot.create(DEFAULT_SCHEMA, rootAllocator);

    }

    public void write(List<DefaultArrowEntry> defaultArrowEntries, int batchSize, WritableByteChannel out) {

        if (batchSize <= 0) {

            batchSize = defaultArrowEntries.size();

        }

        DictionaryProvider.MapDictionaryProvider dictProvider = new DictionaryProvider.MapDictionaryProvider();

        try(ArrowStreamWriter writer = new ArrowStreamWriter(vectorSchemaRoot, dictProvider, out)) {

            writer.start();

            VarCharVector childVector1 = (VarCharVector) vectorSchemaRoot.getVector(0);

            IntVector childVector2 = (IntVector) vectorSchemaRoot.getVector(1);

            childVector1.reset();

            childVector2.reset();

            boolean exactBatches = defaultArrowEntries.size()%batchSize == 0;

            int batchCounter = 0;

            for(int i=0; i < defaultArrowEntries.size(); i++) {

                childVector1.setSafe(batchCounter, new Text(defaultArrowEntries.get(i).getCol1()));

                childVector2.setSafe(batchCounter, defaultArrowEntries.get(i).getCol2());

                batchCounter++;

                if(batchCounter == batchSize) {

                    vectorSchemaRoot.setRowCount(batchSize);

                    writer.writeBatch();

                    batchCounter = 0;

                }

            }

            if(!exactBatches) {

                vectorSchemaRoot.setRowCount(batchCounter);

                writer.writeBatch();

            }

            writer.end();

        } catch (IOException e) {

            throw new ArrowExampleException(e);

        }

    }

    @Override

    public void close() throws IOException {

        vectorSchemaRoot.close();

        rootAllocator.close();

    }

}

To display the support of batches on Arrow a simple batch algorithm has been implemented within the function. For our example just take into account that data will be written in batches.

Let’s dive into the function.

The vector allocator discussed previously is created

public DefaultEntriesToBytesConverter() {

        rootAllocator = new RootAllocator();

        vectorSchemaRoot = VectorSchemaRoot.create(DEFAULT_SCHEMA, rootAllocator);

    }

Then when writing to a stream, an arrow stream writer is implemented and started

ArrowStreamWriter writer = new ArrowStreamWriter(vectorSchemaRoot, dictProvider, Channels.newChannel(out));

writer.start();

ArrowStreamWriter writer = new ArrowStreamWriter(vectorSchemaRoot, dictProvider, Channels.newChannel(out));

writer.start();

We shall use the vectors in order to populated them with the data. Also reset them but let the pre-alocated buffers to exist

VarCharVector childVector1 = (VarCharVector) vectorSchemaRoot.getVector(0);

            IntVector childVector2 = (IntVector) vectorSchemaRoot.getVector(1);

            childVector1.reset();

            childVector2.reset();

JVM Exam Prep, JVM Preparation, JVM Career, JVM Certification, Oracle Java Certification, Core Java

We use the setSafe operation when writing data. This way if more buffer needs to be allocated shall be done. For this example it’s done on every write, but can be avoided when the operations and the buffer size needed is taken into account.

childVector1.setSafe(i, new Text(defaultArrowEntries.get(i).getCol1()));

                childVector2.setSafe(i, defaultArrowEntries.get(i).getCol2());

Then we write the batch to the stream.

vectorSchemaRoot.setRowCount(batchSize);

                    writer.writeBatch();

Last but not least we close the writer.

@Override

    public void close() throws IOException {

        vectorSchemaRoot.close();

        rootAllocator.close();

    }


Source: javacodegeeks.com

Tuesday, May 25, 2021

Apache Arrow on the JVM: Get Started and Schemas

Arrow is memory format for flat and hierarchical data. It is a popular format used by various big data tools, among them BigQuery. One of the benefits that Arrow brings is that the format of the data has the same byte representation on the languages supported. So apart from the benefits of a columnar memory format there are also the benefits of zero-copy without the serialization overhead.

Oracle JVM, Oracle Java Certified, Oracle Java Preparation, Oracle Java Career, Oracle Core Java

Apache Arrow defines a language-independent columnar memory format for flat and hierarchical data, organized for efficient analytic operations on modern hardware like CPUs and GPUs. The Arrow memory format also supports zero-copy reads for lightning-fast data access without serialization overhead.

Let’s import the libraries

<dependency>
            <groupId>org.apache.arrow</groupId>
            <artifactId>arrow-memory-netty</artifactId>
            <version>${arrow.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.arrow</groupId>
            <artifactId>arrow-vector</artifactId>
            <version>${arrow.version}</version>
        </dependency>

Before starting it is essential to understand that for Read/Write operations on Arrow, byte buffers are used. Operations like reading and writing is continuous exchange of bytes. To make this efficient Arrow comes with a buffer allocator, which can have a certain size or have an automatic expansion.

The libraries backing the allocation management is arrow-memory-netty and arrow-memory-unsafe. We shall use the netty one.

Storing Data in arrow requires a schema. Schemas can be defined programatically

package com.gkatzioura.arrow;
 
import java.io.IOException;
import java.util.List;
 
import org.apache.arrow.vector.types.pojo.ArrowType;
import org.apache.arrow.vector.types.pojo.Field;
import org.apache.arrow.vector.types.pojo.FieldType;
import org.apache.arrow.vector.types.pojo.Schema;
 
public class SchemaFactory {
 
    public static Schema DEFAULT_SCHEMA = createDefault();
 
    public static Schema createDefault() {
        var strField = new Field("col1", FieldType.nullable(new ArrowType.Utf8()), null);
        var intField = new Field("col2", FieldType.nullable(new ArrowType.Int(32, true)), null);
 
        return new Schema(List.of(strField, intField));
    }
 
    public static Schema schemaWithChildren() {
        var amount = new Field("amount", FieldType.nullable(new ArrowType.Decimal(19,4,128)), null);
        var currency = new Field("currency",FieldType.nullable(new ArrowType.Utf8()), null);
        var itemField = new Field("item", FieldType.nullable(new ArrowType.Utf8()), List.of(amount,currency));
 
        return new Schema(List.of(itemField));
    }
 
    public static Schema fromJson(String jsonString) {
        try {
            return Schema.fromJSON(jsonString);
        } catch (IOException e) {
            throw new ArrowExampleException(e);
        }
    }
 
}

Also they have a parseable json representation.

{
  "fields" : [ {
    "name" : "col1",
    "nullable" : true,
    "type" : {
      "name" : "utf8"
    },
    "children" : [ ]
  }, {
    "name" : "col2",
    "nullable" : true,
    "type" : {
      "name" : "int",
      "bitWidth" : 32,
      "isSigned" : true
    },
    "children" : [ ]
  } ]
}

Plus just like Avro you can have complex schemas and embedded values on a field.

public static Schema schemaWithChildren() {
        var amount = new Field("amount", FieldType.nullable(new ArrowType.Decimal(19,4,128)), null);
        var currency = new Field("currency",FieldType.nullable(new ArrowType.Utf8()), null);
        var itemField = new Field("item", FieldType.nullable(new ArrowType.Utf8()), List.of(amount,currency));
 
        return new Schema(List.of(itemField));

Source: javacodegeeks.com

Friday, May 21, 2021

Difference Between Java and C language

Java vs C language

Java and C are both computer programming languages. Both are used to develop software applications. Java is used to create application based on e-commerce and applets while C language is used to create system software.

C language

Core Java, C Language, Oracle Java Exam Prep, Java Preparation, Oracle Java Career

In 1972, the C language was developed at the Bell labs and it was designed to work with the UNIX operating system. The C language is not only used to develop system software rather it is also used to develop portable application software. The C language employs structural programming and it also allows lexical variable scope as well as recursion. Static type system helps in preventing unintended operations.

All the executable code in C is contained inside the functions and their parameters are passed by value. When parameters are passed by functions, pointer values are used. Semicolon is used in order to terminate a statement. A function called “Main function” is the one in which the execution of the program is done.

Following are the features of C language:

• A wide variety of compound operators such as ++, -=, += etc.

• Ad-hoc run time polymorphism is supported by data and function pointers.

• Conditional compilation, file inclusion of source code and a macro definition preprocessor.

• Reserved keywords are small.

JAVA

Core Java, C Language, Oracle Java Exam Prep, Java Preparation, Oracle Java Career

Java is a purely object oriented programming language and it was developed by Sun Microsystems in 1990s. Although it was designed for small programs that run on the browser called applets but later on, it is also being used to create e-commerce applications.

There are five main features of Java language:

• Built-in support for computer networks.

• The code from the remote source can be executed securely.

• Easy to use as it combines the best properties of other programming languages.

• Provides more flexibility to develop software applications because of object oriented approach.

• Allows code written in Java to run on different platforms or Java code is independent of platform.

There is no such thing as manual memory management in Java rather it supports automatic memory management. This saves a lot of time of programmers as they don’t need to free memory manually rather this is achieved by the implementation of automatic garbage collection. Some programmers think that Java consumes more memory as compared to C and C++ programming languages.

Difference between Java and C language

• Java is an object oriented programming language while C is a procedural or structural language.

• Java was developed by Sun Microsystems while C language was developed at Bell labs.

• Java is used to create applets and e-commerce applications based on the web while c language is used to create system software and applications.

• Java employs the concept of objects and classes while C language does not support them.

• Java supports automatic garbage collection while C language does not even though some programmers believe that Java consumes more memory.

Source: differencebetween.com

Monday, May 17, 2021

Java 8 Streams – Group By Multiple Fields with Collectors.groupingBy()

Java 8 Streams, Oracle Java Tutorial and Material, Oracle Java Exam Prep, Oracle Java Preparation, Oracle Java Certification

A guide to group by two or more fields in java 8 streams api. Examples to grouping List by two fields.

1. Overview

In this post, We will learn how to group by multiple fields in java 8 using Streams Collectors.groupingBy() method and example programs with custom objects.

2. Group By Multiple Fields Example in Java 8

First, Create a class Employee with below properties.

int id

String name

String designation

String gender

long salary

Create argument constructor and setter, getters methods for all properties. And also add toString() method to see the Employee object in readable format.

Next, We will try to implement the group by on two fields such as designation and gender. On these two fields get the group by count.

Look at the below examples, you will see the code with the groupingBy() is used twice. This is called as Collectors chaining and observe the output.

package com.oraclejavacertified.java8.collectors.groupby;

public class Employee {

    private int id;

    private String name;

    private String designation;

    private String gender;

    private long salary;

    public Employee(int id, String name, String designation, String gender, long salary) {

        super();

        this.id = id;

        this.name = name;

        this.designation = designation;

        this.gender = gender;

        this.salary = salary;

    }

    public int getId() {

        return id;

    }

    public void setId(int id) {

        this.id = id;

    }

    public String getName() {

        return name;

    }

    public void setName(String name) {

        this.name = name;

    }

    public String getDesignation() {

        return designation;

    }

    public void setDesignation(String designation) {

        this.designation = designation;

    }

    public String getGender() {

        return gender;

    }

    public void setGender(String gender) {

        this.gender = gender;

    }

    public long getSalary() {

        return salary;

    }

    public void setSalary(long salary) {

        this.salary = salary;

    }

    @Override

    public String toString() {

        return "Employee [id=" + id + ", name=" + name + ", designation=" + designation + ", gender=" + gender

                + ", salary=" + salary + "]";

    }

}

Example – Group By Two Properties:

package com.oraclejavacertified.java8.collectors.groupby;

import java.util.ArrayList;

import java.util.List;

import java.util.Map;

import java.util.stream.Collectors;

public class GroupingByMultipleFieldsExample {

    public static void main(String[] args) {

        // Creating List and adding Employees values.

        List<Employee> employeesList = new ArrayList<>();

        employeesList.add(new Employee(101, "Glady", "Manager", "Male", 25_00_000));

        employeesList.add(new Employee(102, "Vlad", "Software Engineer", "Female", 15_00_000));

        employeesList.add(new Employee(103, "Shine", "Lead Engineer", "Female", 20_00_000));

        employeesList.add(new Employee(104, "Nike", "Manager", "Female", 25_00_000));

        employeesList.add(new Employee(105, "Slagan", "Software Engineer", "Male", 15_00_000));

        employeesList.add(new Employee(106, "Murekan", "Software Engineer", "Male", 15_00_000));

        employeesList.add(new Employee(107, "Gagy", "Software Engineer", "Male", 15_00_000));

        // group by - multiple fields

        // Grouping by designation and Gender two properties and need to get the count.

        Map<String, Map<String, Long>> multipleFieldsMap = employeesList.stream()

                .collect(

                        Collectors.groupingBy(Employee::getDesignation, 

                                Collectors.groupingBy(Employee::getGender, 

                                        Collectors.counting())));

        // printing the count based on the designation and gender.

        System.out.println("Group by on multiple properties" + multipleFieldsMap);

    }

}

Output:

Group by on multiple properties

{Software Engineer={Male=3, Female=1}, Manager={Female=1, Male=1}, Lead Engineer={Female=1}}

From the output, you can clearly observe that we can see the count by designation and gender type.

In this program, we have gathered the count of employees but rather than this we can get the list of Employees.

3. Java 8 – Group By Multiple Fields and Collect Aggregated Result into List

First, Collect the list of employees as List<Employee> instead of getting the count. That means inner aggregated Map value type should be List.

To get the list, we should not pass the second argument for the second groupingBy() method.

// Example 2

// group by - multiple fields

// Grouping by designation and Gender two properties and need to get the count.

Map<String, Map<String, List<Employee>>> multipleFieldsMapList = employeesList.stream()

        .collect(

                Collectors.groupingBy(Employee::getDesignation, 

                        Collectors.groupingBy(Employee::getGender)));

// printing the count based on the designation and gender.

System.out.println("Group by on multiple properties and Map key as List" + multipleFieldsMapList);

Output:

Group by on multiple properties and Map key as List 

{

Software Engineer={Male=[

                        Employee [id=105, name=Slagan, designation=Software Engineer, gender=Male, salary=1500000], Employee [id=106, name=Murekan, designation=Software Engineer, gender=Male, salary=1500000], Employee [id=107, name=Gagy, designation=Software Engineer, gender=Male, salary=1500000]], 

                    Female=[Employee [id=102, name=Vlad, designation=Software Engineer, gender=Female, salary=1500000]]}, 

Manager={

        Female=[Employee [id=104, name=Nike, designation=Manager, gender=Female, salary=2500000]], 

        Male=[Employee [id=101, name=Glady, designation=Manager, gender=Male, salary=2500000]]}, 

Lead Engineer={Female=[Employee [id=103, name=Shine, designation=Lead Engineer, gender=Female, salary=2000000]]}}

4. Java 8 – Group By Multiple Fields – Avoid Collectors Chaining

We can avoid the Collectors chaining such as calling groupingby() function several times. If we want to group by 4 fields then need to call Collectors.groupingBy() also 4 times which makes code ugly and not readable.

Let us create the separate class with the group by properties and write implementation for equals(), hashcode() methods for object comparisons.

Creating new class for GroupBy fields makes us to call only once the groupingBy() method.

Below examples are implemented as described and suggested way.

GroupBy Class:

class GroupBy {

    private String designation;

    private String gender;

    public GroupBy(String designation, String gender) {

        super();

        this.designation = designation;

        this.gender = gender;

    }

    @Override

    public int hashCode() {

        return this.designation.length() + this.gender.length();

    }

    @Override

    public boolean equals(Object obj) {

        GroupBy other = (GroupBy) obj;

        if (other.getDesignation().equals(this.designation) && other.getGender().equals(this.gender))

            return true;

        return false;

    }

    public String getDesignation() {

        return designation;

    }

    public void setDesignation(String designation) {

        this.designation = designation;

    }

    public String getGender() {

        return gender;

    }

    public void setGender(String gender) {

        this.gender = gender;

    }

    @Override

    public String toString() {

        return "GroupBy [designation=" + designation + ", gender=" + gender + "]";

    }

}

Employee Class:

package com.oraclejavacertified.java8.collectors.groupby.multiple;

public class Employee {

    private int id;

    private String name;

    private long salary;

    private GroupBy groupBy;

    public Employee(int id, String name, long salary, GroupBy groupBy) {

        super();

        this.id = id;

        this.name = name;

        this.salary = salary;

        this.groupBy = groupBy;

    }

    // setters and getters

    @Override

    public String toString() {

        return "Employee [id=" + id + ", name=" + name + ", salary=" + salary + ", groupBy=" + groupBy + "]";

    }

}

Optimized Group By Multiple Fields Example  

package com.oraclejavacertified.java8.collectors.groupby.multiple;

import java.util.ArrayList;

import java.util.List;

import java.util.Map;

import java.util.stream.Collectors;

public class GroupingByMultipleFieldsExample {

    public static void main(String[] args) {

        // Creating List and adding Employees values.

        List<Employee> employeesList = new ArrayList<>();

        employeesList.add(new Employee(101, "Glady", 25_00_000, new GroupBy("Manager", "Male")));

        employeesList.add(new Employee(102, "Vlad", 15_00_000, new GroupBy("Software Engineer", "Female")));

        employeesList.add(new Employee(103, "Shine", 20_00_000, new GroupBy("Lead Engineer", "Female")));

        employeesList.add(new Employee(104, "Nike", 25_00_000, new GroupBy("Manager", "Female")));

        employeesList.add(new Employee(105, "Slagan", 15_00_000, new GroupBy("Software Engineer", "Male")));

        employeesList.add(new Employee(106, "Murekan", 15_00_000, new GroupBy("Software Engineer", "Male")));

        employeesList.add(new Employee(107, "Gagy", 15_00_000, new GroupBy("Software Engineer", "Male")));

        // Example 1

        // group by - multiple fields

        // Grouping by designation and Gender two properties and need to get the count.

        Map<GroupBy, Long> multipleFieldsMap = employeesList.stream()

                .collect(Collectors.groupingBy(Employee::getGroupBy, Collectors.counting()));

        // printing the count based on the designation and gender.

        System.out.println("Group by on multiple properties" + multipleFieldsMap);

    }

}

Output:

Group by on multiple properties

    { GroupBy [designation=Lead Engineer, gender=Female]=1, 

      GroupBy [designation=Software Engineer, gender=Male]=3, 

      GroupBy [designation=Software Engineer, gender=Female]=1, 

      GroupBy [designation=Manager, gender=Male]=1, 

      GroupBy [designation=Manager, gender=Female]=1

    }


5. Grouping By Using Apache Commons Pair.of()


If you have only two fields and do not wish to use the Record or Another class with the Group by properties then we can use the Pair.of() from apache commons library.

// Example 3
// group by - multiple fields
// Grouping by designation and Gender two properties with Pair.of()
 
Map<Pair<String, String>, Long> multipleFieldsMapPair = employeesList.stream()
        .collect(Collectors.groupingBy(e -> Pair.of(e.getDesignation(), e.getGender()), Collectors.counting()));
 
// printing the count based on the designation and gender.
System.out.println("Group by on multiple fields with Pair - " + multipleFieldsMapPair);

Output:

Group by on multiple fields with Pair - 
{
    (Software Engineer,Male)=3, 
    (Software Engineer,Female)=1, 
    (Lead Engineer,Female)=1, 
    (Manager,Female)=1, 
    (Manager,Male)=1
}

Source: javacodegeeks.com

Wednesday, May 5, 2021

Java 8 Parallel Streams – Custom Thread Pools Examples

A brief intro to custom thread pools and their use in Java 8 parallel streams. Examples on how to use custom pools with the Parallel streams API which avoids common thread pool usage.

1. Introduction

In this tutorial, You’ll learn how to create custom thread pools in Java 8 for bulk data processing with parallel streams powerful API.

Parallel Stream can work well in concurrent environments and these are improved versions of streams performance at the cost of multi-threading overhead.

The main focus in this article is to look at one of the biggest limitations of Stream API and Examples on how can you use the Parallel Streams with the custom thread pools.

Java 8 Parallel Streams, Core Java, Java 8, Oracle Java Tutorial and Material, Oracle Java Learning, Oracle Java Preparation, Java Career

2. Java 8 Parallel Streams


First, let us see how to create Parallel Streams from a Collection.

To make a stream that can run by multiple cores of the processer, you just need to call parallelStream() method.

package com.oraclejavacertified.java8.streams.parallel.streams;
 
import java.util.Arrays;
import java.util.List;
import java.util.stream.Stream;
 
public class ParallelStreamCreation {
 
    public static void main(String[] args) {
 
        List<Integer> intList = Arrays.asList(10, 20, 30, 40, 50);
 
        Stream<Integer> parallelStream = intList.parallelStream();
 
        parallelStream.forEach(value -> System.out.println(value));
    }
}

Output:

[30
40
50
20
10]

You can observe the output that printed the values randomly by different cores.

Internally, it uses SplitIterator and StreamSupport classes to make it run parallelly.

The default processing is done with ForkJoinPool.commonPool() which is shared by the entire application. If you lots of parallel streams that are running at the same time then you may see performance and delay in processing time.

3. Using Custom Thread Pool


As a result of the above approach will use a common ForkJoinPool for all the parallel Streams.

If you have many parallel streams running at the same time and some of them take time longer than expected due to network slowness and those tasks may be blocking the threads from the common pool. Hence, it causes to slow down the tasks and take a longer time to complete.

In these cases, It is good to go with the custom thread pools with the parallel streams combination.

Look at the below program, that runs with 5 threads using ForkJoinPool and inside creating a new parallel stream to find the sum of all numbers for the given range.

package com.oraclejavacertified.java8.streams.parallel.streams;
 
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.ForkJoinPool;
import java.util.stream.Collectors;
import java.util.stream.IntStream;
 
public class CustomPoolParallelStreams {
 
    public static void main(String[] args) throws ExecutionException, InterruptedException {
        parallelStreamProcess();
    }
 
    private static void parallelStreamProcess() throws ExecutionException, InterruptedException {
 
        int start = 1;
        int end = 10000;
 
        List<Integer> intList = IntStream.rangeClosed(start, end).boxed()
                .collect(Collectors.toList());
        System.out.println(intList.size());
 
        ForkJoinPool newCustomThreadPool = new ForkJoinPool(5);
        int actualTotal = newCustomThreadPool.submit(
                () -> {
                     int a = intList.stream().parallel().reduce(0, Integer::sum).intValue();
                     return a;
                }).get();
 
        System.out.println("actualTotal " + actualTotal);
 
    }
 
}

Output:

[10000

actualTotal 50005000]

Java 8 Parallel Streams, Core Java, Java 8, Oracle Java Tutorial and Material, Oracle Java Learning, Oracle Java Preparation, Java Career
Actually, The above program does not come up with the efficient but I have seen the many websites talking about this solution. In fact, this also creating a parallel stream inside ForkJoinPool which again internally consumes threads from a common pool of ForkJoinPool area.

So, If you are running multiple parallel streams then do not use this Steam api parallel method as this might slow other streams give the results in more time.

Here, we have taken the pool count as 5 but you can change it as per your CPU configuration. If you have more then you can fine-tune based on the other tasks.

If you have only one parallel stream then you can use it with a limited pool count.

But, Wait for a java update that parallel stream can take ForkJoinPool as input to limit the number of parallel processes.

Source: javacodegeeks.com

Tuesday, May 4, 2021

Java 8 – How To Read A File?

Java 8, Core Java, Oracle Java Tutorial and Material, Oracle Java Career, Oracle Java Preparation, Oracle Java Certification

A quick guide on how to read the file in older java and new JDK 8 version with example programs.

1. Overview

In this tutorial, We’ll learn how to read a file line by line in java and print the files content onto console with example program.

Read More: 1Z0-900: Java EE 7 Application Developer

First, let us use the older java version for example demonstration and next will learn how to do the same in the newer java 8 api.

2. Java Read File Example

First, Use BufferedReader.readLine() method to get the each line from the file.

Example:

package com.oraclejavacertified.files.read;

import java.io.BufferedReader;

import java.io.FileReader;

import java.io.IOException;

/**

 * Example to read the file in java

 * 

 * @author oraclejavacertified.blogspot.com

 *

 */

public class ReadFileBeforeJava8 {

    public static void main(String[] args) throws IOException {

        String fileLocation = "/Users/workspace/CoreJava/src/main/java/com/oraclejavacertified/files/read/student.txt";

        FileReader fileReader = new FileReader(fileLocation);

        BufferedReader bufferedReader = new BufferedReader(fileReader);

        String line;

        int index = 1;

        while ((line = bufferedReader.readLine()) != null) {

            System.out.println("line " + index + " : " + line);

            index++;

        }

    }

}

Output:

line 1 : student name - Java Learner

line 2 : Location - India

3. Java Example To Read CSV File

Next, Use the same concept to read the csv file in java but first line will be the csv header and the remaining lines will be holding the actual values.

package com.oraclejavacertified.files.read;

import java.io.BufferedReader;

import java.io.FileReader;

import java.io.IOException;

/**

 * Example to read the CSV file in java

 * 

 * @author oraclejavacertified.blogspot.com

 *

 */

public class ReadCSVFileBeforeJava8 {

    public static void main(String[] args) throws IOException {

        String fileLocation = "/Users/workspace/CoreJava/src/main/java/com/oraclejavacertified/files/read/student.csv";

        FileReader fileReader = new FileReader(fileLocation);

        BufferedReader bufferedReader = new BufferedReader(fileReader);

        String csvLine;

        int index = 1;

        String header = bufferedReader.readLine();

        System.out.println("CSV header : "+header);

        while ((csvLine = bufferedReader.readLine()) != null) {

            System.out.println("csv line " + index + " : " + csvLine);

            index++;

        }

    }

}

Output:

CSV header : student id,name,age

csv line 1 : 100,Jhon,30

csv line 2 : 101,Shella,35

csv line 3 : 102,Dhon,40

4. Java 8 Streams – Read File Example – Files.readLines()

Further more, Use method Files.readLines() method to get the all the lines in the form of java 8 Stream. Next, use forEach() method to get the each line of file and print into the console.

package com.oraclejavacertified.files.read;

import java.io.IOException;

import java.nio.file.Files;

import java.nio.file.Path;

import java.util.stream.Stream;

/**

 * Java 8 example to read the file using Streams.

 * 

 * @author oraclejavacertified.blogspot.com

 *

 */

public class Java8ReadFileExample1 {

    public static void main(String[] args) {

        Path filePath = Path.of(

                "/Users/workspace/CoreJava/src/main/java/com/oraclejavacertified/files/read/student.csv");

        Stream<String> stream = null;

        try {

            stream = Files.lines(filePath);

        } catch (IOException e) {

            e.printStackTrace();

        }

        stream.forEach(System.out::println);

    }

}

Output:

student id,name,age

100,Jhon,30

101,Shella,35

102,Dhon,40

5. Java 8 Streams + BufferedReader.lines()

Java 8, Core Java, Oracle Java Tutorial and Material, Oracle Java Career, Oracle Java Preparation, Oracle Java Certification
Finally, learn another new method of java 8 api is BufferedReader.lines() method which returns the Stream<String>.

package com.oraclejavacertified.files.read;

import java.io.BufferedReader;

import java.io.IOException;

import java.nio.file.Files;

import java.nio.file.Path;

import java.nio.file.Paths;

import java.util.List;

import java.util.stream.Collectors;

/**

 * Java 8 example to read the file using Streams + BufferedReader.lines().

 * 

 * @author oraclejavacertified.blogspot.com

 *

 */

public class Java8ReadFileExample2 {

    public static void main(String[] args) throws IOException {

        Path filePath = Path.of(

                "/Users/workspace/CoreJava/src/main/java/com/oraclejavacertified/files/read/student.csv");

        BufferedReader br = Files.newBufferedReader(filePath);

        List<String> list = br.lines().collect(Collectors.toList());

        list.forEach(System.out::println);

    }

}

Source: javacodegeeks.com