AWS SDK 2 for Java and storing a Json in DynamoDB

AWS DynamoDB is described as a NoSQL key-value and a document database. In my work I mostly use the key-value behavior of the database but rarely use the document database features, however  the document database part is growing on me and this post highlights some ways of using the document database feature of DynamoDB along with introducing a small utility library built on top of AWS SDK 2.X for Java that simplifies using document database features of AWS DynamoDB

The treatment of the document database features will be very high level in this post, I will plan a follow up which goes into more details later

DynamoDB as a document database

So what does it mean for AWS DynamoDB to be treated as a document database. Consider a json representation of an entity, say something representing a Hotel:

{

    "id": "1",

    "name": "test",

    "address": "test address",

    "state": "OR",

    "properties": {

        "amenities":{

            "rooms": 100,

            "gym": 2,

            "swimmingPool": true                   

        }                    

    },

    "zip": "zip"

}

This json has some top level attributes like “id”, a name, an address etc. But it also has a free form “properties” holding some additional “nested” attributes of this hotel. 

A document database can store this document representing the hotel in its entirety OR can treat individual fields say the “properties” field of the hotel as a document. 

A naive way to do this will be to simply serialize the entire content into a json string and store it in, say for eg, for the properties field transform into a string representation of the json and store in the database, this works, but there are a few issues with it. 

1. None of the attributes of the field like properties can be queried for, say if I wanted to know whether the hotel has a swimming pool, there is no way just to get this information of of the stored content. 

2. The attributes cannot be filtered on – so say if wanted hotels with atleast 2 gyms, this is not something that can be filtered down to. 

A document database would allow for the the entire document to be saved, individual attributes, both top level and nested ones, to be queried/filtered on. 

So for eg, in the example of “hotel” document the top level attributes are “id”, “name”, “address”, “state”, “zip” and the nested attributes are “properties.amenities.rooms”, “properties.amenities.gym”, “properties.amenities.swimmingPool” and so on.

AWS SDK 2 for DynamoDB and Document database support

If you are writing a Java based application to interact with a AWS DynamoDB database, then you would have likely used the new AWS SDK 2 library to make the API calls. However one issue with the library is that it natively does not support a json based document model. Let me go into a little more detail here.  

From the AWS SDK 2 for AWS DynamoDB’s perspective every attribute that is saved is an instance of something called an AttributeValue. A row of data, say for a hotel, is a simple map of “attribute” names to Attribute values, and a sample code looks something like this:

val putItemRequest = PutItemRequest.builder()

    .tableName(TABLE_NAME)

    .item(

        mapOf(

            ID to AttributeValue.builder().s(hotel.id).build(),

            NAME to AttributeValue.builder().s(hotel.name).build(),

            ZIP to AttributeValue.builder().s(hotel.zip).build(),

            STATE to AttributeValue.builder().s(hotel.state).build(),

            ADDRESS to AttributeValue.builder().s(hotel.address).build(),

            PROPERTIES to objectMapper.writeValueAsString(hotel.properties),

            VERSION to AttributeValue.builder().n(hotel.version.toString()).build()

        )

    )

    .build()

dynamoClient.putItem(putItemRequest)

Here a map of each attribute to an AttributeValue is being created with an appropriate “type” of content, “s” indicates a string, “n” a number in the above sample.

There are other AttributeValue types like “m” representing a map and “l” representing a list.

The neat thing is that “m” and “l” types can have nested AttributeValues, which maps to a structured json document, however there is no simple way to convert a json to this kind of an Attribute Value and back.

So for eg. if I were to handle the raw “properties” of a hotel which understands the nested attributes, an approach could be this:

val putItemRequest = PutItemRequest.builder()

    .tableName(TABLE_NAME)

    .item(

        mapOf(

            ID to AttributeValue.builder().s(hotel.id).build(),

            NAME to AttributeValue.builder().s(hotel.name).build(),

            ZIP to AttributeValue.builder().s(hotel.zip).build(),

            STATE to AttributeValue.builder().s(hotel.state).build(),

            ADDRESS to AttributeValue.builder().s(hotel.address).build(),

            PROPERTIES to AttributeValue.builder()

                .m(

                    mapOf(

                        "amenities" to AttributeValue.builder()

                            .m(

                                mapOf(

                                    "rooms" to AttributeValue.builder().n("200").build(),

                                    "gym" to AttributeValue.builder().n("2").build(),

                                    "swimmingPool" to AttributeValue.builder().bool(true).build()

                                )

                            )

                            .build()

                    )

                )

                .build(),

            VERSION to AttributeValue.builder().n(hotel.version.toString()).build()

        )

    )

    .build()

Introducing the Json to AttributeValue utility library

This is exactly where the utility library that I have developed comes in. 

Given a json structure as a Jackson JsonNode it converts the Json into an appropriately nested AttributeValue type and when retrieving back from DynamoDB, can convert the resulting nested AttributeValue type back to a json. 

The structure would look exactly similar to the handcrafted sample shown before. So using the utility saving the “properties” would look like this:

val putItemRequest = PutItemRequest.builder()

    .tableName(TABLE_NAME)

    .item(

        mapOf(

            ID to AttributeValue.builder().s(hotel.id).build(),

            NAME to AttributeValue.builder().s(hotel.name).build(),

            ZIP to AttributeValue.builder().s(hotel.zip).build(),

            STATE to AttributeValue.builder().s(hotel.state).build(),

            ADDRESS to AttributeValue.builder().s(hotel.address).build(),

            PROPERTIES to JsonAttributeValueUtil.toAttributeValue(hotel.properties),

            VERSION to AttributeValue.builder().n(hotel.version.toString()).build()

        )

    )

    .build()

dynamoClient.putItem(putItemRequest)

and when querying back from DynamoDB, the resulting nested AttributeValue converted back to a json this way(Kotlin code in case you are baffled by the “?let”):

properties = map[PROPERTIES]?.let { attributeValue ->

    JsonAttributeValueUtil.fromAttributeValue(

        attributeValue

    )

} ?: JsonNodeFactory.instance.objectNode()

The neat thing is even the top level attributes can be generated given a json representing the entire Hotel type. So say a json representing a Hotel is provided:

val hotel = """

    {

        "id": "1",

        "name": "test",

        "address": "test address",

        "state": "OR",

        "properties": {

            "amenities":{

                "rooms": 100,

                "gym": 2,

                "swimmingPool": true                  

            }                    

        },

        "zip": "zip"

    }

""".trimIndent()

val attributeValue = JsonAttributeValueUtil.toAttributeValue(hotel, objectMapper)

dynamoDbClient.putItem(

    PutItemRequest.builder()

            .tableName(DynamoHotelRepo.TABLE_NAME)

            .item(attributeValue.m())

            .build()

    )

Continue reading

How to create AWS Lambda function with Java

In this post, we will see how we can create AWS Lambda function in Java and I tell you, it is quite easy to do so…

Basically, there are three ways in which we can create AWS Lambda function :

– By implementing RequestHandler interface

– By implementing RequestStreamHandler interface

– Custom implementation, which does not require us to implement any AWS specific interface

AWS Lambda function by implementing RequestHandler interface

For using this method of creating AWS lambda function, we need to have following dependency in our project :

<dependency>

 <groupId>com.amazonaws</groupId>

 <artifactId>aws-lambda-java-core</artifactId>

 <version>1.1.0</version>

</dependency>

And below is how your class will look like :

package com.blogspot.javasolutionsguide;

import com.amazonaws.services.lambda.runtime.Context;

import com.amazonaws.services.lambda.runtime.RequestHandler;

public class HelloWorldWithRequestHandler implements RequestHandler<object,object> {

 @Override

 public Object handleRequest(Object input, Context context) {

  return String.format("Hello %s%s.", input ," from " + context.getFunctionName());

 }

} 

</object,object>

Once you have created maven project with above dependency and class in your project, maven build the project, which will create jar for you in the target folder of your project.

Now open the AWS Console, go to Services and search for AWS Lambda.

On the following screen ,click on Create Function.

On following screen, enter Function name “HelloWorld” and choose Runtime as Java 11.

In Permissions section, choose “Create a new role with basic Lambda permissions”and AWS Lambda will create and execution role with name HelloWorld-role-jc6cmpnj. This role is required to allow AWS Lambda to upload logs to AWS Cloudwatch logs.

Click on Create Function.

You will see following screen, where it says that “Successfully created the function HelloWorld.You can now change its code and configuration.To invoke your function with a test event, choose Test”.

Now in the Function code section, click on the upload button and browse on your computer for the jar that you created earlier.

– Also, in the Handler textbox, replace

example with package name where your “HelloWorldWithRequestHandler” class is residing, which in our case it is “

com.blogspot.javasolutionsguide“

– And replace Hello with “HelloWorldWithRequestHandler”.

– And replace handleRequest will stays as is ,because we also have same method name in our class.

Click on Save button on extreme right side of the screen.

Now to test our lambda function, we need to configure test event(s),for which we will click on “Select a Test event” drop down and then click on “Configure test events”.

You will see following screen.Enter Event name as “HelloWorldEvents” and replace following

{

  “key1”: “value1”,

  “key2”: “value2”,

  “key3”: “value3”

}

with just your name like as below :

“Gaurav Bhardwaj”

and click on Create button.

Now click on Test button and you should see your lambda function executed successfully with message “Hello Gaurav Bhardwaj from HelloWorld”,which is the output returned by our lambda function.

If you click on the logs link in the above screen, it will take you to the AWS Cloudwatch screen where you can see that for your lambda function a LogGroup has been created and under which you have LogStream where you can see logs of your lambda function.This was the reason we assigned role to our lambda function, because AWS lambda used that role to push logs to the Cloudwatch.

AWS Lambda function by implementing RequestStreamHandler interface

In this case you need to follow exact same steps as in above case.It is just that in the code you need to implement RequestStreamHandler interface rather than RequestHandler interface as below.

package com.blogspot.javasolutionsguide;

import java.io.IOException;

import java.io.InputStream;

import java.io.OutputStream;

import com.amazonaws.services.lambda.runtime.Context;

import com.amazonaws.services.lambda.runtime.RequestStreamHandler;

public class HelloWorldWithRequestStreamHandler implements RequestStreamHandler {

 @Override

 public void handleRequest(InputStream inputStream, OutputStream outputStream, Context context) throws IOException {

  int letter;

        while((letter = inputStream.read()) != -1)

        {

            outputStream.write(Character.toUpperCase(letter));

        }

 }

}

AWS Lambda function by custom implementation, which does not require us to implement any AWS specific interface

You can also have your custom lambda function ,which does not follow signature from some AWS specific interface.You can even omit Context object as well, if you don’t want it.

In the following code,I have tried to put two handler methods, one with Context object and one without Context object.To test these both ,you just need to change the name of the method in the AWS console and it will start hitting that method.

Also ,we can see that from Context object ,we can get lots of useful information like name of AWS fucnton,its version,ARN,how much memory is allocated to the function(by default it is 512 mb) .

package com.blogspot.javasolutionsguide;

import com.amazonaws.services.lambda.runtime.Context;

public class HelloWorld {

  

        //Handler method without Context

 public String sayHelloWorldWithoutContext(String name) {

  return String.format("Hello %s.", name);

 }

  

 //We need to add aws-lambda-java-core dependency if we add Context as parameter.

 public String sayHelloWorldWithContext(String name, Context context) {

   

  context.getLogger().log("Lambda Function Name:" + context.getFunctionName() +

    "Version:" + context.getFunctionVersion() + 

    "Arn:" + context.getInvokedFunctionArn() +

    "Allocated Memory:" + context.getMemoryLimitInMB() +

    "Remaining Time:"+ context.getRemainingTimeInMillis());

  return String.format("Hello %s.", name);

 }

}

Also in the following example ,we can see that if we have two handler methods with same name in our class,the handler method which has Context object as its last parameter will be called.

package com.blogspot.javasolutionsguide;

import com.amazonaws.services.lambda.runtime.Context;

public class HelloWorldWithMultipleHandlersWithSameName {

  

 public String handler(String name) {

  return String.format("Hello %s.", name);

 }

  

 public String handler(String name, Context context) {

   

  return String.format("Hello %s%s.", name,   " Memory Allocated:" + context.getMemoryLimitInMB());

 }

}

Continue reading