Serverless applications have been gaining popularity recently because of their scalability and simplicity. In this article, we will create a simple TODO application in Golang using serverless AWS technologies: Lambda, APIGateway, and DynamoDB.

Project setup

First of all, we should create the Golang project:

mkdir todo-app-lambda
cd todo-app-lambda
go mod init github.com/CrazyRoka/todo-app-lambda
touch main.go

It will initialize the Golang project for you. Now we can write starting point for our application in main.go:

package main

import (
	"github.com/aws/aws-lambda-go/lambda"
)

func main() {
	lambda.Start(router)
}

You need to download some libs before proceeding:

go get github.com/aws/aws-lambda-go/lambda
go get github.com/aws/aws-sdk-go-v2
go get github.com/go-playground/validator/v10
go get github.com/aws/smithy-go
go get github.com/google/uuid

Database access layer

Structure definition

First of all, we need to define our model. Simple todo item contains the following:

  • id - identifier in the database to access items easily. We will use UUID as id;
  • name - the name of the todo item;
  • description - detailed description of todo item;
  • status - true if the item is done, or false otherwise.

In Golang, we can define the Todo struct like this:

type Todo struct {
	Id          string `json:"id" dynamodbav:"id"`
	Name        string `json:"name" dynamodbav:"name"`
	Description string `json:"description" dynamodbav:"description"`
	Status      bool   `json:"status" dynamodbav:"status"`
}

Note, I’m using Golang tags to change json and DynamoDB names.

DynamoDBClient initialization

According to Lambda documentation, Lambda environment is initialized once (cold start) and executed several times after that (warm execution). That means we can define and reuse database connections between invocations if we put the client outside our executive function. To do that, we should define the database client variable in the global scope and initialize it in the init method.

const TableName = "Todos"

var db dynamodb.Client

func init() {
	sdkConfig, err := config.LoadDefaultConfig(context.TODO())
	if err != nil {
		log.Fatal(err)
	}

	db = *dynamodb.NewFromConfig(sdkConfig)
}

Internally, AWS passes credentials to Lambda, and we use them with config.LoadDefaultConfig() function call. We will define permissions to Lambda later.

Get Todo item

With everything set, we can start implementing the getItem function. It will find the Todo item by id. The process of calling DynamoDB is a little bit different from traditional databases. Here are some notes:

  • We should marshal and unmarshal every object before and after accessing DynamoDB. In this code example, we are marshaling id to aws.String and unmarshalling returned object to Todo struct. It’ll work because we previously added dynamodbav tags to our struct.
  • Each request to DynamoDB requires input and returns output objects. In this example we pass dynamodb.GetItemInput and retrieve dynamodb.GetItemOutput.
  • We have several edge cases in this function. First, marshaling/unmarshalling may fail, and we should return an error in that case. Secondly, a call to DynamoDB may fail if we don’t have enough permissions, use an invalid database, etc. Lastly, a database may successfully return nil if an object is not found. In that case, we should return 404 NotFound as a response. For now, we return nil from this function and process the result later.
func getItem(ctx context.Context, id string) (*Todo, error) {
	key, err := attributevalue.Marshal(id)
	if err != nil {
		return nil, err
	}

	input := &dynamodb.GetItemInput{
		TableName: aws.String(TableName),
		Key: map[string]types.AttributeValue{
			"id": key,
		},
	}

	log.Printf("Calling Dynamodb with input: %v", input)
	result, err := db.GetItem(ctx, input)
	if err != nil {
		return nil, err
	}
	log.Printf("Executed GetItem DynamoDb successfully. Result: %#v", result)

	if result.Item == nil {
		return nil, nil
	}

	todo := new(Todo)
	err = attributevalue.UnmarshalMap(result.Item, todo)
	if err != nil {
		return nil, err
	}

	return todo, nil
}

Insert Todo item

I like creating different objects for different CRUD operations. We want to insert a new Todo item into the database in this function. However, the id is not known before insertion, and we should generate it ourselves. Also, the status is false because the item is not done yet. That’s why we can use different CreateTodo struct that is passed to our function. Inside the function, we will create Todo struct with all the fields and insert it into the DynamoDB.

type CreateTodo struct {
	Name        string `json:"name" validate:"required"`
	Description string `json:"description" validate:"required"`
}

func insertItem(ctx context.Context, createTodo CreateTodo) (*Todo, error) {
	todo := Todo{
		Name:        createTodo.Name,
		Description: createTodo.Description,
		Status:      false,
		Id:          uuid.NewString(),
	}

	item, err := attributevalue.MarshalMap(todo)
	if err != nil {
		return nil, err
	}

	input := &dynamodb.PutItemInput{
		TableName: aws.String(TableName),
		Item:      item,
	}

	res, err := db.PutItem(ctx, input)
	if err != nil {
		return nil, err
	}

	err = attributevalue.UnmarshalMap(res.Attributes, &todo)
	if err != nil {
		return nil, err
	}

	return &todo, nil
}

Delete Todo item

We should pass the id and delete the entry from DynamoDB with such a key to delete an item.

func deleteItem(ctx context.Context, id string) (*Todo, error) {
	key, err := attributevalue.Marshal(id)
	if err != nil {
		return nil, err
	}

	input := &dynamodb.DeleteItemInput{
		TableName: aws.String(TableName),
		Key: map[string]types.AttributeValue{
			"id": key,
		},
		ReturnValues: types.ReturnValue(*aws.String("ALL_OLD")),
	}

	res, err := db.DeleteItem(ctx, input)
	if err != nil {
		return nil, err
	}

	if res.Attributes == nil {
		return nil, nil
	}

	todo := new(Todo)
	err = attributevalue.UnmarshalMap(res.Attributes, todo)
	if err != nil {
		return nil, err
	}

	return todo, nil
}

Update Todo item

To update the Todo item, we should find it by id and replace the fields with the new ones. Let’s create a separate struct for this operation:

type UpdateTodo struct {
	Name        string `json:"name" validate:"required"`
	Description string `json:"description" validate:"required"`
	Status      bool   `json:"status" validate:"required"`
}

DynamoDB update operation is more complex than previous ones. We should find an item by its id key, check if such item exists and set each field with the new value. We use conditions to verify that item exists because DynamoDB will create a new item instead. To make it easier, we can use an expressions package with its DSL (domain-specific language). Apart from that, we should handle 404 not found cases correctly. We assume that the item was not found if the conditional check failed. Here is the code:

func updateItem(ctx context.Context, id string, updateTodo UpdateTodo) (*Todo, error) {
	key, err := attributevalue.Marshal(id)
	if err != nil {
		return nil, err
	}

	expr, err := expression.NewBuilder().WithUpdate(
		expression.Set(
			expression.Name("name"),
			expression.Value(updateTodo.Name),
		).Set(
			expression.Name("description"),
			expression.Value(updateTodo.Description),
		).Set(
			expression.Name("status"),
			expression.Value(updateTodo.Status),
		),
	).WithCondition(
		expression.Equal(
			expression.Name("id"),
			expression.Value(id),
		),
	).Build()
	if err != nil {
		return nil, err
	}

	input := &dynamodb.UpdateItemInput{
		Key: map[string]types.AttributeValue{
			"id": key,
		},
		TableName:                 aws.String(TableName),
		UpdateExpression:          expr.Update(),
		ExpressionAttributeNames:  expr.Names(),
		ExpressionAttributeValues: expr.Values(),
		ConditionExpression:       expr.Condition(),
		ReturnValues:              types.ReturnValue(*aws.String("ALL_NEW")),
	}

	res, err := db.UpdateItem(ctx, input)
	if err != nil {
		var smErr *smithy.OperationError
		if errors.As(err, &smErr) {
			var condCheckFailed *types.ConditionalCheckFailedException
			if errors.As(err, &condCheckFailed) {
				return nil, nil
			}
		}

		return nil, err
	}

	if res.Attributes == nil {
		return nil, nil
	}

	todo := new(Todo)
	err = attributevalue.UnmarshalMap(res.Attributes, todo)
	if err != nil {
		return nil, err
	}

	return todo, nil
}

List todo items

DynamoDB allows you to list all the entries by using scans. It has some differences from traditional relational databases because this operation may return part of the items with one request. You should query DynamoDB again to retrieve the rest of the items. It’s implemented using token, that points to the latest returned item.

func listItems(ctx context.Context) ([]Todo, error) {
	todos := make([]Todo, 0)
	var token map[string]types.AttributeValue

	for {
		input := &dynamodb.ScanInput{
			TableName:         aws.String(TableName),
			ExclusiveStartKey: token,
		}

		result, err := db.Scan(ctx, input)
		if err != nil {
			return nil, err
		}

		var fetchedTodos []Todo
		err = attributevalue.UnmarshalListOfMaps(result.Items, &fetchedTodos)
		if err != nil {
			return nil, err
		}

		todos = append(todos, fetchedTodos...)
		token = result.LastEvaluatedKey
		if token == nil {
			break
		}
	}

	return todos, nil
}

Lambda handler

We create REST API using AWS Lambda and API gateway in this example. We will define the following requests:

  • GET /todo - fetch all todo items;
  • GET /todo/{id} - fetch todo item by id;
  • POST /todo - insert todo item;
  • PUT /todo - update todo item;
  • DELETE /todo/{id}-delete todo item.

In code, it’s defined like this:

func router(ctx context.Context, req events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	log.Printf("Received req %#v", req)

	switch req.HTTPMethod {
	case "GET":
		return processGet(ctx, req)
	case "POST":
		return processPost(ctx, req)
	case "DELETE":
		return processDelete(ctx, req)
	case "PUT":
		return processPut(ctx, req)
	default:
		return clientError(http.StatusMethodNotAllowed)
	}
}

func processGet(ctx context.Context, req events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	id, ok := req.PathParameters["id"]
	if !ok {
		return processGetTodos(ctx)
	} else {
		return processGetTodo(ctx, id)
	}
}

Helper functions

To make our life easier, we define these helper functions:

func clientError(status int) (events.APIGatewayProxyResponse, error) {

	return events.APIGatewayProxyResponse{
		Body:       http.StatusText(status),
		StatusCode: status,
	}, nil
}

func serverError(err error) (events.APIGatewayProxyResponse, error) {
	log.Println(err.Error())

	return events.APIGatewayProxyResponse{
		Body:       http.StatusText(http.StatusInternalServerError),
		StatusCode: http.StatusInternalServerError,
	}, nil
}

Request handlers

All request handlers unmarshal required arguments from the path and body and pass them to the database layer. They also handle all the errors and validate input objects before processing.

func processGetTodo(ctx context.Context, id string) (events.APIGatewayProxyResponse, error) {
	log.Printf("Received GET todo request with id = %s", id)

	todo, err := getItem(ctx, id)
	if err != nil {
		return serverError(err)
	}

	if todo == nil {
		return clientError(http.StatusNotFound)
	}

	json, err := json.Marshal(todo)
	if err != nil {
		return serverError(err)
	}
	log.Printf("Successfully fetched todo item %s", json)

	return events.APIGatewayProxyResponse{
		StatusCode: http.StatusOK,
		Body:       string(json),
	}, nil
}

func processGetTodos(ctx context.Context) (events.APIGatewayProxyResponse, error) {
	log.Print("Received GET todos request")

	todos, err := listItems(ctx)
	if err != nil {
		return serverError(err)
	}

	json, err := json.Marshal(todos)
	if err != nil {
		return serverError(err)
	}
	log.Printf("Successfully fetched todos: %s", json)

	return events.APIGatewayProxyResponse{
		StatusCode: http.StatusOK,
		Body:       string(json),
	}, nil
}

func processPost(ctx context.Context, req events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	var createTodo CreateTodo
	err := json.Unmarshal([]byte(req.Body), &createTodo)
	if err != nil {
		log.Printf("Can't unmarshal body: %v", err)
		return clientError(http.StatusUnprocessableEntity)
	}

	err = validate.Struct(&createTodo)
	if err != nil {
		log.Printf("Invalid body: %v", err)
		return clientError(http.StatusBadRequest)
	}
	log.Printf("Received POST request with item: %+v", createTodo)

	res, err := insertItem(ctx, createTodo)
	if err != nil {
		return serverError(err)
	}
	log.Printf("Inserted new todo: %+v", res)

	json, err := json.Marshal(res)
	if err != nil {
		return serverError(err)
	}

	return events.APIGatewayProxyResponse{
		StatusCode: http.StatusCreated,
		Body:       string(json),
		Headers: map[string]string{
			"Location": fmt.Sprintf("/todo/%s", res.Id),
		},
	}, nil
}

func processDelete(ctx context.Context, req events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	id, ok := req.PathParameters["id"]
	if !ok {
		return clientError(http.StatusBadRequest)
	}
	log.Printf("Received DELETE request with id = %s", id)

	todo, err := deleteItem(ctx, id)
	if err != nil {
		return serverError(err)
	}

	if todo == nil {
		return clientError(http.StatusNotFound)
	}

	json, err := json.Marshal(todo)
	if err != nil {
		return serverError(err)
	}
	log.Printf("Successfully deleted todo item %+v", todo)

	return events.APIGatewayProxyResponse{
		StatusCode: http.StatusOK,
		Body:       string(json),
	}, nil
}

func processPut(ctx context.Context, req events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	id, ok := req.PathParameters["id"]
	if !ok {
		return clientError(http.StatusBadRequest)
	}

	var updateTodo UpdateTodo
	err := json.Unmarshal([]byte(req.Body), &updateTodo)
	if err != nil {
		log.Printf("Can't unmarshal body: %v", err)
		return clientError(http.StatusUnprocessableEntity)
	}

	err = validate.Struct(&updateTodo)
	if err != nil {
		log.Printf("Invalid body: %v", err)
		return clientError(http.StatusBadRequest)
	}
	log.Printf("Received PUT request with item: %+v", updateTodo)

	res, err := updateItem(ctx, id, updateTodo)
	if err != nil {
		return serverError(err)
	}

	if res == nil {
		return clientError(http.StatusNotFound)
	}

	log.Printf("Updated todo: %+v", res)

	json, err := json.Marshal(res)
	if err != nil {
		return serverError(err)
	}

	return events.APIGatewayProxyResponse{
		StatusCode: http.StatusOK,
		Body:       string(json),
		Headers: map[string]string{
			"Location": fmt.Sprintf("/todo/%s", res.Id),
		},
	}, nil
}

AWS SAM

SAM Template

SAM allows you to build serverless infrastructure for your application using templates and simple CLI. In our example, we should create the AWS Lambda function, DynamoDB and APIGateway.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Resources:
  TodoFunction:
    Type: AWS::Serverless::Function
    Properties:
      Timeout: 10
      Handler: main
      Runtime: go1.x
      Policies:
        - AWSLambdaExecute
        - DynamoDBCrudPolicy:
            TableName: !Ref TodoTable
      Events:
        GetTodo:
          Type: Api
          Properties:
            Path: /todo/{id}
            Method: GET
        GetTodos:
          Type: Api
          Properties:
            Path: /todo
            Method: GET
        PutTodo:
          Type: Api
          Properties:
            Path: /todo
            Method: POST
        DeleteTodo:
          Type: Api
          Properties:
            Path: /todo/{id}
            Method: DELETE
        UpdateTodo:
          Type: Api
          Properties:
            Path: /todo/{id}
            Method: PUT
    Metadata:
      BuildMethod: makefile

  TodoTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: Todos
      AttributeDefinitions:
        - AttributeName: id
          AttributeType: S
      KeySchema:
        - AttributeName: id
          KeyType: HASH
      ProvisionedThroughput:
        ReadCapacityUnits: 2
        WriteCapacityUnits: 2

As you can see, we define two resources here. DynamoDB has id as key and 2 Read and Write capacity units. On the other hand, Lambda uses Golang runtime, has 10 seconds timeout, CRUD access to DynamoDB, and defines API endpoints. This definition creates APIGateway behind the scenes for us. Also, SAM uses makefile to build Lambda artifacts.

Makefile

Makefile allows us to simplify the execution of SAM commands. We define five actions:

  • build - builds an application with sam build. Internally, it will create .aws-sam directory with all the artifacts.
  • build-TodoFunction - this action is called by sam build. It passes ARTIFACTS_DIR environment variable that points to .aws-sam. Also, we disable CGO, because Lambda will fail without this argument.
  • init - this action will deploy the initial infrastructure to AWS. It will ask you several questions and creates a config file in your project. After initialization, you can use make deploy.
  • deploy - builds and deploys your application to AWS. You should initialize the project before calling this action.
  • delete - deletes AWS Cloudformation stack and S3 bucket from AWS. Use this command to to remove altogether all the resources created.
.PHONY: build
build:
	sam build

build-TodoFunction:
	GOOS=linux CGO_ENABLED=0 go build -o $(ARTIFACTS_DIR)/main .

.PHONY: init
init: build
	sam deploy --guided

.PHONY: deploy
deploy: build
	sam deploy

.PHONY: delete
delete:
	sam delete

Summary

In this article, we built the serverless solution in the AWS with Golang programming language. We used Lambda as the execution environment and DynamoDB as a data store.

Simplicity and scalability make Lambda the perfect solution for small and dynamic applications. You can view the final code in my public repo.