Streamlining Financial Operations: Automating Transaction Ledgers with AWS Serverless Architecture

Starting a Vinyl Business

Build a scalable application on AWS to sell 1000 Vinyl records in under 1 second.

Goals:

• Create elastic computing system for high-frequency actions
• Ensure seamless transfer of money from customers’ wallets to the company’s bank account.

AI-Powered Developer Experience

Using Amazon Q Developer CLI to create complete AWS Serverless high frequency transaction architecture without writing a single line of code

Technologies Used:

• AWS Lambda: Elastic computing for auto scaling
• Simple Queue (SQS): Event driven system for rapid development
• Elastic Cache: Improves system performance by in-memory storage

Me

AWS Community Builder

• ReCreate: Develop data solutions for business continuity.
• ReBuild: Construct an intelligent data platform for informed decision-making.
• ReThink: Foster an organizational culture that embraces new data technologies.

Agenda

• Understanding the Need for Serverless
• Next Generation Developer Experience (AWS NGDE)
• Demo: Building Serverless Infrastructure with Q Developer CLI without writing a single line of code
• Limitations of Q Developer

Reality of Starting a Vinyl Business: Everyone creates data. Data is stored everywhere.

Reality of Starting a Vinyl Business: Infrastructure falling behind business needs

Communication Channels:

• Share business performance via email, Slack, and Canva.
• Marketers get product info from the pitch deck.
• Designers get materials from Marketers’ Slack chats. Continuous Improvement:

Developers are constantly upgrading infrastructure without fulfilling business value.

From EC2 instance to Lambda Serverless: System Architecture Evolution

Before: Classic EC2 Two-Tier System Architecture:

• Engineers purchase more servers for short periods of high-frequency actions.
• Pay upfront costs.
• Waste idle state computing resources.
• Spend time and efforts to maintain infrastructure.

After: Advantages of Serverless Performance:

• AWS Lambda functions handle all the jobs for you.
• Elastic upscale computing is available at no additional cost.
• No upfront operational and maintenance jobs.
• Helps the business fulfill high-frequency actions within a single second.

AI-Powered Developer Experience

Using Amazon Q Developer to create complete AWS serverless architecture without writing a single line of code

Part 2: AWS NGDE: Next Generation Developer Experience

Amazon Q Developer CLI:

• Generative AI–powered assistant to enhance developer productivity.
• Provides intelligent code suggestions and automation for AWS services.

Building a Scalable Application on AWS

Objective:
Sell 1000 vinyl records in under 1 second.

System Characteristics:

• High-Frequency Transactions: Capable of handling a large number of transactions simultaneously.
• Real-Time Performance Tracking: Monitor system action and track real-time status.

Architecture Overview

• S3: Simple Storage platform, object-based storage
• Lambda: Run Code Without Provisioning or Managing Infrastructure
• Queue: Send, store, receive messages between software components at any volume
• Elastic Cache: Improves system performance by managing info from in-memory caches

Creating Cloudformation with S3 & Lambda

My statement:

CloudFormation Stack: get-smile

Resource 1) S3 Bucket

• Name: get-smile-vinyl

Resource 2) Lambda Function

• Name: get-smile-vinyl
• Runtime: Node.js
• Configuration:
• Function URL enabled (no API Gateway needed)
• Deployment Code; Sample response 'ok'
• Permissions:
• Necessary IAM role with permissions for Lambda to execute and access S3 if needed

3) Additional Notes:

• No extra resources except necessary permission resources.
• Store CloudFormation template file at: "/mnt/c/Users/USER/workspace/get-smile"

Amazon Q Developer builds a complete cloudformation stack to deploy resources.

Q Developer create S3

GetSmileVinylBucket:
   Type: AWS::S3::Bucket
   Properties:
     BucketName: get-smile-vinyl

Q Developer create Lambda, role (execution permission), function URL

GetSmileVinylLambdaFunction:
   Type: AWS::Lambda::Function
   Properties:
     FunctionName: get-smile-vinyl
     Runtime: nodejs18.x
     Role: !GetAtt GetSmileVinylLambdaRole.Arn

 GetSmileVinylLambdaFunctionURL:
   Type: AWS::Lambda::Url
   DependsOn: GetSmileVinylLambdaFunction

 GetSmileVinylLambdaRole:
   Type: AWS::IAM::Role
   Properties:
     ManagedPolicyArns:
       - 'arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole'

Creating a Node.js Lambda with web page

Lambda Get Excel on S3, response web page

Product Excel on S3

Amazon Q Developer build an HTML product list page

My statement:

1) Retrieve product.csv from the get-smile-vinyl bucket using AWS SDK v3.

Product info includes:

• Name (e.g., Smile Dance Extension)
• Model Number (e.g., 20230715-001-123)
• SKU (e.g., 7 inch, 10 inch, 12 inch)
• Image (e.g., full image path, 1024 width x 1024 height)

2) Generate an HTML product list page using information from product.csv.
3) Store the file at the path /mnt/c/Users/USER/workspace/get-smile.

Amazon Q Developer Thinking:

This Lambda function retrieves product information from a CSV file stored in an S3 bucket and generates an HTML product listing page.

Setup Instructions

1. Install dependencies:
2. Configure AWS credentials with appropriate S3 access permissions
3. Deploy to AWS Lambda:
4. Configure function URL to access the function via HTTP

Adding queue and cache to infrastructure

My statement:

• Based on /mnt/c/Users/USER/workspace/get-smile/template.yaml:
• Create a queue named purchase-smile-vinyl.
• Send queue messages to the Lambda function get-smile-vinyl.
• Create an AWS ElastiCache Valkey named get-smile-vinyl for the Lambda function to store and retrieve information.

Amazon Q Developer build a complete cloudformation.

Q Developer create queue

PurchaseSmileVinylQueue:
 Type: AWS::SQS::Queue
 Properties:
   QueueName: purchase-smile-vinyl

Q Developer create Redis Cluster & VPC

# VPC for the resources
VPC:
 Type: AWS::EC2::VPC

GetSmileVinylSubnetGroup:
 Type: AWS::ElastiCache::SubnetGroup

# ElastiCache Valkey (Redis) Security Group
GetSmileVinylSecurityGroup:
 Type: AWS::EC2::SecurityGroup
 Properties:
   VpcId: !Ref VPC

# ElastiCache Valkey (Redis) Cluster
GetSmileVinylCache:
 Type: AWS::ElastiCache::CacheCluster
 Properties:
   ClusterName: get-smile-vinyl
   CacheNodeType: cache.t3.micro
   Engine: redis

Q Developer takes care of everything

Problem:

• Lambda and Redis instances must be on the same VPC.

Amazing performance:

• Q Developer automatically helps me create Lambda network settings for Redis.
• Reduce DevOps workload.
• No need to manually update infrastructure resources.

Intelligent:

• Q Developer knows what is missing to make the infrastructure run, and Q will help me add the configuration.

Reliable:

• No need to specifically state the steps that an Q Developer takes to complete their work.

Advantages:

• Reduce DevOps workload.
• Reduce administrative overhead.
• Reduce human error.

Q Developer create Security Group for Lambda

# Security Group for Lambda to access ElastiCache
LambdaSecurityGroup:
 Type: AWS::EC2::SecurityGroup
 Properties:
   GroupDescription: Security group for Lambda to access ElastiCache
   VpcId: !Ref VPC

# Subnets for ElastiCache
PublicSubnet1:
 Type: AWS::EC2::Subnet
 Properties:
   VpcId: !Ref VPC

Q Developer add VPC config to Lambda

GetSmileVinylLambdaFunction:
 Type: AWS::Lambda::Function
 Properties:
   FunctionName: get-smile-vinyl
   VpcConfig:
     SecurityGroupIds:
       - !GetAtt LambdaSecurityGroup.GroupId
     SubnetIds:
       - !Ref PublicSubnet1
       - !Ref PublicSubnet2
   Environment:
     Variables:
       REDIS_ENDPOINT: !GetAtt GetSmileVinylCache.RedisEndpoint.Address
       REDIS_PORT: !GetAtt GetSmileVinylCache.RedisEndpoint.Port

Fix network issue by Q Developer

Issue:

• Lambda function within a VPC unable to access S3

Error Message:

• Task timed out after 3.01 seconds.
• Duration: 3012.41 ms, Billed Duration: 3000 ms
• Memory Size: 128 MB, Max Memory Used: 34 MB

Solution:

• Establish a VPC endpoint gateway to enable Lambda to access S3.

Demo:

• Fine-tune infrastructure using Amazon Q Developer
• Adding specific resources within existing infrastructure

Fix network issue by Q Developer

Creating VPC endpoint and network setting to infrastructure

My statement:

• Based on /mnt/c/Users/USER/workspace/get-smile/template.yaml:
• Create a VPC with the tag GetSmileVinylVPC.
• Create a VPC Endpoint gateway for S3.
• Store the file at the path /mnt/c/Users/USER/workspace/get-smile.

Q Developer create VPC Endpoint to access S3

# S3 VPC Endpoint
S3VPCEndpoint:
 Type: AWS::EC2::VPCEndpoint
 Properties:
   ServiceName: !Sub 'com.amazonaws.${AWS::Region}.s3'
   VpcId: !Ref VPC
   RouteTableIds:
     - !Ref PublicRouteTable
   PolicyDocument:
     Statement:
       - Effect: Allow
         Resource:
           - !Sub 'arn:aws:s3:::${GetSmileVinylBucket}'
           - !Sub 'arn:aws:s3:::${GetSmileVinylBucket}/*'

Q Developer create route table setting

# Route Table for the VPC
PublicRouteTable:
 Type: AWS::EC2::RouteTable
 Properties:
   VpcId: !Ref VPC


# Associate Route Table with Subnets
PublicSubnet1RouteTableAssociation:
 Type: AWS::EC2::SubnetRouteTableAssociation
 Properties:
   SubnetId: !Ref PublicSubnet1
   RouteTableId: !Ref PublicRouteTable

Let Q Developer create front-end action

My statement:

Objective:

• Implementing a clickable button for claiming an airdrop

Reference File Location:

• /mnt/c/Users/USER/workspace/get-smile/lambda/index.js

Button Details:

• Place under SKU.
• Button Text: "Claim Your Airdrop"

Functionality:

• On button click, call API: {current_url}/purchase?serial_number={serial_number}
• Upon successful API call, display an alert box with the message: “Congratulations!”

Store updated file:

• at: /mnt/c/Users/USER/workspace/get-smile/lambda/index.js

Q Developer create button on HTML

SKU: ${product.SKU}
claim your airdrop

Q Developer create actionable script for button

function claimAirdrop(serialNumber) {
   let currentUrl = window.location.href.split('?')[0];
   let purchaseUrl = \`${currentUrl}/purchase?serial_number=${serialNumber}\`;

   fetch(purchaseUrl)
     .then(data => {
       alert('Congratulations!');
     });
 }

Parallelly Making 1000 API call in under 1 second

Processed all API call within ~3 milliseconds

Benefit:

• No upfront server costs
• Elastic computing power to support peak traffic
• No infrastructure management overhead

Advantage AWS Serverless Architecture

Objective:

• Build organization service-oriented, automated workflow.

Approach:

• Embrace Lean microservice design aligned with business culture.

Flexible:

• Seamlessly integrate with third-party solutions from the AWS Marketplace.

Benefits:

• Highly scalable and elastic, ensuring business continuity and site reliability.

Effective System Design

Integrated Approach:

• A good system design is one where the system itself is integrated into the business environment.
• Avoid designing the system first and then educating others to change the business environment to match the system design.

If system design is not aligned with business:

• Spend time convincing other stakeholders to follow system guidelines.
• Waste resources upgrading functionality without serving any business values.

Reduce Data Silos

Strategy: Integrate business digital assets platform with AWS serverless architecture.

Benefit:

• Enhanced collaboration across teams.
• Improved data integrity and consistency.

Automate Data Management:

• Approach: Adopt Lean microservice design to build a data platform

Benefit:

• Rapid delivery of data services..
• Tailored data platforms that align with unique business cultures.

Think Big

AWS Mission:

• Reduce infrastructure barriers, enabling developers to leverage cutting-edge technology.

Traditional Approach:

• Invest billions in infrastructure.
• Assemble a complete engineering team to support business operations.
• Allocate resources for maintaining and upgrading infrastructure.

With the power of AWS:

• AWS offers comprehensive infrastructure support with over 300 services.
• Customers can concentrate on creating business value.
• No need for upfront infrastructure investment; reduced management overhead.

Part 3: Limitations of Amazon Q Developer

Common problems of AI-powered development assistant

Can Amazon Q Developer CLI Replace DevOps Engineers or Front-End Developers?

Conclusion: No, Q Developer has certain limitations.

Limitations of DevOps:

• Lacks understanding of resource dependencies.
• Fails to ensure business readiness and continuity.
• Utilizes outdated methods and tools for modern business requirements.

Limitations of Front-End Development:

• Does not clarify ambiguous statements by asking further questions.
• Lacks foresight in preparing for future needs.

Q Developer Limitation 1

Issue:

• Uses outdated information as the reference and base point.

Example:

• Latest Lambda JavaScript AWS SDK is v3, but Q CLI provides the AWS SDK v2 codebase.

Solution:

• Declare import AWS SDK v3 to create the system.

Example code snippet:
import { S3Client } from '@aws-sdk/client-s3'; // Using AWS SDK v3

Q Developer Limitation 2

Issue:

• Provides improper code structure when handling tailor-made solutions. Example:
• Lambda function URL
• Tailor-made function URL: A dedicated HTTP(S) URL endpoint for Lambda function.

Incorrect Suggestion:
Error: Schema validation: Property 'FunctionUrlConfig' is not allowed.

FunctionUrlConfig:
   AuthType: NONE

Good News:

• IDE can identify Q Developer incorrect suggestions.
• Combining IDE problem monitoring and AI-powered assistant streamlines the development flow.

Q Developer Suggestion on IDE

GetSmileVinylLambdaFunctionURL:
   Type: AWS::Lambda::Url
   DependsOn: GetSmileVinylLambdaFunction
   Properties:
       AuthType: NONE
       TargetFunctionArn: !GetAtt GetSmileVinylLambdaFunction.Arn

Key Takeaway

• Create a complete AWS serverless architecture without writing any code using Amazon Q Developer
• Design efficient serverless systems for various business cultures
• Build a high-frequency trading serverless system on AWS