What do I want to do?
I was using the features of the following repository to notify Slack about AWS updates. However, since this application was running on Claude, I tried to see if it could be changed to Nova.
Why Nova
Nova has multiple models for various purposes, and above all, it has a significant cost advantage.
see:https://aws.amazon.com/bedrock/pricing/?nc1=h_ls
Let's try it
The article summary is created on line 124 of hogehoge.py in the repository.
def summarize_blog(
blog_body,
language,
persona,
):
"""Summarize the content of a blog post
Args:
blog_body (str): The content of the blog post to be summarized
language (str): The language for the summary
persona (str): The persona to use for the summary
Returns:
str: The summarized text
"""
boto3_bedrock = get_bedrock_client(
assumed_role=os.environ.get("BEDROCK_ASSUME_ROLE", None),
region=MODEL_REGION,
)
beginning_word = ""
prompt_data = f"""
{blog_body}
You are a professional {persona}.
Describe a new update in tags in bullet points to describe "What is the new feature", "Who is this update good for". description shall be output in tags and each thinking sentence must start with the bullet point "- " and end with "\n". Make final summary as per tags. Try to shorten output for easy reading. You are not allowed to utilize any information except in the input. output format shall be in accordance with tags.
In {language}.
The final summary must consists of 1 or 2 sentences. Output format is defined in tags.
(bullet points of the input)(final summary)
Follow the instruction.
"""
max_tokens = 4096
user_message = {
"role": "user",
"content": [
{
"type": "text",
"text": prompt_data,
}
],
}
assistant_message = {
"role": "assistant",
"content": [{"type": "text", "text": f"{beginning_word}"}],
}
messages = [user_message, assistant_message]
body = json.dumps(
{
"anthropic_version": "bedrock-2023-05-31",
"max_tokens": max_tokens,
"messages": messages,
"temperature": 0.5,
"top_p": 1,
"top_k": 250,
}
)
accept = "application/json"
contentType = "application/json"
outputText = "\n"
try:
response = boto3_bedrock.invoke_model(
body=body, modelId=MODEL_ID, accept=accept, contentType=contentType
)
response_body = json.loads(response.get("body").read().decode())
outputText = beginning_word + response_body.get("content")[0]["text"]
print(outputText)
# extract contant inside tag
summary = re.findall(r"([\s\S]*?)", outputText)[0]
detail = re.findall(r"([\s\S]*?)", outputText)[0]
except ClientError as error:
if error.response["Error"]["Code"] == "AccessDeniedException":
print(
f"\x1b[41m{error.response['Error']['Message']}\
\nTo troubeshoot this issue please refer to the following resources.\ \nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\
\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\x1b[0m\n"
)
else:
raise error
return summary, detail
By default, it uses InvokeAPI, so it's not easy to switch between multiple models. Therefore, I tried to switch the model to Nova and rewrote it as follows.
def summarize_blog(
blog_body,
language,
persona,
):
"""Summarize the content of a blog post
Args:
blog_body (str): The content of the blog post to be summarized
language (str): The language for the summary
persona (str): The persona to use for the summary
Returns:
str: The summarized text
"""
boto3_bedrock = boto3.client("bedrock-runtime", region_name=MODEL_REGION)
beginning_word = ""
persona_data = f"""
{persona}
"""
system = [{ "text" : persona_data}]
prompt_data = f"""
{blog_body}
Describe a new update in tags in bullet points to describe "What is described", "Who is this update good for" in a way that a new engineer can follow.
Description shall be output in tags and each thinking sentence must start with the bullet point "- " .
Make final summary as per tags.
Try to shorten output for easy reading.
You are not allowed to utilize any information except in the input.
Output format shall be in accordance with tags.
{language}
The final summary must consist of at least three sentences, including specific use cases in which it is useful.
Output format is defined in tags.
(bullet points of the input)(final summary)
Follow the instruction.
"""
messages = [
{
"role": "user",
"content": [{"text": prompt_data}]
}
]
inf_params = {
"maxTokens": 4096,
"topP": 0.1,
"temperature": 0.5
}
additionalModelRequestFields = {
"inferenceConfig": {
"topK": 20
}
}
outputText = "\n"
try:
response = boto3_bedrock.converse(
modelId=MODEL_ID,
messages=messages,
system=system,
inferenceConfig=inf_params,
additionalModelRequestFields=additionalModelRequestFields
)
# response_body = json.loads(response.get("body").read().decode())
outputText = beginning_word + response['output']['message']['content'][0]['text']
print(outputText)
# extract contant inside tag
summary = re.findall(r"([\s\S]*?)", outputText)[0]
detail = re.findall(r"([\s\S]*?)", outputText)[0]
except ClientError as error:
if error.response["Error"]["Code"] == "AccessDeniedException":
print(
f"\x1b[41m{error.response['Error']['Message']}\
\nTo troubeshoot this issue please refer to the following resources.\ \nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\
\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\x1b[0m\n"
)
else:
raise error
return summary, detail
The following is a sample executed on Nova.
Conclusion
I was able to easily switch from Claude to Nova. You can also try it for a lower cost.
I also loved how easy it was to switch models using the Converse API.
Try it out and see for yourself!
Extra
The cover image was written by Nova Canvas!!
Cool, right?