Introduction

Hey everyone! Ready to make Google Gemini do some real work? We're diving into function calling, which lets you give Gemini its own set of tools. Think of it like teaching your AI to use your code to solve problems!

I'm Gunpal, the person who created the Google_GenerativeAI SDK for .Net. And I'm thrilled to show you how to make Gemini do amazing things in your C# .NET apps. We'll start by exploring ways to declare and use the "Function Tools", and after that we'll build a super cool project: A Javascript Code Generator and Executor. Get ready to turn Gemini into a code-executing wizard!

Part 1: Easy Ways to Add Tools to Gemini

We've got a few ways to add tools, and they're all designed to be simple and powerful using C# Attributes.

1. QuickTools

Got a quick task for Gemini? QuickTools are your go-to. They turn your methods into tools in a flash!

//Define a method, it can be a named method
var doThisJob = (async ([Description("Find a student's info")] StudentQuery query) =>
{
    // ... your code to find student info ...
    return new StudentInfo { Name = query.StudentName, Grade = "A+" };
});

//Create a QuickTool
var myTool = new QuickTool(doThisJob, "FindStudent", "Get a student's info.");

//Initialize Generative Model
var gemini = new GenerativeModel(apiKey: "YOUR_API_KEY", GoogleAIModels.Gemini2Flash);

//Add Function Tool into model
gemini.AddFunctionTool(myTool);

//Usage
var answer = await gemini.GenerateContentAsync("What's John's grade?").ConfigureAwait(false);

Why it's cool: It's the fastest way to add a simple tool!

2. Turn Methods into Function Tools with Source Code Generation

Want a cleaner, more efficient way to add tools? Just Annotate your methods with FunctionTool attribute, and our built-in source generator will do the rest! It takes care of all the behind-the-scenes work, like creating JSON schemas and serializers, making everything run smoother and faster. and Guess what! it will be NativeAOT/Trimming compatible right away!

//Define Method
[FunctionTool(GoogleFunctionTool = true)]
[Description("Get a page from a book.")]
public static Task<string> GetBookPageAsync(string book, int page, CancellationToken cancellationToken=default)
{
    return Task.FromResult("The sun was shining...");
}

//Initialize GenerativeModel
var gemini = new GenerativeModel(apiKey: "YOUR_API_KEY", GoogleAIModels.Gemini2Flash);

//Create instance of auto generated Tools class
var myTools = new Tools([GetBookPageAsync]);

//Add to model
gemini.AddFunctionTool(myTools);

// Example usage:
var result = await gemini.GenerateContentAsync("Give me the first page of 'The Adventures of Tom Sawyer'").ConfigureAwait(false);

Why it's awesome: Super clean, efficient, and the source generator handles all the technical details.

3. Making Reusable Plugins with Interfaces

If you need to create reusable sets of functions, using an interface is perfect. It's like building a plugin that you can use over and over! We use the GenerateJsonSchema attribute on the interface.

//Declare Interface
[GenerateJsonSchema(GoogleFunctionTool = true)]
public interface IWeatherTools
{
    [Description("Get the weather.")]
    WeatherInfo GetWeather(
        [Description("City name")]
        string city);
}

//Implement the interface
public class WeatherInfo
{
    public string City { get; set; }
    public string Temp { get; set; }
}

public class WeatherService : IWeatherTools
{
    public WeatherInfo GetWeather(string city)
    {
        return new WeatherInfo { City = city, Temp = "75 degrees" };
    }
}

//Initialize the service/tool class
var weatherService = new WeatherService();

//Initialize Gemini Model
var gemini = new GenerativeModel(apiKey: "YOUR_API_KEY", GoogleAIModels.Gemini2Flash);

//Add function tool to model
var weatherTool = weatherService.AsGoogleFunctionTool();
gemini.AddFunctionTool(weatherTool);

// Example usage:
var result = await gemini.GenerateContentAsync("What is the weather in London?").ConfigureAwait(false);

Why it's awesome: It's super organized and great for building reusable plugins with multiple functions.

Controlling How Gemini Uses Tools

You can tell Gemini how to use your tools.

gemini.FunctionCallingBehaviour = new FunctionCallingBehaviour
{
    AutoCallFunction = false, // For manual control over execution, Gemini suggests the tool and SDK will not execute the function
    AutoReplyFunction = true,  // Reply back with the function results automatically
    AutoHandleBadFunctionCalls = false // Don't try to fix mistakes by Gemini.
};

Part 2: Let's Build a Javascript Code Runner!

Now, let's build something truly exciting! We'll make Gemini write Javascript code to solve user-defined problems and then run it using the Jint library. Imagine telling Gemini, "Calculate the 4th power of the 67th prime number," and it does it!

Step 1: Create a Console App and Install the Necessary Packages

First, create a new console application targeting .NET 8.0 or later. Then, install the Google_GenerativeAI.Tools and Jint NuGet packages:

dotnet new console -f net8.0
dotnet add package Google_GenerativeAI.Tools
dotnet add package Jint

Step 2: Make a Function to Run Code

We'll create a function that takes Javascript code and runs it. Let's start with a simple example:

public static string RunJavascript(string code)
{
    try
    {
        var engine = new Engine();
        var result = engine.Evaluate(code).ToString();
        return result;
    }
    catch (JavaScriptException ex)
    {
        return $"Oops! Error: {ex.Message}";
    }
}

// Example usage:
// var result = RunJavascript("2+2");

Now, let's create a more sophisticated version that captures console output and handles various data types, and add description for Google Gemini to understand it.

//Description attributes for LLM to understand the use case of Tool
[Description("this function executes javascript code. pass any javascript code to execute.")]
public string? RunJavascriptCode([Description("Javascript code to execute")] string code)
{
    //Initialize Javascript Engine
    using var engine = new Engine();
    var sb = new StringBuilder();

    //Console outputs client side
    Console.WriteLine("Result from Javascript Engine:");

    //console.log definition
    engine.SetValue("console", new
    {
        log = new Action<object>(o =>
        {
            var output = o switch
            {
                JsValue jsValue when jsValue.IsString() => jsValue.AsString(),
                JsValue jsValue => jsValue.ToObject()?.ToString() ?? string.Empty,
                string str => str,
                _ => JsonSerializer.Serialize(o)
            };
            Console.WriteLine(output);
            sb.AppendLine(output);
        })
    });

    //execute the code
    var evaluationResult = engine. Evaluate(code);

    //return the value with correct data type
    return evaluationResult.Type switch
    {
        Types.Null => sb.ToString(),
        _ when evaluationResult.IsArray() => evaluationResult.AsArray()
            .Select(j => j.ToObject())
            .Where(obj => obj != null)
            .ToList(),
        _ => evaluationResult.ToObject()
    };
}

// Example usage:
var result = RunJavascriptCode("console.log('Hello from Jint!');

Step 3: Integrate with Gemini and Run the App

Here's the code to set up Gemini and run the Javascript code:

using GenerativeAI.Tools;
using Jint;
using Jint.Runtime;
using Jint.Runtime.Interop;
using System.Text;
using System.Text.Json;

// ... (Your RunJavascriptCode function here) ...

//Initialize Generative Model
var apiKey = Environment.GetEnvironmentVariable("GOOGLE_API_KEY", EnvironmentVariableTarget.User);
restart:
var model = new GenerativeModel(apiKey, GoogleAIModels.Gemini2Flash);


var javascriptExecutor = new QuickTool(RunJavascriptCode);

//Create a multi turn chat session
var chat = model.StartChat();

//Add executor to the chat
chat.AddFunctionTool(javascriptExecutor);

//Specify System instruction
chat.SystemInstruction = "Your job is to create javascript code and pass it to RunJavascriptCode for execution. \r\n- Always use RunJavascriptCode function to execute the javascript code.\r\n- use console.log to print the results.";

//Write some information to Console to make some sense
Console.WriteLine(
    "Tell me what to do, and I'll whip up some JavaScript magic to get it done! For example, I can count from 1 to 100.");
Console.WriteLine("Examples:");
Console.WriteLine("Print fibonacci numbers up to 100");
Console.WriteLine("Print the first 100 prime numbers in reverse order");
Console.WriteLine("calculate the 4th power of 67th prime number");
Console.WriteLine("calculate the acceleration due to gravity on mars");
Console.WriteLine("** Note: If I start to hallucinate or get confused. Just type restart and press enter!");

//Start Loop
while (true)
{
    //Input Prompt
    Console.WriteLine();
    Console.WriteLine("Input: ");
    var input = Console.ReadLine();
    if(input == "restart")
        goto restart;

    var prompt =
        $"\r\nExecute the javascript code for this task:\r\n\r\n{input}\r\n\r\n";

    //Execute    
    var result = await chat.GenerateContentAsync(prompt).ConfigureAwait(false);

    //Print Results
    Console.WriteLine();
    Console.Write("Result from Gemini:\r\n");
    Console.WriteLine(result.Text());
}

Full Source code sample is available here

Conclusion

And there you have it! We've successfully transformed Google Gemini into a powerful Javascript code executor, all thanks to the magic of function tools and the Google_GenerativeAI SDK as well as Jint. With just a few lines of code, you can bridge the gap between AI-generated code and real-world execution.

This example showcases the incredible flexibility and power of function calling. Whether you're building complex AI-powered applications or simply exploring the possibilities of generative AI, the Google_GenerativeAI SDK provides the tools you need to bring your ideas to life.

Remember, the possibilities are endless. You can extend Gemini's capabilities with your own custom functions, integrate it with existing systems, and create truly intelligent applications that solve real-world problems.

Happy coding!

Side Note: I used Gemini help to organise my ideas into presentable blog post and English is not my first language. This post is intended to educate beginners in A.I.