gcp agentic agent - how to use serverless model

GCP serverless model are model like gemini-2.5-flash, gemini-3.5-flash that are generally avaiable. This include Claude Sonnet or Haiku 5.5. 

If you ever wanted  your agent to use these model, all you need to do is the following code. 


banking_agent = Agent(
    name="account_assistant",
    model="gemini-2.5-flash",
    instruction="You are a helpful customer support agent. Answer questions concisely.",
    tools=[get_user_account_balance],
    before_model_callback=inspect_llm_input
)

And that's is all. Can you believe it. Deploy it and then go to the playground to test it out. 



The full code base is here :-

import asyncio
from urllib import response
from google.adk.agents import Agent
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.genai import types
from google.adk.agents.callback_context import CallbackContext
from google.adk.models.llm_request import LlmRequest
from vertexai import types as vertex_types
from google.genai import types
import vertexai
import agentplatform
from agentplatform import types   # check below: this is my assumption

# 1. Define the callback function
def inspect_llm_input(callback_context: CallbackContext, llm_request: LlmRequest):
    """Callback triggered directly before ADK sends payload to the LLM."""
    print("=" * 60)
    print("INSPECTING INPUT PASSED TO LLM BY ADK")
    print("=" * 60)
    print(llm_request)
    print("=" * 60 + "\n")
   
    # Return None to continue execution normally
    return None

# -------------------------------------------------------------
# 1. DEFINE THE TOOL FUNCTION
# ADK inspects docstrings and type hints to auto-generate
# the FunctionDeclaration for Gemini.
# -------------------------------------------------------------
def get_user_account_balance(user_id: str, include_pending: bool = False) -> dict:
    """Retrieves the current bank balance for a specific user ID.

    Args:
        user_id: The unique account identifier for the customer.
        include_pending: Whether to include pending uncleared transactions.
    """
    return {
        "user_id": user_id,
        "balance": "$4,250.00",
        "pending": "$120.00" if include_pending else "$0.00"
    }

# -------------------------------------------------------------
# 2. CREATE THE ADK AGENT
# -------------------------------------------------------------
banking_agent = Agent(
    name="account_assistant",
    model="gemini-2.5-flash",
    instruction="You are a helpful customer support agent. Answer questions concisely.",
    tools=[get_user_account_balance],
    before_model_callback=inspect_llm_input
)

# -------------------------------------------------------------
# 3. RUNNER & SESSION ORCHESTRATION
# -------------------------------------------------------------
async def main():

    client = agentplatform.Client(
        project="your-project-id",
        location="us-central1",
    )
   
    # Define resource ID of your gateway
    gateway_resource_id = (
        "projects/your-project-id/locations/us-central1/agentGateways/
my-first-gateway"
    )

    # Deploy agent with gateway configurations attached
    remote_agent = client.runtimes.create(
        agent=banking_agent,
        config={
            "display_name": "Serverless Agent with Gateway1",
            "requirements": ["google-cloud-aiplatform[agent_engines,adk]"],
            "staging_bucket": "gs://my-vertex-bucket-ai",
            "identity_type": vertex_types.IdentityType.AGENT_IDENTITY,
            # Attach the Agent Gateway configuration block
            "agent_gateway_config": {
                "client_to_agent_config": {
                    "agent_gateway": gateway_resource_id
                }
            },
        },
    )

    print(remote_agent.api_resource.spec.deployment_spec.agent_gateway_config)

asyncio.run(main())



Comments

Post a Comment

Popular posts from this blog

NodeJS: Error: spawn EINVAL in window for node version 20.20 and 18.20

Encounter this issue when running autorest (that uses nodejs).  I am using node version:  v18.20.2 After i revert to use node v18.12.0, then i was able to get my app to run without this error. Since i am working with Azure devops yaml, i was able to use the following to resolve my issue.  - task : NodeTool@0   inputs :     versionSource : 'spec'     versionSpec : '18.18.0'
Read more

llama cpp running it in google colab

Image
There are 2 options to do this in Google  colab.  Option 1 (Easiest)  With this option, we can just install the relevant package and then run the model.  First we need to install the required packages using the following command:-  ! pip install -U llama-cpp-python Next,  we will try to get the model using the following command # !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained(   repo_id= "unsloth/Qwen3-0.6B-GGUF" ,   filename= "Qwen3-0.6B-IQ4_NL.gguf" , ) We can see the model being downloaded:- And then we will run the following command to test this model llm.create_chat_completion(   messages = [     {       "role" : "user" ,       "content" : "What is the capital of France?"     }   ] ) And the output will look something like this:-  Option 2 llama.cpp is a powerful inference engine and if you wanted to get it running in google colab, you co...
Read more