mitzen

GLM OCR is pretty lightweight and fit for purpose when i tried it out on invoice images.  Here is the code for setting up and running it locally.  """ Extract line items from an invoice image using a local model served at http://localhost:8888/v1, called through LiteLLM. Setup:     pip install litellm pydantic     export LOCAL_API_KEY="your-local-server-key" Usage:     python extract_invoice_local.py invoice.jpg """ import base64 import json import mimetypes import os import re import sys from typing import Optional import litellm from pydantic import BaseModel # Same settings as your LiteLlm config --------------------------------------- MODEL = "openai/ggml-org/GLM-OCR-GGUF" API_BASE = "http://localhost:8888/v1" API_KEY = "sk-unsloth-4d0a1b198bd177a2a72ee1954585342a"   # keep the key out of source code TEMPERATURE = 0.0 # Ask the server to constrain output to the schema (llama.cpp-based server...

We have been dealing with alot of text completion and it is time that we try to use model with image. This is a sample code that we can use to extract contents from an image (invoice) and return line item in json format - so we can proceed it or fill up a form.  I assume you have enable llama-4-maverick serveless instance.  """ Extract line items from an invoice image using Llama 4 Maverick on Vertex AI, through the OpenAI-compatible Chat Completions endpoint. Setup:     pip install openai pydantic google-auth requests     gcloud auth application-default login      # or use a service account Usage:     python extract_invoice_vertex.py invoice.jpg """ import base64 import json import mimetypes import re import sys from typing import Optional import google . auth import google . auth . transport . requests from openai import OpenAI from pydantic import BaseModel PROJECT_ID = "YOUR-PROJECT-ID" LOCATION = "us-ea...

Sometimes we would like to make use of a deployed model in AI platform and this is what post intent to achieve. We wrap our class in BaseLLM and then implement the relevant code in this case override the based method generate_content_async. And plug it into our root agent and use runner to execute this code. This make it really easy to use. import asyncio from typing import AsyncGenerator from google . cloud import aiplatform from google . adk . agents import Agent from google . adk . models import BaseLlm , LlmRequest , LlmResponse from google . genai import types from google . adk . runners import Runner from google . adk . sessions import InMemorySessionService from google . genai import types class VertexEndpointLlm ( BaseLlm ):     project : str     location : str     endpoint_id : str     async def generate_content_async (         self , llm_request : LlmRequest , stream : bool = False  ...

You don't necessary have to deploy llama-4 to use it, all we need to do is 'enable' and then we will get an endpoint where we can consume it.  We can use curl command :- export ENDPOINT=us-east5-aiplatform.googleapis.com exort REGION=us-east5 export PROJECT_ID="project-id"  curl \   - X POST \   - H "Authorization: Bearer $(gcloud auth print-access-token)" \   - H "Content-Type: application/json" https : // $ { ENDPOINT } / v1 / projects / $ { PROJECT_ID } / locations / $ { REGION } / endpoints / openapi / chat / completions \   - d '{"model":"meta/llama-4-maverick-17b-128e-instruct-maas", "stream":true, "messages":[{"role": "user", "content": "Summer travel plan to Paris"}]}'  Or we can just use to consume it via python OpenAI endpoint :- from openai import OpenAI   import google . auth , google . auth . transport . requests creds , _ = ...

Your AI customer service agent passed every test on your local machine. Then it went live and started promising out-of-stock inventory and inventing fake refund policies. We perform evaluation on the target model to ensure it working as expected. To set it up, goto AI platform and then under "Evaluation" click on "New evaluation". We have options to setup eval set for completed dataset (where you have prompt and response) - this response is of course generated by your AI.  Data format required is jsonl with your own pre-defined column - as long as you know which column your prompt and response lies.  In this setup, we are going to use static approach - where we are dealing with eval set from a static data file.  And then when prompt to upload a jsonl file, we can choose "Upload" Then we will use a sample jsonl file here :- { "eval_id" : "product_info_check" , "prompt" : "Do you have wireless headphones in stock?" , ...

This is a design for invoice processing system - the focus is performance, reliability and cost and the workflow is after client made their submission, they expect to be getting json as an output for entry or every item in the invoice and then fill up their web form and then user validate if everything is ok and finally click submit.  In our agent, we also have a validator but that to ensure we returning legit data.  Agent is specialized and do not have too many context poisoning. If model validator checks out ok, then we proceed to move to the next - which return json results back to user.  Another reason reason for this design, lets say our model is down - somewhere somehow - we are able to retain the client submission and help to processes it later. If for some reason, the default model is not able to decipher the image, model validator can try better model to see if it helps. But this can incur additioan cost. Given that the model can't recognized it, do we want to re...

 When we are using the model to perform tool calling it is important to prompt correctly. Let''s see the difference if we were to do it this way  Prompt #1  When we using the prompt below and notice the output  banking_agent = Agent (     name = "account_assistant" ,     model = model ,     instruction = "You are a helpful customer support agent. Answer questions concisely." ,     tools = [ get_user_account_balance ],     before_model_callback = inspect_llm_input ) Output from prompt1 If i specifiy or prompt the model to specifically "use the tool", the model would call the function. Prompt #2  When we change our prompt slight to ask it to use the tool.  # ------------------------------------------------------------- # 2. CREATE THE ADK AGENT # ------------------------------------------------------------- banking_agent = Agent (     name = "account_assistant" ,     model = model , ...