gcp agentic platform - monitoring serverless model for example gemini flash

How can you tell if your model is being responsive enough especially for serverless model which you do not have to deploy and configure monitoring?

You can do that via Metric Explorer. 

https://console.cloud.google.com/projectselector/monitoring/metrics-explorer?supportedpurview=project,folder,organizationId

In the metrics, please look for Vertex AI. 



And then you will be able to identify relevant metric as shown here. Here we are able to show breakdown 



And you can see more refine analysis especially if you hitting 429, all the way down to which model you're currently using. 

In my case it is hydrogen and gemini flash







 





Comments

Popular posts from this blog

NodeJS: Error: spawn EINVAL in window for node version 20.20 and 18.20

llama cpp running it in google colab