Fixing - gateway: Invalid : error cause: gceSync: generic::invalid_argument

 

Bump into this error here and this is because to deploy inference gateway, we need to configure subnet proxy as defined here: https://cloud.google.com/kubernetes-engine/docs/how-to/deploying-gateways#configure_a_proxy-only_subnet

So basically you need to create a new proxy-subnet in a specific region say australia-southeast2. Once you have done that you can create gateway and httproute resources. 

When you create your gke in autopilot mode, the gateway API is enabled by default. But you still need to create a subnet. 

Gateway: Invalid : error cause: gceSync: generic::invalid_argument: Insert: Invalid value for field 'resource.target': 'regions/australia-southeast2/targetHttpProxies/gkegw1-ctc9-default-inference-gateway-3ehzmvfui5bd'. An active proxy-only subnetwork is required in the same region and VPC as the forwarding rule. ===== Route/default/gateway-route: (1.) error cause: gceSync: generic::invalid_argument: Insert: Invalid value for field 'resource.target': 'regions/australia-southeast2/targetHttpProxies/gkegw1-ctc9-default-inference-gateway-3ehzmvfui5bd'. An active proxy-only subnetwork is required in the same region and VPC as the forwarding rule., (2.) error cause: reference-not-found: Error GWCER104: HTTPRoute "default/gateway-route" is misconfigured, err: failed to validate inference pool extension ref: InferencePool {default/vllm-llama3.1-8b-instruct-ips-7b6e0b8f /v1, Resource=services} not found.





Comments

Post a Comment

Popular posts from this blog

gemini cli getting file not defined error

Image
 While trying to run gemini cli on my windows linux subsystem, i bump into this error here - "/Roaming/npm/node_modules/@google/gemini-cli/node_modules/undici/lib/web/webidl/index.js:512 webidl.is.File = webidl.util.MakeTypeAssertion(File)"  "ReferenceError: File is not defined" In Node.js, there’s no built-in File (until very recent Node 20+ with experimental WHATWG APIs). So, unless polyfilled, File is undefined, leading to the ReferenceError. And to resolve this, I had to upgrade to node 22 (might work for node 20) and then I was able to run it.
Read more

NodeJS: Error: spawn EINVAL in window for node version 20.20 and 18.20

Encounter this issue when running autorest (that uses nodejs).  I am using node version:  v18.20.2 After i revert to use node v18.12.0, then i was able to get my app to run without this error. Since i am working with Azure devops yaml, i was able to use the following to resolve my issue.  - task : NodeTool@0   inputs :     versionSource : 'spec'     versionSpec : '18.18.0'
Read more

vllm : Failed to infer device type

Trying to run vllm without gpu will typicaly lands you into this issue. To resolve this issue: we can use set these both. Setting one does not work for me. So i had to configure both settings below:- 1.  environment variable CUDA_VISIBLE_DEVICES = "" 2. in the command line, change to use device cpu and remove  tensor-parallel-size. For example, python3 -m vllm.entrypoints.openai.api_server --port 8080 --model deepseek-ai/DeepSeek-R1 --device cpu --trust-remote-code --max-model-len 4096" References https://docs.vllm.ai/en/latest/getting_started/troubleshooting.html#failed-to-infer-device-type
Read more