Posts

gcp setting up workload identity authentication with github actions

Image
We need to setup our :- 1. Workload identity pool  2 Create a provider  3. Create service account called -  4. And setup the required attributes mapping. 5. Create our yaml pipeline  I create a pool call github. And next, I create my provider and call it githubprovider.  You can see my mappings here :- Run the following command to add service account to the github worker pool :-   gcloud iam service-accounts add-iam-policy-binding "github-wif@project-f54eacb9-b4e5-40b9-b0c.iam.gserviceaccount.com" \   --project="project-f54eacb9-b4e5-40b9-b0c" \   --role="roles/iam.workloadIdentityUser" \   --member="principalSet://iam.googleapis.com/projects/1092851139093/locations/global/workloadIdentityPools/github/attribute.repository/kepungnzai/testrepo" And once you run this command, you will get something like this :- This is the github actions yaml that we can use the run and test our workload identity federation.  # This is a basic workflow...
Post a Comment
Read more

gcp creating organization folder and assured workload

Image
To createa an organization folder to manage your project and underlying resources, goto https://console.cloud.google.com/cloud-resource-manager.  Assure workload is used to create project for ensuring data residency, access transparency and other localized support. Make sure you have the required permission to create folder in the selected organization which is "Folder creator" assigned to an organization level (not project level) Then select Create folder. And then goto create a folder. If you have the required permission then you can create it. And then from here, you can move your project into the folders
Post a Comment
Read more

gcp orgnaization policy - gcp.resourceLocations

Image
in:us-locationsIn GCP we can use organization policy to restrict user to a specific location for example, we only want to create storage in us region only. Goto IAM & Admin -> Organization Policy -> search for gcp.resourceLocations. The click on edit where you can override the setting here. Click on "override parent's policy" And then add a custom rule here by clicking on Policy Type - allow and then the input text - 'in:us-locations'. Here we are restricting resources only to us region. And click on "set policy":- And if you try to create a bucket, you will get this :-  
Post a Comment
Read more

In a crazy world, our health, mental health and positive thinking are our biggest assets

Post a Comment
Read more

integrating cline and nvidia

Image
We can easily use cline with nvidia which provide free ll model to you. How to do the integration? Lets gets stareted by clicking on cline extension and then goto Settings -> API configuration where we will configure our Provider.  Make sure you select Nvidia and then provide the base URL:- 'https://integrate.api.nvidia.com/v1'  And for the OpenAI Compatible API Key, please generate that from Nvidia portal Then you need to select a valid model - meaning there can be many model but if that model is not hosted in Nvidia, then you will get an error. In my case I am using nvidia/nemotron-3.5-lightning-30b-a3b This is much faster than the one provided by cline.  That's it.
Post a Comment
Read more

installing rea into your laptop and performing analysis

Image
To install the infamour REA (reverse engineer anything) we require node 24x. If you do not have that, then you will not get the prompt below and the installation will just quitely go ahead. I think many developer would be caught off guard thinking that it went well. When you have all the pre-req, you will get a prompt like this :- And then you will asked a few questions and choose your coding agent.  Next, we will perform our first analysis by copying and pasting the following to my opencode. Download https://rea.tools/examples/notes-example.zip into a new example folder and unpack it. Use REA to find what happens when I click Export notes. Show where notes.csv is written, which fields it contains, and the relevant files and lines. And the final reesults here:- 
Post a Comment
Read more

otel tail sampling intro

Otel tail sampling is a really powerful feature that allow us to configure what tracses, log and metrics that we're going to be pushing to Application insights or other monitoring tools.  I had a go with filtering where I tried to send traces only for transactions greater than 2 secs. Soon I realized that is quite limiting and not usable because we need those for performance analysis. All we ever see is slowness in our system.   Tail sampling is a more customizable feature that allow us to configure stuff like  1. send error traces always 2. send request that has high latency times  3. high value transactions to ensure we are able to trace it  4. we can also impose sampling percentage to send over like 10% or 50%  These are the configuration that we keep on the otel collector. processors :   tail_sampling :     decision_wait : 10s           # Time to wait for all spans of a trace to arrive     num_trac...
Post a Comment
Read more

google adk what will happened when we did not set the output key correctly in the template

Image
 What will happen when we set output wrongly?   pipeline_runner = LlmAgent (     name = "pipeline_runner" ,     model = MODEL ,     description = (         "Runs and coordinates the delivery pipeline from dev to test to "         "pre-prod to prod, enforcing promotion gates between stages."     ),     instruction = RUNNER_INSTRUCTION ,     tools = [ run_pipeline_stage , get_stage_status , get_stage_logs ],     output_key = "pipeline_run_report1111" , ) Generally this is what you will get:-  raise KeyError(f'Context variable not found: `{var_name}`.') KeyError: 'Context variable not found: `pipeline_run_report`.' And it looksl like this:
Post a Comment
Read more

google ADK agent output_schema tend to stop tool calling

Image
If we specify output_schema here for our agent and provide tools for it to call - you will notice that tool calling stops and other times it return default python model (using default values) :- pipeline_runner = LlmAgent (     name = "pipeline_runner" ,     model = MODEL ,     description = (         "Runs and coordinates the delivery pipeline from dev to test to "         "pre-prod to prod, enforcing promotion gates between stages."     ),     instruction = RUNNER_INSTRUCTION ,     output_schema = PipelineRunReport ,     tools = [ run_pipeline_stage , get_stage_status , get_stage_logs ],     output_key = "pipeline_run_report" , ) And this is our agentic setup:-  pipeline_coordinator = SequentialAgent (     name = "pipeline_coordinator" ,     description = (         "Coordinates a pipeline run: execute stages, log what ...
Post a Comment
Read more

kubeflow - fine tuning gemini with text-based dataset

Image
To fine tune gemini using kubeflow pipeline, we are using the following setup. There is no validation being configure here. You can easily added it tho. This will be a much more practical example of using kubeflow for :-  from kfp import compiler, dsl   # ---------------------------- edit these ---------------------------- PROJECT_ID = "project-id" LOCATION = "us-central1" BUCKET = "my-vertex-bucket-ai"   # the bucket where you uploaded train.jsonl   TRAIN_URI = f "gs:// {BUCKET} /text/train.jsonl" VALIDATION_URI = f "gs:// {BUCKET} /text/validation.jsonl"   # set to "" to skip validation   # Must be a Gemini model that supports supervised tuning. Check Google's # supported-models list, because it changes over time. SOURCE_MODEL = "gemini-2.5-flash" TUNED_MODEL_NAME = "ticket-classifier"   PIPELINE_ROOT = f "gs:// {BUCKET} /pipeline-root"   # where pipeline files are stored # ------------...
Post a Comment
Read more

kubeflow condition and parallel tasks

Image
We are going to look at a simple kubeflow condition and parallel task when working with pipeline, This pipeline will greet pet owner and if there's pets we greet them too.  This is our example components:- @component def sayhi ( name : str ) -> int :   return 1 if name == "eric" else 2 @component def args_generator_op () -> list :     return [{ "cats" : "1" , "dogs" : "2" }, { "cats" : "10" , "dogs" : "20" }] And this is our pipeline code:- import json from kfp import dsl @dsl.pipeline (name= "control" , pipeline_root=PIPELINE_ROOT) def pipeline ():     name = sayhi(name= "greet-human" )       with dsl.If(name.output == 1 , "if has pets" ):           cat_and_dog = args_generator_op()           with dsl.Else( "if has pets" ):         cat_and_dog = args_generator_op()       with dsl.ParallelFor(cat_and_dog.output, name= "hi-to-pe...
Post a Comment
Read more

kubeflow pipeline conditional and parallel task

Image
We can easily run ML pipeline using aiplatfor sdk using the following setup - notebook  PROJECT_ID = "project-your-id"   # @param {type:"string"} LOCATION = "us-central1"   # @param {type:"string"} BUCKET_URI = f "gs://your-bucket-name- {PROJECT_ID} -unique"   # @param {type:"string"} ! gcloud storage buckets create --location={LOCATION} --project={PROJECT_ID} {BUCKET_URI} SERVICE_ACCOUNT = "ml-pipeline-dev-sa"   # @param {type:"string"} import sys IS_COLAB = "google.colab" in sys.modules if (     SERVICE_ACCOUNT == ""     or SERVICE_ACCOUNT is None     or SERVICE_ACCOUNT == "ml-pipeline-dev-sa" ):     # Get your service account from gcloud     if not IS_COLAB:         shell_output = ! gcloud auth list 2 >/dev/null         SERVICE_ACCOUNT = shell_output[ 2 ].replace( "*" , "" ).strip()     if IS_COLAB:         shell_output = !...
Post a Comment
Read more

creating agentic pipeline image training

Image
We can create gcp agent platform pipeline using kubeflow or tensorflow SDK (this is for large dataset which span terrabytes).  Kubeflow sdk are basically python script that you defines pipelines or components. @kfp.dsl.pipeline and @kfp.dsl.componet. Lets gets started by installing the required modules  uv install kfp.dsl  google-cloud-pipeline-components  docker In its simplistic form,  a pipeline can look like this (this will not work for Windows setup with docker) local_pipeline.py from kfp import dsl # Define a component to add two numbers. @ dsl . component def add ( a : int , b : int ) -> int :     return a + b # Define a simple pipeline using the component. @ dsl . pipeline def addition_pipeline ( x : int , y : int , z : int ) -> int :     task1 = add ( a = x , b = y )     task2 = add ( a = task1 . output , b = z )     return task2 . output And here is our runner  from kfp import local fro...
Post a Comment
Read more