Posts

Showing posts from January, 2025

llama stack - setting up using wdl ubuntu 24 and downloading model

Image
Python 3.12 is installed by default in ubuntu 24. You can try to install pip by running The following command " sudo apt install python3-pip " You would received an email by Meta. Goto https://github.com/meta-llama/llama-models And follow the readme instructions.  Next install llama stack " pip install llama-stack ".  Run pip show llama-stack and see where you have install the global executable file.  Ensure you have set your path the proper folder, in my case it is  export PATH=$PATH:/home/jeremy/.local/bin   Then run "llama model list" and you get the following outputs. Depending on the llama version that you've requested, the will dictates which model you can download. for example, i have requested llama 3.3 but when i tried to download llama 3.2 model, it doesn't allow me to do so. Llama 3.3 is really huge. So I think i should be requesting for llama 3.2 instead.   

python pip - This environment is externally managed

Image
 If you're getting the following error, when running  pip install llama-stack as shown in the diagram below:- Then you can proceed to add/append --break-system-packages to it and run "pip install llama-stack --break-system-packages" but this can be potentially harmful. So it might be better to create your own python environment instead python3 -m venv myllm source myllm /bin/activate  pip install llama-stack

windows python installed but not accessible via powershell

 Sometime ago, I installed python using Microsoft store. I can get access if i choose the command icon but not accessible from powershell. So I need to update my path accordingly.  So I added the followings to my path. Please not the \scripts is mostly for allowing access to pip  C:\Users\usertest\AppData\Local\Programs\Python\Python312 C:\Users\usertest\AppData\Local\Programs\Python\Python312\Scripts

gcp authenticating using client sdk - service account approach.

Create a service account in IAM and Admin. Once you have done that, go to your newly created service account and then click on Create Key Pair. Then it will automatically download the private key json Create your gcp storage bucket. The under permission tab, grant your service account access perhaps with a role "storage object creator". Let's switch to your laptop and fireup a console c# project. Add google bucket sdk libraries into the by running: dotnet add package Google.Cloud.Storage.V1 --version 4.10.0 Then configure the environment variables  GOOGLE_APPLICATION_ CREDENTIALS=path-to-your-json. key  for example "C:\\work\\gcp-cred\\gcp- myprivate-project- 14197e38ef68.json" Then use the following code to test out your configurations using Google.Cloud.Storage.V1; // upload var objectName = "readme.md"; var storage = StorageClient.Create(); using var fileStream = File.OpenRead("c:\\work\\ foctest\\README.md"); storage.UploadObject(" m...

gcp clound run function - configuring scaling and concurrency

Image
To change your cloud function cpu and scaling configurations, go to your cloud run function and select "Edit and Deploy new revision" as shown below:- Proceed to update the memory, cpu allocations.  Update the concurrency if required to take advantage of the higher cpus According to the docs, each instance can handle up to 80 concurrency request but you can change this to 1000. More details https://cloud.google.com/run/docs/about-concurrency