integrating cline and nvidia

We can easily use cline with nvidia which provide free ll model to you. How to do the integration? Lets gets stareted by clicking on cline extension and then goto Settings -> API configuration where we will configure our Provider. 

Make sure you select Nvidia and then provide the base URL:- 'https://integrate.api.nvidia.com/v1' 

And for the OpenAI Compatible API Key, please generate that from Nvidia portal

Then you need to select a valid model - meaning there can be many model but if that model is not hosted in Nvidia, then you will get an error. In my case I am using nvidia/nemotron-3.5-lightning-30b-a3b

This is much faster than the one provided by cline. 




That's it.


Comments

Post a Comment

Popular posts from this blog

NodeJS: Error: spawn EINVAL in window for node version 20.20 and 18.20

Encounter this issue when running autorest (that uses nodejs).  I am using node version:  v18.20.2 After i revert to use node v18.12.0, then i was able to get my app to run without this error. Since i am working with Azure devops yaml, i was able to use the following to resolve my issue.  - task : NodeTool@0   inputs :     versionSource : 'spec'     versionSpec : '18.18.0'
Read more

llama cpp running it in google colab

Image
There are 2 options to do this in Google  colab.  Option 1 (Easiest)  With this option, we can just install the relevant package and then run the model.  First we need to install the required packages using the following command:-  ! pip install -U llama-cpp-python Next,  we will try to get the model using the following command # !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained(   repo_id= "unsloth/Qwen3-0.6B-GGUF" ,   filename= "Qwen3-0.6B-IQ4_NL.gguf" , ) We can see the model being downloaded:- And then we will run the following command to test this model llm.create_chat_completion(   messages = [     {       "role" : "user" ,       "content" : "What is the capital of France?"     }   ] ) And the output will look something like this:-  Option 2 llama.cpp is a powerful inference engine and if you wanted to get it running in google colab, you co...
Read more