mitzen

The $@ symbol are basically expanding or executing arguments passed from a client/entry point to the executing code.  Let's say for example we have a file call ./hello2.sh #!/bin/bash echo "Hello, World! - Argument: $@ " if we run hello2.sh test1 test2 test3 We will get the following output From a docker perspective, we can make a highly customizable image. For example, let's say we have this entrypoint.sh with the following contents entrypoint.sh #!/bin/bash ldconfig 2> /dev/null || echo 'unable to refresh ld cache, not a big deal in most cases' source /usr/src/.venv/bin/activate exec text-generation-launcher $@ And in my dockerfile,  Dockerfile ENTRYPOINT [ "/entrypoint.sh" ] CMD [ "--model-id" , "tiiuae/falcon-40b-instruct"] When my container runs, essentially what i am executing will be as follows. /entrypoint.sh --model-id tiiuae/falcon-40b-instruct which in turn calls  ./text-generation-launcher  --model-id tiiu...

To deploy this, it requires GPU. I was trying to deploy without one and wasn't successfully mainly because the images  huggingface-text-generation-inference does not have a cpu compatible image.  This is the image that is required image : us-docker.pkg.dev/ deeplearning-platform-release/gcr.io/huggingface-text- generation-inference-cu124.2-3.ubuntu2204.py311 First you need to create a secret and ensure you have accepted mistral agreement. If not head over to hugging face and accept the license agreement. Then create your secret kubectl create secret generic l4-demo \   --from-literal=HUGGING_FACE_TOKEN=hf_your-key-here Then deploy gradio and your model to k8s cluster. You can find additional detail here.  The gradio deployment looks like this  apiVersion : apps/v1 kind : Deployment metadata :   name : gradio   labels :     app : gradio spec :   strategy :     type : Recreate   replicas : 1   selector :     m...

To setup ssr on a new project, simply run the following command  ng new --ssr and to setup existing project  ng add @angular/ssr Let's check src/app/app.config.ts file where we have provideClientHydration added.  export const appConfig : ApplicationConfig = {   providers : [ provideZoneChangeDetection ({ eventCoalescing : true }), provideRouter ( routes ), provideClientHydration ( withEventReplay ())] }; To start running your server, run the following command and you will get the outputs shown here ng build  Noticed that server..mjs is generated and if you look at your package.json, it uses node + express to run your server app. Go ahead and run the following command:-  npm run serve:ssr:myssr Then go to your chrome browser and then locate localhost, you will see the response. Notice that the html,css and images is return by the web server.  Looking at server.ts file, we noticed that if route request for static file goes to browser dist fol...

Trying to setup ray serve to run in the gke cluster. Noticed that the sample uses : rayproject/ray-ml:2.9.0 which uses gpu. A non gpu version would be rayproject/ray-ml:2.9.0-cpu I can see from here  that it is not using gpu. The 2.9.0 version does require this - as shown below