mitzen

  Goto this link to see more about this topic https://learn.microsoft.com/en-us/azure/storage/container-storage/container-storage-aks-quickstart?tabs=portal

  Sample for the code can be found here. It has code to export resnet50 into an onnx output. Then reloading it back in. When it is reloaded, please note that the input to the model will no longer be pytorch tensor. Instead it will be array.  https://github.com/mitzen/pytorch_restnet_flask_app More reference  If you're curious how 3 x 224 x 244 is obtained, well that's the standard input size for resnet50. You can find references here  https://www.tensorflow.org/api_docs/python/tf/keras/applications/resnet50/ResNet50 And it is even documented in the paper  https://arxiv.org/pdf/1512.03385.pdf If you wanted to have a play with the code here, please refer to this Colab.  You will be able to see how to work with existing model - loaded by pytorch vs loaded by onnx. https://colab.research.google.com/drive/1o4MmuJBMKZEMWiFY42cz8anW9urKN295#scrollTo=L1BWXN2UDgD1

  The complete gist can be found here: https://colab.research.google.com/gist/mitzen/6cb572bf7eaa57794006c588473bccad/falcon-huggingface-examples.ipynb This example give you an example how we can test run falcon on Google colab. Mine blows up because I am using a free version. 

  You can find more info about pre-training hugging face model - this is specifically for BERT.  https://huggingface.co/docs/transformers/v4.15.0/training Hopefully the same approach is applicable to other model that I'm kinda interested in FALCON and LLAMA 

 If you're using hugging face and getting the error above, technically it means your runtime requires more cpu. You could try to get around it using the following example as a settings = basically settinglow_cpu_mem_usage = false tokenizer = LlamaTokenizer.from_pretrained(model_path) model = LlamaForCausalLM.from_pretrained(     model_path, torch_dtype=torch.float16, low_cpu_mem_usage = False ) Sometimes the code of the issue is because you specify device_map to auto. This will turn on accelerate library. Setting it to 0 means placing your module on to a gpu.    For example, I configure my device_map to auto which will throw more exception if I run it on Google colab using free runtime tokenizer = LlamaTokenizer.from_pretrained(model_path) model = LlamaForCausalLM.from_pretrained(     model_path, torch_dtype=torch.float16, device_map= 'auto' , low_cpu_mem_usage = False )

 

  When installing kubebuilder, using this command, it didn't actually work for me.  curl -L -o kubebuilder "https://go.kubebuilder.io/dl/latest/ $(go env GOOS) / $(go env GOARCH) " So I had to use the following command to download specific releases   curl -L -o kubebuilder https://github.com/kubernetes-sigs/kubebuilder/releases/download/v3.11.0/kubebuilder_linux_amd64

  While running the command aws iam create-policy --policy-name ApplicationTeamRolePolicy --policy-document /AppTeamRole_PolicyDocument.json I ran into this error. An error occurred (MalformedPolicyDocument) when calling the CreatePolicy operation: Syntax errors in policy. Then i had to resort to this. Notice the additional file://./   aws iam create-policy --policy-name ApplicationTeamRolePolicy --policy-document file://./AppTeamRole_PolicyDocument.json

  https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

  I managed to get a good tutorial from this link here: https://catalog.workshops.aws/cfn101/en-US/basics/templates/template-and-stack One of the easiest example, that i worked with is creating a s3 bucket. Create a yaml file, called it test.yaml. Resources :   S3Bucket :     Type : AWS::S3::Bucket     Properties :       BucketEncryption :         ServerSideEncryptionConfiguration :           - ServerSideEncryptionByDefault :               SSEAlgorithm : AES256       BucketName : testtesttestjerwoo       VersioningConfiguration :         Status : Enabled BucketName is the name that you would like to create. If you leave it blank, then a random name will be generated for you.   Then goto your AWS Cloud formation, create a stack, choose upload a file and upload this file. You should accept all the default and then click "Submit".  It will create all the required aws resources for you. 

When you're using appservice and function app, it is recommended to use nameresolver and tcpping when resolving connectivity issues. Tools like nslookup and curl is available but could be misleading and you can't tell the difference whether it worked or not.  https://learn.microsoft.com/en-us/troubleshoot/azure/app-service/troubleshoot-vnet-integration-apps

  The follow code demonstrate decorator in python. When executed, the closest function / decorator will be executed first and the the one above it.  As a result, you will get <c><b><a>hello world</a></b></c> def dec1 ( a : callable ):     def inner ():       x = a ()       return "<a>" + x + "</a>"     return inner def dec2 ( a : callable ):         def inner ():       x = a ()       return "<b>" + a () + "</b>"     return inner def dec3 ( a : callable ):         def inner ():       x = a ()       return "<c>" + a () + "</c>"     return inner @ dec3 @ dec2 @ dec1 def hello ():     return "hello world"

Download and setup aws cli. Once you got that, please issue the following command to get acr password.   aws ecr get-login-password --region ap-southeast-2  Then you can use the following command to login to your ECR region docker login --username AWS your-registry-southeast-2.amazonaws.com Then paste your password, you obtained from above. 

 ` Ran into this error trying to deploy my dotnet lambda.  The application was run as a self-contained app because '/var/task/myapp.runtimeconfig.json' did not specify a framework. To resolve it, i tried using the following target runtime build. dotnet publish -c Release --self-contained --runtime linux-x64

 You could find a registry of AWS lambda images that you can use here https://gallery.ecr.aws/lambda/dotnet And the dockerfile, you can find it from here: https://github.com/aws/aws-lambda-dotnet/tree/master/LambdaRuntimeDockerfiles/Images

  Was getting the error above and then had to run the following command to generate my dll.  dotnet publish -c Release --self-contained --runtime linux-x64 After uploading, i was able to resolve my issues.

 I am getting 403 trying to push code to AWS code commit repository. Probably the user does not have credential/password to push code to code commit. To fix this, goto IAM in AWS, then Generate Credential for AWS Code commit.   That's it - you should be able to pull and push code now.