mitzen

Introduction There are so many options available for log aggregation and monitoring your system. In this section we are going to look at tech stack called telegraf, influxdb and grafana. Telegraf is a log / metric collection and aggregation agent. It is the component which sits on target machine (your target host where your mission critical applications sits) and continuously send metrics data to your other component called influxdb. InfluxDb is a time-series database that telegraf writes metrics to and typically where all your data is stored. It is like any other database, where you can use a client (influxb) to connect to it and start writing queries and pull out the information relevant to you. Grafana is a front end web applications written using react that shows all your metric information. All you need to do is select your database, write up some query and pick your widget. Setting up on a Windows environment.  Now that we are acquinted with the tool. Lets...

By default, telegraf comes as a tar/zip file which are not installed into a specific folder. As part the linux way of doing things, packages are placed in /user/local folder. I needed to configure telegraf as my daemon. So i used the following conf and place it under /etc/systemd/system [Unit] Description=telegraf.servicee [Service] ExecStart=/usr/local/bin/telegraf --config /usr/local/etc/telegraf/telegraf.conf [Install] WantedBy=multi-user.target: To reload and re-run all the daemon.  sudo systemctl daemon-reload To start a single service  systemctl start telegraf.service To stop the status  systemctl status telegraf.service To get the status  systemctl status telegraf.service

Try to update a file called "alerting.jpg" to a blob container. Then get the full path of the blob file. For example it could be :- https://mystorage.blob.core.windows.net/batch/alerting.jpg then append your sas key to the this uri. https://devsapmtgdsdataextracts.blob.core.windows.net/batch/alerting.jpg?sv=2017-07-29&ss=b&srt=sco&sp=rl&se=2099-12-31&st=2018-03-21&spr=https,http&sig= someweirdkey And once you combined it, just paste it into your browser. You should be able to access  and load the file if your SAS key is legit.

Create a non https mvc dotnet core project. dotnet new mvc --no-https Sample csproj configuration. Dotnet core will throw you missing Configuration Error, if you include some packages that are not suppose to be in. If you're doing it from nuget package manager, please ensure you checked "Include Pre-release" dotnet add package Microsoft.Azure.AppConfiguration.AspNetCore --version 3.0.0 -preview- 010560002 - 1165 You need to create a ConnectionString:Appconfig in your app settings. In actual deployment it looks like this The value should be something like this (just copy connection string value)  :- Endpoint=https://your-awesome-appconfig.azconfig.io;Id=fake_id;Secret=fakesecret Let's wire up your view to actually pull data from Application Configuration You need to grab some configuration data from the store and show it like this Our application configuration's Configuration Explorer contains the followi...

To do that, use the following command :- dotnet new angular -o belair_app --no-https

To fix this :- If it is in .net 3.0, remove all these "Microsoft.AspNetCore.Mvc" Version = "2.2.0" "Microsoft.AspNetCore.Mvc.Razor.ViewCompilation" Version = "2.2.0" And make it look like this below :-

Quick and easy way to set you up :) Run the following from the command line :- dotnet new angular -o my-new-app Setting up server side export ASPNETCORE_Environment=Development dotnet build dotnet run Setting up client side npm install --save ng serve Direct docs is availabe from here  

Let's say you created Azure Key Vault and created a KEY (not secret or certificate) that is made up of RSA Key. The following codes can be used to encrpyt and decrpyt value.

Sometimes you might want to create expose your public key / generate your public key to 3rd party apps. This is a code written entirely on Powershell to help you export public key (modulus and exponent) to a legit PEM file: To use it, $pemPublicKey  = GeneratePublicKey  $your_modulus_base64encoded   $your_exponent Write-Host ( $pemPublicKey )

Publicc key are typically stored in a PEM file which contains modulus and exponent value. These get converted into RSAParameters so that it can be used by RSA to do its magic. It looks something lke this :- General Idea  var rsa = new RSACryptoServiceProvider(); rsa.ImportParameters(p); To use a nuget package called PemUtil.  This allows you to read a PEM file into a RSAParameters. Go and install a nuget package called PemUtil and then you can use the following code. Then you can plug it into code above.  public static RSAParameters ReadPEMFileB(string PemFilePath)         {             RSAParameters rsaParameters;             using (var stream = File.OpenRead(PemFilePath))             using (var reader = new PemReader(stream))             {                 rsaPar...

What is this? A PEM file which contains your public key.  Public key will contain your Modulus and Exponent. (P and Q).  It is different from private key which contains alot more information. -----BEGIN PUBLIC KEY----- MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCxSxqCESCdpeXWCprQGMrpwlEg 91EF4Qt1R247gTGaUMgEjFEq9c/UffmtYyccmoIq3n8m/ZcsiCNJNFfGx5h/YEdR ZyzQjSLg9zt3zu5JbBNx4zb63vvhpCGoMf65y/RvMs+XjSBl8ybl0vbbrC62YH1I 7ZJYSbKhXr3ILHyUSwIDAQAB -----END PUBLIC KEY----- OpenSSL To generate a rsa key openssl genrsa -out private.pem 1024 To view your private key openssl pkey -in private.pem -text To generate a public key (your private key already contain your public key) openssl pkey -in private.pem -out public.pem -pubout View our public key (noticed the additional -pubin argument) openssl pkey -in public.pem -pubin -text

If you encounter this issue, this is because the latest code base is not package to pypi for some reason. So i use a rather hacky way to resolve this. If you're running on your computer Clone  https://github.com/huggingface/transformers  and locate your installed python package for "transformer". It should be somewhere usr/lib/python/dist-packages. Replace all code in the dist-package transfomer folder with your clone folder. Yes, replace all as there are alot of dependencies. :( Using colab. This is almost the same, clone the repository and start upload to the destination folder. Loads of work and hope you have fun. To be honest, i didn't get it to do those hard core training. It just stall after a few runs.

Perhaps the best way to get started is to look at this Colab notebook. Just run it and you will see loads of fake content being generated. https://colab.research.google.com/drive/1OqF99q0-G1RCfmvcEw_qZvjuiJL_066Q#scrollTo=TsNXf-0zgaHn

No really sure why we need to specify a type for creating / initializing logger (as indicated by code below)

Sometimes when things are not so clear, and it suddenly turns mysterious when another person say else. All you ever get is a huge mess. This is one of those cases Operation Tab In this tab, you get to see "What and how long an operations are running on my server" Operation times give you how long a spike last. What is the peak operation time happened on the server. Just underneath the "Operation time", you can sub-divide by Request, CPU, Available Memory, Storage. The story it is telling is, during those time when the spike occurs, how is my resources holding up? Does memory got sucked out and is the CPU reaching certain limit (since it is showing a number like 0, 1,2) i am thinking it means, average limit between available CPU. The distribution chart, shows where percentage of operations lies. In my case, 50% of the time, there are over 100 request processed within 64 miliseconds, 90% of the time, under 10 request served within 6.8 seconds windo...

This is the easiest Q and A example that you can find. Workflow goes something like this, 1. tokenizer to convert into torch tensor. You need to encode beginning with [CLS] and [SEP]. Statement or knowledge is encode with 1. Question is 0. 2. create BERT QA model and run.  3. get the score out. Interpreting the score is key. You get start and end index of a answer. Represented by start_scores and end_scores below. It is like using a highlighter and crossing out answer to our face. ## initialize the model tokenizer = BertTokenizer.from_pretrained( 'bert-base-uncased' ) model = BertForQuestionAnswering.from_pretrained( 'bert-large-uncased-whole-word-masking-finetuned-squad' ) ### Please note here that encoding are given as   #                 0 0 0 0 0 0 0 (den...

Here is the code for startup function for doing logger injection And then in your function app constructor, this is where you start to get instances of your logger.

Azure function app v2 uses standarized approach for getting  app settings / app configuration information. This is what your start up function looks like : Metric settings is place to hold your settings info: And finally, wiring it up to your function app This is what your configuration looks like :-

I faced this issues after i upgrade Microsoft.Logging.Extensions. Jezz man, just trying to add more visibility. Remove this and you're set to go but if you really need to do logging and stuff, probably add reference to the correct nuget package. For my case it is "Microsoft.ApplicationInsights.AspNetCore" Version="2.8.2" 

Check out those models that are making impact https://gluebenchmark.com/leaderboard/submission/zlssuBTm5XRs0aSKbFYGVIVdvbj1/-LhijX9VVmvJcvzKymxy