Posts

FluentD Logging with .Net Core

You can install the NLog plugin to logging to a Fluentd node.  Fluentd is a unified logging layer and if you're wondering if we're talking about the same logger, check it out here. There is a difference between fluentd and fluentbit. Fluentd is targeted for servers with larger processing capacity while fluentbt is for IOT devices with small memory footprint.


Getting your project ready with the following nuget

Install-Package NLog.Targets.Fluentd -Version 1.0.4

Install-Package NLog -Version 2.1.0

Install-Package System.Configuration.ConfigurationManager -Version 4.4.1

Setting up Fluentd Docker


This is the docker command to setup Fluentd as an endpoint to gather all your log.


docker run -d -p 24224:24224 -p 24224:24224/udp -v /data:/fluentd/log fluent/fluentd


This will start your docker container with fluentd.


Then we will use the following code to write to docker container :-












Swagger UI - Getting better documentation and playing around with your REST service

We so used to building REST or micro service. Now it makes sense to build better REST based services.

Better documentation with Swashbuckle.AspNetCore

It is always good to have SwaggerUI in your system as it gives you auto-generated, nicely layout way of documenting and interacting with your REST service.


Install-Package Swashbuckle.AspNetCore


Next, configure your startup.cs










Remember to browse to your swagger UI by :- http://localhost:5050/swagger/




CNTK : Working with MNIST dataset using logistic regression

Here is the code to download MNIST dataset.

Just run python download_mnist.py





Next you would like to try out logistic regression with this dataset.


Kubernetes - deploying to a cluster (real cluster)

Image
Surprisingly deploying to a cluster is pretty straight forward.

To run or deploy your application to cluster, issue the following command :-



kubectl run --kubeconfig connect_config.txt connectcd --image=kepung/dotnetapp-prod --port=5050



The only different is --kubeconfig - which is a file storing your cluster info including access.

Next, to ensure, we expose our port with a service, using command below :-


kubectl --kubeconfig connect_config.txt expose deployment/connectcd --type="LoadBalancer" --port=5050 --name=connectcd-http

That's pretty much it.

To get the service url for your pod, you can use the following command :-

kubectl --kubeconfig connect_config.txt get service connectcd-http -o yaml


You will see the following output and it contains the DNS of the newly deployed service. Please wait for a few seconds for AWS DNS to have it registered in there.








Working with Microsoft bot adaptive card

When working with Microsoft Bot, you there are a few gotchas when working with adaptive cards.

Nuget package

You need to install "Microsoft.AdaptiveCards" nuget package


SubmitAction 

If you have data in a property called  "DataJson", it needs to be in Json format, otherwise your layout will not be rendered.





Formally written C# new features

https://docs.microsoft.com/en-us/dotnet/csharp/whats-new/csharp-7#discards

CNTK - working with linear regression

The code sample below gives











creating fast forward network using CNTK

Here is a good example of how you can create and evolve your simple fast forward network for a simple classification problem





sigmoid vs softmax

If outcome is in a binary form, e.g true or false, or 1 or 0 then use sigmoid

Otherwise use softmax where outcome can be more than 2 possible states.



CNTK - A ML framework from Microsoft for large scale processing

CNTK is suppose to be a large scale corporate development of ML. You can call it via C#.
Similar to tensorflow, it is designed as a graph based ML development framework.


A couple of interesting things i find out

Installation CNTK 

To install CNTK on Windows. If you're installing for different platform, you can refer to documentation here.

pip install https://cntk.ai/PythonWheel/CPU-Only/cntk-2.4-cp35-cp35m-win_amd64.whl

Get started 

The best place to get started is following instruction here.


Using CNTK with C#

Yeap, you can use CNTK with C#.


Light RNN

https://github.com/Microsoft/CNTK/tree/master/Examples/Text/LightRNN


Tutorials

https://cntk.ai/pythondocs/CNTK_200_GuidedTour.html?highlight=cross_entropy_with_softmax


Docker image 

You can find the docker image here.

docker pull microsoft/cntk
















Microsoft REST guidelines

Microsoft's guidelines for building REST based services can be found here.

primeng datatable refresh

When you change individual rows in primeng datatable, datatable will not be refreshed automatically,  for example

this.rows[i]['gender'] = 'Female' In order to refresh table, you need to do a manual, shallow copy using code below :-

this.rows[i]['gender'] = 'Female'this.rows = [...this.rows]; // refresh manually here ..

Getting javascript object property and values

Here are some common functions to discover properties and values of an object using properties.

a) Object.keys - Get all the keys of an object

b) Object.values

c) Object.getOwnPropertyDescriptors



The following code snippet demonstrate how to work with these methods.






angular data bindings - further look

There are 3 different types of bindings in Angular

a) Source to View bindings

The following syntax are valid 

{{expression}}

[target]="expression"

bind-target="expression"


b) View to source binginds

The following syntax are valid 

(target)="statement"

on-target="statement"


c) Two way bindings

The following syntax are valid 

[(target)]="expression"


bindon-target="expression"




Pushing docker image to ECS (Elastic Container Service)

In order to push to Amazon's ECS registry you need to setup and configure your account.

First of all you need to setup aws and docker.

Goto AWS console to create a repository and follow the instructions. If you needed a sample project, please clone one here.

To summarize, the steps, do the following. (i assume you created a repository called "jeremy".

Get a docker login command (setup your credential) by running the following command :-



aws ecr get-login --no-include-email --region ap-southeast-2
Then build your image



docker build -t dotnetapp-prod .



docker tag dotnetapp-prod 312611170176.dkr.ecr.ap-southeast-2.amazonaws.com/jeremy:dotnetapp-prod

And push your image to Amazon


docker push 312611170176.dkr.ecr.ap-southeast-2.amazonaws.com/jeremy:dotnetapp-prod


To browse your ECS repository, please go here.


Possible error message :-


Can't push image to Amazon ECR - fails with “no basic auth credentials” - This means you didn't setup your aws credential - please set or …

kubernetes pods keeps restarting

I had this situation whereby kubernetes pods keep on restarting even if I tried deleting it.
As soon as it gets deleted, two new instance get created again.

So i tried looking for what causes it. For my case, it is the replicationcontroller.

kubectl get replicationcontroller

kubectl delete replicationcontroller "connectcd"


And this solves it! :)


Setting up your simple .net core application using kubernetes

Image
This is a simple guide how to deploy .net core docker image to kubernetes (minikube).
You need to setup your minikube, docker. Once you have done that, we're ready to go.

Summary of tasks :-

a) setup Docker image

b) push to Docker registry

c) run your application on Kubernetes and scale it


1. Setup Docker image

Our .netcore service exposes /hello/index url which returns "Hello World!".

The complete source code is available here. Once you have clone the git repo, we just need to build our docker image. This is a multistage docker image which means it uses .netcore sdk for building and then deploy built binaries to a .netcore production image.


Go to ConnectCD-NetCoreWebApi, then run the following command :-



docker build -t dotnetapp-prod .


If you want to test your application locally (using DOCKER),


docker run -d -p 5050:5050 dotnetapp-prod





2. Push to Docker registry

Use the following command to push your built image to Docker registry (kepung) is my user id. You should chan…

Pushing your own image to docker registry (over the cloud)

I am doing this because well, i need to use kubernetes to run my docker image. It's a bitch to setup a local docker image, so this is easier.

First you need to create a free user account in docker.

Make sure you setup docker  for the command prompt, using the following command


minikube docker-env

This will give you a list of command to run to activate your docker.


Login to docker and provide your username and password

docker login
Next tag your image (kepung/my_image) where kepung is the name of my user.

docker tag my_image kepung/my_image
Push it

$ docker push kepung/my_image