Posts

Showing posts from May, 2019

Getting spark to read kafka data

Setup kafka

bin/zookeeper-server-start.sh config/zookeeper.properties


bin/kafka-server-start.sh config/server

Running pyspark code / script from cli

In this example, we are running a spark python file called test.py.

 spark-submit test.py /home/jeremy/test/test.dat




pyspark start with a package from command line

pyspark --packages com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.10

Cloud resource provisioning

Static cloud based resource provisioning is bad. Tools such as terraform needs a dynamic scripting language. There are so many resources created with any kinda of FQDN. And there is not alot of choices to choose from and conflict often occurs.

We need a better cloud provision tool that resolve conflict better or provides a scripting interface.

Azure function to access key vault

You can easily configure Azure function to pull secret value from KeyVault. First you need to setup :-

1. Azure function function -> Platform feature -> Identity -> Go ahead and turn on the identity. This is what the called Managed Identity setup as you will get an App Id (guid). Save it and then setup your vault below :-


2. In your Key Vault, Access policy -> Add New -> Select Principal -> Add the name of your Azure function app Id


3. Restart your azure function app.


4. In your settings file, you can have some an configuration entry say, CONN_WEB_APP with the following value,

@Microsoft.KeyVault(SecretUri=https://blahblahblah.vault.azure.net/secrets/Conn/e74dfd34e8e4415fbd09c540de2cf556)


You're all sorted! :)

Azure function tool publish : Can't determine Project to build. Expected 1 .csproj but found 0

If you encounter this issue, you are probably using older version of azure function tool.
To upgrade, use the following command :-


sudo npm i -g azure-functions-core-tools@core --unsafe-perm true


Then try to publish your app again :-
func azure functionapp publish ClaimSubmissionFunction

Vscode - when working with projects

To add new solution

dotnet new sln

To add existing project into the solution

dotnet sln AzureFunctionApp.sln add ClaimSubmission/ClaimSubmission.csproj


Angular : Adding store + reducer

Added reducer or store pretty much the same thing, to me. To setup the required libraries,

npm install @ngrx/store --save

First you need some sort of actions


import { Action } from'@ngrx/store';
exportenumActionTypes {
Login = 'Login',
Logout = 'Logout',
Reset = 'Reset', }
exportclassLoginimplementsAction { readonlytype = ActionTypes.Login; }
exportclassLogoutimplementsAction { readonlytype = ActionTypes.Logout; }
exportclassResetimplementsAction { readonlytype = ActionTypes.Reset; }
exporttypeActionsUnion = Login | Logout | Reset;

Then you setup your reducer :-

import { ActionReducer, ActionReducerMap, createFeatureSelector, createSelector, MetaReducer } from'@ngrx/store'; import { environment } from'../../environments/environment'; import { ActionsUnion, ActionTypes, Login } from'../actions/Login'; import { Action } from'@ngrx/store';
exportinterfaceState { type : string, actionValue : number; }
exportconstinitialState: State = { type:"em…

Angular : Adding effects to your project.

To install effects, use the following cli :-

npm install @ngrx/effects --save

To add effect, you need to setup your Store reducer first. After that everything is so much easier.

You only requires this piece of code to setup your effects :-



import { Injectable } from'@angular/core'; import { Actions, Effect, ofType } from'@ngrx/effects'; import { ActionTypes } from'./actions/Login'; import { map, mergeMap } from'rxjs/operators';
@Injectable() exportclassAppEffects {
constructor(privateactions$: Actions) {
}
@Effect() loginEffects$ = this.actions$.pipe(ofType(ActionTypes.Login), map(action=> { console.log("login effect taking place....") return ({ type:"tested" }); }));
}



And then configure it, in the app.module.ts

imports: [ BrowserModule,FormsModule,AppRoutingModule, StoreModule.forRoot({login :LoginReducer}), EffectsModule.forRoot([AppEffects]) ],

angular adding routing table

Create a file called "app-routing.module.ts" with the following content :-


import { NgModule } from'@angular/core'; import { Routes, RouterModule } from'@angular/router'; import { AppComponent } from'./app.component';
//This is my case constroutes: Routes = [ { path:'', component:AppComponent }, { path:'about', component:AppComponent } ];
@NgModule({ imports: [RouterModule.forRoot(routes)], exports: [RouterModule] }) exportclassAppRoutingModule { }

Next,  open app.module.ts

import { AppRoutingModule } from'./app-routing.module';

And add "AppRoutingModule"


@NgModule({ declarations: [ AppComponent, CarInsuranceComponent, HouseInsuranceComponent, HealthInsuranceComponent, ], imports: [ BrowserModule,FormsModule,AppRoutingModule, StoreModule.forRoot(reducers, { metaReducers }), EffectsModule.forRoot([AppEffects]) ], providers: [], bootstrap: [AppComponent,] }) exportclassAppModule { }

CSS grid as a landing page

<divid="container">
</div>




#container { display : grid; text-align : right; grid-template-columns: autoautoauto; height : 90vh; background: url("assets/background.png") no-repeat; background-position: bottomright; }

Post quantum encrytion last 17 algorithms

BIKEClassic McElieceCRYSTALS-KYBERFrodoKEMHQCLACLEDAcrypt (merger of LEDAkem/LEDApkc)NewHopeNTRU (merger of NTRUEncrypt/NTRU-HRSS-KEM)NTRU PrimeNTS-KEMROLLO (merger of LAKE/LOCKER/Ouroboros-R)Round5 (merger of Hila5/Round2)RQCSABERSIKEThree Bears The 9 Second Round Candidates for digital signatures are: CRYSTALS-DILITHIUMFALCONGeMSSLUOVMQDSSPicnicqTESLARainbowSPHINCS+