PySpark - Working with JDBC Sqlite database

Spark supports connectivity to a JDBC database. In this session, we going to see how you connect to a sqlite database.  As of this writting, i am using Spark 2.1.1.

First, have your spark-defaults.conf file setup. in your conf folder.

Next, fire up your pyspark, then run the following script in your REPL.

The return data is a list.  Then you probably want to access the data using :-

>>> a[0].ID
>>> a[0].Name
>>> a[0].Name

Looping through it :-

 for i in a: print(i.Name)

>> mark
>> eric

That's it.


Popular posts from this blog

OpenCover code coverage for .Net Core

ionic2 cordova build android - Unable resolve gradle 2.2.3

Using Custom DLL with IronPython / Scripts