Python using stream io

You should make use of pytohn io as defined here to work with stream data. So there are basically 3 main types of stream io which are text io, binary io,

Instead of reading everything into memory, you are able to read this line by line as a stream....

import subprocess

## assuming r.csv is a large file

self.streamedOutput = subprocess.Popen(['cat', 'r.csv'], stdout = subprocess.PIPE)

self.sendOutput(self.streamedOutput)

while True:

executionResult = streamedOutput.stdout.readline()

if executionResult:

print(executionResult)

else:

break