SoFunction
Updated on 2024-10-30

Importing and exporting large files in python.

1, csv file import and export

Export to a csv file via a matrix, import a csv file as a matrix

Importing a csv file into a matrix

import numpy 
my_matrix = (open("c:\\","rb"),delimiter=",",skiprows=0) 

Exporting the matrix to a local csv

('', my_matrix, delimiter = ',') 

Unfinished business.

It is also possible to use the pickle module, where the saved files are serialized

python's pickle module implements basic data serialization and deserialization. With the serialization operation of the pickle module we are able to save the information of the objects running in the program to a file for permanent storage; with the deserialization operation of the pickle module we are able to create the objects saved by the last program from the file.

(obj, file, [,protocol]) 

Note: Save the object obj to the file file.

protocol is the version of the protocol used for serialization. 0: ASCII protocol, the serialized object is expressed in printable ASCII code; 1: the old binary protocol; 2: the new binary protocol introduced in version 2.3, which is more efficient than the previous one. Protocols 0 and 1 are compatible with older versions of python. protocol defaults to 0.

file: the class file object that the object is saved to. file must have the write() interface. file can be a file opened in 'w' mode or a StringIO object or any other object that implements the write() interface. If protocol>=1, the file object needs to be opened in binary mode.

(file) 

Note: Reads a string from file and reconstructs it into the original python object.

file: file-like object with read() and readline() interfaces.

Save data

tmpdatapath = "E:\\data\\u_i_matrix.csv" 
savefp = open(tmpdatapath,"w")
(u_i_mat,savefp)
();

Import data

fp_mat = open("E:\\data\\tmpdata\\u_i_matrix.csv","r")
rMat = (fp_mat)
fp_mat.close()

Append: input the contents to the file via print

str=”a string to print to file” 
f=open(‘','w') 
print >>f,str 
()

Above this on python in the large file import and export method details is all I share with you, I hope to be able to give you a reference, and I hope you support me more.