Example of Python reading a csv file to remove a column and then write it to a new file

Two ways were used to solve the problem, both of which were solutions available online.

Scene Notes:

There is a data file, saved as text, now there are three columns user_id,plan_id,mobile_id. the goal is to get the new file with only mobile_id,plan_id.

prescription

Program I:Use python's open-file-write-file approach to jerk the data straight through, process the data within a for loop and write to a new file.

The code is as follows:

def readwrite1( input_file,output_file):
 f = open(input_file, 'r')
 out = open(output_file,'w')
 print (f)
 for line in ():
 a = (",")
 x=a[0] + "," + a[1]+"\n"
 (x)
 ()
 ()

Program II:Use pandas to read data to DataFrame and then do data splitting, and use DataFrame's write function to write directly to a new file.

The code is as follows:

def readwrite2(input_file,output_file): date_1=pd.read_csv(input_file,header=0,sep=',') date_1[['mobile', 'plan_id']].to_csv(output_file, sep=',', header=True,index=False) 

Code-wise, the pandas logic is clearer.

Here's a look at the efficiency of the implementation

def getRunTimes( fun ,input_file,output_file):
 begin_time=int(round(() * 1000))
 fun(input_file,output_file)
 end_time=int(round(() * 1000))
 print("Read and write runtime:",(end_time-begin_time),"ms")

getRunTimes(readwrite1,input_file,output_file) #Jerking data straight up
getRunTimes(readwrite2,input_file,output_file1) #utilizationdataframeRead and write data

Read/write runtime: 976 ms

Read/write runtime: 777 ms

input_file There are about 270,000 data, the efficiency of dataframe is still a little faster than the efficiency of for loop, if the data volume is bigger, is the effect more obvious?

Try increasing the number of input_file records under the next interview, with the following results

From the above test results, the efficiency of dataframe is improved by about 30%.

Above this Python read csv file to remove a column and then write a new file example is all that I have shared with you, I hope to be able to give you a reference, and I hope you will support me more.