通过Pandas读取大文件的实例

当数据文件过大时,由于计算机内存有限,需要对大文件进行分块读取: import pandas as pd f = open('E:/学习相关/Python/数据样例/用户侧数据/test数据.csv') reader = pd.read_csv(f, sep=',', iterator=True) loop = True chunkSize = 100000 chunks = [] while loop: try: chunk = reader.get_chunk(chunkSize) chunks.append(chunk) except StopIteration: loop