pickle常见Error解决

1. pickle OverflowError: cannot serialize a bytes object larger than 4 GiB

进行pickle.dump时出现上述错误，可以加上“protocol=4”参数。依据：https://docs.python.org/3/library/pickle.html#data-stream-format

2. pickle EOFError: Ran out of input

进行pickle.load时出现上述错误，一种可能是文件没有被正常打开和关闭，如执行下面代码，在“query_ids = pkl.load(handle_id)”处会出现此错误：

........
score_file = open("clipv3_query.pkl", "wb")
pkl.dump(query_reps, score_file, protocol=4)with open("clipv3_query.pkl", "rb") as handle_id:try:query_ids = pkl.load(handle_id)print(query_ids.shape)print(query_ids[:5])except Exception as e:print(e)

改成如下代码后，运行正常：

with open("clipv3_query.pkl", "wb") as score_file:pkl.dump(query_reps, score_file, protocol=4)with open("clipv3_query.pkl", "rb") as handle_id:try:query_ids = pkl.load(handle_id)print(query_ids.shape)print(query_ids[:5])except Exception as e:print(e)