diff --git a/python/20231117_save_resultset_to_file.html b/python/20231117_save_resultset_to_file.html new file mode 100644 index 0000000..31aaf8e --- /dev/null +++ b/python/20231117_save_resultset_to_file.html @@ -0,0 +1,63 @@ + +

+Save ResultSet to a File +

+ +

+At work I had a huge query with many inner joins and also an inner query that +was taking forever and wouldn't always return, it would error out +instead. (Turns out the main problem was hitting a very active DB but when I +switched to a backup DB the query worked fine.) +

+ +

+Anyways, in order not to worry about making the query whenever I wanted to test +something I decided to save the Python resultset to a file. I wanted to save it +in such a way as that when I read it back in it was as if I was using the +resultset. +

+ +

+Pickle didn't seem to "easily" work, i.e. after a few tries I just gave up on +it. But after some Googling I found this solution which is just what I wanted. +

+ +

Execute query, fetch data into a resultset

+ +results = cursor.execute(some_query) +rows = results.fetchall() + + +

Convert the result set to tuples and save

+
+toops = [tuple(row) for row in rows]
+
+with open('some_file.dat', 'w') as f:
+    for tup in toops:
+        f.write(f'{tup}\n')
+
+ +

Read data back in

+
+toops = []
+with open('some_file.dat') as f:
+    for line in f.readlines():
+        toops.append(eval(line))
+
+ +

Note:

+

If the data has any python datetime objects or Decimal objects then you will + need to import the following: +

+ +
+import datetime
+from decimal import *
+
+ +

+ You can now use toops as if it were toops = + results.fetchall() +

+ +