Migrate pickle files to format with better longevity #5

New issue

Open

opened 2024-03-05 14:29:18 +00:00 by mih · 0 comments

mih commented

2024-03-05 14:29:18 +00:00

(Migrated from github.com)

>>> import pandas as pd
>>> df = pd.read_pickle('pickle/joe011_3.pickle')
>>> type(df)
>>> dict
>>> from pprint import pprint
>>> pprint({k: type(v) for k, v in df.items()})
{'column_names': <class 'list'>,
 'event_names': <class 'numpy.ndarray'>,
 'eventtimes': <class 'numpy.ndarray'>,
 'gdf_file': <class 'str'>,
 'spiketimes': <class 'numpy.ndarray'>}

This could be put into npz files: https://numpy.org/doc/stable/reference/generated/numpy.lib.format.html#module-numpy.lib.format

>>> df = pd.read_pickle('pickle/joe011_3.pickle')
>>> numpy.savez('dummy.npz', **df)

>>> npz=numpy.load('dummy.npz')
>>> npz['gdf_file']
array('joe011-345.gdf', dtype='<U14')

>>> npz['gdf_file'].item()
'joe011-345.gdf'

but it can complicate the code in some cases.

``` >>> import pandas as pd >>> df = pd.read_pickle('pickle/joe011_3.pickle') >>> type(df) >>> dict >>> from pprint import pprint >>> pprint({k: type(v) for k, v in df.items()}) {'column_names': <class 'list'>, 'event_names': <class 'numpy.ndarray'>, 'eventtimes': <class 'numpy.ndarray'>, 'gdf_file': <class 'str'>, 'spiketimes': <class 'numpy.ndarray'>} ``` This could be put into `npz` files: https://numpy.org/doc/stable/reference/generated/numpy.lib.format.html#module-numpy.lib.format ``` >>> df = pd.read_pickle('pickle/joe011_3.pickle') >>> numpy.savez('dummy.npz', **df) >>> npz=numpy.load('dummy.npz') >>> npz['gdf_file'] array('joe011-345.gdf', dtype='<U14') >>> npz['gdf_file'].item() 'joe011-345.gdf' ``` but it can complicate the code in some cases.

No labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

sfb1451/a06-inf-clustered-network-pub#5

No description provided.

Rows
Columns