Fix Python – Creating dataframe from a dictionary where entries have different lengths

by

Question

Asked By – Josh

Say I have a dictionary with 10 key-value pairs. Each entry holds a numpy array. However, the length of the array is not the same for all of them.

How can I create a dataframe where each column holds a different entry?

When I try:

pd.DataFrame(my_dict)

I get:

ValueError: arrays must all be the same length

Any way to overcome this? I am happy to have Pandas use NaN to pad those columns for the shorter entries.

Now we will see solution for issue: Creating dataframe from a dictionary where entries have different lengths

Answer

In Python 3.x:

import pandas as pd
import numpy as np

d = dict( A = np.array([1,2]), B = np.array([1,2,3,4]) )
    
pd.DataFrame(dict([ (k,pd.Series(v)) for k,v in d.items() ]))

Out[7]: 
    A  B
0   1  1
1   2  2
2 NaN  3
3 NaN  4

In Python 2.x:

replace d.items() with d.iteritems().

This question is answered By – Jeff

This answer is collected from stackoverflow and reviewed by FixPython community admins, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0