TimedeltaIndex/Scalar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 Memory Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 .dt . . . . . . . . . . . . . . . . . . . 341 4 Frequently Asked Questions (FAQ) 343 4.1 DataFrame memory usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343 4.2 Byte-Ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 819 22.11.1 Memory Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 819 22.11

TimedeltaIndex/Scalar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 Memory Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170 .dt . . . . . . . . . . . . . . . . . . . 343 4 Frequently Asked Questions (FAQ) 345 4.1 DataFrame memory usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 345 4.2 Byte-Ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 821 22.11.1 Memory Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 821 22.11

8 Pandas Google BigQuery support has moved . . . . . . . . . . . . . . . . . . . . . 27 1.3.2.9 Memory Usage for Index is more Accurate . . . . . . . . . . . . . . . . . . . . . . 27 1.3.2.10 DataFrame TimedeltaIndex/Scalar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 215 1.16.1.3 Memory Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218 1.16.1.4 .dt accessor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 908 21.11.1 Memory Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 908 21.11

8 Pandas Google BigQuery support has moved . . . . . . . . . . . . . . . . . . . . . 56 1.5.2.9 Memory Usage for Index is more Accurate . . . . . . . . . . . . . . . . . . . . . . 56 1.5.2.10 DataFrame TimedeltaIndex/Scalar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 vi 1.18.1.3 Memory Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 1.18.1.4 .dt accessor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 942 21.12.1 Memory Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 942 21.12

8 Pandas Google BigQuery support has moved . . . . . . . . . . . . . . . . . . . . . 26 1.2.2.9 Memory Usage for Index is more Accurate . . . . . . . . . . . . . . . . . . . . . . 26 i 1.2.2.10 DataFrame TimedeltaIndex/Scalar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213 1.15.1.3 Memory Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216 1.15.1.4 .dt accessor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 904 21.11.1 Memory Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 904 21.11

text_col 3 non-null object float_col 3 non-null float64 dtypes: float64(1), int64(1), object(1) memory usage: 152.0+ bytes pandas 1.0.0 In [34]: df = pd.DataFrame({"int_col": [1, 2, 3], ....: "text_col": (continued from previous page) 2 float_col 3 non-null float64 dtypes: float64(1), int64(1), object(1) memory usage: 200.0+ bytes 1.5.5 pandas.array() inference changes pandas.array() now infers pandas’ new option of downcasting the newly (or already) numeric data to a smaller dtype, which can conserve memory: In [396]: m = ['1', 2, 3] In [397]: pd.to_numeric(m, downcast='integer') # smallest signed int

(FAQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 869 2.24.1 DataFrame memory usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 869 2.24.2 Using if/truth Cabin 204 non-null object 11 Embarked 889 non-null object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB The method info() provides technical information about a DataFrame, so let’s explain considerations Disk vs memory pandas operates exclusively in memory, where a SAS data set exists on disk. This means that the size of data able to be loaded in pandas is limited by your machine’s memory, but also

(FAQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 869 2.24.1 DataFrame memory usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 869 2.24.2 Using if/truth Cabin 204 non-null object 11 Embarked 889 non-null object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB The method info() provides technical information about a DataFrame, so let’s explain considerations Disk vs memory pandas operates exclusively in memory, where a SAS data set exists on disk. This means that the size of data able to be loaded in pandas is limited by your machine’s memory, but also

(FAQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 978 2.26.1 DataFrame memory usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 978 2.26.2 Using if/truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2839 4.9.2 Checking memory leaks with valgrind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2840 4.10 Extending Cabin 204 non-null object 11 Embarked 889 non-null object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB The method info() provides technical information about a DataFrame, so let’s explain

(FAQ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 978 2.26.1 DataFrame memory usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 978 2.26.2 Using if/truth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2837 4.9.2 Checking memory leaks with valgrind . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2838 4.10 Extending Cabin 204 non-null object 11 Embarked 889 non-null object dtypes: float64(2), int64(5), object(5) memory usage: 83.7+ KB The method info() provides technical information about a DataFrame, so let’s explain