and Analytics Vasiliki (Vasia) Kalavri
 vkalavri@bu.edu Spring 2020 4/23: Cardinality and frequency estimation ??? Vasiliki Kalavri | Boston University 2020 Counting distinct elements 2 ??? probability • Counter overestimation is almost certain for very large data streams with high-frequency elements Counting Bloom Filter ??? Vasiliki Kalavri | Boston University 2020 20 • A space-efficient 6 2 3 2 2 9 7 3 0 5 8 5 0 9 0 … ??? Vasiliki Kalavri | Boston University 2020 23 Estimating frequency 0 0 0 6 9 3 3 1 5 0 0 3 8 2 7 9 m counters h1 h2 hp 3 0 0 3 0 5 8 2 0 0 2 9 2 4 5 2 7 6 2

columns, as in an SQL table or Excel spreadsheet • Ordered and unordered (not necessarily fixed-frequency) time series data. 12 Chapter 1. Getting started pandas: powerful Python data analysis toolkit from the ultrafast HDF5 format • Time series-specific functionality: date range generation and frequency conversion, moving window statistics, date shifting, and lagging. Many of these principles are strings is provided in the section on time series indexing. Resample a time series to another frequency Aggregate the current hourly time series values to the monthly maximum value in each of the stations

columns, as in an SQL table or Excel spreadsheet • Ordered and unordered (not necessarily fixed-frequency) time series data. • Arbitrary matrix data (homogeneously typed or heterogeneous) with row and from the ultrafast HDF5 format • Time series-specific functionality: date range generation and frequency conversion, moving window statistics, moving window linear regressions, date shifting and lagging array of whether the timestamp(s) are at the start/end of the month/quarter/year defined by the frequency of the DateTimeIndex / Timestamp (GH4565, GH6998) • Local variable usage has changed in pandas

DateArray properties and functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 3.3 Frequency conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 178 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544 20.4 Frequency Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545 columns, as in an SQL table or Excel spreadsheet • Ordered and unordered (not necessarily fixed-frequency) time series data. • Arbitrary matrix data (homogeneously typed or heterogeneous) with row and

DateArray properties and functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 3.3 Frequency conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534 20.4 Frequency Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535 columns, as in an SQL table or Excel spreadsheet • Ordered and unordered (not necessarily fixed-frequency) time series data. • Arbitrary matrix data (homogeneously typed or heterogeneous) with row and

columns, as in an SQL table or Excel spreadsheet • Ordered and unordered (not necessarily fixed-frequency) time series data. • Arbitrary matrix data (homogeneously typed or heterogeneous) with row and from the ultrafast HDF5 format • Time series-specific functionality: date range generation and frequency conversion, moving window statistics, moving window linear regressions, date shifting and lagging warn with a AttributeConflictWarning if you are attempting to append an index with a different frequency than the existing, or attempting to append an index with a different name than the existing – support

columns, as in an SQL table or Excel spreadsheet • Ordered and unordered (not necessarily fixed-frequency) time series data. • Arbitrary matrix data (homogeneously typed or heterogeneous) with row and from the ultrafast HDF5 format • Time series-specific functionality: date range generation and frequency conversion, moving window statistics, moving window linear regressions, date shifting and lagging divided by another timedelta64[ns] object, or astyped to yield a float64 dtyped Series. This is frequency conversion. See the docs for the docs. In [69]: from datetime import timedelta In [70]: td =

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 642 21.4 Frequency Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643 columns, as in an SQL table or Excel spreadsheet • Ordered and unordered (not necessarily fixed-frequency) time series data. • Arbitrary matrix data (homogeneously typed or heterogeneous) with row and from the ultrafast HDF5 format • Time series-specific functionality: date range generation and frequency conversion, moving window statistics, moving window linear regressions, date shifting and lagging

methods for dt accessor . . . . . . . . . . . . . . . . . . . . . . . . . 177 1.12.1.5 Period Frequency Enhancement . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 1.12.1.6 Support for SAS Timestamps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 847 19.4.1 Custom Frequency Ranges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 849 19.5 Timestamp Lagging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 874 19.9.2 Frequency Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 875 19.9.3

MultiIndex name orders. (GH25760, GH28956) • Bug Series.pct_change() where supplying an anchored frequency would throw a ValueError (GH28664) 1.9. Bug fixes 31 pandas: powerful Python data analysis toolkit columns, as in an SQL table or Excel spreadsheet • Ordered and unordered (not necessarily fixed-frequency) time series data. • Arbitrary matrix data (homogeneously typed or heterogeneous) with row and from the ultrafast HDF5 format • Time series-specific functionality: date range generation and frequency conversion, moving window statistics, date shifting and lagging. Many of these principles are here