Giggly.DataFrame behaviour (giggly v0.1.0)

Copy Markdown View Source

A table of named columns of equal length, as Giggly holds one: the data a plot was given, and each table Giggly.layer_data/2 returns. names/1, row_count/1, dtypes/1 and pull/2 read it.

Column names

The data a plot is given is read once, when the plot is created (Giggly.Plot.new/2), and so is a layer's own table, by its function's data: option; their column names become strings whatever the source used. Every column Giggly computes is named by an atom, such as the aesthetic :x, so a user column called "x" can never be mistaken for one.

Values

A column is a list of values in row order, stored as given, but for one conversion its type makes: an integer in a column of floats becomes a float, as Explorer casts one, so 4 and 4.0 are the same value. nil is a missing value, which is how every Table.Reader source returns a null.

Types

Every column has one type, which dtypes/1 reports. It is worked out from the values when the column enters a frame, by Explorer's rules for building a series from a list:

  • the non-missing values are all of one kind, and a column mixing two kinds is refused with an ArgumentError naming the column and two values that disagree;
  • integers and floats together are :float, the integers stored as floats, and :nan, :infinity and :neg_infinity are floats, as Explorer gives a non-finite float;
  • a column of only nil, or of no rows, is :null;
  • binaries are :string;
  • Date, Time and NaiveDateTime are :date, :time and :naive_datetime, and a DateTime is {:datetime, time_zone}, its time zone's name, as Explorer types one, so date-times in two zones are two types ("Etc/UTC" and "Europe/London") and a column mixing them is refused.

A column of lists is {:list, type}, the type its lists' values have, worked out as a column's is from all of them together, so [[1, 2.0], []] is {:list, :float}, its integers stored as floats, a column holding only empty lists is {:list, :null}, a list of lists is a list of a list type, and lists whose values mix two types are refused. A missing list, or a missing value in one, is nil.

Two types exist here and not in Explorer, since a list holds Elixir values an engine cannot: :atom, for any other atom, and :term, for anything else (maps, tuples, Decimal and other structs). A :term column's values are not checked against each other; it is the frame's equivalent of pandas' and Polars' object type, so a column the plot never uses cannot refuse the table.

One more is never worked out from values, only declared when a column is built, or read from a table whose column is a Giggly.Factor: :category, Explorer's name for a categorical column, which a plot reads as discrete whatever its values are. Its values are typed, and an integer among floats becomes a float, as any column's; it may hold anything but a :term's values or lists. A category has levels, the values it may hold in their order, given when it is declared or else its distinct values in order (numbers by value, minus infinity first and NaN last, dates and times by time), and it may be declared ordered; a value not among its levels is missing.

Summary

Types

aggregate()

What summarise/3 computes over each group's values of a column.

category()

How a column is declared a category: :category, its levels its distinct values in order, or {:category, levels: levels, ordered: ordered}, either option left out to take that default (ordered: false).

column()

A column as built: its name and values, and how it is declared a category if it is one.

dtype()

A column's type, as the moduledoc's Types section describes.

expression()

What a frame computes in each row: a column by its name, a constant, or an operation on expressions by its name.

name()

A column name: a string for a column read from the input, an atom for every other.

t()

A frame, read through the functions here.

Functions

dtypes(frame)

Each column's type, by column name, as the moduledoc's Types section describes.

names(frame)

The column names, in column order.

pull(frame, name)

One column's values, as a list in row order. Raises FunctionClauseError when the frame has no column of that name.

row_count(frame)

The number of rows.

Types

aggregate()

@type aggregate() ::
  {:sum, [name()]}
  | {:sum, [name()], summation: :extended | :plain, missing: :propagate | :skip}
  | {:mean, [name()]}
  | {:variance, [name()]}
  | {:n_distinct, [name()]}
  | {:first, [name()]}
  | {:quantile, [name()], [{:probability, number()}]}
  | {:count, [name()]}
  | {:min, [name()]}
  | {:max, [name()]}
  | {:list, [name()]}

What summarise/3 computes over each group's values of a column.

category()

@type category() :: :category | {:category, levels: list(), ordered: boolean()}

How a column is declared a category: :category, its levels its distinct values in order, or {:category, levels: levels, ordered: ordered}, either option left out to take that default (ordered: false).

column()

@type column() :: {name(), list()} | {name(), list(), category()}

A column as built: its name and values, and how it is declared a category if it is one.

dtype()

@type dtype() ::
  :integer
  | :float
  | :string
  | :boolean
  | :atom
  | :date
  | :time
  | :naive_datetime
  | {:datetime, String.t()}
  | :null
  | :term
  | :category
  | {:list, dtype()}

A column's type, as the moduledoc's Types section describes.

expression()

@type expression() ::
  name()
  | number()
  | boolean()
  | Date.t()
  | Time.t()
  | NaiveDateTime.t()
  | DateTime.t()
  | {:constant, term()}
  | {:is_finite, [expression()]}
  | {:and, [expression()]}
  | {:or, [expression()]}
  | {:not, [expression()]}
  | {:<, [expression()]}
  | {:>, [expression()]}
  | {:log10, [expression()]}
  | {:floor, [expression()]}
  | {:if_else, [expression()]}
  | {:cut, [expression()], breaks: [number()], closed: :right | :left}

What a frame computes in each row: a column by its name, a constant, or an operation on expressions by its name.

name()

@type name() :: atom() | String.t()

A column name: a string for a column read from the input, an atom for every other.

t()

@opaque t()

A frame, read through the functions here.

Functions

dtypes(frame)

@spec dtypes(t()) :: %{optional(name()) => dtype()}

Each column's type, by column name, as the moduledoc's Types section describes.

iex> frame =
...>   %{"cyl" => ["6", "4"], "mpg" => [21.0, 22.8]}
...>   |> Giggly.Plot.new(%{x: :cyl, y: :mpg})
...>   |> Giggly.Plot.geom_point()
...>   |> Giggly.layer_data()
iex> Giggly.DataFrame.dtypes(frame)
%{
  x: :integer,
  y: :float,
  panel: :integer,
  group: :integer,
  shape: :atom,
  color: :string,
  fill: :null,
  size: :float,
  alpha: :null,
  stroke: :float
}

names(frame)

@spec names(t()) :: [name()]

The column names, in column order.

pull(frame, name)

@spec pull(t(), name()) :: list()

One column's values, as a list in row order. Raises FunctionClauseError when the frame has no column of that name.

row_count(frame)

@spec row_count(t()) :: non_neg_integer()

The number of rows.