a2rl.WiDataFrame#

class a2rl.WiDataFrame(data=None, states=None, actions=None, rewards=None, **kwargs)[source]#

Bases: DataFrame, SarMixin

A WiDataFrame object is a pandas.DataFrame with additional metadata on the expected column names for states, actions, and rewards (i.e., the sar columns).

In addition to the standard pandas.DataFrame constructor arguments, a WiDataFrame also accepts the following keyword arguments:

Parameters:

states (Optional[Collection[str]]) – The expected column names for states.
actions (Optional[Collection[str]]) – The expected column names for actions.
rewards (Optional[Collection[str]]) – The expected column names for rewards.

Note

By design, a WiDataFrame itself may miss one or more of the sar columns. Downstream tasks should deal with missing sar columns.

Some downstream tasks such as slicing ignores the discrepancy, while RL-related tasks may require all sar columns presented.

See also

WiSeries, pandas.DataFrame

Examples

Create a new WiDataFrame:

>>> import a2rl as wi
>>> df = wi.WiDataFrame(
...     {
...         "s1": [1, 2, 3],
...         "s2": [3, 4, 5],
...         "sess": [0, 0, 0],
...         "z": [6, 7, 8],
...         "a": ["x", "y", "z"],
...         "r": [0.5, 1.5, 2.5],
...     },
...     states=["s1", "s2"],
...     actions=["a"],
...     rewards=["r"],
... )

>>> df
   s1  s2  sess  z  a    r
0   1   3     0  6  x  0.5
1   2   4     0  7  y  1.5
2   3   5     0  8  z  2.5

Check the metadata:

>>> df.sar
['s1', 's2', 'a', 'r']

>>> df.sar_d
{'states': ['s1', 's2'], 'actions': ['a'], 'rewards': ['r']}

>>> df.states
['s1', 's2']

>>> df.actions
['a']

>>> df.rewards
['r']

Slice the states. The resulted WiDataFrame or WiSeries inherits the expected sar columns from the source DataFrame.

>>> df[df.states]
   s1  s2
0   1   3
1   2   4
2   3   5

>>> df[df.states].sar
['s1', 's2', 'a', 'r']

Take just the sar columns:

>>> df.trim()
   s1  s2  a    r
0   1   3  x  0.5
1   2   4  y  1.5
2   3   5  z  2.5

Methods

`add_value`([alpha, gamma, sarsa, value_col, ...])	Append column `value_col` into this dataframe (restriction: `df` must NOT contain column names `_state`, `_action`, `_reward`, and the `value_col`).
`add_value_for_multi_episode_process`([sarsa, ...])	Append column `value_col` into this dataframe (restriction: `df` must NOT contain column names `_state`, `_action`, `_reward`, and the `value_col`).
`to_csv_dataset`(path_or_buf, *args[, ...])	Save this data frame as a `Whatif` dataset.
`trim`([copy])	Get the sar columns of this data frame.

Attributes

`actions`	The list of the expected column names of actions.
`rewards`	The list of the expected column names of rewards.
`sar`	The list of the expected sar column names.
`sar_d`	The dictionary of 585 expected sar column names.
`sequence`	Return a 1D Numpy representation of the DataFrame, in row-major order.
`states`	The list of the expected column names of states.