# Computation¶

The labels associated with DataArray and Dataset objects enables some powerful shortcuts for computation, notably including aggregation and broadcasting by dimension names.

## Basic array math¶

Arithmetic operations with a single DataArray automatically vectorize (like numpy) over all array values:

In : arr = xr.DataArray(np.random.randn(2, 3),
...:                    [('x', ['a', 'b']), ('y', [10, 20, 30])])
...:

In : arr - 3
Out:
<xarray.DataArray (x: 2, y: 3)>
array([[-2.5308877 , -3.28286334, -4.5090585 ],
[-4.13563237, -1.78788797, -3.17321465]])
Coordinates:
* x        (x) |S1 'a' 'b'
* y        (y) int64 10 20 30

In : abs(arr)
Out:
<xarray.DataArray (x: 2, y: 3)>
array([[ 0.4691123 ,  0.28286334,  1.5090585 ],
[ 1.13563237,  1.21211203,  0.17321465]])
Coordinates:
* x        (x) |S1 'a' 'b'
* y        (y) int64 10 20 30


You can also use any of numpy’s or scipy’s many ufunc functions directly on a DataArray:

In : np.sin(arr)
Out:
<xarray.DataArray (x: 2, y: 3)>
array([[ 0.45209466, -0.27910634, -0.99809483],
[-0.90680094,  0.9363595 , -0.17234978]])
Coordinates:
* x        (x) |S1 'a' 'b'
* y        (y) int64 10 20 30


Data arrays also implement many numpy.ndarray methods:

In : arr.round(2)
Out:
<xarray.DataArray (x: 2, y: 3)>
array([[ 0.47, -0.28, -1.51],
[-1.14,  1.21, -0.17]])
Coordinates:
* x        (x) |S1 'a' 'b'
* y        (y) int64 10 20 30

In : arr.T
Out:
<xarray.DataArray (y: 3, x: 2)>
array([[ 0.4691123 , -1.13563237],
[-0.28286334,  1.21211203],
[-1.5090585 , -0.17321465]])
Coordinates:
* x        (x) |S1 'a' 'b'
* y        (y) int64 10 20 30


## Missing values¶

xarray objects borrow the isnull(), notnull(), count(), dropna() and fillna() methods for working with missing data from pandas:

In : x = xr.DataArray([0, 1, np.nan, np.nan, 2], dims=['x'])

In : x.isnull()
Out:
<xarray.DataArray (x: 5)>
array([False, False,  True,  True, False], dtype=bool)
Coordinates:
* x        (x) int64 0 1 2 3 4

In : x.notnull()
Out:
<xarray.DataArray (x: 5)>
array([ True,  True, False, False,  True], dtype=bool)
Coordinates:
* x        (x) int64 0 1 2 3 4

In : x.count()
Out:
<xarray.DataArray ()>
array(3)

In : x.dropna(dim='x')
Out:
<xarray.DataArray (x: 3)>
array([ 0.,  1.,  2.])
Coordinates:
* x        (x) int64 0 1 4

In : x.fillna(-1)
Out:
<xarray.DataArray (x: 5)>
array([ 0.,  1., -1., -1.,  2.])
Coordinates:
* x        (x) int64 0 1 2 3 4


Like pandas, xarray uses the float value np.nan (not-a-number) to represent missing values.

## Aggregation¶

Aggregation methods have been updated to take a dim argument instead of axis. This allows for very intuitive syntax for aggregation methods that are applied along particular dimension(s):

In : arr.sum(dim='x')
Out:
<xarray.DataArray (y: 3)>
array([-0.66652007,  0.92924868, -1.68227315])
Coordinates:
* y        (y) int64 10 20 30

In : arr.std(['x', 'y'])
Out:
<xarray.DataArray ()>
array(0.9156385956757354)

In : arr.min()
Out:
<xarray.DataArray ()>
array(-1.5090585031735124)


If you need to figure out the axis number for a dimension yourself (say, for wrapping code designed to work with numpy arrays), you can use the get_axis_num() method:

In : arr.get_axis_num('y')
Out: 1


These operations automatically skip missing values, like in pandas:

In : xr.DataArray([1, 2, np.nan, 3]).mean()
Out:
<xarray.DataArray ()>
array(2.0)


If desired, you can disable this behavior by invoking the aggregation method with skipna=False.

## Rolling window operations¶

DataArray objects include a rolling() method. This method supports rolling window aggregation:

In : arr = xr.DataArray(np.arange(0, 7.5, 0.5).reshape(3, 5),
....:                    dims=('x', 'y'))
....:

In : arr
Out:
<xarray.DataArray (x: 3, y: 5)>
array([[ 0. ,  0.5,  1. ,  1.5,  2. ],
[ 2.5,  3. ,  3.5,  4. ,  4.5],
[ 5. ,  5.5,  6. ,  6.5,  7. ]])
Coordinates:
* x        (x) int64 0 1 2
* y        (y) int64 0 1 2 3 4


rolling() is applied along one dimension using the name of the dimension as a key (e.g. y) and the window size as the value (e.g. 3). We get back a Rolling object:

In : arr.rolling(y=3)
Out: DataArrayRolling [window->3,center->False,dim->y]


The label position and minimum number of periods in the rolling window are controlled by the center and min_periods arguments:

In : arr.rolling(y=3, min_periods=2, center=True)
Out: DataArrayRolling [window->3,min_periods->2,center->True,dim->y]


Aggregation and summary methods can be applied directly to the Rolling object:

In : r = arr.rolling(y=3)

In : r.mean()
Out:
<xarray.DataArray (y: 5, x: 3)>
array([[ nan,  nan,  nan],
[ nan,  nan,  nan],
[ 0.5,  3. ,  5.5],
[ 1. ,  3.5,  6. ],
[ 1.5,  4. ,  6.5]])
Coordinates:
* x        (x) int64 0 1 2
* y        (y) int64 0 1 2 3 4

In : r.reduce(np.std)
Out:
<xarray.DataArray (y: 5, x: 3)>
array([[        nan,         nan,         nan],
[        nan,         nan,         nan],
[ 0.40824829,  0.40824829,  0.40824829],
[ 0.40824829,  0.40824829,  0.40824829],
[ 0.40824829,  0.40824829,  0.40824829]])
Coordinates:
* x        (x) int64 0 1 2
* y        (y) int64 0 1 2 3 4


Note that rolling window aggregations are much faster (both asymptotically and because they avoid a loop in Python) when bottleneck is installed. Otherwise, we fall back to a slower, pure Python implementation.

Finally, we can manually iterate through Rolling objects:

In : for label, arr_window in r:
# arr_window is a view of x


DataArray objects are automatically align themselves (“broadcasting” in the numpy parlance) by dimension name instead of axis order. With xarray, you do not need to transpose arrays or insert dimensions of length 1 to get array operations to work, as commonly done in numpy with np.reshape() or np.newaxis.

This is best illustrated by a few examples. Consider two one-dimensional arrays with different sizes aligned along different dimensions:

In : a = xr.DataArray([1, 2], [('x', ['a', 'b'])])

In : a
Out:
<xarray.DataArray (x: 2)>
array([1, 2])
Coordinates:
* x        (x) |S1 'a' 'b'

In : b = xr.DataArray([-1, -2, -3], [('y', [10, 20, 30])])

In : b
Out:
<xarray.DataArray (y: 3)>
array([-1, -2, -3])
Coordinates:
* y        (y) int64 10 20 30


With xarray, we can apply binary mathematical operations to these arrays, and their dimensions are expanded automatically:

In : a * b
Out:
<xarray.DataArray (x: 2, y: 3)>
array([[-1, -2, -3],
[-2, -4, -6]])
Coordinates:
* x        (x) |S1 'a' 'b'
* y        (y) int64 10 20 30


Moreover, dimensions are always reordered to the order in which they first appeared:

In : c = xr.DataArray(np.arange(6).reshape(3, 2), [b['y'], a['x']])

In : c
Out:
<xarray.DataArray (y: 3, x: 2)>
array([[0, 1],
[2, 3],
[4, 5]])
Coordinates:
* y        (y) int64 10 20 30
* x        (x) |S1 'a' 'b'

In : a + c
Out:
<xarray.DataArray (x: 2, y: 3)>
array([[1, 3, 5],
[3, 5, 7]])
Coordinates:
* x        (x) |S1 'a' 'b'
* y        (y) int64 10 20 30


This means, for example, that you always subtract an array from its transpose:

In : c - c.T
Out:
<xarray.DataArray (y: 3, x: 2)>
array([[0, 0],
[0, 0],
[0, 0]])
Coordinates:
* y        (y) int64 10 20 30
* x        (x) |S1 'a' 'b'


You can explicitly broadcast xaray data structures by using the broadcast() function:

a2, b2 = xr.broadcast(a, b2) a2 b2

## Automatic alignment¶

xarray enforces alignment between index Coordinates (that is, coordinates with the same name as a dimension, marked by *) on objects used in binary operations.

Similarly to pandas, this alignment is automatic for arithmetic on binary operations. Note that unlike pandas, this the result of a binary operation is by the intersection (not the union) of coordinate labels:

In : arr + arr[:1]
Out:
<xarray.DataArray (x: 1, y: 5)>
array([[ 0.,  1.,  2.,  3.,  4.]])
Coordinates:
* x        (x) int64 0
* y        (y) int64 0 1 2 3 4


If the result would be empty, an error is raised instead:

In : arr[:2] + arr[2:]
ValueError: no overlapping labels for some dimensions: ['x']


Before loops or performance critical code, it’s a good idea to align arrays explicitly (e.g., by putting them in the same Dataset or using align()) to avoid the overhead of repeated alignment with each operation. See Align and reindex for more details.

Note

There is no automatic alignment between arguments when performing in-place arithmetic operations such as +=. You will need to use manual alignment. This ensures in-place arithmetic never needs to modify data types.

## Coordinates¶

Although index coordinates are aligned, other coordinates are not, and if their values conflict, they will be dropped. This is necessary, for example, because indexing turns 1D coordinates into scalar coordinates:

In : arr
Out:
<xarray.DataArray (y: 5)>
array([ 0. ,  0.5,  1. ,  1.5,  2. ])
Coordinates:
x        int64 0
* y        (y) int64 0 1 2 3 4

In : arr
Out:
<xarray.DataArray (y: 5)>
array([ 2.5,  3. ,  3.5,  4. ,  4.5])
Coordinates:
x        int64 1
* y        (y) int64 0 1 2 3 4

# notice that the scalar coordinate 'x' is silently dropped
In : arr - arr
Out:
<xarray.DataArray (y: 5)>
array([ 2.5,  2.5,  2.5,  2.5,  2.5])
Coordinates:
* y        (y) int64 0 1 2 3 4


Still, xarray will persist other coordinates in arithmetic, as long as there are no conflicting values:

# only one argument has the 'x' coordinate
In : arr + 1
Out:
<xarray.DataArray (y: 5)>
array([ 1. ,  1.5,  2. ,  2.5,  3. ])
Coordinates:
x        int64 0
* y        (y) int64 0 1 2 3 4

# both arguments have the same 'x' coordinate
In : arr - arr
Out:
<xarray.DataArray (y: 5)>
array([ 0.,  0.,  0.,  0.,  0.])
Coordinates:
x        int64 0
* y        (y) int64 0 1 2 3 4


## Math with datasets¶

Datasets support arithmetic operations by automatically looping over all data variables:

In : ds = xr.Dataset({'x_and_y': (('x', 'y'), np.random.randn(3, 5)),
....:                  'x_only': ('x', np.random.randn(3))},
....:                  coords=arr.coords)
....:

In : ds > 0
Out:
<xarray.Dataset>
Dimensions:  (x: 3, y: 5)
Coordinates:
* y        (y) int64 0 1 2 3 4
* x        (x) int64 0 1 2
Data variables:
x_only   (x) bool True False True
x_and_y  (x, y) bool True False False False False True True False False ...


Datasets support most of the same methods found on data arrays:

In : ds.mean(dim='x')
Out:
<xarray.Dataset>
Dimensions:  (y: 5)
Coordinates:
* y        (y) int64 0 1 2 3 4
Data variables:
x_only   float64 -0.2799
x_and_y  (y) float64 0.2553 0.08145 -0.4308 -1.411 -0.2989

In : abs(ds)
Out:
<xarray.Dataset>
Dimensions:  (x: 3, y: 5)
Coordinates:
* y        (y) int64 0 1 2 3 4
* x        (x) int64 0 1 2
Data variables:
x_only   (x) float64 0.1136 1.478 0.525
x_and_y  (x, y) float64 0.1192 1.044 0.8618 2.105 0.4949 1.072 0.7216 ...


Unfortunately, a limitation of the current version of numpy means that we cannot override ufuncs for datasets, because datasets cannot be written as a single array . apply() works around this limitation, by applying the given function to each variable in the dataset:

In : ds.apply(np.sin)
Out:
<xarray.Dataset>
Dimensions:  (x: 3, y: 5)
Coordinates:
* x        (x) int64 0 1 2
* y        (y) int64 0 1 2 3 4
Data variables:
x_only   (x) float64 0.1134 -0.9957 0.5012
x_and_y  (x, y) float64 0.1189 -0.8645 -0.759 -0.8609 -0.475 0.8781 ...


Datasets also use looping over variables for broadcasting in binary arithmetic. You can do arithmetic between any DataArray and a dataset:

In : ds + arr
Out:
<xarray.Dataset>
Dimensions:  (x: 3, y: 5)
Coordinates:
* x        (x) int64 0 1 2
* y        (y) int64 0 1 2 3 4
Data variables:
x_only   (x, y) float64 0.1136 0.6136 1.114 1.614 2.114 1.022 1.522 ...
x_and_y  (x, y) float64 0.1192 -0.5442 0.1382 -0.6046 1.505 3.572 3.722 ...


Arithmetic between two datasets matches data variables of the same name:

In : ds2 = xr.Dataset({'x_and_y': 0, 'x_only': 100})

In : ds - ds2
Out:
<xarray.Dataset>
Dimensions:  (x: 3, y: 5)
Coordinates:
* x        (x) int64 0 1 2
* y        (y) int64 0 1 2 3 4
Data variables:
x_only   (x) float64 -99.89 -101.5 -99.48
x_and_y  (x, y) float64 0.1192 -1.044 -0.8618 -2.105 -0.4949 1.072 ...


Similarly to index based alignment, the result has the intersection of all matching variables, and ValueError is raised if the result would be empty.

  In some future version of NumPy, we should be able to override ufuncs for datasets by making use of __numpy_ufunc__.