Combining data¶

Concatenate¶

To combine arrays along existing or new dimension into a larger arrays, you can use concat(). concat takes an iterable of DataArray or Dataset objects, as well as a dimension name, and concatenates along that dimension:

In [1]: arr = xray.DataArray(np.random.randn(2, 3),
...:                      [('x', ['a', 'b']), ('y', [10, 20, 30])])
...:

In [2]: arr[:, :1]
Out[2]:
<xray.DataArray (x: 2, y: 1)>
array([[ 0.4691123 ],
[-1.13563237]])
Coordinates:
* x        (x) |S1 'a' 'b'
* y        (y) int64 10

# this resembles how you would use np.concatenate
In [3]: xray.concat([arr[:, :1], arr[:, 1:]], dim='y')
Out[3]:
<xray.DataArray (x: 2, y: 3)>
array([[ 0.4691123 , -0.28286334, -1.5090585 ],
[-1.13563237,  1.21211203, -0.17321465]])
Coordinates:
* x        (x) |S1 'a' 'b'
* y        (y) int64 10 20 30

In addition to combining along an existing dimension, concat can create a new dimension by stacking lower dimensional arrays together:

In [4]: arr[0]
Out[4]:
<xray.DataArray (y: 3)>
array([ 0.4691123 , -0.28286334, -1.5090585 ])
Coordinates:
x        |S1 'a'
* y        (y) int64 10 20 30

# to combine these 1d arrays into a 2d array in numpy, you would use np.array
In [5]: xray.concat([arr[0], arr[1]], 'x')
Out[5]:
<xray.DataArray (x: 2, y: 3)>
array([[ 0.4691123 , -0.28286334, -1.5090585 ],
[-1.13563237,  1.21211203, -0.17321465]])
Coordinates:
* y        (y) int64 10 20 30
* x        (x) object 'a' 'b'

If the second argument to concat is a new dimension name, the arrays will be concatenated along that new dimension, which is always inserted as the first dimension:

In [6]: xray.concat([arr[0], arr[1]], 'new_dim')
Out[6]:
<xray.DataArray (new_dim: 2, y: 3)>
array([[ 0.4691123 , -0.28286334, -1.5090585 ],
[-1.13563237,  1.21211203, -0.17321465]])
Coordinates:
* y        (y) int64 10 20 30
* new_dim  (new_dim) int64 0 1
x        (new_dim) object 'a' 'b'

This is actually the default behavior for concat:

In [7]: xray.concat([arr[0], arr[1]])
Out[7]:
<xray.DataArray (concat_dim: 2, y: 3)>
array([[ 0.4691123 , -0.28286334, -1.5090585 ],
[-1.13563237,  1.21211203, -0.17321465]])
Coordinates:
* y           (y) int64 10 20 30
* concat_dim  (concat_dim) int64 0 1
x           (concat_dim) object 'a' 'b'

The second argument to concat can also be an Index or DataArray object as well as a string, in which case it is used to label the values along the new dimension:

In [8]: xray.concat([arr[0], arr[1]], pd.Index([-90, -100], name='new_dim'))
Out[8]:
<xray.DataArray (new_dim: 2, y: 3)>
array([[ 0.4691123 , -0.28286334, -1.5090585 ],
[-1.13563237,  1.21211203, -0.17321465]])
Coordinates:
* y        (y) int64 10 20 30
* new_dim  (new_dim) int64 -90 -100
x        (new_dim) object 'a' 'b'

Of course, concat also works on Dataset objects:

In [9]: ds = arr.to_dataset(name='foo')

In [10]: xray.concat([ds.sel(x='a'), ds.sel(x='b')], 'x')
Out[10]:
<xray.Dataset>
Dimensions:  (x: 2, y: 3)
Coordinates:
* y        (y) int64 10 20 30
* x        (x) object 'a' 'b'
Variables:
foo      (x, y) float64 0.4691 -0.2829 -1.509 -1.136 1.212 -0.1732

concat() has a number of options which provide deeper control over which variables and coordinates are concatenated and how it handles conflicting variables between datasets. However, these should rarely be necessary.

Merge and update¶

To combine variables and coordinates between multiple Datasets, you can use the merge() and update() methods. Merge checks for conflicting variables before merging and by default it returns a new Dataset:

In [11]: ds.merge({'hello': ('space', np.arange(3) + 10)})
Out[11]:
<xray.Dataset>
Dimensions:  (space: 3, x: 2, y: 3)
Coordinates:
* y        (y) int64 10 20 30
* x        (x) |S1 'a' 'b'
* space    (space) int64 0 1 2
Variables:
foo      (x, y) float64 0.4691 -0.2829 -1.509 -1.136 1.212 -0.1732
hello    (space) int64 10 11 12

In contrast, update modifies a dataset in-place without checking for conflicts, and will overwrite any existing variables with new values:

In [12]: ds.update({'space': ('space', [10.2, 9.4, 3.9])})
Out[12]:
<xray.Dataset>
Dimensions:  (space: 3, x: 2, y: 3)
Coordinates:
* y        (y) int64 10 20 30
* x        (x) |S1 'a' 'b'
* space    (space) float64 10.2 9.4 3.9
Variables:
foo      (x, y) float64 0.4691 -0.2829 -1.509 -1.136 1.212 -0.1732

However, dimensions are still required to be consistent between different Dataset variables, so you cannot change the size of a dimension unless you replace all dataset variables that use it.

Equals and identical¶

xray objects can be compared by using the equals() and identical() methods. These methods are used by the optional compat argument on concat and merge.

equals checks dimension names, indexes and array values:

In [13]: arr.equals(arr.copy())
Out[13]: True

identical also checks attributes, and the name of each object:

In [14]: arr.identical(arr.rename('bar'))
Out[14]: False

In contrast, the == operation performs element-wise comparison (like numpy):

In [15]: arr == arr.copy()
Out[15]:
<xray.DataArray (x: 2, y: 3)>
array([[ True,  True,  True],
[ True,  True,  True]], dtype=bool)
Coordinates:
* y        (y) int64 10 20 30
* x        (x) |S1 'a' 'b'

Like pandas objects, two xray objects are still equal or identical if they have missing values marked by NaN in the same locations.