Documentation

NumPy

class kymatio.numpy.HarmonicScattering3D(J, shape, L=3, sigma_0=1, max_order=2, rotation_covariant=True, method='integral', points=None, integral_powers=(0.5, 1.0, 2.0), backend='numpy')

Bases: kymatio.frontend.numpy_frontend.ScatteringNumPy, kymatio.scattering3d.frontend.base_frontend.ScatteringBase3D

The 3D solid harmonic scattering transform

This class implements solid harmonic scattering on a 3D input image. For details see https://arxiv.org/abs/1805.00571.

Example

# Set the parameters of the scattering transform.
J = 3
M, N, O = 32, 32, 32

# Generate a sample signal.
x = np.random.randn(M, N, O)

# Define a HarmonicScattering3D object.
S = HarmonicScattering3D(J, (M, N, O))

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S(x)
Parameters
  • J (int) – Number of scales.

  • shape (tuple of ints) – Shape (M, N, O) of the input signal

  • L (int, optional) – Number of l values. Defaults to 3.

  • sigma_0 (float, optional) – Bandwidth of mother wavelet. Defaults to 1.

  • max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

  • rotation_covariant (bool, optional) –

    If set to True the first-order moduli take the form:

    \sqrt{\sum_m (x \star \psi_{j,l,m})^2)}

    if set to False the first-order moduli take the form:

    x \star \psi_{j,l,m}

    The second order moduli change analogously. Defaults to True.

  • method (string, optional) – Specifies the method for obtaining scattering coefficients. Currently, only ‘integral’ is available. Defaults to ‘integral’.

  • integral_powers (array-like) – List of exponents to the power of which moduli are raised before integration.

build()

Defines elementary routines.

This function should always call and create the filters via self.create_filters() defined below. For instance, via: self.filters = self.create_filters()

scattering(input_array)

Apply the scattering transform

Parameters

input_array (np.ndarray) – Input of size (batch_size, M, N, O).

Returns

output – If max_order is 1 it returns an np.ndarray with the first-order scattering coefficients. If max_order is 2 it returns an np.ndarray with the first- and second- order scattering coefficients, concatenated along the feature axis.

Return type

np.ndarray

class kymatio.numpy.Scattering1D(J, shape, Q=1, max_order=2, average=True, oversampling=0, vectorize=True, out_type='array', backend='numpy')

Bases: kymatio.frontend.numpy_frontend.ScatteringNumPy, kymatio.scattering1d.frontend.base_frontend.ScatteringBase1D

The 1D scattering transform

The scattering transform computes a cascade of wavelet transforms alternated with a complex modulus non-linearity. The scattering transform of a 1D signal x(t) may be written as

S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]

where

S_J^{(0)} x(t) = x \star \phi_J(t),

S_J^{(1)} x(t, \lambda) = |x \star \psi_\lambda^{(1)}| \star \phi_J, and

S_J^{(2)} x(t, \lambda, \mu) = |\,| x \star \psi_\lambda^{(1)}| \star \psi_\mu^{(2)} | \star \phi_J.

In the above formulas, \star denotes convolution in time. The filters \psi_\lambda^{(1)}(t) and \psi_\mu^{(2)}(t) are analytic wavelets with center frequencies \lambda and \mu, while \phi_J(t) is a real lowpass filter centered at the zero frequency.

The Scattering1D class implements the 1D scattering transform for a given set of filters whose parameters are specified at initialization. While the wavelets are fixed, other parameters may be changed after the object is created, such as whether to compute all of S_J^{(0)} x, S_J^{(1)} x, and S_J^{(2)} x or just S_J^{(0)} x and S_J^{(1)} x.

Given an input np.ndarray x of shape (B, T), where B is the number of signals to transform (the batch size) and T is the length of the signal, we compute its scattering transform by passing it to the scattering method (or calling the alias __call__). Note that B can be one, in which case it may be omitted, giving an input of shape (T,).

# Set the parameters of the scattering transform.
J = 6
T = 2 ** 13
Q = 8

# Generate a sample signal.
x = np.random.randn(T)

# Define a Scattering1D object.
S = Scattering1D(J, T, Q)

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S(x)

Above, the length of the signal is T = 2^{13} = 8192, while the maximum scale of the scattering transform is set to 2^J = 2^6 =
64. The time-frequency resolution of the first-order wavelets \psi_\lambda^{(1)}(t) is set to Q = 8 wavelets per octave. The second-order wavelets \psi_\mu^{(2)}(t) always have one wavelet per octave.

Jint

The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2^J.

shapeint

The length of the input signals.

Qint >= 1

The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave). Defaults to 1.

max_orderint, optional

The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

averageboolean, optional

Determines whether the output is averaged in time or not. The averaged output corresponds to the standard scattering transform, while the un-averaged output skips the last convolution by \phi_J(t). This parameter may be modified after object creation. Defaults to True.

oversamplinginteger >= 0, optional

Controls the oversampling factor relative to the default as a power of two. Since the convolving by wavelets (or lowpass filters) and taking the modulus reduces the high-frequency content of the signal, we can subsample to save space and improve performance. However, this may reduce precision in the calculation. If this is not desirable, oversampling can be set to a large value to prevent too much subsampling. This parameter may be modified after object creation. Defaults to 0.

vectorizeboolean, optional

Determines wheter to return a vectorized scattering transform (that is, a large array containing the output) or a dictionary (where each entry corresponds to a separate scattering coefficient). This parameter may be modified after object creation. Deprecated in favor of out_type (see below). Defaults to True.

out_typestr, optional

The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering coefficient with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.

Jint

The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2 ** J.

shapeint

The length of the input signals.

Qint

The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave).

J_padint

The logarithm of the padded length of the signals.

pad_leftint

The amount of padding to the left of the signal.

pad_rightint

The amount of padding to the right of the signal.

phi_fdictionary

A dictionary containing the lowpass filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.

psi1_fdictionary

A dictionary containing all the first-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.

psi2_fdictionary

A dictionary containing all the second-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.

max_orderint

The maximum scattering order of the transform.

averageboolean

Controls whether the output should be averaged (the standard scattering transform) or not (resulting in wavelet modulus coefficients). Note that to obtain unaveraged output, the vectorize flag must be set to False or out_type must be set to ‘list’.

oversamplingint

The number of powers of two to oversample the output compared to the default subsampling rate determined from the filters.

vectorizeboolean

Controls whether the output should be vectorized into a single Tensor or collected into a dictionary. Deprecated in favor of out_type. For more details, see the documentation for scattering.

out_typestr

Specifices the output format of the transform, which is currently one of ‘array’ or ‘list’. If ‘array’, the output is a large array containing the scattering coefficients. If ‘list’, the output is a list of dictionaries, each containing a scattering coefficient along with meta information. For more information, see the documentation for scattering.

scattering(x)

Apply the scattering transform

Given an input np.ndarray of size (B, T), where B is the batch size (it can be potentially an integer or a shape) and T is the length of the individual signals, this function computes its scattering transform. If the vectorize flag is set to True (or if it is not available in this frontend), the output is in the form of a np.ndarray or size (B, C, T1), where T1 is the signal length after subsampling to the scale 2^J (with the appropriate oversampling factor to reduce aliasing), and C is the number of scattering coefficients. If vectorize is set False, however, the output is a dictionary containing C keys, each a tuple whose length corresponds to the scattering order and whose elements are the sequence of filter indices used.

Note that the vectorize flag has been deprecated in favor of the out_type parameter. If this is set to ‘array’ (the default), the vectorize flag is still respected, but if not, out_type takes precedence. The two current output types are ‘array’ and ‘list’. The former gives the type of output described above. If set to ‘list’, however, the output is a list of dictionaries, each dictionary corresponding to a scattering coefficient and its associated meta information. The coefficient is stored under the ‘coef’ key, while other keys contain additional information, such as ‘j’ (the scale of the filter used) and ‘n’ (the filter index).

Furthermore, if the average flag is set to False, these outputs are not averaged, but are simply the wavelet modulus coefficients of the filters.

Parameters

x (np.ndarray) – An input np.ndarray of size (B, T).

Returns

S – If out_type is ‘array’ and the vectorize flag is True, the output is an np.ndarray containing the scattering coefficients, while if vectorize is False, it is a dictionary indexed by tuples of filter indices. If out_type is ‘list’, the output is a list of dictionaries as described above.

Return type

tensor or dictionary

class kymatio.numpy.Scattering2D(J, shape, L=8, max_order=2, pre_pad=False, backend='numpy', out_type='array')

Bases: kymatio.frontend.numpy_frontend.ScatteringNumPy, kymatio.scattering2d.frontend.base_frontend.ScatteringBase2D

The 2D scattering transform

The scattering transform computes two wavelet transform followed by modulus non-linearity. It can be summarized as

S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]

where

S_J^{(0)} x = x \star \phi_J,

S_J^{(1)} x = [|x \star \psi^{(1)}_\lambda| \star \phi_J]_\lambda, and

S_J^{(2)} x = [||x \star \psi^{(1)}_\lambda| \star
\psi^{(2)}_\mu| \star \phi_J]_{\lambda, \mu}.

where \star denotes the convolution (in space), \phi_J is a lowpass filter, \psi^{(1)}_\lambda is a family of bandpass filters and \psi^{(2)}_\mu is another family of bandpass filters. Only Morlet filters are used in this implementation. Convolutions are efficiently performed in the Fourier domain.

Example

# Set the parameters of the scattering transform.
J = 3
M, N = 32, 32

# Generate a sample signal.
x = np.random.randn(M, N)

# Define a Scattering2D object.
S = Scattering2D(J, (M, N))

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S(x)
Parameters
  • J (int) – Log-2 of the scattering scale.

  • shape (tuple of ints) – Spatial support (M, N) of the input

  • L (int, optional) – Number of angles used for the wavelet transform. Defaults to 8.

  • max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

  • pre_pad (boolean, optional) – Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally. Defaults to False.

  • backend (object, optional) – Controls the backend which is combined with the frontend.

  • out_type (str, optional) – The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering path with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.

J

Log-2 of the scattering scale.

Type

int

shape

Spatial support (M, N) of the input

Type

tuple of ints

L

Number of angles used for the wavelet transform.

Type

int, optional

max_order

The maximum order of scattering coefficients to compute. Must be either 1 or 2.

Type

int, optional

pre_pad

Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally.

Type

boolean

Psi

Contains the wavelets filters at all resolutions. See filter_bank.filter_bank for an exact description.

Type

dictionary

Phi

Contains the low-pass filters at all resolutions. See filter_bank.filter_bank for an exact description.

Type

dictionary

M_padded, N_padded

Spatial support of the padded input.

Type

int

out_type

The format of the scattering output. See documentation for out_type parameter above and the documentation for scattering.

Type

str

Notes

The design of the filters is optimized for the value L = 8.

The pre_pad flag is particularly useful when cropping bigger images because this does not introduce border effects inherent to padding.

scattering(input)

Apply the scattering transform

Parameters

input (np.ndarray) – An input np.ndarray of size (B, M, N).

Raises
  • RuntimeError – In the event that the input does not have at least two dimensions, or the tensor is not contiguous, or the tensor is not of the correct spatial size, padded or not.

  • TypeError – In the event that the input is not of type np.ndarray.

Returns

S – Scattering transform of the input. If out_type is set to ‘array’ (or if it is not availabel for this frontend), this is an np.ndarray of shape (B, C, M1, N1) where M1 = M // 2 ** J and N1 = N // 2 ** J. The C is the number of scattering channels calculated. If out_type is ‘list’, the output is a list of dictionaries, with each dictionary corresponding to a scattering coefficient and its meta information. The actual coefficient is contained in the ‘coef’ key, while other keys hold additional information, such as ‘j’ (the scale of the filter used), and ‘theta’ (the angle index of the filter used).

Return type

np.ndarray

Scikit-learn

class kymatio.sklearn.HarmonicScattering3D(J, shape, L=3, sigma_0=1, max_order=2, rotation_covariant=True, method='integral', points=None, integral_powers=(0.5, 1.0, 2.0), backend='numpy')

Bases: kymatio.frontend.sklearn_frontend.ScatteringTransformerMixin, kymatio.numpy.HarmonicScatteringNumPy3D

The 3D solid harmonic scattering transform

This class implements solid harmonic scattering on a 3D input image. For details see https://arxiv.org/abs/1805.00571.

This class inherits from BaseEstimator and TransformerMixin in sklearn.base. As a result, it supports calculating the scattering transform by calling the predict and transform methods. By extension, it can be included as part of a scikit-learn Pipeline.

Example

# Set the parameters of the scattering transform.
J = 3
M, N, O = 32, 32, 32

# Generate a sample signal.
x = np.random.randn(np.prod(M, N, O))

# Define a HarmonicScattering3D object.
S = HarmonicScattering3D(J, (M, N, O))

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S.predict(x)
Parameters
  • J (int) – Number of scales.

  • shape (tuple of ints) – Shape (M, N, O) of the input signal

  • L (int, optional) – Number of l values. Defaults to 3.

  • sigma_0 (float, optional) – Bandwidth of mother wavelet. Defaults to 1.

  • max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

  • rotation_covariant (bool, optional) –

    If set to True the first-order moduli take the form:

    \sqrt{\sum_m (x \star \psi_{j,l,m})^2)}

    if set to False the first-order moduli take the form:

    x \star \psi_{j,l,m}

    The second order moduli change analogously. Defaults to True.

  • method (string, optional) – Specifies the method for obtaining scattering coefficients. Currently, only ‘integral’ is available. Defaults to ‘integral’.

  • integral_powers (array-like) – List of exponents to the power of which moduli are raised before integration.

class kymatio.sklearn.Scattering1D(J, shape, Q=1, max_order=2, average=True, oversampling=0, vectorize=True, out_type='array', backend='numpy')

Bases: kymatio.frontend.sklearn_frontend.ScatteringTransformerMixin, kymatio.numpy.ScatteringNumPy1D

The 1D scattering transform

The scattering transform computes a cascade of wavelet transforms alternated with a complex modulus non-linearity. The scattering transform of a 1D signal x(t) may be written as

S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]

where

S_J^{(0)} x(t) = x \star \phi_J(t),

S_J^{(1)} x(t, \lambda) = |x \star \psi_\lambda^{(1)}| \star \phi_J, and

S_J^{(2)} x(t, \lambda, \mu) = |\,| x \star \psi_\lambda^{(1)}| \star \psi_\mu^{(2)} | \star \phi_J.

In the above formulas, \star denotes convolution in time. The filters \psi_\lambda^{(1)}(t) and \psi_\mu^{(2)}(t) are analytic wavelets with center frequencies \lambda and \mu, while \phi_J(t) is a real lowpass filter centered at the zero frequency.

The Scattering1D class implements the 1D scattering transform for a given set of filters whose parameters are specified at initialization. While the wavelets are fixed, other parameters may be changed after the object is created, such as whether to compute all of S_J^{(0)} x, S_J^{(1)} x, and S_J^{(2)} x or just S_J^{(0)} x and S_J^{(1)} x.

This class inherits from BaseEstimator and TransformerMixin in sklearn.base. As a result, it supports calculating the scattering transform by calling the predict and transform methods. By extension, it can be included as part of a scikit-learn Pipeline.

Given an input np.ndarray x of shape (B, T), where B is the number of signals to transform (the batch size) and T is the length of the signal, we compute its scattering transform by passing it to the scattering method (or calling the alias predict). Note that B can be one, in which case it may be omitted, giving an input of shape (T,).

Example

# Set the parameters of the scattering transform.
J = 6
T = 2 ** 13
Q = 8

# Generate a sample signal.
x = np.random.randn(np.prod(T))

# Define a Scattering1D object.
S = Scattering1D(J, T, Q)

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S.predict(x)

Above, the length of the signal is T = 2^{13} = 8192, while the maximum scale of the scattering transform is set to 2^J = 2^6 =
64. The time-frequency resolution of the first-order wavelets \psi_\lambda^{(1)}(t) is set to Q = 8 wavelets per octave. The second-order wavelets \psi_\mu^{(2)}(t) always have one wavelet per octave.

Parameters
  • J (int) – The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2^J.

  • shape (int) – The length of the input signals.

  • Q (int >= 1) – The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave). Defaults to 1.

  • max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

  • oversampling (integer >= 0, optional) – Controls the oversampling factor relative to the default as a power of two. Since the convolving by wavelets (or lowpass filters) and taking the modulus reduces the high-frequency content of the signal, we can subsample to save space and improve performance. However, this may reduce precision in the calculation. If this is not desirable, oversampling can be set to a large value to prevent too much subsampling. This parameter may be modified after object creation. Defaults to 0.

J

The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2 ** J.

Type

int

shape

The length of the input signals.

Type

int

Q

The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave).

Type

int

J_pad

The logarithm of the padded length of the signals.

Type

int

pad_left

The amount of padding to the left of the signal.

Type

int

pad_right

The amount of padding to the right of the signal.

Type

int

phi_f

A dictionary containing the lowpass filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.

Type

dictionary

psi1_f

A dictionary containing all the first-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.

Type

dictionary

psi2_f

A dictionary containing all the second-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.

Type

dictionary

max_order

The maximum scattering order of the transform.

Type

int

oversampling

The number of powers of two to oversample the output compared to the default subsampling rate determined from the filters.

Type

int

class kymatio.sklearn.Scattering2D(J, shape, L=8, max_order=2, pre_pad=False, backend='numpy', out_type='array')

Bases: kymatio.frontend.sklearn_frontend.ScatteringTransformerMixin, kymatio.numpy.ScatteringNumPy2D

The 2D scattering transform

The scattering transform computes two wavelet transform followed by modulus non-linearity. It can be summarized as

S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]

where

S_J^{(0)} x = x \star \phi_J,

S_J^{(1)} x = [|x \star \psi^{(1)}_\lambda| \star \phi_J]_\lambda, and

S_J^{(2)} x = [||x \star \psi^{(1)}_\lambda| \star
\psi^{(2)}_\mu| \star \phi_J]_{\lambda, \mu}.

where \star denotes the convolution (in space), \phi_J is a lowpass filter, \psi^{(1)}_\lambda is a family of bandpass filters and \psi^{(2)}_\mu is another family of bandpass filters. Only Morlet filters are used in this implementation. Convolutions are efficiently performed in the Fourier domain.

This class inherits from BaseEstimator and TransformerMixin in sklearn.base. As a result, it supports calculating the scattering transform by calling the predict and transform methods. By extension, it can be included as part of a scikit-learn Pipeline.

Example

# Set the parameters of the scattering transform.
J = 3
M, N = 32, 32

# Generate a sample signal.
x = np.random.randn(np.prod(M, N))

# Define a Scattering2D object.
S = Scattering2D(J, (M, N))

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S.predict(x)
Parameters
  • J (int) – Log-2 of the scattering scale.

  • shape (tuple of ints) – Spatial support (M, N) of the input

  • L (int, optional) – Number of angles used for the wavelet transform. Defaults to 8.

  • max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

  • pre_pad (boolean, optional) – Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally. Defaults to False.

  • backend (object, optional) – Controls the backend which is combined with the frontend.

J

Log-2 of the scattering scale.

Type

int

shape

Spatial support (M, N) of the input

Type

tuple of ints

L

Number of angles used for the wavelet transform.

Type

int, optional

max_order

The maximum order of scattering coefficients to compute. Must be either 1 or 2.

Type

int, optional

pre_pad

Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally.

Type

boolean

Psi

Contains the wavelets filters at all resolutions. See filter_bank.filter_bank for an exact description.

Type

dictionary

Phi

Contains the low-pass filters at all resolutions. See filter_bank.filter_bank for an exact description.

Type

dictionary

M_padded, N_padded

Spatial support of the padded input.

Type

int

Notes

The design of the filters is optimized for the value L = 8.

The pre_pad flag is particularly useful when cropping bigger images because this does not introduce border effects inherent to padding.

PyTorch

class kymatio.torch.HarmonicScattering3D(J, shape, L=3, sigma_0=1, max_order=2, rotation_covariant=True, method='integral', points=None, integral_powers=(0.5, 1.0, 2.0), backend='torch')

Bases: kymatio.frontend.torch_frontend.ScatteringTorch, kymatio.scattering3d.frontend.base_frontend.ScatteringBase3D

The 3D solid harmonic scattering transform

This class implements solid harmonic scattering on a 3D input image. For details see https://arxiv.org/abs/1805.00571.

This class inherits from torch.nn.Module. As a result, it has all the same capabilities, including transferring the object to the GPU using the cuda or to methods. This object would then take GPU tensors as input and output the scattering coefficients of those tensors.

Example

# Set the parameters of the scattering transform.
J = 3
M, N, O = 32, 32, 32

# Generate a sample signal.
x = torch.randn(M, N, O)

# Define a HarmonicScattering3D object.
S = HarmonicScattering3D(J, (M, N, O))

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S.forward(x)
Parameters
  • J (int) – Number of scales.

  • shape (tuple of ints) – Shape (M, N, O) of the input signal

  • L (int, optional) – Number of l values. Defaults to 3.

  • sigma_0 (float, optional) – Bandwidth of mother wavelet. Defaults to 1.

  • max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

  • rotation_covariant (bool, optional) –

    If set to True the first-order moduli take the form:

    \sqrt{\sum_m (x \star \psi_{j,l,m})^2)}

    if set to False the first-order moduli take the form:

    x \star \psi_{j,l,m}

    The second order moduli change analogously. Defaults to True.

  • method (string, optional) – Specifies the method for obtaining scattering coefficients. Currently, only ‘integral’ is available. Defaults to ‘integral’.

  • integral_powers (array-like) – List of exponents to the power of which moduli are raised before integration.

build()

Defines elementary routines.

This function should always call and create the filters via self.create_filters() defined below. For instance, via: self.filters = self.create_filters()

register_filters()

This function should be called after filters are generated, saving those arrays as module buffers.

scattering(input_array)

Apply the scattering transform

Parameters

input_array (torch.Tensor) – Input of size (batch_size, M, N, O).

Returns

output – If max_order is 1 it returns a torch.Tensor with the first-order scattering coefficients. If max_order is 2 it returns a torch.Tensor with the first- and second- order scattering coefficients, concatenated along the feature axis.

Return type

torch.Tensor

class kymatio.torch.Scattering1D(J, shape, Q=1, max_order=2, average=True, oversampling=0, vectorize=True, out_type='array', backend='torch')

Bases: kymatio.frontend.torch_frontend.ScatteringTorch, kymatio.scattering1d.frontend.base_frontend.ScatteringBase1D

The 1D scattering transform

The scattering transform computes a cascade of wavelet transforms alternated with a complex modulus non-linearity. The scattering transform of a 1D signal x(t) may be written as

S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]

where

S_J^{(0)} x(t) = x \star \phi_J(t),

S_J^{(1)} x(t, \lambda) = |x \star \psi_\lambda^{(1)}| \star \phi_J, and

S_J^{(2)} x(t, \lambda, \mu) = |\,| x \star \psi_\lambda^{(1)}| \star \psi_\mu^{(2)} | \star \phi_J.

In the above formulas, \star denotes convolution in time. The filters \psi_\lambda^{(1)}(t) and \psi_\mu^{(2)}(t) are analytic wavelets with center frequencies \lambda and \mu, while \phi_J(t) is a real lowpass filter centered at the zero frequency.

The Scattering1D class implements the 1D scattering transform for a given set of filters whose parameters are specified at initialization. While the wavelets are fixed, other parameters may be changed after the object is created, such as whether to compute all of S_J^{(0)} x, S_J^{(1)} x, and S_J^{(2)} x or just S_J^{(0)} x and S_J^{(1)} x.

This class inherits from torch.nn.Module. As a result, it has all the same capabilities, including transferring the object to the GPU using the cuda or to methods. This object would then take GPU tensors as input and output the scattering coefficients of those tensors.

Given an input torch.Tensor x of shape (B, T), where B is the number of signals to transform (the batch size) and T is the length of the signal, we compute its scattering transform by passing it to the scattering method (or calling the alias forward). Note that B can be one, in which case it may be omitted, giving an input of shape (T,).

# Set the parameters of the scattering transform.
J = 6
T = 2 ** 13
Q = 8

# Generate a sample signal.
x = torch.randn(T)

# Define a Scattering1D object.
S = Scattering1D(J, T, Q)

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S.forward(x)

Above, the length of the signal is T = 2^{13} = 8192, while the maximum scale of the scattering transform is set to 2^J = 2^6 =
64. The time-frequency resolution of the first-order wavelets \psi_\lambda^{(1)}(t) is set to Q = 8 wavelets per octave. The second-order wavelets \psi_\mu^{(2)}(t) always have one wavelet per octave.

Jint

The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2^J.

shapeint

The length of the input signals.

Qint >= 1

The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave). Defaults to 1.

max_orderint, optional

The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

averageboolean, optional

Determines whether the output is averaged in time or not. The averaged output corresponds to the standard scattering transform, while the un-averaged output skips the last convolution by \phi_J(t). This parameter may be modified after object creation. Defaults to True.

oversamplinginteger >= 0, optional

Controls the oversampling factor relative to the default as a power of two. Since the convolving by wavelets (or lowpass filters) and taking the modulus reduces the high-frequency content of the signal, we can subsample to save space and improve performance. However, this may reduce precision in the calculation. If this is not desirable, oversampling can be set to a large value to prevent too much subsampling. This parameter may be modified after object creation. Defaults to 0.

vectorizeboolean, optional

Determines wheter to return a vectorized scattering transform (that is, a large array containing the output) or a dictionary (where each entry corresponds to a separate scattering coefficient). This parameter may be modified after object creation. Deprecated in favor of out_type (see below). Defaults to True.

out_typestr, optional

The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering coefficient with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.

Jint

The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2 ** J.

shapeint

The length of the input signals.

Qint

The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave).

J_padint

The logarithm of the padded length of the signals.

pad_leftint

The amount of padding to the left of the signal.

pad_rightint

The amount of padding to the right of the signal.

phi_fdictionary

A dictionary containing the lowpass filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.

psi1_fdictionary

A dictionary containing all the first-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.

psi2_fdictionary

A dictionary containing all the second-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.

max_orderint

The maximum scattering order of the transform.

averageboolean

Controls whether the output should be averaged (the standard scattering transform) or not (resulting in wavelet modulus coefficients). Note that to obtain unaveraged output, the vectorize flag must be set to False or out_type must be set to ‘list’.

oversamplingint

The number of powers of two to oversample the output compared to the default subsampling rate determined from the filters.

vectorizeboolean

Controls whether the output should be vectorized into a single Tensor or collected into a dictionary. Deprecated in favor of out_type. For more details, see the documentation for scattering.

out_typestr

Specifices the output format of the transform, which is currently one of ‘array’ or ‘list’. If ‘array’, the output is a large array containing the scattering coefficients. If ‘list’, the output is a list of dictionaries, each containing a scattering coefficient along with meta information. For more information, see the documentation for scattering.

load_filters()

This function loads filters from the module’s buffer

register_filters()

This function run the filterbank function that will create the filters as numpy array, and then, it saves those arrays as module’s buffers.

scattering(x)

Apply the scattering transform

Given an input torch.Tensor of size (B, T), where B is the batch size (it can be potentially an integer or a shape) and T is the length of the individual signals, this function computes its scattering transform. If the vectorize flag is set to True (or if it is not available in this frontend), the output is in the form of a torch.Tensor or size (B, C, T1), where T1 is the signal length after subsampling to the scale 2^J (with the appropriate oversampling factor to reduce aliasing), and C is the number of scattering coefficients. If vectorize is set False, however, the output is a dictionary containing C keys, each a tuple whose length corresponds to the scattering order and whose elements are the sequence of filter indices used.

Note that the vectorize flag has been deprecated in favor of the out_type parameter. If this is set to ‘array’ (the default), the vectorize flag is still respected, but if not, out_type takes precedence. The two current output types are ‘array’ and ‘list’. The former gives the type of output described above. If set to ‘list’, however, the output is a list of dictionaries, each dictionary corresponding to a scattering coefficient and its associated meta information. The coefficient is stored under the ‘coef’ key, while other keys contain additional information, such as ‘j’ (the scale of the filter used) and ‘n’ (the filter index).

Furthermore, if the average flag is set to False, these outputs are not averaged, but are simply the wavelet modulus coefficients of the filters.

Parameters

x (torch.Tensor) – An input torch.Tensor of size (B, T).

Returns

S – If out_type is ‘array’ and the vectorize flag is True, the output is a torch.Tensor containing the scattering coefficients, while if vectorize is False, it is a dictionary indexed by tuples of filter indices. If out_type is ‘list’, the output is a list of dictionaries as described above.

Return type

tensor or dictionary

class kymatio.torch.Scattering2D(J, shape, L=8, max_order=2, pre_pad=False, backend='torch', out_type='array')

Bases: kymatio.frontend.torch_frontend.ScatteringTorch, kymatio.scattering2d.frontend.base_frontend.ScatteringBase2D

The 2D scattering transform

The scattering transform computes two wavelet transform followed by modulus non-linearity. It can be summarized as

S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]

where

S_J^{(0)} x = x \star \phi_J,

S_J^{(1)} x = [|x \star \psi^{(1)}_\lambda| \star \phi_J]_\lambda, and

S_J^{(2)} x = [||x \star \psi^{(1)}_\lambda| \star
\psi^{(2)}_\mu| \star \phi_J]_{\lambda, \mu}.

where \star denotes the convolution (in space), \phi_J is a lowpass filter, \psi^{(1)}_\lambda is a family of bandpass filters and \psi^{(2)}_\mu is another family of bandpass filters. Only Morlet filters are used in this implementation. Convolutions are efficiently performed in the Fourier domain.

This class inherits from torch.nn.Module. As a result, it has all the same capabilities, including transferring the object to the GPU using the cuda or to methods. This object would then take GPU tensors as input and output the scattering coefficients of those tensors.

Example

# Set the parameters of the scattering transform.
J = 3
M, N = 32, 32

# Generate a sample signal.
x = torch.randn(M, N)

# Define a Scattering2D object.
S = Scattering2D(J, (M, N))

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S.forward(x)
Parameters
  • J (int) – Log-2 of the scattering scale.

  • shape (tuple of ints) – Spatial support (M, N) of the input

  • L (int, optional) – Number of angles used for the wavelet transform. Defaults to 8.

  • max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

  • pre_pad (boolean, optional) – Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally. Defaults to False.

  • backend (object, optional) – Controls the backend which is combined with the frontend.

  • out_type (str, optional) – The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering path with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.

J

Log-2 of the scattering scale.

Type

int

shape

Spatial support (M, N) of the input

Type

tuple of ints

L

Number of angles used for the wavelet transform.

Type

int, optional

max_order

The maximum order of scattering coefficients to compute. Must be either 1 or 2.

Type

int, optional

pre_pad

Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally.

Type

boolean

Psi

Contains the wavelets filters at all resolutions. See filter_bank.filter_bank for an exact description.

Type

dictionary

Phi

Contains the low-pass filters at all resolutions. See filter_bank.filter_bank for an exact description.

Type

dictionary

M_padded, N_padded

Spatial support of the padded input.

Type

int

out_type

The format of the scattering output. See documentation for out_type parameter above and the documentation for scattering.

Type

str

Notes

The design of the filters is optimized for the value L = 8.

The pre_pad flag is particularly useful when cropping bigger images because this does not introduce border effects inherent to padding.

load_filters()

This function loads filters from the module’s buffers

register_filters()

This function run the filterbank function that will create the filters as numpy array, and then, it saves those arrays as module’s buffers.

scattering(input)

Apply the scattering transform

Parameters

input (torch.Tensor) – An input torch.Tensor of size (B, M, N).

Raises
  • RuntimeError – In the event that the input does not have at least two dimensions, or the tensor is not contiguous, or the tensor is not of the correct spatial size, padded or not.

  • TypeError – In the event that the input is not of type torch.Tensor.

Returns

S – Scattering transform of the input. If out_type is set to ‘array’ (or if it is not availabel for this frontend), this is a torch.Tensor of shape (B, C, M1, N1) where M1 = M // 2 ** J and N1 = N // 2 ** J. The C is the number of scattering channels calculated. If out_type is ‘list’, the output is a list of dictionaries, with each dictionary corresponding to a scattering coefficient and its meta information. The actual coefficient is contained in the ‘coef’ key, while other keys hold additional information, such as ‘j’ (the scale of the filter used), and ‘theta’ (the angle index of the filter used).

Return type

torch.Tensor

TensorFlow

class kymatio.tensorflow.HarmonicScattering3D(*args: Any, **kwargs: Any)

Bases: tensorflow.

The 3D solid harmonic scattering transform

This class implements solid harmonic scattering on a 3D input image. For details see https://arxiv.org/abs/1805.00571.

This class inherits from tf.Module. As a result, it has all the same capabilities as a standard TensorFlow Module.

Example

# Set the parameters of the scattering transform.
J = 3
M, N, O = 32, 32, 32

# Generate a sample signal.
x = np.random.randn(M, N, O)

# Define a HarmonicScattering3D object.
S = HarmonicScattering3D(J, (M, N, O))

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S(x)
Parameters
  • J (int) – Number of scales.

  • shape (tuple of ints) – Shape (M, N, O) of the input signal

  • L (int, optional) – Number of l values. Defaults to 3.

  • sigma_0 (float, optional) – Bandwidth of mother wavelet. Defaults to 1.

  • max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

  • rotation_covariant (bool, optional) –

    If set to True the first-order moduli take the form:

    \sqrt{\sum_m (x \star \psi_{j,l,m})^2)}

    if set to False the first-order moduli take the form:

    x \star \psi_{j,l,m}

    The second order moduli change analogously. Defaults to True.

  • method (string, optional) – Specifies the method for obtaining scattering coefficients. Currently, only ‘integral’ is available. Defaults to ‘integral’.

  • integral_powers (array-like) – List of exponents to the power of which moduli are raised before integration.

build()

Defines elementary routines.

This function should always call and create the filters via self.create_filters() defined below. For instance, via: self.filters = self.create_filters()

scattering(x)

Apply the scattering transform

Parameters

input_array (tf.Tensor) – Input of size (batch_size, M, N, O).

Returns

output – If max_order is 1 it returns a tf.Tensor with the first-order scattering coefficients. If max_order is 2 it returns a tf.Tensor with the first- and second- order scattering coefficients, concatenated along the feature axis.

Return type

tf.Tensor

class kymatio.tensorflow.Scattering1D(*args: Any, **kwargs: Any)

Bases: tensorflow.

The 1D scattering transform

The scattering transform computes a cascade of wavelet transforms alternated with a complex modulus non-linearity. The scattering transform of a 1D signal x(t) may be written as

S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]

where

S_J^{(0)} x(t) = x \star \phi_J(t),

S_J^{(1)} x(t, \lambda) = |x \star \psi_\lambda^{(1)}| \star \phi_J, and

S_J^{(2)} x(t, \lambda, \mu) = |\,| x \star \psi_\lambda^{(1)}| \star \psi_\mu^{(2)} | \star \phi_J.

In the above formulas, \star denotes convolution in time. The filters \psi_\lambda^{(1)}(t) and \psi_\mu^{(2)}(t) are analytic wavelets with center frequencies \lambda and \mu, while \phi_J(t) is a real lowpass filter centered at the zero frequency.

The Scattering1D class implements the 1D scattering transform for a given set of filters whose parameters are specified at initialization. While the wavelets are fixed, other parameters may be changed after the object is created, such as whether to compute all of S_J^{(0)} x, S_J^{(1)} x, and S_J^{(2)} x or just S_J^{(0)} x and S_J^{(1)} x.

This class inherits from tf.Module. As a result, it has all the same capabilities as a standard TensorFlow Module.

Given an input tf.Tensor x of shape (B, T), where B is the number of signals to transform (the batch size) and T is the length of the signal, we compute its scattering transform by passing it to the scattering method (or calling the alias __call__). Note that B can be one, in which case it may be omitted, giving an input of shape (T,).

# Set the parameters of the scattering transform.
J = 6
T = 2 ** 13
Q = 8

# Generate a sample signal.
x = np.random.randn(T)

# Define a Scattering1D object.
S = Scattering1D(J, T, Q)

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S(x)

Above, the length of the signal is T = 2^{13} = 8192, while the maximum scale of the scattering transform is set to 2^J = 2^6 =
64. The time-frequency resolution of the first-order wavelets \psi_\lambda^{(1)}(t) is set to Q = 8 wavelets per octave. The second-order wavelets \psi_\mu^{(2)}(t) always have one wavelet per octave.

Jint

The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2^J.

shapeint

The length of the input signals.

Qint >= 1

The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave). Defaults to 1.

max_orderint, optional

The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

averageboolean, optional

Determines whether the output is averaged in time or not. The averaged output corresponds to the standard scattering transform, while the un-averaged output skips the last convolution by \phi_J(t). This parameter may be modified after object creation. Defaults to True.

oversamplinginteger >= 0, optional

Controls the oversampling factor relative to the default as a power of two. Since the convolving by wavelets (or lowpass filters) and taking the modulus reduces the high-frequency content of the signal, we can subsample to save space and improve performance. However, this may reduce precision in the calculation. If this is not desirable, oversampling can be set to a large value to prevent too much subsampling. This parameter may be modified after object creation. Defaults to 0.

vectorizeboolean, optional

Determines wheter to return a vectorized scattering transform (that is, a large array containing the output) or a dictionary (where each entry corresponds to a separate scattering coefficient). This parameter may be modified after object creation. Deprecated in favor of out_type (see below). Defaults to True.

out_typestr, optional

The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering coefficient with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.

Jint

The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2 ** J.

shapeint

The length of the input signals.

Qint

The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave).

J_padint

The logarithm of the padded length of the signals.

pad_leftint

The amount of padding to the left of the signal.

pad_rightint

The amount of padding to the right of the signal.

phi_fdictionary

A dictionary containing the lowpass filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.

psi1_fdictionary

A dictionary containing all the first-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.

psi2_fdictionary

A dictionary containing all the second-order wavelet filters, each represented as a dictionary containing that filter at all resolutions. See filter_bank.scattering_filter_factory for an exact description.

max_orderint

The maximum scattering order of the transform.

averageboolean

Controls whether the output should be averaged (the standard scattering transform) or not (resulting in wavelet modulus coefficients). Note that to obtain unaveraged output, the vectorize flag must be set to False or out_type must be set to ‘list’.

oversamplingint

The number of powers of two to oversample the output compared to the default subsampling rate determined from the filters.

vectorizeboolean

Controls whether the output should be vectorized into a single Tensor or collected into a dictionary. Deprecated in favor of out_type. For more details, see the documentation for scattering.

out_typestr

Specifices the output format of the transform, which is currently one of ‘array’ or ‘list’. If ‘array’, the output is a large array containing the scattering coefficients. If ‘list’, the output is a list of dictionaries, each containing a scattering coefficient along with meta information. For more information, see the documentation for scattering.

scattering(x)

Apply the scattering transform

Given an input tf.Tensor of size (B, T), where B is the batch size (it can be potentially an integer or a shape) and T is the length of the individual signals, this function computes its scattering transform. If the vectorize flag is set to True (or if it is not available in this frontend), the output is in the form of a tf.Tensor or size (B, C, T1), where T1 is the signal length after subsampling to the scale 2^J (with the appropriate oversampling factor to reduce aliasing), and C is the number of scattering coefficients. If vectorize is set False, however, the output is a dictionary containing C keys, each a tuple whose length corresponds to the scattering order and whose elements are the sequence of filter indices used.

Note that the vectorize flag has been deprecated in favor of the out_type parameter. If this is set to ‘array’ (the default), the vectorize flag is still respected, but if not, out_type takes precedence. The two current output types are ‘array’ and ‘list’. The former gives the type of output described above. If set to ‘list’, however, the output is a list of dictionaries, each dictionary corresponding to a scattering coefficient and its associated meta information. The coefficient is stored under the ‘coef’ key, while other keys contain additional information, such as ‘j’ (the scale of the filter used) and ‘n’ (the filter index).

Furthermore, if the average flag is set to False, these outputs are not averaged, but are simply the wavelet modulus coefficients of the filters.

Parameters

x (tf.Tensor) – An input tf.Tensor of size (B, T).

Returns

S – If out_type is ‘array’ and the vectorize flag is True, the output is a tf.Tensor containing the scattering coefficients, while if vectorize is False, it is a dictionary indexed by tuples of filter indices. If out_type is ‘list’, the output is a list of dictionaries as described above.

Return type

tensor or dictionary

class kymatio.tensorflow.Scattering2D(*args: Any, **kwargs: Any)

Bases: tensorflow.

The 2D scattering transform

The scattering transform computes two wavelet transform followed by modulus non-linearity. It can be summarized as

S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]

where

S_J^{(0)} x = x \star \phi_J,

S_J^{(1)} x = [|x \star \psi^{(1)}_\lambda| \star \phi_J]_\lambda, and

S_J^{(2)} x = [||x \star \psi^{(1)}_\lambda| \star
\psi^{(2)}_\mu| \star \phi_J]_{\lambda, \mu}.

where \star denotes the convolution (in space), \phi_J is a lowpass filter, \psi^{(1)}_\lambda is a family of bandpass filters and \psi^{(2)}_\mu is another family of bandpass filters. Only Morlet filters are used in this implementation. Convolutions are efficiently performed in the Fourier domain.

This class inherits from tf.Module. As a result, it has all the same capabilities as a standard TensorFlow Module.

Example

# Set the parameters of the scattering transform.
J = 3
M, N = 32, 32

# Generate a sample signal.
x = np.random.randn(M, N)

# Define a Scattering2D object.
S = Scattering2D(J, (M, N))

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S(x)
Parameters
  • J (int) – Log-2 of the scattering scale.

  • shape (tuple of ints) – Spatial support (M, N) of the input

  • L (int, optional) – Number of angles used for the wavelet transform. Defaults to 8.

  • max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

  • pre_pad (boolean, optional) – Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally. Defaults to False.

  • backend (object, optional) – Controls the backend which is combined with the frontend.

  • out_type (str, optional) – The format of the output of a scattering transform. If set to ‘list’, then the output is a list containing each individual scattering path with meta information. Otherwise, if set to ‘array’, the output is a large array containing the concatenation of all scattering coefficients. Defaults to ‘array’.

J

Log-2 of the scattering scale.

Type

int

shape

Spatial support (M, N) of the input

Type

tuple of ints

L

Number of angles used for the wavelet transform.

Type

int, optional

max_order

The maximum order of scattering coefficients to compute. Must be either 1 or 2.

Type

int, optional

pre_pad

Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally.

Type

boolean

Psi

Contains the wavelets filters at all resolutions. See filter_bank.filter_bank for an exact description.

Type

dictionary

Phi

Contains the low-pass filters at all resolutions. See filter_bank.filter_bank for an exact description.

Type

dictionary

M_padded, N_padded

Spatial support of the padded input.

Type

int

out_type

The format of the scattering output. See documentation for out_type parameter above and the documentation for scattering.

Type

str

Notes

The design of the filters is optimized for the value L = 8.

The pre_pad flag is particularly useful when cropping bigger images because this does not introduce border effects inherent to padding.

scattering(input)

Apply the scattering transform

Parameters

input (tf.Tensor) – An input tf.Tensor of size (B, M, N).

Raises
  • RuntimeError – In the event that the input does not have at least two dimensions, or the tensor is not contiguous, or the tensor is not of the correct spatial size, padded or not.

  • TypeError – In the event that the input is not of type tf.Tensor.

Returns

S – Scattering transform of the input. If out_type is set to ‘array’ (or if it is not availabel for this frontend), this is a tf.Tensor of shape (B, C, M1, N1) where M1 = M // 2 ** J and N1 = N // 2 ** J. The C is the number of scattering channels calculated. If out_type is ‘list’, the output is a list of dictionaries, with each dictionary corresponding to a scattering coefficient and its meta information. The actual coefficient is contained in the ‘coef’ key, while other keys hold additional information, such as ‘j’ (the scale of the filter used), and ‘theta’ (the angle index of the filter used).

Return type

tf.Tensor

Keras

class kymatio.keras.Scattering1D(*args: Any, **kwargs: Any)

Bases: tensorflow.keras.layers.

The 1D scattering transform

The scattering transform computes a cascade of wavelet transforms alternated with a complex modulus non-linearity. The scattering transform of a 1D signal x(t) may be written as

S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]

where

S_J^{(0)} x(t) = x \star \phi_J(t),

S_J^{(1)} x(t, \lambda) = |x \star \psi_\lambda^{(1)}| \star \phi_J, and

S_J^{(2)} x(t, \lambda, \mu) = |\,| x \star \psi_\lambda^{(1)}| \star \psi_\mu^{(2)} | \star \phi_J.

In the above formulas, \star denotes convolution in time. The filters \psi_\lambda^{(1)}(t) and \psi_\mu^{(2)}(t) are analytic wavelets with center frequencies \lambda and \mu, while \phi_J(t) is a real lowpass filter centered at the zero frequency.

The Scattering1D class implements the 1D scattering transform for a given set of filters whose parameters are specified at initialization. While the wavelets are fixed, other parameters may be changed after the object is created, such as whether to compute all of S_J^{(0)} x, S_J^{(1)} x, and S_J^{(2)} x or just S_J^{(0)} x and S_J^{(1)} x.

This class inherits from tf.keras.layers.Layer. As a result, it has all the same capabilities as a standard Keras Layer.

Given an input tf.Tensor x of shape (B, T), where B is the number of signals to transform (the batch size) and T is the length of the signal, we compute its scattering transform by passing it to the scattering method (or calling the alias call). Note that B can be one, in which case it may be omitted, giving an input of shape (T,).

Example

# Set the parameters of the scattering transform.
J = 6
T = 2 ** 13
Q = 8

# Generate a sample signal.
x = np.random.randn(T)

# Define a Scattering1D object.
S = Scattering1D(J, Q)

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S(x)

Above, the length of the signal is T = 2^{13} = 8192, while the maximum scale of the scattering transform is set to 2^J = 2^6 =
64. The time-frequency resolution of the first-order wavelets \psi_\lambda^{(1)}(t) is set to Q = 8 wavelets per octave. The second-order wavelets \psi_\mu^{(2)}(t) always have one wavelet per octave.

Parameters
  • J (int) – The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2^J.

  • Q (int >= 1) – The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave). Defaults to 1.

  • max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

  • oversampling (integer >= 0, optional) – Controls the oversampling factor relative to the default as a power of two. Since the convolving by wavelets (or lowpass filters) and taking the modulus reduces the high-frequency content of the signal, we can subsample to save space and improve performance. However, this may reduce precision in the calculation. If this is not desirable, oversampling can be set to a large value to prevent too much subsampling. This parameter may be modified after object creation. Defaults to 0.

J

The maximum log-scale of the scattering transform. In other words, the maximum scale is given by 2 ** J.

Type

int

Q

The number of first-order wavelets per octave (second-order wavelets are fixed to one wavelet per octave).

Type

int

max_order

The maximum scattering order of the transform.

Type

int

oversampling

The number of powers of two to oversample the output compared to the default subsampling rate determined from the filters.

Type

int

build(input_shape)

Set up padding and filters

Certain internal data, such as the amount of padding and the wavelet filters to be used in the scattering transform, need to be computed from the parameters given during construction. This function is called automatically during object creation and no subsequent calls are therefore needed.

class kymatio.keras.Scattering2D(*args: Any, **kwargs: Any)

Bases: tensorflow.keras.layers.

The 2D scattering transform

The scattering transform computes two wavelet transform followed by modulus non-linearity. It can be summarized as

S_J x = [S_J^{(0)} x, S_J^{(1)} x, S_J^{(2)} x]

where

S_J^{(0)} x = x \star \phi_J,

S_J^{(1)} x = [|x \star \psi^{(1)}_\lambda| \star \phi_J]_\lambda, and

S_J^{(2)} x = [||x \star \psi^{(1)}_\lambda| \star
\psi^{(2)}_\mu| \star \phi_J]_{\lambda, \mu}.

where \star denotes the convolution (in space), \phi_J is a lowpass filter, \psi^{(1)}_\lambda is a family of bandpass filters and \psi^{(2)}_\mu is another family of bandpass filters. Only Morlet filters are used in this implementation. Convolutions are efficiently performed in the Fourier domain.

This class inherits from tf.keras.layers.Layer. As a result, it has all the same capabilities as a standard Keras Layer.

Example

# Set the parameters of the scattering transform.
J = 3
M, N = 32, 32

# Generate a sample signal.
x = np.random.randn(M, N)

# Define a Scattering2D object.
S = Scattering2D(J)

# Calculate the scattering transform.
Sx = S.scattering(x)

# Equivalently, use the alias.
Sx = S(x)
Parameters
  • J (int) – Log-2 of the scattering scale.

  • L (int, optional) – Number of angles used for the wavelet transform. Defaults to 8.

  • max_order (int, optional) – The maximum order of scattering coefficients to compute. Must be either 1 or 2. Defaults to 2.

  • pre_pad (boolean, optional) – Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally. Defaults to False.

  • backend (object, optional) – Controls the backend which is combined with the frontend.

J

Log-2 of the scattering scale.

Type

int

L

Number of angles used for the wavelet transform.

Type

int, optional

max_order

The maximum order of scattering coefficients to compute. Must be either 1 or 2.

Type

int, optional

pre_pad

Controls the padding: if set to False, a symmetric padding is applied on the signal. If set to True, the software will assume the signal was padded externally.

Type

boolean

Notes

The design of the filters is optimized for the value L = 8.

The pre_pad flag is particularly useful when cropping bigger images because this does not introduce border effects inherent to padding.

build(input_shape)

Defines elementary routines.

This function should always call and create the filters via self.create_filters() defined below. For instance, via: self.filters = self.create_filters()