Diff / List Layer

Calculate the difference between consecutive elements in numeric lists, shifted by a specified number of positions. Similar to Polars' list.diff() or NumPy's diff(). Returns a list column containing the differences between elements.

Example transformation:

With n=1 (default), null_behavior=Ignore:

lists	diff
[1, 2, 3, 4]	[null, 1, 1, 1]
[10, 2, 1]	[null, -8, -1]
[5]	[null]
[]	[]

With n=2, null_behavior=Ignore:

lists	diff
[1, 2, 3, 4]	[null, null, 2, 2]
[10, 2, 1]	[null, null, -9]

With n=2, null_behavior=Drop:

lists	diff
[1, 2, 3, 4]	[2, 2]
[10, 2, 1]	[-9]

Common applications:

Analyzing price changes in financial data
Calculating velocity from position data
Detecting trends in time series
Measuring rate of change in measurements
Computing gradients in numerical analysis
Analyzing sequential differences in any numeric sequence

Note: Works with numeric lists of any length. First n elements are null (or dropped based on NullBehavior). Empty lists remain empty. Useful for analyzing changes or trends in sequential data.

Table

Select

column

The numeric list column to compute differences. Common patterns:

Price series: [100, 102, 97, 105]
Measurements: [1.2, 1.5, 1.8, 2.2]
Coordinates: [-1, 0, 2, 5]
Cumulative values: [10, 25, 45, 70] Lists can have different lengths. Only numeric types supported.

N

i32

Number of positions to shift (N) when calculating differences. Must be >= 1. Examples:

1: Adjacent differences (x[1]-x[0], x[2]-x[1], ...)
2: Skip-one differences (x[2]-x[0], x[3]-x[1], ...)
3: Skip-two differences (x[3]-x[0], x[4]-x[1], ...)

NullBehavior

enum

Ignore

Ignore ~

Keep null values in the result. First n elements will be null. Default behavior.

Drop ~

Remove null values from the result. Output list will be shorter by n elements.

AsColumn

name

Name for the new column. If not provided, the system generates a unique name. If AsColumn matches an existing column, the existing column is replaced. The name should follow valid column naming conventions.