pyspark.sql.functions.timestamp_add#

pyspark.sql.functions.timestamp_add(unit, quantity, ts)[source]#

Gets the difference between the timestamps in the specified units by truncating the fraction part.

New in version 4.0.0.

Parameters

unitliteral string: This indicates the units of the difference between the given timestamps. Supported options are (case insensitive): “YEAR”, “QUARTER”, “MONTH”, “WEEK”, “DAY”, “HOUR”, “MINUTE”, “SECOND”, “MILLISECOND” and “MICROSECOND”.
quantityColumn or column name: The number of units of time that you want to add.
tsColumn or column name: A timestamp to which you want to add.

Returns

Column: the difference between the timestamps.

See also

pyspark.sql.functions.dateadd()
pyspark.sql.functions.date_add()

Examples

>>> import datetime
>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame(
...     [(datetime.datetime(2016, 3, 11, 9, 0, 7), 2),
...      (datetime.datetime(2024, 4, 2, 9, 0, 7), 3)], ['ts', 'quantity'])
>>> df.select('*', sf.timestamp_add('year', 'quantity', 'ts')).show()
+-------------------+--------+--------------------------------+
|                 ts|quantity|timestampadd(year, quantity, ts)|
+-------------------+--------+--------------------------------+
|2016-03-11 09:00:07|       2|             2018-03-11 09:00:07|
|2024-04-02 09:00:07|       3|             2027-04-02 09:00:07|
+-------------------+--------+--------------------------------+

>>> df.select('*', sf.timestamp_add('WEEK', sf.lit(5), df.ts)).show()
+-------------------+--------+-------------------------+
|                 ts|quantity|timestampadd(WEEK, 5, ts)|
+-------------------+--------+-------------------------+
|2016-03-11 09:00:07|       2|      2016-04-15 09:00:07|
|2024-04-02 09:00:07|       3|      2024-05-07 09:00:07|
+-------------------+--------+-------------------------+

>>> df.select('*', sf.timestamp_add('day', sf.lit(-5), 'ts')).show()
+-------------------+--------+-------------------------+
|                 ts|quantity|timestampadd(day, -5, ts)|
+-------------------+--------+-------------------------+
|2016-03-11 09:00:07|       2|      2016-03-06 09:00:07|
|2024-04-02 09:00:07|       3|      2024-03-28 09:00:07|
+-------------------+--------+-------------------------+