> For the complete documentation index, see [llms.txt](https://docs.e6data.com/ingestion-engine/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.e6data.com/ingestion-engine/architecture/watermarks.md). # Watermarks e6 Ingestion Engine processes time-based queries using **event time**: the timestamp carried by a record, rather than the wall-clock time at which the engine happens to process it. A watermark is the pipeline's declaration of event-time progress. When the watermark passes the end of a window, e6 Ingestion Engine can close that window, emit its result, and eventually release state that is no longer needed. ## Event-time assignment Declare event time with `WATERMARK FOR` in the source table: ```sql CREATE TABLE events ( event_id BIGINT, event_time TIMESTAMP NOT NULL, WATERMARK FOR event_time AS event_time - INTERVAL '5 seconds' ); ``` The selected field must be a timestamp, and the watermark expression must produce a non-null timestamp. e6 Ingestion Engine uses the selected field as the record's event time. See the [SQL DDL reference](/ingestion-engine/sql/sql-reference/ddl.md) for the complete `CREATE TABLE` form. If a table does not declare `WATERMARK FOR`, e6 Ingestion Engine uses the source-provided `_timestamp`, such as a Kafka record timestamp. Choose a field whose values reflect when events occurred. Processing-time fields can produce misleading results during source delays or replays. ## Generating watermarks Sources periodically derive a watermark from the timestamps they have observed. An emitted watermark is a progress boundary: downstream operators may treat an older event as late. Configure watermark expressions so progress does not move backward for a source partition. Without an explicit `WATERMARK FOR ... AS ...` expression, e6 Ingestion Engine uses `_timestamp - 1 second`. A custom expression controls the allowed out-of-order delay before a record is considered late. The subtraction in the watermark expression is an intentional tradeoff: * A smaller delay closes windows sooner but tolerates less disorder. * A larger delay accepts later events but increases result latency and state retention. For example, `_timestamp - 30 seconds` allows approximately 30 seconds of event-time disorder while delaying window completion by the same amount. ## Propagation through a pipeline ![Watermark propagation](https://1725648790-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FuztQ31u5OyLCkPgUTosp%2Fuploads%2Fgit-blob-6896035094bd5343688a68f8d4097f8308f0a216%2Fwatermark-diagrams-watermark-propagation.svg?alt=media) Most transformations preserve the watermark. Time-aware operators use it to close windows, expire join state, or finish sessions. For an operator with multiple inputs, progress is limited by the slowest input: | Input state | Combined result | | -------------------------------------- | ------------------------------------ | | Every input has reported event time | Minimum reported timestamp | | Any input has not reported a watermark | No combined watermark is emitted yet | Using the minimum prevents an operator from closing time-based state while another input may still provide earlier records. ### Quiet inputs The current public SQL configuration does not provide an idle-input timeout. A quiet partition or join input can therefore hold back watermark progress for a multi-input operator. Design source partitioning so every active input can make progress, and monitor inputs that stop producing timestamps unexpectedly. ## Windows and sessions Watermarks determine when event-time results become eligible for emission: * **Tumbling windows** close after the watermark passes the fixed window end. * **Sliding windows** emit each completed slide whose covered interval is behind the watermark. * **Session windows** close after the watermark passes the configured inactivity gap for a key. * **SQL window functions** can emit completed event-time buckets once their boundary has passed. The watermark controls completeness, not the amount of data in a window. A high-volume and a low-volume window with the same time boundary close at the same event-time point. ## Late events A record with an event timestamp earlier than the current watermark is late. e6 Ingestion Engine does not provide a universal grace period or late-event side output, and late records are not guaranteed to update results that have already been finalized. To tolerate more out-of-order arrival, configure a larger delay in the watermark expression. This makes the watermark policy part of the pipeline's data-quality contract. Choose it using observed source delay rather than assuming records arrive in order. ## State retention Watermarks also bound time-based state. Conceptually: `expiration boundary = watermark - retention` Records older than that boundary can be removed from joins, windows, and other retained state. Longer retention accommodates more delayed matches but increases checkpoint size, storage use, and recovery time. Retention must be at least as long as the time relationship the query needs to preserve. ## Checkpoints and recovery Completed checkpoints preserve event-time progress together with operator state. On recovery, e6 Ingestion Engine restores both from the same checkpoint so windows and retention decisions remain consistent with restored data. Records replayed after the checkpoint are evaluated against the restored watermark policy in the same way as newly arriving records. ## Operational guidance Watch for: * Growing difference between source event time and the current watermark * A partition or join input that stops advancing its watermark * Records arriving behind the current watermark * Increasing state size or checkpoint duration * A single input holding back multi-input operators These signals usually indicate source skew, a quiet input, or a watermark delay that does not match real arrival patterns. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.e6data.com/ingestion-engine/architecture/watermarks.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.