Integrations · LessonBy Praveen T, ServiceNow Trainer, 9 yrs · Published · ServiceNow · all levels
Data stream actions
Process large paginated API responses in a flow without running out of memory.
Quick answer
Process large paginated API responses in a flow without running out of memory.
Key takeaways
- Data streams paginate large responses safely
- Use For Each Item to process records one at a time
- Filter at the source rather than in the flow
- Make runs restartable and log volumes
The problem it solves
A REST action returns one payload. When an endpoint has fifty thousand rows across two hundred pages, you need something that streams pages and hands records to the flow one at a time. That is a data stream action.
How it works
The action defines the request, the pagination strategy and the output structure. The flow then uses For Each Item on the stream, processing records without holding everything in memory.
Design notes
Streams still cost time and transactions.
- Filter at the source, never stream everything and discard in the flow
- Handle partial failures, a stream that dies at page 140 should be restartable
- Batch writes where possible instead of one update per record
- Log page counts and record totals so runs are comparable
Practice challenge
+0 XPStreak ×0
Question 1 of 2
What consumes the output of a data stream action?
Frequently asked questions
Can we just increase the REST timeout?
No. Large payloads exhaust memory regardless of timeout. Streaming is the supported pattern.
Does it work with SOAP?
Data stream actions target REST style paginated APIs. For SOAP, page manually and process in batches.