Amazon S3 · Arazzo Workflow

Amazon S3 Paginate Through Bucket Objects

Version 1.0.0

List a first page of objects then fetch the next page by continuation token.

1 workflow 1 source API 1 provider
View Spec View on GitHub ArchiveBackupCloud StorageData StorageObject StorageScalable StorageArazzoWorkflows

Provider

amazon-s3

Workflows

paginate-list-objects
List a first page of objects then the next page via continuation token.
Lists a capped first page and, when truncated, fetches the next page using the NextContinuationToken from the first response.
2 steps inputs: bucket, pageSize, prefix outputs: firstKeyCount, nextToken, secondKeyCount
1
listFirstPage
ListObjectsV2
List the first page of objects, capped to the requested page size.
2
listNextPage
ListObjectsV2
Fetch the next page of objects using the continuation token returned by the first page.

Source API Descriptions

Arazzo Workflow Specification

amazon-s3-paginate-list-objects-workflow.yml Raw ↑
arazzo: 1.0.1
info:
  title: Amazon S3 Paginate Through Bucket Objects
  summary: List a first page of objects then fetch the next page by continuation token.
  description: >-
    A pagination flow for buckets that hold more keys than fit in a single
    response. The workflow lists a first capped page of objects and, when the
    response is truncated, follows the NextContinuationToken to fetch the second
    page. ListObjectsV2 returns at most 1,000 keys per call, so large buckets are
    walked one page at a time. Every step spells out its request inline so the
    flow can be read and executed without opening the underlying OpenAPI
    description.
  version: 1.0.0
sourceDescriptions:
- name: s3RestApi
  url: ../openapi/amazon-s3-rest-api-openapi.yml
  type: openapi
workflows:
- workflowId: paginate-list-objects
  summary: List a first page of objects then the next page via continuation token.
  description: >-
    Lists a capped first page and, when truncated, fetches the next page using
    the NextContinuationToken from the first response.
  inputs:
    type: object
    required:
    - bucket
    - pageSize
    properties:
      bucket:
        type: string
        description: The bucket to list.
      pageSize:
        type: integer
        description: The maximum number of keys to return per page (max-keys), 1 to 1000.
      prefix:
        type: string
        description: An optional key prefix to scope the listing.
  steps:
  - stepId: listFirstPage
    description: >-
      List the first page of objects, capped to the requested page size.
    operationId: ListObjectsV2
    parameters:
    - name: Bucket
      in: path
      value: $inputs.bucket
    - name: list-type
      in: query
      value: 2
    - name: max-keys
      in: query
      value: $inputs.pageSize
    - name: prefix
      in: query
      value: $inputs.prefix
    successCriteria:
    - condition: $statusCode == 200
    outputs:
      isTruncated: $response.body#/IsTruncated
      nextToken: $response.body#/NextContinuationToken
      firstKeyCount: $response.body#/KeyCount
    onSuccess:
    - name: morePages
      type: goto
      stepId: listNextPage
      criteria:
      - context: $response.body
        condition: $.IsTruncated == true
        type: jsonpath
    - name: lastPage
      type: end
      criteria:
      - context: $response.body
        condition: $.IsTruncated == false
        type: jsonpath
  - stepId: listNextPage
    description: >-
      Fetch the next page of objects using the continuation token returned by the
      first page.
    operationId: ListObjectsV2
    parameters:
    - name: Bucket
      in: path
      value: $inputs.bucket
    - name: list-type
      in: query
      value: 2
    - name: max-keys
      in: query
      value: $inputs.pageSize
    - name: prefix
      in: query
      value: $inputs.prefix
    - name: continuation-token
      in: query
      value: $steps.listFirstPage.outputs.nextToken
    successCriteria:
    - condition: $statusCode == 200
    outputs:
      isTruncated: $response.body#/IsTruncated
      nextToken: $response.body#/NextContinuationToken
      secondKeyCount: $response.body#/KeyCount
  outputs:
    firstKeyCount: $steps.listFirstPage.outputs.firstKeyCount
    secondKeyCount: $steps.listNextPage.outputs.secondKeyCount
    nextToken: $steps.listNextPage.outputs.nextToken