Skip to content

Scheduling a Bulk Export

Bulk Exports can be scheduled to:

  • Start immediately or at a specific future date and time,
  • Run only once or Recur on a fixed schedule, and
  • End the recurrence at a specific future date and time.

For example, scheduling allows all of the following scenarios and more:

  1. Schedule a one-time export to begin next Tuesday at 5 AM.
  2. Schedule a recurring export to:
    1. Begin immediately and run every day at 5 PM indefinitely,
    2. Begin immediately, run once each week on Wednesdays at 3 AM, and stop after 3 months,
    3. Begin in 3 months on Tuesday at 12 PM, run every Friday at 9 PM, and stop after 1 year.

These are just a few examples to demonstrate the flexibility of the scheduling system.

Availability

Scheduling of Bulk Exports is available for all customer sites. However, schedules currently can only be created and managed using the new (BETA) CLI, not the Web App.

Additionally, scheduling is not supported when exporting data to "My Computer".

Scheduling to My Computer

An alternative option for scheduling recurring exports from a local machine ("My Computer") is to use other, 3rd-party tools (e.g., crontab on Linux) to schedule execution of the Flywheel CLI.

For example, a crontab file similar to the following can be used to schedule the Flywheel CLI to run on a recurring schedule.

30 5 * * FRI fw-beta export run -p 'fw://demo/ACME Trial' -s /mnt/data/

In this example, the cron system in Linux will invoke the Flywheel CLI at 05:30 (local) every Friday to run a new Bulk Export from project fw://demo/ACME Trial to /mnt/data.

Create a Schedule

Setting up a schedule for Bulk Export is similar to running a one-time export but with just a few extra parameters. To setup a new scheduled export use the export schedule create command instead of the export run command.

1
fw-beta export schedule create -p <project> -s <storage> --start <start-date-time> --end <end-date-time> --cron <cron-string>

Where:

  • <start-date-time>: Is the date and time when the schedule should begin in ISO 8601 format.
    • E.g., --start '2025-09-03T06:40:00+00:00' means "begin on September 3, 2025 at 06:40 UTC+0".
    • The export job will not run prior to this date and time, regardless of the recurrence setting.
  • <end-date-time>: Is the date and time when the schedule should terminate in ISO 8601 format.
    • E.g., --end '2026-03-12T14:15:00+00:00' means "end after March 3, 2026 at 14:15 UTC+0".
    • The export job will not start again after this date and time, regardless of the recurrence setting.
  • <cron-string>: Is a cron-style pattern defining when the export job should recur.
    • E.g., --cron '30 5 * * FRI' means "at 05:30 every Friday"

For example, consider this command:

1
2
fw-beta export schedule create -p 'fw://demo/ACME Trial' -s 682e21f7487fc46c72718887 \
--start '2025-09-03T06:40:00+00:00' --end '2026-03-12T14:15:00+00:00' --cron '30 5 * * FRI'

This command will set up a schedule that launches a new export every Friday at 05:30 (UTC+0) from project fw://demo/ACME Trial into storage 682e21f7487fc46c72718887, where the first occurrence begins after September 3, 2025 at 06:40 UTC+0 and the last occurrence begins before March 3, 2026 at 14:15 UTC+0.

For more information, see the new CLI documentation for export schedule create.

Modify a Schedule

To modify an existing Export Schedule use the export schedule update command. This command works the same as the export schedule create command for creating a new schedule, but only the timing-related options can be modified (i.e., --start, --end, --cron).

1
fw-beta export schedule update <ID> --start <start-date-time> --end <end-date-time> --cron <cron-string>

Where <id> is replaced with the ID of the Export Schedule to be modified. If you do not know the Export Schedule ID, see the article locating the ID of an Export Schedule.

Note

Only the timing-related options can be modified for an existing Export Schedule (i.e., --start, --end, --cron). If the export-related options need to be modified (e.g., -p, -s, --mapping, etc.), a new schedule will need to be created instead.

Delete a Schedule

To delete an existing Export Schedule use the export schedule delete command.

1
fw-beta export schedule delete <ID>

Where <id> is replaced with the ID of the Export Schedule to be modified. If you do not know the Export Schedule ID, see the article locating the ID of an Export Schedule.

Locating the ID of an Export Schedule

To locate the Export Schedule ID, use the export schedule list command to list all the existing Export Schedules on the site:

1
fw-beta export schedule list

This output is paginated, and only the 20 most recent export schedules will returned by default. To view the next "page" of export schedules, take the ID of the last schedule in the list, then run the following command:

1
fw-beta export schedule list --after-id <id>

Where <id> is replaced with the ID of the last schedule in the list.

Repeat this process to continue stepping through each "page" of export schedules.

Tip

There are additional options for filtering, sorting, and changing the length of the list to help locate a particular export schedule. These options can be listed using the following command:

1
fw-beta export schedule list -h

For more information, see the new CLI documentation for export schedule list.

References

  1. New (BETA) CLI Docs on export schedule