Skip to content

Scheduling a Bulk Import

Bulk Imports can be scheduled to:

  • Start immediately or at a specific future date and time,
  • Run only once or Recur on a fixed schedule, and
  • End the recurrence at a specific future date and time.

For example, scheduling allows all of the following scenarios and more:

  1. Schedule a one-time import to begin next Tuesday at 5 AM.
  2. Schedule a recurring import to:
    1. Begin immediately and run every day at 5 PM indefinitely,
    2. Begin immediately, run once each week on Wednesdays at 3 AM, and stop after 3 months,
    3. Begin in 3 months on Tuesday at 12 PM, run every Friday at 9 PM, and stop after 1 year.

These are just a few examples to demonstrate the flexibility of the scheduling system.

Availability

Scheduling of Bulk Imports is available for all customer sites. However, schedules currently can only be created and managed using the new (BETA) CLI, not the Web App.

Additionally, scheduling is not supported when importing data from "My Computer".

Scheduling from My Computer

An alternative option for scheduling recurring imports from a local machine ("My Computer") is to use other, 3rd-party tools (e.g., crontab on Linux) to schedule execution of the Flywheel CLI.

For example, a crontab file similar to the following can be used to schedule the Flywheel CLI to run on a recurring schedule.

30 5 * * FRI fw-beta import run -p 'fw://demo/ACME Trial' -s /mnt/data/

In this example, the cron system in Linux will invoke the Flywheel CLI at 05:30 (local) every Friday to run a new Bulk Import from /mnt/data to project fw://demo/ACME Trial.

Create a Schedule

Setting up a schedule for Bulk Import is similar to running a one-time import but with just a few extra parameters. To setup a new scheduled import use the import schedule create command instead of the import run command.

1
fw-beta import schedule create -p <project> -s <storage> --start <start-date-time> --end <end-date-time> --cron <cron-string>

Where:

  • <start-date-time>: Is the date and time when the schedule should begin in ISO 8601 format.
    • E.g., --start '2025-09-03T06:40:00+00:00' means "begin on September 3, 2025 at 06:40 UTC+0".
    • The import job will not run prior to this date and time, regardless of the recurrence setting.
  • <end-date-time>: Is the date and time when the schedule should terminate in ISO 8601 format.
    • E.g., --end '2026-03-12T14:15:00+00:00' means "end after March 3, 2026 at 14:15 UTC+0".
    • The import job will not start again after this date and time, regardless of the recurrence setting.
  • <cron-string>: Is a cron-style pattern defining when the import job should recur.
    • E.g., --cron '30 5 * * FRI' means "at 05:30 every Friday"

For example, consider this command:

1
2
fw-beta import schedule create -p 'fw://demo/ACME Trial' -s 682e21f7487fc46c72718887 \
--start '2025-09-03T06:40:00+00:00' --end '2026-03-12T14:15:00+00:00' --cron '30 5 * * FRI'

This command will set up a schedule that launches a new import every Friday at 05:30 (UTC+0) from storage 682e21f7487fc46c72718887 into project fw://demo/ACME Trial, where the first occurrence begins after September 3, 2025 at 06:40 UTC+0 and the last occurrence begins before March 3, 2026 at 14:15 UTC+0.

For more information, see the new CLI documentation for import schedule create.

Modify a Schedule

To modify an existing Import Schedule use the import schedule update command. This command works the same as the import schedule create command for creating a new schedule, but only the timing-related options can be modified (i.e., --start, --end, --cron).

1
fw-beta import schedule update <ID> --start <start-date-time> --end <end-date-time> --cron <cron-string>

Where <id> is replaced with the ID of the Import Schedule to be modified. If you do not know the Import Schedule ID, see the article locating the ID of an Import Schedule.

Note

Only the timing-related options can be modified for an existing Import Schedule (i.e., --start, --end, --cron). If the import-related options need to be modified (e.g., -p, -s, --mapping, etc.), a new schedule will need to be created instead.

Delete a Schedule

To delete an existing Import Schedule use the import schedule delete command.

1
fw-beta import schedule delete <ID>

Where <id> is replaced with the ID of the Import Schedule to be modified. If you do not know the Import Schedule ID, see the article locating the ID of an Import Schedule.

Locating the ID of an Import Schedule

To locate the Import Schedule ID, use the import schedule list command to list all the existing Import Schedules on the site:

1
fw-beta import schedule list

This output is paginated, and only the 20 most recent import schedules will returned by default. To view the next "page" of import schedules, take the ID of the last schedule in the list, then run the following command:

1
fw-beta import schedule list --after-id <id>

Where <id> is replaced with the ID of the last schedule in the list.

Repeat this process to continue stepping through each "page" of import schedules.

Tip

There are additional options for filtering, sorting, and changing the length of the list to help locate a particular import schedule. These options can be listed using the following command:

1
fw-beta import schedule list -h

For more information, see the new CLI documentation for import schedule list.

References

  1. New (BETA) CLI Docs on import schedule