Skip to content

Same-day updates are not processed #16

Open

Description

@ctgraham

After having run a migration, and then updating ASpace resources on the same day, the updated ASpace resources are not picked up in an additional same-day run. Setting the highwatermark back to the prior day does pick up the changed resources. This is non-intuitive.

It suggests that the greater_than comparator with a precision of day means "anything after the end of day", not "anything after the start of day".

"comparator" => "greater_than",
"field" => "system_mtime",
"value" => $last_import_date,
"precision" => "day"

If we change this to "greater than the previous day", we will get the changed records from today, but we will also get them repeated for multiple runs on the same day. I think this is ok, as they will already be present in the map table.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions