Improved file
source checkpointing
The file
source checkpointing strategy has been improved to solve surprising edge cases.
The Vector file
source included an option called start_at_beginning
that would determine where
Vector would begin reading a file based on a variety of conditions. As you can see, these conditions were
quite confusing. To resolve this, PR 6178 deprecated the start_at_beginning
option and replaced it with new
ignore_checkpoints
and read_from
options. Migrating is easy:
[sources.file]
type = "file"
-start_at_beginning = true
+ignore_checkpoints = false # default
+read_from = "beginning" # default
Adjust as necessary. The above values are the defaults and are not required to be specified.