What's wrong w/ creating a custom field on the document for it's status?
-- The absence of the field could indicate that it's a new document.
-- Setting the field's value to 1 could mean some initial state, like "processed".
If you then create a view of documents who's "status" field was absent and you then have a handy list of documents on which to run your agent, without the overhead of creating/maintaining a document collection. When your agent is done processing each document, the last step would be to set the "status" field's value to "1".
With this approach, it doesn't matter what time the agent last ran, or completed, or even what documents were created while it ran.
Even if you had 1m documents, that would only be 1MB of data used by the extra field.
Am I oversimplifying things and missing the point, entirely?
What's wrong w/ creating a custom field on the document for it's status?
-- The absence of the field could indicate that it's a new document.
-- Setting the field's value to 1 could mean some initial state, like "processed".
If you then create a view of documents who's "status" field was absent and you then have a handy list of documents on which to run your agent, without the overhead of creating/maintaining a document collection. When your agent is done processing each document, the last step would be to set the "status" field's value to "1".
With this approach, it doesn't matter what time the agent last ran, or completed, or even what documents were created while it ran.
Even if you had 1m documents, that would only be 1MB of data used by the extra field.
Am I oversimplifying things and missing the point, entirely?