Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is not even close to complete. It is missing a few features, how ever I hope it shows where this is going and how I think
archivistwill be.I introduce the server side endpoints. Both at post based. The archive receives a manifest, creates a unique id for it and queues it. The extract end point will be similar, but I am not sure yet about the type of data it will receive.
Storage can either be Posix or HPSS. A secondary thread will monitor the archive queue. As soon as there is something, it will pull it, create a tarball if needed and push it to the storage.
All transactions will be recorded in a local postgres, or sqlite DB. A third thread will probe the DB and based on the status of archives it will communicate with the librarian which data points are archived.
Claude is used as a help to make choices and quickly implement things that are not in my head immediately.