feat: Enable auto-purge and set defaults

[xeniape]

Description

Implementation for #1029

This PR only sets default values for the properties autopurge.snapRetainCount, autopurge.purgeInterval (this one basically enables auto-purge), snapSizeLimitInKb, and preAllocSize. The bulk of the PR is the additional docs explaining the feature and adding a reference page for the defaults.

Verified that this feature works with a local kind cluster load test with smaller values. Snapshots were successfully deleted except for the 3 we keep.

Integrationtests

--- PASS: kuttl (329.75s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/smoke_zookeeper-3.9.4_use-server-tls-false_use-client-auth-tls-false_openshift-false (71.26s)
        --- PASS: kuttl/harness/smoke_zookeeper-3.9.5_use-server-tls-false_use-client-auth-tls-false_openshift-false (77.79s)
        --- PASS: kuttl/harness/smoke_zookeeper-3.9.5_use-server-tls-true_use-client-auth-tls-false_openshift-false (83.93s)
        --- PASS: kuttl/harness/smoke_zookeeper-3.9.5_use-server-tls-true_use-client-auth-tls-true_openshift-false (84.30s)
        --- PASS: kuttl/harness/smoke_zookeeper-3.9.5_use-server-tls-false_use-client-auth-tls-true_openshift-false (83.12s)
        --- PASS: kuttl/harness/smoke_zookeeper-3.9.4_use-server-tls-true_use-client-auth-tls-false_openshift-false (84.67s)
        --- PASS: kuttl/harness/smoke_zookeeper-3.9.4_use-server-tls-true_use-client-auth-tls-true_openshift-false (83.38s)
        --- PASS: kuttl/harness/smoke_zookeeper-3.9.4_use-server-tls-false_use-client-auth-tls-true_openshift-false (82.97s)
PASS

Definition of Done Checklist

• Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
• Please make sure all these things are done and tick the boxes

Author

• Changes are OpenShift compatible
• CRD changes approved
• CRD documentation for all fields, following the style guide.
• Helm chart can be installed and deployed operator works
• Integration tests passed (for non trivial changes)
• Changes need to be "offline" compatible
• Links to generated (nightly) docs added
• Release note snippet added

Reviewer

• Code contains useful comments
• Code contains useful logging statements
• (Integration-)Test cases added
• Documentation added or updated. Follows the style guide.
• Changelog updated
• Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

• Feature Tracker has been updated
• Proper release label has been added
• Links to generated (nightly) docs added
• Release note snippet added
• Add type/deprecation label & add to the deprecation schedule
• Add type/experimental label & add to the experimental features tracker

[xeniape]

I added the change as BREAKING in the changelog, but not sure. My reasoning was that it changes behavior, which might impact users and their assumptions of the data directory state. But if someone objects to that, that's fine too.

[lfrancke]

I think BREAKING is fine. We want to point people at behavior that can delete data, even if it's only old journals.
But please add a release note snippet and that must be a bit more detaled than the changelog

[xeniape]

Yeah, that was the plan after it gets approved and the exact changes are set

[razvan]

lgtm with one suggestion

[xeniape]

Release Note

⚠️ BREAKING: ZooKeeper now enables the auto-purge feature for the data directory cleanup per default.

With the default settings, this means, that every 6 hours a cleanup runs, which checks for snapshots and their related transaction logs and only keeps the 3 most recent ones, the remaining ones are deleted. This mitigates the data directory filling up the PersistentVolume. More on the process, check the docs and for the defaults the reference.