You can use

neve bpe.nv

To train and test the inference of a Byte-Pair Encoding tokenizer. You will need to change the path in BPE and File object instantiations. The path is "/Datasets/openwebtxt/openwebtext.txt"