v0.2.2

What's Changed

• Do not build python bindings by default by @kelvich in #6

New Contributors

• @kelvich made their first contribution in #6

Full Changelog: v0.2.1...v0.2.2

v0.2.1

Full Changelog: v0.2.0...v0.2.1

v0.2.0

What's Changed

• BREAKING CHANGE: Moved core methods to the root namespace
• Vendor out the tiktoken code by @zurawiki in #4
• Update tiktoken implementation
• Add get_encoder_for_model function
• Add p50k_edit encoder

Full Changelog: v0.1.4...v0.2.0

v0.1.4

What's Changed

• Refactor tokenization and special tokens by @zurawiki in #3

Full Changelog: v0.1.3...v0.1.4

v0.1.3

What's Changed

• Implement cl100k tokenizer by @zurawiki in #2
• Improve documentation

New Contributors

• @zurawiki made their first contribution in #2

Full Changelog: v0.1.2...v0.1.3

v0.1.2

Full Changelog: v0.1.1...v0.1.2

v0.1.1

Full Changelog: https://github.com/zurawiki/tiktoken-rs/commits/v0.1.1