Skip to content

Commit

Permalink
nit
Browse files Browse the repository at this point in the history
  • Loading branch information
ArthurZucker committed Jun 4, 2024
1 parent fa16167 commit d998a7a
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion tokenizers/src/tokenizer/added_vocabulary.rs
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ fn space_rightmost_at_start(sentence: &str) -> usize {
/// exist as required.
///
#[derive(Clone, Debug, Display)]
#[display(fmt="{:?}, encode_special_tokens={}", added_tokens_map, encode_special_tokens)]
#[display(fmt="AddedVocabulary(added_tokens_map_r={:#?}, encode_special_tokens={})", "added_tokens_map_r", encode_special_tokens)]
pub struct AddedVocabulary {
/// Contains the mapping from String (token content) to ID. This map contains both special
/// tokens and classic added tokens that were added to the this vocabulary.
Expand Down

0 comments on commit d998a7a

Please sign in to comment.