Skip to content

Commit

Permalink
Update utils.py
Browse files Browse the repository at this point in the history
  • Loading branch information
loubnabnl committed Aug 24, 2023
1 parent 8c02f86 commit 52a15e8
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion pii/ner/pii_redaction/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ def is_key(matched_str):
# download the training corpora from https://raw.githubusercontent.com/domanchi/gibberish-detector/master/examples/big.txt
# run gibberish-detector train big.txt > big.model to generate the model (it takes 3 seconds)
Detector = detector.create_from_model(
"/fsx/loubna/code/bigcode-dataset/pii/gibberish_data/big.model"
"/bigcode-dataset/pii/gibberish_data/big.model"
)
is_gibberish = Detector.is_gibberish(matched_str.lower())
return is_gibberish and len(matched_str) > 8
Expand Down

0 comments on commit 52a15e8

Please sign in to comment.