Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-12057. Handle '/' in key names for the readreplicas command #7735

Merged
merged 1 commit into from
Jan 22, 2025

Conversation

ptlrs
Copy link
Contributor

@ptlrs ptlrs commented Jan 22, 2025

What changes were proposed in this pull request?

Key names may include arbitrary number of / characters, such as /volumename/bucketname/test1/test2.

The read-replicas command assumes key names will be single level or not include any / characters. This results in failures when the command is run on multilevel keys.

This PR flattens the keynames by replacing / with _.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-12075
Also see discussion in #7693

How was this patch tested?

Manual testing:

bash-5.1$ ozone debug --verbose read-replicas /ockrwvolume/ockrwbucket/test1/test2/1
Volume Name : ockrwvolume
Bucket Name : ockrwbucket
Key Name : test1/test2/1
Creating directory : ockrwvolume_ockrwbucket_test1_test2_1_20250122073427
Successfully created!
Writing : test1_test2_1_block1_ozone-datanode-1.ozone_default
Writing manifest file : test1_test2_1_manifest

bash-5.1$ cd ockrwvolume_ockrwbucket_test1_test2_1_20250122073427/

bash-5.1$ ls -alht
total 1.1M
-rw-r--r--  1 hadoop hadoop  365 Jan 22 07:34 test1_test2_1_manifest
drwxr-xr-x  4 hadoop hadoop  128 Jan 22 07:34 .
-rw-r--r--  1 hadoop hadoop 1.0M Jan 22 07:34 test1_test2_1_block1_ozone-datanode-1.ozone_default
drwxr-xr-x 25 hadoop hadoop  800 Jan 22 07:34 ..

bash-5.1$ cat test1_test2_1_manifest 
{
  "filename" : "ockrwvolume/ockrwbucket/test1/test2/1",
  "datasize" : 1048576,
  "blocks" : [ {
    "blockIndex" : 1,
    "containerId" : 1,
    "localId" : 115816896921600001,
    "length" : 1048576,
    "offset" : 0,
    "replicas" : [ {
      "hostname" : "ozone-datanode-1.ozone_default",
      "uuid" : "5e14c3a9-2a94-490a-a34c-a657354c1198"
    } ]
  } ]

CI: https://github.com/ptlrs/ozone/actions/runs/12903408829

@ptlrs ptlrs marked this pull request as ready for review January 22, 2025 07:46
@dombizita dombizita added the tools Tools that helps with debugging label Jan 22, 2025
Copy link
Contributor

@dombizita dombizita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for fixing this @ptlrs, it looks good to me!

@dombizita dombizita requested a review from adoroszlai January 22, 2025 14:45
@adoroszlai adoroszlai merged commit 642b1c7 into apache:master Jan 22, 2025
54 checks passed
@adoroszlai
Copy link
Contributor

Thanks @ptlrs for the patch, @dombizita for the review.

@ptlrs ptlrs deleted the HDDS-12075-use-underscore-filename branch January 24, 2025 02:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
tools Tools that helps with debugging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants