Skip to content

Commit

Permalink
bulk: handle absolute/relaive paths in uniform fashion
Browse files Browse the repository at this point in the history
Motivation:
----------
Recent change(s) that massaged user input target paths and
stored absolute paths on bulk backend lead to ambiguity between
user provided and dcache paths and also resulted in inability
to use full paths (i.e. only relative paths are supported). At
Fermilab we need to use both - relative and absolute paths

Modification:
------------
Revert all recent changes that appended prefix to user
supplied paths, stored the result and then stripped the
prefix so that only "original" paths are exposed to the
user.
Instead, like before, store user supplied paths but carry
over request prefix which is computed from user root and
door root. When calling PnfsManager using paths the full
patths of the targets are reassembled using the prefix

Result:
------
Restored ability to use absolute paths when using REST API.

Issue: dCache#7693
Target: trunk
Request: 10.2, 10.1, 10.0, 9.2
Require-book: no
Require-notes: yes
Signed-off-by: Dmitry Litvintsev [email protected]
  • Loading branch information
DmitryLitvintsev committed Nov 27, 2024
1 parent 8bcfc2a commit db4becc
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -167,8 +167,10 @@ public List<ArchiveInfo> getArchiveInfo(
}

var archiveInfos = archiveInfoCollector.getInfo(HandlerBuilders.roleAwarePnfsHandler(pnfsManager),
paths);

rootPath.toString(),
paths);
return archiveInfos;
}


}
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
import com.google.common.base.Throwables;
import diskCacheV111.util.CacheException;
import diskCacheV111.util.FileLocality;
import diskCacheV111.util.FsPath;
import diskCacheV111.util.PnfsHandler;
import dmg.cells.nucleus.CellCommandListener;
import dmg.util.command.Argument;
Expand Down Expand Up @@ -117,12 +118,12 @@ public String call() throws Exception {
private ExecutorService service;
private int maxPaths;

public List<ArchiveInfo> getInfo(PnfsHandler pnfsHandler, List<String> paths) {
public List<ArchiveInfo> getInfo(PnfsHandler pnfsHandler, String prefix, List<String> paths) {
Map<String, Future<FileLocality>> futures = new HashMap<>();
List<ArchiveInfo> infoList = new ArrayList<>();

for (String path : paths) {
futures.put(path, service.submit(() -> getInfo(path, pnfsHandler)));
futures.put(path, service.submit(() -> getInfo(path, prefix, pnfsHandler)));
}

for (Entry<String, Future<FileLocality>> future : futures.entrySet()) {
Expand Down Expand Up @@ -162,9 +163,23 @@ public void setService(ExecutorService service) {
this.service = service;
}

private FileLocality getInfo(String path, PnfsHandler pnfsHandler) throws CacheException {
FileAttributes attributes = pnfsHandler.getFileAttributes(path, REQUIRED_ATTRIBUTES,
private FileLocality getInfo(String path, String prefix, PnfsHandler pnfsHandler) throws CacheException {
String absolutePath = computeFsPath(prefix, path).toString();
FileAttributes attributes = pnfsHandler.getFileAttributes(absolutePath, REQUIRED_ATTRIBUTES,
ACCESS_MASK, false);
return poolMonitor.getFileLocality(attributes, "localhost");
}

public static FsPath computeFsPath(String prefix, String target) {
FsPath absolutePath = FsPath.create(FsPath.ROOT + target);
if (prefix != null) {
FsPath pref = FsPath.create(prefix);
if (!absolutePath.hasPrefix(pref)) {
absolutePath = FsPath.create(
FsPath.ROOT + (prefix.endsWith("/") ? prefix : prefix + "/") + target);
}
}
return absolutePath;
}

}

0 comments on commit db4becc

Please sign in to comment.