Skip to content

Commit

Permalink
feat: Rename non-sandbox mode for clarity
Browse files Browse the repository at this point in the history
Rename sandbox and non-sandbox modes as gVisor and non-gVisor mode respectively for clarity.

Bug: b/389682874
Change-Id: Icde11a28153cfe9d991e5b10ad9f516c359429ac
GitOrigin-RevId: 46063a3ccbc12dbbb24977c963ca141953a0930c
  • Loading branch information
Privacy Sandbox Team authored and copybara-github committed Jan 14, 2025
1 parent 5b1390c commit 98f481a
Show file tree
Hide file tree
Showing 18 changed files with 149 additions and 134 deletions.
42 changes: 21 additions & 21 deletions docs/roma/byob/Benchmarking.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ Namely,
- `dist/roma_byob/benchmark_nonroot_debug_image.tar`
- `dist/roma_byob/benchmark_root_debug_image.tar`

Based on the choice of configuration, you can compare the numbers between sandbox and non-sandbox
mode. In non-sandbox mode, the gVisor sandbox layer is not used. The clone-pivot_root-exec construct
is utilized in both the sandbox and non-sandbox modes.
Based on the choice of configuration, you can compare the numbers between gVisor and non-gVisor
mode. In non-gVisor mode, the gVisor layer is not used. The clone-pivot_root-exec construct is
utilized in both the gVisor and non-gVisor modes.

The remainder of this doc focuses on non-debug images. Debug images are largely the same image with
the addition of a shell. See
Expand Down Expand Up @@ -122,24 +122,24 @@ nitro-cli run-enclave --cpu-count 2 --memory 1684 --eif-path roma_byob_benchmark

Running the benchmarks on an ARM64 machine with `8 X 50 MHz CPUs` yielded the following results:

| Benchmark | Time | CPU | Iterations | Details |
| ------------------------------------------------------- | ------- | -------- | ---------- | ---------------- |
| BM_ProcessRequestPrimeSieve/mode:0/prime_count:100000 | 4.37 ms | 0.067 ms | 1000 | mode:Sandbox |
| BM_ProcessRequestPrimeSieve/mode:1/prime_count:100000 | 15.7 ms | 0.071 ms | 1000 | mode:Non-Sandbox |
| BM_ProcessRequestPrimeSieve/mode:0/prime_count:500000 | 5.20 ms | 0.071 ms | 1000 | mode:Sandbox |
| BM_ProcessRequestPrimeSieve/mode:1/prime_count:500000 | 15.6 ms | 0.078 ms | 1000 | mode:Non-Sandbox |
| BM_ProcessRequestPrimeSieve/mode:0/prime_count:1000000 | 5.69 ms | 0.071 ms | 1000 | mode:Sandbox |
| BM_ProcessRequestPrimeSieve/mode:1/prime_count:1000000 | 15.7 ms | 0.084 ms | 1000 | mode:Non-Sandbox |
| BM_ProcessRequestPrimeSieve/mode:0/prime_count:5000000 | 18.3 ms | 0.085 ms | 1000 | mode:Sandbox |
| BM_ProcessRequestPrimeSieve/mode:1/prime_count:5000000 | 17.7 ms | 0.073 ms | 1000 | mode:Non-Sandbox |
| BM_ProcessRequestPrimeSieve/mode:0/prime_count:10000000 | 37.4 ms | 0.094 ms | 100 | mode:Sandbox |
| BM_ProcessRequestPrimeSieve/mode:1/prime_count:10000000 | 36.7 ms | 0.091 ms | 100 | mode:Non-Sandbox |
| BM_ProcessRequestSortList/mode:0/n_items:10000 | 4.60 ms | 0.065 ms | 1000 | mode:Sandbox |
| BM_ProcessRequestSortList/mode:1/n_items:10000 | 15.6 ms | 0.072 ms | 1000 | mode:Non-Sandbox |
| BM_ProcessRequestSortList/mode:0/n_items:100000 | 7.05 ms | 0.071 ms | 1000 | mode:Sandbox |
| BM_ProcessRequestSortList/mode:1/n_items:100000 | 15.7 ms | 0.073 ms | 1000 | mode:Non-Sandbox |
| BM_ProcessRequestSortList/mode:0/n_items:1000000 | 64.6 ms | 0.094 ms | 100 | mode:Sandbox |
| BM_ProcessRequestSortList/mode:1/n_items:1000000 | 64.9 ms | 0.096 ms | 100 | mode:Non-Sandbox |
| Benchmark | Time | CPU | Iterations | Details |
| ------------------------------------------------------- | ------- | -------- | ---------- | --------------- |
| BM_ProcessRequestPrimeSieve/mode:0/prime_count:100000 | 4.37 ms | 0.067 ms | 1000 | mode:gVisor |
| BM_ProcessRequestPrimeSieve/mode:1/prime_count:100000 | 15.7 ms | 0.071 ms | 1000 | mode:Non-gVisor |
| BM_ProcessRequestPrimeSieve/mode:0/prime_count:500000 | 5.20 ms | 0.071 ms | 1000 | mode:gVisor |
| BM_ProcessRequestPrimeSieve/mode:1/prime_count:500000 | 15.6 ms | 0.078 ms | 1000 | mode:Non-gVisor |
| BM_ProcessRequestPrimeSieve/mode:0/prime_count:1000000 | 5.69 ms | 0.071 ms | 1000 | mode:gVisor |
| BM_ProcessRequestPrimeSieve/mode:1/prime_count:1000000 | 15.7 ms | 0.084 ms | 1000 | mode:Non-gVisor |
| BM_ProcessRequestPrimeSieve/mode:0/prime_count:5000000 | 18.3 ms | 0.085 ms | 1000 | mode:gVisor |
| BM_ProcessRequestPrimeSieve/mode:1/prime_count:5000000 | 17.7 ms | 0.073 ms | 1000 | mode:Non-gVisor |
| BM_ProcessRequestPrimeSieve/mode:0/prime_count:10000000 | 37.4 ms | 0.094 ms | 100 | mode:gVisor |
| BM_ProcessRequestPrimeSieve/mode:1/prime_count:10000000 | 36.7 ms | 0.091 ms | 100 | mode:Non-gVisor |
| BM_ProcessRequestSortList/mode:0/n_items:10000 | 4.60 ms | 0.065 ms | 1000 | mode:gVisor |
| BM_ProcessRequestSortList/mode:1/n_items:10000 | 15.6 ms | 0.072 ms | 1000 | mode:Non-gVisor |
| BM_ProcessRequestSortList/mode:0/n_items:100000 | 7.05 ms | 0.071 ms | 1000 | mode:gVisor |
| BM_ProcessRequestSortList/mode:1/n_items:100000 | 15.7 ms | 0.073 ms | 1000 | mode:Non-gVisor |
| BM_ProcessRequestSortList/mode:0/n_items:1000000 | 64.6 ms | 0.094 ms | 100 | mode:gVisor |
| BM_ProcessRequestSortList/mode:1/n_items:1000000 | 64.9 ms | 0.096 ms | 100 | mode:Non-gVisor |

These results include only some of the benchmarks run. For all benchmarks, consider running them
yourself. Refer also to the source code for the benchmarks in
Expand Down
2 changes: 1 addition & 1 deletion src/roma/byob/benchmark/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -161,7 +161,7 @@ Step 2: Run the following from a command line:
docker run --privileged --volume="${PWD}"/dist:/tmp/runsc-log --cpus=64 --memory=512gb \
privacy-sandbox/roma-byob/traffic_generator:v1-root \
--num_workers=84 --queries_per_second=10 --burst_size=14 \
--num_queries=1000 --sandbox=off --lib_mounts="" --binary_path="/udf/sample_go_udf"
--num_queries=1000 --gvisor=off --lib_mounts="" --binary_path="/udf/sample_go_udf"
"""

[
Expand Down
2 changes: 1 addition & 1 deletion src/roma/byob/benchmark/byob_rps
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ function run_test() {
CMD_ARGS+=("--input_args=${INPUT_ARGS}")
fi
else
CMD_ARGS+=("--sandbox=${SANDBOX}")
CMD_ARGS+=("--gvisor=${SANDBOX}")
fi

set -o xtrace
Expand Down
2 changes: 1 addition & 1 deletion src/roma/byob/benchmark/memory_usage_benchmark.cc
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ int main(int argc, char** argv) {
{
.roma_container_name = "roma_server",
},
Mode::kModeSandbox);
Mode::kSandboxModeWithGvisor);
CHECK_OK(sample_interface);

// Load UDF.
Expand Down
17 changes: 9 additions & 8 deletions src/roma/byob/benchmark/roma_byob_benchmark.cc
Original file line number Diff line number Diff line change
Expand Up @@ -67,8 +67,8 @@ constexpr std::string_view kNewUdfOutput = "I am a new UDF!";
constexpr std::string_view kJavaOutput = "Hello, world from Java!";
constexpr std::string_view kGoBinaryOutput = "Hello, world from Go!";
constexpr Mode kModes[] = {
Mode::kModeSandbox,
Mode::kModeNoSandbox,
Mode::kSandboxModeWithGvisor,
Mode::kSandboxModeWithoutGvisor,
};

enum class Language {
Expand Down Expand Up @@ -167,10 +167,10 @@ std::filesystem::path GetFilePathFromLanguage(Language lang) {

std::string GetModeStr(Mode mode) {
switch (mode) {
case Mode::kModeSandbox:
return "mode:Sandbox";
case Mode::kModeNoSandbox:
return "mode:Non-Sandbox";
case Mode::kSandboxModeWithGvisor:
return "mode:gVisor";
case Mode::kSandboxModeWithoutGvisor:
return "mode:Non-gVisor";
default:
return "mode:Unknown";
}
Expand Down Expand Up @@ -365,7 +365,7 @@ void BM_ProcessRequestMultipleLanguages(benchmark::State& state) {
.lib_mounts = std::move(mounts),
};
ByobSampleService<> roma_service =
GetRomaService(Mode::kModeSandbox, std::move(config));
GetRomaService(Mode::kSandboxModeWithGvisor, std::move(config));

std::string code_token =
LoadCode(roma_service, GetFilePathFromLanguage(lang),
Expand Down Expand Up @@ -568,7 +568,8 @@ void BM_ProcessRequestSortList(benchmark::State& state) {

void BM_ProcessRequestDevNullVsLogBinary(benchmark::State& state) {
const Log log = static_cast<Log>(state.range(0));
ByobSampleService<> roma_service = GetRomaService(Mode::kModeSandbox);
ByobSampleService<> roma_service =
GetRomaService(Mode::kSandboxModeWithGvisor);
const bool enable_log_egress = [](Log log) {
switch (log) {
case Log::kLogToDevNull:
Expand Down
8 changes: 4 additions & 4 deletions src/roma/byob/benchmark/traffic_generator.cc
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,9 @@ ABSL_FLAG(int, num_queries, 10'000, "Number of queries to be sent");
ABSL_FLAG(int, total_invocations, 0,
"Number of invocations to be sent. If non-zero, overrides "
"num_queries, and num_queries = total_invocations / burst_size.");
ABSL_FLAG(privacy_sandbox::server_common::byob::Mode, sandbox,
privacy_sandbox::server_common::byob::Mode::kModeSandbox,
"Run BYOB in sandbox mode.");
ABSL_FLAG(privacy_sandbox::server_common::byob::Mode, gvisor,
privacy_sandbox::server_common::byob::Mode::kSandboxModeWithGvisor,
"Run BYOB with gVisor.");
ABSL_FLAG(std::string, lib_mounts, LIB_MOUNTS,
"Mount paths to include in the pivot_root environment. Example "
"/dir1,/dir2");
Expand Down Expand Up @@ -106,7 +106,7 @@ int main(int argc, char** argv) {
const std::string lib_mounts = absl::GetFlag(FLAGS_lib_mounts);
const std::string binary_path = absl::GetFlag(FLAGS_binary_path);
const privacy_sandbox::server_common::byob::Mode sandbox =
absl::GetFlag(FLAGS_sandbox);
absl::GetFlag(FLAGS_gvisor);

const std::string udf_path = absl::GetFlag(FLAGS_udf_path);
const std::string handler_name = absl::GetFlag(FLAGS_handler_name);
Expand Down
2 changes: 1 addition & 1 deletion src/roma/byob/benchmark/traffic_generator_image_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -39,5 +39,5 @@ commandTests:
- --num_queries=1
- --burst_size=2
- --queries_per_second=1
- --sandbox=on
- --gvisor=on
exitCode: 0
18 changes: 9 additions & 9 deletions src/roma/byob/config/config.h
Original file line number Diff line number Diff line change
Expand Up @@ -36,23 +36,23 @@ inline constexpr std::string_view kLdLibraryPath =
"LD_LIBRARY_PATH=" LIB_MOUNTS;

enum class Mode {
kModeSandbox,
kModeNoSandbox,
kModeSandboxDebug,
kSandboxModeWithGvisor,
kSandboxModeWithoutGvisor,
kSandboxModeWithGvisorDebug,
};

inline bool AbslParseFlag(absl::string_view text, Mode* mode,
std::string* error) {
if (text == "on") {
*mode = Mode::kModeSandbox;
*mode = Mode::kSandboxModeWithGvisor;
return true;
}
if (text == "debug") {
*mode = Mode::kModeSandboxDebug;
*mode = Mode::kSandboxModeWithGvisorDebug;
return true;
}
if (text == "off") {
*mode = Mode::kModeNoSandbox;
*mode = Mode::kSandboxModeWithoutGvisor;
return true;
}
*error = "Supported values: on, off, debug.";
Expand All @@ -61,11 +61,11 @@ inline bool AbslParseFlag(absl::string_view text, Mode* mode,

inline std::string AbslUnparseFlag(Mode mode) {
switch (mode) {
case Mode::kModeSandbox:
case Mode::kSandboxModeWithGvisor:
return "on";
case Mode::kModeSandboxDebug:
case Mode::kSandboxModeWithGvisorDebug:
return "debug";
case Mode::kModeNoSandbox:
case Mode::kSandboxModeWithoutGvisor:
return "off";
default:
return absl::StrCat(mode);
Expand Down
8 changes: 4 additions & 4 deletions src/roma/byob/interface/roma_service.h
Original file line number Diff line number Diff line change
Expand Up @@ -111,16 +111,16 @@ class RomaService final {
return absl::ErrnoToStatus(errno, "fork()");
}
switch (mode) {
case Mode::kModeSandbox:
case Mode::kModeSandboxDebug:
case Mode::kSandboxModeWithGvisor:
case Mode::kSandboxModeWithGvisorDebug:
handle_.emplace<internal::roma_service::ByobHandle>(
pid, config.lib_mounts, control_socket_path.c_str(),
udf_socket_path.c_str(), socket_dir_.c_str(),
std::move(config.roma_container_name), log_dir_.c_str(),
config.memory_limit_soft, config.memory_limit_hard,
/*debug=*/mode == Mode::kModeSandboxDebug);
/*debug=*/mode == Mode::kSandboxModeWithGvisorDebug);
break;
case Mode::kModeNoSandbox:
case Mode::kSandboxModeWithoutGvisor:
handle_.emplace<internal::roma_service::LocalHandle>(
pid, config.lib_mounts, control_socket_path.c_str(),
udf_socket_path.c_str(), socket_dir_.c_str(), log_dir_.c_str());
Expand Down
6 changes: 3 additions & 3 deletions src/roma/byob/sample_udf/shell_cli_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,16 +45,16 @@ commandTests:
Flags from bazel-out/.*/bin/src/roma/byob/sample_udf/sample_shell.cc:
--commands_file \(Text file containing shell commands to execute\.\);
default: ;
--gvisor \(Run BYOB with gVisor\.\); default: on;
--num_workers \(Number of workers\.\); default: 1;
--sandbox \(Run BYOB in sandbox mode\.\); default: on;
--udf_log_file \(Directory for storing UDF logs\.\); default: ;
- name: shell-cli sandbox flag
command: /tools/shell-cli
args: ['--sandbox=unknown']
args: ['--gvisor=unknown']
exitCode: 1
expectedError:
- "ERROR: Illegal value 'unknown' specified for flag 'sandbox'; Supported values: on, off, debug."
- "ERROR: Illegal value 'unknown' specified for flag 'gvisor'; Supported values: on, off, debug."

- name: shell-cli help commands file
command: /tools/shell-cli
Expand Down
2 changes: 1 addition & 1 deletion src/roma/byob/test/example_roma_byob_app_service.h
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ class ByobEchoService final {
using Mode = privacy_sandbox::server_common::byob::Mode;

static absl::StatusOr<ByobEchoService<TMetadata>> Create(
Config config, Mode mode = Mode::kModeSandbox) {
Config config, Mode mode = Mode::kSandboxModeWithGvisor) {
auto roma_service = std::make_unique<AppService>();
PS_RETURN_IF_ERROR(roma_service->Init(std::move(config), mode));
return ByobEchoService<TMetadata>(std::move(roma_service));
Expand Down
7 changes: 4 additions & 3 deletions src/roma/byob/test/example_shell.cc
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,9 @@
ABSL_FLAG(int, num_workers, 1, "the number of workers");
ABSL_FLAG(std::optional<std::string>, commands_file, std::nullopt,
"a text file with a list of CLI commands to execute");
ABSL_FLAG(bool, sandbox, true, "run BYOB in sandbox mode");
ABSL_FLAG(privacy_sandbox::server_common::byob::Mode, gvisor,
privacy_sandbox::server_common::byob::Mode::kSandboxModeWithGvisor,
"Run BYOB with gVisor.");
ABSL_FLAG(std::optional<std::string>, udf_log_file, std::nullopt,
"path with directory to a file in which UDF logs will be stored");

Expand All @@ -60,8 +62,7 @@ int main(int argc, char** argv) {

// Initialize BYOB.
absl::StatusOr<ByobEchoService<>> echo_service = ByobEchoService<>::Create(
/*config=*/{},
absl::GetFlag(FLAGS_sandbox) ? Mode::kModeSandbox : Mode::kModeNoSandbox);
/*config=*/{}, absl::GetFlag(FLAGS_gvisor));
CHECK_OK(echo_service);

// Create load and execute RPC handlers.
Expand Down
2 changes: 1 addition & 1 deletion src/roma/byob/test/example_shell_image_test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -166,8 +166,8 @@ commandTests:
Flags from src/roma/byob/test/example_shell\.cc:
--commands_file \(a text file with a list of CLI commands to execute\);
default: ;
--gvisor \(Run BYOB with gVisor\.\); default: on;
--num_workers \(the number of workers\); default: 1;
--sandbox \(run BYOB in sandbox mode\); default: true;
--udf_log_file \(path with directory to a file in which UDF logs will be\
stored\); default: ;
Expand Down
24 changes: 12 additions & 12 deletions src/roma/byob/test/example_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -65,8 +65,8 @@ ByobEchoService<> GetRomaService(Mode mode) {
return GetRomaService(/*config=*/{}, std::move(mode));
}

TEST(RomaByobExampleTest, LoadCppBinaryInSandboxMode) {
ByobEchoService<> roma_service = GetRomaService(Mode::kModeSandbox);
TEST(RomaByobExampleTest, LoadCppBinaryInGvisorMode) {
ByobEchoService<> roma_service = GetRomaService(Mode::kSandboxModeWithGvisor);

absl::Notification notif;
absl::Status notif_status;
Expand All @@ -79,8 +79,8 @@ TEST(RomaByobExampleTest, LoadCppBinaryInSandboxMode) {
EXPECT_TRUE(notif_status.ok());
}

TEST(RomaByobExampleTest, LoadCppBinaryInNonSandboxMode) {
Mode mode = Mode::kModeNoSandbox;
TEST(RomaByobExampleTest, LoadCppBinaryInNonGvisorMode) {
Mode mode = Mode::kSandboxModeWithoutGvisor;
if (!HasClonePermissionsByobWorker(mode)) {
GTEST_SKIP() << "HasClonePermissionsByobWorker check returned false";
}
Expand All @@ -97,8 +97,8 @@ TEST(RomaByobExampleTest, LoadCppBinaryInNonSandboxMode) {
EXPECT_TRUE(notif_status.ok());
}

TEST(RomaByobExampleTest, LoadGoBinaryInSandboxMode) {
ByobEchoService<> roma_service = GetRomaService(Mode::kModeSandbox);
TEST(RomaByobExampleTest, LoadGoBinaryInGvisorMode) {
ByobEchoService<> roma_service = GetRomaService(Mode::kSandboxModeWithGvisor);
absl::Notification notif;
absl::Status notif_status;

Expand All @@ -110,8 +110,8 @@ TEST(RomaByobExampleTest, LoadGoBinaryInSandboxMode) {
EXPECT_TRUE(notif_status.ok());
}

TEST(RomaByobExampleTest, LoadGoBinaryInNonSandboxMode) {
Mode mode = Mode::kModeNoSandbox;
TEST(RomaByobExampleTest, LoadGoBinaryInNonGvisorMode) {
Mode mode = Mode::kSandboxModeWithoutGvisor;
if (!HasClonePermissionsByobWorker(mode)) {
GTEST_SKIP() << "HasClonePermissionsByobWorker check returned false";
}
Expand All @@ -128,7 +128,7 @@ TEST(RomaByobExampleTest, LoadGoBinaryInNonSandboxMode) {
}

TEST(RomaByobExampleTest, NotifProcessRequestCppBinary) {
ByobEchoService<> roma_service = GetRomaService(Mode::kModeSandbox);
ByobEchoService<> roma_service = GetRomaService(Mode::kSandboxModeWithGvisor);
const std::string message = "I am a test Cpp message.";
const std::string code_token = LoadCode(
roma_service, kUdfPath / kCPlusPlusBinaryFilename, /*num_workers=*/2);
Expand All @@ -147,7 +147,7 @@ TEST(RomaByobExampleTest, NotifProcessRequestCppBinary) {
}

TEST(RomaByobExampleTest, AsyncCallbackProcessRequestCppBinary) {
ByobEchoService<> roma_service = GetRomaService(Mode::kModeSandbox);
ByobEchoService<> roma_service = GetRomaService(Mode::kSandboxModeWithGvisor);
const std::string message = "I am a test Cpp message.";
const std::string code_token = LoadCode(
roma_service, kUdfPath / kCPlusPlusBinaryFilename, /*num_workers=*/2);
Expand All @@ -173,7 +173,7 @@ TEST(RomaByobExampleTest, NotifProcessRequestGoBinary) {
{
.lib_mounts = "",
},
Mode::kModeSandbox);
Mode::kSandboxModeWithGvisor);
const std::string message = "I am a test Go binary message.";
const std::string code_token = LoadCode(
roma_service, kUdfPath / kGoLangBinaryFilename, /*num_workers=*/2);
Expand All @@ -196,7 +196,7 @@ TEST(RomaByobExampleTest, AsyncCallbackProcessRequestGoBinary) {
{
.lib_mounts = "",
},
Mode::kModeSandbox);
Mode::kSandboxModeWithGvisor);
const std::string message = "I am a test Go binary message.";
const std::string code_token = LoadCode(
roma_service, kUdfPath / kGoLangBinaryFilename, /*num_workers=*/2);
Expand Down
Loading

0 comments on commit 98f481a

Please sign in to comment.