Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Create new proxy service to store (and relay) broadcasted daft data #3774

Closed
wants to merge 18 commits into from

Conversation

raunakab
Copy link
Contributor

@raunakab raunakab commented Feb 6, 2025

Overview

This PR introduces a new Rust proxy-service which captures metrics outputted by Daft, and exposes a simple HTTP endpoint for a dashboard to hit to query that data.

Usage

# Terminal 1:
cd dashboard/proxy-service
cargo run

# Terminal 2:
# Create a new test.py file and put some code that uses Daft in there.
# Make sure it executes the query (by calling `.collect()` or something similar).
# Then, run it.
python test.py

# The proxy-service should have caught it and stored it.
# Test that it did by running the following:
curl http://localhost:3239

# You should see a (non-empty) JSON array being printed out.

@github-actions github-actions bot added the feat label Feb 6, 2025
Copy link

codspeed-hq bot commented Feb 6, 2025

CodSpeed Performance Report

Merging #3774 will improve performances by 99.98%

Comparing proxy-service (c1a4c71) with main (86adc44)

Summary

⚡ 2 improvements
✅ 25 untouched benchmarks

Benchmarks breakdown

Benchmark BASE HEAD Change
test_iter_rows_first_row[100 Small Files] 342.5 ms 171.3 ms +99.98%
test_show[100 Small Files] 24.2 ms 16.4 ms +47.65%

@raunakab raunakab closed this Feb 7, 2025
@raunakab raunakab deleted the proxy-service branch February 7, 2025 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant