diff --git a/README.md b/README.md index 1eeeddb..4c0569f 100644 --- a/README.md +++ b/README.md @@ -8,27 +8,28 @@ Export your favorite GitHub repositories to Prometheus -Use it _as a service_: https://gh.skuzzle.de +* Use it _as a service_: See https://gh.skuzzle.de for instructions +* Deploy it _on-premise_: `docker pull ghcr.io/skuzzle/gh-prom-exporter/gh-prom-exporter:0.0.5` ## On-Premise deployment with docker This application can easily be run as a docker container in whatever environment you like: ``` docker run -p 8080:8080 \ - -e WEB_ALLOWANONYMOUSSCRAPE=true - ghcr.io/skuzzle/gh-prom-exporter/gh-prom-exporter:0.0.4 + -e WEB_ALLOWANONYMOUSSCRAPE=true \ + ghcr.io/skuzzle/gh-prom-exporter/gh-prom-exporter:0.0.5 ``` With _anonymous scraping_ allowed, you can now easily view the scrape results directly in the browser by navigating to `https://your.docker.host:8080/YOUR-GITHUB-USERNAME/YOUR-REPOSITORY`. -The scraped repository can just as easiy be added as static scrape target to your prometheus' scrape configs: +The scraped repository can just as easy be added as static scrape target to your prometheus' scrape configs: ``` scrape_configs: - job_name: CHANGE_ME scrape_interval: 2m - metrics_path: /YOUR-GITHUB-USERNAME/YOUR-REPOSITORY + metrics_path: /YOUR-GITHUB-USERNAME/YOUR-REPOSITORY1,YOUR-REPOSITORY2 static_configs: - targets: ['your.docker.host:8080'] ``` @@ -36,7 +37,7 @@ scrape_configs: In case you want to enforce authenticated scrapes only, use this configuration instead: ``` docker run -p 8080:8080 \ - ghcr.io/skuzzle/gh-prom-exporter/gh-prom-exporter:0.0.4 + ghcr.io/skuzzle/gh-prom-exporter/gh-prom-exporter:0.0.5 ``` And diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 43b71f8..a5e7aae 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -1,11 +1,10 @@ -[![Coverage Status](https://coveralls.io/repos/github/skuzzle/gh-prom-exporter/badge.svg?branch=master)](https://coveralls.io/github/skuzzle/gh-prom-exporter?branch=master) -[![Twitter Follow](https://img.shields.io/twitter/follow/skuzzleOSS.svg?style=social)](https://twitter.com/skuzzleOSS) +[![Coverage Status](https://coveralls.io/repos/github/skuzzle/gh-prom-exporter/badge.svg?branch=master)](https://coveralls.io/github/skuzzle/gh-prom-exporter?branch=master) [![Twitter Follow](https://img.shields.io/twitter/follow/skuzzleOSS.svg?style=social)](https://twitter.com/skuzzleOSS) -* Configuration option to allow anonymous scraping -* Minimal documentation for on-premise setup +* Update to Spring-Boot 2.6.2 +* [#1](https://github.com/skuzzle/gh-prom-exporter/issues/1) Allow to scrape multiple repository with single call ``` -docker pull ghcr.io/skuzzle/gh-prom-exporter/gh-prom-exporter:0.0.4 +docker pull ghcr.io/skuzzle/gh-prom-exporter/gh-prom-exporter:0.0.5 ``` \ No newline at end of file diff --git a/pom.xml b/pom.xml index 1482033..bb5fd72 100644 --- a/pom.xml +++ b/pom.xml @@ -5,13 +5,13 @@ de.skuzzle skuzzle-parent - 3.0.2-SNAPSHOT + 3.0.2 de.skuzzle.ghpromexporter gh-prom-exporter - 0.0.4 + 0.0.5 gh-prom-exporter Export GitHub repository metrics in prometheus format @@ -34,7 +34,7 @@ ghcr.io ${docker.registry.name}/${github.user}/${github.name}/${project.artifactId} - 2.6.1 + 2.6.2 2021.0.0-RC1 31.0.1-jre 1.135 diff --git a/readme/README.md b/readme/README.md index 9a2c609..531feeb 100644 --- a/readme/README.md +++ b/readme/README.md @@ -8,27 +8,28 @@ Export your favorite GitHub repositories to Prometheus -Use it _as a service_: https://gh.skuzzle.de +* Use it _as a service_: See https://gh.skuzzle.de for instructions +* Deploy it _on-premise_: `docker pull ${docker.image.name}:${project.version}` ## On-Premise deployment with docker This application can easily be run as a docker container in whatever environment you like: ``` docker run -p 8080:8080 \ - -e WEB_ALLOWANONYMOUSSCRAPE=true + -e WEB_ALLOWANONYMOUSSCRAPE=true \ ${docker.image.name}:${project.version} ``` With _anonymous scraping_ allowed, you can now easily view the scrape results directly in the browser by navigating to `https://your.docker.host:8080/YOUR-GITHUB-USERNAME/YOUR-REPOSITORY`. -The scraped repository can just as easiy be added as static scrape target to your prometheus' scrape configs: +The scraped repository can just as easy be added as static scrape target to your prometheus' scrape configs: ``` scrape_configs: - job_name: CHANGE_ME scrape_interval: 2m - metrics_path: /YOUR-GITHUB-USERNAME/YOUR-REPOSITORY + metrics_path: /YOUR-GITHUB-USERNAME/YOUR-REPOSITORY1,YOUR-REPOSITORY2 static_configs: - targets: ['your.docker.host:8080'] ``` diff --git a/readme/RELEASE_NOTES.md b/readme/RELEASE_NOTES.md index 0b63843..a029a0d 100644 --- a/readme/RELEASE_NOTES.md +++ b/readme/RELEASE_NOTES.md @@ -1,10 +1,9 @@ -[![Coverage Status](https://coveralls.io/repos/github/${github.user}/${github.name}/badge.svg?branch=${github.main-branch})](https://coveralls.io/github/${github.user}/${github.name}?branch=${github.main-branch}) -[![Twitter Follow](https://img.shields.io/twitter/follow/skuzzleOSS.svg?style=social)](https://twitter.com/skuzzleOSS) +[![Coverage Status](https://coveralls.io/repos/github/${github.user}/${github.name}/badge.svg?branch=${github.main-branch})](https://coveralls.io/github/${github.user}/${github.name}?branch=${github.main-branch}) [![Twitter Follow](https://img.shields.io/twitter/follow/skuzzleOSS.svg?style=social)](https://twitter.com/skuzzleOSS) -* Configuration option to allow anonymous scraping -* Minimal documentation for on-premise setup +* Update to Spring-Boot 2.6.2 +* [#1](https://github.com/skuzzle/gh-prom-exporter/issues/1) Allow to scrape multiple repository with single call ``` docker pull ${docker.image.name}:${project.version} diff --git a/src/main/java/de/skuzzle/ghpromexporter/clock/ApplicationClock.java b/src/main/java/de/skuzzle/ghpromexporter/clock/ApplicationClock.java deleted file mode 100644 index 37f9196..0000000 --- a/src/main/java/de/skuzzle/ghpromexporter/clock/ApplicationClock.java +++ /dev/null @@ -1,11 +0,0 @@ -package de.skuzzle.ghpromexporter.clock; - -import java.time.Clock; - -@FunctionalInterface -public interface ApplicationClock { - - static final ApplicationClock DEFAULT = StaticApplicationClock::get; - - Clock get(); -} diff --git a/src/main/java/de/skuzzle/ghpromexporter/clock/ClockConfiguration.java b/src/main/java/de/skuzzle/ghpromexporter/clock/ClockConfiguration.java deleted file mode 100644 index a913a7f..0000000 --- a/src/main/java/de/skuzzle/ghpromexporter/clock/ClockConfiguration.java +++ /dev/null @@ -1,13 +0,0 @@ -package de.skuzzle.ghpromexporter.clock; - -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Configuration; - -@Configuration(proxyBeanMethods = false) -public class ClockConfiguration { - - @Bean - ApplicationClock applicationClock() { - return ApplicationClock.DEFAULT; - } -} diff --git a/src/main/java/de/skuzzle/ghpromexporter/clock/StaticApplicationClock.java b/src/main/java/de/skuzzle/ghpromexporter/clock/StaticApplicationClock.java deleted file mode 100644 index 8a79e1a..0000000 --- a/src/main/java/de/skuzzle/ghpromexporter/clock/StaticApplicationClock.java +++ /dev/null @@ -1,51 +0,0 @@ -package de.skuzzle.ghpromexporter.clock; - -import java.time.Clock; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import com.google.common.base.Preconditions; - -public final class StaticApplicationClock { - - private static final Logger logger = LoggerFactory.getLogger(StaticApplicationClock.class); - - private StaticApplicationClock() { - // hidden - } - - private static volatile Clock APPLICATION_CLOCK = Clock.systemUTC(); - - public static Clock get() { - if (logger.isDebugEnabled()) { - final boolean clockChanged = !APPLICATION_CLOCK.equals(Clock.systemUTC()); - if (clockChanged) { - logger.debug("Accessed global application clock which has been changed from UTC to {}", - APPLICATION_CLOCK); - } - } - return APPLICATION_CLOCK; - } - - public static void resetToDefaultClock() { - final boolean clockChanged = !APPLICATION_CLOCK.equals(Clock.systemUTC()); - if (clockChanged) { - APPLICATION_CLOCK = Clock.systemUTC(); - logger.info("Reset Application clock to UTC"); - } - } - - public static void changeTo(Clock clock) { - Preconditions.checkArgument(clock != null, "global StaticApplicationClock can not be null"); - if (clock.equals(APPLICATION_CLOCK)) { - // early return to avoid warning log message - return; - } - APPLICATION_CLOCK = clock; - final boolean clockChanged = !APPLICATION_CLOCK.equals(Clock.systemUTC()); - if (clockChanged) { - logger.warn("Application clock has been changed globally to {}", clock); - } - } -} diff --git a/src/main/java/de/skuzzle/ghpromexporter/scrape/RepositoryMeters.java b/src/main/java/de/skuzzle/ghpromexporter/scrape/RepositoryMeters.java new file mode 100644 index 0000000..6114a00 --- /dev/null +++ b/src/main/java/de/skuzzle/ghpromexporter/scrape/RepositoryMeters.java @@ -0,0 +1,66 @@ +package de.skuzzle.ghpromexporter.scrape; + +import io.prometheus.client.CollectorRegistry; +import io.prometheus.client.Counter; +import io.prometheus.client.Summary; + +public final class RepositoryMeters { + + private static final String LABEL_REPOSITORY = "repository"; + private static final String LABEL_OWNER = "owner"; + private static final String NAMESPACE = "github"; + + private final CollectorRegistry registry; + private final Counter additions; + private final Counter deletions; + private final Counter stargazers; + private final Counter forks; + private final Counter open_issues; + private final Counter subscribers; + private final Counter watchers; + private final Counter size; + private final Summary scrapeDuration; + + public static RepositoryMeters newRegistry() { + return new RepositoryMeters(new CollectorRegistry()); + } + + private RepositoryMeters(CollectorRegistry registry) { + this.registry = registry; + this.additions = Counter.build("additions", "Sum of additions over the last 52 weeks") + .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); + this.deletions = Counter.build("deletions", "Negative sum of deletions over the last 52 weeks") + .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); + this.stargazers = Counter.build("stargazers", "The repository's stargazer count") + .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); + this.forks = Counter.build("forks", "The repository's fork count") + .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); + this.open_issues = Counter.build("open_issues", "The repository's open issue count") + .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); + this.subscribers = Counter.build("subscribers", "The repository's subscriber count") + .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); + this.watchers = Counter.build("watchers", "The repository's watcher count") + .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); + this.size = Counter.build("size", "The repository's size in KB") + .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); + this.scrapeDuration = Summary.build("scrape_duration", "Duration of a single scrape") + .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); + } + + public RepositoryMeters addRepositoryScrapeResults(ScrapeRepositoryRequest repository, RepositoryMetrics metrics) { + additions.labels(repository.owner(), repository.name()).inc(metrics.totalAdditions()); + deletions.labels(repository.owner(), repository.name()).inc(metrics.totalDeletions()); + stargazers.labels(repository.owner(), repository.name()).inc(metrics.stargazersCount()); + forks.labels(repository.owner(), repository.name()).inc(metrics.forkCount()); + open_issues.labels(repository.owner(), repository.name()).inc(metrics.openIssueCount()); + subscribers.labels(repository.owner(), repository.name()).inc(metrics.subscriberCount()); + watchers.labels(repository.owner(), repository.name()).inc(metrics.watchersCount()); + size.labels(repository.owner(), repository.name()).inc(metrics.sizeInKb()); + scrapeDuration.labels(repository.owner(), repository.name()).observe(metrics.scrapeDuration()); + return this; + } + + public CollectorRegistry registry() { + return this.registry; + } +} diff --git a/src/main/java/de/skuzzle/ghpromexporter/scrape/RepositoryMetrics.java b/src/main/java/de/skuzzle/ghpromexporter/scrape/RepositoryMetrics.java index a4a8b1a..e8f51cb 100644 --- a/src/main/java/de/skuzzle/ghpromexporter/scrape/RepositoryMetrics.java +++ b/src/main/java/de/skuzzle/ghpromexporter/scrape/RepositoryMetrics.java @@ -1,9 +1,5 @@ package de.skuzzle.ghpromexporter.scrape; -import io.prometheus.client.CollectorRegistry; -import io.prometheus.client.Counter; -import io.prometheus.client.Summary; - public record RepositoryMetrics( long totalAdditions, long totalDeletions, @@ -13,61 +9,4 @@ public record RepositoryMetrics( int subscriberCount, int watchersCount, int sizeInKb, - long scrapeDuration) { - - public CollectorRegistry toRegistry(ScrapeRepositoryRequest repository) { - final Meters meters = new Meters(); - meters.additions.labels(repository.owner(), repository.name()) - .inc(totalAdditions()); - meters.deletions.labels(repository.owner(), repository.name()) - .inc(totalDeletions()); - meters.stargazers.labels(repository.owner(), repository.name()).inc(stargazersCount()); - meters.forks.labels(repository.owner(), repository.name()).inc(forkCount()); - meters.open_issues.labels(repository.owner(), repository.name()).inc(openIssueCount()); - meters.subscribers.labels(repository.owner(), repository.name()).inc(subscriberCount()); - meters.watchers.labels(repository.owner(), repository.name()).inc(watchersCount()); - meters.size.labels(repository.owner(), repository.name()).inc(sizeInKb()); - meters.scrapeDuration.labels(repository.owner(), repository.name()).observe(scrapeDuration()); - return meters.registry; - } - - private static class Meters { - private static final String LABEL_REPOSITORY = "repository"; - private static final String LABEL_OWNER = "owner"; - private static final String NAMESPACE = "github"; - - private final CollectorRegistry registry; - private final Counter additions; - private final Counter deletions; - private final Counter stargazers; - private final Counter forks; - private final Counter open_issues; - private final Counter subscribers; - private final Counter watchers; - private final Counter size; - private final Summary scrapeDuration; - - public Meters() { - this.registry = new CollectorRegistry(); - this.additions = Counter.build("additions", "TBD") - .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); - this.deletions = Counter.build("deletions", "TBD") - .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); - this.stargazers = Counter.build("stargazers", "The repository's stargazer count") - .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); - this.forks = Counter.build("forks", "The repository's fork count") - .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); - this.open_issues = Counter.build("open_issues", "The repository's open issue count") - .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); - this.subscribers = Counter.build("subscribers", "The repository's subscriber count") - .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); - this.watchers = Counter.build("watchers", "The repository's watcher count") - .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); - this.size = Counter.build("size", "The repository's size in KB") - .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); - this.scrapeDuration = Summary.build("scrape_duration", "Duration of a single scrape") - .namespace(NAMESPACE).labelNames(LABEL_OWNER, LABEL_REPOSITORY).register(registry); - } - - } -} + long scrapeDuration) {} diff --git a/src/main/java/de/skuzzle/ghpromexporter/web/MultipleRepositories.java b/src/main/java/de/skuzzle/ghpromexporter/web/MultipleRepositories.java new file mode 100644 index 0000000..97b5c61 --- /dev/null +++ b/src/main/java/de/skuzzle/ghpromexporter/web/MultipleRepositories.java @@ -0,0 +1,28 @@ +package de.skuzzle.ghpromexporter.web; + +import java.util.Arrays; +import java.util.List; + +import de.skuzzle.ghpromexporter.scrape.ScrapeRepositoryRequest; +import reactor.core.publisher.Flux; + +final class MultipleRepositories { + + private final String owner; + private final List repositories; + + private MultipleRepositories(String owner, List repositories) { + this.owner = owner; + this.repositories = repositories; + } + + public static MultipleRepositories parse(String owner, String repositoriesString) { + final String[] repositories = repositoriesString.split(","); + return new MultipleRepositories(owner, Arrays.asList(repositories)); + } + + Flux requests() { + return Flux.fromStream(repositories.stream() + .map(repository -> ScrapeRepositoryRequest.of(owner, repository))); + } +} diff --git a/src/main/java/de/skuzzle/ghpromexporter/web/PromController.java b/src/main/java/de/skuzzle/ghpromexporter/web/PromController.java index ca8633d..910ce92 100644 --- a/src/main/java/de/skuzzle/ghpromexporter/web/PromController.java +++ b/src/main/java/de/skuzzle/ghpromexporter/web/PromController.java @@ -13,7 +13,8 @@ import de.skuzzle.ghpromexporter.github.AuthenticationProvider; import de.skuzzle.ghpromexporter.github.GitHubAuthentication; import de.skuzzle.ghpromexporter.scrape.AsynchronousScrapeService; -import de.skuzzle.ghpromexporter.scrape.ScrapeRepositoryRequest; +import de.skuzzle.ghpromexporter.scrape.RepositoryMeters; +import io.prometheus.client.CollectorRegistry; import reactor.core.publisher.Mono; @RestController @@ -24,10 +25,10 @@ record PromController( AbuseLimiter abuseLimiter, WebProperties properties) { - @GetMapping(path = "{user}/{repo}") + @GetMapping(path = "{owner}/{repositories}") public Mono> createStats( - @PathVariable String user, - @PathVariable String repo, + @PathVariable String owner, + @PathVariable String repositories, ServerHttpRequest request) { final GitHubAuthentication gitHubAuthentication = authenticationProvider.authenticateRequest(request); @@ -39,10 +40,11 @@ public Mono> createStats( final InetAddress origin = request.getRemoteAddress().getAddress(); final MediaType contentType = MediaType.TEXT_PLAIN;// determineContentType(request); - final ScrapeRepositoryRequest scrapeRepositoryRequest = ScrapeRepositoryRequest.of(user, repo); + + final MultipleRepositories multipleRepositories = MultipleRepositories.parse(owner, repositories); return abuseLimiter.blockAbusers(origin) - .flatMap(__ -> freshResponse(gitHubAuthentication, scrapeRepositoryRequest, contentType)) + .flatMap(__ -> freshResponse(gitHubAuthentication, multipleRepositories, contentType)) .doOnError(e -> abuseLimiter.recordCall(e, origin)) .onErrorResume(e -> Mono.just(ResponseEntity.badRequest().body(e.getMessage()))) .switchIfEmpty( @@ -52,11 +54,21 @@ public Mono> createStats( } private Mono> freshResponse(GitHubAuthentication authentication, - ScrapeRepositoryRequest request, MediaType contentType) { - return scrapeService.scrapeReactive(authentication, request) - .map(result -> serializer.serializeRegistry(result.toRegistry(request), contentType)) + MultipleRepositories repositories, MediaType contentType) { + + return scrapeAll(authentication, repositories) + .map(registry -> serializer.serializeRegistry(registry, contentType)) .map(serializedMetrics -> ResponseEntity.ok() .contentType(contentType) .body(serializedMetrics)); } + + private Mono scrapeAll(GitHubAuthentication authentication, MultipleRepositories repositories) { + final RepositoryMeters meters = RepositoryMeters.newRegistry(); + + return repositories.requests() + .flatMap(req -> scrapeService.scrapeReactive(authentication, req) + .doOnNext(scrapeResult -> meters.addRepositoryScrapeResults(req, scrapeResult))) + .then(Mono.just(meters.registry())); + } } diff --git a/src/main/java/de/skuzzle/ghpromexporter/web/RegistrySerializer.java b/src/main/java/de/skuzzle/ghpromexporter/web/RegistrySerializer.java index db3921b..6dd9e21 100644 --- a/src/main/java/de/skuzzle/ghpromexporter/web/RegistrySerializer.java +++ b/src/main/java/de/skuzzle/ghpromexporter/web/RegistrySerializer.java @@ -17,7 +17,7 @@ class RegistrySerializer { public String serializeRegistry(CollectorRegistry registry, MediaType mediaType) { try (final var stringWriter = new StringWriter()) { - if (mediaType.equals(OPEN_METRICS)) { + if (mediaType.isCompatibleWith(OPEN_METRICS)) { TextFormat.writeOpenMetrics100(stringWriter, registry.metricFamilySamples()); } else { TextFormat.write004(stringWriter, registry.metricFamilySamples()); diff --git a/src/main/resources/templates/index.html b/src/main/resources/templates/index.html index 704499b..3db1e58 100644 --- a/src/main/resources/templates/index.html +++ b/src/main/resources/templates/index.html @@ -105,8 +105,8 @@

FAQ

  • I don't want to use the public service. Can I host this on-premise?

    - Not yet, but technically everything is prepared so that you can run this application yourself in a - environment that supports docker containers. + Yes, you can run this exact application yourself using Docker. You can find a detailed explanation on + GitHub.

  • @@ -171,6 +171,7 @@

    FAQ

    \ No newline at end of file diff --git a/src/test/java/de/skuzzle/ghpromexporter/web/PromControllerTest.java b/src/test/java/de/skuzzle/ghpromexporter/web/PromControllerTest.java index 4aae4a1..cc40dc0 100644 --- a/src/test/java/de/skuzzle/ghpromexporter/web/PromControllerTest.java +++ b/src/test/java/de/skuzzle/ghpromexporter/web/PromControllerTest.java @@ -16,6 +16,7 @@ import org.springframework.http.ResponseEntity; import org.springframework.web.reactive.function.client.WebClient; +import de.skuzzle.ghpromexporter.github.GitHubAuthentication; import de.skuzzle.test.snapshots.SnapshotAssertions; import de.skuzzle.test.snapshots.SnapshotDsl.Snapshot; import reactor.core.publisher.Mono; @@ -56,60 +57,83 @@ private Mono> getStatsFor(String owner, String repository @Test void scrape_anonymously_forbidden() throws Exception { final var serviceCall = getStatsFor("skuzzle", "test-repo"); + final GitHubAuthentication gitHubAuthentication = successfulAuthenticationForRepository( + withName("skuzzle", "test-repo") + .withStargazerCount(1337)) + .setAnonymous(true); - authentication.with(successfulAuthenticationForRepository(withName("skuzzle", "test-repo") - .withStargazerCount(1337)).setAnonymous(true), () -> { + authentication.with(gitHubAuthentication, () -> { + StepVerifier.create(serviceCall) + .assertNext( + response -> assertThat(response.getStatusCode()).isEqualTo(HttpStatus.UNAUTHORIZED)) + .verifyComplete(); + }); + } - StepVerifier.create(serviceCall) - .assertNext( - response -> assertThat(response.getStatusCode()).isEqualTo(HttpStatus.UNAUTHORIZED)) - .verifyComplete(); - }); + @Test + void scrape_multiple_repositories(Snapshot snapshot) throws Exception { + final var serviceCall = getStatsFor("skuzzle", "test-repo1,test-repo2"); + final var gitHubAuthentication = successfulAuthenticationForRepository( + withName("skuzzle", "test-repo1") + .withForkCount(5)); + + gitHubAuthentication.modify().withRepository(withName("skuzzle", "test-repo2").withStargazerCount(1337)); + authentication.with(gitHubAuthentication, () -> { + StepVerifier.create(serviceCall) + .assertNext(response -> snapshot.assertThat(response.getBody()) + .as(canonicalPrometheusRegistry()) + .matchesSnapshotText()) + .verifyComplete(); + }); } @Test void test_successful_initial_scrape(Snapshot snapshot) throws Exception { final var serviceCall = getStatsFor("skuzzle", "test-repo"); + final GitHubAuthentication gitHubAuthentication = successfulAuthenticationForRepository( + withName("skuzzle", "test-repo") + .withStargazerCount(1337) + .withForkCount(5) + .withOpenIssueCount(2) + .withWatchersCount(1) + .withSubscriberCount(4) + .withAdditions(50) + .withDeletions(-20) + .withSizeInKb(127)); + + authentication.with(gitHubAuthentication, () -> { - authentication.with(successfulAuthenticationForRepository(withName("skuzzle", "test-repo") - .withStargazerCount(1337) - .withForkCount(5) - .withOpenIssueCount(2) - .withWatchersCount(1) - .withSubscriberCount(4) - .withAdditions(50) - .withDeletions(-20) - .withSizeInKb(127)), () -> { - - StepVerifier.create(serviceCall) - .assertNext(response -> snapshot.assertThat(response.getBody()) - .as(canonicalPrometheusRegistry()) - .matchesSnapshotText()) - .verifyComplete(); - }); + StepVerifier.create(serviceCall) + .assertNext(response -> snapshot.assertThat(response.getBody()) + .as(canonicalPrometheusRegistry()) + .matchesSnapshotText()) + .verifyComplete(); + }); } @Test void test_successful_anonymous_scrape(Snapshot snapshot) throws Exception { final var serviceCall = getStatsFor("skuzzle", "test-repo"); webProperties.setAllowAnonymousScrape(true); + final GitHubAuthentication gitHubAuthentication = successfulAuthenticationForRepository( + withName("skuzzle", "test-repo") + .withStargazerCount(1337) + .withForkCount(5) + .withOpenIssueCount(2) + .withWatchersCount(1) + .withSubscriberCount(4) + .withAdditions(50) + .withDeletions(-20) + .withSizeInKb(127)).setAnonymous(true); + + authentication.with(gitHubAuthentication, () -> { - authentication.with(successfulAuthenticationForRepository(withName("skuzzle", "test-repo") - .withStargazerCount(1337) - .withForkCount(5) - .withOpenIssueCount(2) - .withWatchersCount(1) - .withSubscriberCount(4) - .withAdditions(50) - .withDeletions(-20) - .withSizeInKb(127)).setAnonymous(true), () -> { - - StepVerifier.create(serviceCall) - .assertNext(response -> snapshot.assertThat(response.getBody()) - .as(canonicalPrometheusRegistry()) - .matchesSnapshotText()) - .verifyComplete(); - }); + StepVerifier.create(serviceCall) + .assertNext(response -> snapshot.assertThat(response.getBody()) + .as(canonicalPrometheusRegistry()) + .matchesSnapshotText()) + .verifyComplete(); + }); } @Test diff --git a/src/test/resources/de/skuzzle/ghpromexporter/web/PromControllerTest_snapshots/scrape_multiple_repositories_0.snapshot b/src/test/resources/de/skuzzle/ghpromexporter/web/PromControllerTest_snapshots/scrape_multiple_repositories_0.snapshot new file mode 100644 index 0000000..0c80036 --- /dev/null +++ b/src/test/resources/de/skuzzle/ghpromexporter/web/PromControllerTest_snapshots/scrape_multiple_repositories_0.snapshot @@ -0,0 +1,54 @@ +# HELP github_additions_created Sum of additions over the last 52 weeks +# HELP github_additions_total Sum of additions over the last 52 weeks +# HELP github_deletions_created Negative sum of deletions over the last 52 weeks +# HELP github_deletions_total Negative sum of deletions over the last 52 weeks +# HELP github_forks_created The repository's fork count +# HELP github_forks_total The repository's fork count +# HELP github_open_issues_created The repository's open issue count +# HELP github_open_issues_total The repository's open issue count +# HELP github_scrape_duration Duration of a single scrape +# HELP github_scrape_duration_created Duration of a single scrape +# HELP github_size_created The repository's size in KB +# HELP github_size_total The repository's size in KB +# HELP github_stargazers_created The repository's stargazer count +# HELP github_stargazers_total The repository's stargazer count +# HELP github_subscribers_created The repository's subscriber count +# HELP github_subscribers_total The repository's subscriber count +# HELP github_watchers_created The repository's watcher count +# HELP github_watchers_total The repository's watcher count +# TYPE github_additions_created gauge +# TYPE github_additions_total counter +# TYPE github_deletions_created gauge +# TYPE github_deletions_total counter +# TYPE github_forks_created gauge +# TYPE github_forks_total counter +# TYPE github_open_issues_created gauge +# TYPE github_open_issues_total counter +# TYPE github_scrape_duration summary +# TYPE github_scrape_duration_created gauge +# TYPE github_size_created gauge +# TYPE github_size_total counter +# TYPE github_stargazers_created gauge +# TYPE github_stargazers_total counter +# TYPE github_subscribers_created gauge +# TYPE github_subscribers_total counter +# TYPE github_watchers_created gauge +# TYPE github_watchers_total counter +github_additions_total{owner="skuzzle",repository="test-repo1",} 0.0 +github_additions_total{owner="skuzzle",repository="test-repo2",} 0.0 +github_deletions_total{owner="skuzzle",repository="test-repo1",} 0.0 +github_deletions_total{owner="skuzzle",repository="test-repo2",} 0.0 +github_forks_total{owner="skuzzle",repository="test-repo1",} 5.0 +github_forks_total{owner="skuzzle",repository="test-repo2",} 0.0 +github_open_issues_total{owner="skuzzle",repository="test-repo1",} 0.0 +github_open_issues_total{owner="skuzzle",repository="test-repo2",} 0.0 +github_scrape_duration_count{owner="skuzzle",repository="test-repo1",} 1.0 +github_scrape_duration_count{owner="skuzzle",repository="test-repo2",} 1.0 +github_size_total{owner="skuzzle",repository="test-repo1",} 0.0 +github_size_total{owner="skuzzle",repository="test-repo2",} 0.0 +github_stargazers_total{owner="skuzzle",repository="test-repo1",} 0.0 +github_stargazers_total{owner="skuzzle",repository="test-repo2",} 1337.0 +github_subscribers_total{owner="skuzzle",repository="test-repo1",} 0.0 +github_subscribers_total{owner="skuzzle",repository="test-repo2",} 0.0 +github_watchers_total{owner="skuzzle",repository="test-repo1",} 0.0 +github_watchers_total{owner="skuzzle",repository="test-repo2",} 0.0 \ No newline at end of file diff --git a/src/test/resources/de/skuzzle/ghpromexporter/web/PromControllerTest_snapshots/test_successful_anonymous_scrape_0.snapshot b/src/test/resources/de/skuzzle/ghpromexporter/web/PromControllerTest_snapshots/test_successful_anonymous_scrape_0.snapshot index c230075..7669068 100644 --- a/src/test/resources/de/skuzzle/ghpromexporter/web/PromControllerTest_snapshots/test_successful_anonymous_scrape_0.snapshot +++ b/src/test/resources/de/skuzzle/ghpromexporter/web/PromControllerTest_snapshots/test_successful_anonymous_scrape_0.snapshot @@ -1,7 +1,7 @@ -# HELP github_additions_created TBD -# HELP github_additions_total TBD -# HELP github_deletions_created TBD -# HELP github_deletions_total TBD +# HELP github_additions_created Sum of additions over the last 52 weeks +# HELP github_additions_total Sum of additions over the last 52 weeks +# HELP github_deletions_created Negative sum of deletions over the last 52 weeks +# HELP github_deletions_total Negative sum of deletions over the last 52 weeks # HELP github_forks_created The repository's fork count # HELP github_forks_total The repository's fork count # HELP github_open_issues_created The repository's open issue count diff --git a/src/test/resources/de/skuzzle/ghpromexporter/web/PromControllerTest_snapshots/test_successful_initial_scrape_0.snapshot b/src/test/resources/de/skuzzle/ghpromexporter/web/PromControllerTest_snapshots/test_successful_initial_scrape_0.snapshot index c230075..7669068 100644 --- a/src/test/resources/de/skuzzle/ghpromexporter/web/PromControllerTest_snapshots/test_successful_initial_scrape_0.snapshot +++ b/src/test/resources/de/skuzzle/ghpromexporter/web/PromControllerTest_snapshots/test_successful_initial_scrape_0.snapshot @@ -1,7 +1,7 @@ -# HELP github_additions_created TBD -# HELP github_additions_total TBD -# HELP github_deletions_created TBD -# HELP github_deletions_total TBD +# HELP github_additions_created Sum of additions over the last 52 weeks +# HELP github_additions_total Sum of additions over the last 52 weeks +# HELP github_deletions_created Negative sum of deletions over the last 52 weeks +# HELP github_deletions_total Negative sum of deletions over the last 52 weeks # HELP github_forks_created The repository's fork count # HELP github_forks_total The repository's fork count # HELP github_open_issues_created The repository's open issue count