Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: JDBC Url INFO logging #33498

Open
2 of 17 tasks
daniel-deychakiwsky opened this issue Jan 5, 2025 · 2 comments · May be fixed by #33514
Open
2 of 17 tasks

[Bug]: JDBC Url INFO logging #33498

daniel-deychakiwsky opened this issue Jan 5, 2025 · 2 comments · May be fixed by #33514

Comments

@daniel-deychakiwsky
Copy link

daniel-deychakiwsky commented Jan 5, 2025

What happened?

I'm using the python beam WriteToJdbc transform with Google Cloud providing the cloud-sql-jdbc-socket-factory Postgres fat jar via the transforms classpath parameter that outline the format of the JDBC url as:

String jdbcUrl = "jdbc:postgresql:///<DATABASE_NAME>?" 
    + "cloudSqlInstance=<INSTANCE_CONNECTION_NAME>" 
    + "&socketFactory=com.google.cloud.sql.postgres.SocketFactory" 
    + "&user=<POSTGRESQL_USER_NAME>" 
    + "&password=<POSTGRESQL_USER_PASSWORD>";

This works as intended in my pipeline, however, I get unwanted INFO log statements from beam Java likely from here: "Failed to parse JdbcUrl {}. Lineage will not be reported.". The cloud-sql-jdbc-socket-factory documents indicate that part of the JDBC url is not used: "Note: The host portion of the JDBC URL is currently unused, and has no effect on the connection process. The SocketFactory will get your instances IP address based on the provided cloudSqlInstance arg.". Perhaps this is correct behavior to log this if cloud-sql-jdbc-socket-factory wants the JDBC URL to be that specific format when indeed this isn't a valid format (as INFO logged from beam Java). Any thoughts on this?

Issue Priority

Priority: 3 (minor)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner
@liferoad
Copy link
Collaborator

liferoad commented Jan 6, 2025

cc @Abacn

@Abacn
Copy link
Contributor

Abacn commented Jan 7, 2025

Thanks for reporting this! Yes it is a bug that a piece of metrics code is supposed to run once but actually on every read/write.

@Abacn Abacn added this to the 2.63.0 Release milestone Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants