Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add product tests for clickhouse clusters #10956

Merged

Conversation

tangjiangling
Copy link
Member

@tangjiangling tangjiangling commented Feb 4, 2022

Related issues, pull requests, and links

Fixes #8391

Documentation

No documentation is needed.

Release notes

No release notes entries required.

@cla-bot cla-bot bot added the cla-signed label Feb 4, 2022
@tangjiangling
Copy link
Member Author

tangjiangling commented Feb 4, 2022

Question

  1. Do we need to add a description of multinode-clickhouse in the README? I see that some of the connector product tests are not described in the README either (e.g. sqlserver), so I'm not sure if clickhouse is needed.
  2. Do we need to enable product testing for clickhouse now? I have it enabled now, see 2dad786

@tangjiangling
Copy link
Member Author

tangjiangling commented Feb 5, 2022

https://github.com/trinodb/trino/runs/5072994398?check_suite_focus=true

I need help. How do I change /etc/hosts on CI machine?

tests               | Caused by: java.net.UnknownHostException: clickhouse
tests               | 	at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:220)
tests               | 	at java.base/java.net.Socket.connect(Socket.java:609)
tests               | 	at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177)
tests               | 	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
tests               | 	at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
tests               | 	at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
tests               | 	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:341)
tests               | 	at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:362)
tests               | 	at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1253)
tests               | 	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1187)
tests               | 	at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)
tests               | 	at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1015)
tests               | 	at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1367)
tests               | 	at java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1342)
tests               | 	at com.clickhouse.client.http.HttpUrlConnectionImpl.post(HttpUrlConnectionImpl.java:158)
tests               | 	at com.clickhouse.client.http.ClickHouseHttpClient.postRequest(ClickHouseHttpClient.java:88)
tests               | 	at com.clickhouse.client.http.ClickHouseHttpClient.execute(ClickHouseHttpClient.java:115)
tests               | 	at com.clickhouse.client.ClickHouseRequest.execute(ClickHouseRequest.java:1375)
tests               | 	at com.clickhouse.jdbc.internal.ClickHouseConnectionImpl.<init>(ClickHouseConnectionImpl.java:200)
tests               | 	... 44 more

@tangjiangling tangjiangling force-pushed the add-product-tests-for-clickhouse-clusters branch from 195836c to 754b3ec Compare February 5, 2022 16:33
@tangjiangling
Copy link
Member Author

Copy link
Member

@ebyhr ebyhr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the main motivations for ClickHouse product tests is #7600, but INSERT statement failed as far as I confirmed. Do you have any idea why this happened? Invalid table definition?

trino> INSERT INTO clickhouse.default.test_distributed VALUES (1, 1, 1.0);
Query 20220207_015644_00008_bv244 failed: Failed to insert data: Code: 55, e.displayText() = DB::Exception: Method write is not supported by storage Distributed with more than one shard and no sharding key provided (version 21.3.2.5 (official build))
, server ClickHouseNode(addr=http:clickhouse:8124, db=default)@311454579

@tangjiangling
Copy link
Member Author

Do you have any idea why this happened? Invalid table definition?

Yes, I assume you are using this PR configuration for ClickHouse cluster (3 shards 1 replica), I guess because you have a problem with the table definition (no sharding_key configured), when the cluster has multiple shards (>1), you must specify the shard key, you can refer to this document.

Here is a description from the document that I quoted.

Second, you can perform INSERT statements on a Distributed table. In this case, the table will distribute the inserted data across the servers itself. In order to write to a Distributed table, it must have the sharding_key parameter configured (except if there is only one shard).

@tangjiangling tangjiangling force-pushed the add-product-tests-for-clickhouse-clusters branch 2 times, most recently from e9faa4b to 81c741d Compare February 8, 2022 19:08
@tangjiangling tangjiangling force-pushed the add-product-tests-for-clickhouse-clusters branch from 81c741d to 05a637c Compare February 9, 2022 14:16
@tangjiangling tangjiangling force-pushed the add-product-tests-for-clickhouse-clusters branch from 05a637c to c25bf44 Compare February 9, 2022 15:19
@ebyhr ebyhr merged commit a99c77d into trinodb:master Feb 12, 2022
@ebyhr
Copy link
Member

ebyhr commented Feb 12, 2022

Merged, thanks!

@github-actions github-actions bot added this to the 371 milestone Feb 12, 2022
@tangjiangling tangjiangling deleted the add-product-tests-for-clickhouse-clusters branch February 12, 2022 04:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Verify ClickHouse clusters can be queried as well in addition to singlenode setups
4 participants