Skip to content

Commit ea5c806

Browse files
joshelserjoshelser
committed
PHOENIX-5827 Add a maven repo to PQS and optionally bundle phoenix-client
* Move the cluster.xml to the prescribed location per Maven convention. * Use maven-assembly-plugin to create a Maven repo in the assembly * Pull the phoenix-client jar from central and localize so PQS can actually function * Fix the phoenix-client jar name so that it's picked up by phoenix_utils.py * Prevent old Jetty versions from sneaking in on HBase 1.x * Build the ServerCustomizer and expose configuration to enable it Move integration tests to a dedicated module to avoid jetty clashing With Hadoop2/HBase1, we have to deal with conflicting versions of Jetty at runtime. Avatica's ServerCustomizers expose an unshaded Jetty class (Server), which causes problems when we have Jetty6 also on the classpath. We can avoid this by moving all things that touch Avatica's version of Jetty into queryserver, shade that, and then only invoke HBase/Hadoop from within a different module (letting their Jetty6 run wild). The only gripe is that BasicAuthenticationServerCustomizer has to live in src/main/java to get shaded, even though it is test-only code. This is a minor grievance for what's a horrible runtime solution. This all gets much better with Hadoop3/HBase2 where Jetty is shaded. Closes apache#25 Signed-off-by: Istvan Toth <[email protected]>
1 parent ac320cf commit ea5c806

30 files changed

+781
-210
lines changed

‎.github/workflows/maven.yml

+1-1
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,4 @@ jobs:
1313
with:
1414
java-version: 1.8
1515
- name: Build with Maven
16-
run: mvn clean install
16+
run: mvn -B clean install

‎.gitignore

+2-1
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
*.class
33
*.war
44
*.jar
5+
dependency-reduced-pom.xml
56

67
# python
78
*.pyc
@@ -26,4 +27,4 @@ target/
2627
release/
2728
RESULTS/
2829
CSV_EXPORT/
29-
.DS_Store
30+
.DS_Store

‎README.md

+37-2
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,41 @@ limitations under the License.
1717

1818
![logo](https://phoenix.apache.org/images/phoenix-logo-small.png)
1919

20-
<b>[Apache Phoenix](http://phoenix.apache.org/)</b> enables OLTP and operational analytics in Hadoop for low latency applications. Visit the Apache Phoenix website <b>[here](http://phoenix.apache.org/)</b>. This is the repo for the Query Server.
20+
<b>[Apache Phoenix](http://phoenix.apache.org/)</b> enables OLTP and operational analytics in Hadoop for low latency applications. Visit the Apache Phoenix website <b>[here](http://phoenix.apache.org/)</b>. This is the repo for the Phoenix Query Server (PQS).
2121

22-
Copyright ©2019 [Apache Software Foundation](http://www.apache.org/). All Rights Reserved.
22+
Copyright ©2020 [Apache Software Foundation](http://www.apache.org/). All Rights Reserved.
23+
24+
## Introduction
25+
26+
The Phoenix Query Server is an JDBC over HTTP abstraction. The Phoenix Query Server proxies the standard
27+
Phoenix JDBC driver and provides a backwards-compatible wire protocol to invoke that JDBC driver. This is
28+
all done via the Apache Avatica project (sub-project of Apache Calcite).
29+
30+
The reference client implementation for PQS is a "thin" JDBC driver which can communicate with PQS. There
31+
are drivers in other languages which exist in varying levels of maturity including Python, Golang, and .NET.
32+
33+
## Building
34+
35+
This repository will build a tarball which is capable of running the Phoenix Query Server.
36+
37+
By default, this tarball does not contain a Phoenix client jar as it is meant to be agnostic
38+
of Phoenix version (one PQS release can be used against any Phoenix version). Today, PQS builds against
39+
the Phoenix 4.15.0-HBase-1.4 release.
40+
41+
```
42+
$ mvn package
43+
```
44+
45+
### Bundling a Phoenix Client
46+
47+
To build a release of PQS which packages a specific version of Phoenix, enable the `package-phoenix-client` profile
48+
and specify properties to indicate a specific Phoenix version.
49+
50+
By default, PQS will package the same version of Phoenix used for build/test. This version is controlled by the system
51+
property `phoenix.version` system property. Depending on the version of Phoenix, you may also be required to
52+
use the `phoenix.hbase.classifier` system property to identify the correct version of Phoenix built against
53+
the version of HBase of your choosing.
54+
55+
```
56+
$ mvn package -Dpackage.phoenix.client -Dphoenix.version=5.1.0-SNAPSHOT -Dphoenix.hbase.classifier=hbase-2.2
57+
```

‎assembly/pom.xml

+38-3
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,6 @@
5252
</dependencies>
5353

5454
<build>
55-
<directory>target</directory>
5655
<plugins>
5756
<plugin>
5857
<artifactId>maven-assembly-plugin</artifactId>
@@ -65,7 +64,7 @@
6564
</goals>
6665
<configuration>
6766
<descriptors>
68-
<descriptor>cluster.xml</descriptor>
67+
<descriptor>src/assembly/cluster.xml</descriptor>
6968
</descriptors>
7069
<finalName>${project.parent.artifactId}-${project.version}</finalName>
7170
<tarLongFileMode>posix</tarLongFileMode>
@@ -74,7 +73,43 @@
7473
</execution>
7574
</executions>
7675
</plugin>
76+
<plugin>
77+
<artifactId>maven-dependency-plugin</artifactId>
78+
<executions>
79+
<!-- Create a mini Maven repository so PQS can serve these jars like a Maven repo -->
80+
<execution>
81+
<id>prepare-client-repo</id>
82+
<!-- Make sure we build this before making the assembly -->
83+
<phase>prepare-package</phase>
84+
<goals>
85+
<goal>copy-dependencies</goal>
86+
</goals>
87+
<configuration>
88+
<includeArtifactIds>phoenix-client,queryserver-client</includeArtifactIds>
89+
<outputDirectory>${project.build.directory}/maven-repo</outputDirectory>
90+
<overWriteIfNewer>true</overWriteIfNewer>
91+
<useRepositoryLayout>true</useRepositoryLayout>
92+
</configuration>
93+
</execution>
94+
</executions>
95+
</plugin>
7796
</plugins>
7897
</build>
79-
98+
<profiles>
99+
<profile>
100+
<id>package-phoenix-client</id>
101+
<activation>
102+
<property>
103+
<name>package.phoenix.client</name>
104+
</property>
105+
</activation>
106+
<dependencies>
107+
<dependency>
108+
<groupId>org.apache.phoenix</groupId>
109+
<artifactId>phoenix-client</artifactId>
110+
<classifier>${phoenix.hbase.classifier}</classifier>
111+
</dependency>
112+
</dependencies>
113+
</profile>
114+
</profiles>
80115
</project>

‎assembly/cluster.xml ‎assembly/src/assembly/cluster.xml

+32-7
Original file line numberDiff line numberDiff line change
@@ -33,22 +33,21 @@
3333
<directory>${project.basedir}/../queryserver/target</directory>
3434
<outputDirectory>${project.parent.artifactId}-${project.parent.version}/queryserver/target</outputDirectory>
3535
<includes>
36-
<include>*.jar</include>
36+
<include>phoenix*.jar</include>
37+
<include>queryserver*.jar</include>
3738
</includes>
3839
</fileSet>
3940
<fileSet>
4041
<directory>${project.basedir}/../queryserver-client/target</directory>
4142
<outputDirectory>${project.parent.artifactId}-${project.parent.version}/queryserver-client/target</outputDirectory>
4243
<includes>
43-
<include>*.jar</include>
44+
<include>phoenix*.jar</include>
45+
<include>queryserver*.jar</include>
4446
</includes>
4547
</fileSet>
4648
<fileSet>
47-
<directory>${project.basedir}/../phoenix-client/target</directory>
48-
<outputDirectory>${project.parent.artifactId}-${project.parent.version}/phoenix-client/target</outputDirectory>
49-
<includes>
50-
<include>*.jar</include>
51-
</includes>
49+
<directory>${project.build.directory}/maven-repo</directory>
50+
<outputDirectory>${project.parent.artifactId}-${project.parent.version}/maven</outputDirectory>
5251
</fileSet>
5352
</fileSets>
5453
<dependencySets>
@@ -59,5 +58,31 @@
5958
<include>sqlline:sqlline:jar:jar-with-dependencies</include>
6059
</includes>
6160
</dependencySet>
61+
<dependencySet>
62+
<unpack>false</unpack>
63+
<outputDirectory>${project.parent.artifactId}-${project.parent.version}/</outputDirectory>
64+
<includes>
65+
<include>org.apache.phoenix:phoenix-client</include>
66+
</includes>
67+
<!-- Unwind the name from phoenix-client-$version to phonix-$version-client -->
68+
<outputFileNameMapping>phoenix-${artifact.version}${dashClassifier}-client.${artifact.extension}</outputFileNameMapping>
69+
</dependencySet>
70+
<dependencySet>
71+
<unpack>false</unpack>
72+
<outputDirectory>${project.parent.artifactId}-${project.parent.version}/queryserver/target</outputDirectory>
73+
<includes>
74+
<include>org.apache.phoenix:queryserver</include>
75+
</includes>
76+
<outputFileNameMapping>phoenix-${project.parent.version}-queryserver.${artifact.extension}</outputFileNameMapping>
77+
</dependencySet>
78+
<!-- Unwind the name from queryserver-client-$version to phoenix-$version-thin-client -->
79+
<dependencySet>
80+
<unpack>false</unpack>
81+
<outputDirectory>${project.parent.artifactId}-${project.parent.version}/queryserver-client/target</outputDirectory>
82+
<includes>
83+
<include>org.apache.phoenix:queryserver-client</include>
84+
</includes>
85+
<outputFileNameMapping>phoenix-${project.parent.version}-thin-client.${artifact.extension}</outputFileNameMapping>
86+
</dependencySet>
6287
</dependencySets>
6388
</assembly>

‎load-balancer/pom.xml

-4
Original file line numberDiff line numberDiff line change
@@ -116,10 +116,6 @@
116116
<groupId>org.apache.curator</groupId>
117117
<artifactId>curator-framework</artifactId>
118118
</dependency>
119-
<dependency>
120-
<groupId>org.apache.calcite.avatica</groupId>
121-
<artifactId>avatica</artifactId>
122-
</dependency>
123119
<dependency>
124120
<groupId>org.slf4j</groupId>
125121
<artifactId>slf4j-api</artifactId>

‎pom.xml

+45-15
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,7 @@
5151

5252
<modules>
5353
<module>queryserver</module>
54+
<module>queryserver-it</module>
5455
<module>queryserver-client</module>
5556
<module>load-balancer</module>
5657
<module>assembly</module>
@@ -67,7 +68,7 @@
6768
<!-- General Properties -->
6869
<top.dir>${project.basedir}</top.dir>
6970

70-
<phoenix.version>4.14.2-HBase-1.4</phoenix.version>
71+
<phoenix.version>4.15.0-HBase-1.4</phoenix.version>
7172

7273
<!-- Hadoop/Hbase Versions -->
7374
<hbase.version>1.4.10</hbase.version>
@@ -372,6 +373,12 @@
372373
</exclusion>
373374
</exclusions>
374375
</dependency>
376+
<dependency>
377+
<groupId>org.apache.hbase</groupId>
378+
<artifactId>hbase-server</artifactId>
379+
<version>${hbase.version}</version>
380+
<scope>test</scope>
381+
</dependency>
375382

376383
<!-- Hadoop Dependencies -->
377384
<dependency>
@@ -422,6 +429,11 @@
422429
</exclusion>
423430
</exclusions>
424431
</dependency>
432+
<dependency>
433+
<groupId>org.eclipse.jetty</groupId>
434+
<artifactId>jetty-util</artifactId>
435+
<version>${jetty.version}</version>
436+
</dependency>
425437
<dependency>
426438
<groupId>org.apache.curator</groupId>
427439
<artifactId>curator-client</artifactId>
@@ -480,6 +492,14 @@
480492
<version>0.8.1</version>
481493
</dependency>
482494

495+
<!-- Phoenix test dependencies -->
496+
<dependency>
497+
<groupId>org.apache.phoenix</groupId>
498+
<artifactId>queryserver</artifactId>
499+
<version>${project.version}</version>
500+
<classifier>tests</classifier>
501+
</dependency>
502+
483503
<!-- Hbase test dependencies -->
484504
<dependency>
485505
<groupId>org.apache.hbase</groupId>
@@ -494,12 +514,6 @@
494514
</exclusion>
495515
</exclusions>
496516
</dependency>
497-
<dependency>
498-
<groupId>org.apache.hbase</groupId>
499-
<artifactId>hbase-server</artifactId>
500-
<version>${hbase.version}</version>
501-
<scope>test</scope>
502-
</dependency>
503517
<dependency>
504518
<groupId>org.apache.hbase</groupId>
505519
<artifactId>hbase-server</artifactId>
@@ -531,17 +545,10 @@
531545
</dependency>
532546

533547
<!-- Other test dependencies -->
534-
<dependency>
535-
<groupId>org.eclipse.jetty</groupId>
536-
<artifactId>jetty-util</artifactId>
537-
<version>${jetty.version}</version>
538-
<scope>test</scope>
539-
</dependency>
540548
<dependency>
541549
<groupId>org.eclipse.jetty</groupId>
542550
<artifactId>jetty-security</artifactId>
543551
<version>${jetty.version}</version>
544-
<scope>test</scope>
545552
</dependency>
546553
<dependency>
547554
<groupId>org.eclipse.jetty</groupId>
@@ -587,5 +594,28 @@
587594
</dependency>
588595
</dependencies>
589596
</dependencyManagement>
590-
597+
<profiles>
598+
<profile>
599+
<id>package-phoenix-client</id>
600+
<activation>
601+
<property>
602+
<name>package.phoenix.client</name>
603+
</property>
604+
</activation>
605+
<properties>
606+
<!-- Not necessary for Phoenix <4.16.0 -->
607+
<phoenix.hbase.classifier></phoenix.hbase.classifier>
608+
</properties>
609+
<dependencyManagement>
610+
<dependencies>
611+
<dependency>
612+
<groupId>org.apache.phoenix</groupId>
613+
<artifactId>phoenix-client</artifactId>
614+
<version>${phoenix.version}</version>
615+
<classifier>${phoenix.hbase.classifier}</classifier>
616+
</dependency>
617+
</dependencies>
618+
</dependencyManagement>
619+
</profile>
620+
</profiles>
591621
</project>

‎queryserver-client/pom.xml

+1-2
Original file line numberDiff line numberDiff line change
@@ -74,8 +74,7 @@
7474
<goal>shade</goal>
7575
</goals>
7676
<configuration>
77-
<finalName>phoenix-${project.version}-thin-client</finalName>
78-
77+
<shadedArtifactAttached>false</shadedArtifactAttached>
7978
<transformers>
8079
<transformer
8180
implementation="org.apache.maven.plugins.shade.resource.IncludeResourceTransformer">

0 commit comments

Comments
 (0)