3.0.0 release

CHANGELOG.md

@@ -4,9 +4,21 @@ This project adheres to [Semantic Versioning](http://semver.org/).
 
 ## [unreleased]
 
+
+
+## [3.0.0]
+
 ### changed
 
+- version changed to 3 and name to "Anassimene"
 - copyright header in some files
+- aligned with sdaas_rdfstore 2.0.0
+- OS moved from alpine to debian
+- default user is jetty
+
+### Removed
+
+- alpinelinux_provisioning.sh script
 
 ## [2.5.0]
 
@@ -141,7 +153,8 @@ aligned with SDaaS [2.1.0]
 First release, aligned with SDaaS [2.0.3] (Talete)
 
 
-[Unreleased]:  https://github.com/linkeddatacenter/sdaas-ce/compare/2.5.0...HEAD
+[Unreleased]:  https://github.com/linkeddatacenter/sdaas-ce/compare/3.0.0...HEAD
+[3.0.0]:  https://github.com/linkeddatacenter/sdaas-ce/compare/2.5.0...3.0.0
 [2.5.0]:  https://github.com/linkeddatacenter/sdaas-ce/compare/2.4.0...2.5.0
 [2.4.0]:  https://github.com/linkeddatacenter/sdaas-ce/compare/2.3.2...2.4.0
 [2.3.2]: https://github.com/linkeddatacenter/sdaas-ce/compare/2.3.1...2.3.2

Dockerfile

@@ -1,20 +1,26 @@
-# Copyright (C) 2019 LinkedData.Center - All Rights Reserved
+# Copyright (C) 2019-2020 LinkedData.Center - All Rights Reserved
 # Permission to copy and modify is granted under the MIT license
-FROM  linkeddatacenter/sdaas-rdfstore:1.1.0
+FROM  linkeddatacenter/sdaas-rdfstore:2.0.0
 
 LABEL authors="[email protected]"
 
 USER root
+RUN set -xe ; \
+	apt-get update ; \
+	apt-get install -y --no-install-recommends curl gettext
 
-COPY tests/system/platform /workspace
-COPY alpinelinux_provisioning.sh /
+## To add debugging tools uncomment following liness:
+#RUN echo "jetty\n$jetty" | passwd jetty
+#RUN apt-get install -y sudo ; usermod -aG sudo jetty
 
 ENV SDAAS_BIN_DIR /usr/local/bin/sdaas
 COPY scripts $SDAAS_BIN_DIR
-RUN chmod -R 0755 $SDAAS_BIN_DIR; /alpinelinux_provisioning.sh
+COPY tests/system/platform /workspace
 
-WORKDIR  /workspace
+RUN chmod -R 0755 $SDAAS_BIN_DIR; \
+	chown -R jetty.jetty /workspace
+ENV PATH="${SDAAS_BIN_DIR}:${PATH}"
 
+USER jetty
+WORKDIR  /workspace
 ENV SD_REASONER_ENDPOINT http://localhost:8080/sdaas
-ENV PATH="${SDAAS_BIN_DIR}:${PATH}"
-CMD /sdaas-start --foreground

Dockerfile.dev

@@ -0,0 +1,21 @@
+# Copyright (C) 2019-2020 LinkedData.Center - All Rights Reserved
+# Permission to copy and modify is granted under the MIT license
+FROM  linkeddatacenter/sdaas-rdfstore:2.0.0
+
+LABEL authors="[email protected]"
+
+USER root
+RUN set -xe ; \
+	apt-get update ; \
+	apt-get install -y --no-install-recommends curl gettext 
+
+RUN apt-get install -y bats sudo 
+
+# Allow jetty to became su (no password)
+RUN echo "jetty\njetty" | passwd jetty; usermod -aG sudo jetty
+
+USER jetty
+WORKDIR  /workspace
+
+ENV SD_REASONER_ENDPOINT http://localhost:8080/sdaas
+ENV PATH="/workspace/scripts:${PATH}"

README.md

@@ -10,107 +10,78 @@ Try a short [on-line demo of the platform](https://en.linkeddata.center/l/sdaas-
 
 ## 🚀 Quickstart
 
-```
-docker build -t linkeddatacenter/sdaas-ce .
-docker run --name sdmp -d -p 8080:8080 linkeddatacenter/sdaas-ce
-```
-browse knowledge base at http://localhost:8080/sdaas
-
-
-To build a demo environment run:
 
+	docker run --name sdmp -d -p 8080:8080 linkeddatacenter/sdaas-ce
 	docker exec -t sdmp sdaas -f build.sdaas --reboot
 
+browse knowledge base at http://localhost:8080/sdaas
+
 when finished:
 
 	docker rm -f sdmp
 
-## Development environment ##
+## Start test environment
 
-You need a gnu compliant system with following packages installed:
+*Note on windows user: to mount local volume from a git bash `export MSYS_NO_PATHCONV=1` [see this note](https://stackoverflow.com/questions/7250130/how-to-stop-mingw-and-msys-from-mangling-path-names-given-at-the-command-line#34386471)*
 
-- bash (version > 4.4 )
-- coreutils: base64 basename cat chgrp chmod chown cp cut date dir dirname echo env
-  head md5sum mkdir mktemp mv pwd realpath rm rmdir sleep sort split stat tail tee test
-  touch tr uniq unlink wc ...
-- curl
-- find
-- awk
-- grep
-- sed
+	docker build -t sdaas-dev -f Dockerfile.dev .
+	docker run --name sdmp -d -p 8080:8080 -v ${PWD}:/workspace sdaas-dev
+	docker exec -ti sdmp bash
 
-Start a standard development environment by executing in docker with the 
-default bash distribution; then install the required packages:
 
+## Smoke tests 
 
-```
-docker run --rm -it -v "$(pwd):/workspace" -w /workspace bash
-./alpinelinux_provisioning.sh
-```
+Test sdaas cli:
 
+	sdaas --no-warmup
+	exit
+
+run a quick platform test:
 
-## smoke test ###
+	cd tests/system/platform 
+	sdaas sdaas -f build.sdaas --reboot
+	cd ../../..
 
-	./scripts/sdaas --no-warmup
-	exit
+point a browser to http://localhost:8080/sdaas/#query  and execute:
+
+	SELECT (COUNT(?s) AS ?edges) WHERE{?s?p?o}
+
+You shoud get about 24544 edges
 
-## Unit tests ###
+## Unit tests
 
 In order to run unit tests bats is needed (see https://github.com/bats-core/bats-core ):
 
-```
-apk --no-cache add bats
-bats tests/unit/
-exit
-```
+	bats tests/unit/
 
 
-## Functional and system tests
+## Functional tests
 
-To run functional and system tests you will need to start a local instance of blazegraph.
+To run functional and system tests you will need the local instance of blazegraph.
 By default, test scripts expect blazegraph endpoint at http://localhost:8080/sdaas 
 but you can configure a different address exporting the the SD_REASONER_ENDPOINT.
 The instance of blazegraph must share /workspace volume with sdaas.
 
 **WARNING**: blazegraph needs at least 2GB RAM to run functional tests.
 
-The easy way, is to start from  the lyrasis/blazegraph blazegraph docker distribution 
-and then installing inside its image the missing components:
-
-```
-docker run -d --name sdmp -v "$(pwd):/workspace" -p 8080:8080  lyrasis/blazegraph:2.1.5
-docker exec -ti -w /workspace sdmp bash
-./alpinelinux_provisioning.sh
-apk --no-cache add bats
-# ... access blazegraph workbench browsing http://localhost:8080/bigdata
-```
-
-
 For functional test execute: 
 
-```
-bats tests/functional
-```
+	bats tests/functional
+
+## System tests
 
 For system test, verify that the host is able to access Internet then  execute 
 
-```
-bats tests/system/platform
-```
+	bats tests/system/platform
 
-You can also launch directly the build process with:
 
-```
-cd tests/system/platform
-../../../scripts/sdaas -f build.sdaas --reboot
-```
+## Free test environment
 
 To free the docker resources:
 
-```
-exit
-docker rm -f sdmp 
-```
+	exit
+	docker rm -f sdmp 
+
 
 Have a look also to the [developer wiki](https://github.com/linkeddatacenter/sdaas-ce/wiki)
 
@@ -119,20 +90,20 @@ Have a look also to the [developer wiki](https://github.com/linkeddatacenter/sda
 
 To push a new docker image to docker hub:
 
-```
-docker build -t linkeddatacenter/sdaas-ce .
-docker login
-# input the docker hub credentials...
-docker tag linkeddatacenter/sdaas-ce linkeddatacenter/sdaas-ce:x.x.x
-docker push linkeddatacenter/sdaas-ce
-```
+
+	docker build -t linkeddatacenter/sdaas-ce .
+	docker login
+	# input the docker hub credentials...
+	docker tag linkeddatacenter/sdaas-ce linkeddatacenter/sdaas-ce:x.x.x
+	docker push linkeddatacenter/sdaas-ce
+
 
 
 ## Credits and license
 
 The sdaas community edition platform is derived from [LinkedData.Center SDaas Product](https://it.linkeddata.center/p/sdaas) and licensed with MIT by LinkedData.Center
 
-Copyright (C) 2018-2019 LinkedData.Center SRL
+Copyright (C) 2018-2020 LinkedData.Center SRL
  - All Rights Reserved
 Permission to copy and modify is granted under the [MIT license](LICENSE)
 

scripts/sdaas

@@ -4,8 +4,8 @@
 
 _SDAAS_DIR="$(dirname "$( realpath "$0" )")"
 
-_SDAAS_VERSION="2"
-_SDAAS_AGENT_NAME="Anassimandro"
+_SDAAS_VERSION="3"
+_SDAAS_AGENT_NAME="Anassimene"
 
 _SD_MUST_EXIT=0
 _SD_REBOOT=0

tests/system/platform/build.sdaas

@@ -13,7 +13,7 @@ SD_INCLUDE testing
 KEES_SRCs=("data/kees.ttl" "data/introspection.ttl" "data/trustmap.ttl")
 
 # Define a transformation pipeline
-ISTAT_GATEWAY=("|in|out|.ttl|iconv -f ISO88592 -t UTF-8|tr -d '\r'|gateways/istat")
+ISTAT_GATEWAY=("|in|out|.ttl|iconv -f ISO88592 -t UTF-8|tr -d '\r' | awk -f gateways/istat.awk")
 
 
 # Ingest data from a set of files

tests/system/platform/gateways/istat → tests/system/platform/gateways/istat.awk

@@ -1,4 +1,2 @@
-#!/usr/bin/awk -f
-
-BEGIN { FS=";"; print "@prefix schema: <http://schema.org/> ." }
-NR>1 { printf "<urn:istat:comune:%s> a schema:City; schema:name \"\"\"%s\"\"\".\n", $5, $6 }
+BEGIN { FS=";"; print "@prefix schema: <http://schema.org/> ." }
+NR>1 { printf "<urn:istat:comune:%s> a schema:City; schema:name \"\"\"%s\"\"\".\n", $5, $6 }

tests/system/platform/platformTest.bats

@@ -27,7 +27,7 @@ function doTest {
 	[[ "${lines[11]}" =~ "completed by replacing graph <http://schema.org/>"  ]]
 	[[ "${lines[12]}" =~ "starded learning of graph <urn:graph:istat>"  ]]
 	[[ "${lines[13]}" =~ "downloaded https://s3-eu-west-1.amazonaws.com/demo.hub1.linkeddata.center/data/comuni.csv"  ]]
-	[[ "${lines[14]}" =~ "transformation pipeline: in/* -> cat -> iconv -f ISO88592 -t UTF-8|tr -d '\r'|gateways/istat -> out"  ]]
+	[[ "${lines[14]}" =~ "transformation pipeline: in/* -> cat -> iconv -f ISO88592 -t UTF-8|tr -d '\r' | awk -f gateways/istat.awk -> out"  ]]
 	[[ "${lines[15]}" =~ "completed by replacing graph <urn:graph:istat>"  ]]
 	[[ "${lines[16]}" =~ "starded reasoning on graph <urn:graph:trustmap_default>"  ]]
 	[[ "${lines[17]}" =~ "evaluating axiom @axioms/default_trustmap.construct..."  ]]