Skip to content

Commit

Permalink
Merge pull request #224 from gematik/feature/tool-update-PTDATA-1418
Browse files Browse the repository at this point in the history
Automatisieren der Publication-Pipeline PTDATA-1006
  • Loading branch information
nikohl-de authored Dec 20, 2024
2 parents d4cb9f0 + 9a85167 commit f19490a
Show file tree
Hide file tree
Showing 21 changed files with 451 additions and 85 deletions.
38 changes: 37 additions & 1 deletion .github/PULL_REQUEST_TEMPLATE/PR_Template_Contributor.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,4 +29,40 @@
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [ ] My code follows the code style of this IG / specification.
- [ ] My change requires a change to the documentation or narrative (intend) of the IG.
- [ ] I have already updated the documentation / narrative (intend) accordingly.
- [ ] I have already updated the documentation / narrative (intend) accordingly.

## Reviewer / Quality Assurance Checklist
<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
<!--- Please do not hesitate to add additional boxes during review -->
- [ ] The Present PR does not need a thorough review process, due to brevity, low complexity or because it represents a rather minor change; otherwise go trough the list! ... and read the linked manual, if you did not do it yet!

### content review steps ([based on Best Practice IG Germany](https://simplifier.net/guide/Best-Practice-bei-der-Implementierung-und-Spezifizierung-mit-HL7/%C3%9Cbersicht/Spezifikation?version=current))
- [ ] no content review needed
- [ ] [cardinalities in profiles](https://simplifier.net/guide/Best-Practice-bei-der-Implementierung-und-Spezifizierung-mit-HL7/%C3%9Cbersicht/Spezifikation/Profilierung/Umgang-mit-Kardinalit%C3%A4ten.page.md?version=current) are added with caution (specially 'max') and are thoroughly motivated in a comment
- [ ] modifying-Elements are [highlighted](https://simplifier.net/guide/Best-Practice-bei-der-Implementierung-und-Spezifizierung-mit-HL7/%C3%9Cbersicht/Spezifikation/Profilierung/Umgang-mit-Modifying-Elements.page.md?version=current)
- [ ] [meta.profile Elements](https://simplifier.net/guide/Best-Practice-bei-der-Implementierung-und-Spezifizierung-mit-HL7/%C3%9Cbersicht/Spezifikation/Profilierung/Umgang-mit-Meta-Daten.page.md?version=current) not added or added with extreme caution
- [ ] [StructureDefinition.description](https://simplifier.net/guide/Best-Practice-bei-der-Implementierung-und-Spezifizierung-mit-HL7/%C3%9Cbersicht/Spezifikation/Profilierung/Dokumentation-und-Nachvollziehbarkeit.page.md?version=current) is filled
- [ ] [Constrained Elements](https://simplifier.net/guide/Best-Practice-bei-der-Implementierung-und-Spezifizierung-mit-HL7/%C3%9Cbersicht/Spezifikation/Profilierung/Dokumentation-und-Nachvollziehbarkeit.page.md?version=current) contain
- [ ] naming in ElementDefinition.short
- [ ] ElementDefinition.comment contains a motivation
- [ ] [Capability Statement](https://simplifier.net/guide/Best-Practice-bei-der-Implementierung-und-Spezifizierung-mit-HL7/%C3%9Cbersicht/Spezifikation/Erstellung-eines-CapabilityStatements.page.md?version=current) exists
- [ ] the textual searchparameters of every instance do correspond to the definition within the CapabilityStatement
- [ ] [relevant examples exist](https://simplifier.net/guide/Best-Practice-bei-der-Implementierung-und-Spezifizierung-mit-HL7/%C3%9Cbersicht/Spezifikation/Erstellung-von-Beispieldaten.page.md?version=current)


### formal intellectual review steps of [Implementation Guide](https://simplifier.net/guide/Best-Practice-bei-der-Implementierung-und-Spezifizierung-mit-HL7/%C3%9Cbersicht/Spezifikation/Erstellung-eines-Implementierungsleitfadens.page.md?version=current)
- [ ] no formal review needed
- [ ] proofreading for orthography

- [ ] there are no (critical) validation Errors in the CI pipeline
- [ ] inspected and no critical errors found(In the "github action view" -> Option button -> "View raw logs") - possibly list non-critical below
- [ ] skim reading for correct rendering of IG (possibly using IG-grep of string String like "Could not render")

### automated review steps:
- [ ] no formal review needed
- [ ] check IG-pages for broken links (possibly using plug-in)
- [ ]

### For reviewers
You migh use this: https://conventionalcomments.org/
55 changes: 31 additions & 24 deletions .github/PULL_REQUEST_TEMPLATE/PR_Template_VersionUpgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,49 +7,56 @@ Date:

## Description
<!--- CAVEAT: You to be an admin to perform all the necessary jobs. -->
This is a Pullreuqest that requires an increase in the Version number. Therefore, multiple outside-github, related Task have to be performed and checked.
This is a Pullrequest that requires an increase in the Version number. Therefore, multiple outside-github, related Task have to be performed and checked.

All jobs with an `x` in the boxes were performed to the best of knowledge.

## Pre-Merge Activities
<!--- Lets check everything before we continue. -->
- [ ] This PR refers to a versioned Branch with a name and a version number in the form of N.n.n, e.g. "TC_3.2.1".
- [ ] This PR has a clean meaningful committ history. Minor committs or committs without descirption have been squashed, at the latest now.
- [ ] The _./github/workflows/main.yml_ refers to the correct Firetly Terminal and SUSHI Version.
> **_Firely Terminal Pipeline_** 0.4.0.
> **_SUSHI Versions_** 3.5.0.
- [ ] By running the _Release_Publish.py_ script, release version and date was updated accordingly. The script ran without errors.
- [ ] Eventually, increase the dependency of to newer Basis Modul (package and sushi-config)
- [ ] New Release Notes were created, aglined to the committ history and cleaned. In Github, go to
- [ ] _-> Releases_ then _-> Draft a new release_ with the _Modul Name and Version_, then
- [ ] _-> Target the main-Branch_ and _->enter a new Tag according to the Version_, then click.
- [ ] Click _-> Generate Release notes_ , _->Adjust them if necessary_ and _-> Copy/Paste the Details in the RealeaseNotes.md_ of the very Branch you want to merge.
- [ ] Finally _-> Save as Draft_
- This PR refers to a versioned Branch with a name and a version number in the form of N.n.n, e.g. "TC_3.2.1".
- This PR has a clean meaningful commit history. Minor commits or commits without description have been squashed, at the latest now.
- The GitHub Actions "TC version update" and "CI (FHIR Validation)" finished successfully; release version and date was updated accordingly by release_publish.py (triggered by action)
- Eventually, increase the dependency of to newer Basis Modul and Basispofil-de and possibly others (package json and sushi-config)
- [ ] All release note items have been annotated as `test modifiying` OR `test stable` (the annotation should already take place within tickets for tracking reasons)
- [ ] the release notes were reviewed with a person responsible for testing
- [ ] New Release Notes were created, alined to the commit history. Possibly, if you want to check the release notes for completeness, check against automatic relesase note generation in GitHub. In Github, go to
- _-> Releases_ then _-> Draft a new release_ with the _Modul Name and Version_, then
- _-> Target the main-Branch_ and _-> enter a new Tag according to the Version_, then click.
- Click _-> Generate Release notes_ , _-> Adjust them if necessary_ and _-> Copy/Paste the Details in the RealeaseNotes.md_ of the very Branch you want to merge.
- Finally _-> Save as Draft_

## Merge and Publishing
<!--- Now lets do the merge. -->
<!--- Include details of your testing environment, especially -->
- [ ] With the updated Version, Dates, and Release Notes (as described above) with the last committ into the Branch you want to merge.
- [ ] In GitHub _-> Actions_ the _->CI (FHIR Validation)_ workflow terminates successfully.
- [ ] Add the Approve / the PR gets posivitly reviewed by a collegue.
- [ ] In GitHub _-> Actions_ the _-> CI (FHIR Validation)_ workflow terminates successfully.
- [ ] Add the Approve / the PR gets positively reviewed by a colleague.
- [ ] Merge (without squash) the PR, delete the Branch.


## Post-Merge Activities
<!--- Do some update, refreshe and cleanup of related resources. -->
<!--- Do some update, refresh and cleanup of related resources. -->

- [ ] Go to the corresponding SIMPLIFIER Project and _-> Github -> Reimport_ the project.
- [ ] Go to the corresponding SIMPLIFIER Project and _-> Packages -> Expand the Dropdown for Create -> Create new package_ for the project.
- [ ] With the corresponding version number, and
- [ ] The Release notes (from above) and a compare-link to the previous Release.
- [ ] Unlist the old package by _-> clicking on the old package_, _-> go to Admininstration_ and _-> click on Unlist_
- [ ] Publish the previosuly drafteted Release, including version number, on GitHub.
- [ ] Provide / Archive the IG in the corresponding _gh-pages_ branch of the GitHub project.
- [ ] Checkout the Branch (no need to merge it later).
- [ ] Export from Simplifier via _-> Guides -> Expand the Modul ... -> Export_
- [ ] Unpack the zip, remove the packages folder (because its kinda big), and move everything else to a (version coressponding) new folder in the branch folder structure.
- [ ] committ the branch.
- [ ] Unlist the old package by _-> clicking on the old package_, _-> go to Administration_ and _-> click on Unlist_
- [ ] Publish the previously drafted Release, including version number, on GitHub.
- [ ] Publish IG in Simplifier
- [ ] Provide a version in the IG title --> (x.x.x)
- [ ] Scope the IG to the published package
- [ ] Set URL key to isik-[module-name]-version (version without dots)
- [ ] Publish via Simplifier GUI (set to overwritable)


## Obsolete

- Provide / Archive the IG in the corresponding _gh-pages_ branch of the GitHub project.
- Checkout the Branch (no need to merge it later).
- Export from Simplifier via _-> Guides -> Expand the Modul ... -> Export_
- Add the zip (other tasks are automated by action called "Unzip and Update IG Version in webpage", in order to provide the Archige in the following page . https://gematik.github.io/spec-ISiK-Basismodul/index.html)
- If ISiK Basismodul was updated all depending Modules should be updated with a renewed dependency to the incremented Basismodul version - possibly including and closing technical corrections


## Finished
46 changes: 7 additions & 39 deletions .github/workflows/RenderAllDiagrams.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,18 +37,18 @@ jobs:
# Download plantUML jar
- name: Download plantuml file
run: |
wget -O plantuml.jar "https://github.com/plantuml/plantuml/releases/latest/download/plantuml.jar"
wget -O plantuml.jar "https://github.com/plantuml/plantuml/releases/download/v1.2024.2/plantuml.jar"
# Ensure Folder exsists, otherwise, create it
# Clean Folder
- name: Ensure and clean folder
run: |
img_dir=Material/images/diagrams
mkdir -p $img_dir
# do not clean
# rm -rf Material/images/diagrams/*.svg
rm -rf Material/images/diagrams/*.svg
# Generate the SVGs from PUML
- name: Render PUML to SVG files
- name: Render PUML to SVG and Move files
run: |
FileNamePaths=$(find . -path "*/images/src/*/*.puml" -exec dirname {} \; | sort -u)
for dir in $FileNamePaths
Expand All @@ -72,34 +72,7 @@ jobs:
format: svg
action-mode: all

# Install note.js and bpmn-to-image
- name: Install Notejs and pbmn-to-image
uses: actions/setup-node@v4
with:
node-version: 18
- run: npm install -g bpmn-to-image

# Generate the SVGs from BPMN
- name: Render BPMN to SVG files
run: |
FileNamePaths=$(find . -path "*/images/src/*/*.bpmn" -exec dirname {} \; | sort -u)
for dir in $FileNamePaths
do
# Render SVGs from BPMN
echo $dir
FileBaseNames=$(find $dir -name "*.bpmn" -exec basename "{}" ".bpmn" \; | sort -u)
#FileBaseNames=$(find $dir -name "*.bpmn" | xargs -L1 -I{} basename "{}")
for base in $FileBaseNames
do
echo $base
bpmn-to-image "$dir/$base.bpmn":"$dir/$base.svg";
done
done
# copies the created SVG files to the images/diagrams folder and deletes the drawio files
# mv for copy and delete, here
- name: Move SVGs to target image folder
run: |
img_dir=Material/images/diagrams
Expand All @@ -108,8 +81,7 @@ jobs:
for dir in $FileNamePaths
do
# Move SVGs to out directory
# find $dir -name "*.svg" -exec rm -rf {} $img_dir \;
find $dir -name "*.svg" -exec mv -f {} $img_dir \;
find $dir -name "*.svg" -exec mv {} $img_dir \;
done
## add and commit the new generated files
Expand All @@ -129,8 +101,4 @@ jobs:
with:
commit_user_name: GitHub Actions Bot
commit_user_email: [email protected]
commit_message: auto-generated diagrams by GitHub Action after source code change

- name: "Run if no changes have been detected"
if: steps.auto-commit-action.outputs.changes_detected == 'false'
run: echo "No Changes!"
commit_message: auto-generated diagrams by GitHub Action after source code change
7 changes: 4 additions & 3 deletions .github/workflows/TC-version-update.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

name: TC version update

# Controls when the action will run.
Expand All @@ -15,12 +16,12 @@ jobs:
steps:
- name: checkout repo content
uses: actions/checkout@v2 # checkout the repository content

- name: setup python
uses: actions/setup-python@v4
with:
python-version: '3.10' # install the python version needed

- name: install python packages
run: |
python -m pip install --upgrade pip
Expand All @@ -34,4 +35,4 @@ jobs:
with:
committer_name: GitHub Actions Bot
committer_email: [email protected]
message: auto-generated file update of TC version by GitHub Actions (CI FSH to FHIR Validation)
message: auto-generated file update of TC version by GitHub Actions (CI FSH to FHIR Validation)
40 changes: 40 additions & 0 deletions .github/workflows/link-check.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Check Links
# This workflow checks for broken links in the repository.

on:
pull_request:
# Trigger the workflow on any pull request targeting any branch.
branches:
- '**'
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

jobs:
link_checker:
# Define a job named "link_checker" that runs the link-checking process.
runs-on: ubuntu-latest
# Specify the virtual machine environment to run the job (latest Ubuntu).

steps:
- name: Checkout repository
# Step 1: Checkout the repository code into the workflow environment.
uses: actions/checkout@v3
# Use the official GitHub checkout action (version 3).

- name: Run Link Checker
# Step 2: Run the Lychee link-checking tool.
uses: lycheeverse/lychee-action@v2
# Use the Lychee GitHub Action (version 2) for link checking.

with:
fail: true
# Configure the workflow to fail if broken links are found.
# Run the link checker on the current directory and all its contents.
# Exclude links matching the pattern for Jira links (e.g., HL7 Jira).
# Exclude all links in the "ImplementationGuide/style" directory.
# Exclude all links in the "Material" directory.
args: >
.
--exclude-path ImplementationGuide/style
--exclude-path Material
70 changes: 59 additions & 11 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
name: Use Basis main Stufe 4 main.yml
# This is a basic workflow to help you get started with Actions

# Controls when the action will run
name: CI (FHIR Validation)

# Controls when the action will run.
on:
# Triggers the workflow on push or pull request events but only for the master branch
push:
Expand All @@ -11,15 +13,61 @@ on:
pull_request:
branches:
- 'main**'
workflow_call:
secrets:
SIMPLIFIER_USERNAME:
required: true
SIMPLIFIER_PASSWORD:
required: true
WORKFLOW_PERMISSION_GITHUB:
required: true

# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:
# Allows you to run this workflow manually from the Actions tab
workflow_dispatch:

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:

# Validate all resources using Firely Terminal
CI_FHIR_VALIDATION:
# The type of runner that the job will run on
runs-on: ubuntu-latest

jobs:
call-tool-update:
uses: gematik/spec-isik-basismodul/.github/workflows/main.yml@main-isik-stufe-4
secrets:
SIMPLIFIER_USERNAME: ${{ secrets.SIMPLIFIER_USERNAME }}
SIMPLIFIER_PASSWORD: ${{ secrets.SIMPLIFIER_PASSWORD }}
WORKFLOW_PERMISSION_GITHUB: ${{ secrets.WORKFLOW_PERMISSION_GITHUB }}
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: Checkout code (PR)
uses: actions/checkout@v4
if: github.event_name == 'pull_request'
with:
ref: ${{ github.event.pull_request.head.ref }}

- name: Checkout code (Main)
uses: actions/checkout@v4
if: github.event_name != 'pull_request'

# Java and .NET are already installed on ubuntu-latest

- name: Firely.Terminal (GitHub Actions)
uses: FirelyTeam/[email protected]
with:
PATH_TO_CONFORMANCE_RESOURCES: Resources/fsh-generated/resources/
#PATH_TO_EXAMPLES: Examples
PATH_TO_QUALITY_CONTROL_RULES: qc/custom
DOTNET_VALIDATION_ENABLED: true
JAVA_VALIDATION_ENABLED: true
OUTPUT_FORMAT: RAW
JAVA_VALIDATION_OPTIONS: -allow-example-urls true
SIMPLIFIER_USERNAME: ${{ secrets.SIMPLIFIER_USERNAME }}
SIMPLIFIER_PASSWORD: ${{ secrets.SIMPLIFIER_PASSWORD }}
SUSHI_ENABLED: true
SUSHI_OPTIONS: Resources/
SUSHI_VERSION: 3.12.1
EXPECTED_FAILS: VALIDATION_CONFORMANCE_DOTNET VALIDATION_CONFORMANCE_JAVA VALIDATION_EXAMPLES_JAVA

- name: Add & Commit
uses: EndBug/add-and-commit@v9
with:
committer_name: GitHub Actions Bot
committer_email: [email protected]
add: 'Resources/fsh-generated/resources/'
message: auto-generated FHIR files by GitHub Actions (CI FSH to FHIR Validation)
6 changes: 6 additions & 0 deletions .lycheeignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
^https?://jira\.hl7\.org
^https?://test\.krankenhaus\.de
^http://hl7\.org/fhir/StructureDefinition/condition-related
^http://hl7\.org/fhir/5\.0/StructureDefinition/extension-Encounter\.plannedStartDate
^http://hl7\.org/fhir/5\.0/StructureDefinition/extension-Encounter\.plannedEndDate
^https://gematik.de/fhir/isik/StructureDefinition/Profile
12 changes: 12 additions & 0 deletions .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,2 +1,14 @@
{
"python.testing.unittestArgs": [
"-v",
"-s",
"C:/Users/nils.kohl/Documents/spec-ISiK-Dokumentenaustausch/tests",
"-p",
"test*.py"
],
"python.testing.unittestEnabled": true,
"python.testing.pytestEnabled": false,
"python.testing.nosetestsEnabled": false,
"python.testing.autoTestDiscoverOnSaveEnabled": true

}
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

{{tree:https://gematik.de/fhir/isik/StructureDefinition/ISiKDokumentenSuchergebnisse, hybrid}}

[Link Simplifier Profil Übersicht](https://simplifier.net/spec-isik-dokumentenaustausch/isikdokumentensuchergebnisse)
[Link Simplifier Profil Übersicht](https://simplifier.net/guide/isik-dokumentenaustausch-v4/ImplementationGuide-markdown-Datenobjekte-Datenobjekte_Bundle?version=current)

Folgende FHIRPath-Constraints sind im Profil zu beachten:

Expand Down
Loading

0 comments on commit f19490a

Please sign in to comment.