Skip to content

Commit

Permalink
Integrate catalog construction into CI chain
Browse files Browse the repository at this point in the history
The construction script now handles multiple input ontology files, but
with the requirement that they be in the same directory.

Interfaces have also been added to handle imported, possibly non-CDO
ontology references in two ways:
* With a TSV file mapping ontology IRIs or version IRIs to files.
* With optional references to (effectively imported) `catalog-v001.xml`
  files.

Another behavior change is implemented: the focus ontologies are now
also added to the `catalog-v001.xml` file, in part to support when
multiple graph files are in one directory, and in part to support
re-consumption of `catalog-v001.xml` by the `catalog-v001.xml`
generating script.

The rationales for how to handle ontology--file mappings outside the
scope of UCO (in both upstream and downstream directions) include:
* Symbolic links could have been used to pool all file references into
  the `/dependencies` directory.  Windows users that run `git clone`
  without symbolic links enabled for their system would encounter
  significantly counter-intuitive errors.
  - This also would not iterate well with consumers of the catalog
    script outside of UCO (e.g. CASE).
* A Makefile could have been made to normalize the dependent ontology
  files into the same Turtle style (or even away from RDF-XML, which the
  Collections Ontology currently uses as sole format).  However, this
  would again be a point of difficulty for Windows users, as they would
  have to run `make` to create the files referenced in the catalog XML.
* Copying files into a Git repository introduces code-drift issues that
  are difficult to manage.  When the copied files were themselves
  tracked in Git, this is counter to the purpose of Git submodules.

This patch goes on the assumption that Git submodules and recursive
cloning are a reasonable minimal requirement to access full local-file
ontology interaction.

The catalog generating script in this patch state has been tested
(offline) with CASE and CASE-Corpora as users, via a submodule chain
starting with CASE-Corpora.

The `CONTRIBUTE.md` file has also been updated to add usage
documentation, and to fix a copy-paste error from some time ago.

A follow-on patch will regenerate Make-managed files.

References:
* #449

Signed-off-by: Alex Nelson <[email protected]>
  • Loading branch information
ajnelson-nist committed Mar 15, 2023
1 parent 66d0c38 commit fa66e99
Show file tree
Hide file tree
Showing 8 changed files with 290 additions and 141 deletions.
13 changes: 12 additions & 1 deletion CONTRIBUTE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,17 @@
# Contributing to the CASE ontology
# Contributing to the UCO ontology


## Testing prerelease states

Practices for users interested in testing prerelease states are documented on the [Cyber Domain Ontology website](https://cyberdomainontology.org/ontology/development/#testing-prereleases).


## Using Protégé catalog files

Interested users of `catalog-v001.xml` files, e.g. users of [Protégé](https://protege.stanford.edu/), can use these XML files to interact with UCO as local files. To do so, UCO must be `git-clone`'d with Git submodules also cloned. This can be done with the following commands:

* `git clone --recursive https://github.com/ucoProject/UCO.git` (all users)
* `git clone https://github.com/ucoProject/UCO.git ; make` (macOS or Linux users)
- The narrowest setup operation strictly for purposes of supporting the `catalog-v001.xml` files is to run `make .git_submodule_init.done.log` instead of the default `make all`.

Protégé should not require network connectivity to load imported ontologies after the above commands are run.
3 changes: 2 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@ SHELL := /bin/bash
PYTHON3 ?= $(shell which python3)

all: \
.lib.done.log
.lib.done.log \
.venv.done.log
$(MAKE) \
--directory ontology

Expand Down
2 changes: 2 additions & 0 deletions etc/dependency_files.tsv
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
http://purl.org/co/ ${top_srcdir}/dependencies/collections-ontology/collections.owl
http://purl.org/spar/error ${top_srcdir}/dependencies/error/docs/current/error.ttl
File renamed without changes.
27 changes: 23 additions & 4 deletions ontology/co/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ check_reference_basenames := $(foreach ttl_basename,$(ttl_basenames),.check-$(tt
check_targets := $(foreach ttl_basename,$(ttl_basenames),check-$(ttl_basename))

all: \
$(check_reference_basenames)
$(check_reference_basenames) \
catalog-v001.xml

.check-%.ttl: \
%.ttl \
Expand All @@ -43,8 +44,25 @@ all: \
--target-format turtle
mv $@_ $@

catalog-v001.xml: \
$(top_srcdir)/.venv.done.log \
$(top_srcdir)/etc/domain_directories.tsv \
$(top_srcdir)/etc/dependency_files.tsv \
$(top_srcdir)/src/create-catalog-v001.xml.py \
$(ttl_basenames)
rm -f _$@
source $(top_srcdir)/venv/bin/activate \
&& python3 $(top_srcdir)/src/create-catalog-v001.xml.py \
_$@ \
$(top_srcdir)/etc/domain_directories.tsv \
$(top_srcdir)/etc/dependency_files.tsv \
"$(top_srcdir)" \
$(ttl_basenames)
mv _$@ $@

check: \
$(check_targets)
$(check_targets) \
catalog-v001.xml

# Reminder: diff exits non-0 on finding any differences.
# Reminder: The $^ automatic Make variable is the name of all recipe prerequisites.
Expand All @@ -55,5 +73,6 @@ check-%.ttl: \
|| (echo "ERROR:ontology/co/Makefile:The local $< does not match the normalized version. If the above reported changes look fine, run 'cp .check-$< $<' while in the sub-folder ontology/co/ to get a file ready to commit to Git." >&2 ; exit 1)

clean:
@rm -f $(check_reference_basenames)

@rm -f \
$(check_reference_basenames) \
catalog-v001.xml
38 changes: 0 additions & 38 deletions ontology/uco/master/Makefile

This file was deleted.

Loading

0 comments on commit fa66e99

Please sign in to comment.