Skip to content

Commit

Permalink
Stub out more tut5/tut6 content
Browse files Browse the repository at this point in the history
  • Loading branch information
adamkewley committed Aug 7, 2024
1 parent de32d32 commit 6dcc4b3
Show file tree
Hide file tree
Showing 7 changed files with 163 additions and 33 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,9 @@
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = []

# enable numbering each figure (e.g. "figure 1", "figure 2" in caption)
numfig = True


# -- Options for HTML output -------------------------------------------------

Expand Down
1 change: 1 addition & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ Table of Contents
tut3
tut4
tut5
tut6

.. toctree::
:maxdepth: 2
Expand Down
142 changes: 109 additions & 33 deletions docs/source/tut5.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,20 @@ Use the Mesh Warper
In this tutorial, we will be using the Mesh Warping UI to perform
landmark-driven mesh warping using the Thin-Plate Spline algorithm
(`explanation <TPS General Info_>`_, `literature source <TPS Primary Literature Source_>`_). The UI
provides tooling for loading two meshes and creating landmark pairs between them.
provides tooling for loading two meshes and creating landmark pairs between them:

.. _cylinder-warp-example:
.. figure:: _static/tut5_mesh-warper-screenshot.png
:width: 60%

The mesh warping UI, which shows the source ("reference", left) destination ("target", middle) and result ("warped", right) meshes.
Here, the warping quality is low. This is because of the low triangle and landmark count.

.. figure:: _static/tut5_mesh-warper-organic-screenshot.png
:width: 60%

Same as :numref:`cylinder-warp-example`, but showing an example of warping a clavicle bone. This example has many paired (left-to-middle) landmarks in a
variety of locations along the surface of the bone, which improves the warp quality (right).

Prerequisites
-------------
Expand All @@ -23,61 +35,124 @@ Prerequisites
Topics Covered by this Tutorial
-------------------------------

- A general overview of the Thin-Plate Spline technique
- A general overview of the mesh warping UI
- Using the mesh warping UI on a basic anatomical example
- An overview of the Thin-Plate Spline technique
- How OpenSim Creator's mesh warping UI works with that technique
- A concrete walkthrough of using the UI on an anatomical mesh


The Thin-Plate Spline Technique
-------------------------------
The Thin-Plate Spline (TPS) Technique
-------------------------------------

TODO: Basic overview of the high-level idea (minimal warping of a thin plate of metal), how
it relates to the maths (fitting points in n-dimensions), how it relates to landmarks (3D corresponding points) and
where meshes (they're clouds of points that the resulting warp polynomial can be applied to) and "non-participating landmarks" (e.g. one-off datapoints)
fit into the picture.
.. note::

This section isn't going to explain the Thin-Plate Spline (TPS) technique in
extensive detail. Instead, it will provide a simplified explanation
that's good enough to get a rough idea of what's happening when you use the
mesh warping UI.

The Mesh Warping UI
-------------------
If you want something more, we recommend consulting the `Relevant References`_,
which lists a variety of content related to the TPS technique.

TODO

Opening the UI
~~~~~~~~~~~~~~
Imagine placing a thin plate with points along its surface onto a table. Now
imagine that each surface point has a corresponding "target" point somewhere
in 3D space. You could minimize the distance between those corresponding points
by physically bending the plate, but you'll need to figure out the optimal way
to bend it.

TODO
The TPS technique models this principle by:

Panels in the UI
~~~~~~~~~~~~~~~~
- Describing "bending" the plate as a bounded linear combination of some
basis function, :math:`U(v)`. The `original paper <TPS Primary Literature Source_>`_
used :math:`U(v) = |v|^2 \log{|v|^2}`, but `other sources <SemilandmarksInThreeDimensions_>`_ use :math:`U(v) = |v|`.
- Treating the problem of transforming "source/reference" points (landmarks),
:math:`x_i`, to "destination/reference" points (landmarks), :math:`y_i`, as an
interpolation problem.
- Solving the coefficients of that linear combination while minimizing the
"bending energy". `Wikipedia example <TPS General Info_>`_:

TODO
.. math::
Points of Interest in the UI (import, export, blending, etc.)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
E_{\mathrm{tps}}(U) = \sum_{i=1}^K \|y_i - U(x_i) \|^2
TODO
The coefficients that drop out of this process can then be used to warp any
point in the same space. **However**, remember that this explanation is
simplifying things, and that you should read `Relevant References`_ if
you want to learn more.

Here's how concepts from TPS apply to OpenSim Creator's mesh warping UI:

Example: Warping a Femur
- OpenSim Creator's mesh warper uses the TPS technique. It's conceptually
handy to think about how that can be used to "bend" mesh surfaces.

- **Source Mesh** and **Source Landmark** refer to data in the "reference", or
"source" space. Each source landmark requires a corresponding destination
landmark with the same name.

- **Destination Mesh** and **Destination Landmark** refer to data in the "target", or
"destination" space. Each destination landmark must have a corresponding
source landmark with the same name.

- **Warp Transform** is the product of the TPS technique after pairing the
landmarks and solving the relevant coefficients. The transform can be applied
to point data to warp it. E.g. in the mesh warping UI, the transform is applied
to the source mesh to produce the result mesh. It's also applied to "non-participating landmarks"
to produce warped points.

- **Result Mesh** is the result of applying the warp transform to the source mesh.

- **Non-Participating Landmark** is a landmark in the source mesh's space that
should be warped by the warp transform but shouldn't participate in solving
the TPS coefficients.


Opening the Mesh Warping UI
---------------------------

.. figure:: _static/tut5_open-mesh-warper-from-splash-screen.png
:width: 60%

The mesh warping UI can be opened from the main splash screen of OpenSim Creator
(highlighted red).


Mesh Warping UI Overview
------------------------

TODO
.. figure:: _static/tut5_mesh-warper-organic-screenshot.png
:width: 60%

Screenshot of the mesh warping UI, with the (optional) "Landmark Navigator" window
open on the right-hand side.

Load Example Meshes
~~~~~~~~~~~~~~~~~~~
Although the TPS technique (explained above) only actually requires paired
landmarks in 3D space, the mesh warping UI pairs those landmarks with the surfaces
of meshes. This pairing is natural when working with physiological meshes, because
"a pair of 3D points" isn't as useful as "the most posterior point on a femur's
lateral condyl in two CT scans" when we want to use those correspondences to
warp associated data (e.g. muscle points).

TODO
The mesh warping UI separates the information into three panels:

Step-By-Step Explanation
~~~~~~~~~~~~~~~~~~~~~~~~
- **Source**: "source", or "reference" meshes and landmarks. These are where
you're starting from. In this panel, you can ``Import`` meshes, landmarks,
and non-participating landmarks (datapoints, such as markers, that should
be warped, but shouldn't participate in fitting TPS parameters).

TODO

Warping Single Points (markers, muscle points)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* TODO: Explain top-level UI design (what each panel does)
* TODO: Explain key features (import, export, blending, etc.)


Example: Warping a Femur
------------------------

* TODO: Provide + Load Example Meshes
* TODO: Step-by-step explanation
* TODO: warping single points/non-participating landmarks

TODO

.. _Relevant References:

Relevant References
-------------------
Expand Down Expand Up @@ -133,3 +208,4 @@ the algorithm.
.. _3D Thin Plate Spline Warping Function: https://uk.mathworks.com/matlabcentral/fileexchange/37576-3d-thin-plate-spline-warping-function
.. _3D Point set warping by thin-plate/rbf function: https://uk.mathworks.com/matlabcentral/fileexchange/53867-3d-point-set-warping-by-thin-plate-rbf-function
.. _A Practical Guide to Sliding and Surface Semilandmarks in Morphometric Analyses: https://doi.org/10.1093/iob/obz016
.. _SemilandmarksInThreeDimensions: https://doi.org/10.1007/0-387-27614-9_3
50 changes: 50 additions & 0 deletions docs/source/tut6.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
.. _tut6:


Use the Model Warper
====================

.. warning::

**Model Warping is 🪄 experimental 🪄.**

You can try it out, get a feel for it, but beware that some things (e.g.
the ``.warpconfig.toml`` file) aren't established yet, which means they
may change, as we rollout more robust designs.


TODO: overview and screenshot, top-level motivations, etc.

Prerequisites
-------------

TODO: model editing, mesh warping, a basic understanding of Thin-Plate Spline


Topics Covered by this Tutorial
-------------------------------

* TODO: how to ensure a model is warp-able
* TODO: basic example of a two-body model
* TODO: more advanced example walkthrough
* TODO: limitations, references, future work


Warpable Model Requirements
---------------------------

TODO: explain what the model warper can/can't warp. Explain ``StationDefinedFrame``
and limitations around warping frames, muscle scaling, etc.

Basic Example: Two-body model
-----------------------------

TODO: provide a very stripped-down model that meets the requirements for warp-ability


Advanced Example: Many-Bodied Model with Custom Requirements
------------------------------------------------------------

TODO: an example model that requires the user to specialize/specify customization
in the warp config (e.g. tell the warp engine to skip some steps, warp X using
technique Y, etc.)

0 comments on commit 6dcc4b3

Please sign in to comment.