Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] openPMD: variableBased IterationEncoding #1909

Closed
wants to merge 11 commits into from
Closed

[WIP] openPMD: variableBased IterationEncoding #1909

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
8126ff4
Added support to write to one file
Apr 23, 2021
0ae19c1
updated with the latest openPMD version
Apr 24, 2021
10c652e
Merge branch 'development' of https://github.com/ECP-WarpX/WarpX into 1f
Apr 24, 2021
78253db
[draft] openPMD-api: dev
ax3l Apr 24, 2021
99cfcd6
backward compatible comparison
Apr 24, 2021
56a5d36
Merge branch '1f' of https://github.com/guj/WarpX into 1f
Apr 24, 2021
e14ac05
updated openpmd_tspf to openpmd_encoding
May 18, 2021
a2f58b4
replaced binary option openpmd_tspf by openpmd_encoding, taking v/g/…
May 18, 2021
9599db0
Revolved conflict with openPMD.cmake
May 18, 2021
49ab520
fixed eol
May 18, 2021
a8d91ff
updated doc
May 19, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .azure-pipelines.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ jobs:
sudo curl -L -o /usr/local/bin/cmake-easyinstall https://git.io/JvLxY
sudo chmod a+x /usr/local/bin/cmake-easyinstall
if [ "${WARPX_CI_OPENPMD:-FALSE}" == "TRUE" ]; then
cmake-easyinstall --prefix=/usr/local git+https://github.com/openPMD/openPMD-api.git@0.13.2 \
cmake-easyinstall --prefix=/usr/local git+https://github.com/openPMD/openPMD-api.git@dev \
-DopenPMD_USE_PYTHON=OFF -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DBUILD_CLI_TOOLS=OFF
python -m pip install --upgrade openpmd-api
fi
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ubuntu.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ jobs:
run: |
.github/workflows/dependencies/nvcc11.sh
export CEI_SUDO="sudo"
cmake-easyinstall --prefix=/usr/local git+https://github.com/openPMD/openPMD-api.git@0.13.2 -DopenPMD_USE_PYTHON=OFF -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DBUILD_CLI_TOOLS=OFF
cmake-easyinstall --prefix=/usr/local git+https://github.com/openPMD/openPMD-api.git@dev -DopenPMD_USE_PYTHON=OFF -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DBUILD_CLI_TOOLS=OFF
- name: build WarpX
run: |
export PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH}
Expand Down Expand Up @@ -103,7 +103,7 @@ jobs:
sudo curl -L -o /usr/local/bin/cmake-easyinstall https://git.io/JvLxY
sudo chmod a+x /usr/local/bin/cmake-easyinstall
export CEI_SUDO="sudo"
CXX=$(which icpc) CC=$(which icc) cmake-easyinstall --prefix=/usr/local git+https://github.com/openPMD/openPMD-api.git@0.13.2 -DopenPMD_USE_PYTHON=OFF -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DBUILD_CLI_TOOLS=OFF
CXX=$(which icpc) CC=$(which icc) cmake-easyinstall --prefix=/usr/local git+https://github.com/openPMD/openPMD-api.git@dev -DopenPMD_USE_PYTHON=OFF -DBUILD_TESTING=OFF -DBUILD_EXAMPLES=OFF -DBUILD_CLI_TOOLS=OFF
- name: build WarpX
run: |
set +e
Expand Down
2 changes: 1 addition & 1 deletion Docs/source/install/cmake.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ CMake Option Default & Values Des
``WarpX_amrex_internal`` **ON**/OFF Needs a pre-installed AMReX library if set to ``OFF``
``WarpX_openpmd_src`` *None* Path to openPMD-api source directory (preferred if set)
``WarpX_openpmd_repo`` ``https://github.com/openPMD/openPMD-api.git`` Repository URI to pull and build openPMD-api from
``WarpX_openpmd_branch`` ``0.13.2`` Repository branch for ``WarpX_openpmd_repo``
``WarpX_openpmd_branch`` ``dev`` Repository branch for ``WarpX_openpmd_repo``
``WarpX_openpmd_internal`` **ON**/OFF Needs a pre-installed openPMD-api library if set to ``OFF``
``WarpX_picsar_src`` *None* Path to PICSAR source directory (preferred if set)
``WarpX_picsar_repo`` ``https://github.com/ECP-WarpX/picsar.git`` Repository URI to pull and build PICSAR from
Expand Down
27 changes: 21 additions & 6 deletions Source/Diagnostics/WarpXOpenPMD.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -267,9 +267,10 @@ void WarpXOpenPMDPlot::CloseStep (bool isBTD, bool isLastBTDFlush)
// close BTD file only when isLastBTDFlush is true
if (isBTD and !isLastBTDFlush) callClose = false;
if (callClose) {
if (m_Series)
m_Series->iterations[m_CurrentStep].close();

if (m_Series) {
auto iterations = m_Series->writeIterations();
iterations[m_CurrentStep].close();
}
// create a little helper file for ParaView 5.9+
if (amrex::ParallelDescriptor::IOProcessor())
{
Expand Down Expand Up @@ -297,7 +298,10 @@ WarpXOpenPMDPlot::Init (openPMD::Access access, bool isBTD)

// close a previously open series before creating a new one
// see ADIOS1 limitation: https://github.com/openPMD/openPMD-api/pull/686
m_Series = nullptr;
if (m_OneFilePerTS)
m_Series = nullptr;
else if (m_Series != nullptr)
return;

if (amrex::ParallelDescriptor::NProcs() > 1) {
#if defined(AMREX_USE_MPI)
Expand All @@ -316,6 +320,13 @@ WarpXOpenPMDPlot::Init (openPMD::Access access, bool isBTD)
m_MPIRank = 1;
}

if ( m_OpenPMDFileType.compare("bp") == 0 )
Copy link
Member

@ax3l ax3l Apr 24, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should make this a user input option.
We can default to variableBased, fallback default: groupBased, but the user might want to pick fileBased for various reasons

Note that the openPMD::Series file name should not include the _%T placeholder for all but fileBased iteration encoding.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, so in addition to parameter "openpmd_backend", I will introduce another one called "openpmd_iteration". It is default to variableBased if available, otherwise, group based.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually, I will rename openpmd_tspf to openpmd_iteration

Copy link
Member

@ax3l ax3l May 21, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could encode it with an enum struct so that the user can pick between all three encodings :)

#if OPENPMDAPI_VERSION_GE(0, 14, 0)
m_Series->setIterationEncoding( openPMD::IterationEncoding::variableBased );
#else
m_Series->setIterationEncoding( openPMD::IterationEncoding::groupBased );
#endif

// input file / simulation setup author
if( WarpX::authors.size() > 0u )
m_Series->setAuthor( WarpX::authors );
Expand Down Expand Up @@ -430,7 +441,9 @@ WarpXOpenPMDPlot::DumpToFile (ParticleContainer* pc,

WarpXParticleCounter counter(pc);

openPMD::Iteration currIteration = m_Series->iterations[iteration];
openPMD::WriteIterations iterations = m_Series->writeIterations();
auto currIteration = iterations[iteration];

openPMD::ParticleSpecies currSpecies = currIteration.particles[name];
// meta data for ED-PIC extension
currSpecies.setAttribute( "particleShape", double( WarpX::noz ) );
Expand Down Expand Up @@ -811,7 +824,9 @@ WarpXOpenPMDPlot::WriteOpenPMDFields ( //const std::string& filename,
auto const dataset = openPMD::Dataset(datatype, global_size);

// meta data
auto series_iteration = m_Series->iterations[iteration];
openPMD::WriteIterations iterations = m_Series->writeIterations();
auto series_iteration = iterations[iteration];

auto meshes = series_iteration.meshes;
if( first_write_to_iteration ) {
series_iteration.setTime( time );
Expand Down
2 changes: 1 addition & 1 deletion cmake/dependencies/openPMD.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ if(WarpX_OPENPMD)
set(WarpX_openpmd_repo "https://github.com/openPMD/openPMD-api.git"
CACHE STRING
"Repository URI to pull and build openPMD-api from if(WarpX_openpmd_internal)")
set(WarpX_openpmd_branch "0.13.2"
set(WarpX_openpmd_branch "dev"
CACHE STRING
"Repository branch for WarpX_openpmd_repo if(WarpX_openpmd_internal)")

Expand Down