svFSI Guide

svFSI is an open-source multi-physics finite element solver. It provides rich features for general computational studies of diffusion, heat transfer, incompressible viscous fluid, nonlinear elastodynamics, fluid-structure interaction and electrophysiology. It also specializes in computational modeling of the whole heart dynamics. As part of the SimCardio project, svFSI is capable of simulating myocardial electrical activity using state-of-the-art cellular activation models, employing nonlinear hyperelastic material models to simulate tissue mechanics and activation coupling strategies, and performing large deformation fluid-structure interaction to simulate the dynamics between the heart tissue and blood flow. svFSI could also be used to perform blood flow simulations by imposing the ventricular wall motion extracted from medical image data.

svFSI is compatible with several mesh formats and can read a wide range of 2D and 3D element types. svFSI could be coupled to Trilinos linear solver library providing a wide choice of linear solvers and preconditioners for various applications. The methods employed are widely popular among the scientific community and continue to evolve through research and development. The solver is parallelized using MPI for inter-process communication and was demonstrated to scale efficiently on large supercomputing clusters. ParMETIS is used for partitioning the computational domains. Simulation results are outputted into VTK format that can be easily visualized in the free software Paraview. Details on how to be obtain and use svFSI is documented below.

Main features

svFSI provides many functionalities not only for cardiac modeling but for general-purpose computational studies as well. It can model heat transfer, diffusion, linear elasticity, nonlinear elastodynamics, Newtonian/Non-Newtonian flow, fluid-structure interaction, electrophysiology and so on. For an exhaustive list of functionalities provided by svFSI along with the examples, please refer to our example repository. Here we provide documentation for the main features of svFSI.

Physics Solved Documentation/Tutorial Examples
Fluid Webpage pipe flow with RCR BC;
dye transportation;
GenBC/cplBC;
Non-Newtonian flow
Structure Webpage; YouTube struct:
block compression;
passive inflation of LV model
ustruct:
block compression;
tension of arterial strip;
active inflation of LV model
Electrophysiology Webpage; YouTube Aliev-Panfilov model;
ten-Tusscher-Panfilov model;
Bueno-Orovio-Cherry-Fenton model;
Purkinje network
FSI Webpage; YouTube ALE:
2D heart valve;
2D flag behind a block;
pressure pulse inside aorta
CMM:
pipe flow with RCR BC;
vein graft

Input file format

svFSI uses plain-text input file. This input file will define the physics solved, mesh, linear solver, boundary conditions and general simulation parameters. An overview of the syntax could be found here. The input file for svFSI is a scripting code that sets values to specific parameters. A colon “:” is used as a separator between the keyword or key phrase and its value. If the parameters being set are single-valued constants, you may use a single line structure as,

<keyword or key phrase>: 1.3

First part, which is the part before “:”, is the keyword or key phrase to the parameter, and the second part, which is the part after “:”, is the value. This kind of single line structures can occur in any order in the script.

There are five types of values that can be supplied as the parameter in the single line structure:

  • logical (true/false)
  • integer values
  • real/float values
  • strings or characters
  • file paths
  • vector of real or integer values

For logical false, any of False, 0, F, f, false can be used. Likewise, for logical true, any of True, 1, T, t, true can be used. Real numbers can be formatted as {10.0, 10.0D0, 1.e1, 1.D1}. Any line that begins with # is treated as a comment. You may add comments, have space before keywords or after “:”.

If the values to be set are a part of a bigger entity, you need to use the block structure, which uses braces (“{”, “}”) to mark the scope of that entity. E.g.:

 Domain: 1 {
    Density:             0.1      # [epsilon - inf)
    Poisson ratio:       0.35     # [0.0 - (0.5-epsilon)]
 }

In this example, “Domain” is a keyword (i.e., the entity to be specified) and “1” refers to the first “Domain”. Within this entity, “Density” is set to “0.1” and “Poisson ratio” is set to “0.35”. Note that the order of lines doesn’t matter within each block enclosed by “{” and “}”. If a keyword doesn’t have a default value, it has to be specified by the user. For the optional arguments, default value is shown.

A comprehensive input template is provided here, which contains all the available parameters and their meanings. The SimVascular GUI currently supports limited input configurations. To access more advanced functions of svFSI, users are recommended to create their own input file by modifying existing templates.

Structurally, all input files are composed of three parts:

  1. General simulation parameters, such as time step size, format of results, location of results etc.
  2. Mesh information, such as volumetric mesh, surface mesh, scaling factor etc.
  3. Equation information, such as physics solved (fluid, structure, electrophysiology etc.), boundary conditions, linear solver etc.

General simulation parameters

General simulation parameters describe the basic parameters about the problem including spatial dimension, time step control, file output, option to restart, etc. They are usually defined using the single line structure. For example:

Number of spatial dimensions: 3
Number of time steps: 1600
Time step size: 0.005
Save results to VTK format: 1
Increment in saving VTK files: 10
Increment in saving restart files: 200
...

Mesh information

Mesh data is provided in the second part. It mainly includes the paths to volumetric meshes and boundary meshes. There may also be additional information such as domains, fiber orientation, initial values, prestress, etc. Multiples meshes can be loaded for the same or different equations within svFSI. svFSI supports reading multiple mesh formats as mentioned here. Regardless of the mesh format, the mesh information is provided using the block structure. Nested keywords “Add mesh” and “Add face” could be used to provide paths to the mesh and face files. The users should also provide a string-based name for each mesh and face immediately after the colon. This name will be useful when when setting boundary conditions later. For example:

Add mesh: mesh_name {
   # Volumetric mesh
   Mesh file path:    ./mesh/mesh-complete.mesh.vtu

   # Surface meshes
   Add face: face_name_1 {
      Face file path: ./mesh/mesh-surfaces/face_1.vtp
   }
   Add face: face_name_ 2 {
      Face file path: ./mesh/mesh-surfaces/face_2.vtp
   }
   Add face: face_name_3 {
      Face file path: ./mesh/mesh-surfaces/face_3.vtp
   }
}

Equation information

Information on equations being solved is provided here. It includes the type of equation, coupling, boundary conditions, settings for the linear and nonlinear solver, and output control. The type of equation can be heat equation, linear elasticity, nonlinear elastodynamics, incompressible viscous fluid, fluid-structure interaction etc. Also, physics-specific material properties can be supplied. *svFSI also provides a rich selection of linear solvers (BIPN, GMRES, BICGSTAB, CG) and preconditioners (Trilinos-Diagonal, Trilinos-BlockJacobi, Trilinos-ILU, Trilinos-ILUT, Trilinos-IC, Trilinos-ICT, Trilinos-ML). In terms of boundary conditions, we provide choices such as Dirichlet, Neumann, Robin etc. These boundary conditions can be steady, unsteady, coupled to lumped parameter network or RCR-type. Boundary conditions are supplied in a nested block structure with keyword “Add BC”. Also, the string immediately after the colon should match the face name defined in mesh section. For example:

Add equation: fluid {
   Coupled: 1
   Min iterations: 3
   Max iterations: 10
   Tolerance: 1e-3

   # Material property
   Density: 1.06
   Viscosity: Constant {Value: 0.04}

   # Linear solver configuration
   LS type: GMRES
   {
      Preconditioner: Trilinos-ILUT
      Max iterations: 100
      Tolerance:      1D-6
      Krylov space dimension: 50
   }

   # Boundary conditions
   Add BC: face_name_1 {
      Type: Dir
      Time dependence: Steady
      Value: 0.0
   }
}

Mesh format

svFSI supports multiple mesh types, such as plain text files (coordinates + connectivity), “.msh” file generated by Gambit, vtk files and NURBS meshes.

SimVascular-based vtu/vtp format for mesh/faces is the default choice. svFSI supports below element types in vtu/vtp format:

  • line (linear, quadratic)
  • triangle (linear, quadratic)
  • quadrilateral (bilinear, serendipity, biquadratic)
  • tetrahedron (linear, quadratic)
  • hexagonal brick (trilinear, quadratic/serendipity, triquadratic)
  • wedge

It is recommended to generate these mesh files using SimVascular. SimVascular GUI provides a complete pipeline that includes model construction, mesh generation, problem configuration, input file generation, and numerical simulation. Although the model construction and mesh generation have been discussed extensively in other parts of the SimVascular user guides, we will go over them briefly for the sake of completeness.

Let’s use an idealized left ventricular model as an example. First, we create a SV Project called LV_Inflation. In cardiovascular modeling, computational models are usually generated from medical images. However, as we are using an idealized ventricular geometry here, we can directly import the model into the SV Data Manager by right clicking Models and selecting Import Solid Model. The solid model can be a surface mesh stored in either stl or vtp format.

Import solid model for mesh generation.

We could name the new model LV. When asked if you would like to extract faces from the model, click yes and accept the default separation angle (50 degree). Oftentimes, the imported model is not centered in the field of view. One can right click on the model and click Global Reinit to recenter the geometry. Double click the model name to open the SV Modeling window and rename each face corresponding to its physiological interpretation (base - LV basal plane; endo - endocardium; epi - epicardium) and set each of these face types as wall. You may also change the color of each face to distinguish them in the Display window.

Extract faces from the imported model.

Next we will generate an unstructured mesh from this idealized LV model. In the SV Data Manager, right click Meshes and select Create Mesh. Accepting the default options will create a mesh object named LV. Double click the mesh object to open the SV Meshing configuration window. The most import parameter here is Global Max Edge Size, which controls the size of the elements. We will set this value to 2.0 and run the mesher. A relatively coarse mesh will be generated. Users can reduce the Global Max Edge Size if a finer mesh is desired.

Meshing configuration.

To convert the generated mesh to the svFSI-ready format, right click the mesh object, LV, and select Export Mesh Complete.

Export the complete mesh.

Open the destination folder to find the mesh files contained in a folder called <mesh object name>-mesh-complete (LV-mesh-complete in this case). In this folder, the vtu file (mesh-complete.mesh.vtu) contains the volume mesh of the ventricle. The surface (boundary) mesh files could be found in the subfolder mesh-surfaces in vtp format. This set of vtk files defines the computational domain of this problem and can be visualized in Paraview.

Layout of the mesh folder.

svFSI also supports loading NURBS meshes. Limited support is available to import a bilinear quadrilateral mesh from Gambit-based “.msh”. An option to convert this into a biquadratic mesh is also available. svFSI also supports importing mesh in the form of a list of nodal coordinates and element connectivity. It is, however, the user’s responsibility to provide information on faces. The face connectivity should include a “GlobalElementID” mapping between the face element and the corresponding mesh element to which the face belongs.

We also provide a separate tool, mesh-converter, which can convert meshes generate by Gambit, GMSH to vtk format files. It also supports converting low-order elements to high-order elements.

Nested keywords “Add mesh” and “Add face” could be used to provide paths to the mesh and face files. The users should also provide a string-based name for each mesh and face immediately after the colon.

Multiples meshes can be loaded for the same or different equations within svFSI. svFSI supports reading multiple mesh formats as noted below. Some additional information specific to an equation may also be loaded within the mesh attribute such as domains, fiber directions, and initial values.

Run Simulation

Though we can run the simulation through the SimVascular GUI. It currently does not support some advanced features. Hence, we recommend use the aforementioned input file to directly run the simulation in terminals. The MPI runs can be initiated through the following command

mpiexec -np {number of MPI processes}  {path to svFSI executable}  {path to the input file}

By default, svFSI will create a results directory called n-procs, where n is the number of MPI processes for the simulation. This directory will have vtu files that contain all the requested output fields and a log file called histor.dat.

Resources

In addition to the documentation on this page, we also maintain collections of video tutorials, pre/post processing tools and public forums.

Tutorials

Pre/Post Processing Tool

We are also maintaining a collection of useful pre and post processing tools that are compatible with svFSI: https://github.com/SimVascular/svFSI-Tools

Contact us

If you have any questions or suggestions, please feel free to contact us through one of the following ways.

  • Seek support, suggest new features or report bugs, please contact us through GitHub Issues or SimTK forum.
  • Contribute your code to svFSI, please submit a pull request through GitHub.
  • Share your novel applications of svFSI with the community, please consider contribute your case to svFSI-Tests.

Installation

svFSI is an open-source project, and its source code is available on GitHub. To suit the needs of different users, we provide different options to use/test svFSI.

Precompiled installer

If you intent to use the stable version, we provide a precompiled installer on SimTK website. We currently support Ubuntu and MacOS. It is recommended to install svFSI alongside SimVascular to take advantage of the established modeling pipeline.

The svFSI executable (either built from source or downloaded from SimTK) is not packaged into each SimVascular release, so users have to manually add its path through SimVascular GUI. The configuration is under Window -> Preference -> svFSI Simulation.

Set up path to svFSI solver in SimVascular.

Docker container

If you want to test the most recent features of svFSI, we provide the option to build and run svFSI in Docker container. All necessary files and instruction are shipped with svFSI source code under Docker directory. We also present the same material here for your reference. The Dockerfile is as follows.

FROM ubuntu:18.04

ENV DEBIAN_FRONTEND=noninteractive

# Install supporting packages
RUN apt-get update
RUN apt-get install -yq --no-install-recommends apt-utils
RUN apt-get install -yq git make cmake
RUN apt-get install -yq libblas-dev liblapack-dev
RUN apt-get install -yq gcc g++ gfortran 
RUN apt-get install -yq openmpi-bin libopenmpi-dev

# Create directory
RUN mkdir -p /home/test

# Add non-root user and set up home directory
RUN useradd testuser -u 1000 -g 100 -m -s /bin/bash
RUN chown testuser /home/test
USER testuser
WORKDIR /home/test

# Obtain source code
RUN git clone https://github.com/SimVascular/svFSI

# Compile svFSI source code
RUN mkdir Build && \
    cd Build && \
    cmake ../svFSI && \
    make

ENV PATH=$PATH:/home/test/Build/svFSI-build/bin

This Dockerfile will build svFSI executable from the most recent source code in the main repository. The following procedure has been successfully tested on MacOS Big Sur, Ubuntu 18.04 and Windows 10 with WSL 2. Assuming you already have Docker installed, please follow the steps below to run svFSI.

1. Build Docker image. In the current directory (PathtosvFSI/Docker), run the following command.

    docker build -t svfsi-image .

This may take a while. Afterwards, run the command docker images, and you should see svfsi-image.

2. Download the examples.

    git clone https://github.com/SimVascular/svFSI-Tests

3. Run the container in interactive mode.

    docker container run --cap-add=SYS_PTRACE -v "$PWD"/svFSI-Tests:/home/test/svFSI-Tests -it --rm --name svfsi-demo svfsi-image

This will open a shell prompt and you can proceed as usual. Here, --cap-add=SYS_PTRACE fixes a known issue of running openmpi in Docker.

4. Let’s take 04-fluid/06-channel-flow-2D for example. In the shell prompt, run the following commands to generate the simulation results.

    cd svFSI-Tests/04-fluid/06-channel-flow-2D && \
   mpiexec -n 4 svFSI ./svFSI_Taylor-Hood.inp

The results will be stored in 4-procs in vtu format, and can be viewed with Paraview.

5. After all tests are done, execute the following commands to exit the docker and delete the image.

    exit && \
    docker rmi <IMAGE ID>

Please note that svFSI built with this Dockerfile won’t work with any example that requires Trilinos. Trilinos takes too long to build within the Docker image, and we encourage any user that needs it to build svFSI from source.

Build from source

The source code for svFSI includes a build system in cmake. To build from source, one needs compilers for c,c++, Fortran, an MPI compiler and the LAPACK and BLAS libraries. LAPACK and BLAS must be installed in a standard location, which cmake will find automatically. For example, on OSX to install using the package manager brew (see https://brew.sh/) one might run

brew install cmake gcc open-mpi lapack 

or, on Ubuntu, run sudo apt-get install to add the following packages:

cmake
cmake-curses-gui
cmake-gui
gcc 
gfortran
libopenmpi-dev
libblas-dev
liblapack-dev
git

We recommend letting cmake find the default compilers for easy of building.


Standard Build

To build the svFSI solver, download the source code from

https://github.com/simvascular/svFSI

to a directory of your choosing. Make a build directory and go there.

mkdir svFSI\_build && cd svFSI\_build 

Initiate the cmake terminal interface to generate makefiles.

ccmake ../svFSI 

This will automatically search for compilers. Follow instructions if necessary. Push “c” for configure repeatedly until cmake presents the option “g” for generate. Hit “g” to create makefiles and exit. Run make:

make 

This will place the solver binary, called “svFSI” in a directory called svFSI_build/svFSI-build/bin. This will also create a script svFSI_build/svFSI-build/mysvfsi. If non-default compilers were passed to cmake, then runtime errors can occur, especially relating to libraries. If this occurs, try using the script, which will set paths appropriately.


Optional Trilinos package

The svFSI solver contains its own linear solver. Optionally, the advanced user may link to the Trilinos package. To accomplish this, the user must build Trilinos using cmake, then provide a path and change a single flag. Before building svFSI, download the Trininos source from github.

git clone https://github.com/trilinos/Trilinos.git

We recommend compiling the following branch, with which we have tested Trilinos.

git checkout remotes/origin/trilinos-release-12-10-branch

Make a build directory and go there.

mkdir Trilinos\_build && cd Trilinos\_build 

Run the cmake GUI to build Trilinos.

ccmake -DCMAKE_BUILD_TYPE=RELEASE \
      -DCMAKE_INSTALL_PREFIX:PATH=~/Trilinos_build \
      -DTPL_ENABLE_MPI:BOOL=ON \
      -DTrilinos_ENABLE_Amesos:BOOL=ON \
      -DTrilinos_ENABLE_AztecOO:BOOL=ON \
      -DTrilinos_ENABLE_Epetra:BOOL=ON \
      -DTrilinos_ENABLE_Ifpack:BOOL=ON \
      -DTrilinos_ENABLE_ML:Bool=ON \
      -DTrilinos_ENABLE_MueLu:Bool=ON \
      -DTrilinos_ENABLE_TESTS:BOOL=OFF \
      ../Trilinos

Note that CMAKEINSTALLPREFIX should point to your build directory. Run make to build the Trilinos source

make 

Run make install to place header and library files

make install 

This will create a directories lib/cmake/Trilinos and inside them a file called TrilinosConfig.cmake

lib/cmake/Trilinos/TrilinosConfig.cmake

If this directory does not exist or is empty, then an error has occurred.

In the svFSI cmake source, change the SVUSETRILINOS option to on. This can be found in svFSI/Code/CMake/SimVascularOptions.cmake.

option(SV_USE_TRILINOS "Use Trilinos Library with svFSI" ON)

Now, when compiling svFSI, add

ccmake -DCMAKE_PREFIX_PATH=~/Trilinos_build/lib/cmake/Trilinos  ../svFSI

Run make.

make 

If Trilinos is found you should see output that says “Found Trilinos!” and displays the associated variables. The output should look something like this, wherein the ellipsis contain many libraries and packages.

Found Trilinos!  Here are the details: 
   Trilinos_DIR = ~/Trilinos_build/lib/cmake/Trilinos
   Trilinos_VERSION = 12.10.1
   Trilinos_PACKAGE_LIST = MueLu;
   ...
   Trilinos_INCLUDE_DIRS = /Users/alex/sfw/Trilinos_build/include
   Trilinos_TPL_LIST = DLlib;LAPACK;BLAS;MPI;Pthread
   Trilinos_TPL_INCLUDE_DIRS = 
   Trilinos_TPL_LIBRARIES = /usr/lib/libdl.dylib
   ...
   Trilinos_BUILD_SHARED_LIBS = FALSE
   Trilinos_CXX_COMPILER = /usr/local/bin/mpicxx
   Trilinos_C_COMPILER = /usr/local/bin/mpicc
   Trilinos_Fortran_COMPILER = /usr/local/bin/mpif90
   Trilinos_CXX_COMPILER_FLAGS =  -std=c++11 -O3 -DNDEBUG
   Trilinos_C_COMPILER_FLAGS =  -O3 -DNDEBUG
   Trilinos_Fortran_COMPILER_FLAGS =  -O3
   Trilinos_LINKER = /usr/bin/ld
   Trilinos_EXTRA_LD_FLAGS = 
   Trilinos_AR = /usr/bin/ar
End of Trilinos details

Setting WITH_TRILINOS to true

If these are filled in, then cmake has found Trilinos and it will be linked. The Trilinos linear solvers are then available for use. In the linear solver options there are now the following Trilinos options for preconditioners.

Trilinos-Diagonal
Trilinos-BlockJacobi 
Trilinos-ILU
Trilinos-ILUT
Trilinos-IC
Trilinos-ML

This will automatically set the Trilinos linear solvers to be called, rather than the svFSILS linear solvers. The three options are GMRES,BICG,CG. There is currently no option to call Trilinos without a preconditioner. If one wishes to use the simplest possible preconditioning with Trilinos, use Trilinos-Diagonal.

For example, set

LS type: GMRES
{
  Preconditioner: Trilinos-ILU
  Tolerance:           1D-3
  Krylov space dimension: 200
}