|
svFSIplus
|
This document describes some of the implementation details of svFSIplus C++ code.
svFSIplus is essentially a direct line-by-line translation of the svFSI Fortran code into C++. This provides a simple mapping between the code of the original Fortran and C++ versions and aided in debugging the C++ code.
The C++ implementation differs from the Fortran implementation in four fundamental ways 1) Custom C++ Array and Vector classes to reproduce Fortran dynamic arrays 2) XML format file replaces the plain-text input file to specify simulation parameters 3) Direct calls to the VTK API replaces the custom code used to read/write VTK format files 4) Uses 0-based indexing into arrays
What was not converted 1) Shells 2) NURBS 3) Immersed Boundary
The following sections describe how the C++ implementation is organized and how it replicates the data structures and flow of control of the Fortran implementation. Some important details of the C++ implementation will also be discussed.
The C++ implementation attempts to replicate the data structures and flow of control of the Fortran implementation and to maintains its organization. The svFSI Fortran is about 58K lines of code spread over about 100 files.
Most of the Fortran code is replicated in C++ using the same file and procedure names converted to lower case with underscores to improve readability. For example
All Fortan procedures located in a particular file will typically have a C++ implementation in a similarly named file. This was done to maintain a simple mapping between the locations of the C++ and Fortran code.
The Fortan svFSI code is implemented using a procedural programming paradigm where data is passed to procedures to carry out a series of computational steps. It makes little or no use of the object orientation features providing by Fortran90. This organization is reproduced in the C++ implementation so there are essentially no class methods used in the core simulation code.
C++ functions are defined within a namespace defined for each Fortran file. For example, the functions in load_msh.cpp are defined within the load_msh namespace. Some namespaces contain a _ns suffix to prevent conflicts with function names (e.g. read_files_ns).
All simulation data is stored in the Simulation class.
This section provides some details about how the svFSI Fortran code was translated into C++ code. This will help to convert any new Fortran code developed in the Fortan svFSI code not included in svFSIplus.
svFSIplus is essentially a direct line-by-line translation of the svFSI Fortran code. The original Fortran variable names are typically small, contain no underscores for readability and are often ambiguous. However, the same variable names are used in both the C++ and Fortran codes in order to maintain a clear relationship between them.
For example, the following section of Fortran code
is replaced by the following section of C++ code
In this example the Fortran DO loops are replaced by C++ for loops using C++ 0-based indexing. Array indexing is discussed in the Fortran Dynamic Arrays section below.
Modules were introduced in Fortran to moralize a large code by splitting it into separate files containing procedures and data specific to a certain application. A module is like a C++ class because it can encapsulate both data and procedures. The svFSI Fortran code uses modules primarily to store and access global variables.
C++ classes are used to implement Fortran modules. Fortran variable names are retained to prevent (or maintain) confusion. A C++ module name uses the same Fortan name converted to camel case. For example, several of the Fortan module names and the files that implements them are given below with the corresponding C++ class name and implementation files.
The Fortan USE command provides access to all the variables defined in a module. Almost all of the svFSI Fortran procedures have a USE COMMOD command that provides access to all of the global variables (about 90) defined in the COMMOD module. For example
svFSIplus does not use any global variables. A C++ module object is passed to each procedure that needs to access its variables. For example, in C++ the ComMod object com_mod is explicitly passed to the construct_usolid function.
All C++ modules are stored as member data in the Simulation Class.
Fortran dynamic arrays have been reproduced using custom Vector, [Array](array_vector_class) and [Array3](array_vector_class) C++ class templates. Note that these user-defined classes will most likely be replaced by a more sophisticated matrix package such as Eigen.
Fortran dynamic arrays are declared using the ALLOCATABLE attribute. For example the REAL, ALLOCATABLE :: A(:,:) statement declares the two dimensional array of type REAL named A. The Fortran ALLOCATE A(3,10) statement then dynamically creates storage for A as a 3x10 array. The DEALLOCATE A statement is used to return the memory used by A. Relocatable arrays are automatically reallocated when going out of scope.
C++ dynamic arrays are declared using the Array<T> template, where T is the array data type: double or int. The two dimensional array of type double named A is declared and memory allocated using Array<double> A(3,10);. Memory is released when A goes out of scope or is explicitly freed using the clear() method.
C++ multidimensional arrays are referenced using 0-based indexing and are traversed in column-major order like Fortran. Array indexes use parenthesis A(i,j) not brackets A[i][j] to access array elements.
For example, the following sections of Fortran code that declare and use dynamics arrays
are replaced by the following section of C++ code
Note that the : array operator used to copy a column of an array is part of the Fortran language. It was not always possible to efficiently (i.e. memory-to-memory copy) and cleanly replace Fortran array operators by C++ Array template methods. In the above example the Fortran : operator was replaced in C++ by an explicit for loop.
The C++ Simulation class encapsulates all of the objects (Fortran modules) used to store simulation data. It also contains a Parameters object used to store simulation parameters read in from an XML file.
The Simulation class does not contain any methods used in the core simulation code. Like the Fortan svFSI code it is only used to pass data to procedures to carry out a series of computational steps.
Fortran dynamic arrays have been reproduced using custom Vector, Array and Array3 C++ class templates. Note that these custom class templates will most likely be replaced by a more sophisticated matrix package such as Eigen.
The class templates are able to reproduce much of the functionality of Fortran arrays and array intrinsic functions (e.g. sum). The challenge is to create class methods that are as efficient as the Fortan array operators. Because the operators are part of the Fortran language the compiler can optimize them as a efficient memory-to-memory copies. For example
The objects created from class templates are not part of the C++ language like arrays (i.e. double A[100]). They have the overhead associated with all C++ objects (construct/destroy). Object copy and assignment operators must also be handled efficiently so that intermediate objects are not created and extra data copys are avoided.
The Vector, Array and Array3 class templates have a data() method that returns a point to the object's internal memory. This is need for MPI calls that take raw C pointers as arguments. For example
The class templates are defined in the Vector.h, Array.h and Array3.h files.
Objects can be created using its constructor
or defined and later resized
Object memory is initialized to 0.
An object's memory is freed using its clear() method
or when it goes out of scope.
C++ multidimensional arrays are referenced using 0-based indexing and are traversed in column-major order like Fortran. Array indexes use parenthesis A(i,j) not brackets A[i][j] to access array elements. This was done to make C++ code look more like Fortran and to simplify the conversion process.
Indexes can be checked by defining the _check_enabled directive within each template include file. An index out of bounds will throw an std::runtime_error exception. Note that index checking will substantially slow down a simulation so it should be disabled when not testing.
Class templates support most mathematical operators: =,+,-,*,/,+=
Some Fortran array intrinsic (e.g. abs, sqrt) are also supported.
Example
The Array * operator performs an element-by-element multiplication, not a matrix multiplication. This was done to replicate Fortran.
It is more efficient to use the += operator A += B than A = A + B which performs a copy.
A lot of Fortran code in svFSI operates on a column of a 2D array. For example
where fs(1)N(:,g) gets the column g of the fs(2)N array.
The operation of getting a column of data from an Array object is supported using two different methods
Use the col method if the column data is not going to be modified
Use the rcol method if the column data is going to be modified; it might also help to speed up a procedure that is called a lot (e.g. in material models).
A lot of Fortran code in svFSI operates on a slice (a 2D sub-array) of a 3D array. For example
where fs(1)Nx(:,:,g) gets a slice (2D sub-array) g of the fs(1)Nx array.
The operation of getting a slice of data from an Array3 object is supported using two different methods
The rslice() method returns an Array object whose internal data is a pointer to the internal data of an Array3 object. This was done to reduce the overhead of copying sub-arrays in some sections of the custom linear algebra code. The Array object will not free its data if it is a reference to the data of a Array3 object. Use the rslice method if the slice data is going to be modified. It can also speed up code that repeatedly extracts sub-arrays used in computations but are not modified.
The Fortran code made use of 0-size arrays in several places, using ALLOCATE with a zero size. For some reason Fortran is OK with using these 0-size arrays.
The C++ code reproduces this by allowing Array objects to be allocated with 0 size rows and columns. This is a total hack but it allowed to get the C++ code working without having to rewrite a lot of code.
The Fortan svFSI solver read in simulation parameters in a custom text format. All parameters were read in at startup and stored as an array of characters (string) using custom code. Parameter values were then retrieved during various stages of the computation. The string representation was converted when the value of a parameter was needed. An error in a parameter value could therefore not be detected until later stages of the computation (e.g., when the mesh is being distributed over processors).
svFSIplus solver simulation parameters are stored in the Extensible Markup Language (XML) file format. XML is a simple text-based format for representing structured data as element tree. The XML tree starts at a root element and branches from the root to sub-elements. All elements can have sub-elements. An XML tag is a markup construct that begins with < and ends with >. There are three types of tag: 1) start-tag, such as <section> 2) end-tag, such as </section> 3) empty-element tag, such as <line-break />
XML tags represent data structures and contain metadata. An XML element is a logical component that either begins with a start-tag and ends with a matching end-tag or consists only of an empty-element tag. The characters between the start-tag and end-tag, if any, are the element's content, and may contain markup, including other elements, which are called child elements. An attribute is a markup construct consisting of a name–value pair that exists within a start-tag or empty-element tag.
Example:
The elements in the svFSIplus simulation file are represented by sections of related parameters. Sub-elements are referred to as sub-sections. The svFSIplus simulation file has four top-level sections
The Parameters class is used to read and store simulation parameters parsed from an XML file using using tinyxml2. Parameter types are checked as they are read so errors in parameter values are immediately detected.
The Parameters class contains objects for each of the top-level sections in the parameters file
Each section is represented as a class containing objects for the parameters defined for that section and objects representing any sub-sections. Objects representing parameters are named the same as the name used in the XML file except with a lower case first character. Each parameter has a name and a value with as a basic type (bool, double, int, etc.) using the Parameter template class.
Example: MeshParameters class parameter objects
All section classes inherit from the ParameterLists class which has methods to set parameter values and store them in a map for processing (e.g. checking that all required parameters have been set).
Parameter names and default values are set in each section object constructor using member data. The ParameterLists::set_parameter() sets the name and default value for a parameter, and if a value for it is required to be given in the XML file.
Example: Setting parameter names and values in the MeshParameters constructor
Parameter values are set using the 'set_values()' method which contains calls to tinyxml2 to parse parameter values from an XML file. The XML elements within a section are extracted in a while loop. Sub-sections or data will need to be checked and processed. The 'ParameterLists::set_parameter_value()' method is used to set the value of a parameter from a string.
Example: Parsing XML and setting parameter values in MeshParameters::set_values()
Sections that contain simple elements (i.e., no sub-sections or special data processing) can be automatically parsed.
Example: Automatically parsing XML and setting parameter values in LinearSolverParameters::set_values(tinyxml2::XMLElement* xml_elem)
Parameter values are accessed from the core simulation code using the Simulation object's Parameters object. The Parameter template class () operator or value() method is used to access the parameter's value, the defined() method is used to check if a parameter's value has been set.
Example: Accessing parameter values
Performance
The following sections briefly outline some problems that might cause simulation failures or incorrect results.
There may still be indexing mistakes
There are a lot places in the code that uses indexes to offset into arrays.
The Fortran code uses of 0-size arrays in several places. The C++ code reproduced this kind of functionality using tests of array size and adding the allocation of 0-sized Array objects. This hack might could fail under certain circumstances.
This section covers some of the C++ implementation details that may be useful to developers adding new capabilities to svFSIplus.
svFSIPlus does not dynamically allocated objects except in Array and Vector classes and C++ containers. All objects defined in module classes are allocated statically and are referenced using a dot. This provided a much cleaner translation from Fortran to C++ by replacing % with ..
Access objects as references to avoid copying
In svFSI constants are defined in the CONSTS.f file using the Fortan PARAMETER statement
svFSIplus uses enum class types defined in consts.h
Constants are accessed using
Some constants have a short-hand representation
which are used like this
This section describes the coding standards and guidelines that must be followed when adding new code to svFSIplus.
Coding standards are important for producing good software
Code that does not follow coding standards will be rejected during code review. However, program structures not mentioned in the following sections may be coded in any (non-hideous) manner favoured by the developer.
Note that svFSIplus maintained the program structure and naming conventions used by the svFSI Fortran code so some of the following coding standards may be violated.
Indentation refers to the spaces at the beginning of a code line. It is a fundamental aspect of code styling and improves readability by showing the overall structure of the code.
Indentation is two spaces for all programming structures: functions, loops, if-then blocks, etc. Do not use tabs to indent.
The if-else class of statements should have the following form
A for statement should have the following form
A switch statement should have the following form
The braces indicating a function body are placed in column 1, function body statements are indented by two spaces.
Whitespace is a term that refers to the spaces and newlines that are used to make code more readable.
The following white space conventions should be followed
Some complex expressions may be better organized without single separating spaces. The following could be written using spaces between sub-expressions only
or written using double spacing between sub-expressions
Use newlines often to separate logical blocks of code: for-loops, if statements, related code blocks.
Refer to the elements in the std namespace by explicit qualification using std::.
It is acceptable to use unqualified names for svFSIplus namespaces
Program readability is improved by using names that would be clear to any developer.
Two naming styles are used svFSIplus
CamelCase is a way to separate the words in a phrase by making the first letter of each word capitalized and not using spaces.
C++ files should end in .cpp and header files should end in .h.
Files that contain a single class should have the name of the class, including capitalization.
Type names are in CamelCase.
Variable and function names use snake_case.
Data members of classes additionally have trailing underscores.
Comments are absolutely vital to keeping code readable. While comments are very important, the best code is self-documenting. Giving sensible names to types and variables is much better than using obscure names that you must then explain through comments.
Don't literally describe what code does unless the behavior is nonobvious to a reader who understands C++ well. Instead, provide higher level comments that describe why the code does what it does, or make the code self describing.
Comments should be included relative to their position in the code
Start each file with license boilerplate. Every file should contain license boilerplate.
File comments describe the contents of a file. A .h file will contain comments describing any classes defined there. A .cpp file will contain comments describing the purpose of the functions or class methods defined there.
Do not duplicate comments in both the .h and the .cpp files.
Every non-obvious class or struct declaration should have an accompanying comment that describes what it is for and how it should be used.
Comments describing the use of the class should go together with its interface definition; comments about the class operation and implementation should accompany the implementation of the class's methods.
Every function declaration should have comments immediately preceding it that describe what the function does and how to use it. If there is anything tricky about how a function does its job, the function definition should have an explanatory comment.
The function implementation should have comments describing tricky, non-obvious, interesting, or important parts of the code.
Function comments should follow Doxygen format for API functions.
Non API functions should have the form
Variables should be initialized where they are declared when possible. This ensures that variables are valid at any time.
Variables must never have dual meaning. This ensures that all concepts are represented uniquely.
Global variable use should be minimized. In C++ there is no reason that global variables need to be used at all.
Variables should be declared in the smallest scope possible. By keeping the operations on a variable within a small scope it is easier to control the effects and side effects of the variable.
Use nullptr instead of 0 and NULL.
Use const rather than #define statements.
Avoid deeply nested code. Code that is too deeply nested is hard to both read and debug. One should replace excessive nesting with function calls.
This section describes the coding guidelines that are recommend when adding new code to svFSIplus.
Where possible, put enums in appropriate classes
Type conversions should be avoided if possible.
When required, type conversions must always be done explicitly using C++ style casts. Never rely on implicit type conversion.
Arguments that are non-primitive types and will not be modified should be passed by const reference.
Output parameters should grouped at the end of the function's parameters.
1.9.1