# FeenoX Software Design Specification

Abstract. This Software Design Specifications (SDS) document applies to an imaginary Software Requirement Specifications (SRS) document issued by a fictitious agency asking for vendors to offer a free and open source cloud-based computational tool to solve engineering problems. The latter can be seen as a request for quotation and the former as an offer for the fictitious tender. Each section of this SDS addresses one section of the SRS. The original text from the SRS is shown quoted at the very beginning before the actual SDS content.

# 1 Introduction

A computational tool (herein after referred to as the tool) specifically designed to be executed in arbitrarily-scalable remote server (i.e. in the cloud) is required in order to solve engineering problems following the current state-of-the-art methods and technologies impacting the high-performance computing world. This (imaginary but plausible) Software Requirements Specification document describes the mandatory features this tool ought to have and lists some features which would be nice the tool had. Also it contains requirements and guidelines about architecture, execution and interfaces in order to fulfill the needs of cognizant engineers as of 2022 (and the years to come) are defined.

On the one hand, the tool should allow to solve industrial problems under stringent efficiency (sec. 2.4) and quality (sec. 4) requirements. It is therefore mandatory to be able to assess the source code for

• independent verification, and/or
• performance profiling, and/or
• quality control

by qualified third parties from all around the world, so it has to be open source according to the definition of the Open Source Initiative.

On the other hand, the initial version of the tool is expected to provide a basic functionality which might be extended (sec. 1.1 and sec. 2.7) by academic researchers and/or professional programmers. It thus should also be free—in the sense of freedom, not in the sense of price—as defined by the Free Software Foundation. There is no requirement on the pricing scheme, which is up to the vendor to define in the offer along with the detailed licensing terms. These should allow users to solve their problems the way they need and, eventually, to modify and improve the tool to suit their needs. If they cannot program themselves, they should have the freedom to hire somebody to do it for them.

FeenoX is a cloud-first computational tool aimed at solving engineering problems.

To better illustrate FeenoX’ unfair advantage (in the entrepreneurial sense), let us first consider what the options are when an engineer needs to to write a technical report, paper or document:

Feature Microsoft Word Google Docs Markdown1 (La)TeX
Aesthetics
Convertibility (to other formats) 😐 😐 😐
Traceability 😐
Mobile-friendliness
Collaborativeness 😐
Licensing/openness
Non-nerd friendliness 😐

After analyzing the pros and cons of each alternative, at some point it should be evident that Markdown (plus friends) gives the best trade off. We can then perform a similar analysis for the options available in order to solve an engineering problem casted as a partial differential equation, say by using a finite-element formulation:

Feature Desktop GUIs Web frontends FeenoX2 Libraries
Flexibility 😐
Scalability 😐
Traceability 😐
Cloud-friendliness
Collaborativeness
Licensing/openness ✅/😐/❌
Non-nerd friendliness 😐

Therefore, on the one hand, FeenoX is—in a certain sense—to desktop FEA programs (like Code_Aster with Salome-Meca or CalculiX with PrePoMax) and libraries (like MoFEM or Sparselizard) what Markdown is to Word and (La)TeX, respectively and deliberately.

Besides noting that software being free (regarding freedom, not price) does not imply the same as being open source, the requirement is clear in that the tool has to be both free and open source, a combination which is usually called FOSS.

FeenoX is licensed under the terms of the GNU General Public License version 3 or, at the user convenience, any later version. This means that users get the four essential freedoms:3

1. The freedom to run the program as they wish, for any purpose.
2. The freedom to study how the program works, and change it so it does their computing as they wish.
3. The freedom to redistribute copies so they can help others.
4. The freedom to distribute copies of their modified versions to others.

So a free program has to be open source, but it also has to explicitly provide the four freedoms above both through the written license and through the mechanisms available to get, modify, compile, run and document these modifications. That is why licensing FeenoX as GPLv3+ also implies that the source code and all the scripts and makefiles needed to compile and run it are available for anyone that requires it. Anyone wanting to modify the program either to fix bugs, improve it or add new features is free to do so. And if they do not know how to program, the have the freedom to hire a programmer to do it without needing to ask permission to the original authors.

It should be noted that not only is FeenoX free and open source, but also all of the libraries it depends (and their dependencies) are. It can also be compiled using free and open source build tool chains running over free and open source operating systems. In addition, the FeenoX documentation is licensed under the terms of the GNU Free Documentation License v1.3 (or any later version).

To sum up this section, FeenoX is…

1. a cloud-first computational tool (not just cloud friendly, but cloud first).
2. both free (as in freedom) and open source.

## 1.1 Objective

The main objective of the tool is to be able to solve engineering problems which are usually casted as differential-algebraic equations (DAEs) or partial differential equations (PDEs), such as

• heat conduction
• mechanical elasticity
• structural modal analysis
• frequency studies
• electromagnetism
• chemical diffusion
• process control dynamics
• computational fluid dynamics

on one or more mainstream cloud servers, i.e. computers with hardware and operating systems (futher discussed in sec. 2) that allows them to be available online and accessed remotely either interactively or automatically by other computers as well. Other architectures such as high-end desktop personal computers or even low-end laptops might be supported but they should not the main target (i.e. the tool has to be cloud-first but laptop-friendly).

The initial version of the tool must be able to handle a subset of the above list of problem types. Afterward, the set of supported problem types, models, equations and features of the tool should grow to include other models as well, as required in sec. 2.7.

The choice of the initial supported features is based on the types of problem that the FeenoX’ precursor codes (namely wasora, Fino and milonga, referred to as “previous versions” from now on) already have been supporting since more than ten years now. It is also a first usable version so scope can be bounded. A subsequent road map and release plans can be designed as requested. FeenoX’ first version includes a subset of the required functionality, namely

• open and closed-loop dynamical systems
• Laplace/Poisson/Helmholtz equations
• heat conduction
• mechanical elasticity
• structural modal analysis
• multi-group neutron transport and diffusion

Sec. 2.7 explains the mechanisms that FeenoX provides in order to add (or even remove) other types of problems to be solved.

Recalling that FeenoX is a “cloud-first” tool, it is designed to be developed and executed primarily on GNU/Linux hosts, which is the architecture of more than 90% of the internet servers which we collectively call “the public cloud.” It should be noted that GNU/Linux is a POSIX-compliant operating system which is compatible with Unix, and that FeenoX follows the rules of Unix philosophy (further explained in sec. 6) for the actual computational implementation. Besides the POSIX standard, as explained further below in sec. 2.5, FeenoX also uses MPI which is a well-known industry standard for massive execution of parallel processes, both in multi-core hosts and multi-hosts environments. Finally, if performance and/or scalability are not important issues, FeenoX can be run in a (properly cooled) local PC, laptop or even in embedded systems such as Raspberry Pi (see sec. 2).

## 1.2 Scope

The tool should allow users to define the problem to be solved programmatically. That is to say, the problem should be completely defined using one or more files either…

1. specifically formatted for the tool to read such as JSON or a particular input format (historically called input decks in punched-card days), and/or
2. written in an high-level interpreted language such as Python or Julia.

It should be noted that a graphical user interface is not required. The tool may include one, but it should be able to run without needing any interactive user intervention rather than the preparation of a set of input files. Nevertheless, the tool might allow a GUI to be used. For example, for a basic usage involving simple cases, a user interface engine should be able to create these problem-definition files in order to give access to less advanced users to the tool using a desktop, mobile and/or web-based interface in order to run the actual tool without needing to manually prepare the actual input files.

However, for general usage, users should be able to completely define the problem (or set of problems, i.e. a parametric study) they want to solve in one or more input files and to obtain one or more output files containing the desired results, either a set of scalar outputs (such as maximum stresses or mean temperatures), and/or a detailed time and/or spatial distribution. If needed, a discretization of the domain may to be taken as a known input, i.e. the tool is not required to create the mesh as long as a suitable mesher can be employed using a similar workflow as the one specified in this SRS.

The tool should define and document (sec. 4.6) the way the input files for a solving particular problem are to be prepared (sec. 3.1) and how the results are to be written (sec. 3.2). Any GUI, pre-processor, post-processor or other related graphical tool used to provide a graphical interface for the user should integrate in the workflow described in the preceding paragraph: a pre-processor should create the input files needed for the tool and a post-processor should read the output files created by the tool.

Since FeenoX is designed to be executed in the cloud, it works very much like a transfer function between one (or more) files and zero or more output files:

                             +------------+
mesh (*.msh)  }             |            |             { terminal
data (*.dat)  } input ----> |   FeenoX   |----> output { data files
input (*.fee) }             |            |             { post (vtk/msh)
+------------+

Technically speaking, FeenoX can be seen as a Unix filter designed to read an ASCII-based stream of characters (i.e. the input file, which in turn can include other input files or contain instructions to read data from mesh and/or other data files) and to write ASCII-formatted data into the standard output and/or other files. The input file can be prepared either by a human or by another program. The output stream and/or files can be read by either a human and/or another programs. A quotation from Eric Raymond’s The Art of Unix Programming helps to illustrate this idea:

Doug McIlroy, the inventor of Unix pipes and one of the founders of the Unix tradition, had this to say at the time:

1. Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new features.

2. Expect the output of every program to become the input to another, as yet unknown, program. Don’t clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don’t insist on interactive input.

[…]

He later summarized it this way (quoted in “A Quarter Century of Unix” in 1994):

• This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface.

Keep in mind that even though both the quotes above and many finite-element programs that are still mainstream today date both from the early 1970s, fifty years later the latter still

• do not make just only one thing well,
• do complicate old programs by adding new features,
• do not expect the their output to become the input to another,
• do clutter output with extraneous information,
• do use stringently columnar and/or binary input (and output!) formats, and/or
• do insist on interactive output.

There are other FEA tools that even though born closer in time, also follow the above bullets literally.

For example, let us consider the famous chaotic Lorenz’ dynamical system. Here is one way of getting an image of the butterfly-shaped attractor using FeenoX to compute it and Gnuplot to draw it. Solve

\begin{cases} \dot{x} &= \sigma \cdot (y - x) \\ \dot{y} &= x \cdot (r - z) - y \\ \dot{z} &= x y - b z \\ \end{cases}

for 0 < t < 40 with initial conditions

\begin{cases} x(0) = -11 \\ y(0) = -16 \\ z(0) = 22.5 \\ \end{cases}

and \sigma=10, r=28 and b=8/3, which are the classical parameters that generate the butterfly as presented by Edward Lorenz back in his seminal 1963 paper Deterministic non-periodic flow.

The following ASCII input file ressembles the parameters, inital conditions and differential equations of the problem as naturally as possible:

PHASE_SPACE x y z     # Lorenz attractor’s phase space is x-y-z
end_time = 40         # we go from t=0 to 40 non-dimensional units

sigma = 10            # the original parameters from the 1963 paper
r = 28
b = 8/3

x_0 = -11             # initial conditions
y_0 = -16
z_0 = 22.5

# the dynamical system's equations written as naturally as possible
x_dot = sigma*(y - x)
y_dot = x*(r - z) - y
z_dot = x*y - b*z

PRINT t x y z        # four-column plain-ASCII output

Indeed, when executing FeenoX with this input file, we get four ASCII columns (t, x, y and z) which we can then redirect to a file and plot it with a standard tool such as Gnuplot. Note the importance of relying on plain ASCII text formats both for input and output, as recommended by the UNIX philosophy and the rule of composition: other programs can easily create inputs for FeenoX and other programs can easily understand FeenoX’ outputs. This is essentially how UNIX filters and pipes work.

As already stated, FeenoX is designed and implemented following the UNIX philosophy in general and Eric Raymond’s 17 Unix Rules ([sec:unix]) in particular. One of the main ideas is the rule of separation that essentially asks to separate mechanism from policy, that in the computational engineering world translates into separating the frontend from the backend. The usage of FeenoX to compute and of Gnuplot to plot is a clear example of separation. Same idea applies to partial differential equations (PDEs), where the mesh is created with Gmsh and the output can be post-processed with Gmsh, Paraview or any other post-processing system (even a web-based interface) that follows rule of separation. Even though most FEA programs eventually separate the interface from the solver up to some degree, there are cases in which they are still dependent such that changing the former needs updating the latter.

From the very beginning, FeenoX is designed as a pure backend which should nevertheless provide appropriate mechanisms for different frontends to be able to communicate and to provide a friendly interface for the final user. Yet, the separation is complete in the sense that the nature of the frontends can radically change (say from a desktop-based point-and-click program to a web-based immersive augmented-reality application) without needing the modify the backend. Not only far more flexibility is given by following this path, but also develop efficiency and quality is encouraged since programmers working on the lower-level of an engineering tool usually do not have the skills needed to write good user-experience interfaces, and conversely.

In the very same sense, FeenoX does not discretize continuous domains for PDE problems itself, but relies on separate tools for this end. Fortunately, there already exists one meshing tool which is FOSS (GPLv2) and shares most (if not all) of the design basis principles with FeenoX: the three-dimensional finite element mesh generator Gmsh.

Strictly speaking, FeenoX does not need to be used along with Gmsh but with any other mesher able to write meshes in Gmsh’s format .msh. But since Gmsh also

• is free and open source,
• works also in a transfer-function-like fashion,
• runs natively on GNU/Linux,
• has a similar (but more comprehensive) API for Python/Julia,
• etc.

it is a perfect match for FeenoX. Even more, it provides suitable domain decomposition methods (through other FOSS third-party libraries such as Metis) for scaling up large problems.

Let us solve the linear elasticity benchmark problem NAFEMS LE10 “Thick plate pressure.” Assuming a proper mesh has already been created in Gmsh, note how well the FeenoX input file matches the problem statement from fig. 2:

# NAFEMS Benchmark LE-10: thick plate pressure
PROBLEM mechanical DIMENSIONS 3
READ_MESH nafems-le10.msh   # mesh in millimeters

BC upper    p=1      # 1 Mpa

# BOUNDARY CONDITIONS:
BC DCD'C'   v=0      # Face DCD'C' zero y-displacement
BC ABA'B'   u=0      # Face ABA'B' zero x-displacement
BC BCB'C'   u=0 v=0  # Face BCB'C' x and y displ. fixed
BC midplane w=0      #  z displacements fixed along mid-plane

# MATERIAL PROPERTIES: isotropic single-material properties
E = 210e3   # Young modulus in MPa
nu = 0.3    # Poisson's ratio

SOLVE_PROBLEM   # solve!

# print the direct stress y at D (and nothing more)
PRINT "σ_y @ D = " sigmay(2000,0,300) "MPa"

The problem asks for the normal stress in the y direction \sigma_y at point “D,” which is what FeenoX writes (and nothing else, rule of economy):

$feenox nafems-le10.fee sigma_y @ D = -5.38016 MPa$ 

Also note that since there is only one material there is no need to do an explicit link between material properties and physical volumes in the mesh (rule of simplicity). And since the properties are uniform and isotropic, a single global scalar for E and a global single scalar for \nu are enough.

For the sake of visual completeness, post-processing data with the scalar distribution of \sigma_y and the vector field of displacements [u,v,w] can be created by adding one line to the input file:

WRITE_MESH nafems-le10.vtk sigmay VECTOR u v w

This VTK file can then be post-processed to create interactive 3D views, still screenshots, browser and mobile-friendly webGL models, etc. In particular, using Paraview one can get a colorful bitmapped PNG (the displacements are far more interesting than the stresses in this problem).

See https://www.caeplex.com for a mobile-friendly web-based interface for solving finite elements in the cloud directly from the browser.

Even though the initial version of FeenoX does not provide an API for high-level interpreted languages such as Python or Julia, the code is written in such a way that this feature can be added without needing a major refactoring. This will allow to fully define a problem in a procedural way, increasing also flexibility.

# 2 Architecture

The tool must be aimed at being executed unattended on remote servers which are expected to have a mainstream (as of the 202s) architecture regarding operating system (GNU/Linux variants and other UNIX-like OSes) and hardware stack, such as

• a few Intel-compatible CPUs per host
• a few levels of memory caches
• a few gigabytes of random-access memory
• several gigabytes of solid-statee storage

It should successfully run on

• bare-metal
• virtual servers
• containerized images

using standard compilers, dependencies and libraries already available in the repositories of most current operating systems distributions.

Preference should be given to open source compilers, dependencies and libraries. Small problems might be executed in a single host but large problems ought to be split through several server instances depending on the processing and memory requirements. The computational implementation should adhere to open and well-established parallelization standards.

Ability to run on local desktop personal computers and/laptops is not required but suggested as a mean of giving the opportunity to users to test and debug small coarse computational models before launching the large computation on a HPC cluster or on a set of scalable cloud instances. Support for non-GNU/Linux operating systems is not required but also suggested.

Mobile platforms such as tablets and phones are not suitable to run engineering simulations due to their lack of proper electronic cooling mechanisms. They are suggested to be used to control one (or more) instances of the tool running on the cloud, and even to pre and post process results through mobile and/or web interfaces.

FeenoX can be seen as a third-system effect, being the third version written from scratch after a first implementation in 2009 and an second one which was far more complex and had far more features circa 2012–2014. The third attempt explicitly addresses the “do one thing well” idea from Unix.

Furthermore, not only is FeenoX itself both free and open-source software but, following the rule of composition, it also is designed to connect and to work with other free and open source software such as

• Gmsh for pre and/or post-processing
• ParaView for post-processing
• Gnuplot for plotting 1D/2D results
• Pyxplot for plotting 1D results
• Pandoc for creating tables and documents
• TeX for creating tables and documents

and many others, which are readily available in all major GNU/Linux distributions.

FeenoX also makes use of high-quality free and open source mathematical libraries which contain numerical methods designed by mathematicians and implemented by professional programmers. In particular, it depends on

Therefore, if one zooms in into the block of the transfer function above, FeenoX can also be seen as a glue layer between the input files defining a physical problem and the mathematical libraries used to solve the discretized equations. This way, FeenoX bounds its scope to do only one thing and to do it well: to build and solve finite-element formulations of physical problems. And it does so on high grounds, both ethical and technological:

Ethical

Since it is free software, all users can

1. run,
2. share,
3. modify, and/or
4. re-share their modifications.

If a user cannot read or write code to make FeenoX suit her needs, at least she has the freedom to hire someone to do it for her.

Technological

Since it is open source, advanced users can detect and correct bugs and even improve the algorithms. Given enough eyeballs, all bugs are shallow.

FeenoX’ main development architecture is Debian GNU/Linux running over 64-bits Intel-compatible processors. All the dependencies are free and/or open source and already available in Debian’s latest stable official repositories, as explained in sec. 2.1.

The POSIX standard is followed whenever possible, allowing thus FeenoX to be compiled in other operating systems and architectures such as Windows (using Cygwin) and MacOS. The build procedure is the well-known and mature ./configure && make command.

FeenoX is written in C conforming to the ISO C99 specification (plus POSIX extensions), which is a standard, mature and widely supported language with compilers for a wide variety of architectures. For its basic mathematical capabilities, FeenoX uses the GNU Scientifc Library. For solving ODEs/DAEs, FeenoX relies on Lawrence Livermore’s SUNDIALS library. For PDEs, FeenoX uses Argonne’s PETSc library and Universitat Politècnica de València’s SLEPc library. All of them are

• free and open source,
• written in C (neither Fortran nor C++),
• mature and stable,
• actively developed and updated,
• very well known both in the industry and academia.

Moreover, PETSc and SLEPc are scalable through the MPI standard (further discussed in sec. 2.5). This means that programs using both these libraries can run on either large high-performance supercomputers or low-end laptops. FeenoX has been run on

• Raspberry Pi
• Laptop (GNU/Linux & Windows 10)
• Macbook
• Desktop PC
• Bare-metal servers
• Vagrant/Virtualbox virtual machines
• Docker/Kubernetes containers
• AWS/DigitalOcean/Contabo instances

Due to the way that FeenoX is designed and the policy separated from the mechanism, it is possible to control a running instance remotely from a separate client which can eventually run on a mobile device (fig. 4).

The following example illustrates how well FeenoX works as one of many links in a chain that goes from tracing a bitmap with the problem’s geometry down to creating a nice figure with the results of a computation:

Say you are Homer Simpson and you want to solve a maze drawn in a restaurant’s placemat, one where both the start and end are known beforehand as show in fig. 5. In order to avoid falling into the alligator’s mouth, you can exploit the ellipticity of the Laplacian operator to solve any maze (even a hand-drawn one) without needing any fancy AI or ML algorithm. Just FeenoX and a bunch of standard open source tools to convert a bitmapped picture of the maze into an unstructured mesh.

1. Create a maze

3. Perform some conversions

• PNG \rightarrow PNM \rightarrow SVG \rightarrow DXF \rightarrow GEO
$wget http://www.mazegenerator.net/static/orthogonal_maze_with_20_by_20_cells.png$ convert orthogonal_maze_with_20_by_20_cells.png -negate maze.png
$potrace maze.pnm --alphamax 0 --opttolerance 0 -b svg -o maze.svg$ ./svg2dxf maze.svg maze.dxf
$./dxf2geo maze.dxf 0.1 4. Open it with Gmsh • Add a surface • Set physical curves for “start” and “end” 5. Mesh it (fig. 6 (a)) gmsh -2 maze.geo 6. Solve \nabla^2 \phi = 0 with BCs \begin{cases} \phi=0 & \text{at “start”} \\ \phi=1 & \text{at “end”} \\ \nabla \phi \cdot \hat{\vec{n}} = 0 & \text{everywhere else} \\ \end{cases} PROBLEM laplace 2D # pretty self-descriptive, isn't it? READ_MESH maze.msh # boundary conditions (default is homogeneous Neumann) BC start phi=0 BC end phi=1 SOLVE_PROBLEM # write the norm of gradient as a scalar field # and the gradient as a 2d vector into a .msh file WRITE_MESH maze-solved.msh \ sqrt(dphidx(x,y)^2+dphidy(x,y)^2) \ VECTOR dphidx dphidy 0  $ feenox maze.fee
$ 7. Open maze-solved.msh, go to start and follow the gradient \nabla \phi! ## 2.1 Deployment The tool should be easily deployed to production servers. Both 1. an automated method for compiling the sources from scratch aiming at obtaining optimized binaries for a particular host architecture should be provided using a well-established procedures, and 2. one (or more) generic binary version aiming at common server architectures should be provided. Either option should be available to be downloaded from suitable online sources, either by real people and/or automated deployment scripts. As already stated, FeenoX can be compiled from its sources using the well-established configure & make procedure. The code’s source tree is hosted on Github so cloning the repository is the preferred way to obtain FeenoX, but source tarballs are periodically released too according to the requirements in sec. 4.1. The configuration and compilation is based on GNU Autotools that has more than thirty years of maturity and it is the most portable way of compiling C code in a wide variety of UNIX variants. It has been tested with FeenoX depends on the four open source libraries stated in sec. 2, although the last three of them are optional. The only mandatory library is the GNU Scientific Library which is part of the GNU/Linux operating system and as such is readily available in all distributions as libgsl-dev. The sources of the rest of the optional libraries are also widely available in most common GNU/Linux distributions. In effect, doing sudo apt-get install gcc make libgsl-dev libsundials-dev petsc-dev slepc-dev is enough to provision all the dependencies needed compile FeenoX from the source tarball with the full set of features. If using the Git repository as a source, then Git itself and the GNU Autoconf and Automake packages are also needed: sudo apt-get install git autoconf automake Even though compiling FeenoX from sources is the recommended way to obtain the tool, since the target binary can be compiled using particularly suited compilation options, flags and optimizations (especially those related to MPI, linear algebra kernels and direct and/or iterative sparse solvers), there are also tarballs with usable binaries for some of the most common architectures—including some non-GNU/Linux variants. These binary distributions contain statically-linked executable files that do not need any other shared libraries to be installed on the target host. However, their flexibility and efficiency is generic and far from ideal. Yet the flexibility of having an execution-ready distribution package for users that do not know how to compile C source code outweighs the limited functionality and scalability of the tool. For example, first PETSc can be built with a -Ofast flag: $ cd $PETSC_DIR$ export PETSC_ARCH=linux-fast
$./configure --with-debug=0 COPTFLAGS="-Ofast"$ make -j8
$cd$HOME

And then not only can FeenoX be configured to use that particular PETSc build but also to use a different compiler such as Clang instead of GNU GCC and to use the same -Ofast flag to compile FeenoX itself:

$git clone https://github.com/seamplex/feenox$ cd feenox
$./autogen.sh$ export PETSC_ARCH=linux-fast
$./configure MPICH_CC=clang CFLAGS=-Ofast$ make -j8
# make install

$wget http://gmsh.info/bin/Linux/gmsh-Linux64.tgz$ wget https://seamplex.com/feenox/dist/linux/feenox-linux-amd64.tar.gz

Appendix has sec. 8 more details about how to download and compile FeenoX. The full documentation contains a compilation guide with further detailed explanations of each of the steps involved. Since all the commands needed to either download a binary executable or to compile from source with customized optimization flags can be automatized, FeenoX can be built into a container such as Docker. This way, deployment and scalability can be customized and tweaked as needed.

## 2.2 Execution

It is mandatory to be able to execute the tool remotely, either with a direct action from the user or from a high-level workflow which could be triggered by a human or by an automated script. The calling party should be able to monitor the status during run time and get the returned error level after finishing the execution.

The tool shall provide a mean to perform parametric computations by varying one or more problem parameters in a certain prescribed way such that it can be used as an inner solver for an outer-loop optimization tool. In this regard, it is desirable if the tool could compute scalar values such that the figure of merit being optimized (maximum temperature, total weight, total heat flux, minimum natural frequency, maximum displacement, maximum von Mises stress, etc.) is already available without needing further post-processing.

As FeenoX is designed to run as a file filter—or as a transfer function between input and output files from a more traditional engineering point of view—and it explicitly avoids having a graphical interface, the binary executable works as any other Unix terminal command. When invoked without arguments, it prints its version (a thorough explanation of the versioning scheme is given in sec. 4.1), a one-line description and the usage options:

$feenox FeenoX v0.2.37-g5b19c32 a free no-fee no-X uniX-like finite-element(ish) computational engineering tool usage: feenox [options] inputfile [replacement arguments] [petsc options] -h, --help display options and detailed explanations of commmand-line usage -v, --version display brief version information and exit -V, --versions display detailed version information --pdes list the types of PROBLEMs that FeenoX can solve, one per line Run with --help for further explanations. The program can also be executed remotely either 1. on a server through a SSH session 2. in a container as part of a provisioning script FeenoX provides mechanisms to inform its progress by writing certain information to devices or files, which in turn can be monitored remotely or even trigger server actions. Progress can be as simple as an ASCII bar (triggered with --progress in the command line or with the keyword PROGRESS in the input file) to more complex mechanisms like writing the status in a shared memory segment. Regarding its execution, there are three ways of solving problems: 1. direct execution 2. parametric runs, and 3. optimization loops. ### 2.2.1 Direct execution When directly executing FeenoX, one gives a single argument to the executable with the path to the main input file. For example, the following input computes the first twenty numbers of the Fibonacci sequence using the closed-form formula f(n) = \frac{\varphi^n - (1-\varphi)^n}{\sqrt{5}} where \varphi=(1+\sqrt{5})/2 is the Golden ratio: # the Fibonacci sequence using the closed-form formula as a function phi = (1+sqrt(5))/2 f(n) = (phi^n - (1-phi)^n)/sqrt(5) PRINT_FUNCTION f MIN 1 MAX 20 STEP 1 FeenoX can be directly executed to print the function f(n) for n=1,\dots,20 both to the standard output and to a file named one (because it is the first way of solving Fibonacci with Feenox): $ feenox fibo_formula.fee | tee one
1   1
2   1
3   2
4   3
5   5
6   8
7   13
8   21
9   34
10  55
11  89
12  144
13  233
14  377
15  610
16  987
17  1597
18  2584
19  4181
20  6765
$ Now, we could also have computed these twenty numbers by using the direct definition of the sequence into a vector \vec{f} of size 20. This time we redirect the output to a file named two: # the fibonacci sequence as a vector VECTOR f SIZE 20 f[i]<1:2> = 1 f[i]<3:vecsize(f)> = f[i-2] + f[i-1] PRINT_VECTOR i f $ feenox fibo_vector.fee > two
$ Finally, we print the sequence as an iterative problem and check that the three outputs are the same: # the fibonacci sequence as an iterative problem static_steps = 20 #static_iterations = 1476 # limit of doubles IF step_static=1|step_static=2 f_n = 1 f_nminus1 = 1 f_nminus2 = 1 ELSE f_n = f_nminus1 + f_nminus2 f_nminus2 = f_nminus1 f_nminus1 = f_n ENDIF PRINT step_static f_n $ feenox fibo_iterative.fee > three
$diff one two$ diff two three
$ These three calls were examples of direct execution of FeenoX: a single call with a single argument to solve a single fixed problem. ## 2.3 Parametric To use FeenoX in a parametric run, one has to successively call the executable passing the main input file path in the first argument followed by an arbitrary number of parameters. These extra parameters will be expanded as string literals $1, $2, etc. appearing in the input file. For example, if hello.fee is PRINT "Hello$1!"

then

$feenox hello.fee World Hello World!$ feenox hello.fee Universe
Hello Universe!
$ To have an actual parametric run, an external loop has to successively call FeenoX with the parametric arguments. For example, say this file cantilever.fee fixes the face called “left” and sets a load in the negative z direction of a mesh called cantilever-$1-$2.msh. The output is a single line containing the number of nodes of the mesh and the displacement in the vertical direction w(500,0,0) at the center of the cantilever’s free face: PROBLEM elastic 3D READ_MESH cantilever-$1-$2.msh # in meters E = 2.1e11 # Young modulus in Pascals nu = 0.3 # Poisson's ratio BC left fixed BC right tz=-1e5 # traction in Pascals, negative z SOLVE_PROBLEM # z-displacement (components are u,v,w) at the tip vs. number of nodes PRINT nodes w(500,0,0) "\#$1 $2" Now the following Bash script first calls Gmsh to create the meshes cantilever-${element}-${c}.msh where • ${element}: tet4, tet10, hex8, hex20, hex27
• ${c}: 1,2,,10 It then calls FeenoX with the input above and passes ${element} and ${c} as extra arguments, which then are expanded as $1 and $2 respectively. #!/bin/bash rm -f *.dat for element in tet4 tet10 hex8 hex20 hex27; do for c in$(seq 1 10); do

# create mesh if not alreay cached
mesh=cantilever-${element}-${c}
if [ ! -e ${mesh}.msh ]; then scale=$(echo "PRINT 1/${c}" | feenox -) gmsh -3 -v 0 cantilever-${element}.geo -clscale ${scale} -o${mesh}.msh
fi

# call FeenoX
feenox cantilever.fee ${element}${c} | tee -a cantilever-${element}.dat done done After the execution of the script, thanks to the design decision that output is 100% defined by the user (in this case with the PRINT instruction), one has several files cantilever-${element}.dat files. When plotted, these show the shear locking effect of fully-integrated first-order elements as illustrated in fig. 9. The theoretical Euler-Bernoulli result is just a reference as, among other things, it does not take into account the effect of the material’s Poisson’s ratio. Note that the abscissa shows the number of nodes, which are proportional to the number of degrees of freedom (i.e. the size of the problem matrix) and not the number of elements, which is irrelevant here and in most problems.

### 2.3.1 Optimization loops

Optimization loops work very much like parametric runs from the FeenoX point of view. The difference is mainly on the calling script that has to implement a certain optimization algorithm such as conjugate gradients, Nelder-Mead, simulated annealing, genetic algorithms, etc. to choose which parameters to pass to FeenoX as command-line argument. The only particularity on FeenoX’ side is that since the next argument that the optimization loop will pass might depend on the result of the current step, care has to be taken in order to be able to return back to the calling script whatever results it needs in order to compute the next arguments. This is usually just the scalar being optimized for, but it can also include other results such as derivatives or other relevant data.

To illustrate how to use FeenoX in an optimization loop, let us consider the problem of finding the length \ell_1 of a tuning fork (fig. 10) such that the fundamental frequency on a free-free oscillation is equal to the base A frequency at 440 Hz.

This extremely simple input file (rule of simplicity) solves the free-free mechanical modal problem (i.e. without any Dirichlet boundary condition) and prints the fundamental frequency:

PROBLEM modal 3D MODES 1  # only one mode needed
E = 2.07e11         # in [Pa]
nu = 0.33
rho = 7829          # in [kg/m^2]

# no BCs! It is a free-free vibration problem
SOLVE_PROBLEM

# write back the fundamental frequency to stdout
PRINT f(1)

Note that in this particular case, the FeenoX input files does not expand any command-line argument. The trick is that the mesh file fork.msh is overwritten in each call of the optimization loop. Since this time the loop is slightly more complex than in the parametric run of the last section, we now use Python. The function create_mesh() first creates a CAD model of the fork with geometrical parameters r, w, \ell_1 and \ell_2. It then meshes the CAD using n structured hexahedra through the fork’s thickness. Both the CAD and the mesh are created using the Gmsh Python API. The detailed steps between gmsh.initialize() and gmsh.finalize() are not shown here, just the fact that this function overwrites the previous mesh and always writes it into the file called fork.msh which is the one that fork.fee reads. Hence, there is no need to pass command-liner arguments to FeenoX. The full implementation of the function is available in the examples directory of the FeenoX distribution.

import math
import gmsh
import subprocess  # to call FeenoX and read back

def create_mesh(r, w, l1, l2, n):
gmsh.initialize()
...
gmsh.write("fork.msh")
gmsh.finalize()
return len(nodes)

def main():
target = 440    # target frequency
eps = 1e-2      # tolerance
r = 4.2e-3      # geometric parameters
w = 3e-3
l1 = 30e-3
l2 = 60e-3

for n in range(1,7):   # mesh refinement level
l1 = 60e-3              # restart l1 & error
error = 60
while abs(error) > eps:   # loop
l1 = l1 - 1e-4*error
# mesh with Gmsh Python API
nodes = create_mesh(r, w, l1, l2, n)
# call FeenoX and read scalar back
# TODO: FeenoX Python API (like Gmsh)
result = subprocess.run(['feenox', 'fork.fee'], stdout=subprocess.PIPE)
freq = float(result.stdout.decode('utf-8'))
error = target - freq

print(nodes, l1, freq)

Since the computed frequency depends both on the length \ell_1 and on the mesh refinement level n, there are actually two nested loops: one parametric over n=1,2\dots,7 and the optimization loop itself that tries to find \ell_1 so as to obtain a frequency equal to 440 Hz within 0.01% of error.

$python fork.py > fork.dat$

Note that the approach used here is to use Gmsh Python API to build the mesh and then fork the FeenoX executable to solve the fork (no pun intended). There are plans to provide a Python API for FeenoX so the problem can be set up, solved and the results read back directly from the script instead of needing to do a fork+exec, read back the standard output as a string and then convert it to a Python float.

Fig. 11 shows the results of the combination of the optimization loop over \ell_1 and a parametric run over n. The difference for n=6 and n=7 is in the order of one hundredth of millimeter.

## 2.4 Efficiency

It is mandatory to be able to execute the tool automatically in a remote server. The computational resources needed from this server, i.e. costs measured in

• CPU/GPU time
• random-access memory
• long-term storage
• etc.

needed to solve a problem should be comparable to other similar state-of-the-art cloud-based script-friendly finite-element tools.

One of the most widely known quotations in computer science is that one that says “premature optimization is the root of all evil.” that is an extremely over-simplified version of Donald E. Knuth’s analysis in his The Art of Computer Programming. Bottom line is that the programmer should not not spend too much time trying to optimize code based on hunches but based on profiling measurements. Yet a disciplined programmer can tell when an algorithm will be way too inefficient (say something that scales up like O(n^2)) and how small changes can improve performance (say by understanding how caching levels work in order to implement faster nested loops). It is also true that usually an improvement in one aspect leads to a deterioration in another one (e.g. a decrease in CPU time by caching intermediate results in an increase of RAM usage).

Even though FeenoX is still evolving so it could be premature in many cases, it is informative to compare running times and memory consumption when solving the same problem with different cloud-friendly FEA programs. In effect, a serial single-thread single-host comparison of resource usage when solving the NAFEMS LE10 problem introduced above was performed, using both unstructured tetrahedral and structured hexahedral meshes. Fig. 12 shows two figures of the many ones contained in the detailed report. In general, FeenoX using the iterative approach based on PETSc’s Geometric-Algebraic Multigrid Preconditioner and a conjugate gradients solver is faster for (relatively) large problems at the expense of a larger memory consumption. The curves that use MUMPS confirm the well-known theoretical result that direct linear solvers are robust but not scalable.

The large memory consumption shown by FeenoX is due to a high level of caching intermediate results. For instance, all the shape functions evaluated at the integration points are computed once when building the stiffness matrix, stored in RAM and then re-used when recovering the gradients of the displacements needed to compute the stresses. There are also a number of non-premature optimization tasks that can improve both the CPU and memory usage that ought to be performed at later stages of the project.

Regarding storage, FeenoX needs space to store the input file (negligible), the mesh file in .msh format (which can be either ASCII or binary) and the optional output files in .msh or .vtk formats. All of these files can be stored gzip-compressed and un-compressed on demand by exploiting FeenoX’ script-friendliness using proper calls to gzip before and/or after calling the feenox binary.

## 2.5 Scalability

The tool ought to be able to start solving small problems first to check the inputs and outputs behave as expected and then allow increasing the problem size up in order to achieve to the desired accuracy of the results. As mentioned in sec. 2, large problem should be split among different computers to be able to solve them using a finite amount of per-host computational power (RAM and CPU).

The time needed to solve a relatively large problem can be reduced by exploiting the fact that most cloud servers (and even laptop computers) have more than one CPU available. There are some tasks that can be split into several processors sharing a common memory address space that will scale up almost perfectly, such as building the elemental matrices and assembling the global stiffness matrix. There are some other tasks that might not scale perfectly but that nevertheless might (or might not) reduce the overall wall time if split among processors using a common memory space, such as solving the linear system K \cdot \vec{u} = \vec{b}. The usual scheme to parallelize a problem under these conditions is to use the OpenMP framework.

Yet, if the problem is large enough, a server might not have enough physical random-access memory to be able to handle the whole problem. The problem now has to be split among different servers which, in turn, might have several processors each. Some of the processors share the same address space but most of them will only have access to a fraction of the whole global problem data. In these cases, there are no tasks that can scale up perfectly since even when building and assembling the matrices, a processor needs some piece of data which is handled by another processor with a different address space and that has to be conveyed specifically from one process to another one. The usual scheme to parallelize a problem under these conditions is to use the MPI standard and one of its two most well-known implementations, either Open MPI or MPICH.

It might seem that the most effective approach to solve a large problem is to use OpenMP (not to be confused with OpenMPI!) among threads running in processors that share the memory address space and to use MPI among processes running in different hosts. But even though this hybrid OpenMP+MPI scheme is possible, there are at least three main drawbacks with respect to a pure MPI approach:

1. the overall performance is not be significantly better
2. the amount of lines of code that has to be maintained is more than doubled
3. the number of possible points of synchronization failure increases

In many ways, the pure MPI mode has fewer synchronizations and thus should perform better. Hence, FeenoX uses MPI (mainly through PETSc and SLEPc) to handle large parallel problems.

Most of the overhead of parallelized tasks come from the fact that processes need data stored in other processes (i.e. the so-called ghost points) that use a differet virtual memory address space. Therefore, the discretized domain has to be split among processes in such a way as to minimize the number of inter-process communication. This problem, called domain decomposition, can be handled either by the mesher or by the solver itself, usually using a third-part library such as Metis. It should be noted that the domain decomposition problem does not have a unique solution. On the one hand, it depends on the actual mesh being distributed over parallel processes as illustrated in fig. 13. On the other hand, the optimal solution might depend on the kind of topology boundaries to minimize (shared nodes, shared faces) and other subtle parameters and options that partitioning libraries allow.

FeenoX relies on Gmsh to perform the domain decomposition (using Metis at mesh-time) and to provide the partitioning information in the mesh file read by the READ_MESH keyword.

## 2.6 Flexibility

The tool should be able to handle engineering problems involving different materials with potential spatial and time-dependent properties, such as temperature-dependent thermal expansion coefficients and/or non-constant densities. Boundary conditions must be allowed to depend on both space and time as well, like non-uniform pressure loads and/or transient heat fluxes.

The third-system effect mentioned in sec. 2 involves more than ten years of experience in the nuclear industry,4 where complex dependencies of multiple material properties over space through intermediate distributions (temperature, neutronic poisons, etc.) and time (control rod positions, fuel burn-up, etc.) are mandatory. One of the cornerstone design decisions in FeenoX is that everything is an expression (sec. 3.1.5). Here, “everything” means any location in the input file where a numerical value is expected. The most common use case is in the PRINT keyword. For example, the Sophomore’s dream (in contrast to Freshman’s dream) identity

\int_{0}^{1} x^{-x} \, dx = \sum_{n=1}^{\infty} n^{-n}

can be illustrated like this:

VAR x
PRINT %.7f integral(x^(-x),x,0,1)
VAR n
PRINT %.7f sum(n^(-n),n,1,1000)
$feenox sophomore.fee 1.2912861 1.2912860$

Of course most engineering problems will not need explicit integrals—although a few of them do—but some might need summation loops, so it is handy to have these functionals available inside the FEA tool. This might seem to go against the “keep it simple” and “do one thing good” Unix principle, but definitely follows Alan Kay’s idea that “simple things should be simple, complex things should be possible” (further discussion in sec. 3.1.4).

Flexibility in defining non-trivial material properties is illustrated with the following example, where two squares made of different dimensionless materials are juxtaposed in thermal contact (glued?) and subject to different boundary conditions at each of the four sides (fig. 14).

The yellow square is made of a certain material with a conductivity that depends algebraically (and fictitiously) the temperature like

k_\text{yellow}(x,y) = \frac{1}{2} + T(x,y)

The cyan square has a space-dependent temperature given by a table of scattered data as a function of the spatial coordinates x and y (origin is left bottom corner of the yellow square) without any particular structure on the definition points:

x y k_\text{cyan}(x,y)
1 0 1.0
1 1 1.5
2 0 1.3
2 1 1.8
1.5 0.5 1.7

The cyan square generates a temperature-dependent power density (per unit area) given by

q^{\prime \prime}_\text{cyan}(x,y) = 0.2 \cdot T(x,y)

The yellow one does not generate any power so q^{\prime \prime}_\text{yellow} = 0. Boundary conditions are

\begin{cases} T(x,y) = y & \text{at the left edge $y=0$} \\ T(x,y) = 1-\cos\left(\frac{1}{2}\pi \cdot x\right) & \text{at the bottom edge $x=0$} \\ q'(x,y) = 2-y & \text{at the right edge $x=2$} \\ q'(x,y) = 1 & \text{at the top edge $y=1$} \\ \end{cases}

The input file illustrate how flexible FeenoX is and, again, how the problem definition in a format that the computer can understand resembles the humanly-written formulation of the original engineering problem:

PROBLEM thermal 2d            # heat conduction in two dimensions

k_yellow(x,y) = 1/2+T(x,y)    # thermal conductivity
FUNCTION k_cyan(x,y) INTERPOLATION shepard DATA {
1   0    1.0
1   1    1.5
2   0    1.3
2   1    1.8
1.5 0.5  1.7 }

q_cyan(x,y) = 1-0.2*T(x,y)    # dissipated power density
q_yellow(x,y) = 0

BC left   T=y                 # temperature (dirichlet) bc
BC bottom T=1-cos(pi/2*x)
BC right  q=2-y               # heat flux (neumann) bc
BC top    q=1

SOLVE_PROBLEM
WRITE_MESH two-squares-results.msh  T #CELLS k

Note that FeenoX is flexible enough to…

1. handle mixed meshes (the yellow square is meshed with triangles and the other one with quadrangles)
2. use point-wise defined properties even though there is not underlying structure nor topology for the points where the data is defined (FeenoX could have read data from a .msh or .vtk file respecting the underlying topology)
3. understand that the problem is non-linear so as to use PETSc’s SNES framework automatically (the conductivity and power source depend on the temperature).

In the very same sense that variables x, y and z appearing in the input refer to the spatial coordinates x, y and z respectively, the special variable t refers to the time t. The requirement of allowing time-dependent boundary conditions can be illustrated by solving the NAFEMS T3 one-dimensional transient heat transfer benchmark. It consists of a slab of 0.1 meters long subject to a fixed homogeneous temperature on one side, i.e.

T(x=0)=0~\text{°C}

and to a transient temperature

T(x=0.1~\text{m},t)=100~\text{°C} \cdot \sin\left( \frac{\pi \cdot t}{40~\text{s}}\right)

at the other side. There is zero internal heat generation, at t=0 all temperature is equal to 0°C (sic) and conductivity, specific heat and density are constant and uniform. The problem asks for the temperature at location x=0.08~\text{m} at time t=32~\text{s}. The reference result is T(0.08~\text{m},32~\text{s})=36.60~\text{°C}.

PROBLEM thermal DIM 1 # NAFEMS-T3 benchmark: 1d transient heat conduction

end_time = 32      # transient up to 32 seconds
T_0(x) = 0         # initial condition "all temperature is equal to 0°C"

# prescribed temperatures as boundary conditions
BC left  T=0
BC right T=100*sin(pi*t/40)

# uniform and constant properties
k = 35.0           # conductivity [W/(m K)]
cp = 440.5         # heat capacity [J/(kg K)]
rho = 7200         # density [kg/m^3]

SOLVE_PROBLEM

# print detailed evolution into an ASCII file
PRINT FILE nafems-t3.dat %.3f t dt %.2f T(0.05) T(0.08) T(0.1)

# print the asked result into the standard output
IF done
PRINT "T(0.08m,32s) = " T(0.08) "ºC"
ENDIF
$gmsh -1 slab-0.1m.geo [...] Info : Done meshing 1D (Wall 0.000213023s, CPU 0.000836s) Info : 61 nodes 62 elements Info : Writing 'slab-0.1m.msh'... Info : Done writing 'slab-0.1m.msh' Info : Stopped on Sun Dec 12 19:41:18 2021 (From start: Wall 0.00293443s, CPU 0.02605s)$ feenox nafems-t3.fee
T(0.08m,32s) =  36.5996 ºC
$pyxplot nafems-t3.ppl$

Besides “everything is an expression,” FeenoX follows another cornerstone rule: simple problems ought to have simple inputs, akin to Unix’ rule of simplicity—that addresses the first half of Alan Kay’s quote above. This rule is further discussed in sec. 3.1.

## 2.7 Extensibility

It should be possible to add other problem types casted as PDEs (such as the Schröedinger equation) to the tool using a reasonable amount of time by one or more skilled programmers. The tool should also allow new models (such as non-linear stress-strain constitutive relationships) to be added as well.

Even though FeenoX is written in C, it makes extensive use of function pointers to mimic C++’s virtual methods. This way, depending on the problem type given with the PROBLEM keyword, particular routines are called to

1. initialize and set up solver options (steady-state/transient, linear/non-linear, regular/eigenproblem, etc.)
2. parse boundary conditions given in the BC keyword
3. build elemental contributions for
1. volumetric stiffness and/or mass matrices
2. natural boundary conditions
4. compute secondary fields (heat fluxes, strains and stresses, etc.) out of the gradients of the primary fields
5. compute per-problem key performance indicators (min/max temperature, displacement, stress, etc.)
6. write particular post-processing outputs

Indeed, each of the supported problems, namely

is a separate directory under src/pdes that implements these “virtual” methods (recall that they are function pointers) that are resolved at runtime when parsing the main input file.

FeenoX was designed with separated common “mathematical” routines from the particular “physical” ones in such a way that any of these directories can be removed and the code would still compile. For example, let us remove the directory src/pdes/thermal from a temporary clone of the main Git repository. The autogen.sh script is smart enough to

1. tell which types of PDEs are available by reading the subdirectories in src/pdes, and
2. link the available virtual methods to the appropriate function pointers which are resolved at runtime.

In effect,

~$cd tmp/ ~/tmp$ git clone https://github.com/seamplex/feenox
Cloning into 'feenox'...
remote: Enumerating objects: 6908, done.
remote: Counting objects: 100% (4399/4399), done.
remote: Compressing objects: 100% (3208/3208), done.
remote: Total 6908 (delta 3085), reused 2403 (delta 1126), pack-reused 2509
Receiving objects: 100% (6908/6908), 10.94 MiB | 6.14 MiB/s, done.
Resolving deltas: 100% (4904/4904), done.
~/tmp$cd feenox ~/tmp/feenox$ rm -rf src/pdes/thermal/
~/tmp/feenox$./autogen.sh creating Makefile.am... ok creating src/Makefile.am... ok calling autoreconf... configure.ac:18: installing './compile' configure.ac:15: installing './config.guess' configure.ac:15: installing './config.sub' configure.ac:17: installing './install-sh' configure.ac:17: installing './missing' parallel-tests: installing './test-driver' src/Makefile.am: installing './depcomp' done ~/tmp/feenox$ ./configure.sh
[...]
configure: creating ./config.status
config.status: creating Makefile
config.status: creating src/Makefile
config.status: creating doc/Makefile
config.status: executing depfiles commands
~/tmp/feenox$make [...] make[1]: Leaving directory '/home/gtheler/tmp/feenox' ~/tmp/feenox$

Now if one wants to run the thermal problem with the two juxtaposed squares from sec. 2.6 above, the “temporary” FeenoX will complain. But it will still solve the NAFEMS LE10 problem problem right away:

~/tmp/feenox$cd doc/ ~/tmp/feenox/doc$ ../feenox two-squares.fee
error: two-squares.fee: 1: unknown problem type 'thermal'
~/tmp/feenox/doc$cd ../examples ~/tmp/feenox/examples$ ../feenox nafems-le10.fee
sigma_y @ D =   -5.38367        MPa
~/tmp/feenox/examples$ The list of available PDEs that a certain FeenoX binary has can be found by using the --pdes option. They are sorted alphabetically, one type per line: ~/tmp/feenox/examples$ feenox --pdes
laplace
mechanical
modal
neutron_diffusion
~/tmp/feenox/examples$ Besides removals, additions—which are also handled by autogen.sh as describe above— are far more interesting to discuss. Additional elliptic problems can be added by using the laplace directory as a template while using the other directories as examples about how to add further features (e.g. a Robin-type boundary condition in thermal and a vector-valued unknown in mechanical). More information can be found in the FeenoX programming & contributing section. As already discussed in sec. 1, FeenoX is free-as-in-freedom software licensed under the terms of the GNU General Public License version 3 or, at the user convenience, any later version. In the particular case of additions to the code base, this fact has two implications. 1. Every person in the world is free to modify FeenoX to suit their needs, including adding a new problem type either by 1. using one of the existing ones as a template, or 2. creating a new directory from scratch without asking anybody for any kind of permission. In case this person does not how to program, he or she has the freedom to hire somebody else to do it. It is this the sense of the word “free” in the compound phrase “free software:” freedom to do what they think fit (except to make it non-free, see next bullet). 2. People adding code own the copyright of the additional code. Yet, if they want to distribute the modified version they have to do it also under the terms of the GPLv3+ and under a name that does not induce the users to think the modified version is the original FeenoX distribution.5 That is to say, free software ought to remain free—a.k.a. as copyleft. Regarding additional material models, the virtual methods that compute the elemental contributions to the stiffness matrix also use function pointers to different material models (linear isotropic elastic, orthotropic elastic, etc.) and behaviors (isotropic thermal expansion, orthotropic thermal expansion, etc.) that are resolved at run time. Following the same principle, new models might be added by adding new routines and resolving them depending on the user’s input. ## 2.8 Interoperability A mean of exchanging data with other computational tools complying to requirements similar to the ones outlined in this document. This includes pre and post-processors but also other computational programs so that coupled calculations can be eventually performed by efficiently exchanging information between calculation codes. Sec. 1.2 already introduced the ideas about interoperability behind the Unix philosophy which make up for most the the FeenoX design basis. Essentially, they sum up to “do only one thing but do it well.” Since FeenoX is filter (or a transfer-function), interoperability is a must. So far, this SDS has already shown examples of exchanging information with: • Kate (with syntax highlighting): fig. 2 • Gmsh (both as a mesher and a post-processor): figs. 6-8, 10, 13-15 • Paraview: fig. 3 • Gnuplot: figs. 1, 12 • Pyxplot: figs. 9, 11, 16 To illustrate both the filter approach, consider the following input file that solves Laplace’s equation \nabla^2 \phi = 0 on a square with some space-dependent boundary conditions. Either Gmsh or Paraview can be used to post-process the results: \begin{cases} \phi(x,y) = +y & \text{for$x=-1$(left)} \\ \phi(x,y) = -y & \text{for$x=+1$(right)} \\ \nabla \phi \cdot \hat{\vec{n}} = \sin\left(\frac{\pi}{2} \cdot x\right) & \text{for$y=-1$(bottom)} \\ \nabla \phi \cdot \hat{\vec{n}} =0 & \text{for$y=+1$(top)} \\ \end{cases} PROBLEM laplace 2d READ_MESH square-centered.msh # [-1:+1]x[-1:+1] # boundary conditions BC left phi=+y BC right phi=-y BC bottom dphidn=sin(pi/2*x) BC top dphidn=0 SOLVE_PROBLEM # same output in .msh and in .vtk formats WRITE_MESH laplace-square.msh phi VECTOR dphidx dphidy 0 WRITE_MESH laplace-square.vtk phi VECTOR dphidx dphidy 0 A great deal of FeenoX interoperability capabilities comes from another design decision: output is 100% controlled by the user (further discussed in sec. 3.2), a.k.a. “no PRINT, no OUTPUT” whose corollary is the UNIX rule of silence. The following input file computes the natural frequencies of oscillation of a cantilevered wire both using the Euler-Bernoulli theory and finite elements. It writes a GFM table into the standard output which is then piped to Pandoc and then converted to HTML: # compute the first five natural modes of a cantilever wire # see https://www.seamplex.com/fino/doc/alambre.pdf (in Spanish) # (note that there is a systematic error of a factor of two in the measured values) # see https://www.seamplex.com/feenox/examples for a slighly more complex example # wire geometry l = 0.5*303e-3 # [ m ] cantilever length d = 1.948e-3 # [ m ] diameter # material properties for copper mass = 0.5*8.02e-3 # [ kg ] total mass (half the measured because of the experimental disposition) volume = pi*(0.5*d)^2*l rho = mass/volume # [ kg / m^3 ] density = mass (measured) / volume E = 2*66.2e9 # [ Pa ] Young modulus (twice because the factor-two error) nu = 0 # Poisson’s ratio (does not appear in Euler-Bernoulli) # compute analytical solution # first compute the first five roots ok cosh(kl)*cos(kl)+1 VECTOR kl[5] kl[i] = root(cosh(t)*cos(t)+1, t, 3*i-2,3*i+1) # then compute the frequencies according to Euler-Bernoulli # note that we need to use SI inside the square root A = pi * (d/2)^2 I = pi/4 * (d/2)^4 VECTOR f_euler[5] f_euler[i] = 1/(2*pi) * kl(i)^2 * sqrt((E * I)/(rho * A * l^4)) # now compute the modes numerically with FEM # note that each mode is duplicated as it is degenerated PROBLEM modal 3D MODES 10 READ_MESH wire-hex.msh BC fixed fixed SOLVE_PROBLEM # write a github-formatted markdown table comparing the frequencies PRINT " \$n\$| FEM | Euler | Relative difference [%]" PRINT ":----:+:------:+:-----:+:-----------------------:" PRINT_VECTOR SEP " | " %g i %.4g f(2*i-1) f_euler %.2f 100*(f_euler(i)-f(2*i-1))/f_euler(i) PRINT PRINT ": Comparison of analytical and numerical frequencies, in Hz" $ gmsh -3 wire-hex.geo
[...]
 feenox wire.fee | pandoc
<table>
<caption>Comparison of analytical and numerical frequencies, in Hz</caption>
<th style="text-align: center;"><span class="math inline"><em>n</em></span></th>
<th style="text-align: center;">FEM</th>
<th style="text-align: center;">Euler</th>
<th style="text-align: center;">Relative difference [%]</th>
</tr>
<tbody>
<tr class="odd">
<td style="text-align: center;">1</td>
<td style="text-align: center;">45.84</td>
<td style="text-align: center;">45.84</td>
<td style="text-align: center;">0.02</td>
</tr>
<tr class="even">
<td style="text-align: center;">2</td>
<td style="text-align: center;">287.1</td>
<td style="text-align: center;">287.3</td>
<td style="text-align: center;">0.06</td>
</tr>
<tr class="odd">
<td style="text-align: center;">3</td>
<td style="text-align: center;">803.4</td>
<td style="text-align: center;">804.5</td>
<td style="text-align: center;">0.13</td>
</tr>
<tr class="even">
<td style="text-align: center;">4</td>
<td style="text-align: center;">1573</td>
<td style="text-align: center;">1576</td>
<td style="text-align: center;">0.24</td>
</tr>
<tr class="odd">
<td style="text-align: center;">5</td>
<td style="text-align: center;">2596</td>
<td style="text-align: center;">2606</td>
<td style="text-align: center;">0.38</td>
</tr>
</tbody>
</table>
$ Of course these kind of FeenoX-generated tables can be inserted verbatim into Markdown documents (just like this one) and rendered as tbl. 1. Table 1: Comparison of analytical and numerical frequencies, in Hz n FEM Euler Relative difference [%] 1 45.84 45.84 0.02 2 287.1 287.3 0.06 3 803.4 804.5 0.13 4 1573 1576 0.24 5 2596 2606 0.38 It should be noted that all of the programs and tools mentioned to be interoperable with FeenoX are free and open source software. This is not a requirement from the SRS, but is indeed a nice-to-have feature. # 3 Interfaces The tool should be able to allow remote execution without any user intervention after the tool is launched. To achieve this goal it is required that the problem should be completely defined in one or more input files and the output should be complete and useful after the tool finishes its execution, as already required. The tool should be able to report the status of the execution (i.e. progress, errors, etc.) and to make this information available to the user or process that launched the execution, possibly from a remote location. FeenoX is provided as a console-only executable which can be run remotely through SSH or inside a containerized environment without any requirement such as graphical servers or special input devices. When executed without any arguments, FeenoX writes a brief message with the version (further discussed in sec. 4.1) and the basic usage on the standard output and return to the calling shell with a return errorlevel zero: $ feenox
FeenoX v0.2.12-gc5934bb-dirty
a free no-fee no-X uniX-like finite-element(ish) computational engineering tool

usage: feenox [options] inputfile [replacement arguments]

-h, --help         display options and detailed explanations of commmand-line usage
-v, --version      display brief version information and exit
-V, --versions     display detailed version information

Run with --help for further explanations.
$echo$?
0
$ The --version option follows the GNU Coding Standards guidelines: $ feenox --version
FeenoX v0.2.14-gbbf48c9-dirty
a free no-fee no-X uniX-like finite-element(ish) computational engineering tool

FeenoX is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ The --versions option shows more information about the FeenoX build and the libraries the binary was linked against: $ feenox -V
FeenoX v0.2.14-gbbf48c9-dirty
a free no-fee no-X uniX-like finite-element(ish) computational engineering tool

Last commit date   : Sat Feb 12 15:35:05 2022 -0300
Build date         : Sat Feb 12 15:35:44 2022 -0300
Build architecture : linux-gnu x86_64
Compiler version   : gcc (Debian 10.2.1-6) 10.2.1 20210110
Compiler expansion : gcc -Wl,-z,relro -I/usr/include/x86_64-linux-gnu/mpich -L/usr/lib/x86_64-linux-gnu -lmpich
Compiler flags     : -O3
Builder            : gtheler@tom
GSL version        : 2.6
SUNDIALS version   : 5.7.0
PETSc version      : Petsc Release Version 3.16.3, Jan 05, 2022
PETSc arch         : arch-linux-c-debug
SLEPc version      : SLEPc Release Version 3.16.1, Nov 17, 2021
$ The --help option gives a more detailed usage: $ feenox --help
usage: feenox [options] inputfile [replacement arguments] [petsc options]

-h, --help         display options and detailed explanations of commmand-line usage
-v, --version      display brief version information and exit
-V, --versions     display detailed version information

--progress         print ASCII progress bars when solving PDEs
--mumps            ask PETSc to use the direct linear solver MUMPS
--linear           force FeenoX to solve the PDE problem as linear
--non-linear       force FeenoX to solve the PDE problem as non-linear

Instructions will be read from standard input if “-” is passed as
inputfile, i.e.

$echo 'PRINT 2+2' | feenox - 4 The optional [replacement arguments] part of the command line mean that each argument after the input file that does not start with an hyphen will be expanded verbatim in the input file in each occurrence of$1,
$2, etc. For example$ echo 'PRINT $1+$2' | feenox - 3 4
7

PETSc and SLEPc options can be passed in [petsc options] as well, with
the difference that two hyphens have to be used instead of only once.
For example, to pass the PETSc option -ksp_view the actual FeenoX
invocation should be

$feenox input.fee --ksp_view See https://www.seamplex.com/feenox/examples for annotated examples. Report bugs at https://github.com/seamplex/feenox/issues Ask questions at https://github.com/seamplex/feenox/discussions Feenox home page: https://www.seamplex.com/feenox/$

The input file provided as the first argument to the feenox binary contains all the information needed to solve the problem, so any further human intervention is not needed after execution begins. If the execution finishes successfully, FeenoX returns a zero errorlevel to the calling shell (and follows the UNIX rule of silence):

$feenox maze.fee$ echo $? 0$

If there is problem during execution (including parsing and run-time errors), a line prefixed with error: is written into the standard error file descriptor and a non-zero errorlevel is returned:

$feenox hello.fee error: input file needs at least one more argument in commandline$ echo $? 1$ feenox hello.fee world
Hello world!
$echo$?
0
$ This way, the error line can easily be parsed with standard UNIX tools like grep and cut or with a proper regular expression parser. Eventually, any error should be forwarded back to the initiating entity—which depending on the workflow can be a human or an automation script—in order for he/she/it to fix it. Following the rule of repair, ill input files with missing material properties or inconsistent boundary conditions are detected before the actual assembly of the matrix begins: $ feenox thermal-1d-dirichlet-no-k.fee
error: undefined thermal conductivity 'k'
$feenox thermal-1d-dirichlet-wrong-bc.fee error: boundary condition 'xxx' does not have a physical group in mesh file 'slab.msh'$ 

Error code are designed to be useful and helpful. An attempt to open a file might fail due to a wide variety of reasons. FeenoX clearly states which one caused the error so it can be remedied:

$cat test.fee READ_MESH cantilever.msh$ feenox test.fee
$chmod -r cantilever.msh$ feenox test.fee
error: 'Permission denied' when opening file 'cantilever.msh' with mode 'r'
$rm cantilever.msh$ feenox test.fee
error: 'No such file or directory' when opening file 'cantilever.msh' with mode 'r'
$ If the command-line option --progress (or the PROGRESS keyword in PROBLEM) is used, then FeenoX writes into the standard output three “bars” showing the progress of 1. (.) the build and assembly of the problem matrices (stiffness and mass if applicable) 2. (-) the iterative solution of the problem (either linear or non-linear) 3. (=) the recovery of gradient-based (i.e. strains and stresses) out of the primary solution Once again, these ASCII-based progress bars can be parsed by the calling entity and then present it back to the user. For example, fig. 19 shows how the web-based GUI CAEplex shows progress inside an Onshape tab. Since FeenoX uses PETSc (and SLEPc), command-line options can be passed from FeenoX to PETSc. The only difference is that since FeenoX follows the POSIX standard regarding options and PETSc does not, double dashes are required instead of PETSc’ single-dash approach. That is to say, instead of -ksp_monitor one would have to pass --ksp_monitor (see sec. 3.1.3 for details about the input files): $ feenox thermal-1d-dirichlet-uniform-k.fee --ksp_monitor
0 KSP Residual norm 1.913149816332e+00
1 KSP Residual norm 2.897817223901e-02
2 KSP Residual norm 3.059845525572e-03
3 KSP Residual norm 1.943995979588e-04
4 KSP Residual norm 7.418444674938e-06
5 KSP Residual norm 1.233527903582e-07
0.5
$ Any PETSc command-line option takes precedence over the settings in the input file, so the preconditioner can be changed even if explicitly given with the PRECONDITIONER keyword: $ feenox thermal-1d-dirichlet-uniform-k.fee --ksp_monitor --pc_type ilu
0 KSP Residual norm 1.962141687033e+00
1 KSP Residual norm 5.362273771017e-16
0.5
$ If PETSc is compiled with MUMPS, FeenoX provides a --mumps option: $ feenox thermal-1d-dirichlet-uniform-k.fee --ksp_monitor --mumps
0 KSP Residual norm 1.004987562109e+01
1 KSP Residual norm 4.699798436762e-15
0.5
$ An illustration of the usage of PETSc arguments and the fact that FeenoX automatically detects whether a problem is linear or not is given below. The case thermal-1d-dirichlet-uniform-k.fee is linear while the two-squares.fee from section sec. 2.6 is not. Therefore, an SNES monitor should give output for the latter but not for the former. In effect: $ feenox thermal-1d-dirichlet-uniform-k.fee --snes_monitor
0.5
$feenox two-squares.fee --snes_monitor 0 SNES Function norm 9.658033489479e+00 1 SNES Function norm 1.616559951959e+00 2 SNES Function norm 1.879821597500e-01 3 SNES Function norm 2.972104258103e-02 4 SNES Function norm 2.624028350822e-03 5 SNES Function norm 1.823396478825e-04 6 SNES Function norm 2.574514225532e-05 7 SNES Function norm 2.511975376809e-06 8 SNES Function norm 4.230090605033e-07 9 SNES Function norm 5.154440365087e-08$

As already explained in sec. 2.3, FeenoX supports run-time replacement arguments that get replaced verbatim in the input file. This feature is handy when the same problem has to be solved over different meshes, such as when investigating the h-convergence order over Gmsh’s element scale factor -clscale:

PROBLEM thermal 1D
READ_MESH slab-$1.msh k(x) = 1+T(x) BC left T=0 BC right T=1 SOLVE_PROBLEM PRINT nodes %+.2e integral((T(x)-(sqrt(1+(3*x))-1))^2,x,0,1) $ for c in $(feenox steps.fee); do gmsh -v 0 -1 slab.geo -clscale${c} -o slab-${c}.msh; feenox thermal-1d-dirichlet-temperature-k-parametric.fee${c}; done  | sort -g
11      +6.50e-07
13      +3.15e-07
14      +2.29e-07
15      +1.70e-07
17      +1.00e-07
20      +5.04e-08
24      +2.34e-08
32      +7.19e-09
39      +3.46e-09
49      +1.31e-09
$ Since the main input file is the first argument (not counting POSIX options starting with at least one dash), FeenoX might be invoked indirectly by adding a shebang line to the input file with the location of the system-wide executable and setting execution permissions on the input file itself. So if we modify the above hello.fee example as hello #!/usr/local/bin/feenox PRINT "Hello$1!"

and then we can do

$chmod +x hello$ ./hello world
Hello world!
$./hello universe Hello universe!$

For example, the following she-banged input file can be used to compute the derivative of a column with respect to the other as a UNIX filter:

#!/usr/local/bin/feenox
FUNCTION f(t) FILE - INTERPOLATION steffen

a = vecmin(vec_f_t)
b = vecmax(vec_f_t)

# time step from arguments (or default 10 steps)
DEFAULT_ARGUMENT_VALUE 1 (b-a)/10
h = $1 VAR t' f'(t) = derivative(f(t'),t',t) PRINT_FUNCTION f' MIN a+0.5*h MAX b-0.5*h STEP h $ feenox f.fee "sin(t)" 1 | ./derivative.fee
0.05    0.998725
0.15    0.989041
0.25    0.968288
0.35    0.939643
0.45    0.900427
0.55    0.852504
0.65    0.796311
0.75    0.731216
0.85    0.66018
0.95    0.574296
$ ## 3.1 Problem input The problem should be completely defined by one or more input files. These input files might be 1. particularly formatted files to be read by the tool in an ad-hoc way, and/or 2. source files for interpreted languages which can call the tool through and API or equivalent method, and/or 3. any other method that can fulfill the requirements described so far. Preferably, these input files should be plain ASCII files in order to allow to manage changes using distributed version control systems such as Git. If the tool provides an API for an interpreted language such as Python, then the Python source used to solve a particular problem should be Git-friendly. It is recommended not to track revisions of mesh data files but of the source input files, i.e. to track the mesher’s input and not the mesher’s output. Therefore, it is recommended not to mix the problem definition with the problem mesh data. It is not mandatory to include a GUI in the main distribution, but the input/output scheme should be such that graphical pre and post-processing tools can create the input files and read the output files so as to allow third parties to develop interfaces. It is recommended to design the workflow as to make it possible for the interfaces to be accessible from mobile devices and web browsers. It is expected that 80% of the problems need 20% of the functionality. It is acceptable if only basic usage can be achieved through the usage of graphical interfaces to ease basic usage at first. Complex problems involving non-trivial material properties and boundary conditions not be treated by a GUI and only available by needing access to the input files. FeenoX currently works by reading an input file (which in turn can recursively include further input files) with an ad-hoc format, whose rationale is described in this section. Therefore, it does satisfy requirement a. but eventually could also satisfy requirement b. As already explained in sec. 1, the motto is “FeenoX is—in a certain sense—to desktop FEA programs and libraries what Markdown is to Word and (La)TeX, respectively and deliberately.” Hence, the input files act as the Markdown source: instructions about what to do but not how to do it. The input files are indeed plain-text ASCII files with English-like keywords that fully define the problem. The main features of the input format, thoroughly described below, are: 1. It is syntactically sugared by using English-like keywords. 2. Nouns are definitions and verbs are instructions. 3. Simple problems need simple inputs. 4. Simple things should be simple, complex things should be possible. 5. Whenever a numerical value is needed an expression can be given (i.e. “everything is an expression.”) 6. The input file should match as much as possible the paper (or blackboard) formulation of the problem. 7. It provides means to compare numerical solutions against analytical ones. 8. It should be possible to read run-time arguments from the command line. 9. Input files are distributed version control-friendly. ### 3.1.1 Syntactic sugar & highlighting The first argument not starting with a dash to the feenox executable is the path to the main input file. This main input file can in turn include other FeenoX input files and/or read data from other files (such as meshes) or resources (such as shared memory objects). The input files are plain text files, either pure ASCII or UTF-8 (more details in sec. 3.1.9). The extension can be anything, but a particular one is recommended so that per-extension syntax highlighting can be enabled in text editors (both graphical such as Kate and cloud-friendly such as Vim as illustrated in fig. 20) and in documentation (e.g. both HTML and PDF using Pandoc/LaTeX as in the FeenoX website). Throughout the FeenoX repository and documentation the extension .fee is used. But again, any extension (even no extension) can be used. Show that including .geo works (spinning disk) .fee is removed from $0

The ultimate goal of FeenoX is to solve mathematical equations that are hard to solve with pencil and paper. In particular, to integrate differential equations (recall that the first usable computer was named ENIAC, which stands for Electronic Numerical Integrator and Computer). The input file format was designed as to how to ask the computer what to compute. The syntax, based on keywords and alphanumerical arguments was chosen as to sit in the middle of the purely binary numerical system employed by digital computers6 and the purely linguistical nature of human communication. The rationale behind its design is that an average user can peek a FeenoX input file and tell what it is asking the computer to compute, as already illustrated for the NAFEMS LE10 problem in fig. 2. Even if the input files are created by a computer and not by a human, the code used to create a human-friendly input file will be human-friendlier than a code that writes only zeroes and ones as its output (that will become the input of another one following the Unix rule of composition). As an exercise, compare the input file in fig. 2 (or in fig. 20) with the inputs files used by other open source FEA solvers shown in appendix sec. 9.

### 3.1.2 Definitions and instructions

The way to tell the computer what problem it has to solve and how to solve it is by using keywords in the input file. Each non-commented line of the input file should start with either

1. a primary keyword such as PROBLEM or READ_MESH, or
2. a variable such as end_time or a vector or matrix with the corresponding index(es) such as v[2] or A[i][j] followed by the = keyword, or
3. a function name with its arguments such as f(x,y) followed by the = keyword.

A primary keyword usually is followed by arguments and/or secondary keywords, which in turn can take arguments as well. For example, in

PROBLEM mechanical DIMENSIONS 3
[...]
# print the direct stress y at D (and nothing more)
PRINT "σ_y @ D = " sigmay(2000,0,300) "MPa"

we have PROBLEM acting as a primary keyword, taking mechanical as its first argument and then DIMENSIONS as a secondary keyword with 3 being an argument to the secondary keyword. Then READ_MESH is another primary keyword taking nafems-le10.msh as its argument.

A primary keyword can be

1. a definition,
2. an instruction, or
3. both.

Definitions are English nouns and instructions are English verbs. In the example above, PROBLEM is a definition because it tells FeenoX about something it has to do (i.e. that it has to solve a three-dimensional problem), but does not do anything actually. On the other hand, READ_MESH is both a definition and an instruction: it defines that there exists a mesh named nafems-le10.msh which might be referenced later (for example in an INTEGRATE or WRITE_MESH instructions), but it also asks FeenoX to read the mesh at that point of the instruction list (more details below). Finally, PRINT is a primary keyword taking different types and number or arguments. It is an instruction because it does not define anything, it just asks FeenoX to print the value of the function named sigmay evaluated at 2000,0,300. In this case, sigmay is a function which is implicitly defined when PROBLEM is set to mechanical. If sigmay was referenced before PROBLEM, FeenoX would not find it. And if the problem was of any other type, FeenoX would not find it even when referenced from the last line of the input file.

The following example further illustrates the differences between definitions and instructions. It compares the result of (numerically but adaptively) integrating f(x,y,z) = \sin(x^3 + y^2 + z) over a unit cube against using a 3D Gauss integration scheme over a fixed set of quadrature points on the same unit cube meshes with two regular hexahedra in each direction (totalling 8 hexahedra). In one case hex20 are used and in the other one, hex27. Both cases use 27 quadrature points per element.

# these two are instructions to read a two meshes
# but they also define two mesh names that can be referred to later

# these three lines are definitions, they define three functions
# the first two also define four vectors for each function
#  1. vec_f20_x and vec_f27_x with the x coordinates of the mesh' nodes
#  2. vec_f20_y and vec_f27_y with the y coordinates of the mesh' nodes
#  3. vec_f20_z and vec_f27_z with the z coordinates of the mesh' nodes
#  4. vec_f20 and vec_f27 with the value of the function at the i-th node
# these definitions do not evaluate the functions, but they fill vectors 1-3
# (we'll fill vectors 4 below)
# note that these definitions refer to the meshes defined above in READ_MESH
FUNCTION f20(x,y,z) MESH hex20.msh
FUNCTION f27(x,y,z) MESH hex27.msh
f(x,y,z) = sin(x^3 + y^2 + z)

# these two lines are assignment instructions, they "fill" in
# the vectors with the value of the functinos f20(x,y,z) and f27(x,y,z)
# by evaluating f(x,y,z) at the nodes of the two meshes
# (there is a implicit loop for the index i over the size of the vectors)
vec_f20[i] = f(vec_f20_x[i], vec_f20_y[i], vec_f20_z[i])
vec_f27[i] = f(vec_f27_x[i], vec_f27_y[i], vec_f27_z[i])

# this line is an assignment, that first defines a variale If0
# and then calls the functional integral three times to perform a
# "continuous" (in the sense that it is numeric but adapative) triple integration
If0 = integral(integral(integral(f(x,y,z), z, 0, 1), y, 0, 1), x, 0, 1)

# these two lines are instructions, they integrate functions f20 and f27 over
# each of the meshes and then they store the results in the (implicitly defined)
# variables If20 and If27
INTEGRATE f20  MESH hex20.msh RESULT If20
INTEGRATE f27  MESH hex27.msh RESULT If27

# these lines are instructions, they print stuff to the standard output
# nothing is defined
PRINT %.10f If0
PRINT %.10f If20  %+.2e If20-If0
PRINT %.10f If27  %+.2e If27-If0
 feenox  integral_over_hex.fee
0.7752945175
0.7753714586    +7.69e-05
0.7739155101    -1.38e-03
$ ### 3.1.3 Simple inputs Consider solving heat conduction on a one-dimensional slab spanning the unitary range x \in [0,1]. The slab has a uniform unitary conductivity k=1 and Dirichlet boundary conditions \begin{cases} T(0) &= 0 \\ T(1) &= 1 \end{cases} This simple problem has the following simple input: PROBLEM thermal 1D # tell FeenoX what we want to solve READ_MESH slab.msh # read mesh in Gmsh's v4.1 format k = 1 # set uniform conductivity BC left T=0 # set fixed temperatures as BCs BC right T=1 # "left" and "right" are defined in the mesh SOLVE_PROBLEM # tell FeenoX we are ready to solve the problem PRINT T(0.5) # ask for the temperature at x=0.5 $ feenox thermal-1d-dirichlet-uniform-k.fee
0.5
$ Now, if instead of having a uniform conductivity the problem had a space-dependent k(x) = 1+x then the input would read PROBLEM thermal 1D READ_MESH slab.msh k(x) = 1+x # space-dependent conductivity BC left T=0 BC right T=1 SOLVE_PROBLEM PRINT T(1/2) log(1+1/2)/log(2) # print numerical and analytical solutions $ feenox thermal-1d-dirichlet-space-k.fee
0.584893    0.584963
$ Finally, if the conductivity depended on temperature (rendering the problem non-linear) say like k(x) = 1 + T(x) then PROBLEM thermal 1D READ_MESH slab.msh k(x) = 1+T(x) # temperature-dependent conductivity BC left T=0 BC right T=1 SOLVE_PROBLEM PRINT T(1/2) sqrt(1+(3*0.5))-1 # print numerical and analytical solutions $ feenox thermal-1d-dirichlet-space-k.fee
0.581139    0.581139
$ Note that FeenoX could figure out by itself that the two first cases were linear while the last one was not. This can be verified by passing the extra argument --snes_view. In the first two cases, there will be no extra output. In the last one, the details of the non-linear solver used by PETSc will be written into the standard output. The experienced reader should take some time to compare the effort and level of complexity that other FEA solvers require in order to set up simple problems like these. ### 3.1.4 Complex things Alan Kay’s idea that “simple things should be simple, complex things should be possible” has already been mentioned. The first part of the quote was addressed in the previous section. Of course, complexity can scale up almost indefinitely so we cannot show an example right here. But the following repositories contain the scripts and input files (for Gmsh, FeenoX and other common Unix tools such as Sed and Awk) that solve non-trivial problems using FeenoX as the main tool and employing free and open source software only, both for the computation and for the creation of figures and result tables. Convergence study on stress linearization of an infinite pipe according to ASME A parametric study over the number of elements through the thickness of a pipe and the error committed when computing membrane and bending stresses according to ASME VIII Div 2 Sec 5. The study uses the following element types • unstructured tet4 • unstructured straight tet10 • unstructured curved tet10 • structured straight tet10 • structured curved tet10 • structured hex8 • structured straight hex20 • structured curved hex20 • structured straight hex27 • structured curved hex27 The linearized stresses for different number of elements through the pipe thickness are compared against the analytical solution. Figures with the meshes employed in each case and with plots of profiles vs. the radial coordinate and linearized stresses vs. number of elements through the thickness are created. Environmentally-assisted fatigue analysis of dissimilar material interfaces in piping systems of a nuclear power plant A case that studies environmentally-assisted fatigue using environment factors applied to traditional in-air ASME fatigue analysis for operational an incidental transients in nuclear power plant as propose by EPRI. A fictitious CAD geometry representing a section of a piping system is studied. Four operational transients are made up with time-dependent data for pressure and water temperature. 1. A transient heat conduction problem with temperature-dependent material properties (according to ASME property tables) are solved over a small region around a material interface between carbon and stainless steel. 2. Primary stresses according to ASME are computed for each of the operational transients. 3. The results of a modal analysis study are convoluted with a frequency spectrum of a design-basis earthquake using the SRSS method to obtain an equivalent static volumetric force distribution. 4. The time-dependent temperature distribution for each transient is then used in quasi-static mechanical problems to compute secondary stresses according to ASME, including the equivalent seismic loads at the moment of higher thermal stresses. 5. The history of linearized Tresca stresses are juxtaposed to compute the cumulative usage factors using the ASME peak-valley method. 6. Environmental data is used to affect each cumulative usage factors with an environment factor to account for in-water conditions. These repositories contain a run.sh that, when executed in a properly-set-up GNU/Linux host (either on premises or in the cloud), will perform a number of steps including • creation of appropriate meshes • execution FeenoX • generation post-processing views, plots or tables with the results • etc. Refer to the READMEs in each repository for further details about the mathematical models involved. ### 3.1.5 Everything is an expression As explained in the history of FeenoX (sec. 7), the first objective of the code was to solve ODEs written in an ASCII file as human-friendly as possible. So even before any other feature, the first thing I coded in the FeenoX ancestors was an algebraic parser and evaluator. This was back in 2009, and I performed an online search before writing the whole thing from scratch. I found a nice post in Slack Overflow7 that discussed some cool ideas and even had some code published under the terms of the Creative Commons license. Besides ODEs, algebraic expressions are a must if one will be dealing with arbitrary functions in general and spatial distributions in particular—which is essentially what PDE solvers are for. If a piece of software wants to allow features ranging from comparing numerical results with analytical expression to converting material properties from a system of units to another one in a straightforward way for the user (either an actual human being or a computer creating an input file), it ought to be able to handle algebraic expressions. Appropriately handling algebraic expressions leads to fulfilling the Unix rule of least surprise. If the user needs to define a function f(x) = 1/2 \cdot x^2, all she has to do is write f(x) = 1/2 * x^2 And conversely, if someone reads the line above, she can rest assure that there’s a function called f(x) that will evaluate to 1/2 \cdot x^2 when needed. In effect, anyone can expect the output of this instruction PRINT integral(f(x), x, 0, 1) to be one third. Of course expressions are needed to get 100%-user defined output (further discussed in sec. 3.2), as with the PRINT instruction above. But once an algebraic parser and evaluator is available, it does not make sense to keep force the user to write numerical data only. What if a the angular speed is in RPM and the model needs it in radians per second? Instead of having to write 104.72, FeenoX allows the user to write w = 1000 * 60*pi/180 This way, 1. it is easy to see what the speed in RPM is 2. precision is not lost 3. should the speed change, it is trivial to change the 1000 for anything else. Whenever an input keyword needs a numerical value, any expression will do: n = 3 VECTOR x SIZE 2+n x[i] = i^2 PRINT x $ feenox vector_size_one_plus_n.fee
1       4       9       16      25
$ It goes without saying that algebraic expressions are a must when defining transient and/or space-dependent boundary conditions for PDEs: PROBLEM thermal 1D READ_MESH slab.msh end_time = 10 k = 1 kappa = 0.1 FUNCTION f(t) DATA { 0 0 1 1 2 1 3 2 4 0 10 0 } BC left T=f(t) w = 0.5*pi BC right T=1+sin(w*t) SOLVE_PROBLEM PRINT t T(0) T(0.5) T(1) Besides purely algebraic expressions, FeenoX can handle point-wise defined functions which can then be used in algebraic expressions. A useful example is allowing material properties (e.g. Young modulus) to depend on temperature. Consider a simple plane-strain square [-1,+1]\times[-1,+1] fixed on one side and with a uniform tension in the opposite one while leaving the other two free. The square’s Young modulus depends on temperature according to a one-dimensional point-wise defined function E_\text{carbon}(T) given by pairs stated according to one of the many material properties tables from ASME II and interpolated using Steffen’s method. Althouhg in this example the temperature is given as an algebraic expression of space, in particular T(x,y)~[\text{\textdegree}C] = 200 + 350 \cdot y PROBLEM mechanical plane_strain READ_MESH square-centered.msh # [-1:+1]x[-1:+1] # fixed at left, uniform traction in the x direction at right BC left fixed BC right tx=50 # ASME II Part D pag. 785 Carbon steels with C<=0.30% FUNCTION E_carbon(temp) INTERPOLATION steffen DATA { -200 216 -125 212 -75 209 25 202 100 198 150 195 200 192 250 189 300 185 350 179 400 171 450 162 500 151 550 137 } # known temperature distribution # (we could have read it from an outpout of a thermal problem) T(x,y) := 200 + 350*y # Young modulus is the function above evaluated at the local temperature E(x,y) := E_carbon(T(x,y)) # uniform Poisson's ratio nu = 0.3 SOLVE_PROBLEM WRITE_MESH mechanical-square-temperature.vtk E VECTOR u v 0  By replacing T(x,y) = 200 + 350*y with T(x,y) = 200 we can compare the results of the temperature-dependent case with the uniform-properties case (fig. 21): $ feenox mechanical-square-temperature.fee
$diff mechanical-square-temperature.fee mechanical-square-uniform.fee 29c29 < T(x,y) := 200 + 350*y --- > T(x,y) := 200 38c38 < WRITE_MESH mechanical-square-temperature.vtk E VECTOR u v 0 --- > WRITE_MESH mechanical-square-uniform.vtk E VECTOR u v 0$ feenox mechanical-square-uniform.fee
$ In real applications this distribution T(x,y) can be read from the output of an actual heat conduction problem. See sec. 3.2.1 for a revisit of this case, reading the temperature from an unstructured triangular mesh instead of hard-coding it as an algebraic expression of space. So remember, in FeenoX everything is an expression—especially temperature, as also shown in the next section. ### 3.1.6 Matching formulations The Lorenz’ dynamical system system and the NAFEMS LE10 benchmark problem, both discussed earlier in sec. 1.2, illustrate how well the FeenoX input file matches the usual human-readable formulation of ODE or PDE problems. In effect, fig. 2 shows there is a trivial one-to-one correspondence between the sections of the problem formuated in a sheet of paper and the text file nafems-le10.fee. A further example can be given by solving the following case. Let us consider the NAFEMS LE11 benchmark problem titled “Solid cylinder/taper/sphere-temperature” stated in fig. 22. It consists of an axi-symmetrical geometry subject to thermal loading by a temperature distribution given by an algebraic expression. The material properties are linear, orthotropic and uniform. The boundary conditions prescribe symmetries in all directions. • Loading • Linear temperature gradient in the radial an axial direction T(x,y,z)~\text{[ºC]} = \left(x^2+ y^2\right)^{1/2} + z • Boundary conditions • Symmetry on x-z plane, i.e. zero y-displacement • Symmetry on y-z plane, i.e. zero x-displacement • Face on x-y plane zero z-displacement • Face HIH'I' zero z-displacement • Material properties • Isotropic, E=210 \times 10^3~\text{MPa}, \nu = 0.3 • Thermal expansion coefficient \alpha = 2.3 \times 10^{-4}~\text{ºC}^{-1} • Output • Direct stress \sigma_{zz} at point A To solve this problem, we can use the following FeenoX input file that exactly matches the human-readable formulation: PROBLEM mechanical READ_MESH nafems-le11.msh # linear temperature gradient in the radial and axial direction T(x,y,z) = (x^2 + y^2)^(1/2) + z # Boundary conditions BC xz symmetry BC yz symmetry BC xy w=0 BC HIH'I' w=0 # material properties (isotropic & uniform so we can use scalar constants) E = 210e3*1e6 # mesh is in meters, so E=210e3 MPa -> Pa nu = 0.3 # dimensionless alpha = 2.3e-4 # in 1/ºC as in the problem SOLVE_PROBLEM WRITE_MESH nafems-le11.vtk VECTOR u v w T sigma1 sigma2 sigma3 sigma sigmaz PRINT "sigma_z(A) =" sigmaz(0,1,0)/1e6 "MPa (target was -105 MPa)" SEP " " $ time feenox nafems-le11.fee
sigma_z(A) = -105.041 MPa (target was -105 MPa)

real    0m1.766s
user    0m1.642s
sys     0m0.125s

This feature can be better appreciated by comparing the input files needed to solve these kind of NAFEMS benchmarks with other finite-element tools. Sec. 9 gives a glympse for the NAFEMS LE10 problem, where the input files are way too cryptic and cumbersome as compared to what FeenoX needs.

### 3.1.7 Comparison of solutions

One cornerstone design feature is that FeenoX has to provide a way to compare its numerical results with other already-know solutions—either analytical or numerical—in order to verify their validity. Indeed, being able to take the difference between the numerical result and an algebraic expression evaluated at arbitrary locations (usually quadrature points to compute~L_p norms of the error) is a must if code verification through the Method of Manufactured Solutions is required (see sec. 4.4.2).

Let us consider a one-dimensional slab reactor with uniform macroscopic cross sections

\begin{aligned} \Sigma_t &= 0.54628~\text{cm}^{-1} \\ \Sigma_s &= 0.464338~\text{cm}^{-1} \\ \nu\Sigma_f &= 1.70 \cdot 0.054628~\text{cm}^{-1} \end{aligned}

such that, if computed with neutron transport theory, is exactly critical with a width of a = 2 \cdot 10.371065~\text{cm}. Just to illustrate a simple case, we can solve it using the diffusion approximation with zero flux at both as. This case has an analytical solution for both the effective multiplication factor

k_\text{eff} = \frac{\nu\Sigma_f}{(\Sigma_t - \Sigma_s) + D \cdot \left(\frac{\pi}{a} \right)^2}

and the flux distribution

\phi(x) = \frac{\pi}{2} \cdot \sin\left(\frac{x}{a} \cdot \pi\right)

provided the diffusion coefficient D is defined as

D = \frac{1}{3 \cdot \Sigma_t}

We can solve both the numerical and analytical problems in FeenoX, and more importantly, we can subtract them at any point of space we want:

PROBLEM neutron_diffusion 1D

a = 2 * 10.371065 # critical size of the problem UD20-1-0-SL (number 22 report Los Alamos)

Sigma_t1 = 0.54628
Sigma_s1.1 = 0.464338
nuSigma_f1 = 1.70*0.054628
D1 = 1/(3*Sigma_t1)

# null scalar flux at both ends of the slab
BC left  null
BC right null

SOLVE_PROBLEM

# analytical effective multiplication factor (diffusion approximation)
keff_diff = nuSigma_f1/((Sigma_t1-Sigma_s1.1) + D1*(pi/a)^2)

# analytical normalized flux distribution (diffusion approximation)
phi_diff(x) = pi/2 * sin(x/a * pi)

PRINT_FUNCTION FORMAT %+.3f phi1 phi_diff phi1(x)-phi_diff(x) HEADER
PRINT TEXT "\# keff      = " %.8f keff
PRINT TEXT "\# kdiff     = " %.8f keff_diff
PRINT TEXT "\# rel error = " %+.2e (keff-keff_diff)/keff
$feenox neutron-diffusion-1d-null.fee # x phi1 phi_diff phi1(x)-phi_diff(x) +0.000 +0.000 +0.000 +0.000 +10.371 +1.574 +1.571 +0.003 +20.742 +0.000 +0.000 -0.000 +1.474 +0.348 +0.348 +0.001 +2.829 +0.654 +0.653 +0.001 +4.074 +0.911 +0.909 +0.002 +5.217 +1.118 +1.116 +0.002 +6.268 +1.280 +1.277 +0.002 +7.233 +1.399 +1.397 +0.003 +8.120 +1.483 +1.480 +0.003 +8.935 +1.537 +1.534 +0.003 +9.683 +1.565 +1.562 +0.003 +11.059 +1.565 +1.562 +0.003 +11.807 +1.537 +1.534 +0.003 +12.622 +1.483 +1.480 +0.003 +13.509 +1.399 +1.397 +0.003 +14.474 +1.280 +1.277 +0.002 +15.525 +1.118 +1.116 +0.002 +16.668 +0.911 +0.909 +0.002 +17.913 +0.654 +0.653 +0.001 +19.268 +0.348 +0.348 +0.001 # keff = 0.96774162 # kdiff = 0.96797891 # rel error = -2.45e-04$

Something similar could have been done for two groups of energy, although the equations get a little bit more complex so we leave it as an example in the Git repository.

A notable non-trivial thermo-mechanical problem that nevertheless has an analytical solution for the displacement field is the “Parallelepiped whose Young’s modulus is a function of the temperature” (fig. 24). The problem consists of finding the non-dimensional temperature T and displacements u, v and w distributions within a solid parallelepiped of length \ell whose base is a square of h\times h. The solid is subject to heat fluxes and to a traction pressure at the same time. The non-dimensional Young’s modulus E of the material depends on the temperature T in a know algebraically way, whilst both the Poisson coefficient \nu and the thermal conductivity k are uniform and do not depend on the spatial coordinates:

\begin{aligned} E(T) &= \frac{1000}{800-T} \\ \nu &= 0.3 \\ k &= 1 \end{aligned}

The thermal boundary conditions are

• Temperature at point A at (\ell,0,0) is zero
• Heat flux q^{\prime \prime} through x=\ell is +2
• Heat flux q^{\prime \prime} through x=0 is -2
• Heat flux q^{\prime \prime} through y=h/2 is +3
• Heat flux q^{\prime \prime} through y=-h/2 is -3
• Heat flux q^{\prime \prime} through z=h/2 is +4
• Heat flux q^{\prime \prime} through z=-h/2 is -4

The mechanical boundary conditions are

• Point O at (0,0,0) is fixed
• Point B at (0,h/2,0) is restricted to move only in the y direction
• Point C at (0,0,/h2) cannot move in the x direction
• Surfaces x=0 and x=\ell are subject to an uniform normal traction equal to one

The analytical solution is

\begin{aligned} T(x,y,z) &= -2x -3y -4z + 40 \\ u(x,y,z) &= \frac{A}{2} \cdot\left[x^2 + \nu\cdot\left(y^2+z^2\right)\right] + B\cdot xy + C\cdot xz + D\cdot x - \nu\cdot \frac{Ah}{4} \cdot \left(y+z\right) \\ v(x,y,z) &= -\nu\cdot \left[A\cdot x y + \frac{B}{2} \cdot \left(y^2-z^2+\frac{x^2}{\nu}\right) + C\cdot y z + D\cdot y -A\cdot h/4\cdot x - C\cdot h/4\cdot z\right] \\ w(x,y,z) &= -\nu\cdot \left[A\cdot x z + B\cdot yz + C/2\cdot \left(z^2-y^2+\frac{x^2}{\nu}\right) + D\cdot z + \frac{Ch}{4} \cdot y - \frac{Ah}{4} \cdot x\right] \\ \end{aligned}

where~A=0.002, B=0.003, C=0.004 and~D=0.76. The reference results are the temperature at points O and D and the displacements at points A and D ([@tab:parallelepiped]} (table~).

Reference results the original benchmark problem {#tab:parallelepiped}
Point Unknown Reference value
O T +40.0
D T -35.0
A u +15.6
v -0.57
w -0.77
D u +16.3
v -1.785
w -2.0075

First, the thermal problem is solved with FeenoX and the temperature distribution T(x,y,z) is written into a .msh file.

PROBLEM neutron_diffusion 1D

a = 2 * 10.371065 # critical size of the problem UD20-1-0-SL (number 22 report Los Alamos)

Sigma_t1 = 0.54628
Sigma_s1.1 = 0.464338
nuSigma_f1 = 1.70*0.054628
D1 = 1/(3*Sigma_t1)

# null scalar flux at both ends of the slab
BC left  null
BC right null

SOLVE_PROBLEM

# analytical effective multiplication factor (diffusion approximation)
keff_diff = nuSigma_f1/((Sigma_t1-Sigma_s1.1) + D1*(pi/a)^2)

# analytical normalized flux distribution (diffusion approximation)
phi_diff(x) = pi/2 * sin(x/a * pi)

PRINT_FUNCTION FORMAT %+.3f phi1 phi_diff phi1(x)-phi_diff(x) HEADER
PRINT TEXT "\# keff      = " %.8f keff
PRINT TEXT "\# kdiff     = " %.8f keff_diff
PRINT TEXT "\# rel error = " %+.2e (keff-keff_diff)/keff

Then, the mechanical problem reads two meshes: one for solving the actual mechanical problem and another one for reading T(x,y,z) from the previous step. Note that the former contains second-order hexahedra and the latter first-order tetrahedra. After effectively solving the problem, it writes again [@tab:parallelepiped] in Markdown.

The FeenoX implementation illustrates several design features, namely

### 3.1.9 Git and macro-friendliness

The FeenoX input files as plain ASCII files by design. This means that they can be tracked with Git or any other version control system so as to allow reliable traceability of computations. Along with the facts that FeenoX interacts well with

1. Gmsh, that can either use ASCII input files as well or be used as an API from C, C++, Python and Julia, and
2. Other scripting languages such as Bash, Python or even AWK, whose input files are ASCII files as well,

makes it possible to track a whole computation using FeenoX as a Git repository, as already exemplified in sec. 3.1.4. It is important to note that what files that should be tracked in Git include

1. READMEs and documentation in Markdown
2. Shell scripts
3. Gmsh input files and/or scripts
4. FeenoX input files

Files that should not be tracked include

1. Documentation in HTML or PDF
2. Mesh files
3. VTK and result files

since in principle they could be generated from the files in the Git repository by executing the scripts, Gmsh and/or FeenoX.

Even more, in some cases, the FeenoX input files—following the Unix rule of generation–can be created out of generic macros that create particular cases. For example, say one has a mesh of a fin-based dissipator where all the surfaces are named surf_1_i for i=1,...,26. All of them will have a convection boundary condition while surface number 6 is the one that is attached to the electronic part that has to be cooled. Instead of having to “manually” giving the list of surfaces that have the convection condition, we can use M4 to do it for us:

PROBLEM thermal 3d

include(forloop.m4)
BC convection h=10 Tref=-5 forloop(i, 1, 5, PHYSICAL_GROUP "surf_1_'i"' ) forloop(i, 7, 26, PHYSICAL_GROUP "surf_1_'i"' )

BC surf_1_6 q=1000
k = 237
SOLVE_PROBLEM
WRITE_MESH fins.vtk T

Note that since FeenoX was born in Unix, we can pipe the output of m4 to FeenoX directly by using - as the input file in the command line:

$m4 fins.fee.m4 | feenox -$

Fig. 25 confirms that all the faces have the right boundary conditions: face number six got the power BC and all the rest got the convection BC.

Besides being ASCII files, should special characters be needed for any reason within a particular application of FeenoX, UTF-8 characters can be used natively as illustrated in fig. 26.

macro-friendly inputs, rule of generation ejemplo del alpha-flex

VCS tracking, example with hello world. data separated from mesh

## 3.2 Results output

The output ought to contain useful results and should not be cluttered up with non-mandatory information such as ASCII art, notices, explanations or copyright notices. Since the time of cognizant engineers is far more expensive than CPU time, output should be easily interpreted by either a human or, even better, by other programs or interfaces—especially those based in mobile and/or web platforms. Open-source formats and standards should be preferred over privative and ad-hoc formatting to encourage the possibility of using different workflows and/or interfaces.

JSON/YAML, state of the art open post-processing formats. Mobile & web-friendly.

Common and preferably open-source formats.

100% user-defined output with PRINT, rule of silence rule of economy, i.e. no RELAP yaml/json friendly outputs vtk (vtu), gmsh, frd?

90% is serial (vtk), no need to complicate due to 10%

### 3.2.1 Data exchange between non-conformal meshes

To illustrate how the output of a FeenoX execution can be read by another FeenoX instance, let us revisit the plane-strain square from sec. 3.1.5. This time, instead of setting the temperature with an algebraic expression, we will solve a thermal problem that gives rise to the same temperature distribution but on a different mesh.

First, we solve a thermal problem on the same square [-1,+1]\times[-1,+1] such that the resulting temperature field is T(x,y) = 200 + 350\cdot y:

PROBLEM thermal 2D

BC bottom    T=-150
BC top       T=+550
k = 1

SOLVE_PROBLEM
WRITE_MESH thermal-square-temperature.msh T

Now, we read the temperature T(x,y) from the thermal output mesh file thermal-square-temperature.msh (which is a triangular unstructured grid) into the mechanical input mesh file square-centered.msh (which is a structured quadrangular grid):

PROBLEM mechanical plane_strain

# fixed at left, uniform traction in the x direction at right
BC left    fixed
BC right   tx=50

# ASME II Part D pag. 785 Carbon steels with C<=0.30%
FUNCTION E_carbon(temp) INTERPOLATION steffen DATA {
-200  216
-125  212
-75   209
25    202
100   198
150   195
200   192
250   189
300   185
350   179
400   171
450   162
500   151
550   137
}

# read the temperature from a previous result

# Young modulus is the function above evaluated at the local temperature
E(x,y) := E_carbon(T(x,y))

# uniform Poisson's ratio
nu = 0.3

SOLVE_PROBLEM
WRITE_MESH mechanical-square-temperature-from-msh.vtk  E T VECTOR u v 0   

Indeed, the terminal mimic shows the difference between the mechanical input from this section and the one that used an explicit algebraic expression.

$gmsh -2 square-centered-unstruct.geo [...] Info : Done meshing 2D (Wall 0.012013s, CPU 0.033112s) Info : 65 nodes 132 elements Info : Writing 'square-centered-unstruct.msh'... Info : Done writing 'square-centered-unstruct.msh' Info : Stopped on Wed Aug 3 17:47:39 2022 (From start: Wall 0.0208329s, CPU 0.064825s)$ feenox thermal-square.fee
$feenox mechanical-square-temperature-from-msh.fee$ diff mechanical-square-temperature-from-msh.fee mechanical-square-temperature.fee
26,27c26,29
< # read the temperature from a previous result
---
>
> # known temperature distribution
> # (we could have read it from an outpout of a thermal problem)
> T(x,y) := 200 + 350*y
36c38
< WRITE_MESH mechanical-square-temperature-from-msh.vtk  E T VECTOR u v 0
---
> WRITE_MESH mechanical-square-temperature.vtk  E  VECTOR u v 0
$ # 4 Quality assurance Since the results obtained with the tool might be used in verifying existing equipment or in designing new mechanical parts in sensitive industries, a certain level of software quality assurance is needed. Not only are best-practices for developing generic software such as • employment of a version control system, • automated testing suites, • user-reported bug tracking support. • etc. required, but also since the tool falls in the category of engineering computational software, verification and validation procedures are also mandatory, as discussed below. Design should be such that governance of engineering data including problem definition, results and documentation can be efficiently performed using state-of-the-art methodologies, such as distributed control version systems ## 4.1 Reproducibility and traceability The full source code and the documentation of the tool ought to be maintained under a control version system. Whether access to the repository is public or not is up to the vendor, as long as the copying conditions are compatible with the definitions of both free and open source software from the FSF and the OSI, respectively as required in sec. 1. In order to be able to track results obtained with different version of the tools, there should be a clear release procedure. There should be periodical releases of stable versions that are required • not to raise any warnings when compiled using modern versions of common compilers (e.g. GNU, Clang, Intel, etc.) • not to raise any errors when assessed with dynamic memory analysis tools (e.g. Valgrind) for a wide variety of test cases • to pass all the automated test suites as specified in sec. 4.2 These stable releases should follow a common versioning scheme, and either the tarballs with the sources and/or the version control system commits should be digitally signed by a cognizant responsible. Other unstable versions with partial and/or limited features might be released either in the form of tarballs or made available in a code repository. The requirement is that unstable tarballs and main (a.k.a. trunk) branches on the repositories have to be compilable. Any feature that does not work as expected or that does not even compile has to be committed into develop branches before being merge into trunk. If the tool has an executable binary, it should be able to report which version of the code the executable corresponds to. If there is a library callable through an API, there should be a call which returns the version of the code the library corresponds to. It is recommended not to mix mesh data like nodes and element definition with problem data like material properties and boundary conditions so as to ease governance and tracking of computational models and the results associated with them. All the information needed to solve a particular problem (i.e. meshes, boundary conditions, spatially-distributed material properties, etc.) should be generated from a very simple set of files which ought to be susceptible of being tracked with current state-of-the-art version control systems. In order to comply with this suggestion, ASCII formats should be favored when possible. simple <-> simple similar <-> Similar ## 4.2 Automated testing A mean to automatically test the code works as expected is mandatory. A set of problems with known solutions should be solved with the tool after each modification of the code to make sure these changes still give the right answers for the right questions and no regressions are introduced. Unit software testing practices like continuous integration and test coverage are recommended but not mandatory. The tests contained in the test suite should be • varied, • diverse, and • independent Due to efficiency issues, there can be different sets of tests (e.g. unit and integration tests, quick and thorough tests, etc.) Development versions stored in non-main branches can have temporarily-failing tests, but stable versions have to pass all the test suites. make check regressions, example of the change of a sign ## 4.3 Bug reporting and tracking A system to allow developers and users to report bugs and errors and to suggest improvements should be provided. If applicable, bug reports should be tracked, addressed and documented. User-provided suggestions might go into the back log or TO-DO list if appropriate. Here, “bug and errors” mean failure to • compile on supported architectures, • run (unxepected run-time errors, segmentation faults, etc.) • return a correct result github mailing listings ## 4.4 Verification Verification, defined as The process of determining that a model implementation accurately represents the developer’s conceptual description of the model and the solution to the model. i.e. checking if the tool is solving right the equations, should be performed before applying the code to solve any industrial problem. Depending on the nature and regulation of the industry, the verification guidelines and requirements may vary. Since it is expected that code verification tasks could be performed by arbitrary individuals or organizations not necessarily affiliated with the tool vendor, the source code should be available to independent third parties. In this regard, changes in the source code should be controllable, traceable and well documented. Even though the verification requirements may vary among problem types, industries and particular applications, a common method to verify the code is to compare solutions obtained with the tool with known exact solutions or benchmarks. It is thus mandatory to be able to compare the results with analytical solutions, either internally in the tool or through external libraries or tools. This approach is called the Method of Exact Solutions and it is the most widespread scheme for verifying computational software, although it does not provide a comprehensive method to verify the whole spectrum of features. In any case, the tool’s output should be susceptible of being post-processed and analysed in such a way to be able to determine the order of convergence of the numerical solution as compared to the exact one. Another possibility is to follow the Method of Manufactured Solutions, which does address all the shortcomings of MES. It is highly encouraged that the tool allows the application of MMS for software verification. Indeed, this method needs a full explanation of the equations solved by the tool, up to the point that [@sandia-mms] says that Difficulties in determination of the governing equations arises frequently with commercial software, where some information is regarded as proprietary. If the governing equations cannot be determined, we would question the validity of using the code. To enforce the availability of the governing equations, the tool has to be open source as required in sec. 1 and well documented as required in sec. 4.6. A report following either the MES and/or MMS procedures has to be prepared for each type of equation that the tool can solve. The report should show how the numerical results converge to the exact or manufactured results with respect to the mesh size or number of degrees of freedom. This rate should then be compared to the theoretical expected order. Whenever a verification task is performed and documented, at least one of the cases should be added to the test suite. Even though the verification report must contain a parametric mesh study, a single-mesh case is enough to be added to the test suite. The objective of the tests defined in sec. 4.2 is to be able to detect regressions which might have been inadvertently introduced in the code and not to do any actual verification. Therefore a single-mesh case is enough for the test suites. open source (really, not like CCX -> mostrar ejemplo) GPLv3+ free Git + gitlab, github, bitbucket ### 4.4.1 Method of Exact Solutions ### 4.4.2 Method of Manufactured Solutions ## 4.5 Validation As with verification, for each industrial application of the tool there should be a documented procedure to perform a set of validation tests, defined as The process of determining the degree to which a model is an accurate representation of the real world from the perspective of the intended uses of the model. i.e. checking that the right equations are being solved by the tool. This procedure should be based on existing industry standards regarding verification and validation such as ASME, AIAA, IAEA, etc. There should be a procedure for each type of physical problem (thermal, mechanical, thermomechanical, nuclear, etc.) and for each problem type when a new • geometry, • mesh type, • material model, • boundary condition, • data interpolation scheme or any other particular application-dependent feature is needed. A report following the validation procedure defined above should be prepared and signed by a responsible engineer in a case-by-case basis for each particular field of application of the tool. Verification can be performed against • known analytical results, and/or • other already-validated tools following the same standards, and/or • experimental results. already done for Fino hip implant, 120+ pages, ASME, cases of increasing complexity ## 4.6 Documentation Documentation should be complete and cover both the user and the developer point of view. It should include a user manual adequate for both reference and tutorial purposes. Other forms of simplified documentation such as quick reference cards or video tutorials are not mandatory but highly recommended. Since the tool should be extendable (sec. 2.7), there should be a separate development manual covering the programming design and implementation, explaining how to extend the code and how to add new features. Also, as non-trivial mathematics which should be verified (sec. 4.4) are expected, a thorough explanation of what equations are taken into account and how they are solved is required. It should be possible to make the full documentation available online in a way that it can be both printed in hard copy and accessed easily from a mobile device. Users modifying the tool to suit their own needs should be able to modify the associated documentation as well, so a clear notice about the licensing terms of the documentation itself (which might be different from the licensing terms of the source code itself) is mandatory. Tracking changes in the documentation should be similar to tracking changes in the code base. Each individual document ought to explicitly state to which version of the tool applies. Plain ASCII formats should be preferred. It is forbidden to submit documentation in a non-free format. The documentation shall also include procedures for • reporting errors and bugs • releasing stable versions • performing verification and validation studies • contributing to the code base, including • code of conduct • coding styles • variable and function naming convections it’s not compact, but almost! Compactness is the property that a design can fit inside a human being’s head. A good practical test for compactness is this: Does an experienced user normally need a manual? If not, then the design (or at least the subset of it that covers normal use) is compact. unix man page markdown + pandoc = html, pdf, texinfo # 5 Appendix: Downloading and compiling FeenoX ## 5.1 Binary executables Browse to https://www.seamplex.com/feenox/dist/ and check what the latest version for your architecture is. Then do feenox_version=0.2.85-g48a2b76 wget -c https://www.seamplex.com/feenox/dist/linux/feenox-v${feenox_version}-linux-amd64.tar.gz
tar xzf feenox-v${feenox_version}-linux-amd64.tar.gz sudo cp feenox-v${feenox_version}-linux-amd64/bin/feenox /usr/local/bin

You’ll have the binary under bin and examples, documentation, manpage, etc under share. Copy bin/feenox into somewhere in the PATH and that will be it. If you are root, do

sudo cp feenox-v${feenox_version}-linux-amd64/bin/feenox /usr/local/bin If you are not root, the usual way is to create a directory $HOME/bin and add it to your local path. If you have not done it already, do

mkdir -p $HOME/bin echo 'expot PATH=$PATH:$HOME/bin' >> .bashrc Then finally copy bin/feenox to $HOME/bin

cp feenox-v${feenox_version}-linux-amd64/bin/feenox$HOME/bin

Check if it works by calling feenox from any directory (you might need to open a new terminal so .bashrc is re-read):

$feenox FeenoX v0.2.102-g0cb44d2-dirty a free no-fee no-X uniX-like finite-element(ish) computational engineering tool usage: feenox [options] inputfile [replacement arguments] [petsc options] -h, --help display options and detailed explanations of commmand-line usage -v, --version display brief version information and exit -V, --versions display detailed version information --pdes list the types of PROBLEMs that FeenoX can solve, one per line Run with --help for further explanations.$ 

## 5.2 Source tarballs

To compile the source tarball, proceed as follows. This procedure does not need git nor autoconf but a new tarball has to be downloaded each time there is a new FeenoX version.

1. Install mandatory dependencies

sudo apt-get install gcc make libgsl-dev

If you cannot install libgsl-dev, you can have the configure script to download and compile it for you. See point 4 below.

2. Install optional dependencies (of course these are optional but recommended)

sudo apt-get install libsundials-dev petsc-dev slepc-dev

wget https://www.seamplex.com/feenox/dist/src/feenox-v0.1.66-g1c4b17b.tar.gz
tar xvzf feenox-v0.1.66-g1c4b17b.tar.gz
4. Configure, compile & make

cd feenox-v0.1.66-g1c4b17b
./configure
make -j4

If you cannot (or do not want) to use libgsl-dev from a package repository, call configure with --enable-download-gsl:

./configure --enable-download-gsl

If you do not have Internet access, get the tarball manually, copy it to the same directory as configure and run again.

5. Run test suite (optional)

make check
6. Install the binary system wide (optional)

sudo make install

## 5.3 Git repository

To compile the Git repository, proceed as follows. This procedure does need git and autoconf but new versions can be pulled and recompiled easily. If something goes wrong and you get an error, do not hesitate to ask in FeenoX’ discussion page.

1. Install mandatory dependencies

sudo apt-get install gcc make git automake autoconf libgsl-dev

If you cannot install libgsl-dev but still have git and the build toolchain, you can have the configure script to download and compile it for you. See point 4 below.

2. Install optional dependencies (of course these are optional but recommended)

sudo apt-get install libsundials-dev petsc-dev slepc-dev
3. Clone Github repository

git clone https://github.com/seamplex/feenox
4. Boostrap, configure, compile & make

cd feenox
./autogen.sh
./configure
make -j4

If you cannot (or do not want) to use libgsl-dev from a package repository, call configure with --enable-download-gsl:

./configure --enable-download-gsl

If you do not have Internet access, get the tarball manually, copy it to the same directory as configure and run again. See the detailed compilation instructions for an explanation.

5. Run test suite (optional)

make check
6. Install the binary system wide (optional)

sudo make install

To stay up to date, pull and then autogen, configure and make (and optionally install):

git pull
./autogen.sh; ./configure; make -j4
sudo make install

# 6 Appendix: Rules of UNIX philosophy

## 6.1 Rule of Modularity

Developers should build a program out of simple parts connected by well defined interfaces, so problems are local, and parts of the program can be replaced in future versions to support new features. This rule aims to save time on debugging code that is complex, long, and unreadable.

• FeenoX uses third-party high-quality libraries
• GNU Scientific Library
• SUNDIALS
• PETSc
• SLEPc

## 6.2 Rule of Clarity

Developers should write programs as if the most important communication is to the developer who will read and maintain the program, rather than the computer. This rule aims to make code as readable and comprehensible as possible for whoever works on the code in the future.

• Example two squares in thermal contact.
• LE10 & LE11: a one-to-one correspondence between the problem text and the FeenoX input.

## 6.3 Rule of Composition

Developers should write programs that can communicate easily with other programs. This rule aims to allow developers to break down projects into small, simple programs rather than overly complex monolithic programs.

• FeenoX uses meshes created by a separate mesher (i.e. Gmsh).
• FeenoX writes data that has to be plotted or post-processed by other tools (Gnuplot, Gmsh, Paraview, etc.).
• ASCII output is 100% controlled by the user so it can be tailored to suit any other programs’ input needs such as AWK filters to create LaTeX tables.

## 6.4 Rule of Separation

Developers should separate the mechanisms of the programs from the policies of the programs; one method is to divide a program into a front-end interface and a back-end engine with which that interface communicates. This rule aims to prevent bug introduction by allowing policies to be changed with minimum likelihood of destabilizing operational mechanisms.

• FeenoX does not include a GUI, but it is GUI-friendly.

## 6.5 Rule of Simplicity

Developers should design for simplicity by looking for ways to break up program systems into small, straightforward cooperating pieces. This rule aims to discourage developers’ affection for writing “intricate and beautiful complexities” that are in reality bug prone programs.

• Simple problems need simple input.
• Similar problems need similar inputs.
• English-like self-evident input files matching as close as possible the problem text.
• If there is a single material there is no need to link volumes to properties.

## 6.6 Rule of Parsimony

Developers should avoid writing big programs. This rule aims to prevent overinvestment of development time in failed or suboptimal approaches caused by the owners of the program’s reluctance to throw away visibly large pieces of work. Smaller programs are not only easier to write, optimize, and maintain; they are easier to delete when deprecated.

• Parametric and/or optimization runs have to be driven from an outer script (Bash, Python, etc.)

## 6.7 Rule of Transparency

Developers should design for visibility and discoverability by writing in a way that their thought process can lucidly be seen by future developers working on the project and using input and output formats that make it easy to identify valid input and correct output. This rule aims to reduce debugging time and extend the lifespan of programs.

• Written in C99
• Makes use of structures and function pointers to give the same functionality as C++’s virtual methods without needing to introduce other complexities that make the code base harder to maintain and to debug.

## 6.8 Rule of Robustness

Developers should design robust programs by designing for transparency and discoverability, because code that is easy to understand is easier to stress test for unexpected conditions that may not be foreseeable in complex programs. This rule aims to help developers build robust, reliable products.

## 6.9 Rule of Representation

Developers should choose to make data more complicated rather than the procedural logic of the program when faced with the choice, because it is easier for humans to understand complex data compared with complex logic. This rule aims to make programs more readable for any developer working on the project, which allows the program to be maintained.

## 6.10 Rule of Least Surprise

Developers should design programs that build on top of the potential users’ expected knowledge; for example, ‘+’ in a calculator program should always mean ‘addition’. This rule aims to encourage developers to build intuitive products that are easy to use.

• If one needs a problem where the conductivity depends on x as k(x)=1+x then the input is

k(x) = 1+x
• If a problem needs a temperature distribution given by an algebraic expression T(x,y,z)=\sqrt{x^2+y^2}+z then do

T(x,y,z) = sqrt(x^2+y^2) + z

## 6.11 Rule of Silence

Developers should design programs so that they do not print unnecessary output. This rule aims to allow other programs and developers to pick out the information they need from a program’s output without having to parse verbosity.

• No PRINT (or WRITE_MESH), no output.

## 6.12 Rule of Repair

Developers should design programs that fail in a manner that is easy to localize and diagnose or in other words “fail noisily”. This rule aims to prevent incorrect output from a program from becoming an input and corrupting the output of other code undetected.

• Input errors are detected before the computation is started:

$feenox thermal-error.fee error: undefined thermal conductivity 'k'$ 
• Run-time errors can be user controled, they can be fatal or ignored.

## 6.13 Rule of Economy

Developers should value developer time over machine time, because machine cycles today are relatively inexpensive compared to prices in the 1970s. This rule aims to reduce development costs of projects.

• Output is 100% user-defined so the desired results is directly obtained instead of needing further digging into tons of undesired data.The approach of “compute and write everything you can in one single run” made sense in 1970 where CPU time was more expensive than human time, but not anymore.
• Example: LE10 & LE11.

## 6.14 Rule of Generation

Developers should avoid writing code by hand and instead write abstract high-level programs that generate code. This rule aims to reduce human errors and save time.

• Inputs are M4-like-macro friendly.
• Parametric runs can be done from scripts through command line arguments expansion.
• Documentation is created out of simple Markdown sources and assembled as needed.

## 6.15 Rule of Optimization

Developers should prototype software before polishing it. This rule aims to prevent developers from spending too much time for marginal gains.

• Premature optimization is the root of all evil
• We are still building. We will optimize later.
• Code optimization: TODO
• Parallelization: TODO
• Comparison with other tools: TODO

## 6.16 Rule of Diversity

Developers should design their programs to be flexible and open. This rule aims to make programs flexible, allowing them to be used in ways other than those their developers intended.

• Either Gmsh or Paraview can be used to post-process results.
• Other formats can be added.

## 6.17 Rule of Extensibility

Developers should design for the future by making their protocols extensible, allowing for easy plugins without modification to the program’s architecture by other developers, noting the version of the program, and more. This rule aims to extend the lifespan and enhance the utility of the code the developer writes.

• FeenoX is GPLv3+. The ‘+’ is for the future.
• Each PDE has a separate source directory. Any of them can be used as a template for new PDEs, especially laplace for elliptic operators.

# 7 Appendix: FeenoX history

Very much like UNIX in the late 1960s, FeenoX is a third-system effect: I wrote a first hack that seemed to work better than I had expected. Then I tried to add a lot of features and complexities which I felt the code needed. After ten years of actual usage, I then realized

1. what was worth keeping,
2. what needed to be rewrittenm and

The first version was called wasora, the second was “The wasora suite” (i.e. a generic framework plus a bunch of “plugins”, including a thermo-mechanical one named Fino) and then finally FeenoX. The story that follows explains why I wrote the first hack to begin with.

It was at the movies when I first heard about dynamical systems, non-linear equations and chaos theory. The year was 1993, I was ten years old and the movie was Jurassic Park. Dr. Ian Malcolm (the character portrayed by Jeff Goldblum) explained sensitivity to initial conditions in a memorable scene, which is worth watching again and again. Since then, the fact that tiny variations may lead to unexpected results has always fascinated me. During high school I attended a very interesting course on fractals and chaos that made me think further about complexity and its mathematical description. Nevertheless, it was not not until college that I was able to really model and solve the differential equations that give rise to chaotic behavior.

In fact, initial-value ordinary differential equations arise in a great variety of subjects in science and engineering. Classical mechanics, chemical kinetics, structural dynamics, heat transfer analysis and dynamical systems, among other disciplines, heavily rely on equations of the form

\dot{\mathbf{x}} = F(\mathbf{x},t)

During my years of undergraduate student (circa 2004–2007), whenever I had to solve these kind of equations I had to choose one of the following three options:

1. to program an ad-hoc numerical method such as Euler or Runge-Kutta, matching the requirements of the system of equations to solve, or
2. to use a standard numerical library such as the GNU Scientific Library and code the equations to solve into a C program (or maybe in Python), or
3. to use a high-level system such as Octave, Maxima, or some non-free (and worse, see below) programs.

Of course, each option had its pros and its cons. But none provided the combination of advantages I was looking for, namely flexibility (option one), efficiency (option two) and reduced input work (partially given by option three). Back in those days I ended up wandering between options one and two, depending on the type of problem I had to solve. However, even though one can, with some effort, make the code read some parameters from a text file, any other drastic change usually requires a modification in the source code—some times involving a substantial amount of work—and a further recompilation of the code. This was what I most disliked about this way of working, but I could nevertheless live with it.

Regardless of this situation, during my last year of Nuclear Engineering, the tipping point came along. Here’s a slightly-fictionalized of a dialog between myself and the teacher at the computer lab, as it might have happened (or not):

— (Prof.) Open MATLAB.™
— (Me) It’s not installed here. I type mathlab and it does not work.
— (Prof.) It’s spelled matlab.
— (Me) Ok, working. (A screen with blocks and lines connecting them appears)
— (Me) What’s this?
— (Prof.) The point reactor equations.
— (Me) It’s not. These are the point reactor equations:

\begin{cases} \dot{\phi}(t) = \displaystyle \frac{\rho(t) - \beta}{\Lambda} \cdot \phi(t) + \sum_{i=1}^{N} \lambda_i \cdot c_i \\ \dot{c}_i(t) = \displaystyle \frac{\beta_i}{\Lambda} \cdot \phi(t) - \lambda_i \cdot c_i \end{cases}

— (Me) And in any case, I’d write them like this in a computer:

phi_dot = (rho-Beta)/Lambda * phi + sum(lambda[i], c[i], i, 1, N)
c_dot[i] = beta[i]/Lambda * phi - lambda[i]*c[i]

This conversation forced me to re-think the ODE-solving issue. I could not (and still cannot) understand why somebody would prefer to solve a very simple set of differential equations by drawing blocks and connecting them with a mouse with no mathematical sense whatsoever. Fast forward fifteen years, and what I wrote above is essentially how one would solve the point kinetics equations with FeenoX.

FeenoX is distributed under the terms of the GNU General Public License version 3 or (at your option) any later version. See licensing below for details.

 GNU/Linux binaries https://www.seamplex.com/feenox/dist/linux Windows binaries https://www.seamplex.com/feenox/dist/windows Source tarballs https://www.seamplex.com/feenox/dist/src Github repository https://github.com/seamplex/feenox/
• Be aware that FeenoX is a backend. It does not have a GUI. Read the documentation, especially the description and the FAQs. Ask for help on the GitHub discussions page.

• Binaries are provided as statically-linked executables for convenience. They do not support MUMPS nor MPI and have only basic optimization flags. Please compile from source for high-end applications. See detailed compilatation instructions.

• Try to avoid Windows as much as you can. The binaries are provided as transitional packages for people that for some reason still use such an outdated, anachronous, awful and invasive operating system. They are compiled with Cygwin and have no support whatsoever. Really, really, get rid of Windows ASAP.

“It is really worth any amount of time and effort to get away from Windows if you are doing computational science.”

https://lists.mcs.anl.gov/pipermail/petsc-users/2015-July/026388.html

These detailed compilation instructions are aimed at amd64 Debian-based GNU/Linux distributions. The compilation procedure follows the POSIX standard, so it should work in other operating systems and architectures as well. Distributions not using apt for packages (i.e. yum) should change the package installation commands (and possibly the package names). The instructions should also work for in MacOS, although the apt-get commands should be replaced by brew or similar. Same for Windows under Cygwin, the packages should be installed through the Cygwin installer. WSL was not tested, but should work as well.

## 8.1 Quickstart

Note that the quickest way to get started is to download an already-compiled statically-linked binary executable. Note that getting a binary is the quickest and easiest way to go but it is the less flexible one. Mind the following instructions if a binary-only option is not suitable for your workflow and/or you do need to compile the source code from scratch.

On a GNU/Linux box (preferably Debian-based), follow these quick steps. See sec. 8.2 for the actual detailed explanations.

To compile the Git repository, proceed as follows. This procedure does need git and autoconf but new versions can be pulled and recompiled easily. If something goes wrong and you get an error, do not hesitate to ask in FeenoX’ discussion page.

1. Install mandatory dependencies

sudo apt-get install gcc make git automake autoconf libgsl-dev

If you cannot install libgsl-dev but still have git and the build toolchain, you can have the configure script to download and compile it for you. See point 4 below.

2. Install optional dependencies (of course these are optional but recommended)

sudo apt-get install libsundials-dev petsc-dev slepc-dev
3. Clone Github repository

git clone https://github.com/seamplex/feenox
4. Boostrap, configure, compile & make

cd feenox
./autogen.sh
./configure
make -j4

If you cannot (or do not want) to use libgsl-dev from a package repository, call configure with --enable-download-gsl:

./configure --enable-download-gsl

If you do not have Internet access, get the tarball manually, copy it to the same directory as configure and run again. See the detailed compilation instructions for an explanation.

5. Run test suite (optional)

make check
6. Install the binary system wide (optional)

sudo make install

To stay up to date, pull and then autogen, configure and make (and optionally install):

git pull
./autogen.sh; ./configure; make -j4
sudo make install

## 8.2 Detailed configuration and compilation

The main target and development environment is Debian GNU/Linux, although it should be possible to compile FeenoX in any free GNU/Linux variant (and even the in non-free MacOS and/or Windows platforms) running in virtually any hardware platform. FeenoX can run be run either in HPC cloud servers or a Raspberry Pi, and almost everything that sits in the middle.

Following the UNIX philosophy discussed in the SDS, FeenoX re-uses a lot of already-existing high-quality free and open source libraries that implement a wide variety of mathematical operations. This leads to a number of dependencies that FeenoX needs in order to implement certain features.

There is only one dependency that is mandatory, namely GNU GSL (see sec. 8.2.1.1), which if it not found then FeenoX cannot be compiled. All other dependencies are optional, meaning that FeenoX can be compiled but its capabilities will be partially reduced.

As per the SRS, all dependencies have to be available on mainstream GNU/Linux distributions and have to be free and open source software. But they can also be compiled from source in case the package repositories are not available or customized compilation flags are needed (i.e. optimization or debugging settings).

In particular, PETSc (and SLEPc) also depend on other mathematical libraries to perform particular operations such as low-level linear algebra operations. These extra dependencies can be either free (such as LAPACK) or non-free (such as Intel’s MKL), but there is always at least one combination of a working setup that involves only free and open source software which is compatible with FeenoX licensing terms (GPLv3+). See the documentation of each package for licensing details.

### 8.2.1 Mandatory dependencies

FeenoX has one mandatory dependency for run-time execution and the standard build toolchain for compilation. It is written in C99 so only a C compiler is needed, although make is also required. Free and open source compilers are favored. The usual C compiler is gcc but clang can also be used. Nevertheless, the non-free icc has also been tested.

Note that there is no need to have a Fortran nor a C++ compiler to build FeenoX. They might be needed to build other dependencies (such as PETSc and its dependencies), but not to compile FeenoX if all the dependencies are installed from the oeprating system’s package repositories. In case the build toolchain is not already installed, do so with

sudo apt-get install gcc make

If the source is to be fetched from the Git repository then not only is git needed but also autoconf and automake since the configure script is not stored in the Git repository but the autogen.sh script that bootstraps the tree and creates it. So if instead of compiling a source tarball one wants to clone from GitHub, these packages are also mandatory:

sudo apt-get install git automake autoconf

Again, chances are that any existing GNU/Linux box has all these tools already installed.

#### 8.2.1.1 The GNU Scientific Library

The only run-time dependency is GNU GSL (not to be confused with Microsoft GSL). It can be installed with

sudo apt-get install libgsl-dev

In case this package is not available or you do not have enough permissions to install system-wide packages, there are two options.

1. Pass the option --enable-download-gsl to the configure script below.

If the configure script cannot find both the headers and the actual library, it will refuse to proceed. Note that the FeenoX binaries already contain a static version of the GSL so it is not needed to have it installed in order to run the statically-linked binaries.

### 8.2.2 Optional dependencies

FeenoX has three optional run-time dependencies. It can be compiled without any of these, but functionality will be reduced:

• SUNDIALS provides support for solving systems of ordinary differential equations (ODEs) or differential-algebraic equations (DAEs). This dependency is needed when running inputs with the PHASE_SPACE keyword.

• PETSc provides support for solving partial differential equations (PDEs). This dependency is needed when running inputs with the PROBLEM keyword.

• SLEPc provides support for solving eigen-value problems in partial differential equations (PDEs). This dependency is needed for inputs with PROBLEM types with eigen-value formulations such as modal and neutron_transport.

In absence of all these, FeenoX can still be used to

These optional dependencies have to be installed separately. There is no option to have configure to download them as with --enable-download-gsl. When running the test suite (sec. 8.2.6), those tests that need an optional dependency which was not found at compile time will be skipped.

#### 8.2.2.1 SUNDIALS

SUNDIALS is a SUite of Nonlinear and DIfferential/ALgebraic equation Solvers. It is used by FeenoX to solve dynamical systems casted as DAEs with the keyword PHASE_SPACE, like the Lorenz system.

Install either by doing

sudo apt-get install libsundials-dev

or by following the instructions in the documentation.

#### 8.2.2.2 PETSc

The Portable, Extensible Toolkit for Scientific Computation, pronounced PET-see (/ˈpɛt-siː/), is a suite of data structures and routines for the scalable (parallel) solution of scientific applications modeled by partial differential equations. It is used by FeenoX to solve PDEs with the keyword PROBLEM, like the NAFEMS LE10 benchmark problem.

Install either by doing

sudo apt-get install petsc-dev

or by following the instructions in the documentation.

Note that

• Configuring and compiling PETSc from scratch might be difficult the first time. It has a lot of dependencies and options. Read the official documentation for a detailed explanation.
• There is a huge difference in efficiency between using PETSc compiled with debugging symbols and with optimization flags. Make sure to configure --with-debugging=0 for FeenoX production runs and leave the debugging symbols (which is the default) for development and debugging only.
• FeenoX needs PETSc to be configured with real double-precision scalars. It will compile but will complain at run-time when using complex and/or single or quad-precision scalars.
• FeenoX honors the PETSC_DIR and PETSC_ARCH environment variables when executing configure. If these two do not exist or are empty, it will try to use the default system-wide locations (i.e. the petsc-dev package).

#### 8.2.2.3 SLEPc

The Scalable Library for Eigenvalue Problem Computations, is a software library for the solution of large scale sparse eigenvalue problems on parallel computers. It is used by FeenoX to solve PDEs with the keyword PROBLEM that need eigen-value computations, such as modal analysis of a cantilevered beam.

Install either by doing

sudo apt-get install slepc-dev

or by following the instructions in the documentation.

Note that

• SLEPc is an extension of PETSc so the latter has to be already installed and configured.
• FeenoX honors the SLEPC_DIR environment variable when executing configure. If it does not exist or is empty it will try to use the default system-wide locations (i.e. the slepc-dev package).
• If PETSc was configured with --download-slepc then the SLEPC_DIR variable has to be set to the directory inside PETSC_DIR where SLEPc was cloned and compiled.

### 8.2.3 FeenoX source code

There are two ways of getting FeenoX’ source code:

1. Cloning the GitHub repository at https://github.com/seamplex/feenox

#### 8.2.3.1 Git repository

The main Git repository is hosted on GitHub at https://github.com/seamplex/feenox. It is public so it can be cloned either through HTTPS or SSH without needing any particular credentials. It can also be forked freely. See the Programming Guide for details about pull requests and/or write access to the main repository.

Ideally, the main branch should have a usable snapshot. All other branches can contain code that might not compile or might not run or might not be tested. If you find a commit in the main branch that does not pass the tests, please report it in the issue tracker ASAP.

After cloning the repository

git clone https://github.com/seamplex/feenox

the autogen.sh script has to be called to bootstrap the working tree, since the configure script is not stored in the repository but created from configure.ac (which is in the repository) by autogen.sh.

Similarly, after updating the working tree with

git pull

it is recommended to re-run the autogen.sh script. It will do a make clean and re-compute the version string.

#### 8.2.3.2 Source tarballs

When downloading a source tarball, there is no need to run autogen.sh since the configure script is already included in the tarball. This method cannot update the working tree. For each new FeenoX release, the whole source tarball has to be downloaded again.

### 8.2.4 Configuration

To create a proper Makefile for the particular architecture, dependencies and compilation options, the script configure has to be executed. This procedure follows the GNU Coding Standards.

./configure

Without any particular options, configure will check if the mandatory GNU Scientific Library is available (both its headers and run-time library). If it is not, then the option --enable-download-gsl can be used. This option will try to use wget (which should be installed) to download a source tarball, uncompress, configure and compile it. If these steps are successful, this GSL will be statically linked into the resulting FeenoX executable. If there is no internet connection, the configure script will say that the download failed. In that case, get the indicated tarball file manually, copy it into the current directory and re-run ./configure.

The script will also check for the availability of optional dependencies. At the end of the execution, a summary of what was found (or not) is printed in the standard output:

$./configure [...] ## ----------------------- ## ## Summary of dependencies ## ## ----------------------- ## GNU Scientific Library from system SUNDIALS IDA yes PETSc yes /usr/lib/petsc SLEPc no [...]  If for some reason one of the optional dependencies is available but FeenoX should not use it, then pass --without-sundials, --without-petsc and/or --without-slepc as arguments. For example $ ./configure --without-sundials --without-petsc
[...]
## ----------------------- ##
## Summary of dependencies ##
## ----------------------- ##
GNU Scientific Library  from system
SUNDIALS                no
PETSc                   no
SLEPc                   no
[...]  

If configure complains about contradicting values from the cached ones, run autogen.sh again before configure and/or clone/uncompress the source tarball in a fresh location.

To see all the available options run

./configure --help

### 8.2.5 Source code compilation

After the successful execution of configure, a Makefile is created. To compile FeenoX, just execute

make

Compilation should take a dozen of seconds. It can be even sped up by using the -j option

make -j8

The binary executable will be located in the src directory but a copy will be made in the base directory as well. Test it by running without any arguments

$./feenox FeenoX v0.2.14-gbbf48c9-dirty a free no-fee no-X uniX-like finite-element(ish) computational engineering tool usage: feenox [options] inputfile [replacement arguments] [petsc options] -h, --help display options and detailed explanations of commmand-line usage -v, --version display brief version information and exit -V, --versions display detailed version information Run with --help for further explanations.$

The -v (or --version) option shows the version and a copyright notice:8

$./feenox -v FeenoX v0.2.14-gbbf48c9-dirty a free no-fee no-X uniX-like finite-element(ish) computational engineering tool Copyright © 2009--2022 Seamplex, https://seamplex.com/feenox GNU General Public License v3+, https://www.gnu.org/licenses/gpl.html. FeenoX is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.$

The -V (or --versions) option shows the dates of the last commits, the compiler options and the versions of the linked libraries:

$./feenox -V FeenoX v0.1.24-g6cfe063 a free no-fee no-X uniX-like finite-element(ish) computational engineering tool Last commit date : Sun Aug 29 11:34:04 2021 -0300 Build date : Sun Aug 29 11:44:50 2021 -0300 Build architecture : linux-gnu x86_64 Compiler version : gcc (Debian 10.2.1-6) 10.2.1 20210110 Compiler expansion : gcc -Wl,-z,relro -I/usr/include/x86_64-linux-gnu/mpich -L/usr/lib/x86_64-linux-gnu -lmpich Compiler flags : -O3 Builder : gtheler@chalmers GSL version : 2.6 SUNDIALS version : 4.1.0 PETSc version : Petsc Release Version 3.14.5, Mar 03, 2021 PETSc arch : PETSc options : --build=x86_64-linux-gnu --prefix=/usr --includedir=${prefix}/include --mandir=${prefix}/share/man --infodir=${prefix}/share/info --sysconfdir=/etc --localstatedir=/var --with-option-checking=0 --with-silent-rules=0 --libdir=${prefix}/lib/x86_64-linux-gnu --runstatedir=/run --with-maintainer-mode=0 --with-dependency-tracking=0 --with-debugging=0 --shared-library-extension=_real --with-shared-libraries --with-pic=1 --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90 --with-cxx-dialect=C++11 --with-opencl=1 --with-blas-lib=-lblas --with-lapack-lib=-llapack --with-scalapack=1 --with-scalapack-lib=-lscalapack-openmpi --with-ptscotch=1 --with-ptscotch-include=/usr/include/scotch --with-ptscotch-lib="-lptesmumps -lptscotch -lptscotcherr" --with-fftw=1 --with-fftw-include="[]" --with-fftw-lib="-lfftw3 -lfftw3_mpi" --with-superlu_dist=1 --with-superlu_dist-include=/usr/include/superlu-dist --with-superlu_dist-lib=-lsuperlu_dist --with-hdf5-include=/usr/include/hdf5/openmpi --with-hdf5-lib="-L/usr/lib/x86_64-linux-gnu/hdf5/openmpi -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lhdf5 -lmpi" --CXX_LINKER_FLAGS=-Wl,--no-as-needed --with-hypre=1 --with-hypre-include=/usr/include/hypre --with-hypre-lib=-lHYPRE_core --with-mumps=1 --with-mumps-include="[]" --with-mumps-lib="-ldmumps -lzmumps -lsmumps -lcmumps -lmumps_common -lpord" --with-suitesparse=1 --with-suitesparse-include=/usr/include/suitesparse --with-suitesparse-lib="-lumfpack -lamd -lcholmod -lklu" --with-superlu=1 --with-superlu-include=/usr/include/superlu --with-superlu-lib=-lsuperlu --prefix=/usr/lib/petscdir/petsc3.14/x86_64-linux-gnu-real --PETSC_ARCH=x86_64-linux-gnu-real CFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-pVufYp/petsc-3.14.5+dfsg1=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fPIC" CXXFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-pVufYp/petsc-3.14.5+dfsg1=. -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fPIC" FCFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-pVufYp/petsc-3.14.5+dfsg1=. -flto=auto -ffat-lto-objects -fstack-protector-strong -fPIC -ffree-line-length-0" FFLAGS="-g -O2 -ffile-prefix-map=/build/petsc-pVufYp/petsc-3.14.5+dfsg1=. -flto=auto -ffat-lto-objects -fstack-protector-strong -fPIC -ffree-line-length-0" CPPFLAGS="-Wdate-time -D_FORTIFY_SOURCE=2" LDFLAGS="-Wl,-Bsymbolic-functions -flto=auto -Wl,-z,relro -fPIC" MAKEFLAGS=w SLEPc version : SLEPc Release Version 3.14.2, Feb 01, 2021$

### 8.2.6 Test suite

The test directory contains a set of test cases whose output is known so that unintended regressions can be detected quickly (see the programming guide for more information). The test suite ought to be run after each modification in FeenoX’ source code. It consists of a set of scripts and input files needed to solve dozens of cases. The output of each execution is compared to a reference solution. In case the output does not match the reference, the test suite fails.

After compiling FeenoX as explained in sec. 8.2.5, the test suite can be run with make check. Ideally everything should be green meaning the tests passed:

$make check Making check in src make[1]: Entering directory '/home/gtheler/codigos/feenox/src' make[1]: Nothing to be done for 'check'. make[1]: Leaving directory '/home/gtheler/codigos/feenox/src' make[1]: Entering directory '/home/gtheler/codigos/feenox' cp -r src/feenox . make check-TESTS make[2]: Entering directory '/home/gtheler/codigos/feenox' make[3]: Entering directory '/home/gtheler/codigos/feenox' XFAIL: tests/abort.sh PASS: tests/algebraic_expr.sh PASS: tests/beam-modal.sh PASS: tests/beam-ortho.sh PASS: tests/builtin.sh PASS: tests/cylinder-traction-force.sh PASS: tests/default_argument_value.sh PASS: tests/expressions_constants.sh PASS: tests/expressions_variables.sh PASS: tests/expressions_functions.sh PASS: tests/exp.sh PASS: tests/i-beam-euler-bernoulli.sh PASS: tests/iaea-pwr.sh PASS: tests/iterative.sh PASS: tests/fit.sh PASS: tests/function_algebraic.sh PASS: tests/function_data.sh PASS: tests/function_file.sh PASS: tests/function_vectors.sh PASS: tests/integral.sh PASS: tests/laplace2d.sh PASS: tests/materials.sh PASS: tests/mesh.sh PASS: tests/moment-of-inertia.sh PASS: tests/nafems-le1.sh PASS: tests/nafems-le10.sh PASS: tests/nafems-le11.sh PASS: tests/nafems-t1-4.sh PASS: tests/nafems-t2-3.sh PASS: tests/neutron_diffusion_src.sh PASS: tests/neutron_diffusion_keff.sh PASS: tests/parallelepiped.sh PASS: tests/point-kinetics.sh PASS: tests/print.sh PASS: tests/thermal-1d.sh PASS: tests/thermal-2d.sh PASS: tests/trig.sh PASS: tests/two-cubes-isotropic.sh PASS: tests/two-cubes-orthotropic.sh PASS: tests/vector.sh XFAIL: tests/xfail-few-properties-ortho-young.sh XFAIL: tests/xfail-few-properties-ortho-poisson.sh XFAIL: tests/xfail-few-properties-ortho-shear.sh ============================================================================ Testsuite summary for feenox v0.2.6-g3237ce9 ============================================================================ # TOTAL: 43 # PASS: 39 # SKIP: 0 # XFAIL: 4 # FAIL: 0 # XPASS: 0 # ERROR: 0 ============================================================================ make[3]: Leaving directory '/home/gtheler/codigos/feenox' make[2]: Leaving directory '/home/gtheler/codigos/feenox' make[1]: Leaving directory '/home/gtheler/codigos/feenox'$

The XFAIL result means that those cases are expected to fail (they are there to test if FeenoX can handle errors). Failure would mean they passed. In case FeenoX was not compiled with any optional dependency, the corresponding tests will be skipped. Skipped tests do not mean any failure, but that the compiled FeenoX executable does not have the full capabilities. For example, when configuring with ./configure --without-petsc (but with SUNDIALS), the test suite output should be a mixture of green and blue:

$./configure --without-petsc [...] configure: creating ./src/version.h ## ----------------------- ## ## Summary of dependencies ## ## ----------------------- ## GNU Scientific Library from system SUNDIALS yes PETSc no SLEPc no Compiler gcc checking that generated files are newer than configure... done configure: creating ./config.status config.status: creating Makefile config.status: creating src/Makefile config.status: creating doc/Makefile config.status: executing depfiles commands$ make
[...]
$make check Making check in src make[1]: Entering directory '/home/gtheler/codigos/feenox/src' make[1]: Nothing to be done for 'check'. make[1]: Leaving directory '/home/gtheler/codigos/feenox/src' make[1]: Entering directory '/home/gtheler/codigos/feenox' cp -r src/feenox . make check-TESTS make[2]: Entering directory '/home/gtheler/codigos/feenox' make[3]: Entering directory '/home/gtheler/codigos/feenox' XFAIL: tests/abort.sh PASS: tests/algebraic_expr.sh SKIP: tests/beam-modal.sh SKIP: tests/beam-ortho.sh PASS: tests/builtin.sh SKIP: tests/cylinder-traction-force.sh PASS: tests/default_argument_value.sh PASS: tests/expressions_constants.sh PASS: tests/expressions_variables.sh PASS: tests/expressions_functions.sh PASS: tests/exp.sh SKIP: tests/i-beam-euler-bernoulli.sh SKIP: tests/iaea-pwr.sh PASS: tests/iterative.sh PASS: tests/fit.sh PASS: tests/function_algebraic.sh PASS: tests/function_data.sh PASS: tests/function_file.sh PASS: tests/function_vectors.sh PASS: tests/integral.sh SKIP: tests/laplace2d.sh PASS: tests/materials.sh PASS: tests/mesh.sh PASS: tests/moment-of-inertia.sh SKIP: tests/nafems-le1.sh SKIP: tests/nafems-le10.sh SKIP: tests/nafems-le11.sh SKIP: tests/nafems-t1-4.sh SKIP: tests/nafems-t2-3.sh SKIP: tests/neutron_diffusion_src.sh SKIP: tests/neutron_diffusion_keff.sh SKIP: tests/parallelepiped.sh PASS: tests/point-kinetics.sh PASS: tests/print.sh SKIP: tests/thermal-1d.sh SKIP: tests/thermal-2d.sh PASS: tests/trig.sh SKIP: tests/two-cubes-isotropic.sh SKIP: tests/two-cubes-orthotropic.sh PASS: tests/vector.sh SKIP: tests/xfail-few-properties-ortho-young.sh SKIP: tests/xfail-few-properties-ortho-poisson.sh SKIP: tests/xfail-few-properties-ortho-shear.sh ============================================================================ Testsuite summary for feenox v0.2.6-g3237ce9 ============================================================================ # TOTAL: 43 # PASS: 21 # SKIP: 21 # XFAIL: 1 # FAIL: 0 # XPASS: 0 # ERROR: 0 ============================================================================ make[3]: Leaving directory '/home/gtheler/codigos/feenox' make[2]: Leaving directory '/home/gtheler/codigos/feenox' make[1]: Leaving directory '/home/gtheler/codigos/feenox'$

To illustrate how regressions can be detected, let us add a bug deliberately and re-run the test suite.

Edit the source file that contains the shape functions of the second-order tetrahedra src/mesh/tet10.c, find the function feenox_mesh_tet10_h() and randomly change a sign, i.e. replace

      return t*(2*t-1);

with

      return t*(2*t+1);

Save, recompile, and re-run the test suite to obtain some red:

$git diff src/mesh/ diff --git a/src/mesh/tet10.c b/src/mesh/tet10.c index 72bc838..293c290 100644 --- a/src/mesh/tet10.c +++ b/src/mesh/tet10.c @@ -227,7 +227,7 @@ double feenox_mesh_tet10_h(int j, double *vec_r) { return s*(2*s-1); break; case 3: - return t*(2*t-1); + return t*(2*t+1); break; case 4:$ make
[...]
$make check Making check in src make[1]: Entering directory '/home/gtheler/codigos/feenox/src' make[1]: Nothing to be done for 'check'. make[1]: Leaving directory '/home/gtheler/codigos/feenox/src' make[1]: Entering directory '/home/gtheler/codigos/feenox' cp -r src/feenox . make check-TESTS make[2]: Entering directory '/home/gtheler/codigos/feenox' make[3]: Entering directory '/home/gtheler/codigos/feenox' XFAIL: tests/abort.sh PASS: tests/algebraic_expr.sh FAIL: tests/beam-modal.sh PASS: tests/beam-ortho.sh PASS: tests/builtin.sh PASS: tests/cylinder-traction-force.sh PASS: tests/default_argument_value.sh PASS: tests/expressions_constants.sh PASS: tests/expressions_variables.sh PASS: tests/expressions_functions.sh PASS: tests/exp.sh PASS: tests/i-beam-euler-bernoulli.sh PASS: tests/iaea-pwr.sh PASS: tests/iterative.sh PASS: tests/fit.sh PASS: tests/function_algebraic.sh PASS: tests/function_data.sh PASS: tests/function_file.sh PASS: tests/function_vectors.sh PASS: tests/integral.sh PASS: tests/laplace2d.sh PASS: tests/materials.sh PASS: tests/mesh.sh PASS: tests/moment-of-inertia.sh PASS: tests/nafems-le1.sh FAIL: tests/nafems-le10.sh FAIL: tests/nafems-le11.sh PASS: tests/nafems-t1-4.sh PASS: tests/nafems-t2-3.sh PASS: tests/neutron_diffusion_src.sh PASS: tests/neutron_diffusion_keff.sh FAIL: tests/parallelepiped.sh PASS: tests/point-kinetics.sh PASS: tests/print.sh PASS: tests/thermal-1d.sh PASS: tests/thermal-2d.sh PASS: tests/trig.sh PASS: tests/two-cubes-isotropic.sh PASS: tests/two-cubes-orthotropic.sh PASS: tests/vector.sh XFAIL: tests/xfail-few-properties-ortho-young.sh XFAIL: tests/xfail-few-properties-ortho-poisson.sh XFAIL: tests/xfail-few-properties-ortho-shear.sh ============================================================================ Testsuite summary for feenox v0.2.6-g3237ce9 ============================================================================ # TOTAL: 43 # PASS: 35 # SKIP: 0 # XFAIL: 4 # FAIL: 4 # XPASS: 0 # ERROR: 0 ============================================================================ See ./test-suite.log Please report to jeremy@seamplex.com ============================================================================ make[3]: *** [Makefile:1152: test-suite.log] Error 1 make[3]: Leaving directory '/home/gtheler/codigos/feenox' make[2]: *** [Makefile:1260: check-TESTS] Error 2 make[2]: Leaving directory '/home/gtheler/codigos/feenox' make[1]: *** [Makefile:1791: check-am] Error 2 make[1]: Leaving directory '/home/gtheler/codigos/feenox' make: *** [Makefile:1037: check-recursive] Error 1$

### 8.2.7 Installation

To be able to execute FeenoX from any directory, the binary has to be copied to a directory available in the PATH environment variable. If you have root access, the easiest and cleanest way of doing this is by calling make install with sudo or su:

$sudo make install Making install in src make[1]: Entering directory '/home/gtheler/codigos/feenox/src' gmake[2]: Entering directory '/home/gtheler/codigos/feenox/src' /usr/bin/mkdir -p '/usr/local/bin' /usr/bin/install -c feenox '/usr/local/bin' gmake[2]: Nothing to be done for 'install-data-am'. gmake[2]: Leaving directory '/home/gtheler/codigos/feenox/src' make[1]: Leaving directory '/home/gtheler/codigos/feenox/src' make[1]: Entering directory '/home/gtheler/codigos/feenox' cp -r src/feenox . make[2]: Entering directory '/home/gtheler/codigos/feenox' make[2]: Nothing to be done for 'install-exec-am'. make[2]: Nothing to be done for 'install-data-am'. make[2]: Leaving directory '/home/gtheler/codigos/feenox' make[1]: Leaving directory '/home/gtheler/codigos/feenox'$

If you do not have root access or do not want to populate /usr/local/bin, you can either

• Configure with a different prefix (not covered here), or

• Copy (or symlink) the feenox executable to $HOME/bin: mkdir -p${HOME}/bin
cp feenox ${HOME}/bin If you plan to regularly update FeenoX (which you should), you might want to symlink instead of copy so you do not need to update the binary in $HOME/bin each time you recompile:

mkdir -p ${HOME}/bin ln -sf feenox${HOME}/bin

Check that FeenoX is now available from any directory (note the command is feenox and not ./feenox):

$cd$ feenox -v
FeenoX v0.2.14-gbbf48c9-dirty
a free no-fee no-X uniX-like finite-element(ish) computational engineering tool

FeenoX is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ If it is not and you went through the $HOME/bin path, make sure it is in the PATH (pun). Add

export PATH=${PATH}:${HOME}/bin

to your .bashrc in your home directory and re-login.

### 8.3.1 Compiling with debug symbols

By default the C flags are -O3, without debugging. To add the -g flag, just use CFLAGS when configuring:

./configure CFLAGS="-g -O0"

### 8.3.2 Using a different compiler

Without PETSc, FeenoX uses the CC environment variable to set the compiler. So configure like

./configure CC=clang

When PETSc is detected FeenoX uses the mpicc executable, which is a wrapper to an actual C compiler with extra flags needed to find the headers and the MPI library. To change the wrapped compiler, you should set MPICH_CC or OMPI_CC, depending if you are using MPICH or OpenMPI. For example, to force MPICH to use clang do

./configure MPICH_CC=clang CC=clang

To know which is the default MPI implementation, just run ./configure without arguments and pay attention to the “Compiler” line in the “Summary of dependencies” section. For example, for OpenMPI a typical summary would be

## ----------------------- ##
## Summary of dependencies ##
## ----------------------- ##
GNU Scientific Library  from system
SUNDIALS                yes
PETSc                   yes /usr/lib/petsc
SLEPc                   yes /usr/lib/slepc
Compiler                gcc -I/usr/lib/x86_64-linux-gnu/openmpi/include/openmpi -I/usr/lib/x86_64-linux-gnu/openmpi/include -pthread -L/usr/lib/x86_64-linux-gnu/openmpi/lib -lmpi

For MPICH:

## ----------------------- ##
## Summary of dependencies ##
## ----------------------- ##
GNU Scientific Library  from system
SUNDIALS                yes
PETSc                   yes /home/gtheler/libs/petsc-3.15.0 arch-linux2-c-debug
SLEPc                   yes /home/gtheler/libs/slepc-3.15.1
Compiler                gcc -Wl,-z,relro -I/usr/include/x86_64-linux-gnu/mpich -L/usr/lib/x86_64-linux-gnu -lmpich

Other non-free implementations like Intel MPI might work but were not tested. However, it should be noted that the MPI implementation used to compile FeenoX has to match the one used to compile PETSc. Therefore, if you compiled PETSc on your own, it is up to you to ensure MPI compatibility. If you are using PETSc as provided by your distribution’s repositories, you will have to find out which one was used (it is usually OpenMPI) and use the same one when compiling FeenoX.

The FeenoX executable will show the configured compiler and flags when invoked with the --versions option:

$feenox --versions FeenoX v0.2.14-gbbf48c9-dirty a free no-fee no-X uniX-like finite-element(ish) computational engineering tool Last commit date : Sat Feb 12 15:35:05 2022 -0300 Build date : Sat Feb 12 15:35:44 2022 -0300 Build architecture : linux-gnu x86_64 Compiler version : gcc (Debian 10.2.1-6) 10.2.1 20210110 Compiler expansion : gcc -Wl,-z,relro -I/usr/include/x86_64-linux-gnu/mpich -L/usr/lib/x86_64-linux-gnu -lmpich Compiler flags : -O3 Builder : gtheler@tom GSL version : 2.6 SUNDIALS version : 5.7.0 PETSc version : Petsc Release Version 3.16.3, Jan 05, 2022 PETSc arch : arch-linux-c-debug PETSc options : --download-eigen --download-hdf5 --download-hypre --download-metis --download-mumps --download-parmetis --download-pragmatic --download-scalapack SLEPc version : SLEPc Release Version 3.16.1, Nov 17, 2021$

Note that the reported values are the ones used in configure and not in make. Thus, the recommended way to set flags is in configure and not in make.

### 8.3.3 Compiling PETSc

Particular explanation for FeenoX is to be done. For now, follow the general explanation from PETSc’s website.

export PETSC_DIR=$PWD export PETSC_ARCH=arch-linux-c-opt ./configure --with-debugging=0 --download-mumps --download-scalapack --with-cxx=0 --COPTFLAGS=-O3 --FOPTFLAGS=-O3  export PETSC_DIR=$PWD
./configure --with-debugging=0 --with-openmp=0 --with-x=0 --with-cxx=0 --COPTFLAGS=-O3 --FOPTFLAGS=-O3
make PETSC_DIR=/home/ubuntu/reflex-deps/petsc-3.17.2 PETSC_ARCH=arch-linux-c-opt all

# 9 Appendix: Inputs for solving LE10 with other FEA programs

This appendix illustrates the differences in the input file formats used by FeenoX and the ones used by other open source finite-element solvers. The problem being solved is the NAFEMS LE10 benchmark, first discussed in sec. 1.2:

# NAFEMS Benchmark LE-10: thick plate pressure
PROBLEM mechanical DIMENSIONS 3
READ_MESH nafems-le10.msh   # mesh in millimeters

BC upper    p=1      # 1 Mpa

# BOUNDARY CONDITIONS:
BC DCD'C'   v=0      # Face DCD'C' zero y-displacement
BC ABA'B'   u=0      # Face ABA'B' zero x-displacement
BC BCB'C'   u=0 v=0  # Face BCB'C' x and y displ. fixed
BC midplane w=0      #  z displacements fixed along mid-plane

# MATERIAL PROPERTIES: isotropic single-material properties
E = 210e3   # Young modulus in MPa
nu = 0.3    # Poisson's ratio

SOLVE_PROBLEM   # solve!

# print the direct stress y at D (and nothing more)
PRINT "σ_y @ D = " sigmay(2000,0,300) "MPa"

See the following URL and its links for further details about solving this problem with the other codes: https://cofea.readthedocs.io/en/latest/benchmarks/004-eliptic-membrane/tested-codes.html

## 9.1 CalculiX

** Mesh ++++++++++++++++++++++++++++++++++++++++++++++++++++

*INCLUDE, INPUT=Mesh/fine-lin-hex.inp		# Path to mesh for ccx solver

** Mesh ++++++++++++++++++++++++++++++++++++++++++++++++++++

*MATERIAL, NAME=Steel				# Defining a material
*DENSITY
7800						# Defining a density
*ELASTIC,
2.1e11, 0.3					# Defining Young modulus and Poisson's ratio

** Sections ++++++++++++++++++++++++++++++++++++++++++++++++

*SOLID SECTION, ELSET=ELIPSE, MATERIAL=Steel 	# Assigning material and plane stress elements
0.1,						# to the elements sets in mesh and adding thickness

** Steps +++++++++++++++++++++++++++++++++++++++++++++++++++

*STEP						# Begin of analysis
*STATIC, SOLVER=SPOOLES				# Selection of elastic analysis

** Field outputs +++++++++++++++++++++++++++++++++++++++++++

*EL FILE					# Commands responsible for saving results
E, S
*NODE FILE
U

** Boundary conditions +++++++++++++++++++++++++++++++++++++

*BOUNDARY,					# Applying translation = 0 on desired nodes
AB,1,1,0
*BOUNDARY
CD,2,2,0

*INCLUDE, INPUT=Pressure/fine-lin-hex.dlo

** End step ++++++++++++++++++++++++++++++++++++++++++++++++

*END STEP					# End on analysis



## 9.2 Code Aster

mesh = LIRE_MAILLAGE(identifier='0:1',				# Reading a mesh
FORMAT='IDEAS',
UNITE=80)

model = AFFE_MODELE(identifier='1:1',				# Assignig plane stress
AFFE=_F(MODELISATION=('C_PLAN', ),		# elements to mesh
PHENOMENE='MECANIQUE',
TOUT='OUI'),
MAILLAGE=mesh)

mater = DEFI_MATERIAU(identifier='2:1',				# Defining elastic material
ELAS=_F(E=210000000000.0,
NU=0.3))

materfl = AFFE_MATERIAU(identifier='3:1',			# Assigning material to model
AFFE=_F(MATER=(mater, ),
TOUT='OUI'),
MODELE=model)

mecabc = AFFE_CHAR_MECA(identifier='4:1',			# Applying boundary conditions
DDL_IMPO=(_F(DX=0.0,			# displacement = 0
GROUP_MA=('AB', )),	# to the selected group of elements
_F(DY=0.0,
GROUP_MA=('CD', ))),
MODELE=model)

mecach = AFFE_CHAR_MECA(identifier='5:1',			# Applying pressure to the
MODELE=model,				# group of elements
PRES_REP=_F(GROUP_MA=('BC', ),
PRES=-10000000.0))

result = MECA_STATIQUE(identifier='6:1',			# Defining the results of
CHAM_MATER=materfl,			# simulation
EXCIT=(_F(CHARGE=mecabc),
_F(CHARGE=mecach)),
MODELE=model)

SYY = CALC_CHAMP(identifier='7:1',				# Calculating stresses in
CHAM_MATER=materfl,				# computed domain
CONTRAINTE=('SIGM_NOEU', ),
MODELE=model,
RESULTAT=result)

IMPR_RESU(identifier='8:1',					# Saving the results
FORMAT='MED',
RESU=(_F(RESULTAT=result),
_F(RESULTAT=SYY)),
UNITE=80)

FIN()


## 9.3 Elmer

Header
CHECK KEYWORDS Warn
Mesh DB "." "."				# Path to the mesh
Include Path ""
Results Directory ""				# Path to results directory
End

Simulation					# Settings and constants for simulation
Max Output Level = 5
Coordinate System = Cartesian
Coordinate Mapping(3) = 1 2 3
Steady State Max Iterations = 1
Output Intervals = 1
Timestepping Method = BDF
BDF Order = 1
Solver Input File = case.sif
Post File = case.vtu
End

Constants
Gravity(4) = 0 -1 0 9.82
Stefan Boltzmann = 5.67e-08
Permittivity of Vacuum = 8.8542e-12
Boltzmann Constant = 1.3807e-23
Unit Charge = 1.602e-19
End

Body 1						# Assigning the material and equations to the mesh
Target Bodies(1) = 10
Name = "Body Property 1"
Equation = 1
Material = 1
End

Solver 2					# Solver settings
Equation = Linear elasticity
Procedure = "StressSolve" "StressSolver"
Calculate Stresses = True
Variable = -dofs 2 Displacement
Exec Solver = Always
Stabilize = True
Bubbles = False
Lumped Mass Matrix = False
Optimize Bandwidth = True
Steady State Convergence Tolerance = 1.0e-5
Nonlinear System Convergence Tolerance = 1.0e-7
Nonlinear System Max Iterations = 20
Nonlinear System Newton After Iterations = 3
Nonlinear System Newton After Tolerance = 1.0e-3
Nonlinear System Relaxation Factor = 1
Linear System Solver = Direct
Linear System Direct Method = Umfpack
End

Solver 1					# Saving the results from node at point D
Equation = SaveScalars
Save Points = 26
Procedure = "SaveData" "SaveScalars"
Filename = file.dat
Exec Solver = After Simulation
End

Equation 1					# Setting active solvers
Name = "STRESS"
Calculate Stresses = True
Plane Stress = True				# Turning on plane stress simulation
Active Solvers(1) = 2
End

Equation 2
Name = "DATA"
Active Solvers(1) = 1
End

Material 1					# Defining the material
Name = "STEEL"
Poisson ratio = 0.3
Porosity Model = Always saturated
Youngs modulus = 2.1e11
End

Boundary Condition 1				# Applying the boundary conditions
Target Boundaries(1) = 12
Name = "AB"
Displacement 1 = 0
End

Boundary Condition 2
Target Boundaries(1) = 13
Name = "CD"
Displacement 2 = 0
End

Boundary Condition 3
Target Boundaries(1) = 14
Name = "BC"
Normal Force = 10e6
End


1. Here “Markdown” means (Pandoc + Git + Github / Gitlab / Gitea)↩︎

2. Here “FeenoX” means (FeenoX + Gmsh + Paraview + Git + Github / Gitlab / Gitea)↩︎

3. There are some examples of pieces of computational software which are described as “open source” in which even the first of the four freedoms is denied. The most iconic case is that of Android, whose sources are readily available online but there is no straightforward way of updating one’s mobile phone firmware with a customized version, not to mention vendor and hardware lock ins and the possibility of bricking devices if something unexpected happens. In the nuclear industry, it is the case of a Monte Carlo particle-transport program that requests users to sign an agreement about the objective of its usage before allowing its execution. The software itself might be open source because the source code is provided after signing the agreement, but it is not free (as in freedom) at all.↩︎

4. This experience also shaped many of the features that FeenoX has and most of the features is does deliberately not have.↩︎

5. Even better, these authors should ask to merge their contributions into FeenoX’ main code base.↩︎

6. Analog and quantum computers are out of the scope.↩︎

7. The word dirty means the current Git worktree used to compile the binary had some changes that were not commited yet.↩︎