From f5511c2a5f638e090a948aef98dcdb6e328b650b Mon Sep 17 00:00:00 2001 From: Axel Huebl Date: Mon, 19 Aug 2024 11:12:03 -0700 Subject: [PATCH] Doc: Reorganize Sections for Collective Effects (#684) * Doc: Reorganize Sections for Collective Effects Reorganize input option sections for collective effects. * Fix grammar. Co-authored-by: Chad Mitchell <46825199+cemitch99@users.noreply.github.com> --------- Co-authored-by: Chad Mitchell <46825199+cemitch99@users.noreply.github.com> --- docs/source/usage/parameters.rst | 381 +++++++++++++++---------------- docs/source/usage/python.rst | 19 +- 2 files changed, 197 insertions(+), 203 deletions(-) diff --git a/docs/source/usage/parameters.rst b/docs/source/usage/parameters.rst index 7438377ba..e222110c6 100644 --- a/docs/source/usage/parameters.rst +++ b/docs/source/usage/parameters.rst @@ -3,122 +3,7 @@ Parameters: Inputs File ======================= -This documents on how to use ImpactX with an inputs file (``impactx input_file.in``). - -.. note:: - The AMReX parser (see :ref:`running-cpp-parameters-parser`) is used for the right-hand-side of all input parameters that consist of one or more integers or floats, so expressions like ``.density_max = "2.+1."`` and/or using user-defined constants are accepted. - -.. _running-cpp-parameters-overall: - -Overall simulation parameters ------------------------------ - -* ``max_step`` (``integer``) - The number of PIC cycles to perform. - -* ``stop_time`` (``float``; in seconds) - The maximum physical time of the simulation. Can be provided instead of ``max_step``. If both - ``max_step`` and ``stop_time`` are provided, both criteria are used and the simulation stops - when the first criterion is hit. - -* ``amrex.abort_on_out_of_gpu_memory`` (``0`` or ``1``; default is ``1`` for true) - When running on GPUs, memory that does not fit on the device will be automatically swapped to host memory when this option is set to ``0``. - This will cause severe performance drops. - Note that even with this set to ``1`` ImpactX will not catch all out-of-memory events yet when operating close to maximum device memory. - `Please also see the documentation in AMReX `__. - -* ``amrex.the_arena_is_managed`` (``0`` or ``1``; default is ``0`` for false) - When running on GPUs, device memory that is accessed from the host will automatically be transferred with managed memory. - This is useful for convenience during development, but has sometimes severe performance and memory footprint implications if relied on (and sometimes vendor bugs). - For all regular ImpactX operations, we therefore do explicit memory transfers without the need for managed memory and thus changed the AMReX default to false. - `Please also see the documentation in AMReX `__. - -* ``amrex.omp_threads`` (``system``, ``nosmt`` or positive integer; default is ``nosmt``) - An integer number can be set in lieu of the ``OMP_NUM_THREADS`` environment variable to control the number of OpenMP threads to use for the ``OMP`` compute backend on CPUs. - By default, we use the ``nosmt`` option, which overwrites the OpenMP default of spawning one thread per logical CPU core, and instead only spawns a number of threads equal to the number of physical CPU cores on the machine. - If set, the environment variable ``OMP_NUM_THREADS`` takes precedence over ``system`` and ``nosmt``, but not over integer numbers set in this option. - -* ``amrex.abort_on_unused_inputs`` (``0`` or ``1``; default is ``0`` for false) - When set to ``1``, this option causes the simulation to fail *after* its completion if there were unused parameters. - It is mainly intended for continuous integration and automated testing to check that all tests and inputs are adapted to API changes. - -* ``impactx.always_warn_immediately`` (``0`` or ``1``; default is ``0`` for false) - If set to ``1``, ImpactX immediately prints every warning message as soon as it is generated. - It is mainly intended for debug purposes, in case a simulation crashes before a global warning report can be printed. - -* ``impactx.abort_on_warning_threshold`` (string: ``low``, ``medium`` or ``high``) optional - Optional threshold to abort as soon as a warning is raised. - If the threshold is set, warning messages with priority greater than or equal to the threshold trigger an immediate abort. - It is mainly intended for debug purposes, and is best used with ``impactx.always_warn_immediately=1``. - For more information on the warning logger, see `this section `__ of the WarpX documentation. - -* ``impactx.verbose`` (int: ``0`` for silent, higher is more verbose; default is ``1``) optional - Controls how much information is printed to the terminal, when running ImpactX. - - -.. _running-cpp-parameters-box: - - -Setting up the field mesh -------------------------- - -ImpactX uses an AMReX grid of boxes to organize and parallelize the simulation domain. -These boxes also contain a field mesh, if space charge calculations are enabled. - -* ``amr.n_cell`` (3 integers) optional (default: 1 `blocking_factor `__ per MPI process) - The number of grid points along each direction (on the **coarsest level**) - -* ``amr.max_level`` (``integer``, default: ``0``) - When using mesh refinement, the number of refinement levels that will be used. - - Use ``0`` in order to disable mesh refinement. - -* ``amr.ref_ratio`` (``integer`` per refined level, default: ``2``) - When using mesh refinement, this is the refinement ratio per level. - With this option, all directions are fined by the same ratio. - -* ``amr.ref_ratio_vect`` (3 integers for x,y,z per refined level) - When using mesh refinement, this can be used to set the refinement ratio per direction and level, relative to the previous level. - - Example: for three levels, a value of ``2 2 4 8 8 16`` refines the first level by 2-fold in x and y and 4-fold in z compared to the coarsest level (level 0/mother grid); compared to the first level, the second level is refined 8-fold in x and y and 16-fold in z. - -.. note:: - - Field boundaries for space charge calculation are located at the outer ends of the field mesh. - We currently assume `Dirichlet boundary conditions `__ with zero potential (a mirror charge). - Thus, to emulate open boundaries, consider adding enough vacuum padding to the beam. - This will be improved in future versions. - -.. note:: - - Particles that move outside the simulation domain are removed. - -* ``geometry.dynamic_size`` (``boolean``) optional (default: ``true`` for dynamic) - Use dynamic (``true``) resizing of the field mesh, via ``geometry.prob_relative``, or static sizing (``false``), via ``geometry.prob_lo``/``geometry.prob_hi``. - -* ``geometry.prob_relative`` (positive ``float`` array with ``amr.max_level`` entries, unitless) optional (default: ``3.0 1.0 1.0 ...``) - By default, we dynamically extract the minimum and maximum of the particle positions in the beam. - The field mesh spans, per direction, multiple times the maximum physical extent of beam particles, as given by this factor. - The beam minimum and maximum extent are symmetrically padded by the mesh. - For instance, ``1.2`` means the mesh will span 10% above and 10% below the beam; - ``1.0`` means the beam is exactly covered with the mesh. - -* ``geometry.prob_lo`` and ``geometry.prob_hi`` (3 floats, in meters) optional (required if ``geometry.dynamic_size`` is ``false``) - The extent of the full simulation domain relative to the reference particle position. - This can be used to explicitly size the simulation box and ignore ``geometry.prob_relative``. - - This box is rectangular, and thus its extent is given here by the coordinates of the lower corner (``geometry.prob_lo``) and upper corner (``geometry.prob_hi``). - The first axis of the coordinates is x and the last is z. - - -.. _running-cpp-parameters-bc: - -Domain Boundary Conditions --------------------------- - -.. note:: - - TODO :-) +This documents how to use ImpactX with an input file (``impactx input_file.in``). .. _running-cpp-parameters-particle: @@ -592,25 +477,122 @@ Lattice Elements Note: If ``reverse`` and ``repeat`` both appear, then ``reverse`` is applied before ``repeat``. -.. _running-cpp-parameters-parallelization: +.. _running-cpp-parameters-collective: -Distribution across MPI ranks and parallelization -------------------------------------------------- +Collective Effects +------------------ -* ``amr.max_grid_size`` (``integer``) optional (default: ``128``) - Maximum allowable size of each **subdomain** - (expressed in number of grid points, in each direction). - Each subdomain has its own ghost cells, and can be handled by a - different MPI rank ; several OpenMP threads can work simultaneously on the - same subdomain. +.. _running-cpp-parameters-collective-spacecharge: - If ``max_grid_size`` is such that the total number of subdomains is - **larger** that the number of MPI ranks used, than some MPI ranks - will handle several subdomains, thereby providing additional flexibility - for **load balancing**. +Space Charge +^^^^^^^^^^^^ - When using mesh refinement, this number applies to the subdomains - of the coarsest level, but also to any of the finer level. +Space charge kicks are applied in between slices of thick :ref:`lattice elements `. +See there ``nslice`` option on lattice elements for slicing. + +* ``algo.space_charge`` (``boolean``, optional, default: ``false``) + Whether to calculate space charge effects. + +ImpactX uses an AMReX grid of boxes to organize and parallelize space charge simulation domain. +These boxes also contain a field mesh, if space charge calculations are enabled. + +* ``amr.n_cell`` (3 integers) optional (default: 1 `blocking_factor `__ per MPI process) + The number of grid points along each direction (on the **coarsest level**) + +* ``amr.max_level`` (``integer``, default: ``0``) + When using mesh refinement, the number of refinement levels that will be used. + + Use ``0`` in order to disable mesh refinement. + +* ``amr.ref_ratio`` (``integer`` per refined level, default: ``2``) + When using mesh refinement, this is the refinement ratio per level. + With this option, all directions are fined by the same ratio. + +* ``amr.ref_ratio_vect`` (3 integers for x,y,z per refined level) + When using mesh refinement, this can be used to set the refinement ratio per direction and level, relative to the previous level. + + Example: for three levels, a value of ``2 2 4 8 8 16`` refines the first level by 2-fold in x and y and 4-fold in z compared to the coarsest level (level 0/mother grid); compared to the first level, the second level is refined 8-fold in x and y and 16-fold in z. + +.. note:: + + Particles that move outside the simulation domain are removed. + +* ``geometry.dynamic_size`` (``boolean``) optional (default: ``true`` for dynamic) + Use dynamic (``true``) resizing of the field mesh, via ``geometry.prob_relative``, or static sizing (``false``), via ``geometry.prob_lo``/``geometry.prob_hi``. + +* ``geometry.prob_relative`` (positive ``float`` array with ``amr.max_level`` entries, unitless) optional (default: ``3.0 1.0 1.0 ...``) + By default, we dynamically extract the minimum and maximum of the particle positions in the beam. + The field mesh spans, per direction, multiple times the maximum physical extent of beam particles, as given by this factor. + The beam minimum and maximum extent are symmetrically padded by the mesh. + For instance, ``1.2`` means the mesh will span 10% above and 10% below the beam; + ``1.0`` means the beam is exactly covered with the mesh. + +* ``geometry.prob_lo`` and ``geometry.prob_hi`` (3 floats, in meters) optional (required if ``geometry.dynamic_size`` is ``false``) + The extent of the full simulation domain relative to the reference particle position. + This can be used to explicitly size the simulation box and ignore ``geometry.prob_relative``. + + This box is rectangular, and thus its extent is given here by the coordinates of the lower corner (``geometry.prob_lo``) and upper corner (``geometry.prob_hi``). + The first axis of the coordinates is x and the last is z. + +* ``algo.particle_shape`` (``integer``; ``1``, ``2``, or ``3``) + The order of the shape factors (splines) for the macro-particles along all spatial directions: `1` for linear, `2` for quadratic, `3` for cubic. + Low-order shape factors result in faster simulations, but may lead to more noisy results. + High-order shape factors are computationally more expensive, but may increase the overall accuracy of the results. + For production runs it is generally safer to use high-order shape factors, such as cubic order. + +* ``algo.poisson_solver`` (``string``, optional, default: ``"multigrid"``) + The numerical solver to solve the Poisson equation when calculating space charge effects. + Currently, this is a 3D solver. + An additional `2D/2.5D solver `__ will be added in the near future. + + Options: + + * ``fft``: Poisson's equation is solved using an Integrated Green Function method (which requires FFT calculations). + See these references for more details `Qiang et al. (2006) `__ (+ `Erratum `__). + This requires the compilation flag ``-DImpactX_FFT=ON``. + If mesh refinement (MR) is enabled, this FFT solver is used only on the coarsest level and a multi-grid solver is used on refined levels. + The boundary conditions are assumed to be open. + + * ``multigrid``: Poisson's equation is solved using an iterative multigrid (MLMG) solver. + See the `AMReX documentation `__ for details of the MLMG solver. + Field boundaries for MLMG space charge calculation are located at the outer ends of the field mesh. + For the MLMG solver, we assume `Dirichlet boundary conditions `__ with zero potential (a mirror charge). + Thus, to emulate open boundaries, consider adding enough vacuum padding to the beam. + +Multigrid-specific numerical options: + +* ``algo.mlmg_relative_tolerance`` (``float``, optional, default: ``1.e-7``) + The relative precision with which the electrostatic space-charge fields should be calculated. + More specifically, the space-charge fields are computed with an iterative Multi-Level Multi-Grid (MLMG) solver. + This solver can fail to reach the default precision within a reasonable time. + +* ``algo.mlmg_absolute_tolerance`` (``float``, optional, default: ``0``, which means: ignored) + The absolute tolerance with which the space-charge fields should be calculated in units of V/m^2. + More specifically, the acceptable residual with which the solution can be considered converged. + In general this should be left as the default, but in cases where the simulation state changes very + little between steps it can occur that the initial guess for the MLMG solver is so close to the + converged value that it fails to improve that solution sufficiently to reach the + mlmg_relative_tolerance value." + +* ``algo.mlmg_max_iters`` (``integer``, optional, default: ``100``) + Maximum number of iterations used for MLMG solver for space-charge fields calculation. + In case if MLMG converges but fails to reach the desired self_fields_required_precision, + this parameter may be increased. + +* ``algo.mlmg_verbosity`` (``integer``, optional, default: ``1``) + The verbosity used for MLMG solver for space-charge fields calculation. + Currently MLMG solver looks for verbosity levels from 0-5. + A higher number results in more verbose output. + + +.. _running-cpp-parameters-collective-csr: + +Coherent Synchrotron Radiation (CSR) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +.. note:: + + Coming soon, see `PR #627 `__. .. _running-cpp-parameters-parser: @@ -618,12 +600,13 @@ Distribution across MPI ranks and parallelization Math parser and user-defined constants -------------------------------------- -ImpactX uses AMReX's math parser that reads expressions in the input file. -It can be used in all input parameters that consist of one or more integers or floats. -Integer input expecting boolean, 0 or 1, are not parsed. +The AMReX parser is used for the right-hand-side of all input parameters that consist of one or more integers or floats. +Thus, expressions like ``beam.alphaY = beam.alphaX`` and/or using user-defined constants or simple math operations are accepted. + Note that when multiple values are expected, the expressions are space delimited. For integer input values, the expressions are evaluated as real numbers and the final result rounded to the nearest integer. -See `this section `_ of the AMReX documentation for a complete list of functions supported by the math parser. +See `this section `__ of the AMReX documentation for a complete list of functions supported by the math parser. + ImpactX constants ^^^^^^^^^^^^^^^^^ @@ -632,8 +615,6 @@ ImpactX will provide a few pre-defined constants, that can be used for any param .. note:: - Develop, such as: - ======== =================== q_e elementary charge m_e electron mass @@ -645,7 +626,6 @@ ImpactX will provide a few pre-defined constants, that can be used for any param pi math constant pi ======== =================== - See in WarpX the file ``Source/Utils/WarpXConst.H`` for the values. User-defined constants ^^^^^^^^^^^^^^^^^^^^^^ @@ -658,10 +638,10 @@ by ImpactX, and cannot be used as user-defined constants: ``x``, ``y``, ``z``, ` The values of the constants can include the predefined ImpactX constants listed above as well as other user-defined constants. For example: -* ``my_constants.a0 = 3.0`` -* ``my_constants.z_plateau = 150.e-6`` -* ``my_constants.n0 = 1.e22`` -* ``my_constants.wp = sqrt(n0*q_e**2/(epsilon0*m_e))`` +* ``my_constants.my_alpha = 3.0`` +* ``my_constants.my_beta = 12.e-6`` +* ``my_constants.abc = 1.23e10`` + Coordinates ^^^^^^^^^^^ @@ -677,57 +657,6 @@ define functions by intervals. Alternatively the expression above can be written as ``if(x>0, a0*x**2 * (1-y*1.e2), 0)``. -.. _running-cpp-parameters-numerics: - -Numerics and algorithms ------------------------ - -* ``algo.particle_shape`` (``integer``; ``1``, ``2``, or ``3``) - The order of the shape factors (splines) for the macro-particles along all spatial directions: `1` for linear, `2` for quadratic, `3` for cubic. - Low-order shape factors result in faster simulations, but may lead to more noisy results. - High-order shape factors are computationally more expensive, but may increase the overall accuracy of the results. - For production runs it is generally safer to use high-order shape factors, such as cubic order. - -* ``algo.space_charge`` (``boolean``, optional, default: ``false``) - Whether to calculate space charge effects. - -* ``algo.poisson_solver`` (``string``, optional, default: ``"multigrid"``) - The numerical solver to solve the Poisson equation when calculating space charge effects. - Options: - - * ``multigrid``: Poisson's equation is solved using an iterative multigrid (MLMG) solver. - See the `AMReX documentation `__ for details of the MLMG solver. - - * ``fft``: Poisson's equation is solved using an Integrated Green Function method (which requires FFT calculations). - See these references for more details `Qiang et al. (2006) `__ (+ `Erratum `__). - It only works in 3D and it requires the compilation flag ``-DImpactX_FFT=ON``. - If mesh refinement is enabled, this solver only works on the coarsest level. - On the refined patches, the Poisson equation is solved with the multigrid solver. - The boundary conditions are assumed to be open. - -* ``algo.mlmg_relative_tolerance`` (``float``, optional, default: ``1.e-7``) - The relative precision with which the electrostatic space-charge fields should be calculated. - More specifically, the space-charge fields are computed with an iterative Multi-Level Multi-Grid (MLMG) solver. - This solver can fail to reach the default precision within a reasonable time. - -* ``algo.mlmg_absolute_tolerance`` (``float``, optional, default: ``0``, which means: ignored) - The absolute tolerance with which the space-charge fields should be calculated in units of V/m^2. - More specifically, the acceptable residual with which the solution can be considered converged. - In general this should be left as the default, but in cases where the simulation state changes very - little between steps it can occur that the initial guess for the MLMG solver is so close to the - converged value that it fails to improve that solution sufficiently to reach the - mlmg_relative_tolerance value." - -* ``algo.mlmg_max_iters`` (``integer``, optional, default: ``100``) - Maximum number of iterations used for MLMG solver for space-charge fields calculation. - In case if MLMG converges but fails to reach the desired self_fields_required_precision, - this parameter may be increased. - -* ``algo.mlmg_verbosity`` (``integer``, optional, default: ``1``) - The verbosity used for MLMG solver for space-charge fields calculation. - Currently MLMG solver looks for verbosity levels from 0-5. - A higher number results in more verbose output. - .. _running-cpp-parameters-diagnostics: Diagnostics and output @@ -767,6 +696,7 @@ In-situ visualization TODO :-) + .. _running-cpp-parameters-cp-restart: Checkpoints and restart @@ -774,13 +704,15 @@ Checkpoints and restart .. note:: - ImpactX will support checkpoints/restart via AMReX. + Future version of ImpactX will support checkpoints/restart via AMReX. + This is not yet implemented. The checkpoint capability can be turned with regular diagnostics: ``.format = checkpoint``. * ``amr.restart`` (`string`) Name of the checkpoint file to restart from. Returns an error if the folder does not exist or if it is not properly formatted. + Intervals parser ---------------- @@ -832,3 +764,64 @@ This is essentially the python slicing syntax except that the stop is inclusive Note that if a given period is zero or negative, the corresponding slice is disregarded. For example, ``something_intervals = -1`` deactivates ``something`` and ``something_intervals = ::-1,100:1000:25`` is equivalent to ``something_intervals = 100:1000:25``. + + +.. _running-cpp-parameters-overall: + +Overall simulation parameters +----------------------------- + +* ``amrex.abort_on_out_of_gpu_memory`` (``0`` or ``1``; default is ``1`` for true) + When running on GPUs, memory that does not fit on the device will be automatically swapped to host memory when this option is set to ``0``. + This will cause severe performance drops. + Note that even with this set to ``1`` ImpactX will not catch all out-of-memory events yet when operating close to maximum device memory. + `Please also see the documentation in AMReX `__. + +* ``amrex.the_arena_is_managed`` (``0`` or ``1``; default is ``0`` for false) + When running on GPUs, device memory that is accessed from the host will automatically be transferred with managed memory. + This is useful for convenience during development, but has sometimes severe performance and memory footprint implications if relied on (and sometimes vendor bugs). + For all regular ImpactX operations, we therefore do explicit memory transfers without the need for managed memory and thus changed the AMReX default to false. + `Please also see the documentation in AMReX `__. + +* ``amrex.omp_threads`` (``system``, ``nosmt`` or positive integer; default is ``nosmt``) + An integer number can be set in lieu of the ``OMP_NUM_THREADS`` environment variable to control the number of OpenMP threads to use for the ``OMP`` compute backend on CPUs. + By default, we use the ``nosmt`` option, which overwrites the OpenMP default of spawning one thread per logical CPU core, and instead only spawns a number of threads equal to the number of physical CPU cores on the machine. + If set, the environment variable ``OMP_NUM_THREADS`` takes precedence over ``system`` and ``nosmt``, but not over integer numbers set in this option. + +* ``amrex.abort_on_unused_inputs`` (``0`` or ``1``; default is ``0`` for false) + When set to ``1``, this option causes the simulation to fail *after* its completion if there were unused parameters. + It is mainly intended for continuous integration and automated testing to check that all tests and inputs are adapted to API changes. + +* ``impactx.always_warn_immediately`` (``0`` or ``1``; default is ``0`` for false) + If set to ``1``, ImpactX immediately prints every warning message as soon as it is generated. + It is mainly intended for debug purposes, in case a simulation crashes before a global warning report can be printed. + +* ``impactx.abort_on_warning_threshold`` (string: ``low``, ``medium`` or ``high``) optional + Optional threshold to abort as soon as a warning is raised. + If the threshold is set, warning messages with priority greater than or equal to the threshold trigger an immediate abort. + It is mainly intended for debug purposes, and is best used with ``impactx.always_warn_immediately=1``. + For more information on the warning logger, see `this section `__ of the WarpX documentation. + +* ``impactx.verbose`` (int: ``0`` for silent, higher is more verbose; default is ``1``) optional + Controls how much information is printed to the terminal, when running ImpactX. + + +.. _running-cpp-parameters-parallelization: + +Distribution across MPI ranks and parallelization +------------------------------------------------- + +* ``amr.max_grid_size`` (``integer``) optional (default: ``128``) + Maximum allowable size of each **subdomain** + (expressed in number of grid points, in each direction). + Each subdomain has its own ghost cells, and can be handled by a + different MPI rank ; several OpenMP threads can work simultaneously on the + same subdomain. + + If ``max_grid_size`` is such that the total number of subdomains is + **larger** that the number of MPI ranks used, than some MPI ranks + will handle several subdomains, thereby providing additional flexibility + for **load balancing**. + + When using mesh refinement, this number applies to the subdomains + of the coarsest level, but also to any of the finer level. diff --git a/docs/source/usage/python.rst b/docs/source/usage/python.rst index d45dd7c47..a47f23bf5 100644 --- a/docs/source/usage/python.rst +++ b/docs/source/usage/python.rst @@ -5,8 +5,8 @@ Parameters: Python This documents on how to use ImpactX as a Python script (``python3 run_script.py``). -General -------- +Collective Effects & Overall Simulation Parameters +-------------------------------------------------- .. py:class:: impactx.ImpactX @@ -71,16 +71,18 @@ General The numerical solver to solve the Poisson equation when calculating space charge effects. Either ``"multigrid"`` (default) or ``"fft"``. - * ``multigrid``: Poisson's equation is solved using an iterative multigrid (MLMG) solver. - See the `AMReX documentation `__ for details of the MLMG solver. + Currently, this is a 3D solver. + An additional `2D/2.5D solver `__ will be added in the near future. * ``fft``: Poisson's equation is solved using an Integrated Green Function method (which requires FFT calculations). See these references for more details `Qiang et al. (2006) `__ (+ `Erratum `__). - It only works in 3D and it requires the compilation flag ``-DImpactX_FFT=ON``. - If mesh refinement is enabled, this solver only works on the coarsest level. - On the refined patches, the Poisson equation is solved with the multigrid solver. + This requires the compilation flag ``-DImpactX_FFT=ON``. + If mesh refinement (MR) is enabled, this FFT solver is used only on the coarsest level and a multi-grid solver is used on refined levels. The boundary conditions are assumed to be open. + * ``multigrid``: Poisson's equation is solved using an iterative multigrid (MLMG) solver. + See the `AMReX documentation `__ for details of the MLMG solver. + .. py:property:: mlmg_relative_tolerance Default: ``1.e-7`` @@ -166,7 +168,6 @@ General The number of periods to repeat the lattice. - .. py:property:: abort_on_warning_threshold (optional) Set to "low", "medium" or "high". @@ -229,7 +230,7 @@ General .. warning:: - By default, OpenMP spawns as many threads as there are available virtual cores on a host. + By default, OpenMP spawns as many threads as there are available physical CPU cores on a host. When MPI and OpenMP support are used at the same time, it can easily happen that one over-subscribes the available physical CPU cores. This will lead to a severe slow-down of the simulation.