Function t8_cmesh_get_first_element_of_process

Function Documentation

constexpr uint64_t t8_cmesh_get_first_element_of_process(const uint32_t process, const uint32_t mpisize, const uint64_t global_num_elements)

Compute the first element of a process in a partitioned mesh, via floor(process * global_num_elements / mpisize). Prevents overflowing by using division with remainder to store partial results in uint64_t. We also take into account that process <= mpisize. Both process and global_num_elements can be written as:

global_num_elements = elem_over_size * mpisize + remainder_0 process = proc_over_size * mpisize + remainder_1 with: elem_over_size = global_num_elements / mpisize proc_over_size = process / mpisize and remainders: remainder_0 = global_num_elements % mpisize remainder_1 = process % mpisize

Putting this together in the computation of first_element = global_num_elements * process / mpisize gives:

= elem_over_size * proc_over_size * mpisize + elem_over_size * process % mpisize

  • proc_over_size * global_num_elements % mpisize + (remainder_0 * remainder_1) / mpisize

Each variable is less than 2^32-1 taking into account that process <= mpisize we can assure that the first summand does not overflow

This enables us to partition 2^64-1 elements over 2^32-1 processes. Update this function if we have supercomputers with more than 2^32-1 processes, or need larger meshes.

Warning

This function assumes that process <= mpisize. mpisize has to be greater than 0. Otherwise the result of process * global_num_elements / mpisize can exceed the range of uint64_t.

Parameters:

  • process – [in] The number of processes

  • mpisize – [in] The size of the MPI communicator

  • global_num_elements – [in] The number of elements in the global mesh

Returns:

The first element of the process in the partitioned mesh