This shows you the differences between two versions of the page.
Both sides previous revision Previous revision | Next revision Both sides next revision | ||
software:specfem [2018/03/05 15:04] wphase |
software:specfem [2018/03/05 16:04] wphase |
||
---|---|---|---|
Line 286: | Line 286: | ||
==== Running multiple SPECFEM3D_GLOBE jobs in parallel ==== | ==== Running multiple SPECFEM3D_GLOBE jobs in parallel ==== | ||
- | To launch batch of SEM simulations: | + | Some instructions to use custom scripts enabling parallel SEM simulations on the HPC cluster |
+ | |||
+ | === Preparing the input files === | ||
First, create an event list "Events.txt" with 3 collumns: | First, create an event list "Events.txt" with 3 collumns: | ||
Line 293: | Line 296: | ||
* 3nd column: path to ''STATION'' file for this event (can be the same for all events) | * 3nd column: path to ''STATION'' file for this event (can be the same for all events) | ||
- | Then you must setup a ''Par_file'' as described above (be careful to use a version of ''Par_file'' that is compatible with your SEM version) | + | Then you must setup a ''Par_file'' (be careful to use a version of ''Par_file'' that is compatible with your SEM version) |
+ | |||
+ | Finally, you must setup hostfiles named ''nodelistN'' files where N=0,...,Np-1 (Np, the number of parallel SEM simulations). These files must specify host names and number of slots per node. Here is an example: | ||
+ | <code> | ||
+ | $ cat nodelist0 | ||
+ | hpc-n443 slots=8 | ||
+ | hpc-n444 slots=8 | ||
+ | hpc-n445 slots=8 | ||
+ | </code> | ||
+ | (see ''/b/home/eost/zac/jobs/specfem/parallelSEM/nodelist0'') | ||
+ | |||
+ | === Running the simulations in parallel === | ||
- | The SPECFEM runs will be handled using 3 scripts: | + | Parallel SEM simulations are handled using 3 scripts: |
- | * ''parallelSEM.sh'': is the main script | + | * ''parallelSEM.sh'': is the main script, that compiles the code and run the simulations |
* ''run_gpu_nodelist.sh'' is the script used to run the mesher and solver | * ''run_gpu_nodelist.sh'' is the script used to run the mesher and solver | ||
* ''sleep.slurm'' is a script to reserve the GPU nodes | * ''sleep.slurm'' is a script to reserve the GPU nodes | ||
All these scripts are available in ''/b/home/eost/zac/jobs/specfem/parallelSEM'' | All these scripts are available in ''/b/home/eost/zac/jobs/specfem/parallelSEM'' | ||
- | Before running it, make sure the input parameters in ''parallelSEM.sh'' are consistent with the input parameters stated above (see ''INPUT PARAMS'' in the main script). Then run: | + | Before running your job, make sure that the input parameters in ''parallelSEM.sh'' are consistent with the input parameters stated above (see ''INPUT PARAMS'' in the main script). Specifically: |
+ | * ''SPECFEMDIR'': path to SPECFEM3D_GLOBE directory | ||
+ | * ''Par_file'': path to the Par_file used in simulations | ||
+ | * ''Nparallel'': Number of SEM simulations in parallel (make sure enough GPUs are available) | ||
+ | * ''event_list'': List of events with the format given above | ||
+ | |||
+ | Then run your simulations: | ||
<code> | <code> | ||
./parallelSEM.sh | ./parallelSEM.sh |