User Tools

Site Tools


software:specfem

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision Both sides next revision
software:specfem [2018/03/05 15:04]
wphase
software:specfem [2018/03/05 16:04]
wphase
Line 286: Line 286:
 ==== Running multiple SPECFEM3D_GLOBE jobs in parallel ==== ==== Running multiple SPECFEM3D_GLOBE jobs in parallel ====
  
-To launch batch of SEM simulations:+Some instructions to use custom scripts enabling parallel ​SEM simulations ​on the HPC cluster 
 + 
 +=== Preparing the input files === 
  
 First, create an event list "​Events.txt"​ with 3 collumns: First, create an event list "​Events.txt"​ with 3 collumns:
Line 293: Line 296:
   * 3nd column: path to ''​STATION''​ file for this event (can be the same for all events)   * 3nd column: path to ''​STATION''​ file for this event (can be the same for all events)
  
-Then you must setup a ''​Par_file'' ​as described above (be careful to use a version of ''​Par_file''​ that is compatible with your SEM version)+Then you must setup a ''​Par_file''​ (be careful to use a version of ''​Par_file''​ that is compatible with your SEM version) 
 + 
 +Finally, you must setup hostfiles named ''​nodelistN''​ files where N=0,​...,​Np-1 (Np, the number of parallel SEM simulations). These files must specify host names and number of slots per node. Here is an example: 
 +<​code>​ 
 +$ cat nodelist0  
 +hpc-n443 slots=8 
 +hpc-n444 slots=8 
 +hpc-n445 slots=8 
 +</​code>​ 
 +(see ''/​b/​home/​eost/​zac/​jobs/​specfem/​parallelSEM/​nodelist0''​) 
 + 
 +=== Running the simulations in parallel ===
  
-The SPECFEM runs will be handled using 3 scripts: +Parallel SEM simulations are handled using 3 scripts: 
-  * ''​parallelSEM.sh'':​ is the main script+  * ''​parallelSEM.sh'':​ is the main script, that compiles the code and run the simulations
   * ''​run_gpu_nodelist.sh''​ is the script used to run the mesher and solver   * ''​run_gpu_nodelist.sh''​ is the script used to run the mesher and solver
   * ''​sleep.slurm''​ is a script to reserve the GPU nodes   * ''​sleep.slurm''​ is a script to reserve the GPU nodes
 All these scripts are available in ''/​b/​home/​eost/​zac/​jobs/​specfem/​parallelSEM''​ All these scripts are available in ''/​b/​home/​eost/​zac/​jobs/​specfem/​parallelSEM''​
  
-Before running ​it, make sure the input parameters in ''​parallelSEM.sh''​ are consistent with the input parameters stated above (see ''​INPUT PARAMS''​ in the main script). Then run:+Before running ​your job, make sure that the input parameters in ''​parallelSEM.sh''​ are consistent with the input parameters stated above (see ''​INPUT PARAMS''​ in the main script). ​Specifically:​ 
 +  * ''​SPECFEMDIR'':​ path to SPECFEM3D_GLOBE directory 
 +  * ''​Par_file'':​ path to the Par_file used in simulations 
 +  * ''​Nparallel'':​ Number of SEM simulations in parallel (make sure enough GPUs are available) 
 +  * ''​event_list'':​ List of events with the format given above 
 +  
 +Then run your simulations:
 <​code>​ <​code>​
 ./​parallelSEM.sh ./​parallelSEM.sh
software/specfem.txt · Last modified: 2018/03/05 16:06 by wphase