Batch Processing - Generate Cases#
This notebook demonstrates creating cases in batch and running them in parallel.
Create Cases in Batch#
The approach to create cases in batch following this procedure:
Load the base case from file
For each desired case output:
Alter parameters to the desired value
Save each system to a new case file
import andes
import numpy as np
from andes.utils.paths import get_case
andes.config_logger(stream_level=30) # brief logging
# create directory for output cases
!rm -rf batch_cases
!mkdir -p batch_cases
kundur = get_case('kundur/kundur_full.xlsx')
ss = andes.load(kundur)
We demonstrate running the Kundur's system under different loading conditions.
Cases are created by modifying the p0 of PQ with idx == PQ_0.
As always, input parameters can be inspected by accessing Model.as_df(vin=True).
p0_base = ss.PQ.get('p0', "PQ_0")
Create 3 cases so that the load increases from p0_base to 1.2 * p0_base.
N_CASES = 3 # Note: increase `N_CASES` as necessary
p0_values = np.linspace(p0_base, 1.2 * p0_base, N_CASES)
for value in p0_values:
ss.PQ.alter('p0', 'PQ_0', value)
file_name = f'batch_cases/kundur_p_{value:.2f}.xlsx'
andes.io.dump(ss, 'xlsx', file_name, overwrite=True)
Parallel Simulation#
Parallel simulation is easy with the command line tool.
Change directory to batch_cases:
import os
# change the Python working directory
os.chdir('batch_cases')
!ls -la
total 56
drwxrwxr-x 2 hacui hacui 4096 Dec 14 14:51 .
drwxrwxr-x 6 hacui hacui 4096 Dec 14 14:51 ..
-rw-rw-r-- 1 hacui hacui 14640 Dec 14 14:51 kundur_p_11.59.xlsx
-rw-rw-r-- 1 hacui hacui 14641 Dec 14 14:51 kundur_p_12.75.xlsx
-rw-rw-r-- 1 hacui hacui 14640 Dec 14 14:51 kundur_p_13.91.xlsx
Running from Command line#
!andes run *.xlsx -r tds
_ _ | Version 1.5.7.post27.dev0+g9e0e253e
/_\ _ _ __| |___ ___ | Python 3.9.7 on Linux, 12/14/2021 02:51:51 PM
/ _ \| ' \/ _` / -_|_-< |
/_/ \_\_||_\__,_\___/__/ | This program comes with ABSOLUTELY NO WARRANTY.
Working directory: "/home/hacui/repos/andes/examples/batch_cases"
-> Processing 3 jobs on 32 CPUs.
Process 0 for "kundur_p_11.59.xlsx" started.
Process 1 for "kundur_p_12.75.xlsx" started.
Process 2 for "kundur_p_13.91.xlsx" started.
0%| | 0/100 [00:00<?, ?%/s]PQ.vcmp out of limits <vmin>
idx | Flag | Input Value | Limit
------+------+-------------+------
PQ_1 | zl | 0.888 | 0.900
0%| | 0/100 [00:00<?, ?%/s]PQ.vcmp out of limits <vmin>
idx | Flag | Input Value | Limit
------+------+-------------+------
PQ_1 | zl | 0.833 | 0.900
<Toggler 1>: Line.Line_8 status changed to 0 at t=2.0 sec.
<Toggler 1>: Line.Line_8 status changed to 0 at t=2.0 sec.
<Toggler 1>: Line.Line_8 status changed to 0 at t=2.0 sec.
100%|███████████████████████████████| 100/100 [00:00<00:00, 198.45%/s]
100%|███████████████████████████████| 100/100 [00:00<00:00, 188.95%/s]
100%|███████████████████████████████| 100/100 [00:00<00:00, 187.96%/s]
Log saved to "/tmp/andes/andes-kqrgn4n8/andes.log".
-> Multiprocessing finished in 1.2658 seconds.
Number of CPUs#
In some cases, you don't want the simulatino to use up all resources.
ANDES allows to control the number of processes to run in parallel through --ncpu NCPU, where NCPU is the maximum number of processes (equivalent to the number of CPU cores) allowed.
!andes run *.xlsx -r tds --ncpu 4
_ _ | Version 1.5.7.post27.dev0+g9e0e253e
/_\ _ _ __| |___ ___ | Python 3.9.7 on Linux, 12/14/2021 02:51:53 PM
/ _ \| ' \/ _` / -_|_-< |
/_/ \_\_||_\__,_\___/__/ | This program comes with ABSOLUTELY NO WARRANTY.
Working directory: "/home/hacui/repos/andes/examples/batch_cases"
-> Processing 3 jobs on 4 CPUs.
Process 0 for "kundur_p_11.59.xlsx" started.
Process 1 for "kundur_p_12.75.xlsx" started.
Process 2 for "kundur_p_13.91.xlsx" started.
0%| | 0/100 [00:00<?, ?%/s]PQ.vcmp out of limits <vmin>
idx | Flag | Input Value | Limit
------+------+-------------+------
PQ_1 | zl | 0.833 | 0.900
<Toggler 1>: Line.Line_8 status changed to 0 at t=2.0 sec.
<Toggler 1>: Line.Line_8 status changed to 0 at t=2.0 sec.
10%|███▏ | 10/100 [00:00<00:00, 559.13%/s]PQ.vcmp out of limits <vmin>
idx | Flag | Input Value | Limit
------+------+-------------+------
PQ_1 | zl | 0.888 | 0.900
<Toggler 1>: Line.Line_8 status changed to 0 at t=2.0 sec.
100%|███████████████████████████████| 100/100 [00:00<00:00, 200.92%/s]
100%|███████████████████████████████| 100/100 [00:00<00:00, 187.29%/s]
100%|███████████████████████████████| 100/100 [00:00<00:00, 191.10%/s]
Log saved to "/tmp/andes/andes-8fnenj9r/andes.log".
-> Multiprocessing finished in 1.2691 seconds.
Running with APIs#
Setting pool = True allows returning all system instances in a list.
This comes with a penalty in computation time but can be helpful if you want to extract data directly.
systems = andes.run('*.xlsx', routine='tds', pool=True, verbose=10)
Cases are processed in the following order:
"kundur_p_11.59.xlsx"
"kundur_p_12.75.xlsx"
"kundur_p_13.91.xlsx"
PQ.vcmp out of limits <vmin>
idx | Flag | Input Value | Limit
------+------+-------------+------
PQ_1 | zl | 0.888 | 0.900
0%| | 0/100 [00:00<?, ?%/s]
PQ.vcmp out of limits <vmin>
idx | Flag | Input Value | Limit
------+------+-------------+------
PQ_1 | zl | 0.833 | 0.900
<Toggler 1>: Line.Line_8 status changed to 0 at t=2.0 sec.
<Toggler 1>: Line.Line_8 status changed to 0 at t=2.0 sec.
<Toggler 1>: Line.Line_8 status changed to 0 at t=2.0 sec.
100%|███████████████████████████████| 100/100 [00:00<00:00, 202.24%/s]
100%|███████████████████████████████| 100/100 [00:00<00:00, 189.78%/s]
100%|███████████████████████████████| 100/100 [00:00<00:00, 190.62%/s]
Log saved to "/tmp/andes/andes-u_3twnzn/andes.log".
-> Multiprocessing finished in 2.1970 seconds.
systems[0]
<andes.system.System at 0x7ffb2cd41f10>
systems
[<andes.system.System at 0x7ffb2cd41f10>,
<andes.system.System at 0x7ffb2cd53fd0>,
<andes.system.System at 0x7ffb2cd45730>]
Example plots#
Plotting or data analyses can be carried out as usual.
ss = systems[0]
systems[0].TDS.plotter.plot(ss.GENROU.omega, latex=False)
(<Figure size 600x400 with 1 Axes>, <AxesSubplot:xlabel='Time [s]'>)
systems[2].TDS.plotter.plot(ss.GENROU.omega, latex=False)
(<Figure size 600x400 with 1 Axes>, <AxesSubplot:xlabel='Time [s]'>)
!andes misc -C
!rm -rf batch_cases
_ _ | Version 1.5.7.post27.dev0+g9e0e253e
/_\ _ _ __| |___ ___ | Python 3.9.7 on Linux, 12/14/2021 02:51:57 PM
/ _ \| ' \/ _` / -_|_-< |
/_/ \_\_||_\__,_\___/__/ | This program comes with ABSOLUTELY NO WARRANTY.
No output file found in the working directory.